notabel

Well, I certainly hope that news.yc users are a bit more than Ricardo-programmed economic optimization automata. :)

One solution to older users having a perpetual advantage would be to implement some sort of aging of karma, or at least of the karma value used to determine throttling. Such a thing would actually be a win in terms of SNR--it doesn't really matter how karmic someone was a year ago, if they have only made low-quality posts/comments in the past month, they should be treated (in the submission system) as a low-karma user.

This is an interesting notion; there is an implicit assumption that the low-karma user has something to lose by posting comments. Am I missing something here?

I do see your point, though, wrt top users not spamming the site. That's largely why I liked your idea of throttling submission rates based on karma. New users should have to develop karma by being useful, rather than just prolific.

That is certainly the status quo; I would ask, however, whether it is desirable. I think the system should provide more incentive for thoughtful comments, or at least some sort of disincentive for shotgun-style submissions. It is difficult to build karma by posting lots of comments; you have to put the effort into generating good content to get karma. The same should ideally be true for submissions; the difficulty, of course, is that since the content of submissions is generated by someone other than the submitter, simply posting lots of them /will/ eventually result in a karma-boosting hit.

I'll be honest, I've not been nearly so good about doing my time on the 'new' page. In your experience, are there any strong patterns in good submissions? What I'm getting at is basically: are there any views on the new submissions list that would make finding the under-rated ones substantively simpler?

One thing I find interesting about news.yc is the relatively small set of active contributors to the comments, and their rather high availability: case in point, this thread, which is feeling to me rather more like an ongoing conversation than a /. style shout-fest.

Anything that can be done to encourage this feeling of intellectual discourse is a win, in my opinion.

I was pondering this phenomenon earlier today; it would be interesting to do a quantitative analysis of post and moderation rates (this would require access to the backing database). I have noticed that there tend to be definite hot times, and that posts made outside of those hot times tend to never get modded.

This, combined with a burying system to get rid of junk stories, could clean up the 'new' section substantially. It would also be useful to factor snr into the computation; the nature of the karma system, which makes it more advantageous (at a marginal level) to submit a story than a comment, encourages a shot-gun approach to submission; SNR would combat that.

EDIT: Ha, points to jwecker for posting about SNR while I was writing about it.

"mid twenties" -- pg in an email

This is interesting, I'd never heard about it. Time for some research, I suppose. Yahoo's gambit is reminiscent of a real-world retailer telling a customer to go to a competitor to find the product they really want (my local bike shop did that once, and won my lasting trust).

In the case of bloggers, though, there is another effect in play: blogs succeed basically on connectedness, so for a new or up-coming blog, the most important thing is tying into the network of existing blogs in the relevant domain. It makes sense that praising, referencing, critiquing, generally interacting with "competing" blogs is the best way to jack into the community.

Yeah, Guy Kawasaki is a great example of the maxim that what really matters is "getting it", not so much "being it". Even though he's no hacker (and makes no claim to be) he understand hackers, which has made him incredibly effective at interfacing with hackers--which is really the hard problem of what he does.

The code for the bookmarklet is refreshingly simple; glad to see that PG can design an interface that is minimal and functional, even in this age of wrapping everything in 12 layers of XML, JSON, SOAP, $ACRONYM.

Thanks for doing this, phil.

The ideas embodied in Shaker design are (in my opinion) almost always correct design rules (the almost being that, as always, you should break your design rules when they interfere with your goal), but aesthetics aside, there is a very relevant message here: in the startup phase, the focus should always be on creating what is necessary, useful, and beautiful. Useful explains itself (though it's often forgotten); necessary is the real Occam's razor of the group, and perhaps the important one for start-ups (i.e. during the initial angel phase, get your critical functionality implemented--a VC won't be impressed by great features if the core functionality doesn't work); beautiful is tricky. It's aesthetic. When it comes to code, though, beauty is usually directly related to maintainability and extensibility. Sure, beauty is in the eye of the beholder, and what I think is atrocious C++ might be beautiful to you, and my beautiful Lisp might make you cry; what matters is that the code is beautiful to the person who has to interact with it.

However, you need the correct subset of the 15 billion pages, which is only fractionally easier, and in some ways harder: google can just grab everything, and then pull semantics out. If you want to leverage your limited domain, you need to be able to be able to have semantics in your indexer/crawler, otherwise you're going to end up having to index everything anyway.

One exception, of course, is in genuinely finite-domain search engines, like Octopart. There, you know exactly where to send your indexer, so you can be very efficient.

I entirely agree. The best way to build something is to get alot of smart people working on it--or rather, just enough smart people. The thing that Wikipedia leverages so powerfully is the ability of a wiki to let people work in parallel. Unfortunately, that advantage does not translate to building a search engine--you can't have people working separately, each contributing intelligence in their own domain, and expect to build a working algorithm.

I'll give them the benefit of the doubt, but as far as I've seen, Jimbo is pretty much just flying around in his roflcopter talking up vapor on this one.

QUOTE: Considering that relying sites can filter which providers they allow authentication from, I am overall positive about an increased usage of OpenID.

I'm hoping that this sort of thinking will not proliferate, as it nullifies what I see as one of the big advantages of OpenID: user privacy. The core architecture of OpenID makes it possible for users or small groups of users to operate their own OpenID servers, avoiding the need to hand their personal information over to any third party to prove their (pseudonymous) identity. If it becomes standard practice to filter OpenID requests to only allow authentication from certain large providers, then OpenID becomes no better than any other single-sign-on system.

Moreover, there is no advantage to filtering authentication sources. An OpenID authentication is merely an assertion that the user is authorized to use a given universally unique identifier. The only way that the authentication source can cheat is by permitting auths that it should deny, or vice versa. Because users can choose the authentication source they use, there is no incentive for such cheating. In the case of a private person running their own OpenID server, the only attack would be for that person to attack their own account, nonsensically.

If you're unconvinced of the security of permitting anyone to serve, consider the analogy to Jabber servers: anyone can run their own, federating into the wider XMPP network, and the network maintains security because the misbehavior of one server only affects that server's users.

I'd be curious to know how many people were polled to generate those numbers. For instance, if 25 people[1] were polled, then 23 of 25 -- 92% -- put Sequoia on their list. That would be quite impressive indeed.[2]

[1] I'm pulling that number from the ether, and the fact that there are 95 points visible; 25 people would have 125 available, and an excess of 30 seems reasonable. [2] Not that the data aren't impressive anyway.[3] [3] Yes, I have a footnote problem.

As you pointed out, Google already competes both directly and indirectly with myspace; NewsCorp knew that going in, and went with Google anyway. I doubt that serving ads for yet another social network would be enough to drive NewsCorp away--and I even more seriously doubt that their contract with Google would allow it.

One point about Orkut: while its growth is phenomenal, most of that growth is in non-US markets, so it does not pose any direct threat to either Friendster or myspace.

Thank you! I had the nagging feeling that there was a more precise bit of terminology that I was missing. You've gotten it off the tip of my tongue. As you pointed out, though, priming properly refers immediate-time phenomena. Whereas priming arises from manipulation of operational memory, confirmation bias arises from deeper heuristic phenomena.

\begin{ego}Welcome to the wonderful world of the learned.\end{ego} Seriously, though, I've had problems like this ever since I started studying psychology--self examination can be risky!

Ultimately, while the human mind is excellent at fuzzy inference, it's not perfect, and augmenting inductive inference with declarative rules often just swings the error to the other side, as you observed: confirmation bias + knowledge of confirmation bias - reverse confirmation bias. If you want /real/ self-examination, do what you did: quantify it!

Unfortunately, the author largely glosses over what I consider the most difficult part of search--extracting semantic information from unstructured source materials (also refered to as "magicks"). However, the discussion of bandwidth and server allocation - of starting with what you can afford, and spending your effort/money on what matters (i.e. your algorithms) resonates with a lot of the ideas behind YC. When it comes right down to it, anyone who can sweet-talk a VC can get fat pipes and big boxes--the (human) brains behind the (algorithmic) brains are what differentiate Google from Lycos.

Yes, actually, there is a word for that--sort of. It is essentially a special case of confirmation bias. Confirmation bias is a concept from cognitive psychology; basically, the human mind heuristically prefers data which support (confirm) its nascent inferences. In this case, the inference being developed is roughly the importance/prevalence of the word.

So, there isn't a word for this precise phenomenon (to my knowledge), but you're welcome to coin one. If there's one thing cognitive psychology loves, it's terminology. (Don't get me wrong, cognitive psych is delightful, but it loves its lingo.)

N.B. Related phenomena are the cocktail party effect, and, in the large, figure-ground phenomena, and selection bias. EDIT: And priming. Thanks to python_kiss. (Wikipedia has decent articles on these things, but if you're really interested, go to the library and get a good psych book.)

This is certainly a great product, and the fact that the founders are in fact "scratching an itch" gives me confidence that they'll continue to do fine on the tech side of things.

I do wonder, however, what the business model is here. My hope is that the major parts vendors will recognize that Octopart is a good thing for them, and will play ball--both on the technical/database integration front, and, hopefully in terms of monetizing the product. Because it really is a great product.

One technical quibble, though: Octopart could improve its result quality a good bit by incorporating some semantic analysis. An example: a search for "blue led 3mm leaded" returns as its first result an SMD (i.e. non-leaded) part. What's going on here? It's matching the phrase "Leaded Process Compatible: No;" with a bit of not-terribly-deep semantic matching (leveraging the fact that part specs speak a relatively contrived and regularized language) it should be possible to recognize that "Leaded Process Compatible: No" is not a match for leaded.

Quibbles aside, I'm looking forward to watching Octopart improve, and, hopefully, succeed. We all know the competition sucks, after all.