HNHacker News
TopNewBestAskShowJobs

codahale

1,027 karma · joined June 13, 2008

submissionscomments
codahale··on Advanced programming languages
That doesn't sound unreasonable, no -- at some point in the future, Scala's hash array mapped trie will be robust and highly optimized. If your plan is to wait until then, you might be able to speed things up by helping them out yourself.

In the meantime, I benchmarked Yoink's PeristentHashMap against 2.8.0.RC2's immutable.HashMap by a) creating a map of 100,000 sequential keys and then removing them all, and b) creating a map of 100,000 random keys and then removing 100,000 random keys. In the first case, Yoink was 20.7041% (±1.83693%) faster. In the second case, Yoink was 24.7834% (±4.17055%) faster. (Student's t distribution, 95% confidence interval, source is in src/test on the GitHub project and I used ministat for the statistical analysis [a cross-platform port of which is also available on my GitHub page].)

So to summarize, I have a provably faster hash array mapped trie at almost zero cost right now because I spent a few hours with a "good learning experience."

codahale··on Advanced programming languages
The hash array mapped trie was committed to trunk 3 months ago and was first released in 2.8.0.RC1 (which no one uses due to various bugs) and is now available in 2.8.0.RC2 (which very few people are using due to binary incompatibility).

In contrast, Clojure's PersistentHashMap dates back to 2007 and was in both the 1.0 and 1.1.0 releases.

codahale··on Advanced programming languages
I invite you to benchmark the two.

I'm also slightly confused as to why you think reusing stable, proven, and reviewed data structure implementations is a "good learning experience" as opposed to being SOP.

codahale··on My latest anti-java rant
I've seen java apps commonly take up a whole rack of servers, not just one

Ok, and I've seen Java apps not do that. Argument by anecdote is bankrupt.

codahale··on My latest anti-java rant
You know, you can set a maximum heap size for JVM processes. It defaults to 1/4th of the physical memory or 1GB, whichever is the smaller.
codahale··on Free will is an illusion, biologist says
And yet the nerve impulses which caused the muscles in your arm to contract left your brain about 500ms before you became aware of deciding to raise your arm. (http://www.ncbi.nlm.nih.gov/pubmed/6640273) Additionally, the mechanisms which sent that nerve impulse were spinning up for up to 10s before that (http://www.nature.com/neuro/journal/v11/n5/abs/nn.2112.html).

The you that lifts your arm and the you which feels like it decided to lift your arm are on different schedules.

codahale··on Huge iceberg breaks off Antarctica
Oh, ok. So it takes making predictions to quality as real science.

Here's a real prediction, with actual confirming evidence: giving someone with a high MADRS score an SSRI will reduce their score.

That's beyond seeing and naming depression, it's beyond recording patients' reactions to SSRIs, it's beyond spotting patterns in their reactions, it's even beyond making falsifiable theories about its mechanism of action. It's making predictions which are confirmed on a statistically significant basis.

Does this mean you think psychiatry has the same epistemological standing as physics?

codahale··on Huge iceberg breaks off Antarctica
You have got to be kidding. 150-300 years?

Agriculture dates back to at least ~10,000 BC (the "Neolithic revolution"). Effective farming (i.e., not dying of starvation) requires planning for changing weather conditions. If you think Neolithic farmers didn't have a vested interest in detecting patterns in weather cycles, you're sadly mistaken.

Hell, the term "meteorology" was coined by Aristotle... in a book he wrote... called Meteorology... in 350 BC.

codahale··on Huge iceberg breaks off Antarctica
The reason we know where Pluto will be in 9 years is that we've been watching the solar system for two thousand years.

How long do you think people have been watching the weather for?

codahale··on Huge iceberg breaks off Antarctica
Seriously. No one jumps all over NASA yelling about how the science "isn't in yet" when they want to send a probe out to where Pluto will be in 9 years.
codahale··on How To Safely Store A Password
They might. I don't know. They didn't ask me for help getting it tuned.
codahale··on How To Safely Store A Password
The bcrypt algorithm has salting built-in.
codahale··on How To Safely Store A Password
They're two completely different things. AES is an encryption algorithm. bcrypt is a special-purpose hash algorithm.

If you're confused, I'd highly recommend Practical Cryptography by Niels Furguson and Bruce Schneier: http://www.schneier.com/book-practical.html

It will give you much better advice than the internet will.

codahale··on How To Safely Store A Password
I should have been more precise in the article. Here:

A system which uses an adaptive hash function like bcrypt is ~6 orders of magnitude less effed in the event of a compromised database than a system which uses a standard hash algorithm and a salt, ceteris paribus.

I would hope you agree that those ~6 orders of magnitude could well be the difference between "not noticeably effed" and "profoundly effed."

codahale··on How To Safely Store A Password
Awww. :(
codahale··on How To Safely Store A Password
You're asking about the relative merit of two hash functions, one of which allows your attacker to test a candidate password in 1 millisecond, the other of which allows the same in less than 1 nanosecond. Using the latter provides your attacker with a 1,000,000x productivity boost. Personally, I'm not that generous.

As far as CPU exhaustion, there are some huge sites which use bcrypt (like, in the top 10)[1]. It is not a problem, provided you choose your work factor carefully.

As far as complex passwords, use them. Try to get your users to use them. But it's an orthogonal concern: your attacker will still be able to work their way through the gigantic keyspace of your monster passwords at 1,000,000x the rate of what they could do if you'd have used bcrypt.

[1] Just looked this up. Not the top 10, but the top 15 for sure.

codahale··on How To Safely Store A Password
Toss it up on GitHub, man.
codahale··on How To Safely Store A Password
Uh. All of the current SHA-3 candidates are as fast or faster than SHA-1. That's one of the contest's design goals:

NIST expects SHA–3 to have a security strength that is at least as good as the hash algorithms currently specified in FIPS 180–2, and that this security strength will be achieved with significantly improved efficiency.

http://csrc.nist.gov/groups/ST/hash/documents/FR_Notice_Nov0...

codahale··on How To Safely Store A Password
He's implying that using bcrypt (and other iterative hash functions like PBKDF2 and scrypt) raises the cost of mounting a dictionary attack compared to commonly-used and -recommended hash algorithms like SHA256.

Which, uh, is an objectively verifiable fact.

codahale··on How To Safely Store A Password
Also, you point out that "successfully discouraging your users from using a crappy password has much better repercussions." This is true, and I'm curious to know how you personally achieve this with your users, given that the vast majority[1] of users choose spectacularly crap passwords.

[1] http://www.imperva.com/docs/WP_Consumer_Password_Worst_Pract...

codahale··on How To Safely Store A Password
I'm familiar with cperciva's scrypt, and I think it's a great solution. That said, I've only seen C and Ruby bindings for it.

I'd love to recommend its use over bcrypt, as memory constraints are much more expensive than computational constraints, but recommending something most developers don't have access to would result in more weak hashing schemes in production.

codahale··on How To Safely Store A Password
I'm guessing you haven't read Provos and Mazières' USENIX paper. In the design considerations, they say:

In general, a password algorithm, whatever its cost, should execute with near optimal efficiency in any setting in which it sees legitimate use, while offering little opportunity for speedup in other contexts.

PBKDF1 and PBKDF2 both rely upon general-purpose hash functions, which are much faster in dedicated computing environments such as FPGA or GPU clusters. The Eksblowfish algorithm at the heart of bcrypt is extremely resistent to optimization.

This is crucial for password storage, since if a CUDA implementation is 3-4 orders of magnitude faster than the implementation your application uses, you've just chipped off a huge chunk of the advantage offered by your hash function.

codahale··on How To Safely Store A Password
I'm curious to know why you think innovation in the security domain will come from web developers working on lolcat apps and not, say, cryptographers.
codahale··on How To Safely Store A Password
Foiled again! Curse you, Ptacek!
codahale··on Freenode sucks
It's true. That is a ridiculous policy.

They should refund him his money.

codahale··on Don’t Hash Secrets
Good god, no. Here is much better advice, in much shorter form:

If you're storing a password, use bcrypt or I'll bite your thumbs off.

If you're making sure something hasn't been modified by someone and you absolutely can't use a signed GPG message, use HMAC-SHA256 and a constant-time comparison algorithm or I'll bite your thumbs off.

All the business about hash functions is like advice on how to build your own car brakes using a backyard smelter and a sand cast. You might get something of roughly the right shape and weight but someone's going to get hurt down the road.

codahale··on Ad Hominem Is More Important Than You Recognize
From TFA: And then some fool calling himself SoSerious posted _an article_ about how much he feels DC was an idiot.

From his link: Joshua Blankenship is a designer living in Anderson, SC.

codahale··on Dygraphs: Better Time Series Graphs
When an "upstream" owner doesn't take patches, the usual result is that there's complete chaos and mutually incompatible forks with the same name and version information spring up.

If the upstream owner doesn't accept patches the project is hosed, regardless of how it's hosted.

The big difference with a DVCS is that those chaotic forks are public and can themselves accept patches. Developers can still pool their bug fixes, added features, etc., and perhaps the project can rise from the ashes.

If the project is hosted on a centralized VCS, all of those bug fixes and new features languish as one-off forks in private repositories. Instead of being a chaotic mess, it's moribund. You still have the same issue of having to maintain your own fork; you just don't have any help in doing so.

codahale··on Ruby Gets An Official Spec: Heading To Become An ISO International Standard
I want to be excited about this, but there are big gaps in this spec.

For example, "blah"[0] should return an Object instance which represents "b". Could be a Fixnum. Could be a String. Could be a JPEG of some ASCII art representing the essence of "B." Any of those responses would be valid according to RubySpec.

What happens when you shadow a local variable with a block variable is unspecified, too. In 1.8.x, it overwrites the variable. In 1.9, it doesn't.

Even with a "RubySpec-compatible" implementation, you have no way of knowing if your application will run or not, which kind of defeats the purpose.

codahale··on Flawed climate data
Ah yes, The Financial Post. The newspaper I turn to for all of my climatological analysis. It's like International Journal of Climatology, but with better op-eds and stock listings.
← PreviousPage 3 of 5Next →