cmurphycode

Forgive me, but I think you've misunderstood something here. The most important thing about the cuckoo is that your bucket choice functions (i.e. hashing function) are independent. If two keys hash to the same first choice bucket, their alternate hashes will (almost certainly) hash to DIFFERENT buckets. For instance, take the SHA-1 of the key. Your first bucket choice is defined by the first x bits, your second bucket choice is defined by the next x bits. But inside a bucket, you can have a bunch of slots.

It works like this. A typical sizing is 4 slots in a bucket, with 2 bucket choices. So your first choice, you arrive at a bucket, and it is just a linear array of length 4. You can put your element anywhere! After a while, when you go to insert into your first choice and you find all 4 slots taken, then you can take ANY of those and try to put it in its alternate bucket. If that alternate bucket is full, you evict from that bucket, too! Thanks to the random nature of your hash functions, it works out very well. 2 bucket 4 slot gets you near 95% occupancy without many evictions on insert.

When you go to lookup, you check your first bucket, and you check all 4 elements until you find it. If you don't find it, you go to your alternate bucket and check all 4 elements. But that's it - only two random accesses (the 4 slots will easily fit in a cache line). It's not so bad to check all 4 elements because the cost is really dominated by loading the line to begin with.

I really recommend a read of some of the cuckoo papers, it's cool stuff.

Imgur album now compares brotli and zstd. Looks like zstd is always significantly faster, but Brotli compresses slightly better for every dataset except the XML one.

This is with brotli level 1, by the way. My understanding is that brotli is pretty quick through the first few levels, but the levels that ask for the highest compression are insanely slow (which is a valuable thing to have as an option, for things like game assets or something, which are compressed once and delivered many times!)

There's a nice comparison of compression algorithms (including zlib, zstd, brotli, snappy, etc) here: https://quixdb.github.io/squash-benchmark/ It's nice because it uses many datasets and machine platforms.

Unfortunately the graphs leave a bit to be desired, especially when you're trying to compare two algorithms. They provide the raw data, but it needs a little munging to make it workable.

For my day job I made the following graph to compare zlib level1 and zstd. This was using the "peltast" platform which is a Xeon based system, because that was most relevant for us.

http://imgur.com/a/0a3kK

I'll make one with brotli now.

Just to clarify, I think you meant to ask why random access is still slower than sequential on ssd.

Writes smaller than the ssd erase size are more expensive per byte because of write amplification. This is not nearly as big of a factor as for disks, but it does seem to matter even on enterprise flash, if you benchmark properly (truly random and enough data so that it cannot be buffered by clever controllers). And as my sibling comment mentioned, the write amplification wastes flash lifetime, too!

On enterprise flash, I've found that random reads top out at the same bandwidth ad sequential reads, but it takes a LOT of parallelism to get there, so sequential reading, if possible, is still nice (let the controller do the parallelism for you!)

Nearly everyone thinks they're an above-average driver. A slightly better than average autopilot will be viewed as riskier!

There's also some sort of bias against negative events that are (more) out of your control, I think.

I'm having trouble understanding what the practical difference between the transaction concept and something like RAII. It may just be the example they chose (goroutine processes an incoming request, does not "share" the contents with anyone else, and terminates when complete) but nonetheless I'm having a mental block. Can anyone clarify?

Note that I'm not saying that they should've used RAII along with existing GC, I'm just trying to distinguish the transaction concept and the RAII concept.

The main reason I bought my Galaxy S7 is because they made it thicker and with a bigger battery than the S6.

From talking to my colleague with an ecobee, that's not the point. If the ecobee detects a big swing between the two sensors, it'll run the fan, but not the heat/cool. That balances the rooms. It makes sense.

For 4K random reads, you'll probably want more than 32 iodepth, even on a consumer drive like this. And you'll want to try to write to each offset in the drive beforehand in case they have a "return 0" shortcut (like many enterprise drives).

That is very true (and a valid concern) but it's hard to understand what the filesystem should do in this case. Seems to me that the best you can hope for is that the data is corrupted in a way that doesn't make "sense" and the filesystem has a check for that, and quits.

Other than duplicating the pages (which is certainly a valid approach) how else can you recover from a data stomp?

Having read that slide, I can only guess that they were fixing up checksums for the pages they modified in known valid, but wrong ways. In other words, if there's a page on disk containing a timestamp and the test wants to bump the timestamp downwards to see what happens later on, then you need to fix that page's checksum otherwise the test will not do what you wanted.

Of course that means your mutations have to be "fair" in the sense that your new values should still be valid. Because as you stated, just stomping on blocks and fixing up their checksums seems like a silly test.

How's that? If you go read your data, re-checksum it, and that matches the original checksum, then you have confidence (to the strength of your checksum function) that the data is not corrupted.

without spending the time to make a better prime generator, you can get your test going much faster by only testing factors up till sqrt(p).

With just that change to your program, and asking for 100k primes:

{'9-1': 8829, '1-1': 4104, '9-7': 5671, '3-9': 8387, '3-7': 7419, '7-1': 6438, '1-3': 7961, '3-3': 3604, '7-9': 8022, '1-7': 8297, '7-3': 6928, '1-9': 4605, '3-1': 5596, '7-7': 3627, '9-3': 6513, '9-9': 3994}

Wow. Lots of interesting points, and a few things that made me say WTF and wonder how much this will be torn apart in the comments.

In other words, a very worthy read.

Well, if they did understand, they would say the chance is ~1 in 2^61, not 1 in 2^122, and they would've based the math comparison to RAM failure on 2^61, which changes the comparison entirely.

In any case, I should have been more clear. I'm not necessarily worried about this case. But if you design systems and miss a factor of a square root, you tend to break things that we're all relying on to work.

Yeah, I really don't understand how they are ignoring this. It's actually very concerning to me that they don't understand this.

seltzer water (i like Polar) is this. It's an enormous market in the US.

This is a great point and a huge mental problem I have when looking at (and writing) benchmarks. If anyone knows more about this (i.e. if they can show why it isn't true) please speak up.

" Most of those players are not famous. Most never made a dime off the game. They are relatively anonymous men who played the sport in college and only later, for some reason or another, have found themselves struggling in life."

College players aren't paid. Most college players never make it to a professional paying level.

https://www.ncaa.org/sites/default/files/Probability-of-goin...

Actually, you can use convergent encryption to dedup encrypted files. It leaks information (because I can tell whether a file that I possess already exists on the server if the server is silly enough to tell me that it took no space/time) but it is indeed encrypted.

Check out this discussion of Mega for more info: https://news.ycombinator.com/item?id=5072680

Have you ever had your legs measured by a physical therapist or equivalent? I used to do the same thing. The problem is that resting on one leg tends to push the hips out of alignment (it's easy to see in the mirror; the hipbones won't make a line parallel to the ground).

For me, I would always get pain/numbness in my lower back on the side of my taller leg. Putting a lift in the other shoe and being mindful to keep straight helped to fix it. I had to be mindful (at least at first) because my body was so used to slouching sideways that it would still try to default in one way or another. I think that it's gotten better as the muscles re-learn the necessary patterns to keep straight.

EDIT: BTW, I don't use a standing desk- I prefer to walk around a lot. Before my PT visit, I would get pain within 15 minutes of standing!

Yep, the math makes no sense if you assume the two options are 0 or 1B. I guessed that he was using the actual outcomes of the entire spread- considering that he references the 67% chance of $0 shortly thereafter, it is clear he doesn't think the other 99.999.... percent result in $0.

Even if my excuses for him are correct, I think you all have made a valid point that this is not a really productive way to measure :) Nonetheless, the lesson "don't start a company to get rich" is, I think, a good thing to say.

"expected" payoff is the key, here. He's not just dividing a billion by 3.

Your theory of salting-per-user is correct.

Rainbow tables are useful if the time it takes to compute is big enough that storing a large list is useful.

The GPU crackers are so fast nowadays that that isn't the case. If you can try 500M per second, that's about 10 GB of SHA1 generated! Imagine storing a day's worth of work in a rainbow table :)

It's better to use a proper password hash, which will be slow, and will likely have per-user salting built in, anyway.

Here's your answer: "The XL1 has a coefficient of drag of 0.189" from the article.

That's an enormous improvement on all other production cars. Which is why it uses " 8.3 horsepower to maintain [60 mph]— one-third that of a Jetta"

Of course, they don't mention the frontal area, which is just as important and yet never referenced in these discussions.

Software Engineers in Boston area (Waltham, MA)

Actifio (http://www.actifio.com) is looking for software engineers. Actifio is radically simple copy data management. We've invented some really cool technology to do things no other system can, and we're dead easy to use. This is not your mother's enterprise software!

Actifio has been invested in by top VC firms (one of the very few East Coast companies in Andreessen Horowitz's portfolio) and is growing super fast. Recently, we closed a 50 million dollar round of funding.

We're always looking for smart engineers in every team, but since I'm on the deduplication team, I'll have to pitch you on that :) Briefly, our other teams include Java, C++, and Adobe Air, and work on things like VMware interaction, snapshot management, scheduling, databases, etc.

As for the dedup team: we write in C, and use Python for debugging and support scripts. Dedup seems like an easy problem to solve, because it's so conceptually simple: split the data up into blocks and make sure you only store one of each. But at scale, it becomes a whole different animal. When you've got 128TB of 4KB blocks, you have 32 billion unique blocks of data to manage. Making our dedup ingest, restore, and garbage collect fast is a great challenge, especially with "only" 128 GB of RAM.

If you'd like to chat about a job opportunity (in dedup or otherwise!), I'm at chris.murphy@actifio.com. Since I'm an engineer, I'm not going to type a list of silly requirements. Just send me a resume and whatever else you think I'd like to know.

I've been getting pretty sick of the whole "look how dumb I am, but I'm iterating!!!11" meme, but I really like this post. As a young person working in a company of people with far more experience, I've asked my share of stupid-simple questions and made my share of dumb mistakes. Luckily, I've got enough experience with programmers to get the most out of these interactions (also helpful to have decent human beings for co-workers!)

The important thing is to remember that there aren't (m)any super geniuses for which this stuff spews out first try. It's just that some people had the benefit of working through their ridiculously terrible code at the age of 8, writing BASIC, or whatever. And while they may have to learn jQuery the same as you did, they have a massive class of "intuitive" knowledge, which is really just experience, to help them through most of it.

I set up gitlab for some "skunkworks" projects at my day job, and it's been really great. The Gitolite dep never bothered me (the gitlab install is well documented), but seems like a good thing.

Thanks for your hard work, guys and gals!

Lucky :) Even if I don't use my phone at all, it'll be below 80% by this time, and Maps & co seem to be a major culprit, since Now keeps on chugging even when the phone hasn't woken in a long time.

Interestingly, GSam battery monitor (which I just got last week, and is really neat!) shows the number of times each app woke the device. I've been on for 4.5 hours, and Maps is at 441, with the next Android System at 85 and Google Search (which is Now) at 53.

Android System has also shown up more in usage % since I started having Now.

I think there's definitely a significant drain with Now. I've had it enabled since the Verizon release, somewhere around 12/20, but I think I will disable it at the start of next month and keep an eye on how it behaves. I certainly feel that my battery life has gone down since the upgrade.

Actually, Now's time goes under Maps- at least the location services, which are the big draw. I have all of the tracking enabled, and maps uses between 20 and 30% of my overall.