Freaky

> Physical drives fail due to physical reasons, but SSDs fail for logical reasons.

HDD's also have a long and distinguished history of nasty firmware bugs, and this is only going to get worse as things like drive-managed SMR and hybrid flash caches get more common and their internal complexity ramps up.

Both also fail due to electrical reasons. Chips fail due to manufacturing defects, solder degrades, capacitors dry out, etc.

And I'll reiterate the uncorrected bit error rate. Transient errors are more common with SSDs, and are unlikely to happen identically with multiple units.

> If you have a mirror, and you have two identical SSDs, and you produce such a condition ... then no more mirror. They both die identically

I have a pair of mirrored SanDisk Extreme Pro's in my main server. Both suffered from a firmware bug that caused data corruption. ZFS was able to repair all damage, because they didn't fail identically.

Also thanks to the mirror I was able to upgrade the firmware without taking the server down.

Mixing different SSDs might be a good idea, but you can make much the same arguments for doing the same with HDD's, and like with HDD's it's still better than nothing to have redundancy with "identical" drives.

> As if both drives do exactly the same writes at the same time, chances are they will fail at the same time.

You mean due to exhausting their endurance? This is something you monitor, you should have plenty of time to replace the drives before it becomes a concern.

For other failures, how's it going to be any different from normal HD's? There's always the risk that having the same models/batches in the same conditions might lead to a cluster of failures, but RAID's still likely to save you from plenty of other failure modes.

For instance, the recent Google study[1] suggests that uncorrectable errors with SSDs are actually more common than HD's: "More than 20% of flash drives develop uncorrectable errors in a four year period... significantly higher rates .. than hard disk drives".

I've certainly seen enough IO errors from SSDs to know I want to defend against them. Not to mention the silent data corruption I've seen (and been protected from, yay ZFS).

1: http://0b4af6cdc2f0c5998459-c0245c5c937c5dedcca3f1764ecc9b2f...

curl -Ro does, however.

> The reason is that when ZFS detects corruption, it'll lock down the whole fs... and prevent reading/recovery data from it

Depends on what exactly is corrupt, but for file corruption it's generally just a case of warnings in logs/zpool status (which will suggest restoring the file from backup), and IO errors trying to access that specific file. The pool itself should remain intact and online.

It's less clear cut if it's important metadata that's damaged, but as you mention, ZFS is quite aggressive about maintaining multiple copies even on standalone devices.

Meanwhile fsck on a traditional filesystem will gleefully mangle data that's actually fine in face of transient corruptions.

I've had this happen more than once, both with bad RAM and bad IO controllers - previously fine static data suddenly being detached from the filesystem and appearing in little bits in lost+found, because bit flips effectively causes it to hallucinate problems to "fix".

As mentioned elsewhere in the thread, this isn't a "you must", this is a "you might as well".

Timing attacks depend on an attacker having control over the hash being compared (e.g. they have a HMAC in a cookie they're sending you, and they can adjust it character by character) - with randomised secret salts and server-side hashing, this isn't the case.

The SCrypt example is the one I'm more concerned with. The defaults there are 1MB of RAM and 64-bit salts, both of which could do with increasing. I have an open issue on this: https://github.com/pbhogan/scrypt/issues/25

As it is, the example would probably be better as this:

    password = SCrypt::Password.create(usersPassword, salt_size: 32, max_mem: 16*1024*1024)

You'll be pleased to know SCrypt::Password#== is at least constant-time.

> base64_encode(hash('sha384', $password, true))

> ...

> The above construction may invite theoretical concerns about entropy reduction (i.e. 72 characters of raw binary without any NUL bytes comes out to about 573 bits of possible entropy, but a SHA-384 hash outputs are clearly limited to 384 bits).

Given BCrypt hashes are a mere 184 bits, I don't see how this is a meaningful concern even in principle. If you're brute-forcing search spaces this big you're no longer looking to recover a password, but find a collision.

Just because I want a less bright background for reading text doesn't mean I want my monitor to be incapable of displaying a bright white - it's not only used for reading text.

Also: https://www.dragonflybsd.org/cgi/web-man?command=sys_checkpo...

It's notable that the Solaris unexec() implementation[1] is basically a thin wrapper around dldump()[2].

1: http://git.savannah.gnu.org/cgit/emacs.git/tree/src/unexsol....

2: https://docs.oracle.com/cd/E19455-01/806-0627/6j9vhfmop/inde...

Losing to a Forgotten Beast made out of vomit and pus doesn't sound quite as fun in real life :(

The cvsweb links are semi-broken. Relevant SVN commit: http://v4.freshbsd.org/commit/freebsd/src/38520

You can use a Yubikey NEO to handle key authentication on your computer's behalf:

https://blog.habets.se/2013/02/GPG-and-SSH-with-Yubikey-NEO

64-bit is very strongly recommended, but it does work on 32-bit with tuning: https://wiki.freebsd.org/ZFSTuningGuide

I wouldn't expect a passphrase that long to be particularly entropy-dense - quite the opposite really. Some people do things like pick phrases from Hamlet and stuff some numbers on the end, so truncating them would seriously compromise their expected strength.

Or XKCD style passphrases. Using the Oxford 3000 dictionary gets you about 1 bit of entropy per character. If I have my password manager generate 128 bit passphrases using them for ease of use in the real world, plain bcrypt will quietly reduce their strength by about 17 orders of magnitude. If nothing else that's obnoxious.

And that's ignoring the soft 55 byte limit both the bcrypt spec and the original scrypt paper mention. If we're looking at less than one bit per byte that's starting to get into worryingly weak territory.

Either way, it's all solved by stuffing a (e.g.) base64 SHA384 in there. Bam, every input bit equally effects the output hash regardless of length and position, and the entire thing even becomes NULL byte safe.

> imposing max lengths .. ends up hurting people

In that case you'll want to pre-hash your BCrypt-encoded passwords, otherwise you'll be imposing a silent 72 character limit.

Remember to encode the hashes, since BCrypt also silently truncates after NULL bytes.

This is why ABX testing is so big in lossy audio circles. People can and do demonstrate their ability to distinguish between lossy and lossless encodings with certain samples in double-blind tests, at all sorts of bitrates. I've done it myself occasionally.

That people have been doing this for many years is one of the big reasons modern encoders are so good - they've needed tonnes of careful tuning to get to this point.

LHA and LZX were the popular pair when my Amiga days came to an end. Being a commercial product the latter kind of occupied a similar position RAR does on Windows.

Microsoft ended up adopting LZX for things like CAB and CHM files.

You're also describing everyone else's experience:

> Really at 320kps you are entering the realm of fantasy if you think you can hear any difference.

It depends on the encoder, the track, your equipment, and how good you are at picking out artifacts. Some people do surprisingly well in double-blind tests, though I doubt anyone can do it all the time on every sample.

I'm fairly sure bsdtar had automatic compression detection before GNU tar did. Been the default on FreeBSD since version 5.3 (2004).

Also works with non-tar formats like Zip, RAR and 7z: https://www.freebsd.org/cgi/man.cgi?query=libarchive-formats...

I.. would probably have done that before concluding I didn't have any need for it. Even just do a quick one-off mcelog run to see what events have happened recently - you don't have to do some big statistical analysis.

I'd estimate about half the machines I've used with ECC have had at least one correction a year, maybe a quarter had one every few months. I've seen one-off weird bursts that never happen again, and I've had quite a few cases where ongoing corrections have indicated a DIMM needed reseating. An actual blatantly faulty DIMM's been quite rare.

I've had plenty of corrupt JPGs - they're certainly noticable since a single error tends to mess up an entire macroblock, often even the entire rest of the image.

Was fun finding a bunch of my own photos like that. Luckily that was on a disposable copy and not primary storage, but it was enough to put me completely off non-checksumming filesystems.

> "statistically speaking, most computers in the world do not use ECC". How can both of these things be true?

... most computers aren't particularly reliable.

I'm curious - are you not monitoring or at least keeping a vague eye on ECC correction events with your existing hardware? If so, are you just not seeing any?

I've never really operated at any sort of "large" scale - a handful of racks at most - but I've always found correction events to be about as routine as IO errors, and certainly way more common than outright disk failures.

You'd probably be better off linking to something like this: http://www.rifters.com/crawl/?p=6116

Or this: http://rifters.com/real/2009/01/iterating-towards-bethlehem....

Less than a million neurons and Portia's interpreting complex visual scenes and making detailed attack plans.

While "we only use 10% of our brains" is clearly bollocks, the implication that our brains may not be particularly efficient for our level of intelligence doesn't seem too unlikely given the way evolution works - our brain's a crudely scaled-up pile of accidental hacks guided by endless local maxima it can't readily escape from, it'd be very surprising if you couldn't improve on it.

You might draw parallels with modern computers. All these transistors to play with, but we're certainly not using them as well as we could - we're limited by the software we want to run and know how to write using our existing tools.

Yup: https://openconnect.netflix.com/software/

They're actively giving back a lot of their work too: http://v4.freshbsd.org/search?q=netflix

See also The Log File Navigator (lnav) if you're in the market for a fancy tail replacement: http://lnav.org/

Rogue System looks promising as a semi-realistic combat space sim: http://imagespaceinc.com/rogsys/

Pioneer also exists as an open source Frontier remake, with the same sort of flight model and universe: http://pioneerspacesim.net/

Not to mention how badly stress can screw with memory.

If you're used to entering your password in a specific relaxed situation and now you're filled with dread every time you think about it, damn right you might not be able to remember it. That's just basic neurology.

My bare Linux repository comes to 1.2GB. A git gc run got it down to 1.1GB.

Work on packaging up FreeBSD base is ongoing: https://www.bsdcan.org/2015/schedule/events/563.en.html