Freaky

Just coming up to 100,000 flight hours with my 6. No real complaints, but quietly planning retirement to secondary mass storage. Picked up some surprisingly cheap Toshiba 5TB's for evaluation[1].

Did you notice anything in particular leading up to failure? IO errors, reallocated/pending sectors, checksum errors?

1: http://www.toshiba.co.uk/hard-drives/internal-hard-drives/sa...

> Testing with `openssl speed -evp AES128` on an i5-4670K I'm able to get 6.7 gigabits of encryption out of AES-NI

Bit of a crappy test, though - tight loop over a single 8k buffer (used in-place). It probably executes entirely in L1/L2.

They provide details of the hardware they're using: https://openconnect.itp.netflix.com/hardware/

The fanciest in their "IO-optimized" SSD-driven beast is a 2.7GHz 12 core Ivy Bridge. You've got a 20% clock advantage, and Haswell reportedly reduced AES-NI clock latency from 8 to 7 cycles for a ~14% improvement. Doing the math I get 55Gbps for the lot.

That's without doing any packet processing, no waiting on memory accesses, no interrupts, no context switching, just perfect scaling with a tight loop over the same 8k buffer - and they're driving 4 10Gbps NICs. Not much left over.

If you're just pushing raw bytes down a socket, you can use sendfile(2) to direct cached data directly to the NIC without copying it to a private buffer first. This is a pretty big deal at 40Gbps, and Netflix have funded quite a bit of work towards improving it:

  https://secure.freshbsd.org/search?project=freebsd&q=netflix+sendfile

TLS kicks all that in the teeth - each encrypted stream needs its own private buffer which has to go in and out of the CPU before it gets anywhere near the NIC. That costs memory bandwidth (a mere 400Gbps with quad-channel DDR3), memory you could have otherwise used as cache (keeping in mind you need to hold onto it until the client ACKs it), and CPU cycles (probably still a decent chunk of overhead even with AES-NI).

BSD's have tcpdrop(8):

     If	a connection to	httpd(8) is causing congestion on a network link, one
     can drop the TCP session in charge:

	   # sockstat -c | grep	httpd
	   www	    httpd      16525 3	tcp4 \
		   192.168.5.41:80	192.168.5.1:26747

     The following command will	drop the connection:

	   # tcpdrop 192.168.5.41 80 192.168.5.1 26747

I wrote a partial quicksort for Newzbin - once I saw how simple it was to add I was surprised it wasn't more widespread: https://github.com/Freaky/pqsort

There are still inputs that will make it go quadratic, that just obscures it slightly. A proper fix is to fall back on a different sort if it looks like quicksort is doing that: https://en.wikipedia.org/wiki/Introsort

Unfortunately S3QL uses MAC-then-encrypt[1], which is pretty strongly discouraged[2]. Very nice there's a detailed writeup on the details in the docs though, wish more projects did that.

1: http://www.rath.org/s3ql-docs/impl_details.html (last paragraph) 2: http://www.daemonology.net/blog/2009-06-24-encrypt-then-mac....

Tarsnap is fantastic, and I'd love to be able to use it more, but it's 25x more expensive than Glacier (not counting the whacky retrieval costs) and 10x more expensive than S3, so it's not really practical for bulk backups unless your data is particularly valuable.

Only for JPGs, though, no RAWS or anything else. And upload speeds are capped. And all your photos are in cleartext on Amazon's end.

Add OPTIONS_UNSET=gnome to /etc/make.conf, plus whatever other options you want excluded system-wide. Ditto OPTIONS_SET. You can override for specific ports with ${PORTNAME}_SET/_UNSET too.

End-to-end encryption means it's encrypted between the endpoints - any intermediate servers just see ciphertext and never have access to the key to decrypt it. SSL just encrypts the link to Google's servers - it should be basically irrelevant since the data should already be encrypted and digitally signed.

Things are looking a little brighter on the ZFS pool shrinking front thanks to Delphix working on device removal: http://blog.delphix.com/alex/2015/01/15/openzfs-device-remov...

Note there are quarterly ports branches you can track if you just want security fixes - you're not just limited to tracking HEAD: https://secure.freshbsd.org/search?project=freebsd-ports&bra...

There's work ongoing on FreeBSD to add libxo support to tools in the base system, which will allow you to get (amongst others) JSON out of various commands: https://github.com/Juniper/libxo

Medusa seems like a better approach, since it doesn't require detonating as close to the spacecraft and can forgo the big heavy pusher plate: http://www.centauri-dreams.org/?p=23754

Juniper gate.

Probably just more sensitive to vibration in long term use. 40% AFR in normal conditions would be insane, and I doubt we'd need Backblaze to point it out.

I have six ST3000DM001's and seven ST31500341AS's, all with about 2 years on the clock (~20,000 power-on hours). With 40% and 25% AFR's respectively, the odds of all thirteen surviving are:

1 / ((0.6 12) * (0.75 14)) = 1 in 25,782.

I doubt I'm that lucky.

Speaking of C APIs, JRuby+Truffle is going all-out:

> Our new solution in JRuby+Truffle is pretty radical - we're going to interpret the C source code of your extension. We use the same high performance language implementation framework, Truffle, and dynamic compiler Graal, to implement C in the same way as we have implemented Ruby.

http://www.chrisseaton.com/rubytruffle/cext/

aboutthebsds.wordpress.com is a well known troll site, don't link to it like it contains anything meaningful.

Missed that, thanks. Direct link: http://www.bbc.co.uk/iplayer/episode/b04w7ytd/charlie-brooke...

Yep, aka rcNG: https://www.usenix.org/legacy/events/usenix01/freenix01/full...

They were in 7.3-RELEASE, March 2010.

https://www.freebsd.org/releases/7.3R/announce.html

> FreeBSD needs to either port their ZFS drivers to their bootloader, or employ a hacky method of having a UFS boot volume that then points to a ZFS root partition (which, sadly, is how FreeBSD currently works).

A ZFS-aware loader hit CURRENT in late 2008, and a dedicated zfsloader for use from (gpt)zfsboot hit the stable branches in late 2009.

As an anecdotal data point, I see ECC corrections much, much more often than ZFS CKSUM errors. If you're in any way concerned about bitrot, ECC memory should be the first thing you go for regardless of your choice of filesystem.

> Under a regular user account?

Yep. Only special factors I can think of are VirtualBox guest additions and passphraseless sudo - though I don't recall any similar problems when I last had it on my laptop which had neither.

Also starting from console and not a display manager, because they're vile things that are almost as bad as splash/boot screens.

> I know their team is much smaller so it's just not feasible, but it's definitely something that would be greatly appreciated.

Very much so - the biggest thing I miss from the Debian world is unattended-upgrades. Maybe with some more commercial support..

> Is OpenBSD's MP-safe yet? I believe that's partly related to why FreeBSD's has ALTQ disabled

It's still under a big giant lock as far as I'm aware. They're also removing ALTQ in favour of a simpler framework: http://undeadly.org/cgi?action=article&sid=20140419151959

Snapshots with LVM still imply a relatively costly(?) quiescing of the filesystem for however long it takes to complete pending writes and perform the snapshot. This is certainly my experience with snapshots on UFS2 - they're supported, but you're not going to be too pleased performing them automatically from a cronjob every 15 minutes.

Snapshots can also be promoted to fully-fledged filesystems using cloning, which is very handy for spawning off fully-featured jail environments and for setting up boot environments for a host.

Filesystem replication. My primary backups involve using `zfs send` to maintain remote copies of my snapshots on another machine with minimal cost in bandwidth and disk IO.

Filesystem compression. My root pool is just 0.63x the size it would have been thanks to lz4, basically for free.

As X-Isence says, end-to-end data protection. Every bit of data and metadata are checksummed so any errors are detected, and if possible automatically repaired. Crucial metadata is duplicated even on single-disk configurations.

Copy-on-write semantics make for more robustness in face of problems - active data is never overwritten in-place. This also makes for much safer parity RAID than traditional approaches.

> I have to make this org.freedesktop.consolekit.pkla file and add entries to it in order to get the restart and shutdown buttons in Xfce to work

These Just Worked for me. My only real annoyance with Xfce is that it's unresponsive to mouse clicks for the first few seconds after starting.

> I really, really appreciate Debian's branches. If you install Wheezy, you can get security updates for packages, but not get version bumps. With FreeBSD, you have to choose between "the packages made at release time", or "the absolute bleeding edge."

There are quarterly port branches now to give people more of a choice: https://lists.freebsd.org/pipermail/freebsd-ports-announce/2...

> encrypted swap

Be nice to see this on by default these days, but if you're not already using it: swapoff -a && $EDITOR /etc/fstab && plop ".eli" at the end of the swap device name && swapon -a. Done.

> I also really like pf a whole lot more than iptables

I just wish it were more actively maintained and kept at least vaguely in-sync with upstream. Makes me reluctant to switch from ipfw.

FreeBSD migrated to Subversion in 2008.

No. There is a vfs.zfs.min_auto_ashift sysctl you can poke now instead of messing about with gnop, but it still defaults to 9 (512b).

They're less labels for purpose and more labels for lifecycle - data you'd consider "cached" can appear just about anywhere. From what I remember (corrections more than welcome):

Active is fully-fledged in use memory, mapped into one or more processes

Inactive is where Active goes when it's less in-use. Cheap to reactivate, but relatively costly to free: can still be mapped into processes, and may be dirty (modified) and thus require writing to disk before being unmapped and cleared for reuse.

Cached is where lesser used Inactive cached data goes before it dies. More costly to reactivate, but cheap to free: No longer mapped directly into any process, and strictly consists of only clean (unmodified) pages that don't need writing back to disk before clearing.

Wired is pinned-down memory that can't be swapped out. ZFS's data and metadata caches are counted here, since it maintains its own (known as the ARC - Adaptive Replacement Cache, after the algorithm it's based on) instead of just relying on the traditional VM page cache.