aidanhs

One trick that helped when I was doing log crunching (where time parsing was a good 10%) was to cache the parse.

All log lines began with a date+time like "2015-12-10 14:42:54.432" and there's maybe 100 lines per second. You can therefore just take the first 19 characters, parse that to a millisecond unix time and then separately parse the milliseconds to an int and add that. All you need is one cache entry (since logs are mostly in order) and then you can just do a string comparison (i.e. no hashmap lookup) to check the cache - instantly 100x fewer time parsing calls.

The best way to speed up a function is to not call it!

1. I'm not certain on the precise binary layouts etc, but I believe the answer is "yes, except Go binaries can't be stripped". In terms of practical usage, they'll run on the same machines.

2. This is a long one.

The statement you've quoted is false. However, it's both a) a common misunderstanding and b) a sort-of-ok simplification (the full story is much more involved). I'd personally alter it to say "it is technically challenging to statically link glibc". You can read more about the whole musl Rust origin story at [0]. The 'problem' with glibc is that it uses NSS, a feature that allows you to dynamically load libraries installed on the system to let you change how some libc functions work (if neither you nor your libraries use these functions, static linking with glibc works). For example, musl will look up users from /etc/password, whereas with glibc an admin can install an LDAP library, change a config file and all programs using glibc will magically use LDAP. But, you can disable NSS in glibc at compile time [1], which then allows you to truly statically compile with glibc.

Go is interesting because doesn't use libc at all (when using the standard compiler)...except when doing some network things that NSS is useful for, when it does link against it. People who want totally static binaries pass a few additional flags to say "don't use NSS, use the Go implementation of these features"...and you then end up with bugs like [2].

3. Yes [3], but you lose (by default) a) the ability to use shared libraries from the system, b) NSS. Given that one angle Rust is pushed from is "C/C++ replacement", not being able to link to system libraries without using arguably cryptic command line arguments would be a bit sad. But I'm ambivalent about this.

[0] https://internals.rust-lang.org/t/static-binary-support-in-r...

[1] https://sourceware.org/glibc/wiki/FAQ#Even_statically_linked...

[2] https://github.com/docker/docker/issues/1715

[3] (specific post from [0]) https://internals.rust-lang.org/t/static-binary-support-in-r...

To me this raises a question about selling security vulnerabilities to state actors in general (in the context of the Facebook vulnerability thread where the standard discussion about value is being hashed out).

Specifically, I live in the UK and one of the complaints law enforcement has is that US companies can (and do) totally ignore valid court orders because they don't apply in the US (reddit being an arbitrary concrete example).

So, what would be the impact of GCHQ setting up a scheme where you can sell vulnerabilities to them (assuming they do the legwork to make it legal)? Would it violate some kind of trade agreement? I assume at minimum it would harm diplomatic relations given the pressure the big companies would exert on the US to push back.

The linked article abstract from Psychological Science says:

> The way to maintain high comprehension and get through text faster is to practice reading and to become a more skilled language user (e.g., through increased vocabulary). This is because language skill is at the heart of reading speed.

So, in short, moving your eyes over the page faster only helps if your speed of language interpretation can keep up (the nytimes article has a slightly paraphrased version).

I wasn't really impressed by the article. It seemed to add very little to the abstract, just a couple of concrete examples of speed reading techniques (and one accompanying study).

Defining speed reading as 'eye movement speedups' and explicitly excluding 'language comprehension speedups' is fine, but it makes me want an article about what happens when you combine both, which I know for a fact can result in reading speedups by a whole number multiple.

This looks very cool, definitely something I'm tempted to play with (having never done any dev in this area before).

However, the feedback loop for debugging drivers on Windows seems quite manual (https://msdn.microsoft.com/en-us/library/windows/hardware/mt...). On Linux I'd have SSH at my disposal - are there any best practices for speeding up this copy/install process? Cygwin sshd could be an option, off the top of my head?

Your comment reminded me of this highly entertaining talk - https://www.destroyallsoftware.com/talks/the-birth-and-death...

Although tongue in cheek, I think it gives some food for thought. I feel like WebAssembly is to asm.js what the modern JS profession is to old follow-your-cursor effect on webpages - it becomes something to take seriously and use, and having done a bunch of porting things with Emscripten the idea of a browser within a browser doesn't sound as crazy as it used to!

> and it didn't exist with Docker (because it uses layers)

There's a big spectrum of 'solving the deduplication problem' and Docker is towards the 'when all you have is a sledgehammer' end.

Saying "you can manually arrange Dockerfiles so that they cooperate and share layers" is not solving things in an architectual sense! You essentially have to be using a single custom tool (Dockerfiles) to create your images and then you need to apply thinking power to consider how best to arrange your images (e.g. having a 'base' package.json to install a bunch of things common to many apps, then additional an additional package.json per-app).

It's getting better with 1.10 (layer content is no longer linked to parent layers, so ADDing the same file in different places should reuse the same layer) but it's still pretty imperfect. I created https://github.com/aidanhs/dayer to demonstrate the ability to extract common files from a set of images and use it as a base layer, which is another improvement. Even better would be a pool of data split with a rolling checksum, like the one bup creates - short of domain-specific improvements (e.g. LTO when compiling C), I think this is probably the best architectural thing you can do.

Writing such a thing from scratch is pretty impressive - the twelve rules of Tcl syntax are readable (http://www.tcl.tk/man/tcl/TclCmd/Tcl.htm), but it's up to you to figure out the edge cases!

As a different take on the same problem, I compiled Tcl to JS with Emscripten - http://aidanhs.github.io/emtcl/. I also wrote a toy DOM library (https://github.com/aidanhs/emtcl/blob/master/opt/dom.c) which plugs into it which you can see demonstrated on the page.

One of the nice things about EmTcl is that you can include arbitrary C extensions by also compiling them with Emscripten, which is how the DOM library above works. As a more comprehensive demonstration, have a look at TclDis (http://aidanhs.github.io/tcldis/). That has Tcl, a Tcl C extension I wrote for Tcl <-> Python communication, a Python interpreter and the Python script I wrote for performing Tcl decompilation all in the same page! (it probably won't work well on a mobile browser)

Emscripten truly is incredible.

A few years ago I had a friend visiting Iran who wanted unrestricted access to sites. I didn't have any personal Linux servers on the internet at the time, but I did have a Windows one with Remote Desktop licenses.

It turned out that RDP actually worked pretty well. I did hesitate to post this in case it's seen by the wrong people(!), though given it's a while since it was necessary to use, it may be blocked by now anyway.

I wonder if it was available because it was relatively little known and, if so, what other little known protocols might be available.

The links in this article go to some very good case studies on getting the correct tone for downtime status updates.

However, I found the most interesting link the one about open salaries [1] - are there any other businesses doing similar things? I'd be inclined to consider it a very strong mark in their favour as a workplace.

[1] https://open.bufferapp.com/introducing-open-salaries-at-buff...

I think you're being overly dismissive when it comes to easy dev environments. I was recently fixing the reddit vagrant environment and it was absolutely excruciating:

- Creating a scratch VM (not even provisioning) is a speed bump when you want to re-running scripts on a pristine environment to validate them. Starting a Docker image takes ~1s.

- Provisioning is slow! Reddit suggests installing a plugin (vagrant-cachier) to keep you sane. I ended up downloading a plugin to take VirtualBox snapshots of my VM and even that was depressingly slow. Docker commit takes maybe 20s on a big layer.

- VirtualBox shared folders are pretty bad, so apparently I should install vagrant-bindfs (and NFS packages). -v is effectively zero effort.

- I had to keep on shutting down and starting up the VM to tweak memory settings - too big and my laptop died, too small and nothing would start. You get far more flexible controls in Docker.

I couldn't help but think all the way through that putting everything in a Docker container would just be a far nicer experience, even if it does violate the single process rule - for this use case, Docker solves some papercuts very effectively.

This article is interesting because there seems to be an assumption that there is some ideal the web should strive towards ("free exchange of information"), when actually everyone is optimising for their local maxima.

Google wanted to challenge Facebook and had a huge userbases on lots of different platforms...you can see the reasoning which would lead them to add everyone to an umbrella google social network. The Verge saw a way to make money with relatively little investment on their part - a seductive offer. Soundcloud (presumably) found a business plan with a clear path to profit and evaluated it to be the best option.

In the end, all of these companies exist to make money. Those that do stick around to be criticised, those that fail are irrelevant! Buzzfeed is a great example because it's so polarising - it's criticised for having low quality content, and yet it's extremely popular, a perfect example of something that people clearly do actively want to use. If you had an isolated choice reduce your userbase by 20% but increase profit by 10x, that's an easy business decision to make.

I just find it difficult to make sweeping criticisms of these companies and the internet as a whole - these are independent evolutions, each of which can be criticised on their own demerits. Of course the internet will evolve to consist mostly of what the majority wants - this isn't inherently good or bad.

Also an impressive achievement on the software side of things - no room for showstopper bugs here! It's not quite as simple as taking it offline, deploying some new code and starting it up again.

Is there any information on the processes used to have such a high level of confidence in the software? Lots of simulation? Relatedly, it might also be interesting to see a list of 'known bugs' and ways they've been worked around.

I've tried reading the circular [1] (which seems to be a summary of the changes) and can't see anything about immunity. The article also lacks direct quotes or links to relevant sections.

Does anyone have more specific source on this detail?

Edit: this factsheet [2] on the computer misuse amendments only mentions a clarification that investigation may be performed against UK citizens even if they're located outside the UK at the time of the crime. No mention of immunity or extra powers.

[1] https://www.gov.uk/government/uploads/system/uploads/attachm...

[2] https://www.gov.uk/government/uploads/system/uploads/attachm...

A little frustrating that github's down, as I'd be interested to see their build process - with some trickery I was able to get empython [0] down to 2.5MB gzipped from 7MB, though I'm aware pypy as a binary is quite a bit larger anyway.

I also wish github was up so I could look at the benchmark code - startup looks much faster with empython, but I'd like to know how much slower interpreting would be.

[0] http://aidanhs.github.io/empython/

Has there been any CP system passing a call-me-maybe test first time apart from Zookeeper [1]?

[1] https://aphyr.com/posts/291-call-me-maybe-zookeeper

The problem I have with statements like this is that it gives no context for what 'not healthy' actually means (and, by extension, what a 'perfectly healthy' diet is).

For example, I often don't hit 3 of the 5 recommended fruit/veg in a day and I consume far too many fatty foods. Over an extended time frame this will probably cause me to become overweight and deficient in vitamins. Given this, it'd be useful to know what the advantages and disadvantages of soylent would be in my current situation.

Do you have a link to the studies? They presumably give more information about their benchmarking.

At present it uses asciidoctor[1] to go from asciidoc to docbook, then to .docx via pandoc[2] and .pdf via a Manning-provided tool (for styling reasons).

pandoc is pretty incredible. As you can see from its page, it can produce epubs and many other things - it's likely that with pandoc and a docbook file you can build a pipeline to take you wherever you need. For your purposes, you'd probably go from EPUB to kindle with KindleGen.

[1] http://asciidoctor.org/ [2] http://johnmacfarlane.net/pandoc/

This is pretty interesting. For the book I'm co-authoring at the moment[0] I ended up putting together my own toolchain, including:

1. A dead-simple web IDE for asciidoc with live preview[1]

2. A script for checking for common AsciiDoc mistakes (e.g. an unterminated table will mess up all text following it).

3. A script which enforces the house style.

4. A makefile which runs docker (of course) to build the book with Manning styling.

5. A buildbot configuration on my server to run 2, 3 and 4 above on `git push` and move the generated pdf and docx into a directory served by nginx.

Though it may seem like a lot, this investment has been totally worth it - the ability to push a change and be viewing the updated pdf in ~1 minute is very liberating. I can't imagine what it'd be like to not have this kind of thing in place. I can share some more of the above if there's interest.

If you're ever thinking of writing a book yourself, I'd highly recommend making the investment into a process like the above (or a tool like GitBook, though I've never tried it and it seems to be more than just a build tool) so you can get the boring stuff out of the way and focus on what you want to be doing - writing.

[0] http://docker-in-practice.github.io/ [1] https://github.com/aidanhs/AsciiDocIde

Networking with Docker is a pretty hefty topic. One of the interesting things talked about in the book in this area is skydock[1], which provides automatic dns registration inside docker when you start your containers. We've found it very handy for simulating a network of real machines!

[1] https://github.com/crosbymichael/skydock

For a long time I couldn't figure out why moving from bash to Python felt like such a big leap, and it was quite frustrating (path manipulation is particularly difficult).

Recently I discovered a post [1] which touches on this:

> The whole point of short code is saving human bandwidth, which is the single thing in a computing environment that doesn't obey Moore's law and doesn't double once in 18 months. Now, which kind of bandwidth is the most valuable? I'll tell you which. It's the interactive command bandwidth.

It makes the argument that syntax is actually really important in building an interactive language - '[]' is better than '()', ' ' is better than ','. And it's true, bash (and, as it happens, tcl) ruthlessly pursue this - strings don't require quotes, function calls just require spaces (rather than brackets and commas) and there's generally very little punctuation for the most common operations.

I bring this up because it's interesting to see bish has gone in the other direction (curly braces, semicolons, brackets, quoted strings) in the name of a 'modern feel'. It makes me wonder if there's a way to bring the power of python (etc) into the shell in a syntax-friendly way.

[1] http://yosefk.com/blog/i-cant-believe-im-praising-tcl.html

I too like my Lumia more than any previous phone I've had.

Unfortunately there's one key problem which means I won't be getting another one - I have no way of getting an offline copy of my phone backups.

I didn't realise for a while after purchase and it's not such an issue to make me immediately want to get a new phone, but I refuse to relinquish that control by choice.

> May as well just use a virtual machine at that point.

While this isn't true (as explained by others), I don't know why there's such dismissal in the docker community for a VM with ~zero boot time, good tools without requiring ssh (`docker cp`, `docker diff`, `docker exec`, volumes) and layers. These are good things! Of course packaging in a light-weight way is nice if you can manage that as well but it's not everything.

> Nice and composable, reproducible infrastructure.

Phusion passenger is also reproducible you're still using (the highly touted) Dockerfiles. I've no idea how you want to define composability.

> There is definitely a shortage of developers

Even this isn't quite true - I'd qualify 'developers' with 'good'. There are a lot of average/bad developers out there and companies don't really want to hire them...but many don't have a huge pool of candidates like Facebook and Google.

For my purposes, I wrote a makefile which starts up a container with ssh, then provisions it with ansible (as you would with any normal remote host).

This meant a) I could reuse existing scripts to provision normal server b) docker could be used to test the ansible scripts were actually capturing everything required.

The cache point was a bit of a pain though, I confess.

> Startup times will vary - mostly based on the app. Docker advertises millisecond startup times, Thinapp is probably in the 10s of milliseconds to startup [...]

The simplest test I can think of (`docker run busybox true`) takes 0.38s wall clock, pretty far out from your statement even when allowing for it stopping as well.

I've not seen millisecond startup times since pre-1.0 on any of my machines (there was a post on the mailing list about the mailing list about this slowdown a while back). When/where have you seen this advertised?

Edit: unless we're excluding the time for the implicit `create` from this, which doesn't seem like a very helpful metric.

Let's say PulseAudio is really good now (I can't disagree). It was initially released 10 years ago. So 6-7 years after initial release ("3-4 years ago") it was causing people grief.

I'm not sure that's a great vote of confidence for the road ahead of systemd (given that systemd presumably has a bit more to it than PulseAudio). To quote the article, "I do honestly believe this will end up being the start of a rocky period for Linux".

Ubuntu 14.04 will keep me happy for ~5 years, then I can take another look at what the current state of things are.

Never thought about this, interesting question.

You can try this equivalent, killing the server when it tells you to - https://gist.github.com/aidanhs/e40417381e9aecb87b35. It prints '/', i.e. would remove your root directory if it was an `rm` command.

Won't be doing that again then.

> Currently there is definitely a case where there are too many calls in Rust that will just panic.

So, as a newcomer to rust, how can I identify which calls could blow up my single-task app? The docs on println! (http://doc.rust-lang.org/std/macro.println!.html) say nothing about this.

I could use task::try everywhere (http://doc.rust-lang.org/guide-tasks.html#handling-task-pani...), but I'd prefer to have an indication of which functions are pure and cannot error (modulo bugs).

I quite like Jake as a make-equivalent in Javascript and was relieved to discover it after experiencing 20s grunt build times.