HNHacker News
TopNewBestAskShowJobs

pmahoney

632 karma · joined December 22, 2011

submissionscomments
pmahoney··on Interview: Thomas Voß of Mir
> A program that runs on your computer has access to all your files anyway.

This is also a security problem. If I have the means to run a program (downloaded from the internet; don't fully trust) in a sandbox that restricts its filesystem access, it'd be nice to also not allow it to control my whole GUI session.

pmahoney··on Single Sign-on at Rackspace
I found the tools weak at best, confusing and overcomplicated at worst. The code from OneLogin [1] was the clearest example I found, but it leaves various things as an exercise for the reader, such as obtaining the relevant public key and establishing trust in that key.

I ended up doing a lot of custom stuff that is highly specific to the particular SAML implementation in use at my company. For example, the public signing key is included with the SAML response (on successful sign on) as an x509 certificate. I extract if from there and establish trust by verifying the signature chain (up to the root SSL certs included in my distro). But, SAML also allows embedding raw RSA or DSA keys, or presumably none at all (and you'd store the trusted key somewhere as a configuration value). So I don't support "SAML" but "the specific SAML implementation currently in use at my company".

But I've worked closely with an OAuth client before as well, and in the end I'd say they are roughly equivalent in terms of ease of use, but there was a larger learning curve for SAML (and XML-DSig is frustrating due to whitespace sensitivity among other things. Implementation seems like a huge burden if your language lacks it).

[1] https://github.com/onelogin/java-saml But see other language impls. as well

pmahoney··on Aptly: Owning Your Debian Distribution
I deploy a couple things via Nix package manger to non-NixOS systems, and I manage my own system configs, etc. within Nix. My "external configuration management system" is pared down to a small script that symlinks everything in a directory into another directory (such as /etc/init).

One thing missing from Nix is ability to store passwords and the like in the nix store, and from what I understand this applies to NixOS as well as using Nix outside NixOS. For example, every file in the nix store is world readable.

https://github.com/NixOS/nix/issues/8

pmahoney··on Shellshock DHCP Remote Code Execution – Proof of Concept
Execline is a little mindbending at first, but a really nice way to write scripts for launching other programs.
pmahoney··on Bent iPhone claims put Apple under pressure to respond
The aluminum phone yields, undergoing plastic deformation, while the plastic phone deforms elastically.
pmahoney··on Switch default desktop to Gnome
But none of the reasons he lists have much to do with Gnome:

> Machine would suspend and resume without problem, WiFi just worked, audio did not stop working, ... [no] fighting the video drivers

pmahoney··on The Road to Rust 1.0
> reason that you want it to do the installation is that a lockfile is supposed to represent the way to do a build successfully

Sure, and I'd like to do that build within Nix (and someone else might want to do it with another packager), which gives a stronger guarantee than Bundler since it incorporates C library dependencies and more. Anyway, the specifics aren't relevant to this discussion, and it seems you have a grasp of the issues, so carry on!

pmahoney··on The Road to Rust 1.0
I would probably make the same decision, but I hope in the end Caro is easy to wrap with Nix, which is a breath of fresh air particularly when needing to mix dependencies that cross language boundaries and share those build recipes with a team.

Previously, I wrote shell scripts and worried whether everyone on the team had rsync installed, or xmlstarlet, or some other less common tool. Now I wrap those scripts in a Nix package that explicitly depends on all those and distribute with confidence. It's fantastic.

Bundler and rubygems, for example, do various things that make good support within Nix rough. Two examples: 1. rubygems has no standard way of declaring dependencies on C libraries; 2. as far as I know there is no way to ask Bundler to resolve dependencies, create a Gemfile.lock, but not install any gems (I realize github gems must be downloaded to see the gemspec...)

pmahoney··on We're building the GNU system [pdf]
Excellent, thank you!
pmahoney··on We're building the GNU system [pdf]
I've not used Guix, but looking at those slides, the ability to take the build definition of the "hello" program, inherit everything but change the version, looks very interesting.

I'm not aware of a way to do that in Nix, but I think it's more a consequence that the convention is to call

    stdenv.mkDerivation { descriptor }
(where the descriptor includes things like the upstream url, version, etc.)

without providing a direct handle to the descriptor (which one might inherit and modify just the upstream url, for example).

(In practice, I find it an order of magnitude easier to create custom packages in Nix compared to .deb.)

pmahoney··on Show HN: Chan – pure C implementation of Go channels
The struct is currently storing a pointer to the mutex (which was allocated with malloc). The call to pthread_mutex_destroy does not free that memory.

Valgrind is your friend :)

http://valgrind.org/docs/manual/quick-start.html#quick-start...

pmahoney··on The fundamental problem of programming language package management
There are Java jars with native binaries, just sayin. Though I do like Maven and Java quite a bit.

https://bitbucket.org/xerial/sqlite-jdbc

https://github.com/twall/jna

pmahoney··on Unix Wildcards Gone Wild
> portably in shell scripts (spoiler: it's hard)

At what point do you give up and, if you must have portability, bootstrap a saner environment?

Are there many options in this area? Assume perl is available? Use autoconf-like shell script compiler? Starting with Lua (which I hope/assume can build anywhere, though I know it's missing lots of functionality out of the box)?

pmahoney··on Quality Software Costs Money – Heartbleed Was Free
> C has not caused a single buffer overrun. Lazy coders have.

That's like saying exposed high voltage lab equipment has not caused a single shock, careless lab techs bumping into it have.

When you design for safety, you assume mistakes will happen, and you design accordingly. Continuing with the voltage example, for a low voltage, one that will shock but not cause lingering pain, a simple barrier would suffice to prevent careless bumping into. For higher voltage, a box with a door connected to an interlock that cuts power when the door is open is better. To receive a shock, either the interlock would need to fail, or the operator would need to deliberately defeat it. In even higher voltage situations, where the shock could cause permanent damage or death, trusting that single interlock would be insufficient, and further protections would be needed.

In general, assume mistakes will happen, and design so that a serious failure requires multiple mistakes and/or multiple component failures. Never trust your life to a single component not failing.

Whether C "causes" buffer overruns is irrelevant semantics. Using plain C without strong static analysis is equivalent to working directly on exposed voltage. Perfectly fine when nothing bad will happen as a result of inevitable mistakes. Negligent when securing a bank.

pmahoney··on An underrated feature in Python 3
Can str() call into a C extension that then loops forever?
pmahoney··on What is type safety?
> Sure, I'm giving up a hell of a lot of performance for this safety, but it's more than worth it for my company's use case.

Surely you're not trading performance for safety, as dozens of languages are "safe" while having much higher performance over Ruby including similarly dynamic languages JavaScript and Lua, or others such as Java, Scala, Clojure, Haskell, OCaml, and so on.

There are, of course, other reasons to choose Ruby, perhaps because of Rails, large talent pool, code conciseness, etc.

pmahoney··on Personal Privacy Is Only One of the Costs of NSA Surveillance
I assume tertia is referring to something like https://en.wikipedia.org/wiki/Wickard_v._Filburn
pmahoney··on The Mediocrity Principle
> the conditions under which intelligent life needs to have in order to exist

We know what these are?

pmahoney··on How recursion got into programming: a comedy of errors
Well, if the compiler will also report error for recursive calls not in tail position...

There are strategies to prove various things about recursive calls. See Ada/Spark's explanation for Rule 17.2 in [1].

I'm not familiar with Coq and Agda, but I believe they allow recursive calls if they can prove termination, though "termination" may be different than "using too much stack".

[1] http://www.spark-2014.org/entries/detail/misra-c-2012-vs-spa...

pmahoney··on An open competition to build a smaller power inverter, with a $1M prize
I actually worked a little on this problem some years ago (as a lab tech; take my knowledge with a gain of salt).

One of the largest components in an inverter (such as found in a Toyota Prius) is the capacitor bank. I'll ignore the electrical design and just assert you need X capacitance to get this done. At least in automotive world, polymer film capacitors are used for this purpose. A polymer film capacitor is made from a (very long) sheet of polymer coated on both sides with a thin layer of metal and rolled up into a cylinder. They get quite bulky at capacitances required by these inverters. The other downside is polymer film cannot handle high temperature. I believe the Prius includes an whole extra cooling loop (in addition to the main loop attached to the engine, which runs hotter than the capacitors) to keep the capacitors cool, so that's even more bulk.

A multi-layer ceramic capacitor of similar capacitance can be much smaller, and can handle far greater temperatures than any polymer. The reason why polymer film is preferred is that when ceramic capacitors fail, they do so catastrophically in much the same way as a ceramic dinner plate shatters. At sufficient voltage, or at lower voltage with a sufficient defect, the ceramic will breakdown: a conduction path will form between the electrodes through the ceramic, which will heat the surrounding area, causing thermal expansion, shattering, and permanent destruction of the capacitor.

The same thing happens in polymer film capacitors, except that because the material is flexible, it does not shatter, and only a small hole around the defect will be ablated away. The remaining capacitor loses some capacitance, but otherwise functions normally.

So one way to create a smaller inverter is to use smaller capacitors, but you've got to match capacitance, voltage-handling ability, and fail gracefully.

pmahoney··on Babel – Nimrod's package manager
> dependency solver that generates nix expressions

I have some code that uses Bundler to convert its Gemfile.lock (resolved dependencies) into a set of nix expressions. I think it's a nice way to go.

(Though Bundler is aimed at end users, which for my use case means an annoyingly large amount of functionality and no stable api; e.g. it insists on installing gems as part of the dependency resolution process.)

Last I checked, Nix has no way to publish just a small nix expression for one project. You'd really need to include the entire stack of nix expressions for all your dependencies, all the way down to gcc (which is exactly what nixpkgs is), or somehow declare "depends on http://nixpkgs/2014-07-24.tar.xz".

pmahoney··on Babel – Nimrod's package manager
I don't really disagree with your points, but "the best solution we've found given those constraints" include a lot of pain. Something like Nix comes with a lot of different pain, but I suspect many pain points could be lessened if language-specific package managers were designed with consideration for more generic tools.

For example, rubygems cannot properly express dependencies on C libraries. Rubygems that include C extensions run the compilation step during "gem install" [1] (which typically assumes the presence of a C compiler, etc.). A gem like libxml-ruby assumes the presence of libxml development headers. (Nokogiri punted and includes an entire libxml source tree in its gem which leads to its own problems.)

Nix has some means to convert rubygems into nix packages, and it has a list of overlay dependencies to apply to various common gems (e.g. libxml-ruby gets enhanced with that dependency on libxml).

Language-specific package managers handle simple cases well, but other cases make increasingly large assumptions about the external world that complicate automation and confound repeatability.

[1] There is support for binary gems, but I don't know of anyone who uses these in practice, nor do I know of a repository of pre-compiled gems.

pmahoney··on Purely Functional Configuration Management with Nix and NixOS
> nix-pkg -i github.com/my/project

What you describe would be fairly easy to script. I'm playing around with similar ideas. It's not exposed by default, but the included script `nix-prefetch-git` dumps a git repo into the nix store, and when you run it from your shell, you have access to ssh keys and ssl certs (that are unavailable within a normal nix build).

The next step is harder like you say, you'd either need to include nix expressions for all dependencies on down through libc and gcc, or just assume/hope that "whatever happens to be in the local nixpkgs" is appropriate, but then you kinda lose out on a lot of the point of nix. It'd be nice if there were some way to reference a version of nixpkgs from which the expressions for "ruby20" and its dependencies would be drawn (but then you'd end up with hundreds of slightly different versions of nixpkgs tarballs from every github project with no easy way to update).

pmahoney··on Tail Calls, Optimization, and ES6
Sounds like the ST monad.

http://www.haskell.org/haskellwiki/Monad/ST

pmahoney··on Clib: Package manager for C
Regarding the computational difficulty of dependency resolution, I ran into a pathological case with Bundler a while back [1]. The money quote from the EDOS folk is:

> automatic package installation tools ... live dangerously on the edge of intractability, and must carefully apply heuristics that may be either safe, and hence still not guaranteed to avoid intractability, or unsafe, thus accepting the risk of not always finding a solution when it exists.

In other words, you either risk running into pathological cases (as I did with Bundler) that will come up with a solution only after many hours (or more), or you use methods that terminate quickly but will not find every solution.

[1] http://polycrystal.org/2012/03/07/bundler_searches_an_np_com...

pmahoney··on Clib: Package manager for C
I've actually been exploring nix myself recently. It has some interesting features. I'm finding it quite nice to maintain some C-based packages across both Linux and Mac OS X (where it is somewhat of a competitor to Homebrew).

I've been trying to get it to manage some Ruby applications, complete with gems, private github gems, etc. This has been difficult, in part because some areas of nix are a bit rough, and in part because Rubygems has incomplete dependency information (i.e. there's no standard way to know that, say typhoeus gem depends on C library libcurl, and I'm still unclear on how/if dynamic loading can be made to load the nix packaged libcurl).

Something like nix is what I had in mind with my original question. I'd like to see new languages at least design their custom package managers with these more generic systems in mind. Rubygems has some quirks, and Bundler just includes everything and the kitchen sink (sourcing gems from local files, github, various other features) that complicate matters.

Nix is good for keeping things more or less isolated and tracking complete dependencies, though an alternate way to accomplish similar is something like Docker and then just scripting all the native package managers...

pmahoney··on Clib: Package manager for C
Nice, but can anything challenge the "every language gets its own unique package manager"? (And own unique build system for that matter).
pmahoney··on The 18th Century four-minute mile
I think if you consider the outliers on the "talent" scale, such as Jim Ryun, Sebastian Coe, current record holder Hicham El Guerrouj (3:43.13) compared to the entirety of humanity over the past few centuries, you've got to assume there have been other outliers of equal or greater "talent".

So the question is, how fast can an outlier in talent run with minimal or suboptimal training? Or what kind of training can one get "by accident"?

Jim Ryun [1] (along with a handful of others) ran under 4 minutes as a high-school student (a junior in Ryun's case). He didn't exactly fall out of bed and do that, but neither was he a professional having many years of training leading up to the achievement. Ryun's training was probably not optimal. Running enthusiasts often complain that his coach was a former swim coach who did not use training routines commonly recommended today.

Many sports played on foot involve lots of aerobic running combined with occasional sprinting. Also consider Cliff Young [2] who claimed to have "run for two to three days straight rounding up sheep", presumably as a routine-ish part of his job. He went on to win an 875km ultra marathon.

It's all speculation of course, but given all that and the fact that modern mile record is quite a bit faster than 4 minutes, I think it's plausible that a highly talented individual got enough training from work, sports, or perhaps even dedicated training to run a 4 minute mile.

[1] https://en.wikipedia.org/wiki/Jim_Ryun

[2] https://en.wikipedia.org/wiki/Cliff_Young_%28athlete%29

pmahoney··on What's wrong with systemd
I agree with your points, though am disappointed by this reality. I've been aware of daemontools for some years, but only recently read over the design (which I find quite brilliant) and began using an implementation (s6 at the monent [1]).

I know you weren't arguing against daemontools, but just wanted to share some thoughts:

> Service directories make sense, but they tend to be shell scripts all the way down

This is true, but what are you doing in your "run" script that is so complex? Also, any executable will suffice. s6 provides 'execline' [2] which simplifies some use cases (and complicates others...)

> Logging is quite different and non-centralized

Arguably a feature. I'm quite enjoying the "logging chain" where every service logs to stderr which either goes to its specific logger process or bubbles up to that of the parent process.

The typical usage of syslog is to collect all log messages in a centralized inbox and then (by some pattern matching; log messages must contain enough to be categorized) split them up again and write them to separate files. Skip all that with logger-per-service.

> Services need to be set not to fork and do their own daemon management.

Clearly a feature. There is no need for the redundancy of multiple, error-prone implementations of daemonization routines in every service. Also, language nitpick, it's not that a service can't fork. For example, daemontools can happily supervise a Unicorn master process that will fork off its own workers. The service must not background itself by severing ties with its parent (which involves forking of course).

But there are downsides and quirks as you say. Two examples:

If you really want something like Nginx or Unicorn to be able to restart without dropping http connections (e.g. fork+exec new master process, gracefully shutdown and kill old master process), then you will run into issues. It's possible to hackishly wrap this in a helper script. Daemontools would supervise the helper, which would in turn manage nginx via plain old racy PID files, just like init.d scripts.

I'm only familiar with daemontools and s6, but they assume TERM is the signal to use to stop a service, and don't provide configuration to send, say, QUIT which Unicorn interprets as "graceful shutdown". (Upstart also did not allow this until fairly recent versions.) It's one thing to get a service to not daemonize itself. It's another thing to expect upstream to modify its signal handling.

[1] http://skarnet.org/software/s6/ [2] http://skarnet.org/software/execline/

pmahoney··on The Nix project – Atomic upgrades, rollbacks and multi-user package management
I found the nix expression language somewhat confusing at first, but I was able to create simple packages after a couple days. You can find many examples in nixpkgs, e.g.

https://github.com/NixOS/nixpkgs/blob/master/pkgs/applicatio...

Is that not AwesomeWM btw? There are some rough edges around package discovery (or holes in my knowledge).

If you install nix locally (just the package manager, not NixOS), I usually query like this:

    nix-env -qaP | grep awesome
Or I search through my local checkout of nixpkgs source.

I'll say that I'm finding nix (just using the package manager, not NixOS) to be a very nice way to manage some custom-compiled packages. I was able to upgrade nginx, for example, from 1.5 to 1.6 with all my plugins and compile flags in place with just a few small changes. This is surely possible through other means, but my previous method of manually downloading/compiling/saving configure flags in my $HOME/src was sloppy.

← PreviousPage 4 of 7Next →