This is also a security problem. If I have the means to run a program (downloaded from the internet; don't fully trust) in a sandbox that restricts its filesystem access, it'd be nice to also not allow it to control my whole GUI session.
632 karma · joined December 22, 2011
This is also a security problem. If I have the means to run a program (downloaded from the internet; don't fully trust) in a sandbox that restricts its filesystem access, it'd be nice to also not allow it to control my whole GUI session.
I ended up doing a lot of custom stuff that is highly specific to the particular SAML implementation in use at my company. For example, the public signing key is included with the SAML response (on successful sign on) as an x509 certificate. I extract if from there and establish trust by verifying the signature chain (up to the root SSL certs included in my distro). But, SAML also allows embedding raw RSA or DSA keys, or presumably none at all (and you'd store the trusted key somewhere as a configuration value). So I don't support "SAML" but "the specific SAML implementation currently in use at my company".
But I've worked closely with an OAuth client before as well, and in the end I'd say they are roughly equivalent in terms of ease of use, but there was a larger learning curve for SAML (and XML-DSig is frustrating due to whitespace sensitivity among other things. Implementation seems like a huge burden if your language lacks it).
[1] https://github.com/onelogin/java-saml But see other language impls. as well
One thing missing from Nix is ability to store passwords and the like in the nix store, and from what I understand this applies to NixOS as well as using Nix outside NixOS. For example, every file in the nix store is world readable.
> Machine would suspend and resume without problem, WiFi just worked, audio did not stop working, ... [no] fighting the video drivers
Sure, and I'd like to do that build within Nix (and someone else might want to do it with another packager), which gives a stronger guarantee than Bundler since it incorporates C library dependencies and more. Anyway, the specifics aren't relevant to this discussion, and it seems you have a grasp of the issues, so carry on!
Previously, I wrote shell scripts and worried whether everyone on the team had rsync installed, or xmlstarlet, or some other less common tool. Now I wrap those scripts in a Nix package that explicitly depends on all those and distribute with confidence. It's fantastic.
Bundler and rubygems, for example, do various things that make good support within Nix rough. Two examples: 1. rubygems has no standard way of declaring dependencies on C libraries; 2. as far as I know there is no way to ask Bundler to resolve dependencies, create a Gemfile.lock, but not install any gems (I realize github gems must be downloaded to see the gemspec...)
I'm not aware of a way to do that in Nix, but I think it's more a consequence that the convention is to call
stdenv.mkDerivation { descriptor }
(where the descriptor includes things like the upstream url, version, etc.)without providing a direct handle to the descriptor (which one might inherit and modify just the upstream url, for example).
(In practice, I find it an order of magnitude easier to create custom packages in Nix compared to .deb.)
Valgrind is your friend :)
http://valgrind.org/docs/manual/quick-start.html#quick-start...
At what point do you give up and, if you must have portability, bootstrap a saner environment?
Are there many options in this area? Assume perl is available? Use autoconf-like shell script compiler? Starting with Lua (which I hope/assume can build anywhere, though I know it's missing lots of functionality out of the box)?
That's like saying exposed high voltage lab equipment has not caused a single shock, careless lab techs bumping into it have.
When you design for safety, you assume mistakes will happen, and you design accordingly. Continuing with the voltage example, for a low voltage, one that will shock but not cause lingering pain, a simple barrier would suffice to prevent careless bumping into. For higher voltage, a box with a door connected to an interlock that cuts power when the door is open is better. To receive a shock, either the interlock would need to fail, or the operator would need to deliberately defeat it. In even higher voltage situations, where the shock could cause permanent damage or death, trusting that single interlock would be insufficient, and further protections would be needed.
In general, assume mistakes will happen, and design so that a serious failure requires multiple mistakes and/or multiple component failures. Never trust your life to a single component not failing.
Whether C "causes" buffer overruns is irrelevant semantics. Using plain C without strong static analysis is equivalent to working directly on exposed voltage. Perfectly fine when nothing bad will happen as a result of inevitable mistakes. Negligent when securing a bank.
Surely you're not trading performance for safety, as dozens of languages are "safe" while having much higher performance over Ruby including similarly dynamic languages JavaScript and Lua, or others such as Java, Scala, Clojure, Haskell, OCaml, and so on.
There are, of course, other reasons to choose Ruby, perhaps because of Rails, large talent pool, code conciseness, etc.
We know what these are?
There are strategies to prove various things about recursive calls. See Ada/Spark's explanation for Rule 17.2 in [1].
I'm not familiar with Coq and Agda, but I believe they allow recursive calls if they can prove termination, though "termination" may be different than "using too much stack".
[1] http://www.spark-2014.org/entries/detail/misra-c-2012-vs-spa...
One of the largest components in an inverter (such as found in a Toyota Prius) is the capacitor bank. I'll ignore the electrical design and just assert you need X capacitance to get this done. At least in automotive world, polymer film capacitors are used for this purpose. A polymer film capacitor is made from a (very long) sheet of polymer coated on both sides with a thin layer of metal and rolled up into a cylinder. They get quite bulky at capacitances required by these inverters. The other downside is polymer film cannot handle high temperature. I believe the Prius includes an whole extra cooling loop (in addition to the main loop attached to the engine, which runs hotter than the capacitors) to keep the capacitors cool, so that's even more bulk.
A multi-layer ceramic capacitor of similar capacitance can be much smaller, and can handle far greater temperatures than any polymer. The reason why polymer film is preferred is that when ceramic capacitors fail, they do so catastrophically in much the same way as a ceramic dinner plate shatters. At sufficient voltage, or at lower voltage with a sufficient defect, the ceramic will breakdown: a conduction path will form between the electrodes through the ceramic, which will heat the surrounding area, causing thermal expansion, shattering, and permanent destruction of the capacitor.
The same thing happens in polymer film capacitors, except that because the material is flexible, it does not shatter, and only a small hole around the defect will be ablated away. The remaining capacitor loses some capacitance, but otherwise functions normally.
So one way to create a smaller inverter is to use smaller capacitors, but you've got to match capacitance, voltage-handling ability, and fail gracefully.
I have some code that uses Bundler to convert its Gemfile.lock (resolved dependencies) into a set of nix expressions. I think it's a nice way to go.
(Though Bundler is aimed at end users, which for my use case means an annoyingly large amount of functionality and no stable api; e.g. it insists on installing gems as part of the dependency resolution process.)
Last I checked, Nix has no way to publish just a small nix expression for one project. You'd really need to include the entire stack of nix expressions for all your dependencies, all the way down to gcc (which is exactly what nixpkgs is), or somehow declare "depends on http://nixpkgs/2014-07-24.tar.xz".
For example, rubygems cannot properly express dependencies on C libraries. Rubygems that include C extensions run the compilation step during "gem install" [1] (which typically assumes the presence of a C compiler, etc.). A gem like libxml-ruby assumes the presence of libxml development headers. (Nokogiri punted and includes an entire libxml source tree in its gem which leads to its own problems.)
Nix has some means to convert rubygems into nix packages, and it has a list of overlay dependencies to apply to various common gems (e.g. libxml-ruby gets enhanced with that dependency on libxml).
Language-specific package managers handle simple cases well, but other cases make increasingly large assumptions about the external world that complicate automation and confound repeatability.
[1] There is support for binary gems, but I don't know of anyone who uses these in practice, nor do I know of a repository of pre-compiled gems.
What you describe would be fairly easy to script. I'm playing around with similar ideas. It's not exposed by default, but the included script `nix-prefetch-git` dumps a git repo into the nix store, and when you run it from your shell, you have access to ssh keys and ssl certs (that are unavailable within a normal nix build).
The next step is harder like you say, you'd either need to include nix expressions for all dependencies on down through libc and gcc, or just assume/hope that "whatever happens to be in the local nixpkgs" is appropriate, but then you kinda lose out on a lot of the point of nix. It'd be nice if there were some way to reference a version of nixpkgs from which the expressions for "ruby20" and its dependencies would be drawn (but then you'd end up with hundreds of slightly different versions of nixpkgs tarballs from every github project with no easy way to update).
> automatic package installation tools ... live dangerously on the edge of intractability, and must carefully apply heuristics that may be either safe, and hence still not guaranteed to avoid intractability, or unsafe, thus accepting the risk of not always finding a solution when it exists.
In other words, you either risk running into pathological cases (as I did with Bundler) that will come up with a solution only after many hours (or more), or you use methods that terminate quickly but will not find every solution.
[1] http://polycrystal.org/2012/03/07/bundler_searches_an_np_com...
I've been trying to get it to manage some Ruby applications, complete with gems, private github gems, etc. This has been difficult, in part because some areas of nix are a bit rough, and in part because Rubygems has incomplete dependency information (i.e. there's no standard way to know that, say typhoeus gem depends on C library libcurl, and I'm still unclear on how/if dynamic loading can be made to load the nix packaged libcurl).
Something like nix is what I had in mind with my original question. I'd like to see new languages at least design their custom package managers with these more generic systems in mind. Rubygems has some quirks, and Bundler just includes everything and the kitchen sink (sourcing gems from local files, github, various other features) that complicate matters.
Nix is good for keeping things more or less isolated and tracking complete dependencies, though an alternate way to accomplish similar is something like Docker and then just scripting all the native package managers...
So the question is, how fast can an outlier in talent run with minimal or suboptimal training? Or what kind of training can one get "by accident"?
Jim Ryun [1] (along with a handful of others) ran under 4 minutes as a high-school student (a junior in Ryun's case). He didn't exactly fall out of bed and do that, but neither was he a professional having many years of training leading up to the achievement. Ryun's training was probably not optimal. Running enthusiasts often complain that his coach was a former swim coach who did not use training routines commonly recommended today.
Many sports played on foot involve lots of aerobic running combined with occasional sprinting. Also consider Cliff Young [2] who claimed to have "run for two to three days straight rounding up sheep", presumably as a routine-ish part of his job. He went on to win an 875km ultra marathon.
It's all speculation of course, but given all that and the fact that modern mile record is quite a bit faster than 4 minutes, I think it's plausible that a highly talented individual got enough training from work, sports, or perhaps even dedicated training to run a 4 minute mile.
I know you weren't arguing against daemontools, but just wanted to share some thoughts:
> Service directories make sense, but they tend to be shell scripts all the way down
This is true, but what are you doing in your "run" script that is so complex? Also, any executable will suffice. s6 provides 'execline' [2] which simplifies some use cases (and complicates others...)
> Logging is quite different and non-centralized
Arguably a feature. I'm quite enjoying the "logging chain" where every service logs to stderr which either goes to its specific logger process or bubbles up to that of the parent process.
The typical usage of syslog is to collect all log messages in a centralized inbox and then (by some pattern matching; log messages must contain enough to be categorized) split them up again and write them to separate files. Skip all that with logger-per-service.
> Services need to be set not to fork and do their own daemon management.
Clearly a feature. There is no need for the redundancy of multiple, error-prone implementations of daemonization routines in every service. Also, language nitpick, it's not that a service can't fork. For example, daemontools can happily supervise a Unicorn master process that will fork off its own workers. The service must not background itself by severing ties with its parent (which involves forking of course).
But there are downsides and quirks as you say. Two examples:
If you really want something like Nginx or Unicorn to be able to restart without dropping http connections (e.g. fork+exec new master process, gracefully shutdown and kill old master process), then you will run into issues. It's possible to hackishly wrap this in a helper script. Daemontools would supervise the helper, which would in turn manage nginx via plain old racy PID files, just like init.d scripts.
I'm only familiar with daemontools and s6, but they assume TERM is the signal to use to stop a service, and don't provide configuration to send, say, QUIT which Unicorn interprets as "graceful shutdown". (Upstart also did not allow this until fairly recent versions.) It's one thing to get a service to not daemonize itself. It's another thing to expect upstream to modify its signal handling.
[1] http://skarnet.org/software/s6/ [2] http://skarnet.org/software/execline/
https://github.com/NixOS/nixpkgs/blob/master/pkgs/applicatio...
Is that not AwesomeWM btw? There are some rough edges around package discovery (or holes in my knowledge).
If you install nix locally (just the package manager, not NixOS), I usually query like this:
nix-env -qaP | grep awesome
Or I search through my local checkout of nixpkgs source.I'll say that I'm finding nix (just using the package manager, not NixOS) to be a very nice way to manage some custom-compiled packages. I was able to upgrade nginx, for example, from 1.5 to 1.6 with all my plugins and compile flags in place with just a few small changes. This is surely possible through other means, but my previous method of manually downloading/compiling/saving configure flags in my $HOME/src was sloppy.