lambda

Note that the author himself is working on a framework for distributed differential dataflow computations (dataflow computations in which it is very cheap to make small updates to the inputs and see the differences in the outputs), and he has plans to apply the same metrics (comparing to a single-threaded implementation) to his own work once he gets a little further:

https://github.com/frankmcsherry/blog/blob/master/posts/2017...

I think that answering the "fastest way to do X in a cluster" question is only particularly interesting if X is something that you need to do in a cluster. But clearly they are testing X that is capable of being done on a mid-range laptop. There's a real question of whether the X that they are testing will actually scale to larger datasets than the laptop (or a decently beefy single node) could handle; it's entirely possible that the cluster solution will blow up at that point. And of course, if the single-threaded implementation is faster, then you could presumably just do "X in a cluster" by doing "X on a single node" and using the rest of the nodes to mine your CPU bound cryptocurrency of choice.

> He is limited by not having an "expensive macbook with 4 real cores" ?? He gave up on a test because it "paged out the 15GB of data".

Note that he already demonstrated that for many of the tests, his older MacBook was faster than these 160 core clusters. There were a couple where the older MacBook was not faster than a 160 core cluster, but he suspects that a newer one might be.

The one that paged out happened to be one where he artificially broke it up into 160 pieces to simulate what it would be like if he had 160 similar cores, but that caused enough issues with memory locality that swapping dominated the results. This artificial partitioning was to give a sense for what results on a 160 core cluster "should" be, but the swapping distorts the results enough that for that problem it wouldn't even be a very good rough estimate.

> Is this guy for real?

This isn't supposed to be a publication-quality paper. This is a blog post, using the resources he has immediately at hand, to show why if you follow cutting edge distributed database research you might be led down a wrong, and very costly, path, which you could avoid with a much simpler implementation on much cheaper hardware.

Note that he references an earlier paper that he did publish on this topic; this blog post is mostly just doing a similar check on some more recent published work, to see if much has improved in the field:

https://www.usenix.org/system/files/conference/hotos15/hotos...

But it is a distinct city. With a separate mayor, a separate budget, and so separate issues that it has to deal with due to its budget shortfalls. Including issues with the City of Detroit being one of its creditors, due to unpaid water bills.

It is not uncommon for there to be cities or towns in the metro area of other cities that are independent. For instance, Cambridge is different than Boston; or for an even closer example, Brookline which is surrounded on three sides by Boston is independent from Boston. Or Oakland is not San Francisco, nor is Almeda Oakland.

Or for other examples of being completely surrounded, the Vatican is not Rome. Monaco is not France.

The "technical boundary" is not irrelevant here. We're talking about different jurisdictions, with different governments, and an article about how one government is failing to provide some basic services is distinct from another government that is close by but has very different issues due to the differences of scale and available tax base.

> with fields in between them, as opposed to a continuous metropolitan area.

But that's not how cities work. It is very common for two cities to be part of one continuous metropolitan area, and the only way you notice the boundary is by paying very close attention to signs as you pass them.

> My question is whether there might be some political context to the way this article is written. Do you know of any?

I think that the context for this article is just that it's an interesting phenomenon; this small city, which grew very fast during the auto boom and has been fading slowly ever since, and now the government is so cash-strapped that it doesn't even have the personnel to work with independent non-profits to do things as basic as keeping the streetlights on, but also due to the small scale it's possible for individuals and neighborhood organizations to provide some of their own services that the city can't.

The issue of "the government barely exists", as expressed in the title, is not true of Detroit as a whole. While the Detroit government does have issues of being short on funds, it is large enough that there is a recognizable government that is able to do things. But for this small city, which grew very quickly and then has slowly been fading away, the size and scale make it very hard to have a functioning government that can reasonably govern.

And while the state and the federal government obviously do exist, for many of the things that ail Highland Park, the state and federal government generally work by providing grants to cities and towns, but being as cash strapped and small as this city is, it's hard for it to take advantage of programs that could bring in more revenue.

The problem being written about is that the city itself has so little income and revenue that it can't provide certain basic services, and even can't necessarily partner with non-profits which are trying to address problems in different ways.

With a different government, tax base, and so on, Detroit as a whole may have similar problems, but the scale is different, and the larger scale likely means that there are some things Detroit can do that Highland Park cannot.

I'm not sure why you think that an article about one city, Highland Park, should have been about a different city, Detroit, that while it is close by, operates at a very different scale?

It is a separate city, that is surrounded by Detroit except for one small portion touching Hamtramck which is also surrounded by Detroit: https://en.wikipedia.org/wiki/Highland_Park,_Michigan

It's not terribly uncommon for a city like Detroit to annex surrounding towns and cities, but some of them hold out, which looks like is what happened here.

Those aren't all of the instructions for apt-get. The full instructions are on https://docs.microsoft.com/en-us/sql/linux/quickstart-instal... and appear to consist of:

  curl https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -
  sudo add-apt-repository "$(curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list)"
  sudo apt-get update
  sudo apt-get install -y mssql-server
  sudo /opt/mssql/bin/mssql-conf setup
  sudo apt-get install -y mssql-tools unixodbc-dev
  echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bash_profile
  echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
  source ~/.bashrc

And at that point, you have a limited period trial license of a release candidate version installed, and so once you want to use it for production, you have to start dealing with license management issues, which could limit your ability to easily set up dev envs, test envs, hot spares, etc.

All of that is on top of the issues of how heavyweight it is due to containing a platform abstraction layer that consists of a very large amount of Windows running in user mode, as discussed by others in this thread.

I have an SSD (along with a USB SATA enclosure to connect it), a spinning disk, and at least one USB thumb drive in my bag that I bring to work every day (along with my laptop).

This is in part because I switch between working from home and working at the office fairly often, have travel time in between, and need to have access to more VMs than I can fit on my laptop's internal SSD comfortably.

I also may have SIM cards rattling around somewhere; when I travel internationally, I buy a local prepaid SIM, and I may have older SIM cards left over from the last time I replaced one.

No, he said:

> I ran a simple HTTPS web server with net/http, loaded a page on Chrome, and instead of causing a kernel panic...

Followed by demonstrating it working.

However, the point remains that he only got it working up to doing a toy hello world. The part that would be important for performance would be what he mentioned isn't finished, which is allowing it to be used with sendfile so that web servers can just sendfile over a TLS connection and let the kernel handle all of the IO.

What does WeChat do that isn't handled by other, separate apps?

For instance, I don't really see a need to have the same app do chat and payment (in fact, I'd prefer if they didn't).

What about WeChat is particularly innovative, other than just bundling a lot of different functionality into one big app?

Have you looked at Rust? It uses Option types, has generics, also uses return values for error handling but has a lot more syntactic sugar to make it cleaner than in Go.

Huh, I had the opposite reaction. I thought that it was a very nice, clear article, which is phrased to be approachable by as many people as possible with some amount of technical background.

That technical background may not be as programmers. It may be power users, or web developers, or the like.

And I thought the analogies and illustrations helped out with giving some context that people who aren't systems developers might be missing.

The point is that when installing, you are not rebuilding. The hypothetical example here is for some kind of randomizaiton that applies, for instance, to the kernel during build time, such as randomization of structure layout. Having to rebuild every kernel you install to get a security patch, on every system you install it on, would be an enormous amount of time spent (there's a reason Gentoo is a fairly niche distro).

Such randomization can mitigate against certain kinds of attacks, especially making particular instances of an attack work differently between each kernel build, and thus increasing the difficulty of writing exploits that work across a broad range of kernel builds. There are about 30 different patch releases of the kernel for Debian Jessie alone; so such randomness, if applied across the different builds of a kernel within a release, across various different Debian releases, across different architectures, and also applied by other distros, can substantially increase the difficulty of writing an exploit that can affect a substantial fraction of Linux systems on the net.

That helps provide a form of "herd immunity", where the herd is all deployed Linux systems on the net. It doesn't provide any real protection for a particular build; so it's no harder to write an exploit that targets all system running the exact same kernel. But it does dramatically increase the difficulty of writing a widespread worm which relies on the given exploit, and which can easily spread between a variety of different systems.

Anyhow, that footnote was on a policy for explicit exceptions from the general policy. The general policy is that yes, builds should be reproducible, and any randomness should be generated locally. The footnote was giving a few example of potential exceptions which may be required, as a way of demonstrating that it would be fine to have the general requirement be reproducible builds with case-by-case, narrow exceptions for cases in which non-reproducible builds provide significant value.

And note that such exceptions may be based on how an upstream project operates. If the upstream kernel has some modules that build non-reproducibly in such a way, it may be more viable to encode an explicit exception to the policy for that case, than to remove that non-reproducibility.

It's important to keep in mind that someone applying a cleanup once accidentally removed almost all entropy from OpenSSL's random number generator on Debian (they even asked for review from upstream, and failed to get it, because of the confusing naming of upstream's mailing lists). Having an excessively rigid policy could mean that Debian maintainers would be forced to remove features that do provide some kind of benefit.

CloudFlare's "Flexible SSL" (https://www.cloudflare.com/ssl/) offers encryption/authentication from CloudFlare's server to the client, but none from the origin server to CloudFlare's. Which means that is a vector by which the content could be sniffed or modified in transit.

It's a "better than nothing" option, as there are a slightly higher number of actively exploited attack vectors that apply to the client to CDN connection than the CDN to origin server, such as "free" wifi that injects ads, malicious ISP DNS, and the like. But it's not actually secure, as the origin server to CDN connection could be tampered with, and just because there are fewer active attacks that would be likely to affect that connection right now, doesn't mean that someone won't come along later and hijack such a connection.

CloudFlare offers other TLS options that do include encryption and authentication between the origin server and CDN, but they do require that you set up a certificate on your server, so if all you're trying to do is enable TLS (and don't care about the CDN), just installing a cert on the origin server and using TLS is probably a simpler option that using CloudFlare.

  But if you actually try to advocate that people attend to 
  their health first and foremost by eating right, 
  exercising and making good lifestyle choices, you are some
  crazy weirdo?

I don't think that's what the article was sneering at.

It was sneering at the idea of fearmongering about what people eat, but then trying to sell them on a whole bunch of unregulated supplements and quack remedies.

There is an awful lot of "something free on the internet" that is just complete bullshit designed to sell you something.

Not according to http://quoteinvestigator.com/2011/08/09/remain-solvent/. Apparently what Keynes said was something similar, but not that same phrasing:

  There is nothing so disastrous as a rational investment 
  policy in an irrational world.

From what Elon Musk writes, his plan is that either you'll be renting other people's cars, or making some extra money by renting out your own: https://www.tesla.com/blog/master-plan-part-deux

Or as they say, the market can remain irrational longer than you can remain solvent.

And the reason that they are named "uppercase" and "lowercase" is that in the early days of movable type, the majuscule letters were stored in a case above the miniscule letters.

Furthermore, Roman and Italic are two entirely different styles of characters. For highlighting purposes, printers started using Italic within Roman text, and eventually pairs of Roman and Italic fonts became identified with each other, so that now typefaces generally include both of them.

Bazel was first released to the public in March, 2015. By that time, Cargo had already been in development for over a year, had reached release 0.1.0, and was already the official build tool for most Rust projects.

Furthermore, Bazel and Cargo are aimed at slightly different use-cases. Cargo is tailored towards Rust, is integrated with a Rust package repository, and can use it's knowledge of Rust to allow for builds of pure-Rust projects with a lot less configuration. Bazel is a more general purpose build system, but that means it requires a little more configuration. There's no reason you couldn't use Bazel for building a larger project that includes Rust and C, C++, Java, or other languages that need to be built, but Cargo makes it a lot easier and simpler to handle pure-Rust projects than it would be in Bazel.

Many of the other tools that you mention don't require just more support from the build tool, but also significant engineering effort on the part of the compiler, or a separate static analyzer, or other development process tools. These tools are desired, there just isn't anyone who has had the time to design and build them.

As to why Clippy is separate from the compiler, there are a couple of reasons. Adding more lints to the compiler, especially ones that are turned on by default, can cause problems as well. Any time you add a new lint, you may break people's code that has #![deny(warnings)] turned on. Even if people don't, a lot of people will still try to rewrite code to make it comply with lints; but that in itself can cause problems. I've seen many bugs introduced over the years by people trying to do too many lint cleanups at once, and making mistakes in some of their cleanups. For this reason, the Rust compiler is fairly conservative about adding new lints..

That said, it sounds like there are plans to integrate Clippy with the compiler and cargo, but probably as an opt-in separate command rather than being included in the default set of warnings: https://www.reddit.com/r/rust/comments/5ibr2a/cargo_check_ha...

Code coverage is another example. You can use some existing code-coverage tools with Rust (https://users.rust-lang.org/t/tutorial-how-to-collect-test-c...). Because there are tools out there that work, there's less of a pressing need to get code coverage integrated with the Rust toolchain; I think that it would be a good idea to do eventually, as it will make it easier to do cross-platform and out of the box, but for now it's not the highest priority.

So, I don't think that any of these omissions are due to a difference of philosophy; just a difference of focus. Rust is all about admitting that programmers aren't perfect, and that better tooling can help avoid mistakes and thus make developers more productive. Right now, that focus is on things like improving the compiler (adding MIR, which allows for fixing several bugs in the compiler and doing some Rust-specific optimizations before getting to LLVM), implementing the Rust Language Server which can be used as a backend for IDEs, improving the ability of the compiler to do incremental and parallel builds, and adding language features with a focus on making Rust easier to use.

If Gecko is ever able to get rid of all of it's C++, it won't happen for quite a while, that will be an enormous undertaking. I would expect, if Rust is successful enough that that is feasible, then by that time there would be multiple different Rust implementations, and some of them may be written in pure Rust.

There's no fundamental reason that the Rust compiler has to be dependent on LLVM. It's just a good strategic decision as it allows the Rust developers to focus on the parts of the compiler that are unique to Rust, and use a well tested backend with lots of existing optimizations and targets for the parts that aren't particularly unique to Rust. There is actually some discussion already of using a faster, pure-Rust backend for debug builds, and maybe far in the future for release builds as well (https://internals.rust-lang.org/t/possible-alternative-compi...).

I'm an Emacs user, so that's what I've written most of my Rust in, but I tried out VSCode and setting up all of the Rust integration, and it was pretty easy to set up and seemed to work pretty well playing with it briefly. Didn't seem to be worth switching from Emacs for, but I was fairly impressed, and will likely recommend VSCode for anyone looking for a Rust IDE who doesn't already have a strongly preferred editor or IDE.

It's a framework for building cross-platform applications in which the UI is implemented in HTML/CSS/JS using an embedded Chromium and Node.js.

http://electron.atom.io/

For cross-platform apps, Qt is the best I've used. But it does have its warts, and it is big, it will probably bloat your app by about as much as Electron does.

But yeah, there's no one clear winner, which means that Electron is not a uniquely bad choice. Sure, it might be a little heavyweight for a really small simple app, and if you're not aiming for something cross platform your native widget toolkit might be a better idea.

Can vouch for it. I found my current job through SO Careers, and now that we're trying to hire more people, we're finding a good number of our better candidates coming through our SO Careers posting.

> I'm wondering what part of the twelve step compilation process was found to be easier than simply installing Python on the target machine?

That part is something that you can control.

Giving instructions for someone non-technical to install Python and run a script with it, and then helping them figure out how to interpret those instructions, is a heck of a lot more complicated.

And yeah, you can diddle around with cx_freeze and then having to spend some time with InnoSetup or WiX to build an installer, and then do the same on OS X but putting everything into an app bundle inside of a disk image instead, and finally on Linux trying to figure out how to bundle something up to play nicely with all of the major distros and packaging systems, and get something to work, though it doesn't support cross compilation so you now need to set up and maintain three different build machines, which is definitely no simpler than the solution described.

All of these solutions have their ups and downs. But being able to relatively easily (modulo installing a linker and a few libraries) cross compile from Rust to a static executable for any of the major desktop platforms is pretty nice, after you've dealt with enough of these funky build and packaging systems or trying to walk people through figuring out how to add Python to their path on Windows.

"I don't see how you can have a humane prison not be a criminal university."

By having it promote positive improvement and rehabilitation rather than purely punishment.

Here is what a humane prison looks like:

http://www.bbc.com/news/magazine-35813470 http://content.time.com/time/photogallery/0,29307,1989083,00... https://en.wikipedia.org/wiki/Halden_Prison

That's not the DNC, that's the New York Board of Elections. The DNC doesn't run the actual voting of the primaries, that's done by the local election commissions, just like for the general elections. This varies by state, it's generally the state parties that decide how primaries are run in their state, not the DNC.

If you are your own company, thus working as a contractor, that's different than hiring an employee to work in an existing company. There are lot of obligations that companies have for each employee in terms of taxes, health insurance, and the like, and in the US they can differ from state to state. If you hire an independent contractor, however, all of those obligations fall to the contractor themself, and the contractor is likely only dealing with one state's laws, so they don't have that complexity.

This is a little different than codec support, though as other replies indicate, that can be done too. This is just demuxing the stream in one format, re-muxing it in a different format, and feeding it into <video> to actually do the decoding and playback.

On the other hand, something like https://github.com/brion/ogv.js/ is actually doing all of the decoding and playback in JavaScript/asm.js, using C libraries compiled with Emscripten.

Anyhow, yes, it probably would be possible to package up extensions that would apply one of these automatically to <video> tags with unsupported codecs or container formats. It's just a simple matter of programming.

> with the eventual goal of being pure Rust

No, being pure Rust is not the goal. It aims to use Rust as much as possible for the parts that Rust is good at. But core crypto algorithms generally need to be written in assembler, to avoid various timing attacks that could be introduced by optimization. And for things that would require large amounts of `unsafe` in Rust, there's less reason to port that to Rust, and leaving it in C can be more clear.

See the style guidelines from the project:

https://github.com/briansmith/ring/blob/master/STYLE.md#unsa...

The thing that are most appropriate for Rust are parsing, protocol implementation, higher level code that uses core crypto primitives, and providing a safe API to client code. But the core crypto primitives themselves will remain written in C and/or assembler, as appropriate.