mwilliamson

A made a tool called Whack [1] as an attempt to solve a small part of the problem. The idea was that you could write scripts that would build statically-linked versions of binaries, such as Apache HTTPD [2]. Then, you could run a command like:

    whack install git+https://github.com/mwilliamson/whack-package-apache2-mod-php5 apps/apache2

to install to any location you wanted, and the installation would still work if you mv'd the entire directory somewhere else, or even onto another distribution (provided it didn't have an older libc).

I never really the pushed the idea further since I think doing a decent job requires a lot of time and effort, and it solves something which is a minor annoyance for me, rather than causing significant pain. I do still use it for when I need to quickly get a WordPress server for development up and running since it's one command to deploy from scratch.

It feels to me like there's lots of scope for something similar to apt-get that builds and installs static binaries (or, at the very least, lets you install to different directories without root, similar to, say, virtualenv for Python or RVM for Ruby).

EDIT: actually, from a quick glance, it seems like Gentoo Prefix [3], as mentioned by ryao, allows you to install things as you normally would on a Gentoo machine, but without being root and with a prefix of your choosing.

[1] https://github.com/mwilliamson/whack [2] https://github.com/mwilliamson/whack-package-apache2-mod-php... [3] https://www.gentoo.org/proj/en/gentoo-alt/prefix/

This article has a few examples of basic income experiments, with encouraging results. For instance, when the town of Dauphin had basic income implemented, hospital visits went down by 8.5%, which suggests real benefit rather than just frivolous spending.

https://decorrespondent.nl/541/why-we-should-give-free-money...

I believe they're all still Red Gate folk. Specifically, if you take a look at their About page:

> Regard is an open source product built by a team working for Red Gate Ventures in Cambridge, UK.

Only just noticed that it's open source as well. https://github.com/with-regard/

I think this is an excellent idea. It lets companies retain the benefits of analytics, but gives users the appropriate power over information concerning them. A couple of questions occurred to me.

Firstly, how much do end-users worry about the transparency of their data? For many users, I get the impression that this is something that they've simply never worried about (although that's somewhat anecdotal!). Do you think that's going to change (or isn't true), or perhaps something that we, as an industry, should be working to change? What's the motivation for a company to adopt this over other, less transparent analytics?

Secondly, I had a look at "My data". At the moment, it's a list of JSON. I'm not sure how transparent this is to non-technical people, nor how transparent it will be when the list contains hundreds or thousands of entries. Do you have any plans to make this more approachable?

I think a problem is that we tend to overgeneralise our own experience. When we try out ideas, such as TDD, we (hopefully) get a good feel for how they work (or don't work) for us. Reading articles or getting advice might help us to understand and use those ideas better, but the impression is still based on your personal experience. That's fine when you're talking about how you like to work, but I think it comes unstuck when you start trying to apply it to other people. Even if you read other people's perspectives, it's often tempting to consider those that disagree to be idiots (although you might not be so blunt), and those that agree with you to be sensible.

I think it's more productive to try out a technique, try out some variations based on others' advice, and then share the experience: I found this technique useful in these situations, but not in these. Here are some tips that I found effective, and here's what didn't work for me. Then, people can try out techniques for themselves, and learn from your experience.

If we want to generalise what's effective and what's not for software development overall, then I think we need a much rigorous approach then getting a vibe from the community. I've only ever seen a handful of studies that try to be unbiased and somewhat scientific in assessing techniques like TDD.

Having built something similar [1], the reason I didn't use "grep -ri license" is that I wanted to generate the list of licenses for both my project and all of its dependencies. Using grep would still require matching up each project to its own license.

Another problem is that the license file doesn't necessarily state the name of the license, which is often what you're looking for. (My package compares the text of the license to saved copies of common licenses, and picks the closest match, assuming any are a good match.)

[1] https://github.com/mwilliamson/node-license-sniffer

I tend to prefer using testing for getting recent packages on a desktop -- packages automatically migrate from unstable to testing if no bugs are reported within 10 days, which tends to mean a slightly more stable system, but still with reasonably recent versions.

Software developer, Stat.io - Cambridge UK, or REMOTE

We aim to allow users to understand the socio-economics of any location, from the population of a country to the crime rate of a postcode. Organisations can then use this data to improve their decision-making, whether it's businesses making strategic sales and marketing decisions, NGOs trying to understand a country, or somebody else entirely.

The two main technical challenges we face are:

* Importing data from many different sources, from APIs to Excel files, and understanding their structure. For instance, given an Excel file, we need to determine the structure of any tables, such as which columns represent a location. We also want to pick up as much metadata as possible, such as a description of the dataset.

* Designing a UI that allows users to find what they're looking for, and then making that UI fast (we're currently up to about 10,000,000 datasets and 200,000,000 data points)

Our technology stack:

* Postgres + PostGIS to store the data we've imported

* Elasticsearch for... search

* Python code to import data from the original source to our database (with a tiny bit of Redis for some basic queueing)

* A thin PHP web application to display the data (probably soon to be replaced by node.js)

Things we like:

* Test-driven development and good test coverage (when it makes sense -- sometimes, it's better to manually test a very thin UI than to keep worrying about brittle UI tests)

* Releasing early and often. We deploy features as they're ready, and try to break work down to small pieces.

* Doing the simplest thing that could possibly work. Rather than planning for every possibility up front, we implement something small but useful to start with, and then iterate and refactor as necessary.

What we're looking for:

We were founded in Cambridge last year, and we're now working towards a product covering public data for the UK.

We're looking for a software developer to join us, initially part-time for £1,000 a month + stock option plan. We're joining Microsoft Ventures in London from February 2013, so you can join us in Cambridge or London, or work remotely.

If you have any questions, feel free to get in touch: contact@stat.io. If you'd like to apply, please send us a CV and a paragraph describing why you're interested.

You can find out more about us on our website: http://www.stat.io/

I suspect it's because there are backwards-incompatible changes between 0.x versions, whereas 1.x should (arguably) be backwards-compatible.

Software developer, Stat.io - Cambridge UK, or REMOTE

Stat.io is building a web application that lets people use open data to improve decision-making. Our vision is to aggregate data from hundreds of open data portals, which can then be accessed in one place. You can see a very early demo of the application over here:

http://www.stat.io/dev/

The challenge is to understand millions of datasets from many different sources, from APIs to Excel files, and then provide an interface that allows users to find the data they're interested in.

The main technologies we're using for data processing are Python and MySQL, with a bit of Redis and MongoDB for some simple queueing and logging respectively. We're open to using any technologies if it helps get us closer to our vision. For instance, there's a good chance that alternative data stores to MySQL would be a better fit for the problems we face.

We're looking for a software developer to join us, initially part-time. If you're interested or have any questions, get in touch: contact@stat.io.

You can find out more about us on our website: http://www.stat.io/

I used to be a developer for some technical websites (publishing articles, blogs, newsletters, that sort of thing), mainly around SQL Server and .NET. Almost all of the authors used Microsoft Word, the main value being familiarity and using Track Changes. I came to appreciate Track Changes when I wrote or edited the occasional article, and I can't imagine it would be easy to find a replacement for the functionality that would work smoothly enough to justify convincing all the authors and editors to switch. Fortunately, the authors and editors were pretty good about using the house style in Word, meaning that the articles were semantically marked up. I wrote a converter that did a pretty good job of generating clean HTML from the source Word, mainly by mapping Word styles to the corresponding pieces of HTML.

One of the more interesting challenges was the mismatch between the structure of Word docx files and HTML. Whereas HTML has plenty of nesting, docx files have a comparatively flat structure. For instance, suppose you want a nested list of depth 2 that looks like this:

    * Outer
        * Inner

One way a docx (XML) file can represent this is a paragraph element with the style "Bullet1" (and the text "Outer"), followed by a paragraph element with the style "Bullet2" (and the text "Inner"). The two elements are completely separate, and you have to infer that they're part of the same list, and that the second bullet is the child of the first. Once you've done that, you can generate the corresponding HTML, which has the inner list as a child of the outer list element.

If anybody's interested, I wrote a Word to HTML converter for node.js using the same ideas: https://github.com/mwilliamson/mammoth.js

Software developer, ProjectPolicy - Cambridge UK, or REMOTE

ProjectPolicy is a startup building Stat.io, a web application that lets people use open data to improve decision-making. Our vision is to aggregate data from hundreds of open data portals, which can then be visualised in one place. You can see a very early demo of the application over here:

http://www.stat.io/demo/

The main technologies we're using for data processing are Python and MySQL, with a bit of Redis and MongoDB for some simple queueing and logging respectively. We're open to using any technologies if it helps get us closer to our vision.

We're looking for a software developer to join us. At the moment, the technical team consists of me (the CTO) and one intern. If you're interested or have any questions, get in touch: mike@projectpolicy.org.

You can find out more about us on our website: http://www.stat.io/

Originally, I tried using similar tools (I think I used chrpath, I can't remember whether I tried patchelf) to implement whack, but I found it quite time-consuming to get things working, and had limited success. I chose the current implementation since it was simpler, quicker and more reliable, and the setuid requirement wasn't a problem for my particular use case.

Having said that, it would be great to have an implementation that didn't require setuid/root.

I'm no Linux security expert, so any advice on that front is hugely appreciated.

whack-run is supposed to drop any privileges it holds when it runs "setgid(getgid())" and "setuid(getuid())", and it exits early if either of those calls fail. Is there any reason why that wouldn't work?

whack-run is a separate C binary for this very reason:

https://github.com/mwilliamson/whack-run

Whack itself is run with normal privileges.

Containerization solves this problem, but it also solves many more. Whack is intentionally narrow in scope, which means that there's relatively little to learn as a user. Whack certainly isn't a replacement for Docker and the like.

You're absolutely right, but "relocatable" was the best word I could find that's used for the concept that was relatively accurate. The other usual adjective is "portable", but I felt that was misleading since you can't just stick the binaries on a USB stick, since the target computer needs to have whack-run installed, which requires root access.

Suggestions for alternatives are extremely welcome!

You just need to be root to install whack-run, since whack-run requires the setuid flag. Once you've installed whack-run, you can install the applications as an ordinary user.

Yes, so long as the hard-coded path for the the config file is within the output directory, the program will find the right config file for that particular installation. You still won't be able to put the config file in an arbitrary location, but at least you can have isolated instances of the program with isolated config files.

In my experience, a lot of Linux programs aren't straightforward to compile statically. Having said that, that might just be my ineptitude!

Which bit of the caching isn't clear? If you're happy that whack creates relocatable versions, then the caching works just by copying the output directory into ~/.cache, and then copying that directory to the target on subsequent installations. The exact details of how programs are made relocatable are under "How does Whack work?" in the README.

There was an article in NYTimes a little while ago that suggested that getting six hours of sleep a night for two weeks was equivalent to being awake for 24 hours:

http://www.nytimes.com/2011/04/17/magazine/mag-17Sleep-t.htm...

All told, by the end of two weeks, the six-hour sleepers were as impaired as those who, in another Dinges study, had been sleep-deprived for 24 hours straight — the cognitive equivalent of being legally drunk.

My personal experience certainly tells me that a solid eight hours a night makes me far more productive than consistently getting six.

As an example of taking this to an extreme, I was at a coderetreat where we had to implement Conway's Game of Life without any if statements (or similar, such as switches) -- we had to use polymorphism instead. The result was that my partner and I ended up reimplementing a subset of the natural numbers as distinct classes.

http://mike.zwobble.org/2012/12/polymorphism-and-reimplement...

I'm definitely not advocating this as good programming practice, but the point is that if you're used to always using if statements, then it's hard to learn alternatives. By forcing yourself to use the unfamiliar, you might find some situations where polymorphism is better suited to the problem, whereas you would have previously defaulted to using ifs.

(barrkel has already left an excellent comment on when the two styles are useful, so I won't repeat it:

http://news.ycombinator.com/item?id=4977487)

When I was learning to drive (in Hemel Hempstead), I found it easier to get around the magic roundabout if you just thought about it as a road that happened to be a small circle with a few mini-roundabouts on it. Even when it's busy, I've found it quite straightforward to get around, mainly because you rarely have to wait long at a mini-roundabout (if the car to your left is turning right, you can start going since the car on the right is blocked).

I remember reading a comment on another story which I thought was a wonderful explanation of why estimates tend to be too short:

http://news.ycombinator.com/item?id=3522910

To quote one part:

Let's say that you have 20 tasks. Each involves rolling a 10-sided die. If it's a 1 through 8, wait that number of minutes. If it's a 9, wait 15 minutes. If it's a 10, wait an hour.

How long is this string of tasks going to take? Summing the median time expectancy, we get a sum 110 minutes, because the median time for a task is 5.5 minutes. The actual expected time to completion is 222 minutes, with 5+ hours not being unreasonable if one rolls a lot of 9's and 10's.

> It would seem that the tasted of Coke is affected as much by its branding as by its recipe!

There was an interesting experiment where some researchers put subjects inside an MRI machine, and then fed them two samples of cola. Sometimes both drinks were anonymous, and sometimes the drink was labelled as Pepsi or Coca-Cola, while the other was anonymous but actually the same drink. When the drinks were anonymous, people seemed to like both equally, but when told what the drinks were, people tended to prefer Coca-Cola over the anonymous drunk. Even more interestingly, when people were told they were drinking Coca-Cola, extra bits of their brain lit up, specifically memory and cognitive control, which suggests that when you think you're drinking Coca-Cola, it's not just the immediate sensation of drinking the coke that affects how much you enjoy it, but also the effect of all of that marketing that makes your brain think "Mmmm, Coca-Cola is delicious!"

http://health.usnews.com/usnews/health/briefs/mentalhealth/h...

Or even simpler:

    var collected = $('a').map($)

However, map as defined on jQuery objects takes a function that takes the index as the first argument, and the element as the second argument:

http://api.jquery.com/map/

So, what you actually want is:

    var collected = $('a').map(function(index, element) {return $(element)})

However, jQuery.map has the element as the first argument, and the index as the second argument:

http://api.jquery.com/jQuery.map/

I would consider this to be the more intuitive way round, although I'd prefer it if both maps were consistent. So, you could write:

    var collected = $.map($('a'), $)

which strikes me as the neatest version.

I regularly see people getting 12am and 12pm mixed up -- using midnight and noon definitely reduces confusion.

In my opinion, one of the big problems with formal methods is getting the specification right. You might be able to prove that your program conforms to your specification, but what is to say that your specification is any less buggy than your program? Some interesting work has been done with proving that certain errors do not occur, rather than proving the entire program correct. For instance, Microsoft's Terminator project [1] is used to prove that device drivers do not hang.

[1] http://research.microsoft.com/en-us/um/cambridge/projects/te...

What about XenSource? I believe they were later than VMware, and they made a tidy sum when Citrix bought them.