brchr

"CO2 levels higher": Would you mind sharing your source on this?

Here are the last 6,000 years [1] (it cuts off at 4713 BC because that happens to be the earliest timestamp available in Postgres). The source data from the NOAA are here [2] and you can see for yourself that we are something like 30% higher than the highest level of CO2 recorded in the last 800,000 years.

If you have a source that tells a different story, let me know.

[1] https://numer.al/noaa_data/figures/antarctic-ice-cores-revis...

[2] ftp://ftp.ncdc.noaa.gov/pub/data/paleo/icecore/antarctica/antarctica2015co2composite.txt

Well said, and I share this intuition and desire 100%. I’ve been working on building a platform for data-driven journalism at https://numer.al -- It’s a prototype that turns public APIs into live D3 charts. (Working on embedding those into articles so that they stay up to date, link back to show their provenance, etc.) I think there’s a lot that can be done to bring journalism into the 21st century. If you’re interested in the things we’re working on, feel free to drop me a line: brian@numer.al.

I would love to hear any feedback you might have! I completely agree with you about fact-checking. One mini-feature that’s implemented right now is the ability to select a range in a graph and then copy/paste the URL, and when someone opens it in a new window they’ll see exactly the highlight you made with the same % change displayed. It makes it really easy to back up particular assertions across a time series.

And yes, I was having some issues with the DNS because of the way Heroku interacts with root domains, but that should be sorted as of today and the apex numer.al domain should work just as well as www. Thanks for the heads up!

If you’re trying to make a point about a change over a specific period, you can select a range and the selection gets appended by JS to the URL, such that if you send someone else the link they’ll see exactly the selection you made. It’s also possible to follow figures and be updated when they change. But also interesting things become possible across datasets. For instance, it becomes possible to mash up absolute job numbers from the BLS with population estimates from the Census and get accurate per-capita figures, that stay current when either the BLS or the Census publish new numbers.

Agreed that it'd be interesting to try to figure out what explains those cycles and steps. Recessions in general? Would be interesting to see what correlated with those points in time.

Here, for instance, is a graph of manufacturing jobs per capita in the US:

https://www.numer.al/us_bls_data/figures/manufacturing-emplo...

It’s quite clear that manufacturing jobs have just been slowly and steadily disappearing over the past 70+ years.

Viewed at this scale, the debate over the bump in the last 6 years seems totally silly. Moreover it’s hard to imagine a policy that would turn that kind of trend around.

Oh I stand corrected that the BLS doesn’t break that out!

Here’s a comparison of men (in black) vs. women (in blue). Very interesting.

https://www.numer.al/us_bls_data/figures/labor-force-partici...

For me prose is a really hard medium for parsing trends like the ones Will is describing here, so I made a scraper that turns the Bureau of Labor Statistics API into live D3 charts that refresh themselves every day and stay current. I’ve found it really useful.

For instance, here’s manufacturing jobs per capita -- you can see that the bump during the Obama administration, while real, is truly nothing to get excited about.

https://www.numer.al/us_bls_data/figures/manufacturing-emplo...

Conversely it becomes clear that manufacturing jobs have been pretty much smoothly and steadily disappearing since World War II, and it seems (to me) highly unlikely that any kind of immediate policy solution is going to reverse that.

And here’s the civilian labor force participation rate that Will is talking about in the article:

https://www.numer.al/us_bls_data/figures/civilian-labor-forc...

You can see clearly how much lower the value is now than it was at the turn of the millennium -- this is Will’s point at the start of the piece -- but you can also see that we’re still above the entire 40s, 50s, 60s, and most of the 70s.

Although the difference there is that the people not participating in the labor force were, I’d have to guess, women. Interesting that Will focuses specifically on men. It doesn’t look like the BLS breaks labor participation out by gender, which would be interesting.

If there are other BLS stats of interest to HN readers, I haven’t finished making a way for other users to add their own scrapers, but I’m more than happy to add figures for other BLS data. Just let me know.

Thank you! Yes, I’ve been working on it as a way both to visualize and to stay current with some of the things I’ve been seeing in the news that are of interest to me, related to the environment, the energy sector, economy, etc. With the election coming up, I’ve been looking a lot at the types of indicators being mentioned in debates (crime, deficit, etc.).

At the moment you can “follow” figures and view a feed of the ones you follow in reverse-chronological order by the time of the latest update (I’m working on an “email me when this changes” option too), and you can drag the thumbnail graphs over one another and drop them to see two numbers compared. The comparison view also gives the option to make your own charts derived from existing ones: e.g., you can make a “per capita” chart by dragging anything over the Census figures and clicking “Proportion.” The “downstream” chart will automatically update when its “upstream” charts do.

It’s very much a work in progress, and if it proves useful, I’d love to hear more from you about the kinds of things that would be of interest to add to it!

@mastazi Thanks for the heads up! I was testing in Chrome and Safari and missed that one completely -- apparently they allow for a circle radius ("r") to be defined in a stylesheet, but Firefox doesn’t. It’s cross-compatible now -- cheers!

I wanted to make more sense of some of these CO2 headlines, so I put together a scraper that generates live D3 charts of the most recent CO2 measurements from the Scripps Institution of Oceanography and keeps them up to date.

Here’s weekly: https://www.numer.al/scripps_data/figures/atmospheric-c02-co...

Monthly: https://www.numer.al/scripps_data/figures/atmospheric-c02-co...

And yearly (avg): https://www.numer.al/scripps_data/figures/atmospheric-co2-co...

And the really scary one, similar to the recent XKCD, is the ice core data from the NOAA: https://www.numer.al/noaa_data/figures/antarctic-ice-cores-r...

(Fun fact, Postgres date fields underflow after 4713 BC, but the data actually goes back another 700,000 years!)

Even with tools to automate the search through listings and get you ahead of other buyers, one of the tricky parts of apartment hunting is its “optimal stopping” structure: you have to decide when to commit without knowing whether an even better apartment might be out there.

This is a famous math problem (the “secretary problem”) with the lovely solution that you should define a time interval over which to search and not commit to anything until you’ve spent 1/e of that time -- 37% -- noncommittally exploring your options.

https://medium.com/galleys/optimal-stopping-45c54da6d8d0#.lj...

I'm aware of portions of that literature, but haven't studied it explicitly. What do you think are the most interesting things happening in that space at the moment?

For me one of the most salient things in that space -- albeit a different point than the one raised in the essay -- is the debate about what sorts of metrics are most appropriate (if any) for informing decisions about hiring, tenure, and the like. In some ways it's quite like the be-careful-what-you-wish-for problems any business has in choosing which metrics to care about. For instance, the h-index might shape a researcher's choice of whether to publish something as one larger paper or two smaller ones.

As Sam Altman put it, "It really is true that the company will build whatever the CEO decides to measure." I think a similar principle holds for research and scholarship.

Author here. Delighted to see this near the top of HN this afternoon, and to see the healthy discussion it's provoked. Happy to answer any questions as best I can if you've got 'em.