U.S. Daily Temperature Anomalies 1964-2013

Thanks for the excellent comment.

Your first point is very well taken - we also urge caution in drawing too many conclusions regarding an overall trend in mean temperatures. However, I think our conclusion - that we are having more hot outlier days now than ever before - is interesting in and of itself.

Your second point is an interesting one, and I too would be curious to look further into the correlation between proximity to a city and temperature. However, to look for outliers, we created a long-running distribution of seasonal temperatures for each station individually - so in some sense the map is already corrected for this effect. Each anomaly you see is an anomaly for that station alone - meaning if an urban station regularly gets higher temperatures than a rural one, it will take a proportionally higher temperature to trigger an anomaly on the former than on the latter.

Furthermore, NOAA has been good about getting good national distribution of these stations so it's less of a concern than you might think.

That said, urban stations may still show artifacts compared to rural ones, eg. when there is an extreme warm outlier, cities may be more likely to have another warm outlier the following day due to the heat storage effect you mention. I'm not sure.

> A trend in the outliers of a distribution does not imply the same trend in the mean values of the distribution

It does if the distribution is normal and the trend in the outliers is asymmetrical (as it is in this case). Because temperature distributions are (almost certainly) the result of many additive factors, they are normal by the mean value theorem.

The thing about weather temperature outliers, however, is that they're highly relevant to the effects resulting from them: an hard freeze at an unexpected time, or a colder seasonal low, can have huge impacts on agriculture, infrastructure (roads, pipes, overhead wires, and structures can be damaged by freezing or ice storms), similarly heat can result in ruined crops, heat deaths, drought, and fires. Diseases, pests, and parasites can also be strongly influenced by weather.

And in terms of the energy surplus (or deficit) represented, these are huge changes being represented. Takes a lot of energy to heat (or cool) the world.

> Intuitively, it seems like the reporting weather stations would not be uniformly distributed geographically, but would rather show a higher concentration near urban areas

Why would this be intuitive at all?

First, if you play the animation and look at the dots on the map and have a basic understanding of how the US population is distributed, I just can't see how you would draw your conclusion. The dots are pretty evenly distributed geographically.

But even without the visualization, why would you assume there were more reporting stations in urban areas? Why would people be inclined to waste time and money supporting a new reporting station if there is already one close by?

Finally, the "urban heat islands are skewing the data and climate change is not real" argument has been debunked so thoroughly that it is hard to believe anyone who brings it up brings it up in good faith. Especially when it is phrased as an "innocuous question" which could have been answered with 30 seconds and your favorite search engine.

Yeah, that is an interesting point and highlights the difference between "weather" and "climate" - it also brings up a tension between two different objectives of the chart: to visualize weather patterns over time intuitively, and to draw general conclusions about climate trends. In the context of the first intention, climatic oscillations like El Niño are interesting signals - you can see how they affect weather throughout the country in unexpected ways. But in the context of the latter goal, they are noise which should be filtered out/corrected for.

You aren't blind :) We just had trouble fitting a y-scale on the top chart that was both clean and readable and didn't get in the way of the time slider... However this is just supposed to be an overview of the trend - the proportional bar chart further down the page is the same data, plotted in more depth and explained further.

Click and drag to pan the map - Alaska, Hawaii and Puerto Rico all have data :)

Hail is not the same as snow. When I lived in the mid-atlantic region, we got hail almost exclusively in the summer, as it was typically caused by thunderstorms.

Little known fact: Hawaii's record high temperature (100F) is the same as Alaska's record high temperature (100F), and both are the lowest record high temperatures for the 50 states.

Alaska's record low (-80F) is a little bit colder than Hawaii's record low (15F) though.

Yes, this would be a good improvement to the data pipeline. I only built the front-end so I can't speak to exactly what difficulties would be involved, but I agree.

Yes, that's a good point. It would be better to create one distribution for each day of the year, with the values drawn from the 2-3 weeks before / after that day, but we chose a month to minimize the computational complexity. Presumably these artifacts wouldn't affect the aggregate trend over time, only the appearance of more outliers at the beginning of March / October, etc. each year. We also were drawing inspiration from NOAA's Climate Extremes Index which uses monthly data: https://www.ncdc.noaa.gov/extremes/cei/

Thanks for the feedback! Just pushed an update with a max-width.

Hmm, that's odd - it was developed in Chrome and I haven't seen any Chrome-related issues after testing on several computers. Could be due to a badly-behaving browser extension? Let me know if you see any JS errors on the console, or if you figure out what was causing this...

You should've seen it before we found all the cheap wins :) Seriously though, I'd be interested to hear any ideas you have - it has already been optimized quite a bit but it's just a lot of data (3 million rows over 50 years). It uses canvas and throws away all the circle references every day so that helps a lot.

Theoretically no, the database we used contains weather station data from all over the world. However during our analysis we found that the quality of data from international stations was a lot more varied - as measured by many factors such as geographical distribution of stations, regularity of measurements and number of implausible outliers. I would still like to come back to this eventually & try to iron out as many of those issues as possible - but for the scope of this project it was much easier to just use US data as it is extremely well-curated.

It's "dataviz", bro. Not data science.

From the page:

"Armed with this refined dataset, we computed the historical range of low and high temperatures for each station for each month of the year. We then compared each station's daily temperatures to its corresponding monthly distribution. If one or both of these measurements fell in the bottom or top 2% on a given day, we labeled it an "anomaly" according to the typology above."

AFAIK this is based on raw, original weather station data.

Joe D'Aleo, the first director of meteorology at the Weather Channel who now works with WeatherBELL, is explaining to anyone who will listen that NOAA did exactly that.

http://icecap.us/images/uploads/NOAAroleinclimategate.pdf

Closing line:

"The NOAA, NASA and the Hadley Center press releases should be ignored. The reason which is expanded on with case studies in the full report is that the surface based data sets have become seriously flawed and can no longer be trusted for climate trend or model forecast assessment in decision making by congress or the EPA."