this deserves its own post, but the other advantage working with tech datasets on the regular is developing perspective on the methodological shortcomings of what's done at universities.

what i mean is that once i started running product experiments at scale in really hyperoptimized environments, my sense of what's realistically possible with small data sets collected by humans describing meatspace circumstances contracted fairly dramatically. I don't just mean replication crisis issues, although those are real; im instead skeptical of nearly all of the empirical work being done in nearly every social science field.

that is, im not arguing that tech does a great job facilitating research into eg maternity leave policies. rather, im saying i dont think anyone is learning much about them at all.

im happy to acknowledge that this is a minority view.