teej

This seems neat!

Last year I helped a friend diagnose an issue they had with Postgres. A database table with a scheduled full DELETE/INSERT had slowed to the point of failure. It turns out, having slightly less IO than needed led the auto-VACUUM process to get further and further behind each time it ran.

My friend simply provisioned more IO and moved on. Another option would be to rewrite the process to naturally produce fewer dead rows. It would be great to have a third feasible option.

> “As of 2013, NIST-F1 has an uncertainty of ..., which means that it would take 100 years for the clock to be off by a second”

This didn’t sound right to me. The clock being off 100ms every 10 years sounds terrible. This must be a typo because the source says:

“... which means it would neither gain nor lose a second in more than 100 million years!”

The end state is you DIY. Segment isn’t doing anything secret/magic, it’s just well executed and super convenient.

The “how” question has changed every 3 years in the last 10 that I’ve done data work. That trend is driven primarily by data scale and partially that analytics and ML access patterns have evolved a lot.

Segment (and any other SaaS that wants to own the event firehose) has pricing issues as the customer scales. As a customer you are charged on data volume[0], but your data grows at the rate = (Users x Engagement X Features X Data Sources). That rate can easily be the square of your revenue growth.

This pricing issue makes it easy for Segment's fast-growing customers to justify an in house replacement. This puts huge down-pressure on their "revenue retention" rate and makes it hard to raise more VC or go public.

[0] - Segment technically prices on user volume but in reality it's a mix of users + objects depending on what your sources are

Don’t need to draw the line. For better or worse, California has already passed prop 65. This is just suing for the law to be enforced.

DRY is about code you write, not the code artifacts you ship to the client.

You’ve got it right. This “seed” is a seed in the sense of a random number generator. It only controls generating the world, however, it does not control the behavior of NPCs.

Here’s chess in Redshift http://www.michaelburge.us/2017/09/10/injecting-shellcode-to...

I’ve heard this as a rumor, yes.

Citation(s) needed

> my work was "utterly and completely useless

I can never understand the arrogance of folks who would say something like this

No amount of technology can fix a dysfunctional org.

A large amount of the work of BI is getting data into the right shape in order to ask questions about it.

- What’s level on average do new players drop off?

- How are sales of the new skins we introduced performing vs the last set?

- How often are players engaging with feature A vs B?

You might be able to answer all of these questions with just event data in a wide table, but you’ll write very different queries to answer them. dbt is a way to write queries on your source data that leads to systematic, repeatable, and reusable new tables for answering analytics questions.

A few comments based on lots and lots of experience:

- Wide tables in columnar DBs can make some analytics queries easier to write and sometimes more performant.

- Wide tables can come at high storage cost and make other queries less performant (like SELECT *)

- How much of these things happen is extremely DB dependent. How does the DB's underlying storage mechanism work? How is the data partitioned and distributed? How sophisticated and storage-aware is the query planner? How large is your data? How fast is your data growing? How fast do you need a new data point to be reflected in your dashboard?

There's no free lunch when it comes to this stuff. A perfect solution doesn't exist, but the technology is getting better all the time. I've personally never used sql server as a data warehouse but plenty of folks do.

The stuff I use that I recommend very highly - Snowflake, TimescaleDB, vanilla Postgres. Also, use dbt.

Here’s some photos of my personal deck, which I have owned for 10 years. https://imgur.com/a/vTNsN5b

My oblique strategies deck is my most prized desk doodad. I don’t reach for it often but I never regret when I do.

C code is being replaced by Rust fast. The only limit is how quickly programmers can become good at Rust. It's already happening.

> We AB tested this

JTNimoy is a creative technologist who has been on the HN homepage multiple times, especially for their work on Tron Legacy http://www.jtnimoy.net/item.php?handle=14881671-tron-legacy

The counter argument is that the Academy reports to no one but themselves and can set whatever standard they want for their own award show.

I understand a lot better now, thanks!

> data to be stored within the CPU

I don’t know anything about CPU design. Am I wrong to read this line and be worried about memory access vulnerabilities like have been found with Intel CPUs?

I've lost a lot of respect for you after this comment.

That question is psychotic.

No.

If I could add 10 years on to my 30s I would. But adding 10 years into my 80s doesn’t seem worthwhile.

Seems very strange to leave out any discussion of auction models for pricing, like Google and Spotify used.

A second opinion from whom? Another doctor.

Your average joe doesn’t get to just decide what treatment they get. Doctors do, and those doctors assume liability for their decision.

FDA decisions like this are for healthcare providers and hospital systems.

I didn’t claim that the way brains work didn’t include statistical inference. That’s why the dog metaphor works. Both the dog and the machine learning model use statistical inference to perform tricks.

Dogs haven’t been to the moon, however. There’s more to the brain we don’t understand.

Yes, it’s widely believed that statistical inference is a part of how the brain operates. But we have barely scratched the surface in our understanding how the human brain works.

Do you honestly believe statistical inference completely explains a human’s ability to learn?

That’s the thing. Machine Learning is a misnomer, the models don’t “learn” anything about the domain they operate in. It’s just statistical inference.

A dog can learn to turn left or to turn right for treats. But they don’t understand the concept of “direction”, their brain isn’t wired that way.

Machine learning models perform tricks for treats. The tricks they do get more impressive by the day. But don’t be deceived, they aren’t wired to gain knowledge.