HNHacker News
TopNewBestAskShowJobs

andrenotgiant

2,185 karma · joined September 27, 2010

submissionscomments
andrenotgiant··on Postgres: The next generation
Ahh thanks for clarifying - now I better understand the significance of the OP's point about the rarity of younger COMMITTER's.
andrenotgiant··on Postgres: The next generation
I have the honor of working with a Postgres ~committer~ contributor who was just over 25 when they first contributed! The story about their first commit is great:

They were testing SQL behavior for Materialize and thought to check that both systems handle interval functions identically. Being thorough, they tried something like:

  select interval '0.5 months 2147483647 days';
You can try it yourself on dbfiddle[0] Instead of erroring, Postgres returned a bogus value `{"days":-2147483634}` you can read why here[1]

So naturally they decided to fix it in Postgres, which is why they contributed and why it's handled properly in 15+ [2]

[0] https://www.db-fiddle.com/f/ijT76fsmL99bHvXxhAtf7j/0 [1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit... [2] https://www.db-fiddle.com/f/i3KikCb72AN1EZpywErZvr/1

andrenotgiant··on Fixing Penn Station
I commute through Penn Station three days a week, I can't understate how terrible it is for commuters.

Every morning when I exit the train to a blast of heat on cramped platforms, wait in line for my turn to weave up a tiny staircase, into the aging waiting halls with hoses routing ceiling leaks into trash cans, zig zag through a labyrinth of low-ceiling corridors, giving a wide berth to half-naked mentally ill people, to a side exit (since the main exit has been closed for months, and finally, as icing on the cake, I jump over a permanent puddle created by construction barriers RIGHT at the crosswalk where THOUSANDS of people exit the busiest train station in the entire country, I think about the line:

“One entered the city like a god. One scuttles in now like a rat.”

andrenotgiant··on How do domain-specific chatbots work? A retrieval augmented generation overview
Is there something (besides the convenience of being already in the proper vector formats) unique about the combination of semantic search and LLM's that makes it a better fit?

I guess I'm surprised that the approach is to use semantic search here, where in regular search, completely different algorithms (Lucene?) have "won".

andrenotgiant··on How do domain-specific chatbots work? A retrieval augmented generation overview
Great explanation. I understand the process a lot better after reading.

I have questions now: It seems like everything hinges on the search step, where domain-specific content is gathered to serve as prompt input for the LLM.

1. In an area like technical docs, if I can't find what I need it's often because I don't know the right terms - e.g.: I am looking for "inline subquery" but they're really called "lateral joins" - Would the search step have any likelihood of finding the right context to feed the LLM here?

2. How much value is added by feeding the search results through the LLM with the user's prompt, vs just returning the results directly to the user?

3. Are there good techniques being developed for handling citations in the LLM output? IIRC Google had this in Bard

andrenotgiant··on Uses and abuses of cloud data warehouses
The article is separating "operational" and "analytical" use-cases.

IIUC analytical = "what question are you trying to answer" and in analytics, RT/NRT is absolutely novelty/vanity. Operational = "what action are you trying to take" and it makes sense to want to have up-to-date data when, for example, running ML models, triggering notifications, etc...

andrenotgiant··on Uses and abuses of cloud data warehouses
It seems like Snowflake is going all-in on building features and doing marketing that encourage their customers to build applications, serving operational workloads, etc... on them. Things like in-product analytics, usage-based billing, personalization, etc...

Anyone here taking them up on it? I'm genuinely curious how it's going.

andrenotgiant··on We reduced the cost of building Mastodon at Twitter-scale by 100x
incremental view maintenance is the database equivalent of: "recompute only the changed data by checking the relationship between fields,"

Oracle has decent support for incrementally updated materialized views, redshift has some too. Materialize.com is an entire snowflake-like platform built around incrementally maintained materialized views.

andrenotgiant··on Cut out everything that's not surprising (2019)
Write for an audience, read the other work that audience reads, that will help you get a sense of whether something is going to be surprising to them.

Here is a longer lecture on a similar idea from U Chicago Writing Program https://www.youtube.com/watch?v=vtIzMaLkCaM&t=3s

andrenotgiant··on Google engineers want to make ad-blocking (near) impossible
This is a dumb article that's trying and failing to tie Attestation to ad blocking.

> However, how this plays out with browsers that allow extensions or are modified remains a grey area. As the proposal vaguely mentions, "Web Environment Integrity attests the legitimacy of the underlying hardware and software stack, it does not restrict the indicated application’s functionality."

That's not vague at all.

andrenotgiant··on A glitch in the SEO matrix
They actually did have it as an experimental feature for a little while, maybe 10 years ago. [0]

If I recall correctly, it was made available in response to another big wave of criticism directed towards Google about "Content Farms". It has been interesting to see the difference in response between the "Content Farm" debacle and what we are dealing with currently.

[0] https://googleblog.blogspot.com/2011/03/hide-sites-to-find-m...

andrenotgiant··on The many faces of DISTINCT in Postgres (2017)
How would you explain them to someone who knows the syntax but still finds them counterintuitive?
andrenotgiant··on We glued together content moderation to stop soccer pirates
Yeah being pedantic here, but reading the details of the $750k reference, it's (hopefully) nowhere near $750k that they saved:

> That, combined with the fact that these streams usually have large viewership, means we incur a not insignificant cost and have no one to bill. In 2021 alone, Mux had over $750,000 in unpaid invoices due to pirated streams. For an infrastructure company like Mux, this pirating comes with hard costs. Transcoding, storing, and delivering video is not cheap. If these pirated streams were not held in check, they could quickly spiral out of control and have a significant negative impact on our business. By identifying and shutting down these streams, we are able to reduce our costs.

So to actually saved $750k, they'd have to have some combination of:

1. 100% COGS or 0% operating margin. Youch. Probably not, likely closer to 30% COGS, making actual infra/bandwidth cost to them of $750k services closer to $225k 2. Massive growth in attempted abuse. E.g. Their level of attempted abuse grew 3.3x to $2.5M, they stopped all of it, that would have cost them $2.5M x 30% COGS = $750k

But at the end of the day I think they just wrote a clever headline to get upvotes on HN :)

andrenotgiant··on Powering a Serverless Web: Vercel Joins AWS Marketplace
It doesn't really start at $250k. It's just a dumb quirk of the AWS marketplace participation process.

As a vendor, if you want to participate, you have to have a public listing page with a price. But many vendors don't actually want you to initiate the buying process that way. They want you to talk to them first, negotiate a deal/contract, and only then handle the payment through AWS marketplace (presumably with pre-committed spend you already have on AWS) via something called "Private OFfers"

So lots of marketplace pages have an absurdly high public listing price to discourage people from using that route.

andrenotgiant··on NZ’s biggest data breach shows retention is the sleeping giant of data security
I think you're paraphrasing from Maciej Ceglowski amazing "Haunted by Data" talk from seven years ago.

Everyone should watch it https://www.youtube.com/watch?v=GAXLHM-1Psk

andrenotgiant··on Ask HN: What lesser-known accessories do you use with your computer?
I used this one through this winter and it worked fine: https://www.amazon.com/gp/product/B09C5LQ86D/

I'd just look for what people say in the bad reviews. I generally use mine on the lowest temps, 80 or 90. I don't know what kind of maniac would use them at the higher temps. Every time I turn it on I have to press the button to cycle through the temperatures, a mild annoyance but NBD. It automatically turns off after a few hours which is a feature, not a bug. It used to be that I'd sit down at my desk and my hands would never warm up, now they warm up from the mat and stay warm even after the thing turns off.

andrenotgiant··on Incompetent but Nice
not OP. But in my experience there's a big gray area between "Brilliant/Nice" and "Brilliant/Asshole" quadrant where people who are very direct, have less patience for incompetence, or hold their coworkers to high standards go.

IMO these are the best people because they intervene before a company or team veers too far off course.

andrenotgiant··on Dragonfly Is Production Ready (and we raised $21M)
I think maybe I'm just small-potatoes, but the only limitations or constraints I've ever run into with Redis are: 1. memory utilization 2. deployment/orchestration 3. bugs I created for myself related to using caches

What are the use cases that max out Redis speed/throughput?

andrenotgiant··on America’s bad bet on expanding legal sports gambling
I know nothing about FanDuel and DraftKings, or sports betting in general. Do FanDuel and DraftKings only make money on fees, or are there also situations where , like in a Casino, you're playing against the house?

Also, I bet there are SWE's from FanDuel and DraftKings on HN. No judgement I would love to hear how you see things from the inside.

andrenotgiant··on Home food delivery’s questionable value, for consumers and investors
not trying to be snarky, genuinely curious: What benefit does that have over ordering delivery from a restaurant?
andrenotgiant··on Stochastic gradient descent written in SQL
This is really interesting, but a basic part I don't understand: What would it actually look like to run this on a live dataset?

If I understand correctly: you'd run the recursive query, it produces results for every step, effectively showing you the progression of output over time, and then once it hits "present day", it completes and stops?

How would you generate results going forward? I.E. A minute elapses after the results return, do you have to re-run the whole query for all time?

andrenotgiant··on Developer marketing for early-stage startups – what we’ve learned
There's a variation of that in the data space, where early-stage data companies show the logos of _other_ data companies.

It's so obviously just that the other company was like checking to see how your product and there's integrates. They weren't actually using it. But now they're in the "Trusted by" panel on the homepage.

andrenotgiant··on Show HN: Usage 2.0 – Cut AWS Spend by 57% in 5 Minutes
just from reading the site, looks like vantage takes a 5% fee vs Usage 20% - but that's a big difference, not sure if it's an apples-to-apples comparison.
andrenotgiant··on Show HN: Usage 2.0 – Cut AWS Spend by 57% in 5 Minutes
How would you describe the mechanics of this in regular finance terms... Almost like you are providing mortgages for reserved instances?

I think I understand the model here, but I guess I'm curious how you model in risk of suddenly getting stuck with a bunch of 3 year RI commitments?

andrenotgiant··on Minimum Viable Hugo
This is great, but for different reasons than the README implies.

> "Hugo sounds great! Let me try it out!" > "Okay, I made a new theme and a page in content/!!" > "Running hugo server -D now!!! ... Where is it!!!!"

It's not Hugo itself that is complicated, it's all the other JS and CSS dependencies that other getting started repos tend to layer in.

andrenotgiant··on Replacing a SQL analyst with 26 recursive GPT prompts
I had not heard that Churchill quote, but now I feel vindicated that I take the same approach as him!

I think this can also be extended to data tools. I've had so many experiences where a SaaS tool that tries to automate away the manual SQL analysis shows something totally wrong. I tend to "Not trust, verify" anything coming from a SaaS UI when serious business decisions are about to be made based on it.

andrenotgiant··on Show HN: Use Slack Emoji on GitHub
No, Github proxies and caches images through an intermediary (probably for privacy reasons amongst other things.)
andrenotgiant··on Differential Datalog: a programming language for incremental computation
Even more context:

SQL interface on Differential/Timely Dataflow = https://materialize.com/

Python interface on Differential/Timely Dataflow = https://bytewax.io/

Grossly oversimplified, but directionally correct.

andrenotgiant··on The next generation of Materialize
Thanks! To answer your question:

The core components of Materialize have always been licensed under the BSL [0], with a conversion to the Apache v2.0 License four years from the date of publication. These components (now called the storage, compute, and adapter layers) continue to be developed in the open under the same license, but are no longer packaged or supported for on-premise deployment.

Timely Dataflow and Differential Dataflow [1] are the open source frameworks that have always been at the heart of the compute layer in Materialize.

If you're referencing the downloadable binary when you mentioned "the open-source version", we decided the opportunity cost of splitting our attention and continuing to release updated binary versions by "rebundling" Materialize is too high. We have a small team and ambitious plans for Materialize, (maybe some `WITH RECURSIVE`?) so we don't plan to release updates to the binary.

[0] https://github.com/MaterializeInc/materialize/blob/main/LICE... [1] https://timelydataflow.github.io/differential-dataflow/intro...

andrenotgiant··on Sudden disturbing moves for IT in large companies, mandated by CEOs
Fantasy fiction of a sysadmin that uses anonymity on the internet to craft themselves into someone more important than they are in real life.
← PreviousPage 3 of 13Next →