HNHacker News
TopNewBestAskShowJobs

mcdonje

1,757 karma · joined August 10, 2017

submissionscomments
mcdonje··on Scientific datasets are riddled with copy-paste errors
It does have types if you define them in the schema. Not every format needs to be self-describing. It's often more efficient to share the schema once outside of the data feed than have the overhead of restating it for every data point.

It's completely Excel's fault for pushing their type-inference and making it difficult for users to define or supply their own.

Power Query does a better job handling it, but you should be able to just supply a schema on import, like you can with Polars or DuckDb.

It's another example of MS babying their userbase too much. Like how VBA is single threaded only because threads are hard. They're making their product less usable and making it harder for their users to learn how stuff works.

mcdonje··on Scientific datasets are riddled with copy-paste errors
You're just saying when there's no filetype transfer, you don't have to deal with issues related to filetype transfer.
mcdonje··on Scientific datasets are riddled with copy-paste errors
No, it's Excel trying to be too clever. It does the same thing with manual imput if you don't proactively change the field type.

You can import a DSV into Excel without mangling datatypes in a few different ways. Probably the best way is using Power Query.

A DSV generally does have a schema. It's just not in the file format itself. Just because it isn't self-describing doesn't mean it isn't described. It just means the schema is communicated outside of the data interchange.

mcdonje··on Scientific datasets are riddled with copy-paste errors
That's Excel's type inference causing problems. Not an issue with CSV or any other type of DSV.

It is possible to import a CSV into Excel without type conversion. I just tested it two different ways.

While possible, it's not Excel's default way of doing things. Not always obvious or easy. Not enough people who use Excel really know how to use it.

Regardless, Excel mangling files via type inference is an Excel problem. It's not the fault of the file formats Excel reads in.

mcdonje··on Scientific datasets are riddled with copy-paste errors
You're blaming a lot of normal ETL problems on DSVs.

Like, specifying date as a type for a field in JSON isn't going to ensure that people format it correctly and uniformly. You still have parsing issues, except now you're duplicating the ignored schema for every data point. The benefit you get for all of that overhead is more useful for network issues than ensuring a file is well formed before sending it. The people who send garbage will be more likely to send garbage when the format isn't tabular.

There are types and there is a spec WHEN YOU DEFINE IT.

You define a spec. You deal with garbage that doesn't match the spec. You adjust your tools if the garbage-sending account is big. You warn or fire them if they're small. You shit-talk the garbage senders after hours to blow off steam. That's what ETL is.

DSVs aren't the problem. Or maybe they are for you because you're unable to address problems in your process, so you need a heavy unreadable format that enforces things that could be handled elsewhere.

mcdonje··on Coq theorem prover is now called Rocq
While some may lament the departure of the phallus bird, nobody can be sad about the arrival of the giant fierce mythical bird.
mcdonje··on Ada, its design, and the language that built the languages
I imagine an ada dev would find the pattern grating over the decades, so it reads like an expression of that experience.
mcdonje··on 5NF and Database Design
Normalize the app, denormalize the reporting.
mcdonje··on Two Months After I Gave an AI $100 and No Instructions
>I don't know what that proves.

It proves something, but not much. Those models with those inputs (mostly HN articles) were benign or even a net positive for society.

Other models with different training (upstream of the blogging user), or with different inputs (maybe it finds a different article posted to HN or another site that proves foundational to its evolving perspective), could end up behaving differently.

mcdonje··on Good CTE, Bad CTE
While generally a fair critique, the site does have "SQL" in its name.
mcdonje··on Good CTE, Bad CTE
There's some good stuff in here. I didn't know about the issues an aggregation in a CTE can cause and haven't used EXISTS much.

Regarding recursive CTEs, you might be interested in how DuckDb evolved them with USING KEY: https://duckdb.org/2025/05/23/using-key

mcdonje··on Ask HN: How best to plan a 5K on a map for training?
That's a pretty good training plan.

There's been some research about sprint interval training (SIT), which seems to be popular these days, but low heart rate training (LHRT) is also popular.

Seems like your training was closer to LHRT, but after pushing yourself upfront, not as a sprint, but still initially unsustainably fast.

I'm a little over 40, male, out of shape, and I've been just running at 5mph. Much slower than your 7.5mph, but I can keep it up for 5k. (Side note: It's funny and annoying that treadmills in the US are almost entirely imperial even though most popular races are metric.) My plan was to slowly increase my speed, but now I'm considering your plan. It makes sense. It's easy to manage. It gets good results.

mcdonje··on Ask HN: How best to plan a 5K on a map for training?
Thanks for the suggestion. I just fiddled around on that site a bit and it's good!

They're not walling off features or asking for money, so I'm not sure if it's owned by another company or a FOSS project or something else. I did a quick search but it didn't turn up anything about their business structure.

mcdonje··on Ask HN: How best to plan a 5K on a map for training?
Yes, thanks for adding that. Also the titular distance of the competition: 5 Kilometers.
mcdonje··on Astral to Join OpenAI
Seems like you're responding to the wrong person. The person I replied to said the open source model doesn't work. Nobody said the thing in your second quote.

I get the point you're making, but the way you introduced it isn't conducive to productive conversation.

mcdonje··on Astral to Join OpenAI
Google contributed tons of developer hours for things like bug fixes, without which the project might not be where it is today.

There are examples of foundations or other similar entities paying developers, like Linux, SQLite, even Zig.

Maybe the difference is some projects rely on core contributors more because external contributions are more restricted in some way.

But sure, the entire open source model doesn't work, lol

mcdonje··on Astral to Join OpenAI
Are you not aware of foundations?
mcdonje··on Afroman found not liable in defamation case
It works here because it isn't a pleasant song about Ohio.
mcdonje··on Afroman found not liable in defamation case
The ACLU called it a SLAPP lawsuit. If true, they probably didn't care if they won or not.

That said, going on stand when your opponent has proven they can and will use your words and actions against you in the court of public opinion is a... bold strategy.

mcdonje··on Rob Pike's 5 Rules of Programming
Very performance focused. Could more accurately be 5 rules of perf. Good list, though.
mcdonje··on Beyond has dropped “meat” from its name and expanded its high-protein drink line
I think that's rather uncommon. The closest I've seen is someone smashing down some ground beef then putting salt and pepper on it before cooking.
mcdonje··on Beyond has dropped “meat” from its name and expanded its high-protein drink line
You're comparing a burger patty to a burger ingredient. Two different things. Not a reasonable comparison.
mcdonje··on Rack-mount hydroponics
Tell me more about super spinach. B12 doesn't come from plants or animals, but from bacteria. So, I don't know how you could get B12 into spinach by using red lights. You'd also need to introduce the bacteria and somehow make it live inside of the spinach.

Do you have some sort of inoculation step and then use red light to penetrate the spinach leaves to feed light energy to the bacteria?

mcdonje··on Global warming has accelerated significantly
The graph that shows the US as the #3 CO2 generator after China (#1), and all of Asia except China & India (#2)? The one that shows the US generating about half as much as China despite having less than a third of the population? The one that shows the US's contribution to be more than the continents of Africa, South America, and Oceania combined?

Struggling to see how one could look at such a chart and think it's futile for the US to take meaningful action.

mcdonje··on Bluesky CEO Jay Graber is stepping down
Seriously? If a company is publicly traded, they're legally required to prioritize shareholder value, unless they're a benefit corp or something with multiple bottom lines. I suppose you could call it values-driven to drive up the bottom line, but that's not normally what people mean.
mcdonje··on Motorola announces a partnership with GrapheneOS
Synecdoche. The EU governmental body is acting like it wants it.
mcdonje··on Motorola announces a partnership with GrapheneOS
If they can't get it passed because the people don't want it, then why do they keep trying to pass it? Some entities with a lot of power or influence clearly want it. This is the same thing we see in the US. We keep saying "no", and they keep trying.

Maybe the EU people don't want it, but at least some governing body of the EU clearly does.

There's a comment not too far up in this thread saying this is more of a US thing than an EU thing, but it looks like exactly the same pattern from where I'm sitting in the US.

mcdonje··on The Physics and Economics of Moving 44 Tonnes at 56mph
Long haul trucking should be illegal or effectively eliminated by a carbon tax. Use trains for long hauls.

He even said the problem with trains is last mile. Last mile short haul trucks can be and are electric.

It's weird he laid the groundwork for this argument, but he isn't making this argument.

Fix the infrastructure issues that make transferring from rail to truck difficult. Yes, that's challenging and expensive. Guess what else is? The status quo, and the effects of climate change.

mcdonje··on Antirender: remove the glossy shine on architectural renderings
Yes
mcdonje··on I was right about ATProto key management
>you get to use a slightly worse form of Twitter

The protocol can support all sorts of other social networks. People are building things akin to instagram, tiktok, medium, allrecipies, etc

← PreviousPage 3 of 24Next →