HNHacker News
TopNewBestAskShowJobs

thomasmg

654 karma · joined June 19, 2018

Original author of Hypersonic SQL and H2 database (Java relational databases): h2database.com
submissionscomments
thomasmg··on PRQL: a simple, powerful, pipelined SQL replacement
> part of its purpose is to be back-end agnostic

Low-level languages can be backend agnostic. For example C or Rust allows to write low-level code that is back-end agnostic, and doesn't cause cause vendor or technology lock-in.

What I like to have is guarantees similar to what a RTOS (real-time operating system) has. Maybe using hints, or something like that. This could be standardized. Languages like C or Rust have such guarantees naturally: statements have some kind of guaranteed time constraints (that's why RTOSes can be mostly written in C). Sure, loops can be endless, but loops, e.g. in Rust, can't become 1 million times slower suddenly because the backend changed.

thomasmg··on PRQL: a simple, powerful, pipelined SQL replacement
Yes, it's also used to hide implementation complexity and diversity: the advantage of a standard (even a "weak" one).

> SQL is used to reduce network roundtrips.

What I mean is: the database has an API that allows the client to specify (almost) arbitrary complex queries and operations. Regular REST APIs don't support this. GraphQL also supports it. The client sending queries to the database is a great way to reduce network roundtrips. Key-value stores that only support get/put/remove/list require a lot more network roundtrips for slightly complicated operations.

> it makes sense to try a better language.

I argue that the language itself (the syntax) isn't the main problem: part of the problem might be that SQL (even with a better syntax) would _still_ have very similar problems: missing indexes / wrong query plans, missing encapsulation which results in a complex mess of entangled code, things that work fine with small data sets but are extremely slow with large data sets,...

thomasmg··on PRQL: a simple, powerful, pipelined SQL replacement
So that you can generate SQL more easily, and (for hand-written statements) so that you can re-order lines more easily. Many programming languages support trailing commas.
thomasmg··on PRQL: a simple, powerful, pipelined SQL replacement
> How many of those are "innovative"?

In programming languages we have imperative and functional languages, type-safe / untyped, memory managed / unmanaged, low-level vs high level, interpreted / JIT / ahead-of-time. So much innovation! And SQL is almost 50 years old, and basically still the same (plus adding features). And now maybe GraphQL. Very little innovation.

thomasmg··on PRQL: a simple, powerful, pipelined SQL replacement
> regular programming languages are “lower level” in that regard and require you to do that optimisation yourself. The point of a language like SQL is that you describe the shape and content of what you want and don't have to worry about the most efficient way to retrieve that information.

Yes! But I wonder: wouldn't it be better if the developer has to specify how he wants to retrieve the information? In a regular programming language, typically people don't complain too much about this "missing feature" of a query optimizer. It's about modularization / abstraction. In SQL, the data model is king; in a regular programming language, the API is king. Yes, compilers have optimizers, but they optimize the machine instructions, not (so much) the algorithms that are used (scan vs using an index / using which index).

Maybe I just have seen too many cases where developers didn't create the necessary indexes, or databases did't use the (obviously) best indexes, or developers writing 10'000 lines SQL statements (that I then re-wrote in Visual Basic, and the resulting program was much faster). Or 1000 line GraphQL query that returns 50 MB of JSON. Yes, I have also seen Java developers writing exponential algorithms or worse, and running into memory issues - but I personally found it easier to analyze and optimize then SQL. Maybe I have some bias thought.

thomasmg··on PRQL: a simple, powerful, pipelined SQL replacement
Many databases support other languages as well (eg. PostgreSQL supports many including Python, by default). One challenge is lack of standardization. (SQL is a weak standard, but at least a standard).

Weak typing: what about TypeScript?

Slow loops: yes, this is a problem. However, SQL (and even more so, GraphQL) also has a problem of large results / operations spanning too many entries. During development, the number of entries is fine, but not in production. Specially if indexes are missing, this is also a problem in SQL. (Endless loops are actually less of a problem than slow loops: it's easier to detect endless loops during development).

To process large results in a good way, often, pagination is needed; best would be keyset pagination. What if a statement returns a "continuation" statement in addition to the result set? If the client wants to get more results (or process more entries), then he would need to run the "continuation" statement.

Say a database doesn't provide SQL, but instead a set of low-level APIs (plus a ton of tools). Developers would then (be forced to) write properly modularized, versioned code on top of those APIs.

thomasmg··on PRQL: a simple, powerful, pipelined SQL replacement
> prefer writing plain SQL to using ORMs

Oh, ORMs are not the solution. I agree, ORMs often bring new issues (like, loading too many entries in memory), and don't solve many problems (encapsulation).

> I don't ever want to have a lot of SQL logic in my code.

That works if you are a small team. With big teams / multiple teams, the small problems become a huge problem: someone / some team will write huge statements, will use weird syntax, and so on. (C++ has similar problems btw) Specially the lack of proper encapsulation can become a huge problem.

> Some statements do need to be very large, for efficiency.

Well, then the problem is that SQL doesn't give you a good way to avoid large statements.

> there are a million ways it can be abused

Exactly. Other languages (like Java) make it harder to abuse the language. If you really want, you still can; but it's easier to refactor and maintain.

thomasmg··on PRQL: a simple, powerful, pipelined SQL replacement
SQL has many problems: "from" should come first to help auto-complete; null handling is complicated; the syntax in general is weird. Compatibility is a problem (I have implemented some relational databases in Java: Hypersonic SQL, H2; compatibility with other databases is hard).

There is little innovation in the database space: there are hundreds of general-purpose programming languages, but very few "database" languages. I also tried to specify a new language, https://newsql.sourceforge.net/ many years ago. There's GraphQL, but more innovation in this area would be great.

Many developers end up writing huge SQL statements (one statement that is hundreds of lines). You wouldn't do that in a "proper" programming language; but in languages like SQL (or maybe Excel) this will happen.

Another problem is proper encapsulation. Views can be used, but often developers have access to all tables, and if you have many components this becomes a mess. Regular programming languages have module system and good encapsulation. SQL statements often don't use indexes, for one reason or the other. With regular programming languages, it is harder to end up in this position (as you have to define the API properly: encapsulation). Regular programming languages don't require "query optimizers".

SQL is used to reduce network roundtrips. Clients send SQL (or GraphQL) statements to the server, which are actually small programs. Those programs are executed on the server, and the response is sent to the client. (Stored procedures and for GraphQL persisted queries can be used - but that's often happening afterwards.) Possibly it would be better to send small programs written in a "better" language (something like Lua) to the server?

thomasmg··on Java 20: A Sneak Peek on the Panama FFM API
So the computer that simulates the universe is likely a 512-bit machine.
thomasmg··on UK Plans for Three-Hour Power Blackouts in Event of Gas Shortages
Power outages can happen for various reasons (storm, accident,...). I would expect the equipment has batteries, no?
thomasmg··on Making a Website Under 1kB
Or chess (including computer opponent): https://js1k.com/2010-first/demo/750
thomasmg··on Faster offset pagination for Rails apps
Yes, using OFFSET for pagination is often problematic. The best alternative IMHO is "keyset pagination", but it requires a bit of work. See https://use-the-index-luke.com/no-offset
thomasmg··on “After Going Solar, I Felt the Bliss of Sudden Abundance”
The problem isn't the amount of usage of electricity, the problem is CO2.
thomasmg··on “After Going Solar, I Felt the Bliss of Sudden Abundance”
While running, there is no additional "waste" heat: Without solar panels, 100% of the sun rays heats the earth; with solar panels, 20% is converted to electricity, and then converted to heat again.
thomasmg··on How I clean my glasses
I do the same. The problem is at step 5: some faucets (at hotels and so on) are automatic (no touch, due to pandemic maybe). I don't have a solution for this yet.
thomasmg··on Chasing utopian energy: How I wasted 20 years of my life
> All power generation is subsidized

My point is, the combination of wind, solar, storage is orders of magnitude less subsidized. If you account for risks (nuclear) and externalities (fossil fuel). This is not propaganda. But your claims about nuclear are.

thomasmg··on Chasing utopian energy: How I wasted 20 years of my life
Solar + wind are actually fixing problems. They are not perfect, but as you said, perfect is the enemy of improvement. Solar + wind are cheaper than almost anything else, specially nuclear. They have much less detrimental effects to the local environment than fossile fuels (specially coal). Yes, they are not guaranteed, so you need a grid (which you have), and some batteries (for short-term storage; like batteries of electric cars) + pumped storage (for medium term storage) + maybe some other storage like hydrogen / ammonia.

Nuclear is way too expensive, specially if you account for risks of disasters (no insurance company will insure them, so the risk is on the population).

thomasmg··on In defense of flat earthers (2020)
Interesting idea! The sun would have to be about 6'000 km away I think. The full moon / new moon / solar eclipse proves that the moon is much closer than the sun. The moon would have to be dangerously close. Then, we know that the moon causes tides, and we can measure them as well on different cities. Being so close, the sun would have to cause tides as well, unless it is very light. It would have to take some time to calculate all that...
thomasmg··on UUIDs are popular, but bad for performance (2019)
You are right, the internals are important. When I tested this, a long time ago [1] [2], I found that randomly distributed keys on PostgreSQL were indeed faster than e.g. on MySQL. Which surprised me! I still don't quite understand why. But, even with PostgreSQL, sequential (or nearly sequential) are much faster.

[1] https://markmail.org/thread/3jzqjy6cavxcrpbq [2] https://markmail.org/download.xqy?id=3jzqjy6cavxcrpbq&number...

thomasmg··on Omicron at 100% Prevalence, Colorado
(You are right of course. With the data we have now, we are quite certain that Omicron is much milder. Hopefully no lockdown is needed.)

But, personally, I would like to avoid getting Omicron, as long term effects are still unclear: does "long Omicron" exist, does it affect vaccinated, and if yes how common and severe is it. There are some recent news about long Covid: it could be micro-clots.

thomasmg··on The world’s biggest offshore wind farm, Hornsea 2, generates first power
Yes. Worst case, lithium can be extracted from seawater: https://electrek.co/2021/06/04/scientists-have-cost-effectiv... - but it's currently cheaper to get it from somewhere else. And lithium can be recycled. In the near future, I think most cars will use LFP batteries.
thomasmg··on DNA seen through the eyes of a coder
There is an analogy with data compression: if you remove a byte, in for example a zip file, and then try to expand it, it might work and give you a different result (it likely doesn't due to a checksum error). I think something similar was used in the SHA-1 collision attack (shattered.io) to generate different jpeg images that have the same checksum.
thomasmg··on Head of the South African Medical Association says Omicron brings “mild illness”
> not the death rate

I reported the death rate. It didn't increase dramatically yet; only from 12 per day to 30 per day; and that in theory could also be "patient died with Covid" and not "because of Covid". Yes, number of ventilated didn't increase as sharply as would be expected from Delta. So it does look less severe, from the number of ventilated. Oxygenated did increase from around 400 to 800; here as well we are not sure about the cause.

> I think the point here is that hospitalizations in general aren't necessarily increasing.

You are right, the data on this web site doesn't show the total number of hospitalizations (irrespective of Covid).

> there is at least some reason to be hesitantly optimistic.

Yes, I agree.

thomasmg··on Head of the South African Medical Association says Omicron brings “mild illness”
> the website you linked is falling over

I'm sorry about that... It works for me still. You might want to use https://web.archive.org/

> I haven’t seen anything indicating the death rate is increasing

I made a statistic myself, using the data provided there, looking at the "Daily Hospital Surveillance (DATCOV) Report" and then "Died". From the historical data listed there, I got the "Died" number for every second day, and took the average of 7. From December 11 to November 25, for every 2nd day, this is 61, 46, 42, 32, 32, 27, 26, 25, 25. So, 61 is the average of the last 14 days, and so on.

Update: I did some more analysis, and found that the "oxygenated" decreased from Nov 1 to Nov 19, and then increased since then. And "ventilated" decreased since as well, and now increased, but yes, not as high! So this would indicate that in fact, Omicron needs less ventilation. And death rate seems to be a function of ventilated and oxygenated.

> it’s also been reported that 80% of hospitalizations are “incidental”

OK... so yes death could also be incidental, as well as the increase in oxygenated... Yes, it could be. But what would be the explanation for this "incidental" increase? Some other infection? Remember it is summer in South Africa.

I'm not saying I know the answer, but if the data doesn't match what somebody is saying, then you have to ask yourself what is the explanation.

> Test positivity isn’t a measure of severity

No. But according to the article, "Dr Coetzee said that ..., but added that after one month the case rate in South Africa is already decreasing." With such a high test positivity rate, how can someone say "case rate is decreasing"? On the published data, I don't see any evidence that the case rate is decreasing: the numbers are very high, and likely the limit on the number of tests that can be performed have been reached. The WHO target number for test positivity rate is 5% I think. In a wave it is often higher than that, sure.

> So far the little data available indicates it’s likely highly contagious, even compared to earlier strains

I agree.

> is possibly less severe

Dr Coetzee said "the severity of the disease is mild". Of course there are multiple interpretations of that: it is less severe on average than the Delta variant. Or it could mean: it is like a common cold. It is not clear what Dr Coetzee meant here.

thomasmg··on Head of the South African Medical Association says Omicron brings “mild illness”
Hospitalization, and death rate, are both growing in South Africa. The case rate isn't decreasing either. Test positivity rate was 60% in the last 24 hours, according to https://www.nicd.ac.za/diseases-a-z-index/disease-index-covi...

So those statements from Dr Coetzee don't seem to be based on actual data. They might be politically motivated, to get rid of the travel ban? Yes, the travel ban is probably useless now, as Omicron is already the majority of cases in e.g. London. But that's a different story.

thomasmg··on Data seems to indicate omicron milder, more transmissible, hospitalizations low
Well, there is not enough data for _simple_ answers yet (and people like simple, so the media likes simple; scientists also like simple).

But there is a lot of data, it is just hard to interpret, even for experts (which I'm not): It is currently summer in South Africa, but there was a summer wave last year. "Around 60-70% of South Africans have already had a Covid-19 infection" (from another comment). Age distribution. How fast omicron is spreading and why. How many are vaccinated (once, twice,...). How many breakthrough infections. Delta and other variants are spreading faster than the previous mostly due to higher viral load. Omicron seems to have has higher viral load. Omicron also binds to ACE2. What is the typical delay between infection and hospital / ICU admission. Some of this is discussed e.g. in https://yourlocalepidemiologist.substack.com/p/omicron-were-...

I understand that the media doesn't just stay silent. What I'm missing from them is in-depth and up-to-date info, with error bars. They can do that for weather forecasts, they should also be able to do this for omicron.

thomasmg··on Bosch gives go-ahead for volume production of silicon carbide chips
I bought one for my wife's wedding ring as well. The local jeweller (who made the rings) tested it with his diamond tester device -- and the led light said 'diamond'. He was a bit surprised :-)
thomasmg··on There’s a queue coming for Tesla charging
The capital can come from elsewhere. If the location of the cafe/restaurant is good enough, it is likely cheaper to add a charger + small convenience store to an existing cafe/restaurant, than to build a new restaurant when building a new charger.
thomasmg··on Tesla opens charging network to other EVs for the first time
I only read about the german minister of transport, Andreas Scheuer, was in talks to Tesla to open up the charging network. The source I have is here:

https://ecomento.de/2021/06/16/tesla-soll-supercharger-fuer-...

So, I think the logic was: eventually it will happen anyway - so it's better for Tesla to do it themselves before they are forced to, so that they can do it on their own terms. For example, so that Tesla drivers will never have to wait longer than x minutes.

thomasmg··on Non-Transitive Dice
This seem similar to "Penney's game" and the "Humble-Nishiyama Randomness Game" (https://en.wikipedia.org/wiki/Penney%27s_game) - betting on which coin-toss sequence appears first. For the 3-bit game there is also always a better pattern.
← PreviousPage 8 of 10Next →