d_watt

I would think a key difference is once this was found out, Dan Rather was fired.

There will always be failures of integrity. Our expectation can't be infallible people, but rather the rigor of an organization to care about the failures.

I had a Tesla for a couple of years and then sold it.

I'd buy another EV. I wouldn't buy another Tesla, because of the UX. Lack of physical buttons and changing UI. The infotainment screen crashed a couple of times in a year, the car was fine but suddenly the only place for car status feedback was gone. Lastly, not having a console in front of you, and needing to look to the side to see things like speed limit just felt unacceptable.

When I tried it with a copilot trial at one business, it basically just dumped huge text blobs into big text fields.

They push the "generate image based on prompt" as the main add for slides to make them not just bullets, but no real foundational slide creation is baked in.

Things may have changed since I demo'd it ~6 months ago.

Google Slides is just generated images and transform text as far as I've seen.

I really want 365 or gemini to do the obvious thing: "Using this template, summarize this document into a slide deck."

Maybe it won't be good, but it's surprising to me it can't do it barebones. Considering LLMs do an okay at summarizing into markdown, maybe the issue is a lack of proper intermediate DSL to represent slides it can train on?

But autogenerating at least a scaffolding for a deck seems so obvious, it's pretty damning it's so bad 2 years later.

I'm not sure I agree with the framing of this article.

It seems to be comparing two different things, 1. Points as a method for attributing difficulty to a work item, vs 2. The collaboration process of creating a task list as a method of breaking down work.

Running a team, I found pointing useful, more useful than the individual line engineers did, as we had different goals. My goal, in optimizing the team, was to go through all our work, get everyone in a room and create a relatively optimal plan for the next couple of weeks.

By going through the exercise of pointing, we often found that something one person thought was hard, another found easy. That act of estimating would reveal knowledge one person had that another didn't, that made the task easier. Without that process, the work might have been harder do do, because the easy way was never revealed. We also adhered to a 8 points is too many philosophy that meant any item that hard needed to be decomposed to simple tasks to repoint.

The "queuing" section basically implies the planning process should decompose tasks all the way to nothing but 1 point stories (or at least similarly sized work items within some variance). It's basically the same process as pointing, except not calling out some things are chunkier than others because it all comes out in the wash.

TL;DR this articles definition of queuing is basically pointing where task = 1 point items.

It's due diligence to understand your cost of goods sold using open source models where possible.

Don't actually spend the business effort to use them if it's not worth it, but have a business continuity plan to be able to switch over, especially if a "single source provider" going out of business or changing it's pricing could represent a existential thread to your business.

It's the same as any other tech vendor.

Streamlit, for one, was bought by snowflake for 800m, and they've been adding the ability to deploy streamline apps natively to your snowflake account.

I feel like this would do well to compare it to a more traditional CDN delivery method.

Other than the obvious edge benefits of a CDN, this would seem to open up issues around asset versioning. EG it's pretty standard to include asset/build hashes as part of urls, to ensure that the base app page and JS requests all get the correct build. You want JS to be cached between normal refreshes, but that to be busted on a new build.

If you were to deploy a new docker container with this pattern, you'd lose the old hashes, so potentially getting weird intermittent failures if someone's operating the old app in their browser, but the accompanying assets are no longer available due to a new build having gone out? If you don't do hashes, how do you ensure a new page load doesn't use a cached asset? ETAGS?

I think the article is a decent presentation of "docker 101" but left me with more questions than answers on what it actually means to try and do an dockerized ngnix static asset server rather than a CDN.

One of the things I've come to value as a "must have" for databases is the ability to stream CDC (Change Data Capture) out of the store.

With redis's persistance mechanisms (RDP + AOF), I see how one could set up their own CDC on top of those (tailing the AOF and reimplementing the redis instruction protocol to convert that to proper data changes), but does anyone know of a more structured way of piping out a change log?

I don't see anything obvious like a debezium connector.

"NewSQL" is not about analytical performance, it's about multimaster distributed writes and reads. EG, you don't want to run count(*) on spanner.

"HTAP" is the buzzword you're looking for. It's promising, but also complex and nascent. It'll be interesting to see how much traction it gets over time, but things like TiDB and Unistore are pretty early on to call a nail in the coffin for redshift/bigquery/clickhouse etc.

8 million people across 300 square miles, so there’s variety. It’s fine near me, but canceled dinner plans because of train issues.

Media will highlight the worst.

I think it's a sliding scale. The more unique your product is, and the more pain it solves, the more users are willing to put up with hard UI.

I've seen a lot of bad ui make a lot of money, because it found a market segment that had no other options.

> I never understood the appeal of docker-compose

Using a recent HN submission to spin up a local vector stack for analyzing notes, https://github.com/memgraph/odin/blob/main/docker-compose.ym...

How would you suggest a bash script handle configuring all the different images, their ports, and ensuring services are spun up in the correct dependency graph (parallel where possible), and are exposed to each other as a reliable host name over a subnet without polluting the host network?

And then how is that bash script extendable so it's not a custom script every time?

Sure, some OLAP databases let you add the same metadata that a OLTP database gives you as constraints, especially enterprise ones. A lot still don't, like Clickhouse, afaik.

No OLAP database I know of would let you encode other semantic layer things like aggregations or metrics. EG defining a DAU/MAU metric as "The distinct number of users logged in that day vs the distinct number of users in the 28 days before that day."

Those types of definitions usually live in the semantic layer or bi layer, which a LLM analysis tool would need to solve for.

For an article about LLM+OLAP, it doesn't spend much time on that part. Specifically it seems like their strategy is around using an LLM to generate a DSL query for an unnamed semantic layer, then everything downstream of that is normal warehousing, with the semantic layer handling actual SQL creation.

I wish it spent time on talking about how they trained their LLM to reliably generate parsable queries for the semantic layer, and what the accuracy rate of what the user intended vs what they got.

I do think the only way a LLM based analytics tool can succeed is via a semantic layer rather than direct SQL, since database schemas fail to encode a lot of information about the data (EG a warehouse might not even know user.customer_id = customer.id).

Malloy could be an interesting target here.

I'd caveat that it can be a good idea to make it a general api if you're building a B2B app. These days, purchasing checklists often include "do you have a public api?" Saying no to that can be a sign of product immaturity that is worth trying to get ahead of, and having to maintain internal and external apis is a lot of overhead.

It also can be good as you scale to have an API interface you can have a services team learn how to work with to enable "special" flows for customers.

I eventually found it under their pricing page.

I suspect from how "MVP" their site is, plus only NestJS actually has a "use" button instead of a "request demo" button, that the only thing currently supported is NestJS based javascript, and this is a way to test the water if they should build other languages. I could be wrong, ofc.

I definitely want some innovation in this space.

Does this actually track dependancies at the code level, or just at the module import level. EG if have a redis connection file that's imported to a service library file, and only 1 of 8 exposed methods from that service file use the imported Redis, will every file that imports that service library show a dependency to Redis, or just the ones that import the Redis based service.

Also, quick feedback on your site, your hero image shows the view that turns me off these types of things. An overwhelming graph that sprawls 3 screens horizontally, and thus can't be reasoned about in one look. The ones further down the page are more compelling, but perhaps the 1st one is the only one that actually represents the project?

I mean to be fair, even in using something like Postgres logical replication, there's no way to verify whether a particular change has replicated (minus xmin hackery).

If I were to write a tutorial on how to set up PG replication and "observe the replication", I think that would probably involve "write to master, sleep for a bit, then see the data is in the secondary."

It can be https://en.wikipedia.org/wiki/Garden_leave

Regarding your comment on updating models, are you saying you think they're updating the "pinned" models. EGT gpt4-0314?

Otherwise, I think effectively they already have what you're describing with LTS models as pinned versions, and the unversioned model is effectively the "nightly."

From what I've see in the materials, it seems like if you pay for dedicated compute, you can also have some control over your model versions.

If we're going to pedant about status codes, it seems like 401 is correct. 500 is a "server error" range, 503 would imply the service is down due to server side issues.

401:"not authorized" seems exactly correct for status page saying "I know who you are, and what you want, but you can't have it because you chose to disable your account."

Ah, almost like a little bit of a lower level trino, where DuckDB is able to push out queries to different data storage endpoints?

I see. Would it be fair to say you treat it almost like Pandas, except that it has a lower memory footprint since data is written to disk instead of memory. IE you use it for on the fly analysis of large frames of data, not like more traditional database/datawarehouse?

I'm duck-curious.

Looking at how it's deployed, as an in process database, how do people actually use this in production? Trying to figure out where I might actually want to think about replacing current databases or analyses with DuckDB.

EG if you deployed new code

1. Do you have a stateful machine you're doing an old school "Kill the old process, start the new process" deploy, and there's some duckdb file on disk that is maintained?

2. Or do you back that duckdb file in some sort of shared disk (Eg EBS), and have a rolling deploy where multiple applications access the same DB at the same time?

3. Or is DuckDB is treated as ephemeral, and you're using it to process data on the fly, so persisted state isn't an issue?

Because when you have a DAG of 100s of CREATEs that need to be run in dependency, many of which are table materializations and not views, you will end up creating a poorly specified, half baked version of DBT.

On the one hand, the compile targets for MetricFlow and DBT are SQL. So strictly speaking, nothing can be done with these that can't be done with a data warehouse.

SQL doesn't really have a way to define new aggregations well. So taking something like Daily Active Users / Monthly Active Users (DAU/MAU), which is a semi complicated calculation using windowed data, there's no good way to have a repeatable DAU/MAU aggregation concept you could reuse at different levels without having to re-write that sql that defines the aggregation. EG DAU/MAU against the account, or DAU/MAU by user type, or DAU/MAU by region would all be different queries/reports.

Instead if you have a layer that can be given the definitions of how a particular aggregation is structured, and it knows how data is related to each other, you can ask for that metric against different concepts or levels without having to repeat yourself.

Extending that out, if BI tools can understand that layer, then you can get consistent WYSIWYG report building across multiple tools without having to redefine the calculations everywhere.

I don't think I did a great job explaining here, this keynote is a good topic survey IMO: https://www.youtube.com/watch?v=w2weqVjbUTs&feature=emb_imp_...

I only recently discovered the term for this is quaila, https://en.wikipedia.org/wiki/Qualia.

What I personally like about SQL here is that you can start to think about your transformations in a purer "relational algebra" sense. You're able to declare the expected outcomes, and let the engine figure out the best way to handle that, as opposed to actually having to figure out how to join, aggregate, window, etc.

SQL transformations shine in environments where regular batching in short intervals is acceptable. Materialize and Flink are both super interesting for overcoming the batching shortcoming into realtime materialization via SQL.

Regarding the concerns about tests, it's true it's harder to do unit testing of SQL, but for me the tradeoff is it feels like there's a whole class of bug that's eliminated by focusing on the relational algebra. If you can avoid the category of work of defining the procedures to transform the data, and only define the expected outcome, things can be much simpler.

The other day I was thinking about how you have certain generations that are exposed to a new technologies, and are "step function" different from previous generations. EG, most recently "digital natives" being kids for whom the internet was a defacto part of life.

I was wondering what is the thing that's going to come that will make me marvel at that generations ability to use a technology effortlessly. Maybe it's a 3rd thumb from birth!