ankrgyl

It's just a different workflow IMO. AI is effectively real-time, whereas offshoring, no matter the quality, is something you have to do in batches.

> In ??? coding, the developer specifies code changes that must be made, such as adding a feature, modifying an existing function, or removing unused logic. The AI delivers the complete set of changes to the codebase, and the developer reviews it at the code level. Any corrections are given again as updates to the code, and the cycle repeats until the code aligns.

Yes

> If so, I've most seen the latter be called AI pair-programming or AI-assisted coding.

I specifically considered both terms and am not a fan * "pair-programming" is something that involves two people paying attention while writing code, and in this case, i'm not looking at the screen while the AI system writes code * "AI-assisted coding" is generally anchored to copilots/IDE style agents where people are actively writing code, and an AI assists them.

I totally hear you on conflating async. However, I think the appropriate term would clearly indicate that this happens without actively watching the AI write code. Unfortunately I think other terms like "background" may also be confusing for similar reasons.

(Author here)

Hi everyone, thanks for the spirited debate! I think there are some great points in the discussion so far. Some thoughts:

* "This didn't work for offshoring, why will it work all of a sudden?" I think there are good lessons to draw from offshoring around problem definition and what-not but the key difference is the iteration speed. Agents allow you to review stuff much faster, and you can look at smaller pieces of incremental work.

* "I thought this would be about async primitives in python, etc" Whoops sorry, I can understand how the name is confusing/ambiguous! The use of "async" here refers to the fact that I'm not synchronously looking at an IDE while writing code all the time.

* "You can only do this because you used to handwrite code". I don't think this workflow is a replacement for handwriting code. I still love doing that. This workflow just helps me do more.

(Author here) I can certainly appreciate having an alternate perspective, but I think it's unfair to say it's DOA. I've personally used this workflow for the last 6 months and shipped a lot of features into our product, including the lowest levels of infra all the way to UI code. I definitely think there is a lot to improve. But it works, at least for me :)

(Author here) Personally, I try to combat this by synchronously working on 1 task and asynchronously working on others. I am not sure it's perfect, but it definitely helps me avoid atrophy.

(Author here) Haha that is a great point. I was trying to come up with a term that described my personal workflow and specifically felt different than vibe coding (because it's geared towards how professional programmers can use agents). Very open to alternative terms!

Cool to see the balance between UX and security. I really like the UI!

Thanks for mentioning Braintrust!

We are very committed to the proxy :)

Although, to your point, we have seen less market pull for routing, and more for (a) supporting the latest LLMs, (b) basic translation (e.g. tool call API b/w Anthropic & OpenAI), and (c) solid infra features like caching/load balancing api keys/secret management. So that's our focus.

We absolutely can. The proxy already supports running in multiple environments, including your own AWS VPC deployed on Lambda, so I'm sure we could add support for Bedrock (Anthropic or other models).

Feel free to ping me at ankur@braintrustdata.com and we can chat more!

We think a lot about this. I think it's very important to us to provide building blocks that foster a more open ecosystem. Part of that is providing building blocks (like the proxy) that do not lock you too much into "our" way of doing things either.

Model routers are a "semantic" abstraction -- they analyze the contents of a prompt and automatically determine which provider to use. This proxy is (currently) much simpler: you pick the provider you want and we add a bunch of nice features like caching and low-latency on top.

We will likely work with or build some routing capabilities in the future!

LiteLLM is a great project. The key differences (from what I can tell) are that:

a) This is hosted b) Supports caching and load balancing c) Can manage multiple providers behind a single API key d) Implemented in Typescript (vs. Python)

On the other hand, LiteLLM is a more mature project and supports significantly more model providers than we currently do!

It's not open source at the moment, but we're definitely open to it.

Thanks for all the support Bryan and team!

Who do you think is the sweet spot user for Rivet? Anyone building any LLM app, or a certain kind (e.g. agents)? Is there a use case for which you'd advise against using Rivet?

I'm not super familiar with Dolt's SQL implementation, but I'm surprised that a simple `count(*)` query timed out:

https://www.dolthub.com/repositories/dolthub/transparency-in...

Great to see popular SaaS use cases commoditized and opened up like this. After product categories start to fossilize into a standard UX + features, the benefits of using an expensive vendor start to decrease.

Will you ever consider using EBS volumes or another mechanism which supports random writes?

Why use Kubernetes if you’re in control of the cloud environment? What does it bring to the table? Why not firecracker?

I love DuckDB and am cheering for MotherDuck, but I think bragging about how fast you can query small data is really no different than bragging about big data. In reality, big data's success is not about data volume. It's about enabling people to effectively collaborate on data and share a single source of truth.

I don't know much about MotherDuck's plans, but I hope they're focused on making it as easy to collaborate on "small data" as Snowflake/etc. have made it to collaborate on "big data".

It’d be great to show the debugging experience in the video (in fact, I’d prefer seeing that over the breadth of features). E.g. what happens when there’s a syntax error in my sql query or the python code fails on an invalid input?

That tends to be the critical make it or break it feature when you’re writing code in an app builder.

The Rust vs. Go comparison has two key differences:

- The Rust example uses 8 bit unsigned ints vs. Go example uses 32 bit signed ints

- Rust's sort is stable by default whereas Go's is not.

If you tweak the Rust benchmark to use `i32` instead of `u8` and `sort_unstable` instead of `sort`, you should see ~3-4x faster performance.

They release a lot of weights open source, including T5 (the underlying model they used in this work). They also indicated their intent here: https://twitter.com/aleksandrafaust/status/15799326368934420....

This is a "classic" tension. Having worked in the (broader) RPA space for a while, I would say that the true north star of most processes is (a) rewriting the internal procedures to be transformations on data (not UIs) and (b) standardizing communication across companies.

There is a lot of momentum to solve (a) with no code, but it's slow because processes are impossibly complex. I think AI will accelerate this and could result in the "human backup" dystopia. On the other hand, AI can also be used to generate code, and I'm optimistic that technology like this can accelerate humans' ability to encode complex processes robustly (as transformations of data) and would 10 or 100x less work than no/low code.

There is a visual demo here: https://sites.google.com/view/llm4html/home.

This work is very exciting to me for a few reasons:

- HTML is an incredibly rich source of visually structured information, with a semi-structured representation. This is as opposed to PDFs, which are usually fed into models with a "flat" representation (words + bounding boxes). Intuitively, this offers the model a more direct way to learn about nested structure, over an almost unlimited source of unsupervised pre-training data.

- Many projects (e.g. Pix2Struct https://arxiv.org/pdf/2210.03347.pdf, also from Google) operate on pixels, which are expensive (both to render and process in the transformer). Operating on HTML directly means smaller, faster, more efficient models.

- (If open sourced) it will be the first (AFAIK) open pre-trained ready-to-go model for the RPA/automation space (there are several closed projects). They claim they plan to open source the dataset at least, which is very exciting.

I'm particularly excited to extend this and similar (https://arxiv.org/abs/2110.08518) for HTML question answering and web scraping.

Disclaimer: I'm the CEO of Impira, which creates OSS (https://github.com/impira/docquery) and proprietary (http://impira.com/) tools for analyzing business documents. I am not affiliated with this project.

Thank you ultrasounder! Supply chain, construction, purchasing, insurance, financial services, and healthcare are our biggest verticals. Although we have customers doing just about anything you can imagine with documents!

For invoices, we have a pre-trained model (demo here: https://huggingface.co/spaces/impira/invoices) that is pretty good at most fields, but within our product, it will automatically learn about your formats as you upload documents and confirm/correct predictions. The pre-trained model is based on LayoutLM and the additional learning we do uses a generative model (GMMs) that can learn from as little as one example.

LMK if you have any other questions.

It's very easy to try! Just plug that URL here: https://huggingface.co/spaces/impira/docquery.

I tried a few questions:

  What is the development date? -> June 20, 2017
  What is the medicine? -> SPINRAZA® (nusinersen)
  How many doses -> 5 doses
  Did the patient meet the review criteria? -> Patient met initial review criteria.
  Is the patient treated with Evrysdi? -> not

> but my use case is in creating an offline-first local document storage system (data never reaches a cloud).

Makes sense -- this is why we OSS'd DocQuery :)

> The templates sound very cool - are they essentially just using a preset list of (natural language) queries tied to a particular document class? It seems like you're using a version of donut for your document classification?

Yes that's the plan. We've done extensive testing with other approaches (e.g. NER) and realized that the benefits of using use-case specific queries (customizability, accuracy, flexibility for many use cases) outweigh the tradeoffs (NER only needs one execution for all fields).

Currently, we support pre-trained Donut models for both querying and classification. You can play with it by adding the --classify flag to `docquery scan`. We're releasing some new stuff soon that should be faster and more accurate.

Can you tell me a bit more about your use case? A few things that come to mind:

- There are some ML/transformer-based methods for extracting a known schema (e.g. NER) or an unknown schema (e.g. relation extraction). - We're going to add a feature to DocQuery called "templates" soon for some popular document types (e.g. invoices) + a document classifier which will automatically apply the template based on the doc type. - Our commercial product (http://impira.com/) supports all of this + is a hosted solution (many of our customers use us to automate accounts payable, process insurance documents, etc.)

DocQuery (https://github.com/impira/docquery), a project I work on, allows you to do something similar, but search over semantic information in the PDF files (using a large language model that is pre-trained to query business documents).

For example:

  $ docquery scan "What is the due date?" /my/invoices/
  /my/invoices/Order1.pdf       What is the due date?: 4/27/2022
  /my/invoices/Order2.pdf       What is the due date?: 9/26/2022
  ...

It's obviously a lot slower than "grepping", but very powerful.