Flux159

The graphs around programming language & use cases makes sense from my personal usage. What's interesting is that they call out "automation" vs "augmentation", but also mention "directive" vs "feedback loop" for both claude code and claude ai - directive in this case being similar to solving a problem without iteration while feedback loop is the human iterating with an agent.

I think for the near future we're going to continue seeing feedback loop be the primary way we interact with coding agents simply because we generally don't specify all coding tasks with no ambiguity. If we did, then we'd already have some pseudo code that the agent could just directly translate into working code / validate with testing.

Some very quick initial thoughts - the json spec has some similarities to mcp: https://google.github.io/A2A/#/documentation?id=agent-card - there's an agent card that describes capabilities that google wants websites to host at https://DOMAIN/.well-known/agent.json according to https://google.github.io/A2A/#/topics/agent_discovery so crawlers can scrape to discover agents.

The jsonrpc calls look similar-ish to mcp tool calls except the inputs and outputs look closer to the inputs/outputs from calling an LLM (ie messages, artifacts, etc.).

The JS server example that they give is interesting https://github.com/google/A2A/tree/main/samples/js/src/serve... - they're using a generator to send sse events back to the caller - a little weird to expose as the API instead of just doing what express allows you to do after setting up an sse connection (res.send / flush multiple times).

I think the link should be updated to this since it's currently just pointing to a git commit.

I just made an open source CLI based chat client that supports MCP servers to solve this problem yesterday at https://github.com/Flux159/mcp-chat.

The issue I faced during MCP server development of my Kubernetes MCP server (https://github.com/Flux159/mcp-server-kubernetes) is that I wanted to be able to easily test changes to the server without having to deal with Claude Desktop. Realized that I could make a simple CLI client that lets me automate some of the testing & it just happened to also be a decent terminal based chat client for any production MCP servers too. Requires using your own Anthropic API key, but other than that it should "just work".

Is there a way to always accept certain commands across sessions? Specifically for things like reading or updating files I don't want to have to approve that each time I open a new repl.

Also, is there a way to switch models between 3.5-sonnet and 3.5-sonnet-thinking? Got the initial impression that the thinking model is using an excessive amount of tokens on first use.

It's interesting that Anthropic is making their own coding agent with Claude Code - is this a sign of them looking to move up the stack and more into verticals that model wrapper startups are in?

This is super cool & I'm impressed that it's working with WebGL.

From reading the code & seeing the examples in the sandbox, it looks like the sprite layer is defined one time (initialization) with specific sprite & animation properties that are exposed to JS via the SpriteGPULayer config https://github.com/phaserjs/phaser/blob/v4.0.0-beta.5/src/ga... , then that class passes the info over to the vertex shader which has ifdefs to determine which animation, other features are enabled for a given SpriteGPULayer - https://github.com/phaserjs/phaser/blob/v4.0.0-beta.5/src/re...

This way you don't end up having to do any updates per frame on the CPU and it remains one draw call for the entire layer right?

Are there other use cases besides from large scale sprite/animated backgrounds that you've thought of?

To run the full 671B Q8 model relatively cheaply (around $6k), you can get a dual EPYC server with 768GB RAM - CPU inference only at around 6-8 tokens/sec. https://x.com/carrigmat/status/1884244369907278106

There are a lot of low quant ways to run in less RAM, but the quality will be worse. Also, running a distill is not the same thing as running the larger model, so unless you have access to an 8xGPU server with lots of VRAM (>$50k), cpu inference is probably your best bet today.

If the new M4 Ultra Macs have 256GB unified RAM as expected, then you may still need to connect 3 of them together via Thunderbolt 5 in order to have enough RAM to run the Q8 model. Assuming that the speed of that will be faster than the EPYC server, but will need to test empirically once that machine is released.

This is a great list - I was wondering if there was a less research oriented, more experimental or practical reading list that you're planning as well - some things off the top of my head:

- Actual examples of Fine tuning of LLMs or making merges - usually talked about in r/localLlama for specific use cases like role playing or other scenarios that instruction tuned LLMs are not good at. Jupyter notebook or blog post would be great here.

- Specifically around Agents & Code generation - Anthropic's post about SWE-bench verified gives a very practical look at writing a coding agent https://www.anthropic.com/research/swe-bench-sonnet with prompts, tool schema and metrics.

- The wide amount of Loras and fine tunes available on civitai for image models - a guide on making a custom one that you can use in ComfyUI.

- State of the art in audio models in production - Elevenlabs seems to still be the best for closed platforms, but there are some options for open source voice cloning, TTS, or even text to speech with very small parameter models (kokoro 82M).

This looks pretty nice - this is specifically replacing the JupyterLab frontend and keeping the connections to Jupyter kernels - there shouldn't be any theoretical reason that it couldn't support Javascript or other language kernels, although I guess the project has only been tested with IPython kernels.

Would be interested to see where this goes.

Google's answer to o1-mini that is available via API with much higher limits than OpenAI. Performance is said to be very good for it's speed but ymmv https://x.com/JeffDean/status/1869795828493320541

I was in a Waymo last week in San Francisco that had a bicyclist cut in front of us, no hard breaking or jerk just a gradual decrease in speed. I would've hit the break relatively fast myself but Waymo handled it without issue in a very smooth way.

Also see https://x.com/dmitri_dolgov/status/1868778679868047545

I don't know how it'll handle significantly more pedestrians, but I assume that they're confident enough in the models and have run enough simulations to expand to Tokyo.

I've been using Sapling along with the VSCode extension for ISL which mostly gets me a better workflow with diff stacks locally (Github isn't great dealing with stacks though).

The only major limitation for me is git-lfs support so for certain repos I have to revert to git.

Right now I'm doing prompt execution in activities, passing results to other activities.

Workflows currently only started after human events, but going to move towards additional workflows started through an activity soon - I'm keeping the state of the world independent of the execution engine so each workflow will read the new state.

I'm working on building out an AI agent right now - looking through autogen, langgraph and other current frameworks along with just building my own logic on top of a workflow orchestrator.

The ReAct paradigm for agents makes me think of standard workflows because they're not that different conceptually: Read the state of the world, plan / "reason" based on user intent, perform actions that mutate, then go back and read the state of the world if problem hasn't been fully solved (ie start another workflow). Similar in concept from reading from dbs, computing logic, mutating / writing to new table, trigger follow on job from data engineering.

Congrats on launching! From your explanation comparing to traditional workflow engines it would be a step up from something like Airflow, but I was wondering about a comparison to Temporal which does let you create dynamic DAGs as well as provides some additional guarantees around reliability of workers in case of crashes.

> Why would I want to play a 3D game in my fucking browser of all places?

To provide users a way to instantly play a game without having to download all assets at once. Give developers a potential way to avoid app store royalties of up to 30% on desktop or mobile. With wgpu in rust, you can also target WebGPU as a shared 3d runtime that will run across OS's natively rather than having to target Vulkan, Metal, and DirectX.

> Why would I want to run an LLM in the browser, I could just run it natively for better performance?

What about users who don't know how to download a model and run it locally? I would argue this is the vast majority of users in the world. Also, this specific use case is probably not going to be generalized with WebGPU yet due to model sizes, but rather other APIs like the Prompt API in Chrome which will use Gemini Nano embedded into the browser (assume it will eventually get standardized). https://developer.chrome.com/docs/ai/built-in-apis

I agree with you that WASM and WebGPU will be used for adware, targeting, and spyware - but if you don't want to use them, you should disable them in your browser settings - there's definitely value add for other users even if you can't see any benefits.

I like they're adding 4MP images, but after so many models in the past 2 years (diffusion, LLM, etc.), I can't keep up with what model(s) are best for which use cases.

I know civitai has fine tunes specifically for anime style, realistic, etc. but I don't know which one is "state of the art". /r/stablediffusion usually gets overly hyped about new models and isn't really searchable for what is sota today. This doesn't even get into models that are only accessible via api like flux pro or through an app (midjourney).

LLMs have pretty much the same problem for locally runnable models and api based ones (llama vs qwen 2.5 vs sonnet 3.5 for coding vs other tasks).

Does anyone know of a github repo or an app that is keeping these things up to date? Or is that something that other people would also want to collaborate on?

I like it - was just making a diagram for a container architecture & used Mac OS Freeform, but I could keep this in my repo easily & update via text.

Also, like that there’s an online playground for trying it out.

If I already have a postgres operator setup on Kubernetes - what additional benefits is this getting me?

Right now super fast start times aren't really needed since it's a database that I expect to be running for a while - ms vs 30s is fine. It's easier to setup a test database on the same machine I'm running automated tests so that may not be a good usecase either.

I'm glad that they've made improvements to startup speed and size of the containers which would be good if it's open sourced, but I don't know if this is good as a paid service if you already have an easy way of setting up new clusters in k8s.

Ah thanks for clarifying. I understand the focus on being a scripting syntax first after looking at google/zx & rereading the docs.

Also had some more time to look at the code, seems like the CLI is using https://github.com/microsoft/genaiscript/blob/main/packages/... to execute scripts which uses promptrunner in core https://github.com/microsoft/genaiscript/blob/main/packages/... - it looks like it's not trivial to use directly from JS or decouple the runtime? Unless that's on the roadmap somewhere to be able to call the scripts in the same process as a Node app.

This seems like it can be super useful - lot to go over but want to focus on running commands in containers. Having a clean and elegant way of executing LLM commands in a containerized environment is definitely better than running on a single VM/machine. I do wonder how something like this would run in a containerized application though - can you define the "host" to be a kubernetes control plane?

Note that the container link on the homepage to https://microsoft.github.io/genaiscript/referenc/scripts/con... seems to be broken.

Edit: Okay, taking a deeper look - this seems to be a separate runtime on top of node - "$" seems to be global and files are defined as .genai.mjs and run through VSCode. I see that there's a way to run via CLI as well: `npx genaiscript run proofreader path/to/files*.md`, but I wonder what the rationale around these design decisions are. It seems like it's tying these files to a different runtime - can I use them directly in Node or Bun as part of an API? Something like "import {templateStr as $, def} from 'genaiscript'", then use it normally in Node?

I have a Minisforum UM690S, it's about the same size (5x5x2.25 inches) and works well as a small machine. They just announced the EliteMini AI370 today that has the latest AMD laptop chip with 12 cores/24 threads - I assume that would also be a decent linux server (note that ram is soldered on that one).

Time to update all the Mac Mini server racks for the new design

I would also say that more than 1 in 6 companies don’t have the culture or resources to make college graduates successful - I’m more experienced with software engineers, but it takes significant time and effort from management and senior talent to make recent graduates set up for success. It’s less for other senior talent, but sometimes senior talent still needs significant help.

One good thing about most college graduates is that they can generally learn new things quickly, especially if they’re willing to put in some effort.

This looks interesting - is there a comparable ring that also has a temperature sensor? It would be interesting to be able to determine if you're sick a day or two ahead like an Oura ring or Apple's new Vitals app for Apple Watch using an open source app.

Alternatively, does anyone know if it's possible with the sensors just in this ring?

It would be nice to have comparisons to Claude 3.5 for the coder model, only comparing to open source models isn’t super helpful because I would want to compare to the model I’m currently using for development work.

This kind of research is great for reducing training costs as well as enabling more people to experiment with training large models. Hopefully in 5-10 years we'll be able to train a model on par with SD 1.5 with consumer gpus since that would be great for teaching model development.

This would also work for local development and then put up a pr, but I feel like that's a temporary thing as well until a model or agent-like workflow is good enough to directly read from a github issue and make the PR directly.

Right now Cursor can already use Claude 3.5 so I feel like that's very close to integrated into a code editor.

With how good Claude 3.5 is for coding, I would love if this supported private github repos - just connect my github account and provide a repo link which stays up to date as changes are made instead of having to upload content to a project.

I feel like this will probably be an option down the line - I wonder if it would be possible to replicate this functionality via the API as well, kind of like how there are open source projects attempting to replicate artifacts via the API.