never_inline

How many microorganisms and pests have you deprived of livelihood? Why stop at animals?

Can someone elucidate how using a full blown browser is improvisation over using say markitdown / pandoc / whatever? Given that most useful coding docs sites are static (made with sphinx or mkdocs or whatever)

The problem I see with MCP is very simple. It's using JSON as the format and that's nowhere as expressive as a programming language.

Consider a python function signature

list_containers(show_stopped: bool = False, name_pattern: Optional[str] = None, sort: Literal["size", "name", "started_at"] = "name"). It doesn't even need docs

Now convert this to JSON schema which is 4x larger input already.

And when generating output, the LLM will generate almost 2x more tokens too, because JSON. Easier to get confused.

And consider that the flow of calling python functions and using their output to call other tools etc... is seen 1000x more times in their fine tuning data, whereas JSON tool calling flows are rare and practically only exist in instruction tuning phase. Then I am sure instruction tuning also contains even more complex code examples where model has to execute complex logic.

Then theres the whole issue of composition. To my knowledge there's no way LLM can do this in one response.

    vehicle = call_func_1()
    if vehicle.type == "car":
      details = lookup_car(vehicle.reg_no)
    else if vehicle.type == "motorcycle":
      details = lookup_motorcycle(vehicle.reg_ni)

How is JSON tool calling going to solve this?

Wasn't there a tool calling benchmark by docker guys which concluded qwen models are nearly as good as GPT? What is your experience about it?

Personally I am convinced JSON is a bad format for LLMs and code orchestration in python-ish DSL is the future. But local models are pretty bad at code gen too.

People are used to the `click` way, where you can define args as function parameters. It's little more verbose but it helps click is a very established library which also provides many other things needed by CLI tools.

There's also `typer` from the creator of `fastapi` which relies on type annotations. I have not had the opportunity to use it.

At this point you're just flexing that you have 96GiB machine. (Average developer machines are more like 16GiB)

But that's not the point. If every dependency follows same philosophy, costs (compiler time, binary size, dependency supply chain) will add up very quickly.

Not to mention, in big organizations, you have to track each 3rd party and transitive dependency you add to the codebase (for very good reasons).

Doesnt work recursively?

> Breaking up long expressions into smaller ones

If it improves compile time, that sounds like a bug in the compiler or the design of the language itself.

I mean, LLMs you can run on ollama are usually pretty bad ones.

You can annotate `json:"-"` which is equivalent of @JsonIgnore

They can be used in conjunction. You can drop into debugger from REPL and vice versa.

IDEs tend to have a "just my code" option.

I use single-stepping very rarely in practice when using a debugger, except when following through a "value of a variable or two". Yet it's more convenient than pprint.pprint() for that because structured display of values, eval expression, and ability to inspect callers up the stack.

I am young grug who didn't use debuggers much until last year or so.

What sold me on debugger is the things you can do with it

    * See values and eval expressions in calling frames.
    * Modify the course of execution by eval'ing a mutating expression.
    * Set exception breakpoints which stop deep where the exception is raised.

One other such tool is REPL. I see REPL and debugger as complementary to each other, and have some success using both together in VSCode, which is pretty convenient with autoreload set. (https://mahesh-hegde.github.io/posts/vscode-ipython-debuggin...)

Why would I?

The only argument is that it improves the style of writing.

But I am in an ESL environment and no one cares about that.

Even otherwise why would anyone want to read a decompressed version instead of the "prompt" itself?

I run aider in VSCode terminal so that I can fix smaller lint errors myself without another AI back-and-forth.

Here's a system prompt I tend to use

    ## Instructions
    * Be concise
    * Use simple sentences. But feel free to use technical jargon.
    * Do NOT overexplain basic concepts. Assume the user is technically proficient.
    * AVOID flattering, corporate-ish or marketing language. Maintain a neutral viewpoint.
    * AVOID vague and / or generic claims which may seem correct but are not substantiated by the the context.

Cannot completely avoid hallucinations and it's good to avoid AI for text that's used for human-to-human communication. But this makes AI answers to coding and technical questions easier to read.

Because why do you anti-compress your thoughts using LLM at all? It makes things harder to read.

They may be overrepresented in the RLHF

Apart from obvious non-reproducibility, the other problem is lack of navigable structure. I can't command+click or "show usages" or "show definition" any more.

> Don't be afraid to get your hands dirty. Some bugs and styling issues are faster to fix manually than to prompt through. Knowing when to intervene is part of the craft.

This has been my experience as well. to always run the cli tool in the bottom pane of an IDE and not in a standalone terminal.

It actually bothers me it took so long to be discovered.

> When debugging error handling code, being able to quickly add a println or have a dedicated line or source location for setting a breakpoint in a debugger is helpful.

On the flip side, you can't have exception breakpoints in Go.

Stupid question: Why not provide a threadpool-like construct which may not necessarily keep threads around but limits their number?

They are all compatible with existing stuff, which makes it kinda better than js.

Thanks for writing the LLM CLI.

It's been a very useful tool to test out and prototype using various LLM features like multimodal, schema output and now tools as well! I specifically like that I can just write a python function with type annotations and plug it to the LLM.

Even cline documentation seems to be slop generated.

Give it context and copy paste the code yourself.

They're good at coming up with new code.

Give it function signature with types and it will give pretty good implementation.

Tell it to edit something, and it will lose track.

The write-lint-fix workflow with LLMs doesn't work for me - LLM is monkey brain edits unrelated parts of code.

And yet people here keep arguing proper cs education is not important.

Without proper understanding of CS, this is what we get. Lack of rigour.

You literally wrote it can be more powerful than Google search.