Hacker News Reader

funfunfunctionon Show HN: RLM-based local debugger for AI agent traces

There's some benchmarks in the repo for AppWorld. Looks promising

funfunfunctionon Show HN: RLM-based local debugger for AI agent traces

Cool project. A team at work was building something similar to internal use.

I'm curious how this compares to just using Claude Code directly and giving it a dump of the agent traces? It seems like Claude could probably do some of the same diagnostics / trace grouping to identify failure patterns. Why use a custom harness?

funfunfunctionon [dead]

Hi all, we built a super easy way to train small language models from your production data - install a gateway to save your request/response data from a frontier provider, and use the traces to train a open source model that can be hosted privately on your machine.

We've seen some pretty impressive results already, including training an HTML->JSON model that competes with frontier quality at 5% the cost called [schematron](https://huggingface.co/inference-net/Schematron-8B)

It's great for data extraction, chat, search, image classification, sentiment analysis, and more.

Trainings are free for the next 30 days as we test and experiment. We're also going to make it possible to launch and manage training runs on your own GPUs, so everything is private.

funfunfunctionon Show HN: Using LLMs and >1k 4090s to visualize 100k scientific research articles

We'll release the full data explorer soon, with more info.

At the core of this project is a structured-extraction task using a custom Qwen 14B model, which we distilled from larger closed-source models. We needed a model we could run at scale on https://devnet.inference.net, which is comprised mostly of idle consumer-grade NVIDIA devices.

Embeddings were generated using SPECTER2, a transformer model from AllenAI specifically designed for scientific documents. The model processes each paper's title, executive summary, and research context to generate 768-dimensional embeddings optimized for semantic search over scientific literature.

The visualization uses UMAP to reduce the 768D embeddings to 3D coordinates, preserving local and global structure. K-Means clustering groups papers into ~100 clusters based on semantic similarity in the embedding space. Cluster labels are automatically generated using TF-IDF analysis of paper fields and key takeaways, identifying the most distinctive terms for each cluster.

funfunfunctionon The case for the return of fine-tuning

Creator of inference.net / schematron here.

There is growing emphasis on efficiency as more companies adopt and scale with LLMs in their products.

Developers might be fine paying GPT-5-Super-AGI-Thinking-Max prices to use the very best models in Cursors, but (despite what some may think about Silicon Valley), businesses do care about efficiency.

And if you can fine-tune an 8b-parameter Llama model on GPT-5 data in < 48 hours and save $100k/mo, you're going to take that opportunity.

funfunfunctionon UWU – generate CLI commands without leaving the terminal

OP here. I wanted a dead-simple way to quickly generate CLI commands without the overhead of Claude Code or Cursor, so I built it in an afternoon.

The project uses some zsh magic to allow for quick editing of the model's response before running the command. Commands show up in your shell history, just like any other command.

The project is intended to be very simple and hackable -- the whole project is one TypeScript file.

Feedback (and pull requests) welcome!

funfunfunctionon A GPU Calculator That Helps Calculate What GPU to Use

This is a cheap marketing ploy for a GPU reseller with billboards on highway 101 into SF.

funfunfunctionon Distillation makes AI models smaller and cheaper

There are even companies starting to offer distillation as a service https://inference.net/explore/model-training

funfunfunctionon My chatbot builder is over-engineered and I love it

This is cool! I don’t see many people doing write ups on their tech stack as much any more. It’s nice to see the inside of a production-grade app like this.

I’m curious, why command+r for the model? What benefits does it have over other SOTA models?

funfunfunctionon Inference.net: Wholesale LLM Tokens

It’s unlikely an individual would need this much capacity. Folks who need tokens at this level are apps with lots of users that don’t have their own GPUs. Think character.ai type apps.

10B tokens on together.ai is ~$2,000.

funfunfunctionon Ask HN: Who is hiring? (August 2024)

Kuzco, Inc. (https://kuzco.xyz) | Full-stack SWEs and MLE | Full-time | San Francisco, CA

We're building a serverless LLM inference network that makes use of underutilized capacity from GPU data centers. Our product is a scheduler for running LLM inference workloads on GPUs located all over the world. We currently have over 6,000 GPUs on our network, and are growing quickly.

Things we need help with:

* Improve core scheduling algorithms

* Optimize vLLM inference runtime

* Improve logging and observability stack

* Building our user-facing dashboard and APIs

* New products

We're well-funded and have a clear path to profitability. We're currently a four person team of staff-level engineers and looking to add two more engineers to our team. Our office is near the Ferry Building in downtown San Francisco. We pay well and offer significant equity.

Apply here: https://apply.workable.com/kuzco

Or send an email to sam at kuzco.xyz with your resume, GitHub, and a bit about yourself.

Thanks!

funfunfunctionon How Does OpenAI Survive?

> generative AI is a product with no mass-market utility.

> I am neither an engineer nor an economist.

clearly.

funfunfunctionon Ask HN: How to monetize a website that is making views?

Awesome project! I’m sure someone would be willing to buy this from you if the traffic is big enough. If you want to move on to other engineering projects this may be best path. If you’re interested in learning sales and marketing yourself I would look into affiliate marketing with hair loss product companies. Happy to chat more about this if you want help.

funfunfunctionon Employees are feeding sensitive data to ChatGPT, raising security fears

People aren’t using ChatGPT because they can’t do it themselves, they’re using it to save time.

funfunfunctionon Autodoc: Toolkit for auto-generating codebase documentation using LLMs

Hey if your project is public I would love to take a look if you don’t mind sharing a link

funfunfunctionon Autodoc: Toolkit for auto-generating codebase documentation using LLMs

Generally these documents wouldn't be used directly. You can use the `doc q` command to query these documents with natural language questions.

funfunfunctionon Autodoc: Toolkit for auto-generating codebase documentation using LLMs

This is exactly what autodoc does in your terminal. It wouldn't be hard to package it as a VSCode plugin.

funfunfunctionon Autodoc: Toolkit for auto-generating codebase documentation using LLMs

Completely agree. Explaining how systems work in plain english is much more valuable than just giving inputs and outputs of individual functions. We want to understand how a system and it's subsystems work independently and interdependently.

We're not there yet with Autodoc; there is still tons of work to do.

If you have't tried the demo, give it a shot. You might be surprised.

funfunfunctionon Autodoc: Toolkit for auto-generating codebase documentation using LLMs

Generally these documents would be used directly. You can use the `doc q` command to query these documents with natural language questions.

funfunfunctionon Autodoc: Toolkit for auto-generating codebase documentation using LLMs

Running the `doc estimate` command on the Wordpress repository says it will cost $58.47. The estimates are usually +/-10% of the actual cost.

funfunfunctionon Autodoc: Toolkit for auto-generating codebase documentation using LLMs

How many pages? You can get an estimate for how much it would cost using the `estimate` command in Autodoc.

funfunfunctionon Autodoc: Toolkit for auto-generating codebase documentation using LLMs

Hallucination is definitely a problem but can be somewhat mitigated by good prompting. GPT-4 seems less prone to hallucination. This will be better over time.

You can view the prompts used for generating docs here[1] and the prompts used for answering questions here[2]

[1]https://github.com/context-labs/autodoc/blob/master/src/cli/... [2https://github.com/context-labs/autodoc/blob/master/src/cli/...]

funfunfunctionon Autodoc: Toolkit for auto-generating codebase documentation using LLMs

Hi there, creator of autodoc here.

You can do it yourself and make pull request back to the BuildKit codebase :)

If it gets merged everyone who uses BuiltKit would have access.

funfunfunctionon Show HN: Semantic search for video

We’re using Pinecone. Great experience so far.

funfunfunctionon Show HN: Semantic search for video

Hey this is really cool! I built something similar for https://addcontext.xyz over the break. We just launched a self-service version that allows anyone to enter a YouTube playlist link and create a mini search-engine from the content.

I've seen a few other people working on similar stuff in the wild. Very curious to see where this goes.

funfunfunctionon Ask HN: Feeling burned out at an early stage in my career

https://news.ycombinator.com/item?id=15505304

I also went into tech out of high school and burned out after a few years. I wrote a post about it on here and got some really good feedback. There may be some tidbits in there that also apply to you.

Ultimately taking a few months off and traveling/hanging out with family worked wonders. Happy to talk more about it with you. samheutmaker at gmail

funfunfunctionon We don’t use a staging environment

Infra as code + good modern automation solves the parity issue. I empathize with wanting to stay lean but this seems extreme.

funfunfunctionon Tortured phrases: A dubious writing style emerging in science

This is called content spinning.

funfunfunctionon Real world ownership is not a use case for blockchain

This is an interesting idea, do you have any resources to help me better understand how this would work?

funfunfunctionon Real world ownership is not a use case for blockchain

> Mortgage underwriters can pool their blockchain based loans into pooled securities, and sell CDO-like products as smart contracts. This allows investors to completely bypass all the financial regulations and restrictions that exist on subprime mortgages.

This is exactly what https://www.figure.com/ is doing.