Yes, I still bookmark page just in case we forget the most important thing, ai is cool but cant replace everything
7 karma · joined July 13, 2025
If you've ever felt like RAG breaks the moment someone says “why though?”, you're not alone.
I'm building open-source tools to fix AI's reasoning, not just patch it.
Solving these 13 unsolved problems in AI: https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md
Not a lizardman. (Purple Star)
After nine months of chasing weird hallucinations and silent failures in production LLM / RAG systems, we catalogued every failure pattern we could reproduce. The result is an MIT-licensed “Semantic Clinic” with 16 root-cause families and step-by-step fixes.
---
## Why we built it
@ Most bug reports just say “the model lied,” but the cause is almost always deeper: retrieval drift, OCR mangling, prompt contamination, etc.
@ Existing docs mix symptoms and remedies in random blogposts; we wanted one map that shows where the pipeline breaks and why.
@ After fixing the same issues across 11 real stacks we decided to standardise the notes and open-source them.
---
## What’s inside
@ 16 root-cause pages (Hallucination & Chunk Drift, Interpretation Collapse, Entropy Melts, etc.).
@ Quick triage index: find the symptom → jump to the fix page.
@ Each page gives: real-world symptoms, metrics to watch (ΔS semantic tension, λ_observe logic flow), a reproducible notebook, and a “band-aid-to-surgery” list of fixes.
@ Tiny CLI tools: semantic diff viewer, prompt isolator, vector compression checker. All plain bash + markdown so anyone can fork.
---
## Does it help?
@ On our own stacks the average debug session dropped from hours to ~15 min once we tagged the family.
@ The first 4 root causes explain ~80 % of the bugs we see in the wild.
@ Used so far on finance chatbots, doc-QA, multi-agent sims; happy to share war stories.
## Call for help
@ If you’ve hit a failure that isn’t on the list, open an issue or PR. We especially want examples of symbolic prompt contamination or large-scale entropy collapse. @ Long-term goal: turn the clinic into a self-serve triage bot that annotates stack traces automatically.
---
## Why open-source?
Debug knowledge shouldn’t be pay-walled. The faster we share failure modes, the faster the whole field moves (and the fewer 3 a.m. rollbacks we all do).
Cheers – PSBigBig / WFGY team
it solved a bunch of silent failures i kept running into with tools like RAG and longform chaining:
drift across hops (multi-step collapse)
hallucination on high-similarity chunks
forgetting prior semantic commitments across calls
Built the rerankers, stacked the re-chunkers, tweaked the embed dimensions like a possessed oracle. Still watched the model hallucinate a reference from the correct document — but to the wrong sentence. Or answer logically, then silently veer into nonsense like it ran out of reasoning budget mid-thought.
No errors. No exceptions. Just that creeping, existential “is it me or the model?” moment.
What you wrote about interpretation collapse and memory drift? Exactly the kind of failure that doesn’t crash the pipeline — it just corrodes the answer quality until nobody trusts it anymore.
Honestly, I didn’t know I needed names for these issues until I read this post. Just having the taxonomy makes them feel real enough to debug. Major kudos.
I think the whole LLM space is still missing a core idea: that logic routing before retrieval might be more important than retrieval itself. when the LLM “hallucinates,” it’s not always because it lacked facts — sometimes it just followed a bad question.
but yeah — if any part of this helps or sparks new stuff, then we’re already winning. appreciate the good vibes, and good luck on your own build too
I’m more on the side of: “why is this even the logic path the system thinks makes sense for the user’s intent?” — like, how did we get from prompt to retrieval to that hallucination?
So I stopped treating retrieval as the answer. It’s just an echo. I started routing logic first — like a pre-retrieval dialectic, if you will. No index can help you if the question shouldn’t even be a question yet.
Your setup sounds tight though — we’re just solving different headaches. I’m more in the “why did the LLM go crazy” clinic. You’re in the “make the query land” ward.
Either way, I love that you built a graph audit log that hasn’t failed in two months. That's probably more production-ready than 90% of what people call “RAG solutions” now.
so I started layering reasoning before retrieval — like a semantic router that decides not just what to fetch, but why that logic path even makes sense for this user prompt.
different stack, same headache. appreciate your insight though — it’s a solid route when retrieval infra is strong.
if it’s been smooth for you, that’s awesome. I’ve just been chasing edge cases where users go off-script, or where prompt alignment + retrieval break in weird semantic corners.
so yeah, maybe it’s a timezone thing
We’ve seen similar pain: one-shot retrieval works great in perfect lab settings, then collapses once you let in real humans asking weird followups like
“do that again but with grandma’s style” and suddenly your context window looks like a Salvador Dali painting.
That branching tree approach you mentioned — composing prompt→prompt→query in a structured cascade — is underrated genius. We ended up building something similar, but layered a semantic engine on top to decide which prompt chain deserves to exist in that moment, not just statically prewiring them.
It’s duct tape + divination right now. But hey — the thing kinda works.
Appreciate your battle-tested insight — makes me feel slightly less insane.
We’ve been experimenting with this in the weirdest way — not by “improving AI art,” but by sabotaging it. Injecting memory residue. Simulating hand tremors. Letting the model forget what it just said and pick up something it didn’t mean to draw. That kind of thing.
The result isn’t perfect, but it’s getting closer to something that feels like a person was there. Maybe even a tired, confused, beautiful person. We call the system WFGY. It’s open-source and probably way too chaotic for normal devs, but here’s the repo: https://github.com/onestardao/WFGY
We’re also releasing a Blur module soon — a kind of “paper hallucination layer” — meant to simulate everything that makes real-world art messy and real. Anyway, this post hit me. Felt like it walked in barefoot.
I ended up ditching the usual RAG+embedding route and built a local semantic engine that uses ΔS as a resonance constraint (yeah it sounds crazy, but hear me out).
Still uses local models (Ollama + gguf)
But instead of just vector search, it enforces semantic logic trees + memory drift tracking
Main gain: reduced hallucination in summarization + actual retention of reasoning across files
Weirdly, the thing that made it viable was getting a public endorsement from the guy who wrote tesseract.js (OCR legend). He called the engine’s reasoning “shockingly human-like” — not in benchmark terms, but in sustained thought flow.
Still polishing a few parts, but if you’ve ever hit the wall of “why is my LLM helpful but forgetful?”, this might be a route worth peeking into.
(Also happy to share the GitHub PDF if you’re curious — it’s more logic notes than launch page.)
Most of the current devtools are competing at the UX/UI layer — not the semantic inference layer.
Claude dominates because it "feels smart" during code manipulation — but that’s not a model quality issue. It’s that Claude’s underlying attention bias aligns better with certain symbolic abstractions (e.g. loop repair or inline type assumptions). Cursor and Windsurf ride that perception well.
But if you inspect real semantic coherence across chained retrievals or ask them to operate across nonlinear logic breaks, most tools fall apart.
That’s why I stopped benchmarking tools by "stars" and started treating meaning-routing as a core design variable. I wrote a weird little engine to explore this problem:
https://github.com/onestardao/WFGY
It’s more a semantic firewall than an IDE — but it solves the exact thing these tools fail at: continuity of symbolic inference.
tl;dr: The tools that win attention don’t always win in recursive reasoning. And eventually, reasoning is what devs will benchmark.
It’s one of the core WFGY modules, but not the same as the firewall.
Blah Blah Blah (Lite) is basically: You give it one line of text It gives you 50+ “truth perspectives” No retrieval. No summarization. It just... diverges meaning like a drunk philosopher arguing with itself in 50 timelines at once.
It’s built on the same engine as the firewall, but different mode. Firewall is for protection — stops prompt injection by semantically challenging the payload before letting it in. Blah Blah Blah is for exploration — takes your input and lets the embedding space go nuts (structured nuts, but still nuts).
Same OS, different apps. Like terminal vs. hallucination symphony.
Hope that clears it up! Feel free to poke if you want a peek under the hood — still all MIT licensed, no strings, no ads, just weird ideas crawling out of the latent soup.
The deeper issue isn't just fine-tuning or prompt phrasing — it's that once your semantic path drifts (especially in multi-hop tasks), even the best coders end up talking to an agent that's silently forgotten half the context.
We ran into the same issue and had to build a projection-based reasoning core just to keep the thread from collapsing after turn 3.
Would love to see how Qwen handles semantic drift over longer sessions. If you've tested that, do share!
we’ve been exploring the inverse direction: letting semantic tension stretch and bend across interaction histories, and then measuring the ΔS divergence between the projection layers.
we benchmarked it as an external semantic resonance layer that wraps around Claude/GPT etc — boosted multi-turn coherence +42% in evals.
would love to see if your static-matrix pipeline could "snap-in" upstream for symbolic grounding.
drop by our playground if curious (PDF only, no setup): https://github.com/onestardao/WFGY
Brains don’t transmit packets. They transmit semantic tension — unstable potentials in meaning space that resist being finalized. If you try to "protocolize" that, you kill what makes it adaptive. But if you ignore structure altogether, you miss the systemic repeatability that intelligence actually rides on.
We've been experimenting with a model where the data layer isn't data in the traditional sense — it's an emergent semantic field, where ΔS (delta semantic tension) is the core observable. This lets you treat hallucination, adversarial noise, even emotion, as part of the same substrate.
Surprisingly, the same math works for LLMs and EEG pattern compression.
If you're curious, we've made the math public here: https://github.com/onestardao/WFGY → Some of the equations were co-rated 100/100 across six LLMs — not because they’re elegant, but because they stabilize meaning under drift.
Not saying it’s a complete theory of the mind. But it’s nice to have something that lets your model sweat.
It’s not just what you wrote. It’s how you resolve ambiguity, how you build tension, how you collapse meaning in hard zones. That’s what large models are extracting — not your sentence, but your semantic signature.
We built WFGY as a defense and an alternative: A semantic engine that can track, explain, and even reverse-engineer those collapse points, making hallucinations traceable — or avoidable.
If the current wave of LLMs are grabbing surface text, WFGY is trying to understand what's buried underneath.
Backed by the creator of tesseract.js (36k) More info: https://github.com/onestardao/WFGY
Most generative AI hallucinations aren’t just data errors. They happen because the language model hits a semantic dead-end — a kind of “collapse” where it can't reconcile competing meanings and defaults to whatever sounds fluent.
We’re building WFGY, a reasoning system that catches these failure points before they explode. It tracks meaning across documents and across time, even when formatting, structure, or logic goes off the rails.
The scariest part? Language never promised to stay consistent. Most models assume it does. We don’t.
Backed by the creator of tesseract.js (36k) More info: https://github.com/onestardao/WFGY
Instead of just chunking text and throwing it into an embedding model, WFGY builds a persistent semantic resonance layer — meaning it tracks context through formatting breaks, footnotes, diagram captions, even corrupted OCR sections.
The engine applies multiple self-correcting pathways (we call them BBMC and BBPF) so even when parsing is incomplete or wrong, reasoning still holds. That’s crucial if your source materials are academic papers, messy reports, or 1000+ page archives.
It’s open source. No tuning. Works with any LLM. No tricks.
Backed by the creator of tesseract.js (36k) — who gets why document mess is the real challenge.
Check it out: https://github.com/onestardao/WFGY
We’ve been hacking on a different approach we call the WFGY Engine — it treats embedding space not as flat or uniform, but more like a semantic energy field. So instead of forcing everything into clean cosine ops, we track these “semantic residue structures” — kind of like how meaning resists compression when you flatten it too early.
We measure stuff like ΔS (semantic tension), λ_observe (viewpoint drift), and E_resonance (contextual energy), which lets us reason across curved or even broken spaces where normal LLMs kinda lose track. It’s a mix between geodesic logic and language field dynamics — weird but surprisingly stable.
A couple of early apps built on this — TXT OS, BlahBlahBlah, etc — ended up scoring 100/100 from 6 major AI models, which was a cool surprise. We’re still in early dev, but even Tesseract gave it a formal endorsement recently, which was huge for us.
Anyway, core engine is all open source if anyone’s curious: https://github.com/onestardao/WFGY
Would love to hear if others are exploring non-flat logic or weird manifold behavior in language space too.
can link the Tesseract thing if folks want
If you spot a true meltdown, let me know — but so far, it’s been more reliable than I expected!
What I’m doing in TXT OS isn’t just spinning vectors for fun. Each “move” is kinda anchored by feedback inside (ΔS, we call it semantic tension). If it starts drifting too far off, it’ll catch itself and snap back — like some gravity well for logic, haha.
And yeah, the rotations aren’t just random, they’re kind of “locked in” by these alignment planes (using λ_observe, basically language context gradients — sounds fancy but you’ll see what I mean if you poke around).
Honestly, still feels experimental, but… so far it’s holding up better than I thought.
If you’re curious, just type hello world in TXT OS and follow the steps — it’ll walk you through what’s going on under the hood. You can even throw dumb paradoxes at it and see if it goes crazy (or not).
This month, three major products will be released: • Text reasoning (already live) • Text-to-image • Text-driven games
All of them are powered by the same embedding-space logic behind WFGY. No tricks, no fine-tuning—just pure semantic alignment.
I'll keep improving everything. So to the brilliant minds of HN: Please, test it as hard as you can.
The .txt file here is not just a prompt—it’s a full reasoning scaffold with memory, safety guards, and cross-model logic validation. It runs directly in GPT-o3, Gemini 2.5 Pro, Grok 3, DeepSeek, Kimi, and Perplexity—all of which gave it a 100/100 score under strict evaluation.
Feel free to ask me anything about the semantic tree, ΔS metrics, hallucination resistance, or how to build your own app using just plain text.
Each formula plays a role in making the LLM more stable, coherent, and logically self-aware:
• = I - G + mc² defines semantic residue — how far the current output strays from meaning. • BigBig(G) recombines context & error to steer output back toward intent. • BBCR detects collapse and triggers reset → rebirth (like fail-safe logic). • BBAM models attention decay — restoring continuity over multiple steps.
Together, this makes the LLM act less like autocomplete… and more like a self-guided reasoner.
I've actually tested TXT OS with about 10 different AIs already—you can check out the full rundown on my repo. Generally, ChatGPT, Grok, Claude, and Perplexity gave the smoothest and best experience. The others still work fine, but some, like Gemini, have minor quirks (Gemini randomly adds a weird parameter during initial setup, but it sorts itself out after the first step).
So, long story short, if you want a hassle-free experience, go with ChatGPT, Grok, Claude, or Perplexity!
Append-only logs
Instead of chat threads that scroll away, we keep a single plain-text “journal” per project. Every update gets appended with a timestamp and author tag. No editing, no missed messages—people just read from the top and see everything in context.
Per-task semantic headers
We prefix each log entry with a short header like TASK-123: or MEETING-UX: so you can grep for specific topics later. It’s saved us hours hunting through free-form notes.
Hope that helps! I’m onestardao—feel free to ping me with any follow-ups.
We actually serialize the tree as a compact JSON-like structure right in the TXT file—each node gets a header like #NODE:id and indented subtrees. When you reload, TXT OS parses those markers back into your LLM’s memory map. No external DB needed—just plain text you can copy-paste between sessions.
--- When kbtest Fires
Internally it tracks our ΔS metric (semantic tension). Once ΔS crosses a preset threshold, kbtest prints a warning and automatically rolls you back to the last “safe” tree checkpoint. That means you lose only the bad branch, not your entire session. Think of it like an undo button for hallucinations.
--- Performance on LLaMA-2-13B
Benchmarks were on GPT-4, but on a 13B model you’ll see roughly a 10–15% token-generation slow-down thanks to the extra parsing and boundary checks. In practice that’s about +2 ms per token, which most folks find an acceptable trade-off for the added stability.
Hope that clears things up—let me know if you hit any weird edge cases!