duckerduck

I'm working on a development tool for specification-driven development. It uses LLMs to verify that your specification files and implementation do not drift. More specifically, I am trying to lower the number of false positives I'm currently seeing. I find that the LLM will hallucinate issues when there are no discrepancies, or become forgetful in the case of long documents (like RFC texts). The first step in improving this is to expand my evaluation suite so I can reliably measure improvements.

https://github.com/rejot-dev/semcheck/

It seems that many in the field are converging to similar ideas. With AI the emphasis seems to be more on defining clearly what to build rather than how. I made a similar experiment [1] recently where I setup "rules" between code and spec and have the LLM check that they are the same, I think similar to how Kiro uses hooks.

[1]: https://news.ycombinator.com/item?id=44432215

I've made a utility that uses LLMs to compare specifications against implementations, called "semcheck" (semantic checker). I’ve often found that even though I set up Cursor rules or CLAUDE.md files, the implementation tends to drift away from these documents. I built this utility to be run as a pre-commit or CI step to check that they are still the same.

Repo: https://github.com/rejot-dev/semcheck/