VulnHunter: Capital One's agentic AI code security tool | Hacker News Reader

VulnHunter: Capital One's agentic AI code security tool

capitalone.com

78 pointsmedina36 comments

IMHO these type of projects are not tools per-se but methodologies. I think this is a better framing since that's exactly what they are - a bunch of markdown files that describe in general terms how to perform an assessment aligned to some principles.

Btw, these type of methodologies are used all the time. Practically every security consultancy has them so adding them to an LLM makes a lot of sense.

If there is a pentester here who uses mitmproxy, the security skills below (distilled from 4000 h1 disclosures) might help -https://github.com/instavm/security-skills

this is just a side project though for me

Thanks for sharing.

How does your skills comparison to the offering from Capital One, as well,as what Visa published?

https://github.com/visa/visa-vulnerability-agentic-harness

The others work on repository or codebase. Mine works on API traffic realtime captured via mitmproxy.

More like a red team.

I can't imagine claude being able to use any of these without immediately refusing.

All these security/vulnerability scanning harnesses look more or less the same. Not sure what’s the point of bragging or publishing about them anymore, there’s no moat

They're desperate for the hype.

Eh, Capital One has long been surprisingly progressive on open source and whatnot. They were one of the first to properly adopt OAuth to connect accounts, too, back when Plaid/Mint/etc. were mostly proxying logins.

https://www.capitalone.com/tech/open-source/

https://developer.capitalone.com/documentation/o-auth

Vulnhunter isn't that exciting but if you scroll down their feed you'll see "guide to common rust errors" which... also isn't very exciting. I think they just need to publish something every week.

fwiw I have seen good whitepapers from them. A while back I used one about their IVR to get exec buy in for re-doing my company's IVR into something a lot better.

An end product to justify the Billions spent on AI

Also so that the engineers have something for the resume that looks and sounds impressive.

They don't know that Dario told me Fable is going to hack the planet.

I imagine they built it mostly for themselves, but open sourcing it is a nice gesture.

There are few more:

https://github.com/visa/visa-vulnerability-agentic-harness

https://github.com/cloudflare/security-audit-skill

I'm on the fence here, for one as from recent Linux mailing discussions those tools can really find good bugs (51% of them?), but on other side - I'm afraid of false sense of security.

Since you seem informed, can you help me understand how these work.

Are they glorified markdown skill files, or something more?

Why does this feel like an exec trying to justify token spend?

They dont need to justify it.

Sorry to say, tokens aren’t going anywhere, people aren’t going to suddenly stop using AI, and this whole paradigm shift of how people are changing how they work - is a full blown reality.

There is no reversal, no “eh we don’t think the tokens/AI are worth it”. Accept the new reality.

Don’t underestimate how banks operate. Things move very slowly in banking, so they absolutely need to justify it. Many banks still use GPT Mini, and even that requires approval from two levels of management. Even upgrading from Java 8 to Java 17 requires extensive justification.

Forget AI—you even have to justify using a MacBook.

You’re correct, but in the span of time - they are just doing the same thing everyone else is doing in many industries, just way slower

We also have to stop treating any criticism of some practices surrounding AI adoption like a rejection of the technology.

True, but this post in about Capital One.

It doesn’t take a genius or system architect to figure out AI use in banking both internally and for consumers is a huge benefit especially for capital one.

Maybe we should critique the actual agentic products they are creating rather than critique the entire AI spend.

Why does this feel like an exec trying to justify token spend?

I use LLM's everyday, I'm not sure why you took that take, but anyway.

> If you intend to use VulnHunter on Anthropic's first-party platforms (Claude API / Claude Code), we strongly recommend enrolling first via the verification portal.

Has anyone actually had success with this? I applied for my company several weeks ago, and never heard back.

Seems to be very random. I've heard stories like yours, as well as companies who applied and are approved same day.

Open source... Requires Claude Code CLI, authenticated with access to Claude Opus.

So, not Open Source.

Put a wrapper around nessus, stave off a "below strong" rating another six months

I know a few companies that did this about 20 years back (cough Comodo). Charge customers for a free Nessus report that's been rebranded. Profit.

Curious if the team at CapitalOne can share in more detail how this tool is being used internally, including how it’s helped their security practices and culture. That would help address some of the legitimacy concerns.

Wasn't Capital One founded on the premise of massive-scale market and product experimentation? Makes sense that they would design tools that match that approach.

This is a sad. Makes me want to move my bank accounts.

Why?

seems weird to not include shell scripts.

https://github.com/capitalone/VulnHunter/blob/main/vulnhunt/...

I think it's actually interesting how you can use the same model to both find and falsify the findings

you can, but it's better to use a different model or higher effort level at the very least to do the verifying part. (haven't checked to see what it is they are doing exactly), but doing vulnerability validation with the same model and effort you used to find the vulnerabilities isn't going to be a true second set of eyes and the model will likely always just try and justify it was right in the first place. ime, it's better to start with a fresh, clean context and at the very least a higher effort level on the same model. pass the finding(s) to a model that hasn't seen it before and it will judge it with an unbiased perspective.

VulnHunter: Capital One’s open-source, agentic AI code security tool.

Repo will be inactive and obsolete within 12 months.

Reply on news.ycombinator.com