GitHub PM here. It scans using regexes and then applies post-processing. So yep, it can (and does) parse JWTs to understand their properties.
1,399 karma · joined March 25, 2012
However, with secret scanning alerts we look for credentials from service providers we _don't_ have a partnership with, too. Our partnerships team are pretty good, so the delta isn't that big, but Asana, Notion, Intercom and Artifactory are a few of the service providers whose tokens we scan for where we don't (yet!) have a relationship to send detections. We also scan for tokens where a partnership isn't possible or would be much harder (like HashiCorp Vault service tokens).
On standardized formats, if one existed we would scan for it! However, as we've worked with dozens of service providers to update their formats we've found many have specific constraints and everyone has different preferences - as a result, for now, we're pursuing a broad church approach, rather than pushing a standard. If you haven't already read Thomas Ptacek's survey (for fly.io) I recommend it.[2]
[1] https://docs.github.com/en/developers/overview/secret-scanni...
The big benefit of highly identifiable tokens is not just that we can alert on them, but that we can scan for them at pre-receive time and prevent them from leaking (by rejecting the push). We already have that functionality as part of GitHub Advanced Security, and are planning to make it available (for free) on public repos in 2023.
[1] https://github.blog/2021-04-05-behind-githubs-new-authentica...
https://docs.github.com/en/code-security/secret-scanning/sec...
One thing to note is that the full CVSS 3.1 string is included in the database as assessed by NIST. The severity displayed by GitHub is stored as a "database specific" field, so it looks like we're trying to be explicit about the existence of multiple perspectives on severity (one of which is our own), but that we could do more to make that clear.
https://github.com/github/advisory-database/blob/main/adviso...
On backfilling the data to include advisories from before 2017 - absolutely. So far we've done this in a relatively ad-hoc way - you should already find that the most important (severe and wide-reaching) CVEs from before 2017 are in the database (and if there are any that aren't you think should be we'd love you to open an issue on the DB). We want to do a more complete backfill in the near future.
https://github.com/github/advisory-database/blob/main/adviso...
(Not trying to claim the GitHub Advisory Database is perfectly complete - it’s not, and achieving that is part of why we’ve opened it up to community contributions. Just that the comparison with everything in the NVD isn’t apples to apples - the databases have different scopes.)
Right now, our focus is on going deep for a smaller number of ecosystems before going broad. The intention is that anyone using one of the languages in the current list feels “fully covered” by the data in the database.
For repositories using a language the GitHub Dependency Graph supports, we automatically create an inventory of the dependencies the repository uses and create alerts if/when any have a vulnerability (via Dependabot alerts and, as a sibling comment has already mentioned, Dependabot update PRs).
The next improvement we'd like to ship is an API that lets you upload a list of dependencies to us for repositories in which we can't automatically detect them. A good example is repositories using Gradle for dependency management - it's hard for us to understand the dependency tree there without running a build. With the new API you'll be able to upload a list of dependencies (generated using a Gradle command) to GitHub in CI, and GitHub will then be able to send alerts if/when there's a vulnerability in one of those dependencies, just like we do for repos using other package managers.
Your comment specifically mentions containers. That's one area that's a little further off for native GitHub support, but where the open source advisory database should help. Whilst we're currently focussed on scanning source code and surfacing results on repos (not containers), the structured data in the advisory database is just as usable with the results of a container scan. Indeed, I believe all the open source container scanning solutions already use it as a data sources.
On the alerting side, we have a couple of things coming. Neither are magic bullets, but both will help.
- Better handling of vulnerabilities in dev dependencies. Some vulnerabilities matter if they're in a dev dependency - anything that exfiltrates your local filesystem, for example. Other's don't - DoS vulnerabilities, for example. At the moment, GitHub doesn't even tell you whether the dependency a vulnerability affects is a runtime or development dependency. We can and will get better there.
- Analysis of whether the vulnerable code in a dependency is called. You almost certainly want to react faster to vulnerabilities in your code that your application is actually exposed to than to ones that it may be exposed to in future. (You probably want to respond to the unreachable ones, too, especially if you can get an auto-generated PR to do so, but there's much less urgency.) We have this in private beta for Python right now, and expect to have it in public beta in the next few months.
Beyond alerting, the other big thing is that GitHub's incentives for this database and the experiences it triggers are fundamentally different from other vendors. We aren't selling its contents, so don't have an incentive to inflate it. Open source maintainers are at the heart of our platform, and we really don't want low quality advisories go out about their software. And developers are our core customers, and we want to deliver experiences they love above all else. That difference in incentives will likely manifest in lots of little differences, but at a high level, we're aligned on wanting to reduce the alert fatigue.
Sorry we dropped the ball on this for the last couple of years. You're going to see steady improvements from here on.
We have some work to do on the tooling to make it really slick, and a couple of those PRs have taken longer to get reviewed than we'd like, but we're working on it!
On trusted members of language ecosystem - we'd be super interested to explore that. It will require some work on the tooling on our side, so I don't expect progress there overnight, but in the long term is a model I think we could make work really well.
One big reason is that the alternative to this structured data being open source is that it lives in proprietary databases. In that world, attackers still have knowledge about these vulnerabilities - they don't need the structured data as much as defenders, and the licenses on those proprietary databases aren't going to deter them anyway (most are public for SEO reasons). Defenders on the other hand, often won't have as much or as high quality information.
For anything that doesn't already have a CVE, no. We don't want that disclosure process to happen in public - we recommend you reach out to the maintainer privately. (Currently we don't have an on-platform way to do that, but we're planning one.)
On major strand is more work like this to make it easy for the community to collaborate. I expect we'll make a lot of iterative improvements to the database over the next few months, aimed at making it easier to contribute to, maintain and use. We need to improve our APIs for this data, for example (currently only available via GraphQL).
Another big one that we're starting to think about is the security vulnerability disclosure process. Our goal there is to support maintainers as much as possible, and there's more we can do. Recent articles on loguru, beg bounties, and the way log4j initially reached public attention all point to problems GitHub can and should help with. In the next 12 months we'd like to give maintainers the option to receive vulnerability disclosures privately on GitHub, and for us to be able to support them through that process. (GitHub already does a bit here - through maintainer security advisories we issued about 30% of the CVEs in the JavaScript ecosystem last year, for example. But we can and will do more.)
Loguru CVE article: https://tomforb.es/cve-2022-0329-and-the-problems-with-autom...
Beg bounties: https://www.troyhunt.com/beg-bounties/
Log4j PR: https://github.com/apache/logging-log4j2/pull/608#issuecomme...
One big challenge is that it's hard to get service providers to change their token formats. Very few have this at the top of their priority list - they're busy with other things. Here's an example playing out in OSS that is pretty typical: I tried to persuade the (excellent) team at Sentry to update their format, and they essentially told me "we have other priorities" https://github.com/getsentry/sentry/pull/26313. And that's a relatively simple changes, not the adoption of a whole standard.
In addition, as Thomas points out in this article, there are a lot of different token types for someone thinking about minting API tokens to choose between. They might rationally have different preferences over them. A standard that is prescriptive of format and approach is likely to struggle given that diversity.
With that said, I do see an opportunity here for a more modest standard targeted at service providers that already use JWTs or Macaroons. Generic tokens of those types are relatively easy for scanning providers to identify, and it's easy (and hopefully uncontroversial) for service providers to encode more information in them, like an "if found" link. I think a standard that defines the attribute name there, and the API for reporting / responding, would be a good start that might see adoption.
I work on secret scanning at GitHub. When token issuers use easily identifiable formats for their tokens we can easily spot them when they're accidentally committed. We can then work with the token issuer to automatically alert them of those leaks. A good example is AWS - if you commit an AWS key and secret to a public GitHub repo we will tell AWS about it and they will tell you about it (and quarantine the exposed keys) within a few seconds. We work with dozens of other token issuers too, though - some of the latest we added were Linear, PlanetScale and Ionic.
The above relies on tokens being identifiable - we can't send hundreds of partners everything that looks like 32 hex chars. In future we want to be able to do even more sophisticated things, like ask users for confirmation before the push code that contains secrets. We recently changed our own token pattern for that reason.
GitHub secret scanning program: https://docs.github.com/en/developers/overview/secret-scanni...
GitHub's updated token format: https://github.blog/2021-04-05-behind-githubs-new-authentica...
(FWIW, I'm sure we can think of something to fix (1) which balances security and usability - it just needs some thinking. (2) sounds like a bug to me - I'm sure the team will want to look at that. And (3) is a tricky case where Dependabot's Ruby logic needs to be able to unlock a dependency's "parent" or peers in order to make security fix (e.g., rails in the case where the vulnerability is in activerecord). Entirely possible but hard!)
Update: https://github.com/dependabot/dependabot-core/pull/4077