beyang

Hmm, I think we might need to update that ticket.

Here's some docs on Bazel support for Java/Scala precise code nav: https://sourcegraph.github.io/scip-java/docs/getting-started.... Any languages you're looking for in particular?

Bazel support in what sense? We have basic code nav across every major language, select precise indexers known to work with Bazel (with some degree of work), and we have improved code nav for Bazel files themselves (https://github.com/sourcegraph/sourcegraph/pull/37858), too. Search, of course, works across all languages. Happy to chat more about supporting your specific build environment: https://discord.gg/mBVP4JVBcj

LSP has addressed the editor code nav problem, but not the "just works in my web browser across all my code and multiple revisions" problem or the "let me build on top of a structured representation of code to enhance all my dev tools" problem. LSP's sister protocol LSIF aims to address the indexed code nav use case, but Steve calls out some of its shortcomings in the post, and these motivated our development of a new protocol, SCIP: https://about.sourcegraph.com/blog/announcing-scip

* Full code intelligence platform (not just code search) with Code Insights for tracking trends (https://about.sourcegraph.com/code-insights) and Batch Changes for large-scale refactors (https://about.sourcegraph.com/batch-changes)

* Precise code navigation (vs. more fuzzy-level nav), powered by SCIP (spiritual successor to Grok, the system Steve built at Google)

* More powerful search language beyond regex (supports comby.dev) + user-friendly (smart search)

* Works across multiple GitHub instances + other code hosts (GitLab, Bitbucket, Perforce, enterprise Git repositories)

* Self-hosted deployment and enterprise scale

cs.github.com is a significant improvement over github.com/search—kudos to the team there!—but is about feature parity with something like OpenGrok, Hound, or Google Code Search before Steve built and integrated Grok (primarily cross-codebase regex search).

Sourcegraph's licensing is open core (similar to GitLab): https://handbook.sourcegraph.com/departments/engineering/pro....

Our code is public, regardless of whether it is open-source or enterprise licensed. The open source code includes most of the product features; the enterprise code includes things like enterprise AuthN, AuthZ, and admin capabilities that are needed for large companies to use Sourcegraph.

Sourcegraph.com/search indexes over 6 million open-source libraries, nearly every github.com repository with at least 5 stars and the most popular projects across NPM, PyPI, Maven Central, and many more independent projects and code hosts.

Sourcegraph CTO here. We're elated to welcome Steve to the team and will be hosting a Twitter live / AMA (https://twitter.com/sourcegraph/status/1577364344056201236) with him tomorrow at 1pm PT. Join us if you're free!

We're trying out a series at Sourcegraph doing walkthroughs and deep-dives into popular open-source projects:

1. How Caddy web server works: https://sourcegraph.com/notebooks/Tm90ZWJvb2s6MTM2Nw==

2. How TypeScript ESLint works: https://sourcegraph.com/notebooks/Tm90ZWJvb2s6MTA2OA==

The goal is to write these as if we were onboarding new contributors to the codebase. If there are maintainers of projects that'd like to be featured, please hit us up: https://discord.gg/SQCbGe6zTn. And you can subscribe here: https://tourdesource.substack.com.

Author of the post here. Thought it might be of interest to call out some observations about how the world has changed since I wrote this:

1. Code search is more mainstream. It used to be most of Sourcegraph's sales came from ex-Googlers missing code search, but nowadays, most of our new revenue originates from dev teams just struggling with the challenge of big codebases.

2. Investment in dev tools has only accelerated. A lot of pandemic beneficiaries (virtual events, tele-medicine, at-home exercise equipment, crypto) are now reversing, but dev tools growth continues to be strong.

3. Bazel adoption has grown. Were I writing the post today, I would have had a category for monorepo tooling that included companies like EngFlow and Turborepo.

4. Kubernetes is still dominant for container orchestration, but serverless and managed services seem to be gaining more adoption. Classic tension between customer focus and open standards—will be interesting to see how we trade off dev-ex vs proprietary platform dependency.

Since this post was first published, we've also made a lot of advances at Sourcegraph and are looking for people who want to play around with some new experimental search and code intel features. Come say hi in our Discord! https://discord.gg/SSCBGByJeu

Ironically, some of the steps that Bazel takes to make builds hermetic make it more difficult to integrate with to pull the information we need out of the build for precise code navigation. But we're working on it :) If this is an area of interest to you, pop into our community Discord and say hello!

Similar concept: https://news.ycombinator.com/item?id=31948833

Author of the post here. This really resonates with me. One of the best pieces of advice I've received in my life as a programmer, from my first manager actually, was "focus on getting an end-to-end system up and running first, no matter how janky, and then refine from there".

Sourcegraph CTO (and author of the post) here. There's no part of our product that seeks to track software productivity?

We do have a feature (https://about.sourcegraph.com/code-insights) built around code search that lets devs define their own metrics around things like progress of big migrations and anti-patterns + code smells. This tweet is a good summary of my philosophy around code-related metrics: https://twitter.com/beyang/status/1524259425451577344.

Insights from treating code as data are best discovered by the people who know the dataset through and through—i.e., developers. Another project we've released in this vein is https://codestat.dev, built on top of Comby (incidentally also on the HN front page right now), which was created by a member of our team, Rijnard, as part of his PhD thesis on automatic program transformation.

We're developers by trade and by heart, and are motivated by making tools and mental models that devs actually find useful and that boost the creative spark of coding. If there's something we can improve about our product or message to further this mission, I'd love to hear your feedback!

Sourcegraph CTO here. It's interesting to read about GitHub's approach and how it contrasts with the approach we've taken at Sourcegraph. One of the key tradeoffs the article highlights is GitHub's decision to take the "shallow-but-wide" approach to code navigation, which has enabled them to provide some level of code navigation for most open-source repositories on GitHub, but at the expense of precision/accuracy (i.e., the system can't necessarily differentiate between different symbols with the same name).

Sourcegraph decided early on to take the opposite approach, favoring precision and accuracy over supporting every public codebase. Part of the reason why is that we aren't a code host that hosts millions of open-source repositories, so we didn't feel the need to support all of those at once. Another big reason is we heard from our users and customers that code navigation accuracy was critical for exploring their private code and enabling them to stay in flow (inaccurate results would break the train of thought because you'd have to actively think about how to navigate to the referenced symbol). We actually built out a language-agnostic search-based code navigation, but increasingly user feedback has driven us to adopt a more precise model, based at first on our own protocol (https://srclib.org) and also the LSIF protocol open-sourced by Microsoft that now enables code navigation for many popular editor extensions.

This is not to say that GitHub's approach is wrong, but more to say that it's interesting how different goals and constraints have led to systems that are quite different despite tackling the same general problem. GitHub aiming to provide some level of navigation to every repository on GitHub, and Sourcegraph aiming to provide best-in-class navigation for private codebases and dependencies.

(Btw, hats off to the GitHub team for open-sourcing tree-sitter, a great library which we've incorporated into parts of our stack. We actually hosted the creator of tree-sitter, Max Brunsfeld, on our podcast awhile back and it was a really fun and insightful conversation if people are interested in hearing some of the backstory of tree-sitter: https://about.sourcegraph.com/podcast/max-brunsfeld.)