HNHacker News
TopNewBestAskShowJobs

williamzeng0

340 karma · joined November 13, 2022

ML Engineer at Sweep AI (autocomplete & coding agent for JetBrains)
submissionscomments
williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
I really liked the blogpost! We're hoping to change point 2 (Engineering Time is Finite) with Sweep, so hopefully we don't have to make a tradeoff between a high quantity/quality of tests.
williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
Good point, we may allow open source repos to do this in order to credit contributors that wrote the original issues (those contributions are really valuable).
williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
Our code is messy (sweep hasn't gotten around to it yet), but here's where we save the code! https://github.com/sweepai/sweep/blob/main/sweepai/core/vect...

So for context, this is running in a ephemeral function from Modal https://modal.com/docs/reference/modal.Function#modalfunctio....

We need a way to store the computed embeddings, because the function doesn't persist any state by default, so we use Redis. But we don't want to store the actual code as the key, so we hash the code + add some versioning. Because it's a cache, it supports concurrent writes + reads, which a lot of vector dbs do poorly.

So the actual code is only accessed at runtime (using the GitHub app authentication to clone the repo), and we also build the vector db in memory at runtime. It's slow(redis call, embedding the misses, constructing the index), but 1-2s is negligible in the context of Sweep because a single openai call could be 7s+.

And one nice feature is that when you have Sweep running on 10+ branches (which probably share 95%+ of the code) we just use the cache hits/misses to automatically handle diffs in the vector db. It's super easy to setup, we don't need to manage different indices (imagine a new index per branch), and it's very cost efficient.

williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
Much appreciated! I just got a new mic so the audio won't be so bad.

What kinds of videos would you like? We can make anything, the two repos we use the most are Sweep itself and our landing page

williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
Here it is: https://docs.sweep.dev/privacy

The logs from Sweep(which contain snippets of code) are logged for debugging purposes. We don't train on any of your code. These will only be stored for 30 days. We send this data to OpenAI to generate code. We're using the OpenAI api, and OpenAI has an agreement stating they will not train on this data and will persist it for 30 days to monitor trust and safety.

We index your codebase for search, but we use a system that only reads your repo at runtime in Modal. This runs as a serverless function which is torn down after your request completes. Here's a blog we wrote about it! https://docs.sweep.dev/blogs/search-infra

williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
We're on it! Perfect time to ask Sweep to give it a try.
williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
That's exactly the use case we want. We also let you specify the file path (ex: "main.py").

We noticed that Sweep's search works way better if there are comments, because the comments match up really well with the search queries (language <-> language is easier than language <-> code)

williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
I really agree with the second point. Even if there are enough junior devs, there's small issues where you're on the go and delegating is relatively expensive as the expected turn-around time is generally in the hours. Often times I would just do it myself, but then it burns part of the stamina. Also we're trying to make reviewing easier with webpage previews and automated testing through Github actions.
williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
Cool! We're focused on close integration with GitHub and handling smaller, more focused tasks. We also have plans to run Sweep migrations, let me know if you'd like to see that!
williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
Haha, that's a bad way to measure output. Unfortunately the commits are attributed to Sweep ;)
williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
Interesting, that change would take some time to materialize. In the meantime it might be best to adopt both? I don't see it as a complete substitute.

Right now you could have some junior devs picking up work that Sweep can't handle in order to grow and learn, and eventually still become senior devs. Having a small team also helps with mentorship (more focused attention).

williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
Let's do it, we'll be online to help in the discord https://discord.com/invite/sweep-ai. It'll also help if you have GitHub actions to run the tests, do you have it setup?
williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
It's necessary for sure, but we want to let junior devs choose to do the more interesting work.

We're also trying to make it easy to use Sweep. One outcome is an entirely simulated teammate, which is part of what we're doing with allowing you to review Sweep's PR

williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
Yep, small shops definitely benefit. Fewer people means each person already knows more of the codebase. For a 3 person team they might know >50% each, while for a 10 person team they might only know 10%.

The previous belief here would be the 10 person gets more work done, but that will change as AI developers like Sweep become more popular. There are a lot of additional benefits for small teams, like fewer meetings + faster decisions.

williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
+1, The PRs we made 2 months ago were really bad. That's also been the biggest barrier to getting them merged.

Definitely check out what we've been able to merge now though. The ceiling for tools like Sweep is incredibly high.

williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
Sweep is targeted towards senior devs that can do two things. 1. review code quickly 2. articulate requirements well

Also, here's another example of "grunt work". Sweep added a banner to our landing page, and I didn't touch my IDE at all. https://github.com/sweepai/landing-page/pull/226

williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
Yep, these bugs can be trivial but that initial context switch, creating a branch, etc tends to drain your energy.

Sweep can do this right now, you just have to label it yourself. We're doing this right now so you don't get flooded with PRs if you have a lot of open issues.

williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
For sure, I completely agree. Reviewing code can be really annoying, especially if it's not well written/broken. We realized this last month, so we've moved closer to providing tested pull requests.

Also as a dev, writing code is energizing and I love spending my day building a new feature. But when you get into maintenance mode, it's not that fun anymore. There's a good amount of code in the intersection of "easy to review" + "annoying to write", so Sweep is aiming to address that first.

Overall, it's not so much about not writing any more code and more about writing more interesting code. Similarly for junior devs, even in the space of "grunt work", there's more and less interesting options.

williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
Thats a good point, I really dislike when Sweep fails. That's why we're so focused on PR validation like self-review and GitHub actions, which brings it even closer to a junior dev. We wrote another blog on it here: https://docs.sweep.dev/blogs/giving-dev-tools

There's still a long way to go on automated testing, building, and running code, but I don't see any reason it's not possible!

williamzeng0··on Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
Thanks! We have a couple more demos at https://www.youtube.com/channel/UCUmi0YoUNHiITnYUrm5tnLQ, warning the audio is not the best :)
williamzeng0··on Copy, Paste, Repeat: My Frustrations Coding with ChatGPT
Completely agree, coding with chatGPT is hard. I've been using this bot I wrote to code. You describe what you want in your github repo and it just writes the code and pull request for you. https://github.com/sweepai/sweep.
williamzeng0··on AI junior developer looking for a job
Hahaha as evaluated by Sweep
williamzeng0··on Ask HN: How would you go about coding a bot?
Do you mean coding bot? We did this at Sweep AI, happy to share our repo: https://github.com/sweepai/sweep#getting-started-
williamzeng0··on Show HN: Sweep – AI-Powered Junior Developer for Tech Debt
Good point! It works well at documentation changes.

We currently have it check the github actions, but this is definitely not enough. Thanks for bringing this up, we'll continue improving how Sweep tests code.

Currently we only support GPT4. We're really excited for open source, but GPT4 is the best at code generation.

← PreviousPage 3 of 3