Clearly the performance of the "instruct" LLM is due to some odd bug or other issue. I do not believe that it is fundamentally better at chess than the others, even if it was specifically trained on far more chess data, which is unlikely. Lack of correlation is also not causation.
Perhaps the tokenizer is the entire problem? I wonder how the chat-based LLMs would perform if you explained the move with text, rather than chess notation? I can easily imagine that the "instruct" LLM uses a specialized tokenizer designed to process instructions, similar in many ways to chess notation.
We could easily support government grants to Open Source developers and companies without needing to increase taxes. It should already be part of the NSF grant process that funds a large number of universities, who's research is also given away for public use (most of the time). Sure, you'd have to fill out lots of forms and be at the mercy of the bureaucratic grant submission system, but that process already has mechanisms for managing IP and leaving ownership with the grantee. Despite its frustrations, it is already a much healthier way of developing IP compared to the VC-supported route.
Where will the training data for coding come from now that Stack Overflow has effectively been replaced? Will the LLMs share fixes for future problems? As the world moves forward, and the amount of non-LLM generated data decreases, will LLMs actually revert their advancements and become effectively like addled brains, longing for the "good old times"?
Even with the perfect coding Oracle, you need to ask the right questions to get good answers. The LLM will cheerfully give you bad answers if you ask the wrong question. The implication is thet you still need to learn the problem space yourself, often via the LLM, in order to ask good questions. Junior developers will ask bad questions, get dangerous code, and love it. Good programmers will learn stuff much quicker. Asking good questions is not easy. As LLMs get better, we will need to ask better and better questions in order to write more and more complex apps. This happened when we got open source libraries via the Internet (we all used to suffer alone with our immediate coworkers in the before times).
I should also add asyncpg-trek for migrations to that stack above. Took me awhile to find a simple Postgres migration tool that just used asyncpg, without ORM, and simple SQL files.
The vast majority of the security "industry" is about useless compliance, rather than actual security. The chimps have put their fears into large enterprise compliance documents. This teaches the junior security people at enterprise companies that these useless fears are necessary, and they pass them along to their friends. Why? Not just because of chimps and fear, but also $$. There is a ton of money to be made off of silly chimps.
Love it. My new backend app stack is Postgres/FastAPI/PGQueuer. No need for Redis. Perfect for backends that need a fleet of workers to generate, e.g. embeddings.
More generally, any work that a developer does outside of coding will improve their coding. If they learn a bit more about the business side of the company, they will have better conversations with the business teams because they speak (some of) the same language, and the developer will be better at prioritizing their tasks because they consider business concerns in addition to technical concerns. The same for Design, hiring, or testing. The most important task at a startup for both business and technical people is actually talking to customers, which should drive both Design, Business and Technical decisions. Most developers avoid talking to their customers, or are prevented from talking to them by other folks at the startup. The author alludes to this approach when talking about Design. You can and should go deeper, creating customer personas and workflows (like one of those Great Designers) to understand your customers better. The more time you spend talking to and watching your customers use your tools, the better!
I don't use the autocomplete very often. I do use the chat window to do rubber ducking about potential changes. I ask it about features of the libraries I'm using, the built-in features of the language, ways to restructure the code, various suggested improvements to the algorithms and so forth. After a short discussion about various options, I'll ask you to generate code which I use either as a reference or verbatim. For me rubber ducking and discussions are much more valuable than generating little bits of code using autocomplete.
The next interaction mode I want is to have the AI as a pair programming partner, making comments as I type code, including suggestions and pointing out potential errors, and allowing me to ask questions. That is, similar to the rubber ducking, except that I'd like it to interject comments at the right moments automatically.
This fix allows me to run the OpenAI CLIP model in less than a second per image in the browser, whereas before it was taking 30+ seconds, making interactivity possible.
I've rarely met anyone who can get more than four useful hours a day. Sure, there's lots more time when you are thinking about work, writing code in your head, prioritizing, and planning. That stuff can and does happen when you sit at your desk, or while you are biking, running, doing laundry, or watching tv.
Exactly. I didn't start coding until the '80s, but even at that point I was writing a lot of assembly code. I don't do that anymore. Each decade that I've coded I look back and see that I've moved up to higher and higher levels of complexity over the years. I was skeptical of AI coding, and I still am. But I've seen that it is capable of writing single line expressions quite well, and if the function is well scoped enough, it can even write full functions occasionally. My favorite use of AI is as a rubber duck, for me to bounce my ideas off of before I actually write any code. I expect it's capabilities will get better and better over time as everything does. Having The AI code these smaller bits allows me to focus more on gluing the bits together, just like I've done for the last 10 years with all the open source libraries that are available these days. I don't mind that I forgotten how to use the Lisa assembler for my Apple ][. I'm happy to keep forgetting stuff if I keep working on more and more complicated projects.
Who uses previous titles to determine capability or compensation? Just look at their job history, call references, and make good hiring decisions. Perhaps the issue is that people are bad at hiring?
When I'm debugging, I imagine that there is a huge cartoon style 2000lb weight supported by a mish mash of random toothpicks and I have to stick my arm waaay underneath to remove a bit from the middle and replace it with a new, clean and sturdy toothpick.
I'd also include the status of the company. What a startup needs from tests is very different from what an enterprise company needs. If you're searching for product market fit, you need to be able to change things quickly. If you're trying to support a widely used service, you need better test coverage.
That's funny, since I wouldn't code at a place that mandates unit tests. Sure, they have a very minor role, in very specific cases, but I'd say 90% of projects can get 90% of the benefits by writing only integration tests with 90% coverage. If you'd like a more in-depth discussion of why integration testing is better: https://kentcdodds.com/blog/write-tests
Why just unit tests? Integration tests seem much more valuable as documentation of what the users will do in the app. Unit tests have limited benefits overall, and add a bunch of support time, slowing down development. If you have good (90%+) coverage just from integration tests, you are likely doing 90%+ coverage of the unit tests at the same time, without the extra effort or support burden. You can use the same reasoning to describe the benefits for understanding the code, you get a clear understanding of the important usage cases, plus you get the unit-level "documentation" for free.
The trick is finding a problem that increases revenue or decreases costs after providing structure to the data. Sure, it would be great to bring structure to assets, but you can't just provide search or labeling. You have to figure out how providing the structure actually brings value to those companies. You'd hope they would do that for you, but you need to figure it all out, at least for one set of customers who will pay you, before you build the MVP. The details of how it benefits the company has a profound effect on the design of the MVP, including how to access the assets and how to expose the structure in the UX.
Back when I worked in an office and had to do deep thinking without interruptions, I'd string up a bit of yellow police-like tape that I stole from the SGI offices that were closed during Shoreline concerts that said "SILICON GRAPHICS". Worked perfectly.
The topical matching is horrible. I tried two very different topical areas and got the same candidates in the same order. Likely there are just not enough candidates, or their demo is fake. Hard to make an app like this because it needs a big user base, and the first people to try a service like this are the most desperate. Plus there are dozens of companies trying to make co-founder finders. It is one of the most common ideas presented in startup communities, which also undermines the ability to get lots of users. Finally, apps and companies that prey on inexperienced founders are a Bad Thing (tm), since they play into the false belief that anyone can start a company and get rich.
A good senior developer should be able to go out to lunch and know if the candidate has the skills and the right cultural match by the end. There's no need for coding tests or whiteboarding problems. I've seen some of the best developers I've worked with in 40 years of coding get turned away by Google and Apple because of their terrible interview processes. The only thing better than going out to lunch is working with the person for a couple of weeks. Ideally this happens on a paid contract basis. Of course our culture makes it more challenging to switch jobs if you want to work together first, but really no other interview process has ever provided good data.
The best things about WFH are hard to quantify, like the best things in life. You are more relaxed, with lower stress, more control of your time, less probability of auto accidents commuting, ... These are rarely mentioned because they're hard to quantify, but as someone who's been working from home for multiple decades, they are far more important for my happiness and productivity.
When you go to a movie in towns like LA, Seattle or the Bay Area, you can always tell which people work in the industry because they are the only people who stay to watch the credits. Normal humans all leave when the credits start to roll, with a tiny fraction staying in the hope they include a teaser or surprise bit of story in the middle of the credits. Since the producers of the Alien revival had just spent all their money on the interview, they want to make sure people see it, so they put it before the film.
> The reality of the world is that product holds the money and software development is seen as a cost center to be minimized towards zero.
This is only true for mediocre companies. Software development is really a force multiplier, not a cost center. A good optimization for a proprietary app saves time for everyone at the company that uses the app, or for your customers if the app is a product. Also important is that the developers can dramatically alter the cost structure both for the current set of features, as well as the future support and additional features. Product faces outward, helping prioritize features for customers. Development faces inward, minimizing tech debt and future work for the company. Both halves are needed to design and build a good product in a reasonable amount of time.