The change will be slow, not sudden. Slow and devastating.
66 karma · joined October 24, 2018
This was part of testing out how well a tool of mine worked (github.com/jsuppe/loom), which aims to be used to extracts requirements, specs, creates tests. At first I had no intention of using it for code generation but then tried it out with some early success. I tried splitting the work by using the tool with different frontier models, and then providing work to a local ollama instance running one of several models. Not all local models had the same outcome, not all coding languages had the same outcome. I also found in this experiment, when nailing down the coding tasks I wanted to set up positive and negative scenarios- which is where I found setting guardrails can sometimes backfire with inversion- this essentially elaborates on previous work by Khan 2025 (https://arxiv.org/abs/2510.22251); the most interesting finding to me was that if you give guardrails with a rationale, it reduces compliance and may cause the inversion
For coding tasks I found that the improvement was not only ability to use a lower cost model for these broken down tasks, but wall clock time was improved over using frontier model alone, with equivalent outcomes.
However, this is the state of the art today. In the future, the training set will be based on the prompt-result output and refinement process, leading me to believe that next generation tools will be much better at prompting the user to provide the details. I've already seen this enhancement in gpt4 recently, I think this is a common and interesting use case.
Overall, these tools will become more and more advanced and useful for developers. Now is a great time to become proficient and start learning how the future will require you to adapt your workflow in order to best leverage chatGPT for development.
This response was written by a human.
Ultimately, this is all you really need to get a job- get good at these problems. Being able to solve them and explain them to an interviewer. Even if you never plan to get a job, it will force you to learn all of the basics that you should at least know about if you plan a career in software development.
As examples, here are some notes I received recently from Amazon and Meta that they send after your initial phone screen with a recruiter:
Amazon tech screen: ------
Technical Prep:
Our technical questions will test your ability to solve a technical problem. If you are rusty, you should study up on:
Trees (binary/red black trees/N-ary trees, tree symetric (inverse image), tree traversal, sorting trees, prefix tree, radix tree) hash maps, stacks/queues/heap/priority que link lists/reverse link lists (space vs time) recursion/recursive strings binary search Arrays - 2d/3d graphs/graph traversal Big O notation
Meta tech screen: -----------------
Tips/Things to Know: Your initial interview will last approximately 45 minutes. Most people study 1-2 weeks and practice on a coding competition website beforehand (Ex. Medium difficulty problems on a site like Leetcode or HackerRank). Discuss initial ideas and solutions with your interviewer, which will help you to clarify any ambiguity in the problems. Take hints from your interviewer to showcase your thought process and problem-solving ability. Generally, avoid solutions with lots of edge cases or huge if/else if/else blocks. Deciding between iteration and recursion is always an important step. Talk about different algorithms and algorithmic techniques (sorting, divide-and-conquer, dynamic programming/memorization, recursion). Your interviewer may hint that you’re heading in the wrong direction if you move towards one that’s not optimal. Think about data structures, particularly the ones used most often (Array, Stack/Queue, Hashset/Hashmap/Hashtable/Dictionary, Tree/Binary Tree, Heap, Graph, Bloom Filter, etc.) Don’t worry about rote memorization such as runtimes or API/native calls. It's good to know how to figure out approximate runtimes on the fly but the code you write is more important (call out optimizations at the end). You will be asked about o Memory constraints on the complexity of the algorithm you are writing (and its running time - O(N^2) to O(N) etc. Please prepare any questions you have before the interview and also be prepared to answer why you are interested in Facebook.