Large Language Models (LLMs) and ChatGPT have taken the world by storm in the last few months. While GPT-3 and other open-sourced LLMs are great at generic tasks (summarize an email), they fail at specialized tasks (answer a customer support ticket).
This is expected: LLMs are affected by the biases of their training data and channel this bias into downstream applications, hurting their ability to be precise in business applications.
What is data-centric fine-tuning?
Data-centric fine-tuning is the process of training an additional layer on top of a pre-trained LLM (e.g. davinci-003) with additional data (prompts and completions) to steer the model towards achieving consistent behavior in a specific task (e.g. extracting data from documents, classifying conversations, or creating content with a background).
Some of the benefits of fine-tuning with additional data include: - Higher accuracy and quality than doing prompt engineering. - Requires fewer tokens, saving you money. - Faster and lower latency. - Ability to use smaller and cheaper models, like BERT or ada, that are better suited to your task.
Yet, at least 80% of the time invested in fine-tuning models is spent working to get a clean, labeled, and well-balanced dataset. A valuable fine-tuned dataset can require upwards of 10,000 datapoints and a consistent stream of new data to maintain good performance.
Stack’s solution: the CI/CD for Large Language Models.
Here's where Stack comes in. We are a team that has designed, deployed, and maintained machine learning models for over a decade, as part of our PhDs at MIT and previous work experience at SpaceX, Facebook, and NASA JPL.
Consistently, we have noticed that iterating on the model design or engineering a prompt can only take a model from 60% to 80% accuracy. However, having a large and high-quality dataset can take a model from 60% to 99% performance.
Unfortunately, while fine-tuning is straightforward conceptually, curating the data is the difficult part. In our experience, 80% of the time invested in fine-tuning models is spent working to get a clean, labeled, and well-balanced dataset. We also found ourselves building the same internal tools over and over, and using multiple open tabs with different tools available out there. This is not sustainable or scalable.
With Stack AI, we want to make fine-tuning LLMs a painless experience for our customers.
Our software helps you automate data labeling, quickly filter data anomalies, and export different prompts to fine-tune your models to any application.
From our experience, having a streamlined process to label and manage your data can save thousands of hours when building a fine-tuned model.
Reach out if you'd like to learn more about data-centric fine-tuning!