Ask HN: How to Fine Tune LLMs?

2 pointskrishadi3 comments

I wanted to play around with fine tuning LLMs on my Mac. Any recommendations on how (or) where to start? Is there a blog you'd suggest?

These instructions got me started

with the caveat that the default training rate is too high and you get better results with 2e-5

Great! Thanks...

There are a number of ways, with increasing methods and capabilities. It's going to depend on what you are trying to do and the constraints you have

Google put out a paper last week that talks about some of this and proposed a new method that outperforms previous: https://news.ycombinator.com/item?id=35810663