413 karma · joined May 15, 2017
Clark code now working on better motion animation system of itself as I am typing it :-) Check it out - www.clarkchat.com/clark-code - if anything, another tool in the toolbox when codex limits are not reset by Tibo!
Results + new binary https://github.com/clark-labs-inc/clark-browser
it kind of worked. the magic was the smallest UI around it:
- timeline of dials + retries
- "call me back" flags
- when it tried, who picked up
- short summaries with links to the raw transcript
once i could see the behavior, it stopped feeling spooky and started feeling useful.
so yeah, copilots are cool, but i want HUDs: quiet most of the time, glanceable, easy to interrupt, receipts for every action.
Don't just accept promises. Ask for the 409A valuation, liquidation preferences, and pay bands. If a company won’t provide transparency, that’s your signal.
Equity is a lottery ticket. Salary is money in the bank.
What is the real-world use case where GPT-4.5? Anyone actually seeing a game-changing difference? Please share your prompts!
I can say that R1 is on par with O1. But not as deep and capable as O1-pro. R1 is also a lot more useful than Sonnete. I actually haven't used Sonnete in awhile.
R1 is also comparable to the Gemini Flash Thinking 2.0 model, but in coding I feel like R1 gives me code that works without too much tweaking.
I often give entire open-source project's codebase (or big part of code) to all of them and ask the same question - like add a plugin, or fix xyz, etc. O1-pro is still a clear and expensive winner. But if I were to choose the second best, I would say R1.
Gemini refuses to answer or perform even on the simple prompts.
I hope Google team can make it better, but at the moment, for my light coding and text analysis use cases it is not worth $19.99
Simple things done repeatedly can feel magical at Google's scale - like recalculating signals across the entire internet to improve Search. But much improvement comes from manual analysis and labeling data.
Surprising problems get tackled if the right teams are interested - like developing a math image recognition API from scratch in 6 months. But most products face many hurdles to launch.
There is an ever-shifting web of goals and efforts. Politics and frequent re-orgs can derail projects. Smart people argue rather than align.
Technical debt is real, but so is process debt. Layers of reviews and requirements accumulate over time. Top-heavy teams with lots of senior people can cause gridlock. More doers than thinkers are needed.
To drive something big, you must relentlessly sell the vision and get the right leaders on board.
Many acquisitions fail. The Socratic founders left, and some goals weren't achieved, but parts of the product grew significantly.
Overall, amazing things are possible at Google if you navigate politics, rally the right support, and play the long game. But it's challenging due to complex processes, shifting priorities, and ingrained ways of doing things.
The article explores optimizing LoRA hyperparameter settings for finetuning large language models. The goal is to maximize performance while minimizing memory usage and training time.
The base model used is Llama-2 7B. Experiments compare default LoRA, QLoRA (4-bit quantized), AdamW, SGD optimizers, and different choices for rank r and alpha hyperparameters.
Key findings:
QLoRA provides substantial memory savings (6GB less than default LoRA) at the cost of slower training. Performance impact is minor.
AdamW vs SGD makes little difference in memory or performance.
Increasing training iterations from 50k to 100k hurts performance, likely because the Alpaca dataset lacks diversity.
Tuning rank r and alpha is most impactful. Good rule of thumb is to set alpha=2*r. Best model uses r=256, alpha=512. Improves over base model on most tasks, except arithmetic.
The optimized LoRA model was submitted to the NeurIPS efficiency challenge and showed improvements on several benchmarks compared to the base Llama-2 model.
Takeaways are practical tips for tuning LoRA hyperparameters and trading off memory, compute, and model performance.
please, issue corrections in the comments if you have a second