cashsterling

I think David's expressed concerns are valid... there should be more safety engineering around LLM to avoid data loss and other accidental issues caused by LLM models.

And, I find the timing of "all of this" very interesting:

- we have a coordinated marketing campaign around discussions of AI safety. I am not saying David is part of that campaign but it would not surprise me. OpenAI and Anthropic are talking about safety and the need to slow down. People are "resigning". Techno-influencers are also being offered money to talk about AI safety, etc.... and some of them are taking that money and others are not.

- Cheaper models are keeping pace with Frontier models, albeit with a 3-6 month lag... which indicates that frontier labs have no moat unless one is created for them by law or some other market force.

- we have a ballooning debt crisis in LLM model hyperscale spending... revenue from serving LLM models is lagging the debt payments which are coming due by a wide margin. If something dramatic doesn't occur in the near- to mid-term, the financial bottom is going to fall out of this thing and the AI industry is going to suffer a major collapse of some sort.

LeCun is a lot more than a "Grey Beard"... lol, that is a significant understatement of his contributions to the field and position in the field.

Among other things, LeCun is one of the senior people industry who has a deep understanding of the mathematics and analysis underlying neural networks and "neural network like" approaches to machine learning. I think he understands a lot more about neural networks than most researchers today.

I also don't think our current LLM models are AGI and think the LLM approach is, mathematically, incapable of producing an AGI. At the end of the day, the architecture is still a streaming token plinko machine with a lot of guide-rails to achieve good behavior.

I do think it is very impressive how far hundreds of billions of dollars have been able to take LLMs in terms of usefulness (I use LLMs everyday in my job). AGI or not, current LLM models are a pretty incredible achievement and will provide lasting benefit even after the economic implosion of the AI industry occurs (which I think is imminent).

A lot of shopify's engineering probably work on the back-end business logic for payment processing, sales tax, etc. Shopify has to comply with financial norms, rule and laws in so many different jurisdictions.

I know someone who works for Gusto (HR and payroll SaaS for small businesses)... most of their code-base is also business logic and whatnot for all the jurisdictions they support.

I too doubted, from the beginning, that neural networks will be the basis of AGI. As impressive and useful as LLM's are, they are still a long, long way from AGI.

It is not a one dimensional take... it is a stress test of qubit gate fidelity [across all qubits involved in the circuit], state prep and measurement , lifetime (coherence), memory errors, etc.

Now I agree that there are other great stress tests of quantum computer systems... but most of the industry agreed that quantum volume was a great metric several years ago. As many companies systems have been unable to hit decent QV, companies have pivoted away from QV to other metrics... many of them are half baloney.

The better analogy here, in my opinion, is until someone builds a car that actually drives forward...

A lot of these quantum computing papers are just loud engine rev'ing.

But you are right... "zero attention" is hyperbole... I shouldn't have said "I pay zero attention"... I should have said, "I am underwhelmed and don't give a lot of credence to systems until they demonstrate ability to perform high fidelity calculations of reasonable complexity and depth."

I do think superconducting qubits approaches will continue to improve in fidelity and ability... there are just too many brilliant people working on these challenges to count them out.

Until a company can demonstrate a quantum volume of even just 2^16... their computer is just about worthless for any sort of half-real quantum computing.

I pay zero attention to any technical information and marketing speak coming from a quantum computing company until they demonstrate decent quantum volume.

Most companies computers can't even hit 2^16 so they are figuring out means of distracting the market from the poor fidelity of their systems.

Modern high-end fabs have extremely expensive equipment and are highly automated... like they are so automated that people don't actually handle wafers... it is almost all robotic.

Thus, salaries and cost of services do not factor in as heavily as you might think to fab economics.

Data suggests that TSMC's per wafer costs in Arizona are 10-30% higher than Taiwan and that Arizona fab is relatively new. It's economics will probably improve over time, narrowing the margin to 5-15%.

Looking towards the future, power costs and other global supply chain factors could very easily make TSMC's Arizona fab less expensive and more reliable to operate over time. For one, the US is completely energy independent... Taiwan is not.

Atlantic Quantum has some brilliant people and I'm sure they have some great ideas.

That said, I dislike the quantum computing craziness we're in where big claims are made without proof or data.

"We have the fastest clock speeds, lowest error rates, and most scalable architecture... no data, just take our word for it."

It will be interesting to see how they do in the DARPA QBI program.

I always try to communicate a strong sense of hope to my teenage kids. Many nations have an inverted population demographic, including the US (especially without immigration).

I think the US is "as little as 10 years" away from a significant skilled labor shortage.

I think Ada is a great language and it is completely possible that Ada will experience a resurgence in coming years.. especially as LLM's are used more and more to generate software. ADA/Spark can provide very robust guide rails for correctness (so can Rust) of 'AI' generated code.

Yeah... my intro CS class was in C and Ada95 (I'm not a CS guy btw, just took the class). I actually preferred Ada over C... but continued to program in C for other classes because of compiler availability; I had to do all my Ada programming on Sparc workstations at school.

I personally think that AdaCore, and friends, missed an opportunity in the early 2000's to fully embrace open source... they left a big gap which Rust has filled nicely.

I still think Ada is a great programming language. When a question comes up along the lines of: "what's the best programming language nobody's heard of?", or "what's the best programming language that is under used?" Ada is usually my first answer. I think other good answers include F#, <insert LISP flavor>, Odin, Prolog, Haxe, and Futhark (if you're into that sort of thing).

I really hope the best for Oxide and applaud their compensation model.

I applied to one of their roles which required me to write about 10 pages of text to answer all their questions... which I think is a big ask but I did it because "why not".

They took over 3 months to get back to me, but at least they got back to me (with an apology and a polite "no").

100% agree with you!

I have worked US manufacturing and manufacturing R&D for most of my career: pharmaceutical, microelectronics, materials, aerospace, etc. The US is awesome at manufacturing when we want to be.

One problem is that "modern MBA/business philosophy" views manufacturing and manufacturing employees as a cost center and there is so much emphasis on maximizing gross margin to increase shareholder value.

So business leaders scrutinize the hell out of anything that increases the cost of their cost centers:

- employee training & development? hell with that.

- Increasing pay to retain good employees in manufacturing? Why? isn't everything mostly automated?

- manufacturing technology development? Not unless you can show a clear and massive net present value on the investment... and, then, the answer is still no for no good reason. I have pitched internal manufacturing development investments where we conservatively estimated ~50% internal rate of return and the projects still didn't get funded.

There is also a belief that outsourcing is easy and business people are often horrible at predicting and assessing the total cost of outsourcing. I have been on teams doing "insource vs. outsource" trade studies and the amount of costs and risks that MBA decision makers don't think about in these situations really surprised me initially... but now I'm use to it.

Anyhow... the US (and Europe for that matter) can absolutely increase manufacturing. It is not "difficult"... but it would be a slow process. I think it is important to differentiate between difficulty and speed.

I graduated in ChemE in southern California in 1999 when there was a major downturn in the job market. One or two big chemical engineering design firms closed their SoCal offices flooding the market with qualified chemE's, aerospace companies were consolidating, etc.

Some of my school colleagues got good jobs at refineries and whatnot... but they were the fortunate ones. It took me 12 months to land my first "I made it" engineering job with a good salary. In the interim, I worked hourly jobs making between 13-18 USD an hour.

Don't let the current job market deflate you. You are young, intelligent, and you have a degree from MIT... you are going to be fine.

My unpopular opinion, as someone who worked in vaccine development and biotech manufacturing for a lengthy portion of my career, is that:

- vaccination should be a personal choice... no one has the right to compel anyone to get injected with anything. Full stop.

AND

- we need to raise the level of scientific and medical rigor applied to vaccine development, manufacturing, vaccine administration, and ongoing monitoring / pharmacovigilance so that people will feel comfortable taking vaccines.

The fact is that different quality and risk/benefit rules are applied to vaccines versus other injected drugs and this is not okay (read up on the National Childhood Vaccine Injury Act passed on 1986, why it was passed, and how vaccines are tested for efficacy and safety versus other injected drugs).

Under normal circumstances, the COVID mRNA vaccines should have failed the clinical trial requirements for human safety. There were far too many adverse events in the clinical trials and many, many people were harmed by these mRNA vaccines (with quite a few deaths). The NIH & FDA largely swept this under the rug and refused to investigate or fund adequate investigation. Unfortunately, that entire episode has seriously harmed people's faith in the government and has negatively influenced people opinions on vaccine safety.

This is very disappointing because there are a lot of highly effective vaccines with excellent safety profiles when administered properly... vaccines almost everyone should receive. But the average citizen can't tell the difference and I don't blame them if they don't trust the government to protect them at the moment.

I work in system engineering at a quantum computing company... when I read announcements and new papers on quantum computing, qubit technologies or implementations of quantum computers, I loosely apply these thresholds.

threshold one: The authors/company report data on qubit coherence times, single-qubit and two-qubit gate error, state prep and measurement error, etc. The more rigorous data, the better. If they report data from different qubits on the same device or across multiple devices, even better. But without this sort of data, any assertions or forward looking statements about the utility of a device or quantum computing approach are "pure marketing". To me... the Microsoft paper and announcement do not meet threshold one.

Threshold two is performing some sort of useful bench-marking calculation that requires repeated use of multiple qubits in concert. Quantum volume calculation is one such benchmark. It much easier to get great qubit results from a small test device (a hero device) than a larger system. It is tough to make a blanket statement about all QC technologies but system noise levels and calculation error failure modes scale with # of qubits... so be able to achieve high fidelity two-qubits gates repeatedly in a deep ciruit using 20-50 qubits is much, much more difficult and impressive than demonstrations with 1-4 qubits. To me, number of qubits is almost irrelevant if those qubits are not useful together... example: if a company reports 100+ qubits on a device but can't pass a quantum volume 12 or 16 calculation, then I will reserve judgement about the utility of that QC approach. There is engineering development value to scaling number of qubits (like figuring out how to orchestrate massively parallel qubit control at scale) while also working on improving qubits performance metrics... but these two development streams have to converge: lot of qubits AND high fidelity. Demonstrating high fidelity at low qubit count and high qubit counts separately doesn't mean that high fidelity, high qubit counts will be achievable.

I'd consider Advanced Engineering Mathematics by Zill, 6th Edition which is available used for as little as 25 USD. There is also a print solution manual for this book which is great for self study.

At a startup I worked at, we had our adaptive control pilot software flying our drone designs in XPlane for testing and demo purposes.

Rust has functionality in the std lib for saturating arithmetic (and other kinds of arithmetic via crates) so that calculations saturate rather than overflow, etc. https://doc.rust-lang.org/nightly/std/?search=saturating

If your goal is literally to get good at math through Calculus, Differential Equations and Linear Algebra... I feel like the best approach is working through good books (reading, taking notes, and doing problems out of the book then checking answers/solutions) and hitting Youtube and Khan Academy for help with certain topics.

Fortunately, there are a lot of great older-edition, cheap used books on Pre-Algebra, Geometry, College Algebra, Pre-Calc, Calculus, DiffEq and Linear Algebra, etc.

It will take some time to master all the math you want... but if you structure your studying so that are you actual enjoy studying (i.e. enjoy the process), you will 100% get there.

I'd be happy to list out of some book recommendations.

I will echo the sentiment of others that you don't need to be a math expert to use machine learning libraries effectively in many cases. The problem is, without the math expertise you won't always be able to identify the cases when you are using the wrong approach, etc. And you'll have a harder time applying cutting edge ML to new problems.

Going away from physical book-based learning was possibly well intentioned (but I have my doubts)... but it was really dumb.

There are clear studies that show reading a physical book (versus a screen) and using and physical pen or pencil on a piece of paper, versus typing or drawing on a screen, leads to higher comprehension and retention of information... and thus much better overall learning outcomes. This doesn't even consider the fact that youtube, discord, and a bunch of other apps are a swipe away on an iPad.

A common solution to the "carrying books around problem" used to be there was the copy you were issued (and mostly stayed at home) and there was a shared classroom copy.

Carrying around 2-3 books plus a binder is not a big deal (and is not a 30lb backpack... more like 10-15 lbs)... we act like this is some sort of massive hardship yet so many of us did this for over a decade of our childhood with no ill effect.

This is absolutely ridiculous... shame on UW. Their quid pro quo offer is ethically very wrong.

My wife works at a major public university and fights with antiquated software and business systems all day long. The amount of IT system bloat and 'village-idiot dumb' processes for managing course offerings, course catalogs, class schedules, etc. is pretty bad at a lot of universities.

Same... I didn't have to take Linear Algebra in ChemE undergrad. DiffEq had a little bit of LA... and ChemE had few classes where bits of pieces of LA where introduced and applied.

Graduate school definitely made up for lost time... LA was very front and center in the applied math courses.

The real solution to this issue is municipal broadband.

If there is a municipal option for high speed internet, people who don't want deal with companies' monitoring and throttling shenanigans have a viable option from a community-focused provider. This provides price and service competition for the Tel-Co cartels who collude to fix prices and pull all kinds of other nonsense (like taking billions of dollars in federal funds and then deliver next to nothing... then lobby their way out of accountability).

T

All great empires fall, often due an accumulated effect of dumb policy decisions. Ergo, Amazon's grip on market will eventually fail by dumb policy decisions... and this is almost certainly one of them.

The work-life balance & climate in some areas of the US (and other areas of the world) does make it hard to exercise... but that's only part of the story.

Eating healthier requires very little extra time: perhaps 2-3 hours a week for shopping and meal prep.

- making a salad take 5 minutes.

- baking 10 meals worth of chicken takes 30-45 minutes, with about 5-10 minutes of actual prep time.

- baking a few meals worth of salmon, or some other fish is about the same as chicken.

- cutting up an apple takes 1-2 minutes. Bananas, peaches are ready to go, almost no prep required.

- steaming fresh broccoli takes 3-5 min of prep... and about 15-20 minutes total.

- Equally important is making small decisions to avoid or limit less healthy foods, which requires no time at all and often saves us money [that we can spend on healthier foods instead].

Exercising: 30-60 minutes a day, most days, is ideal but anything is better than nothing. A morning and evening 20 minute walk is great place to start (and can be so relaxing). If a person spends time on Hacker News, they could instead allocate that time to exercise. There are free 15 minutes yoga videos on Youtube. A set of push ups requires 60 seconds. Sometimes I do body weight squats at my desk at work (especially during Teams calls, camera off of course).

Sleep: get at least 7 hours of sleep a night... ideally starting to bed before 11pm. more sleep gives us more energy to put towards being awesome during the day and more energy for exercise and doing the extra work to eat better.

I think almost everyone can set aside 6-10 hours in their week towards building their health. It is a matter of priorities... a person's health should be one of their top priorities.

Fitness is in reach for almost everyone on Hacker News. There are folks who have serious health issues or disabilities that preclude exercise... but the rest of us [the majority?] can walk more, do body weight squats, some form of incline push ups, and so forth and build larger successes on top of smaller ones. We can start by taking a morning and evening walk.

We all can make healthier decision about what food we put into our bodies (a lot could be done to make our food supply healthier, as well). I find I have to be more strict about eating right as I get older to maintain my health.

I feel like folks saying 'fitness' isn't in reach [for most/many of us] is a cop out. This is anecdotal, but i know people who have lost 100+ lbs in their 40's and 50's and completely turned their life around; I know people who have reversed Type-2 diabetes through diet and exercise. These people have busy lives with a lot of responsibility... some of these people live in city areas where access to gyms or outdoor spaces for walking, etc. is not convenient.

I have a few doctors and other medical professionals in my family... knowing what I know, I am 'almost terrified' of becoming metabolically unhealthy and overweight. Many of the maladies that afflict us are metabolic disorders (in whole or in part) due to the long term effects of poor diet, lack of exercise & quality sleep (high stress doesn't help either). Poor metabolic health tremendously increases risk of almost every disease and malady: diabetes, heart disease, heart attack, dementia & Alzhiemer's, many cancers, stroke, depression and other mental illnesses, sleep apnea, susceptibility to infectious disease, etc. Poor metabolic health also indirectly leads to higher likelihood of issues causing chronic pain: e.g., herniated discs in the spine, ankle/knee/hip problems, migraine headaches.

I'm glad Ozempic exists but it is not a cure... it is an aid towards building a healthier life. Use Ozempic as a tool, by all means!!, to start your journey and get yourself into a better situation physically. But please use that good start to pursue the 'best/healthy you'... do it for yourself.

If you are on a weight loss drug like Ozempic, make sure you are doing resistance training. Ozempic reduces your appetite and help you go into a calorie deficit. Bone mass, muscle mass, ligament and tendon mass can all suffer on a calorie deficit unless your body is get regular stimulus that 'muscle, bone, and connective tissue strength are still important.'

Also, changing your diet while on Ozempic to (1) cut down carbohydrate intake and (2) include more healthy protein (meat, eggs, fish, etc.), fats (nuts, olive oil, etc.) and fruits/veggie will supercharge your weight loss and your overall health gains.

I'd recommend getting the 9th or 10th edition of Introduction to Operations Research by Hillier and Lieberman. 9th: https://www.amazon.com/dp/0077298349 You can search for the 10th edition. Both are available used for less than 50 USD in good condition. The book covers a lot more than linear programming. A solution manual for both editions can be found on the internet.

A good "free-pdf" optimization book, to support the above is, Algorithms for Optimization by Kochenderfer & Wheeler ( https://algorithmsbook.com/optimization/ ). It has a chapter on constrained linear optimization with Julia code and is a good secondary resource. Kochenderfer, Wheeler, and colleagues also have two other free optimization books that are a little more advanced. It is exceptionally cool that they make the high quality PDF freely available; more authors in the technical space are making their books freely available as pdf and I applaud them for it.