PlayDate Fulfillment Delayed | Hacker News Reader

PlayDate Fulfillment Delayed

lists.play.date

261 pointsgen220206 comments

The lead times for the STM microcontrollers have gone way beyond unreasonable. "730 days" I think is a polite way of saying "we may never supply you with this chip".

I may be way off here but my observation is that chips that are very configurable are the hardest to purchase. For example, the STM32 series have something like ~3000 SKUs ( https://www.digikey.com/en/products/filter/embedded-microcon... ).

Normally, this allows every designer to pick the most suitable part and have the lowest total BoM cost. However, in today's challenging supply/demand environment, the total number of unique parts that must be manufactured must be causing a huge problem.

STM32H7 is the upgrade path, but it doesn't seem to be any more available.

I bought one from their site.

The part number explosion may not be as bad as you think: ST very likely does some relabeling/fuse programming of the same die to make multiple part numbers. As the most classic example, the STM32F103C8 promises 64KB of flash, and the F103CB (note CB vs C8) has 128KB. But it turns out that some of the 64KB actually have 128KB [1], and they've just been fused to report the smaller part number.

[1]: https://mecrisp-stellaris-folkdoc.sourceforge.io/stm32f1xx-d...

True, and then every variant in every package version. But it does cause problems in the current climate as each of those variants do have to make it through the production pipeline (after die fab) which is also constrained at the moment. And of course ST is prioritising "high value customers" (like car manufacturers).

Oh for sure, it's not like fab capacity is the only bottleneck at this point. Seems like everything is the bottleneck.

Car companies are just few percents of MCU market. A car may have a lot of MCUs, but that's probably less than 30 per economy car. The entire car industry is a drop in the ocean for The Big Semi.

On other hand, I've seen very expensive STM32s being used in near semi-disposable goods.

Car companies are doomed because they required special "automotive grade" MCUs, which are usually just very long running die series with a lot of testing.

Being on old die users, they can't share the benefit of much larger mainstream MCU die batches. They are now in the end of the queue waiting for big batches of dies for mainstream MCUs to finish manufacturing.

IMHO, lots of auto parts uses automotive grade MCUs without any particular necessity. It's just because they were well moneyed, and they can. Some Mercedes car literally have one MCU per button, which just sits on CAN bus, and blinks an LED.

> On other hand, I've seen very expensive STM32s being used in near semi-disposable goods.

Like a single-use disposable electronic COVID test: https://www.youtube.com/watch?v=UvArprBmdFA (though this one uses an nRF52)

Contrary to the popular opinion (which can be traced back to one or two newspaper articles), it's not the car companies causing this. All sorts of material has evaporated from markets, in grades beyond automotive.

> I may be way off here but my observation is that chips that are very configurable are the hardest to purchase. For example, the STM32 series have something like ~3000 SKUs ( https://www.digikey.com/en/products/filter/embedded-microcon... ).

Digi-Key is great for prototyping and small-scale production runs, but once you get into mass manufacturing your CM (contract manufacturer) is buying these parts for you through more direct channels.

STM can produce small numbers of the various different SKUs and send them off to stocking distributors like Digi-Key to support small-scale prototyping and smaller production runs, but they don't aim to stock everything in high quantities everywhere.

Unfortunately, when supply chains get tight the smallest players suffer the most. The big manufacturers went out and locked up their supply contracts when this all started. Companies like STM don't mind because it's much easier to do big manufacturing runs for a few customers than to try cater to small orders all over the place.

Of course, they'd prefer to have the capacity to serve everyone and sell as many chips as possible, but obviously we're bottlenecked on the production side relative to all of the demand.

Can they not use an off the shelf board like Raspberry Pi?

Raspberry Pis, as are most consumer sbcs, are extremely unsuitable for an embedded, high reliability consumer device. For a whole bunch of reasons, from reliability, to form factor, to pricing, to technical concerns (such as IO).

Raspberry Pis have commercial and industrial use cases so they can have pretty good reliability. There’s an official “for industry” page [1], and there are PLC products made with the Pi [2]. I’m curious on why do you think the Pi (including the Pi Compute Module)’s form factor make it unsuitable for consumer devices that need computing power a SBC offers?

[1]: https://www.raspberrypi.com/for-industry/

[2]: https://www.industrialshields.com/industrial-plc-raspberry-p...

Generally, actual industry would use the pi compute module instead of the pi itself, and you plug the compute module into a carrier board with better power, storage, and IO options (i.e., rs485, 24V DC, eMMC instead of SD, etc)

One of a million examples: https://pipci.jeffgeerling.com/boards_cm/edatec-cm4-industri...

Having your power connector come loose on a vibrating machine or having your SD card die is a serious problem when you're trying to use a machine to make money reliably and consistently.

Anyway that's unrelated to GP really, raspberry pi sucks too much power for playdate, I think.

> Having your power connector come loose on a vibrating machine or having your SD card die is a serious problem when you're trying to use a machine to make money reliably and consistently.

Hah, I’d never thought of the power connector coming loose but I can see how it will be a real issue in an industrial environment, or on vehicles. I’ve also heard a lot of stories of the Pi’s SD card corrupting or dying, though I agree that actual products might use eMMC instead.

I think the Pi is better suited for less portable products like NAS, or TV boxes, although we already have popular inexpensive SoCs for those.

> commercial and industrial use cases

> unsuitable for consumer devices that need computing power a SBC offers

PLC is not "consumer". "for industry" devices often have service contracts and dedicated staff whose job it is to ensure reliability and spares, and vastly different requirements than a plug and play "consumer" device.

You've compared two very different use cases as if they were alike.

Consumer devices actually have much stricter reliability requirements than many industrial applications - if you look at margins, you'll see that any return is a huge dent. Versus industrial, where swapping out a $50~ sbc part once a year is still vastly cheaper than "high reliability" solutions which cost into the thousands. Maybe you lost half a day of work, but that's probably acceptable for a small shop running a lathe that saved $$$$ on upfront capex.

And, there's no way anyone's using raspberry pis for high availability, such as in manufacturing.

But back to those consumer requirements - there's a lot of software and hardware requirements that you get "for free" with more expensive, traditional solutions. For example,

- verified boot (pre-boot checksum/signing verification, and chain of trust)

- multiple boot partitions with fallback to prevent against bad updates

- image based updates and recovery

- out of band network recovery modes

- guaranteed flash failure rate and yield

- minimal form factor with reduced components for unneeded features

and a bunch of other small stuff that all shaves off failure rates by .5% here and there, and yet is completely worth it to anyone deploying consumer devices. Some of that you could probably implement yourself, but make a mistake versus using a tried and tested solution... well, that's more margins and profit lost.

By the way, in another comment you said "Pi is better suited for less portable products like NAS, or TV boxes". Well, yeah, 'cause it's basically a set-top box SOC that was put into a hobbyist board. ;) The Pi has a TON of "undocumented" features and modes around being a set top box. Broadcom will sell you basically the same GPU/CPU package with a lot of those features listed above.

"730 days" is, of course, exactly two non-leap years.

So it seems possible that "two years" is just an overflow in inventory management restocking projections.

I.e. "more than we can measure", "unknown but not soon", etc.

A little bit off. If you are interested: The link you have there includes several different ways of ordering the same part but packaged differently: Cut Tape, Reels, Tubes, Trays etc. Some of those have will different sizes of packaging (eg tray of 50 vs tray of 200). I'd divide that 3000 by 3-5.

Other filters will be same part but different temperature range; same MCU but different footprint (which would require a board re-spin but not normally the end of the world).

You'll also get SKUs where different peripherals and flash is available which is generally the same die, but not everything worked when tested at the fab so it will get binned appropriately.

There's other filters but I guess I am saying the total number of SKUs of die that are getting made is not anywhere near 3000.

You are right with the 730 day lead time. Actually anything beyond a year you can interpret that way.

Even when there is stock, the parts will be on allocation anyway. Normally the biggest existing customers get first dibs and everyone outside of that circle is given an absurd lead time. Sometimes you can push your way in if your order is big enough.

I feel for the PlayDate team. It's not a fun time when this happens. Even my usual rule of "make sure there are at least two alternatives if possible" is failing in this shortage.

We've even resorted to ordering dev kits and scavenging parts from there which is good fun :)

There has been a number of submission on HN discussing the chip shortage - and I’ve yet to see anyone discussing using second hand chips, scavenging from used electronics. Obviously as described in the post: redesign is always going to be expensive and cause delays if even possible. But I’m still curious as to why this would not be possible or at least create a much bigger immediate market for old chips?

Reliability and in general it not beimg able to scale salvaging 5 chips to salvaging 50 000 chips. Your margins well be gone if there are any significant failures in the field.

Salvage is ok for R&D or high cost assemblies with big FPGA's.

Salvage is very expensive and risky: There's nothing like the high-speed and reliable PCB assembly process for dissasembling PCBs, only manual processes, and even with skilled workers there's some risk of damaging the chips, if they aren't already damaged when you got them. And there's definitely no economic way to test the chips to ensure they meet spec. You can do it in a pinch for very high value low volume products; for a thousand or more consumer devices, no chance.

Seems like everyone's pretty understanding, although I haven't checked twitter yet. The Playdate discord is nothing but supportive of the team over at Panic.

The worst part to me is knowing that they had 5000 completed units, and without the battery issue, mine would probably be at my door already by now since I was one of the first orders.

I think the support has a lot to do with just how transparent Panic have been about the entire process. You can see a lot of people being able to emphasize with the human element behind the product and the company.

I've always been impressed with Panic's level of communication. Many companies could learn a thing or two from them!

From their blog they literally talk to people like they are little clueless kids. If that is how people like their interactions. I personally find it semi-annoying and mildly offensive.

They write in a colloquial, friendly style and use language that is easily understood by people that exist outside of the HN echo chamber.

This is targeted at the HN echo chamber, though. A Nintendo Switch costs less and plays AAA games. This is an STM32 with a black-and-white low rez screen and a hand crank. You could build one yourself for $50. (https://www.adafruit.com/product/3160 but that has a color screen.) Ordinary people that want to play video games are not going to buy this.

I know this verbatim sounds like "No wireless. Less space than a Nomad. Lame." or "you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem", both famously wrong assessments of hit products. But in this case, it's different -- the masses already have video games. If you want to make a 320x200 black-and-white game, fire up your editor, open up an OpenGL canvas at that resolution, and start coding. You can have your game available to billions of users tomorrow morning.

They could easily say "we switched from the STM32F4 series to RP2040s because there is actual availability right now" and not alienate a single potential customer.

I hear enough inscrutable engineering talk at work. I'm perfectly happy to read actual prose that reads like it was written for humans.

What are engineers if not human?

Engineers may be human, but they’re often inhumane, cold, calculating, and callous.

For example, being observably irked by insufficiently technical PR copy.

I'd consider myself deeply steeped in the HN echo chamber. Even though I'm a software guy, I've spent my share of time tinkering with electronics, implementing DIY home automation projects with Raspberry Pis and ESP8266 chips, etc.

But I wouldn't have recognized either of those without going to look up more information, skim the specs or possibly manuals to understand whether or not I care about the change.

The communication from Panic is simple, straight forward, and gives me a bit of context without making me go spend more time on research. They did exactly what they needed to do to help me feel more informed and less frustrated by the change.

On your implication that this is an unnecessary overpriced product, then I suspect this product was not built for you. Sure, someone could purchase those parts and have their own thing up and running in no time. But no one would care.

The PlayDate builds a defacto community all participating in the same product and the hype surrounding it, and this is what people buy into.

I also take issue with this only targeting the HN crowd. There has been plenty of hype in other circles, and the Playdate team will be shipping a no-code project so they're clearly trying to make this accessible to enterprising kids (young and old).

People seem to forget that you can’t evaluate games purely on tech specs. Sure, the graphics will be way better on my Switch. But if I want to play those cute Keita Takahashi or Lucas Pope games, I need to buy a Playdate.

Agreed, and interestingly it’s a similar story for the Switch itself! Relative to a PS5 it’s very underpowered, yet it’s still hugely popular in large part because of the awesome Nintendo games.

Go ahead, make them. I’ll give you $100.

If your initial reaction to something is "I could make that myself for cheaper" then you definitely are not the target market.

> Ordinary people that want to play video games are not going to buy this.

Everyone, including Playdate, knows this. Nobody is thinking this is the next Switch. People who aren't the target audience of the Playdate always seem to think this is the intention and I'm not sure why.

I've said it before, but to me it looks more like a kind of physical Pico8. Again, Pico8 isn't meant for the mass market, but its certainly not useless

Panics' language is definitely designed to be friendly to non-native speakers, who you seem to be discounting as unworthy of such treatment.

But I would say this is probably a bigger market than you seem willing to accommodate in your argument.

STM32 is STM32 in literally every language.

Maybe it sounds like that because they are marketing a toy?

This is actually just really good product/CE communication. Very technical and very untechnical people alike can read those posts and have a very good idea of what is going on, with a high degree of specificity. You don't have to use every piece of jargon available and write like a dick to have an interesting technical conversation. Communication like this is refreshing and clear, and you lose exactly nothing by writing like this to customers, as the technical implications of each sentence are also clear if you are technically inclined.

Compare with the grift of the Amico and the useless secrecy of Analogue.

Panic deserves every ounce of goodwill they’re getting.

I would rather them take the time to get this right, than potentially launch new hardware that flops and slaps them in the face.

When I see this kind of stuff I find them fascinating, because it's always the same story. People building a hardware product and then realizing that building hardware, even when it looks simple has a lot of moving parts. Coming back from that is quite impressive.

But when people boast about their amazing products to reek in VC money I actually get upset. I'm not sure if I'm more upset at the founders or the VCs though.

For example recently they have been raving about oxide, how amazing it is, how amazing the founder is, how amazing the idea is. But if you look at the website with all the ambitious goals and an estimate of early 2022[1] I can already see this company burning money like no other without ever getting close to that estimate. Either they have zero clue about supply chain or their wilfully misleading investors or a little bit of both.

But I guess that's the world we live in.

[1] https://oxide.computer/

Playdate does not seem like VC funded?

Panic is existing Mac software company, that sold (arguably, niche) software at not-that-small prices (well, usual for Mac OS ecosystem, at least in 2000s/2010s), and had a modest success being a distributor of Untitled Goose Game; and PlayDate is made in relatively small numbers; I would think they are self-funded and not backed by VCs.

Clearly I talked about two opposite directions. The first paragraph is about panic. The rest is about the companies that don't act like panic. Why lump it together?

And Firewatch!

People will link back to this comment in two years to point out how wrong / mean spirited HN can be sometimes.

It maybe helped that Steam Deck, Play Date and Analogue Pocket all announced their delays recently. Goes to show that this is beyond any single company and everyone's affected equally.

Yeah the supply chain breakdown study that was posted last week (sorry couldn't find link) points to the fact that demand across the board in almost all industries is quite a bit higher than it was pre-pandemic. Confusingly, supply is also higher across the board compared to pre-pandemic numbers, it's just that demand is so much higher than it was, that the increased supply is still woefully insufficient.

I was really surprised when I found out that the 5 pin JST XH connectors I need are out of stock seemingly worldwide. It’s just metal pins and a plastic housing. I looked at stock levels of alternatives and it looks like Molex is the winner. Hundreds of thousands of their similar connector in stock at Digi-Key.

My hope is that these supply shocks will teach people to better prepare for this kind of thing in the future, leading to more robust supply chains.

Yeah seriously: Why are JST connectors so scarce all the sudden? They're literally just some metal and plastic (Polycarbonate?). Unless there's a shortage of metal wire (cuz that's all it is; pressed into a squareish shape) or plastic why would the world be running out of JST XH (and other JST) connectors?

Are there serious bare wire and/or plastic shortages right now? I've read about price increases but not shortages.

My completely uninformed, wild guess: the manufacturing plants capable of creating those are creating other wires that are more in demand / profitable for them, rather than these cheap ones.

Here at Big Electrical Equipment Manufacturer we are currently dealing with a cable shortage. 3 years ago, if I would have told the people working here that someday we would have a cable shortage, I would have been laughed out of the room.

Do you mean vernacular term JST as in any little plastic PCB connector including all kinds of gray market fake clones or genuine JST components from JST the company?

Its a single supplier. JST is single company in Japan. I'm sure they're busy doing "stuff" and having fun trying to ship things.

Its like trying to buy Nichicon capacitors. There is one company in Japan making Nichicon capacitors. If they're busy, well, they're busy and you wait. There are zillions of "e-waste" reprocessors in China whom will desolder larger electrolytics from old PCBs, peel the plastic shrink, and apply a new shrink with all manner of trademarked names and temperature ratings to random e-waste pulls, including falsely labeled Nichicon. Those are the only two suppliers of Nichicon caps and if they're both busy or sold out you're outta luck.

Now if you're willing to change your BoM or if your safety regulator permits it and your company purchasing dept has a relationship with competitors and no single-sourcing agreement for better prices, you can use fantastic products from Chemi-con or Vishay or tens of serious competitors and hundreds of not-so-serious competitors.

Because you're doing stuff with JST XH I "know" you're doing something with modern RC vehicle batteries. I'm old, we played with RC cars in the 80s using Molex connectors and had the same problem as now, Molex is a single company and if they couldn't ship we didn't get connectors and that was the end of the discussion.

Full size car batteries are not licensed single source technology. Then again a lead post of a given diameter isn't very high tech. AFAIK there are no rechargeable battery pack connectors that are unlicensed at this time. FOSS type projects to issue a standard connector might arise, someday. Could implement 18650 individual cells and holders, those are unlicensed and not trademarked AFAIK but I have not verified that.

BTW JST connectors are extremely complicated not just wire and plastic. The posts are made with a cheap copper alloy, then plated with thick copper and plated again with tin, and the solder tabs are made the same way. I guess the PITA of a stage of pure copper plating is cheaper than paying for pure copper for the entire post. Copper is expen$ive. The plastic is a weird polyamide nylon-alike with some secret sauce flame retardant mix in that supposedly doesn't corrode copper or tin, so the connectors don't burn terribly well. Nylon usually doesn't burn until it melts anyway. Very hand wavy polycarb would melt or be tiresome to deal with at reflow solder temps, nylon is only like fifty degrees tougher but that's tough enough to make it easy to reflow nylon connectors. Or more specifically it'll keep its shape deep into the decimal places after reflow. They used to make polycarb capacitors from like 1960 to 2010 but they gave up, I guess impossible to hot air reflow them I don't know for sure. Polycarb dielectric caps were good higher-temp milspec capacitors...

(edited to note, if you buy genuine JST corporation connectors the above is true. Fake clones from China, who knows maybe made with copper plated aluminum wire and polyethylene?)

I guess I began using JST XH from my time with hobby lithium batteries but then I just used that connector in an unrelated project because I was familiar with it and there was plenty of stock in 2019.

But the Chinese fakes still have plenty of availability :)

The reason is that it's demand shock more than supply shock. Electronic equipment supply is reported to have largely recovered and is above pre pandemic levels. But demand from backlog and increased consumption has outstripped supply. So it's not just microcontrollers that are in higher demand, it's everything.

What I find interesting is the things you can't get chips for vs the things that are shipping without issue that use chips.

Look at what's happened to the auto industry. Cars, which most of the time have a need rather than being a toy or not needed, have issues with chips.

But, IoT devices that are monitoring us to phone home with surveillance on us see to not have a problem.

I'm sure I'm missing something. Am I wrong?

Automakers are notoriously looking for years-old chips because they don't update their products the same way. I believe that stock is just dead, more or less.

Dirty little secret: none of the leading-edge sub-10nm parts are of any use whatsoever without peripheral, power management, and analog parts built on those "dead" processes.

Maybe IoT garbage can ship with just about any chip that will run Linux, while cars need specifically certified components?

Not only certified (e.g. ASIL*) but long term availability.

* https://en.wikipedia.org/wiki/Automotive_Safety_Integrity_Le...

Which neatly explains why STM32 microcontrollers are in such high demand -- most of their microcontroller portfolio has a longevity commitment of at least 10 years.

https://www.st.com/content/st_com/en/about/quality-and-relia...

Espressif also commits to 8 years exactly because they want to capture lazy manufacturers.

>"Espressif also commits to 8 years exactly because they want to capture lazy manufacturers."

It's not about laziness, it's about certainty, certifications, and saving money on unnecessary redevelopment and testing. If each of the ICs on your board gets discontinued every two years, and you have 20 ICs on your board, it means that you will have to go through redesign and re-testing as often as you buy components, even if you're doing ten batches a year.

I don't think so because there's rarely a drop in replacement unless you designed your board ahead of time for multiple options (of course exceptions exist, there's drop in replacements for many devices including the esp8266 I mention below).

But for example, an ESP8266 hasn't really lost much value in the past few years, and is unlikely to lose significant value if it sat on a shelf for another few years, so they don't mind a bit of an overstock.

Meanwhile the current year car model will lose a ton of it's value when the next model comes out so they don't want to risk over-stocking it.

If you've got a low-chip-count product and an agile business structure, when Chip A is unavailable you can respond by redesigning your PCB to use Chip B.

But if you've got loads of chips on different boards from different suppliers, a structure suitable for a large business, lengthy supply chains, complex certification requirements, and you can't afford to pay much more? Well, that's going to make things quite a bit more difficult :)

Automakers effectively pulled the rug out from under their chip suppliers by asking them to severely cut production, then turning around and asking for a lot more than their original quantity. In general, those suppliers canceled their supply capacity with their factory partner, so when automakers turned around and placed new orders, those factories already sold $xx months of that factory time to someone else, making it hard for the semiconductor manufacturers to get back to their own production capacity and fulfill the auto industry’s orders.

https://www.freep.com/story/money/cars/2021/06/15/car-chip-s...

You’re exactly right.

Somebody at a major chip company told me that they went looking for, and found, new customers after the automotive customers canceled their orders.

So not only did the car companies ordered more than before after they realized that demand was not dropping, even if demand for them had returned to what it used to be, the total demand from other customers is now higher too.

Blame this on just-in-time supply chains where no one wants to hold inventory. It’s been MBA dogma for the last two decades.

Neither JIT nor many other things factors in "global pandemic," but proper JIT that knows how to scale up/down theoretically could handle things, its just the human element that fucks up the system. Yeah beancounters are beancounters but trying to blame JIT is really off course.

I disagree, respectfully. Not that I think everyone should inventory a year of everything. But the ethos of JIT is that you can treat your supply chain as if it’s infinite, and this keep it close to your projections.

Automotive got hit especially hard since any feature that requires electrics has the microprocessor embedded in that feature. Very cost effective-you don’t ship extra cpu power for unused features.

Tesla did the opposite though—centralized processing and over allotted cpu cycles by also made it easy to upgrade via software only.

Isn't the problem that is is not easy to scale some things up and down? This sounds a lot like no true scotsman.

> This sounds a lot like no true scotsman.

The claim is more that they are bad planners, orthogonal from whether they use JIT or not.

They're not doing proper supply management. You can do proper supply management with JIT or without JIT. You can also do improper supply management with JIT or without JIT.

For it to be "no true scotsman" they would have to be claiming that "doing JIT" is a solution to supply management. And they're not claiming that.

I guess you are partly right, but parent said "proper JIT" would work well without any evidence to what that means. Surely not everyone is doing the "improper" JIT.

Seems like we are dealing with the opposite issue here where factory production is booked many months in advance and can’t quickly adapt to new demand.

The factory is not in trouble, their customers are.

TAM for semiconductors turned out to be wayyyyyy bigger than the world's manufacturing capacity for semiconductors. I don't know how this caught manufacturers by surprise, but perhaps there's bottlenecks in talent, supporting industry and infrastructure that I'm not aware of that would have made additional expansion difficult previously.

There’s just not flexibility built in to the supply chain because flexibility adds cost.

When things go well this just in time manufacturing is lauded as a huge money saver, when they aren’t… you get now.

The business of running everything to the wire on credit and working “cost of capital” so much into business decisions gets us here.

I remember reading somewhere that automakers trying to get the governments involved to have them to lean on the chip suppliers to give the automaker the highest priority over other clients.

Why isn’t the result of those actions that those auto manufacturers either pay more or go without, rather than every buyer paying more and/or going without?

Because all capacity is being used. So it's not that today's capacity costs more (this capacity was sold and paid for months/years ago), it's future capacity that costs more. Until there are more chip factories, this is going to be an ongoing problem for the chip industry.

Sounds like someone at the car manufacturing has a shit ton of IOT stocks and knew exactly what they were doing

No, they're just being short-sighted and revenue-first. Think about how the rental car companies sold their idle car stocks and now can't rent cars reliably to anyone.

That's what happened, but faced with insolvency, you gotta do what you gotta do. You might really need those cars, but if selling it is the only way you'll make rent and payroll for the month, you gotta sell the cars. It's not a matter of short vs long term thinking, it's a matter of bankruptcy!

> Automakers effectively pulled the rug out from under their chip suppliers by asking them to severely cut production

Automakers only make few percents of MCU market. Just the most profitable one. Plus, a lot of MCU makers have own fabs.

It's very unlikely they took "months" of factory output. They just got their small batches to wait until the tsunami wave of mainstream part runs is done.

This is just the quoted lead time. When the cycle eventually turns, there will be an enormous wave of push outs and cancellations and those who actually wanted the full order will get the product much sooner

All the big industries got scared when COVID hit thinking consumer spending was going to plummet so they drastically cut orders.

Meanwhile the IoT space has been growing rapidly in the past few years, and those suppliers didn't cut their orders. IoT devices like the ESP8266 can sit in a warehouse for the next few years before being sold for use in some smart lightbulb, but automotives will lose their value as they become last year's model.

With the supply chain disruptions and increasing consumer spending automotive industries basically lost their reserved spots are struggling to claw back their manufacturing and shipping capacity.

So basically in this game of musical chairs the automotive industries stood up while the IoT space stayed sitting, and then the supply chain disruption removed the chair the automotive industries had been sitting in before...

This doesn’t serm right. The car manufacturers would surely just buy their way into the front of the line, if it was just this. Chips are a fraction of a car’s value and the cost of stopping car production lines must be a lot higher than buying slots in s chip production facility.

"Buying your way to the front of the line" may work if it involves paying $18 for a part that normally costs $15. (Even so, the automakers will scream bloody murder over much less.)

That's not what's happening. What's happening is that the $15 part now costs $150, and is only available from sketchy Asian brokers that aren't on anyone's list of approved vendors.

The semiconductor houses had better unfuck themselves SOON, or 2022 will go down in history next to 1929. I don't think anyone understands how serious this situation is getting.

Between flooding and Covid-19 lockdowns, the semiconductor houses aren't, like, choosing to be shut, they're closed because of global climate change. This is only going to get worse as the climate changes dramatically within our lifetimes.

> The car manufacturers would surely just buy their way into the front of the line, if it was just this.

Those older nodes "chairs" that the automotive industry used are less profitable in general. One doesn't simply spin down and spin up a semiconductor fabrication plant.

>> the supply chain disruption removed the chair the automotive industries had been sitting in before...

Replacing the (old) chair is easily in the billions of dollars. Not everyone can foot that kind of bill.

In that case it was an incredibly stupid move for car manufacturers to wind down orders for semiconductors, even in the face of covid. The down side risk is huge.

A charitable interpretation of that decision would account for the fact that they have more information than you do about the situation.

Well, they may have had more information, but it certainly wasn't good information.

> The car manufacturers would surely just buy their way into the front of the line, if it was just this. Chips are a fraction of a car’s value and the cost of stopping car production lines must be a lot higher than buying slots in s chip production facility.

There is no excess capacity at fabs, so if someone buys their way to the front of the line, the fab is going to miss delivery dates on all the contracts it currently has.

Fabs do have that option, but it's going to cost an enormous amount. Automakers have to pay enough that the fab makes enough money to compensate for the loss of reputation (it won't be a secret that they let someone skip the line), even after subtracting any contractual fees brought about by missing delivery dates on other contracts. That's why you aren't paying slightly more, you're paying orders of magnitude more. I would also be unsurprised if everyone who can get in an order is paying a little extra to raise the contractual fees for missing a delivery date.

A single jumped order can make 100 other contracts deliver late. That's a lot of fees.

The bigger the BOM, the higher the probability something is not available, blocking the product from being fully manufactured.

Cars have bigger BOMs than IoT devices.

Given the long lead times, tech companies are scrambling to use alternative chips: first finding something functionally equivalent and pin compatible; that failing, qualifying something close enough and redesigning PCBs with an updated component footprint; last case scenario, porting FW to a different chip and redesigning circuits to accommodate.

Traditional car companies largely rely on suppliers to redesign electronics (at even greater cost and lead time). There's also more liability with cars so last minute redesigns have to be taken very seriously.

I suspect that IoT devices also have a much shorter turnaround when they need to redesign the board to use the chips available.

Anecdotally I think this is right on the money. I've been buying lots of LoRA hardware over the course of the year and I think I'm up to 9 different permutations of the "same" board via swaps of the LoRA SXnnnn/ESPnnnn/GPS Mn units for whatever is currently available. Turns out to be a good way of discovering firmware bugs.

I'm guessing BOM doesn't mean Byte Order Marking here?

Bill Of Materials

Bill of Materials - all the components required to build a circuit

In this case, Bill Of Materials, as in a list of all chips, resistors, capacitors and other electronic components.

> Traditional car companies largely rely on suppliers to redesign electronics (at even greater cost and lead time).

Just to put this in perspective for some readers, I used to work at one of the largest automakers and suppliers would bend over backwards for us. They would shut down the line and retool for us when we needed something quickly. But even when things were running smoothly, some parts had a 12 month lead time and there was nothing that could be done to expedite that.

I remember one part that we shared with many other automakers had a recall. It took the industry over 3 years to supply all of those parts to consumers. Not airbags, this was almost 20 years ago.

All of that is without a design change. Now add a design change on top and it can easily move a 3 month lead time to 12 months or more.

I do consulting for another automaker now and one of our chips has become unavailable. Actually, at the beginning of the pandemic the price jumped almost 10,000%. We had no choice to pay it, but that also killed all of the margin on that particular part so we immediately started redesigning the PCB for a new chip. The software had to be rewritten for the new chip; several case components had to be redesigned to accommodate the new PCB, testing, mass production, etc. That was over a year ago and the part is just starting to roll out now.

So, I think PlayDate's messaging is completely reasonable.

The Decoder podcast had on the CEO of Anker and he explained (IIRC) that chips coming off of 12" wafers are impossible to get, while chips from 7" wafers are plentiful. The problem is recertification, which for autos can be a very long process.

Why that occurred is another question.

The automotive industry presumed they were higher delivery priority than they really were. By volume they were not thus they had lower priority when things got squeezed.

The auto industry also had unreasonable chip reliability demands - physics doesn't care that they wanted 30-year life while also having 7-10 nm CPU performance - you can NOT HAVE BOTH!

IoT is primarily NOT 7-10 nm performance nodes - AVRs and many ARMs are not at that level - you can get by with larger geometries which can be made on older processes. So not so much shortage issues.

No, there are legitimate shortage issues, and the "Automakers screwed up their forecasts" excuse is starting to wear awfully thin.

Intel FPGAs aren't typically used in cars, for example, yet we're seeing quotes of $500+ on parts that used to cost $50. Lead times? "Don't ask."

Automakers more or less started the shortage, as a result of that everyone is in squirrel mode buying up what they can because they rightly fear having to reengineer and revalidate every single production run. That our new economy has a greater thirst for chips just compounds the issue.

If there's a shortage and buyers willing to pay a premium in one segment, it affects all other segments because at least part of manufacturing capacity is interchangeable (with switching costs, but those are bearable if someone wants to pay) and it becomes profitable to randomly shut down production of various niche parts to save some car manufacturers arse, or have essentially a bidding war between those niche parts or car parts in order to discard any customers which aren't willing to pay increased prices.

FPGAs are likely short because they are a substitute for SoCs in expensive products.

Everything is sloshing around.

If you want to deliver that part to beat your competitor this year, you will find a substitute.

I don't understand why car manufacturers haven't just started remanufacturing some of their older (say 2010) model cars that don't rely on so many computer chips. I'm sure there's lots of costs to retooling their factories, and these less computerized cars are going to be less desirable to most customers, but with car prices currently 150% of what they "should" be, it seems like it would be profitable for at least one company to go the "low chip" route.

Some manufacturers already did remove advanced vehicle features.

https://www.bloomberg.com/news/articles/2021-05-06/chip-shor...

BMW is pulling the touchscreens off of certain models:

https://www.autoblog.com/2021/11/05/bmws-losing-touchscreen-...

Slightly confusing since it's not the removal of the entire infotainment system, just the capacitive controller. The system is still controllable with the joydial. Seems like that's not really a $500 cost center, more of a way to appease the customer.

And some makers like Mazda pulled the captouch off their IVI with the recent generation of cars pre-COVID, it just wasn't worth the cost.

For Mazda it was also at least theoretically to reduce driver distraction and increase safety. I mostly believe them on that point, given how much they focus on safety ratings in other ways.

Some of them were also mounted pretty far from the driver so they weren't easy to reach, and they had a dial to alternatively interact with them. The first few times I drove one, I didn't even know it was a touch screen.

On the earlier Mazdas the touchscreen actually disabled itself when in drive. Dial only.

Seems like most people didn't need it anyway.

My guess would be they won't be regulatory compliant. Few cars have already removed non essential things like touch screen.

It takes close to a year to re-tool a factory for a different auto model. The suggestion to just produce an older model just doesn't make sense in time or money.

My 2011 car has ECUs for: engine, brakes, ABS, air bag, air conditioning, alarm. All of those are connected to a gateway, that is another ECU. Do you want parking sensors? That's another ECU. Multimedia? That's one or two. Cars of 2000 have not much less ECUs. Consider also that components to build such ECUs are now totally obsolete and out of production.

My old 1997 Town Car has about as many controllers as my 2011 Crown Victoria does (many of them are even the same parts), and the 2021 300S I just ordered doesnt have many more.

Though the 1997 Town Car, had many more controllers than a 1997 Taurus for comparison.

This is partially true. Vehicles are needed but most of the computing the modern ones do is completely unnecessary for the primary function and purpose of the vehicle. There is really nothing important I can do with my Tesla that I can’t with my 1980s Ford ranger. There are definitely important things I can do with the old pickup truck that I can’t do in the Tesla. Until we reach a point where the car can legally drive you home from the bar while you are passed out in the back most of the stuff chips are used for in vehicles pretty much is for sales purposes and turns it into a giant smartphone. Add in the whole remote exploit thing about some of the models of different 2000s models with dash computing that never get firmware updates but still have remote network connectivity and it’s actually a pretty shitty idea overall until actual self driving cars are a thing. Before anyone mentions that you can track your car if it’s stolen, it’s important to remember most of the thieves going after new cars will back up to the vehicle with a metal box truck lined with some kind of faraday material then pull your car into it.

Having an electric vehicle doesn’t mean it needs the fastest hardware. You can run the entire base mechanics sensors and outputs of an electric vehicle on an Arduino with some extra breadboards.

> pretty much is for sales purposes

Features like blind spot detection, backup cameras, and adaptive cruise control are pretty valuable for general collective safety reasons.

It's not even legal to sell a new car in the US without a backup camera.

IoT are OK with high risk and move very fast with startup mentality. You have qty zero of CPU model XYZ in stock and qty essentially-infinite of CPU model ABC in stock? Our new product, which will have zero support after shipping, will ship with CPU ABC which is in stock.

Car mfgrs are in the risk adverse capital preservation mode and it takes years for changes to be made. You only have model ABC CPU in stock? Has that undergone enough approval and testing processes to be an automobile ABS controller? No? Then we wait. It'll take years for model ABC to be approved -or- similar years for us to design a new brake controller system.

If you ship an IoT product its assumed it'll have zero aftermarket support, be wide open to new security holes, and probably stop working in a couple years at most. Cars are expected to have parts available for 20 years, never be hacked, and reliably operate in a life critical environment for decades.

If the IoT temperature sensor is 2 degrees high, nobody cares anyway, ship it and fix the software. If the automotive brake controller only applies the brakes 99.9% of the time perhaps because it has less memory so the stack crashes under weird and rare conditions, it'll pass trivial levels of testing while killing lots of people causing a financial and PR nightmare...

A better comparison to automotive would be aviation, or perhaps ocean shipping.

Nope, they're having trouble too. I was on the phone with an embedded cellular modem module manufacturer earlier this year. The part we spec'd was unavailable because the underlying modem device was on allocation for so long that the module manufacturer decided to just discontinue the device rather than say it would be out of stock for two years.

It's a bad time to be a non-top tier hardware manufacturer. I'm sure Apple, Samsung and the like are having supply trouble as well, but they have the scale, connections and money to handle it. Smaller companies are getting demolished.

It is quite frustrating. I'm a medical device hardware engineer, so our quantities are in the low 1000s per year for our highest-running SKUs. Just today had to deal with a panicked contract manufacturer who was going to bring our line down for an oscillator and a capacitor that they were shorted. Distributors will tell you they're sending a reel of 1000 parts or whatever, but you open the box and they just threw in 30.

Every other week it's another *drop your NPD responsibilities and come up with a solution to this shortage so we don't stop our assembly lines* which is quite frustrating to keeping "on schedule" and "in a flow state"... If it's not in *your* warehouse or allocated to you in *your* contract manufacturer's warehouse, it doesn't exist.

i've started modularizing my designs again

10 years ago it was pretty common to design small submodules and stitch them together into a final assembly, that would get upgraded piece-meal as tech/product needs evolved

about 2-3 years ago it became more economical to just design an entire bespoke board for any new project, and then run an entirely new board for any subsequent changes. i loved not dealing with interconnects anymore

but now I'm back to reusable modules and sub-assemblies again, because by the time i'm finished designing an entire board, a single part will be out of stock and i'd have to start from scratch :(

Yeah but what are you doing about connectors to join the boards together? There's serious shortages of things like JST connectors right now just like chips.

if possible, leave enough clearance to swap a vertical for a horizontal in the 11th hour ;) or use larger/more common widths like 8 or 16 position connectors even if you only need 5 or 11 or them filled

if you can reach for creative solutions, a couple months ago it was cheaper for me to use one of those 6-pin power only USB-C connectors with an off-the-shelf USB-C cable than a 6-pin molex cable assembly...

This was known before they (and Valve) took preorders.

Like in the Great Recession a side effect, or by design, was a banking squeeze and consolidation from small/medium in favor of large players.

The chip/ship supply shortage is again squeezing small/medium in favor of large players.

Very little margin and too much optimization/efficiency is bad for resilience. Couple that with private equity backed near entire market leverage monopolies/duopolies/oligopolies that control necessary supply and you have trouble.

HBS is even realizing too much optimization/efficiency is a bad thing. The slack/margin is squeezed out and with that, an ability to change vectors quickly. It is the large company/startup agility difference with the added weight of physical/expensive manufacturing.

The High Price of Efficiency, Our Obsession with Efficiency Is Destroying Our Resilience [1]

> Superefficient businesses create the potential for social disorder.

> A superefficient dominant model elevates the risk of catastrophic failure.

> If a system is highly efficient, odds are that efficient players will game it.

Hopefully that same mistake is not made in the future. It will take time to build up diversification of market leverage in terms of chips for availability. Hopefully we have learned our lesson about too much concentration, with that comes leverage and sometimes a "gaming" of the market.

This chip shortage, and all the supply chain problems during the pandemic as well, will hopefully introduce more wisdom and knowledge into business institutions that just because things are ok while being overly super efficient, that is almost a bigger risk than higher prices/costs. Competition is a leverage reducer. Margin is a softer ride even if the profit margins aren't as big.

[1] https://hbr.org/2019/01/the-high-price-of-efficiency

I honestly still don't understand the hype around this thing. To me it's up there with Tiger electronic games, but for a lot more.

The story is less about the device, and more about the chip shortage. Having to wait two years for a CPU is just crazy, and it’s interesting to watch the hoops that these companies have to go through in order to ship products. I don’t know anything about the device itself, but do you find it interesting to watch how do chip shortage is affecting different manufacturers.

Panic has a history of delivering thoughtful, well-made software. Their initial foray into games (Firewatch, Untitled Goose Game, Nour) were all excellent and novel. Combine that with Teenage Engineering's similar novel and excellent track record with hardware design and you've got a very interesting product to those that like Panic's software and/or like good hardware design. If nothing else this product will be novel and well considered.

If that's not your cup of tea, cool. But there's a lot here for several communities.

They could release just the software then, which would run fine on Android and iOS machines (which by the way offer way more value for people who appreciate good hardware and good software) Who needs more e-waste?

The playmate has a unique interface with its wind up arm thing. I would assume all the already made launch games will use it

Not all the games for the Playdate use the crank, but a lot do.

They could have sold the crank as an extension for iPhones or Android phones.

The hardware is the whole point of the endeavour.

I agree. In a chip shortage, we should be reassessing exactly what devices need the chips, and priorities. I am very unexcited by this product, it seems practically useless. Tiger Electronics Handhelds indeed.

The steam deck (a real linux handheld computer) on the other hand, gimme gimme gimme

Okay, but people want different things. If you were to ask me, I would much rather ”reassess” the Steam Deck — a device in which I have no interest, made by a company with a poor hardware track record — than the Playdate — a unique device with interesting exclusives made by two companies I like a lot.

> Their initial foray into games (Firewatch, Untitled Goose Game, Nour) were all excellent and novel.

Note that Panic hasn't developed any of those games; they published them.

Right. True.

Continuing that trend, nearly all of PlayDate's announced games are also externally developed and (obviously) published by Panic.

https://en.wikipedia.org/wiki/Playdate_(console)#Games

See also: ClockworkPi, DevTerm, GameShell / OpenPyra, etc.

Seems to be an emerging successful model: take the revenues from a game/software product and use it to bootstrap a hardware company.

There's apparently a pretty big market for plastic doodads that do the same thing your computer can, but with a crank. It's like they ripped a page straight out of Apple's playbook and made an iPod for gaming.

I can see why it's cool and the me from 15 years ago might have bought one. But the current me just tires of more useless crap constantly being built that will just end up in someone's junk drawer 3 months from arrival and in the landfill years later.

I think it's like that statistics about most of purchased Steam games not being played or something like that. You buy an idea that you could have enjoyed this device.

People get super upset when you say this, but like much of Panic's other hardware (like their Mini-ITX case) you're also paying for the mythos.

Like I said from the get-go that most of these will be used as desk ornaments (albeit pretty and functional ones)

Not long after Panic announced a dock with a pen holder.

They know their market, people with a lot of disposable income and a want to a showpiece that's not as flashy as gold pen or something

You're confusing Panic (the software/video game company behind this device's concept and software) with Teenage Engineering (a Swedish hardware company that designed the device)

Teenage Engineering has a variety of products unrelated to this game console, including the ITX case.

Sorry you're right, but my comment stands word for word with the name replaced

Teenage Engineering is behind the physical device, which is what represents most of what I'm describing here

There's also this article:

https://arstechnica.com/gaming/2021/11/playdate-delays-to-20...

(via https://news.ycombinator.com/item?id=29191933, but we merged that thread hither)

Chip shortages are real, but shipping capacity is also very tight at the moment.

Peak Design had some issues[1] with shipping their phone accessories (tripod, bike mount, etc) because of the shipping capacity crunch. Apparently large electronics companies like Apple have booked out most of the air freight capacity.

1. https://www.kickstarter.com/projects/peak-design/mobile-by-p...

The same thing happened to the Steam Deck so it's not really a surprise at this point.

I'll have a Blue... Christmas... without you...

But you'll have a very happy Christmas 2022!

Interesting, that one cannot delay a (pre-)payment (except as a credit) yet the goods can be delayed randomly. Seems like some asymmetry there.

The reservation fee for the Steam Deck was 1€, at least for me.

As a pre-order customer, I don't mind the delay. I'm glad they're doing it right instead of rushing to meet a deadline.

What a shame. Such a niche and cute and nostalgic little project will probably be dealt a deathblow because of this. I have no doubt Panic will find a way to move forward but it's going to take time. And that kind of delay is so much more hurtful to niche projects.

the weird thing about these products if they seem to go two ways.

one is "what a cool quirky company, they always do stuff like this, i like their quirkyness i wonder what they will do next"

"what fever dream did they have that they could put together a gameboy clone? they don't know the market. haven't they seen what happened to the lynx and neo geo pocket and the wonderswan and even the vita? its like invading russia in the winter, you don't make a handheld system in the age of cellphones"

Tbh I was 100% expecting it with all the shortages and chip crunched. Hardware is hard, no way around it If anything, I appreciate the transparency

Does anyone know which exact CPU they were using and which one they’re switching to?

You would think in that long interaction they could have provided some actual details, yet they spend so much space repeating themselves and acting like their customers are small children.

It's a 32bit ARM CPU running around 200mhz I think. I'm sure they just used a similar ARM CPU.

STM32F7 with a Cortex M7. I don't think they've announced what they are moving to, except that it's a variant with better availability.

Disappointing but happy they finally issued an update on the situation detailing how they’re prioritizing QA over on-time delivery. I’ve been checking religiously for signs of a shipment.

It's like the whole world needs STM32F7 parts and nobody can get them. What a pain in the ass.

You might have more luck on marketplaces: https://www.sourcengine.com/search?q=STM32F7

> we found a number of units with batteries so drained, Playdate wouldn’t power on at all — and couldn’t be charged

Wait, wait, wait… Are they saying that if the battery is too drained the device will not charge? It that’s the case, it’s a terrible design decision!

Also, isn’t the + shaped control a Nintendo patent?

Firstly, it wasn't a design "decision". It was an unplanned, unexpected thing.

Secondly, did you even read past that paragraph? They got a new supplier and replaced the batteries with better ones in every device.

You didn’t understand my comment.

Their write up seems to imply that the device wouldn’t charge if the battery is completely drained.

It happens sometimes in cheap devices in which the charging logic is managed in software, but it’s a terrible design decision.

If that’s the case, a better battery will mitigate the problem but the terrible design remains!

I was surprised that a device could drain to the point where it could never be charged again, thanks for the possible explanation.

> It happens sometimes in cheap devices in which the charging logic is managed in software, but it’s a terrible design decision.

In my experience, this is a pretty common practice, not limited to cheap devices. Many modern battery-driven devices I've looked at closely (for example: e-book readers, smartphones, ...) cannot bootstrap from a completely dead battery. They have to bring up the CPU, which then brings up the PMIC, before it can start charging the battery.

The difference between well-designed and less well-designed devices is mostly in how well they protect the battery from discharging beyond the point where this minimal bootstrap process stops working.

Oh hey no one answered your query about the patent on the D-pad; that patent is long expired. (https://patents.google.com/patent/US4687200A/en)

Very cool. I hope more video game hardware appears (like the Steam Deck).

It really opens up a lot more options for developers and a new paradigm for playing games.

I don't think the crank is going to be anything ground-breaking, but I like the idea of designing new controllers. It's cool.

I saw the same thing with Steam Deck. I don't really care since they haven't took my money yet.

I'm actually happy with this since I just brought too many toys recently. Gives me a bit more time to save

Hardware be that way under the best of circumstances, and 2021 certainly isn't the best of circumstances for physical fulfillment. So it makes sense.

Battery issues are so common. I ordered an electric unicycle last spring that came with dud batteries, now I have to pay $150 in shipping to get it fixed.

On their website..

> The Crank. Yes, the crank. Is it a gimmick? Nah. Does it charge Playdate's battery? Nope.

I guess they could revisit that eh ?

Is it bad that their transparency about the aspects affecting the delay made me want to order?

As someone who never plays games, I kinda want this simply because of how cute it looks.

Did you see the dock the plan on releasing? makes the thing look like a tiny little CRT when docked. Very cute indeed.

How about naming and shaming the dishonest battery supplier?

At least tell us what CPU you are using!

180 MHz ARM Cortex M7F

https://www.ifixit.com/Device/Playdate

Thanks. Any idea what CPU will they be moving to?

They haven't released that info yet.

Such a shame, I'll put the xmas wrapping away for this one.

So.. the crank doesn't charge the battery?

Of course not. The crank is an input device, like a joystick.

You say that, but I've never seen an input wheel as a crank; And it isn't obvious to me how it helps fine control (ala a game control / mouse wheel-like input) since I'd associate crank-like features with benefiting power i.e. turning something with high torque more easily.

Incidentally, people are so offended by the suggestion I immediately received surprise downvotes. Who knew it was such a controversy.

Crank is a gimmick, why else would anyone want to pay $200 for ~$30 pay as you go phone specs hardware in 2021?

Would have been cool if it was an input AND charged the battery.

It was only that read the title and thoght

“Yep that is my life?”

Joke's on them. Once I saw it was extremely uninteresting I got a refund.

I swear, do people not even read the articles anymore and just jump straight to comments?? That's exactly what the company has done already.

That's ... what they did. They say so in the article. They swapped the CPU on a revised board.

Yeah whatever,

Every single software company that jumps hardware finds out in a sour way how completely different the two worlds are.

Reply on news.ycombinator.com