Unveiling secrets of the ESP32: creating an open-source MAC layer | Hacker News Reader

Unveiling secrets of the ESP32: creating an open-source MAC layer

zeus.ugent.be

293 pointsredfast00149 comments

I wasnt aware of this wifi blob. This feeds a tiny paranoia I have at the back of my head when dealing with esp32/espressif. I have dozens of esp32s around and I love them, but Espressif is 100% Chinese.

Im uncomfortable with what I read that every company of significant size in China automatically requires CCP party members to be involved in the company at a high level.

So Im very happy to hear people such as these guys are looking deep at this.

Ofcourse since Espressif controls the hardware, so they can do anything eventually. My itch will always be there and Im going to switch once I find something made in preferably the EU when I find something comparable to esp32. Maybe Nordic Semiconductors will make some nice risk-v chips and dev-boards soon.

It is because of FCC certification requirements. Usually, if the end user can modify the lowlevel radio firmware on the device, the device looses it's FCC certification and cannot be sold in USA.

It also seems that Espressif has bought their wifi IP, so their contracts and licensing terms with the IP vendor likely prevent any sharing.

But FCC is the reason for closed binary blob firmware for all wifi radios out there these days.

I don't understand how the license affects certification TBH. As this post clearly shows users can implement their own stack if they really want to, it's not that the license is going to prevent them. Why can't one have an open source stack with specific builds that are approved, tested and certified?

They can, but they lose their certification. Same if you don't use a certified board your antenna might need new certification of your device. It may work, but if authorities find out it may become quite expensive.

I can also remove my lights and breaks from my car and it will still work, but if authorities find out I might be in trouble.

They could, but somebody would have to write that FOSS wifistack. You could not run selfcompiled versions of the stack on any hardware that has been FCC certified, because if you could, the certs would be gone once again.

Wifi is shared spectrum and devices using are licenced to make sure they conform to the local regulations. One size does not fit all. For example 2.4GHz wifi channel 13 is legal in EU, but in USA it falls on a govt owned band. This is why companies like Mikrotik or Ubiquiti have specific hardware versions for USA. So that they verifiably cannot be set on illegal channels by the enduser.

So it's perfectly feasible to have an open source Wifi, or Bluetooth or any other RF, stack, but only certain compiled versions are actually certified. I understand that the openness in this case would be limited: you lose the freedom to modify and run the software as you wish (unless you want to risk to break the law), but you can still help bugfix, improve the software and verify that there no backdoors / spying features.

The reason sometimes given by vendors that "FCC demands the code to be proprietary" is an excuse.

The law does state that "an intentional or unintentional radiator must be constructed such that the adjustments of any control that is readily accessible by or intended to be accessible to the user will not cause operation of the device in violation of the regulations" (https://www.law.cornell.edu/cfr/text/47/15.15 )

So if the manufacturer makes a device where changing the firmware is "readily accessible" to the user and there is an open source firmware available that can circumvent the FCC transmission restrictions (for example, change the power limits or channel limits for wifi physical layer), then that could be grounds for FF refusing to certify that device, as it is not permitted to make, import or sell general unrestricted transmitters to the general public (there are certain exceptions for licensed operators, ham radio, experimental use by manufacturers etc).

It's similar to other clauses that prohibit manufacturers from making it easy for the user to modify the equipment - e.g. 15.203 (https://www.law.cornell.edu/cfr/text/47/15.203) "the use of a standard antenna jack or electrical connector is prohibited." so that the user can't easily replace the antenna with a different one from what was certified.

Is that really why? I know many devices where I can just set whatever country I want :)

This is handy sometimes because in the EU it's the opposite, the 5 and 6 GHz bands are much smaller.

FCC certification is for the entire system, firmware included.

The reason the modules with integrated antenna are so popular is that the module carries its own certification, so you don't need to do it yourself.

If you use the module with an antenna connector instead of integrated, you must get your own FCC certification. It's for the entire system.

> Why can't one have an open source stack with specific builds that are approved, tested and certified?

Mainly because it's heinously expensive and difficult.

Expressif also block some cool packet injection things like sending malformed packets in their firmware. Which is not illegal to offer. They probably are afraid of getting a bad reputation.

This is why open firmware can be really handy for the security community.

Given that they are originated from China, I'm not surprised.

For a while (not any more though) you can't sell an ACR122U RFID card reader online in China, just because mfoc [1] supports it nicely and it got a reputation of "smart card cloner".

[1] https://github.com/nfc-tools/mfoc

Wifi is easy... there's no way to send anything undetected, since you control the routers, etc.

GSM->5G modems are a lot harder to debug... maybe now in recent years with cheaper SDRs, but a lot harder then wifi.

And not sure why you'd be afraid of CCP, we saw the wikileaks, USA does a lot of similiarly bad stuff too and even got caught doing it... and if you live in a "western" country, USA has much easier access to you than China.

There’s no point having a vague Chinese back door in Espressif devices which security researchers could discover with relative ease. The remarkable prevalence of these chips in commodity consumer goods means that they’ve likely already been analysed by countless world governments and strategic enterprises.

If the IoT ecosystem has any weak spots, it is the Tuya software stack. It would be much easier, much more useful layer to put a back door into that.

You are assuming you have full insight into what the board is capable of.

What part don't I have insight into? At least the parts that could be exploited by china in some way that would affect me, and couldn't be detected?

Google 'the thing' and tell me that you could have predicted what it was and how it worked. Hardware is finicky in that way: you look at one thing it can be quite another.

> And not sure why you'd be afraid of CCP, we saw the wikileaks, USA does a lot of similiarly bad stuff...

I find little solace in this whataboutism.

It's not whatboutism when there's literally no proof that they are doing mass dragnet spying on western residents. And when we do have tons of proof of western governments doing exactly just that

Again, it's a trendy buzzword to use but it literally isn't a catch all shield to argue that it's fine when we do it. When there's no proof of something happening, maybe we should focus on the thing that we know is happening instead of chasing literal ghosts

1) The activities of the US gov't have precisely nothing to do with the probability that a device from China has some backdoor or surveillance function. When someone raises this concern, the response "the USA has a history of surveiling its citizens" is not a rebuttal, it's irrelevant. Thus a whataboutism.

2) Your burden of proof might be different from mine, but one needs to be pretty naive to think that the CCP doesn't surveil western citizens. I doubt their intelligence apparatus is that bad at their job. In fact, I have plenty of reason to believe it's pretty great at it.

[0] https://www.csis.org/analysis/how-chinese-communist-party-us...

[1] https://foreignpolicy.com/2023/04/27/china-ccp-spying-us-pol...

1) of course it does. This is an information cold war. If participant 1 is doing something then participant 2 is forced to do at least the same thing so as to ensure they don't fall behind. It's very naive to not expect the actions of one state to not affect the actions of the other.

you either intentionally or unintentionally phrased it in a way that blames unnamed participant 1. And given the context of this conversation, it seems that you blame US for CCP's inevitable spying. Seems weird given the kind of government CCP is(hint, the name).

In my opinion US is only at fault for spying on it's own citizens(im not citizen nor resident of the us), and in doing so it undermined its counter intelegence actions against state adversaries. Next time there is a good guy in the white house, All intelligence apparatus should be dismantled and replaced with something more transparent for the people, even though the reality where most of the world is despotic by design and there for hostile to a supposed to be free nation is still there.

I'm not blaming either party. I don't even know who first started it. I'm just pointing out the fallacy in the argument. That's why I explicitly used generic names instead of CCP and US Gov.

So, usa was "proven" (well.. data was leaked and believed by many to be true) to be spying on many people, both local to US and foreign... and for china, all you have is "you're naive, if you don't think they doo it too".

So, you don't mind being spied on by someone who was already caught spying on their own citizens, (assuming that you're from USA), has access to you, your finances, can lock you up, can suicide you in jail, etc., but you're afraid of china who has access to none of those powers?

How is "but they do it too" not whataboutism?

It's not they do it too, it's "they are the ones who do it". There is no "too" in this context for the average western user. Unless there's proof indicating otherwise.

It's... flips through the list of officially recognized fallacies "tu quoque": https://en.wikipedia.org/wiki/Tu_quoque

"Whataboutism" is trying to deflect to an utterly unrelated topic, like suddenly turning around, pointing at a tree an screaming "but what about the pine needle water content cartel?!".

It certainly isn't pointing out hypocrisy, establishing a common standard or a cureall in online discussions.

I've seen a proper "whataboutism" been used but only once (by a malfunctioning redditor from Eglin Airforce Base).

Op above me was bothered with the company being chinese, and I was pointing out the hypocracy because the chinese didn't get caught at a level nearly as bad as US did with wikileaks and that for most people in the west, it's better that the chinese find out that you did something bad than if your own intelligence agencies find out.

https://www.businesstimes.com.sg/events-awards/singapore-bus...

Learn more about Espressif's founder. And I think the CCP party cannot impact Espressif.

------- Singapore’s bilingual education gave the engineer an adequate command of Chinese; he played translator for his Chinese and non-Chinese speaking staff in meetings during Espressif’s early days.

And the time he spent in national service with the Singapore Armed Forces taught him the importance of being in the front line, of knowing the ground well.

The CEO believes that entrepreneurship cannot be taught. One needs to have a head for risk-taking, creativity, a big-picture perspective, and to be prepared to fail.

And passion, of course.

He has a nugget of wisdom for those who have yet to find theirs: “There are two things that drive people... One is passion, the other is fear... If you lose that fear, you might find your passion.”

Your paranoia about the Chinese is equally applicable to Americans, whose NSA has given itself carte-blanche to infiltrate any computing system it desires, for whatever reason, in total secrecy - without recourse for the public to address any wrongs.

So I'm not sure that framing your paranoia in terms of "the Chinese" is productive - you might just want to update that thought with "any state actor who operates covert torture sites and violates human rights at immense scale", in which case your set of actually hostile actors becomes a little more realistic.

The biggest threat to your freedom and human rights, as an American, is your own government.

> Your paranoia about the Chinese is equally applicable to Americans, whose NSA has given itself carte-blanche to infiltrate any computing system it desires, for whatever reason, in total secrecy - without recourse for the public to address any wrongs.

Yes, except the China also 'infiltrate' Chinese companies themselves, and can perhaps order them to put in backdoors.

The NSA generally does not order US companies around, as evidenced by the fact it's been documented that they intercept shipments and compromise the systems on their own:

* https://arstechnica.com/tech-policy/2014/05/photos-of-an-nsa...

If the NSA had an 'in' into Cisco (or Juniper, or Aruba, etc), they wouldn't need to clandestinely have their own 'compromise factories'.

Yes, both the Chinese and NSA do cyber stuff, but so does every country. At the very least the odds of getting a 'clean' product from an American supplier compared to a Chinese one are higher: the links between Chinese companies and government are often murky.

>Yes, except the China also 'infiltrate' Chinese companies themselves, and can perhaps order them to put in backdoors.

As does the USA's own spy agencies. There is literally no moral authority on this issue that can be claimed by America over China. Did you overlook the multiple NSA backdoors implemented by Microsoft over the years, or just have not caught up to this situation, yet? See also - Intel: TPM.

>The NSA generally does not order US companies around

I believe this to be false on the basis of multiple whistleblower leaks which demonstrate otherwise. Not to mention that American companies have evolved the canary mechanism as a means of bypassing strict secrecy rules around disclosure of this influence by the spooks.

>At the very least the odds of getting a 'clean' product from an American supplier compared to a Chinese one are higher: the links between Chinese companies and government are often murky.

I do not believe this to be true one bit. China and America are equivalent when it comes to trustworthiness, which is to say neither country has the moral authority to claim a more ethical behaviour over the other when it comes to human rights.

China doesn't operate Pine Gap - the worlds biggest, wholesale violator of human rights at massive scale, ever.

And then, there's this:

https://www.androidauthority.com/apple-google-push-notificat...

Not much different to what the CCP can do to its people...

I think that US agencies are not as interested in stealing trade secrets or harming US companies as other countries would be.

I think that is a very, very naive point of view. There are countless examples of this happening - probably they're not on your radar because your nation was the recipient of the stolen goods ..

The USA regularly uses its intelligence apparatus to undermine economy and industry in other countries. I would even say, at a far greater rate, with worse results (for the targets) than anything China or Russia are doing ..

The mass violation of human rights for billions of people (literally) that occurs every millisecond of the day at Pine Gap on behalf of the American government, for example, demonstrates that this naivete is very, very dangerous.

> The USA regularly uses its intelligence apparatus to undermine economy and industry in other countries. I would even say, at a far greater rate, with worse results (for the targets) than anything China or Russia are doing ..

This was my point. How often does it use its power to undermine the US economy or industry in the US?

NSA is infamous for its involvement in stealing trade secrets and generally cooperating clandestinely to for example ensure US company wins the bid. And the competition is often supposed US allies.

It also rather infamously claimed it never partakes in industrial espionage, despite it being somewhat well documented in 1990s that they do.

Generally, if the contract is big enough, and specially if the companies involved have military/intelligence ties otherwise as well (every bigger MIC corp), you can expect ITC, NSA, CIA and occasionally FBI and others to all be involved in unethical work related to getting the contract won by US vendor.

> NSA is infamous for its involvement in stealing trade secrets and generally cooperating clandestinely to for example ensure US company wins the bid. And the competition is often supposed US allies.

This was my point. They don't steal trade secrets in order to harm US companies.

Some old Realtek switch chips featured a protocol called RRCP[1] where you could write to the hardware registers using a specific type of Ethernet frame. So I guess a CCP-designed backdoor would probably detect a specially encrypted WiFi packet and allow then internal memory of the device to be written/read over the air. The key would be hardwired into the chip, part of the random logic - so there will be no visible block to identify on visual inspection of the die.

Or more subtly they could insert (or just not fix) a bug which allows packet descriptors to be overwritten on reception of a certain malformed WiFi packet, e.g. too short or long, which makes it possible to overwrite regions of the device's memory and thus compromise it. A SDR might be required to transmit the malformed packet(s).

By the way, I wonder if modern Realtek switch chips might still support RRCP, and an undocumented EEPROM bit or strapping resistor might re-enable it?

1. https://en.wikipedia.org/wiki/Realtek_Remote_Control_Protoco...

Many modern Realtek chips still support RRCP. You just need to enable it. For example for RTL8370N: Register 0x18d6 configures the 16-bit key, 0x18d4 selects which ports can use it (0xFF for all, normally only the cpu port), 0x18d3=0x1 enables it.

You can write these registers via the management interface or via the EEPROM. It will not respond to discovery packets, but get and set packets work fine.

This chip also has a 8051 core that can access the internal bus and can tx/rx network packets. To use it you either attach external SPI flash (large program) or write the program into the internal RAM in the chip (small program).

All documented stuff, no secret details.

Thanks for that, I wonder how many switches out there have it enabled inadvertently, because of a mistake by the manufacturer?

By the way, Some of the Broadcom chips have an integrated 8051, with on-chip ROM firmware. There are leaked datasheets floating around somewhere as well. If someone has the time to dump the on-chip ROM, it would be interesting to see what's in there. Note, some of the pins marked NC in the datasheet are in fact the 8051 UART TX/RX lines.

I restrict my esps to local network only absolutely no internet access for them. No trust for Chinese products from Chinese companies

> No trust for Chinese products from Chinese companies

That's fair but Espressif is wayyyyyyy more open than ANY Western chip maker. The entire framework and toolchains are open-source for one thing. You get listings and sometimes pseudo-code for the internal ROMs (though no code). You get full access to datasheets, technical reference, and sdk documentation. Everything in their SDK is documented. You even get help on github. All of that accessible to anyone anywhere at any time.

Contrast that to the last time I worked with Nordic in a professional manner, I had to sign NDAs to get the full documentation and toolchain. Their toolchain contained binary blobs that when inquired about you get told "don't worry about it ;)" which is shockingly frustrating when a crash occurs in them and you're left trying to work around it. And if you're not a professional you're basically SOL and left with half-baked community toolchains, when they exist for a particular chip.

> Contrast that to the last time I worked with Nordic in a professional manner, I had to sign NDAs to get the full documentation and toolchain

That has to be at least 5 years ago, but even back then, 99% of their software was out in the open. Now, their SDK is open source, their official toolchain is based on the Linux Foundation's Zephyr toolchain and their docs are open and buildable. Their support is done on an open forum and complete data sheets are available both as PDFs and (with the exception of the nRF51) as web pages. They aren't allowed to publish their LTE stack because of operator licensing, their Bluetooth link layer is still distributed as a library and some upcoming SoCs aren't publicly available yet, but aside from this, they're as open as they can possibly be.

If I'm not mistaken a very significant part of that is due to hobbyist interest in the chips, not in the least by Sprite_tm (also a HN user, now employed by Espressif).

The ESP8266 essentially started out as a wifi modem, responding to AT commands transmitted over serial, but going from there to a full standalone device relied on a lot of work by enthusiasts[0] and a leaked proprietary SDK[1].

[0]: https://hackaday.com/2014/09/06/the-current-state-of-esp8266...

[1]: https://tweakers.net/reviews/7992/community-interview-sprite...

The vast majority of people who use ESP-32 smart devices, however, probably just give it the password to their WiFi.

Sure, but if one in a million people bothers to check what it is actually sending through their router, then any malicious activity would get detected and disclosed to the public - and since that hasn't happened, we can assume that it isn't happening on a large scale.

There is of course a limit to how much damage a device with such limited capacity can do.

And with the amount of tinkering that happens with these, if it were calling home someone would really have figured it out by now.

Some stick them on unrouted lans. But that may not preclude mesh like activity between restricted and unrestricted ESP-32's that are close enough to see each other.

> Maybe Nordic Semiconductors will make some nice risk-v chips and dev-boards soon.

They are, not as CPU necessarily but for exactly those auxiliary functions: https://blog.nordicsemi.com/getconnected/why-nordic-is-getti...

Tbf I think China is more interested in the money espressif makes than anything like spying. Because they'd be so easily caught out by anyone with a decent router. And if they ever happened then the whole company would be gibbed.

Lol. The Chinese fud is pretty hard.

Do you know about this

https://www.swisscommunity.org/en/news-media/swiss-review/ar... https://www.reuters.com/technology/cybersecurity/governments...

So... you are basing your fears on just that, hunches and yet presumably your own government is spying on others and maybe you but that doesnt bother you because USA /five eyes are the "good guys"

No, the general assumption should be that any nation state will take advantage of this kind of opportunity, regardless of whether they are Western or Eastern.

Thats a false dichotomy, a false equivalence- it's just a lot of false stuff

How are the two not equivalent? If anything, for anyone living in the west it's probably less of an issue to have China spy on them than a western government. Sure, at a state security level it's not but for regular individuals I sure would rather have China spy on me since they can't do anything to me directly.

You have at least some civil rights in those countries, you don't in China. Be careful of a layover.

> You have at least some civil rights in those countries

I'm sure the lads that spent decades in Guantanamo with no charges brought against them after being kidnapped based on their watch model will be very glad to hear they have civil rights.

civil rights.... haha

i live under a literal tyrant right now in india. does it really matter if i have "Civil rights" and china and usa and india is snooping on me?

https://www.onmanorama.com/news/kerala/2023/12/02/kollam-chi...

here. this is literally fresh off the oven.

26000 people's internet history was looked with a fine tooth comb in order to find that one IP which had viewed tom and jerry on youtube. youtube wouldnt give information but ISPs are by law or otherwise willing to "help" the police because its a "my way or highway" rule with the government.

so my point is, did i consent to this snooping? i am "supposed" to have laws but "think of the children". next time it is political opponents or dissidents as has been the case.

this isn't just in india or china or usa but everywhere.

The person was talking about "the West" vs China. India is not part of "the West".

>i live under a literal tyrant right now in india.

>civil rights.... haha

I wasn't even talking about you lmao.

I mean, sure yeah don't go to China. That's a pretty solid advice. But again, that's irrelevant for the average western user. Even with civil rights, the potential to get hurt by a western government while living in the west is orders of magnitude more probable than getting anything from the CCP or China. But yeah don't do layovers in China if you are a dissident, though I don't think that's an especially common scenario

what is false?

I would rather have CCP listen than other parties. I am not a Chinese citizen, so I don't care.

I use a Chinese phone with a Chinese ROM. I installed Google service as apps, with limited permissions. I'd rather have Uncle Xi listening than Uncle Sam.

you're either very naive or a pro-china supporter

The article claims that the ESP32 costs $5. The reality is around half of that for the MCU, and around $3 for pre certified modules including crystal, PCB antenna or UF-L connector. So it's really affordable.

Espressif has also launched a new ESP32C3 based on RISC-V, with modules priced at around $2.

That's the development unit price. ESP's MCUs have amazing value when you buy in bulk.

ESP32C3 has great Rust support too.

that second core is worth its weight in gold

cant wait until they can make a dual-core risc-v but p.happy on the -s3

Dual core RISC-V? Presenting ESP32-C6:

It consists of a high-performance (HP) 32-bit RISC-V processor, which can be clocked up to 160 MHz, and a low-power (LP) 32-bit RISC-V processor, which can be clocked up to 20 MHz. It has a 320KB ROM, a 512KB SRAM, and works with external flash. It comes with 30 (QFN40) or 22 (QFN32) programmable GPIOs, with support for SPI, UART, I2C, I2S, RMT, TWAI, PWM, SDIO, Motor Control PWM. It also packs a 12-bit ADC and a temperature sensor.

https://www.espressif.com/en/products/socs/esp32-c6

[The High Performance] CPU . . . has 4-stage, in-order, scalar pipeline optimized for area, power and performance. CPU core complex has a debug module (DM), interrupt-controller (INTC), core local interrupts (CLINT) and system bus (SYS BUS) interfaces for memory and peripheral access.

The ESP32-C6 Low-Power CPU (LP CPU) . . . features ultra-low power consumption and has a 2 stage, in-order, and scalar pipeline. [It has same features but lacks core local interrupts (CLINT)].

https://www.espressif.com/sites/default/files/documentation/...

oh sweet i didnt realize there was a module out https://www.mouser.com/ProductDetail/Espressif-Systems/ESP32...

i prefer using dual-core designs over one uC for WiFi + one uC for realtime UI + TFT screens, so much cleaner and less hardware to worry about

literally porting the design i'm working on rn to that one :) ty!

The section on trying to attenuate outside wifi signals interested me.

There is a bunch of hand wavy information on building faraday cages online, some people suggesting to utilize a microwave oven, since they operate at the same frequency.

There are even wifi faraday cages for sale on amazon.

However I can't really find much actual benchmark data online about how well these various approaches actually attenuate signals.

Because the stuff on amazon is mostly targeted at consumers, not professionals.

Less about protocol development and more fear of 5G and wifi.

The prices of this things can be quite high. Which is why I bought my small one as used surplus.

One can find a great amount of models from various manufacturers here: https://www.everythingrf.com/search/shielded-test-enclosures

And none of them look cheap. But more "please call for quote"-type of things.

If the product doesn't list some form of dB loss (at at least 1 frequency) I assume it's more or less just a standard box. Sometimes that'll be right, sometimes it won't. Some of it comes down to how you use the box as well. If you're feeding un-isolated power in via a big hole in the back then it doesn't really matter how perfect the front is at blocking signals.

I'm surprised their paint can only gave them a 10 dB difference. I've found simply wrapping things in aluminum foil is good for about 40 dB when it comes to Wi-Fi.

At least for microwave ovens, with domestic models reaching 1 kW and commercial (e.g. in food) 2 kW, the shielding on these needs to be pretty darn good to make sure that someone standing right next to it doesn't get their body fried. The maximum allowed leakage power density is 5-10 mW/cm² [1], according to a 2004 study even then most ovens barely get up to 1% of the legally allowed emissions limits [2, page 7].

Out of random interest I tested the "put a phone in it and call it" and it didn't work (as did wifi) that's mentioned in [2].

[1] https://www.hea.de/fachwissen/mikrowellen/sicherheit

[2] https://www.sfu.ca/phys/346/121/resources/physics_of_microwa...

Don't underestimate the dynamic range of a modern wifi transceiver. It is transmitting at ~20dBm and will receive down to at least -80dBm. You need to achieve >100dB attenuation, well beyond what a microwave oven provides.

Even a solid metal box with a few small holes will struggle.

A microwave oven is an excellent (30-40dB) attenuator specifically around 2.4GHz. The legal requirements essentially mandate 30dB + manufacturing safety margin. Your mileage will vary for other frequencies, as they're not actually faraday cages, so it's usually easier to simply use aluminum foil or a copper mesh/PVC box depending on your needs.

Thing is, 30-40dB is not sufficient to properly block wireless connections, they can still work with 40dB attenuation.

To add on to this - I just tested with my home microwave - I get 35-38dB attenuation, but I can still quite reliably get wifi packets to/from an ESP32 in that microwave.

Maybe you are looking for something like this?

https://www.mattblaze.org/blog/faraday

It's a few years old, but seems pretty thorough.

Not sure on specific numbers but based on the picture he posted of it in a can he would have probably done better with metal netting from a window screen or metal chicken wire it destroys wifi signal pretty damn effectively

For someone unexperienced with ESP32 but wanting to dip your toes, I'd highly recommend M5Stack - https://m5stack.com/ . No affiliation whatsoever, but I started playing with some basic boards last year for the first time and the the tiny devices they build have so many different sensors, transmitters, etc that you can start with a lot of early experiments just using a single device and a USB-C cable.

I can attest to the challenges of the section on Dynamic analysis on real hardware and the struggles of attenuating signal interference on the ESP.

Anyone have a recommendation on conducting fabric for RF isolation as briefly mentioned in the article or resources on the subject of rf isolation/Faraday cages for microcontrollers?

As part of the NLNet grant, I will build an affordable Faraday cage; I'll post the BOM, assembly process and a test report in a separate blog post.

I am researching that at this time as well. I have VNA etc and am keen to collaborate !

Charles@turnsys.com

I have quite a collection of research material on RF cages / testing .

There is an email address at the bottom of the blog post, you can send directly there.

We needed a large highly isolating cage for a project and the cost-effective (but still costly) way to do that was to build our own, having local welders make a custom steel box and lining it up with rf anechoic pyramid foam, otherwise the reflections make weird RF noise. A tricky part is the seams where your doors meet the box. It's easier to get good isolation from outside if you don't have to connect any wires, i.e. if any "control" you need is done by a battery-powered computer (laptop or smartphone) inside the isolation.

Based only on my own anecdotal experience... I think the hole the cables go through is the biggest problem in OPs setup. I'd solder the shields to the cage circumferentially around the hole. Shielded USB cables aren't too hard to find, it wouldn't be as good as something optical but it's a lot easier.

Feed through capacitors for power and use fiber for the rest?

If power was really the problem I'd just use a battery inside the cage.

Related, this is a neat experiment which powers a galvanically isolated circuit optically using LEDs and a solar cell: https://youtu.be/9JinSfCKuNQ?t=823&si=kBwesUsVgAGBVVwq

He didn't need to block RF though, the interface seems mechanically challenging to build into a cage.

Not directly answering your question, but on preventing emission in the first place (and the best video I've ever seen on electrical engineering): https://www.youtube.com/watch?v=ySuUZEjARPY

Lots of very useful information here, thanks for the link! Actually there's some answer: as someone else already wrote in this thread, shields of cables must be soldered to the Faraday cage to prevent leakage.

I used the isolation „fabric“ wrapped around the extension cable off a super cheap PCIe riser card. I put an iPhone inside which blocked WiFi, BT and 4G.

…I wonder if this could be used to implement AWDL (Apple Wireless Direct Link) for use with AirDrop… if I recall correctly, the blocker on normal WiFi chipsets is being unable to send the ACK frames, which this should enable?

What kind of programmer does one need to work with ESP32? I bought jlink for stm32 thinking that's the ultimate programmer for all my needs, however it does not claim compatibility with esp32.

Depends a lot on the exact model of ESP32 you get. Even for non-devboards, a lot of the newer modules have a built in USB serial peripheral. If it's got a USB port, you're already ready to go. The ESP32-C3 QtPy modules from adafruit are awesome, they've got a USB-C port on them and you can just plug them in and they're ready to be flashed.

They do have a boot loader that speaks uart, but are also compatible with jtag and openocd. However, I haven’t needed to burn the pins on jtag because they’ve implemented a gdb stub over uart, including the ability to trap interrupts and dump into a paused gdb session. It’s not ideal, but it’s pretty impressive from a preexisting tools perspective.

This doesn’t really answer your question, but it feels like it’s worth mentioning.

The commonly used cheap devboards have everything included on the board, you just plug in USB and that's all you need.

There are also 'premium' devkits with smaller and nicer packaging e.g. I like m5 devkits that are inch-by-inch-by-halfinch blocks at $7.5 for https://docs.m5stack.com/en/core/AtomS3%20Lite or with an integrated screen for $15.5 https://shop.m5stack.com/products/atoms3-dev-kit-w-0-85-inch... is useful for visual feedback.

They have a built in serial bootloader. Most dev boards don't need anything, they come with a USB-to-serial adapter. Some of the new ones even have the USB part integrated on chip.

If you use a bare module then you need a basic 3.3V serial adapter and maybe a jumper wire to ground pin 0 to enter programming mode.

You can also do stuff with JTAG, which will let you use a debugger, but that's not mandatory. Espressif sells a dirt cheap one. I think maybe you can use your jlink for that with some fiddling but I'm not certain.

> Some of the new ones even have the USB part integrated on chip.

Worth noting that this includes JTAG over USB.

I haven't tried the newer ones with USB support since I have a literal stack of older ESP32s to get through for projects, how's support for JTAG/debugging over USB in the common environments like ESP-IDF/Arduino/Platformio/etc? If it's not a hassle to get it set up then I might have to pick one up to develop on.

I'm simply launching openocd and gdb and it works well. But the only way I've ever used Arduino was via arduino-cli, so I may not be the best person to ask :P

You don't need anything. Plug it into your machine's usb, open arduino ide(or their web ide + agent) and upload any of the examples that arduino ide comes with and use its serial monitor to debug/read whatever output you write on the esp32's serial. This applies to windows/mac/linux.

A usb cord to your computer. Flash the firmware with whatever you want. Check out WLED if you have wifi.

If you get a dev kit they'll usually have a usb-serial interface for power and data.

I’ve actually found that ESPHome is sufficient for most basic use cases

The question then is where have you found ESPHome to fall short?

ESPHome is great for most use cases.

But there are things they won't touch it seems, like LoRa support.

Also I might be wrong but so far I don't think you can leverage cool things like ESP-NOW (yet).

It ends up losing contact with home assistant regularly for me. All my devices do this (all based on 8266's anyway) and all devices will fall into unavailable for a few seconds. Which is annoying because their return can trigger automations.

Honestly? Nowhere where an ESP32 was a suitable device. Anything that esphome can’t do generally requires more powerful or specialized hardware anyways. I did have some issues with updates killing WiFi on the boards and needing a reflash but I think they fixed that bug. I could also never get the chrome based flasher to work properly when I tried that out.

I've found it to be nearly unusable. The opaque firmware generation is an anti-feature IMO. It also doesn't fucking work. If you change the config, you have to force a full clean and rebuild or you'll get a corrupt image. Building an esp firmware image from scratch takes several minutes every time you want to change a config.

All of the provided controls are bad. It's incredibly difficult to get it on the network. The documentation is laughable.

It took me three hours to get a simple RGB strip working. All it had to do was drive three PWM signals. The esp has hardware specifically for that. It took a lot of effort to even find out how to use the LEDC hardware, much less how to assign a pin and hook it to the color picker.

I also wanted to hang an IR LED off it to control my TV and such. No sort of configurable button panel, you get one giant widget per action.

I did eventually get it working, but honestly it would have been so much faster to just write my own firmware to talk to HA directly.

I think Espressif have or at least used to have their own in-house developed MAC and PHY, which is not publicly documented.

For the Bouffalo Lab and Beken WiFi SoCs we already have SVD files[1] for the WiFi MAC (and likely the PHY too). Thus we have nearly complete documentation for all chip registers and their bitfields. Both SoCs are based on CEVA RivieraWaves WiFi IP.

Also you might be able to use it as a SDR for the 2.4GHz band, there appears to be registers to send ADC data to on-chip SRAM. And USB 2.0 High Speed device functionality on some of the Bouffalo chips.

I was thinking of hacking it to use as a cheap uplink to the QO-100 amateur radio satellite, which uplinks in the 2.4GHz band. I think 100mW of power might be just enough for CW or some very narrowband PSK mode.

By the way, on the Bouffalo devices, watch out for the eFuse registers, they're not fully lockable and write protectable, one wrong register write and the whole chip itself can be bricked and stuck permanently in secure boot mode. It happened to me, and I'm going to try and work around it by glitching the clock input on boot, just at the right time, to disrupt the eFuse reading, just for the fun of it.

1. https://github.com/bouffalolab/bl_iot_sdk/blob/master/compon...

> 50000 peripheral memory accesses are needed [to initialize the hardware]

Wow, that's a lot. If OP could upload somewhere the list of accesses together with a stack trace for each, I think we could crowd source a rewrite of each function - I'd be willing to bet the vast majority of those are repetitive patterns - ie. 'run this transmission test 1000 times while increasing the power levels each time until the received power = some set value'.

Headline should read "MAC" layer like it does in the article, not "Mac" layer. Two very different things :)

Oops, my bad, copy-pasted it incorrectly; fixed now

I picked up an ESP32 devboard recently. I've always been intrigued by embedded but don't have a background in it at all.

I have no idea what my first project should be. Any ideas?

The barrier to entry has never been lower. Last night I prototyped some code in Python on my Mac to talk to a Bluetooth peripheral, and then had ChatGPT translate it to Arduino C++ code for a $5 ESP32, and it mostly worked on the first go.

You can even run Python on microcontrollers these days. See Adafruit's https://circuitpython.org for which they publish modules for many (almost all?) of the sensors they sell. The modern microcontroller frameworks hide much of the complexity of Wi-Fi, Bluetooth, filesystems, etc. so you can do complicated things with minimal effort. You can really cobble something together in an afternoon.

The "hello world" of microcontrollers is making an LED blink. Then figuring out how to print a message out over serial (print debugging is invaluable). Then maybe figure out how to make a Wi-Fi connection and an HTTP request. Then go on a shopping spree on Adafruit or SparkFun for $9 sensors that spark your imagination and figure out how to talk to them; Adafruit publishes zillions of tutorials you can copy from: https://learn.adafruit.com

Make an LED blink.

Then, connect an RGB LED and experiment with PWM signal generation.

Then, experiment with network programming, accepting a UDP packet to the ESP32 that sets the color of the RGB LED.

Alternatively, you can go high-level immediately - instead of accepting a UDP packet to set the color, run a webserver on it with functionality to change the RGB LED color that you can access from any browser. Modern microcontrollers have enough resources to just spend them like that.

Are we talking lightweight servers like minihttp and shizaru, mid-level lighttpd , or big-ass Apache and nginx?

We're talking lighter than minihttp and shizaru - instead of a separate process on an OS, you'd use library that allows your firmware to respond to http requests, but that still allows you to run all the relatively complex UI/UX code on the user's computer or phone, with the microcontroller only handling the physical functionality; and reduces the need to have and manage more physical buttons/lights/screens/etc on the device itself; but you can do that without writing a separate app to generate some custom command&control messages - as in the grandparent post example of encoding RGB light control in an UDP packet, it would probably need three times the amount of code both on sending and receiving side compared to a http-based rgb control, which can probably be done in ten or so lines of your own code.

Oh, that's pretty neat. Basically trimming the server down to the routing itself.

    1. accept HTTP
    2. check for valid endpoint
    3. if yes, do thing and exit
    4. if no, 'error'

Client side has fancy UI for essentially templating per-HTTP request or command sent to the device.

I guess the only issue I see is you'd need some sort of firewalling or security. Otherwise any rando could fire HTTP requests at the thing and make it do stuff.

How would you structure this on the device's side? If a webserver's too big, then I imagine an init system is also too big.

You generally don't structure it as having a webserver, you'd structure it as an app (you run a single app on the device, there's no separate OS involved) that can react to HTTP requests - i.e. my mental model is that you don't run a webserver on the device, but instead that the device becomes a webserver.

You can structure the on-device app as 'slaved' to the web requests, where it simply waits for requests in a loop and only does stuff in response to a request - for example, take a measurement from some sensors and send them back with some surrounding HTML.

Authentication/authorization is an issue, but it has all the same issues and solutions as webapps - login+session cookies; or whitelisting IP ranges; or TLS client certificates; etc.

Pretty barebones but working web servers are possible. Example: https://randomnerdtutorials.com/esp32-web-server-arduino-ide...

ugh.. arduino.

Better to start with ESP-IDF, there's a pretty full featured well documented web server, and a lot more.

https://github.com/espressif/esp-idf/tree/master/examples/pr...

Just install ESPHome on it :)

It seems to have gone out of favour for some reason I don’t understand, but subscribing to an MQTT topic from an ESP is easy and performant.

Depending on the board and language support, a good first time project is syncing a real-time clock to NTP. At least that was my first one, followed by a wifi signal scanner. I was using MicroPython which is fast and easy, C is a little more work obviously.

Have a look at https://docs.toit.io/tutorials (using Toit).

I wrote lots of easy to follow tutorials there.

It depends a lot on the sensors you have. That said, even without any (or few) sensor(s) you can still have fun with network related applications like a Telegram bot.

There are a lot of interesting sensors out there. A good start can be by picking a sensor you're interested in and use it for monitoring. It can be as basic as temperature monitoring, or gas/CO2.

I tried making a small servo tester as a first small project on the esp32. I obviously took the servo testing code from somewhere else but yeah, the control part was useful.

> 50000 peripheral memory accesses are needed

Have you tried just replaying those 50,000 accesses and seeing if things work? Obviously some things might not be correctly calibrated, but merely knowing that a simple replay works tells you that there are no complex hardware/software handshakes (ie. Take random token from here and write it to there). It also tells you that the process is probably fairly timing independent.

Wouldn't this invalidate the FCC certification on the prebuilt modules? You'd have to get certified with this firmware to ensure you aren't violating transmission power requirements.

Admittedly, this is a non-issue for hobby scale projects, but is potentially a blocker for commercial applications.

I wouldn't say it's necessarily a bad thing, but worth discussion.

> Wouldn't this invalidate the FCC certification on the prebuilt modules?

MAC is at OSI Layer 2. FCC concerns about radio power occur at the PHY layer, OSI Layer 1:

> On the ESP32, the PHY layer is implemented in hardware; most of the MAC layer is implemented in the proprietary blob. One notable exception to this separation is sending acknowlegement frame: if a device receives a frame, it should send a packet back to acknowledge that this packet was received correctly. This ACK packet needs to be sent within ~10 microseconds; it would be hard to get this timing correct in software.

* https://pics.zeus.gent/vYXyQm2t9pJCzpDdWFvq9oWR2DACoUJoTsYf8...

It's not that simple. Besides implementing a SoftMAC, the blob is also responsible for setting up the PHY, so it would definitely be able to adjust things like transmitting power. See the part describing `esp_phy_enable()`.

If you know where to look, I'm quite sure you can already do this with vanilla ESP-IDF. I don't have a VNA or whatever is used to measure power at the antenna, but I do have a precision ammeter that confirmed my changes were effective at the supply. I only changed PHY power parameters and a lot more power was going somewhere.

Also, in my limited understanding, even changing the physical antenna can alter transmission power characteristics in such a way that it likely violates some limit, at least for some very specific wavelength or direction.

Huh, neat!

He should just go with stm and its open source LwMesh library instead.

But the closed radio parts are indeed horrible. Qualcomm (US Intelligence) and Broadcom (Chinese intelligence) controlling the physical layer underneath is as disturbing as the various Intel, AMD, ARM backdoors in their pre-OS layers.

I think bl602 shares the wifi rf/mac layer with esp32. There is a monitor mode implementation here https://github.com/stschake/bl60x-wifimon/

No, it doesn't. I know with 100% certainty ESP WiFi hardware is developed in-house and not shared. I think BL602 uses CEVA IP, not sure about that, but it certainly is not shared with ESP chips.

Sprite, how will Espressif react to this effort to open-source wifi? Will it help by releasing wifi phy documentation? Will it add DRM to block future efforts?

(sprite_tm works at Espressif)

Just a few months ago I was thinking "surely someone must have tried to RE the ESP32 Wi-Fi stack" and tried to find some research on it, but couldn't find anything. Great work!

the esp32 also has a mask ROM (which includes BASIC for some weird reason). Hence fully deblobbing it is a hopeless battle.

https://docs.espressif.com/projects/esp-idf/en/v4.3/esp32/ap...

Nothing an electron microscope can't handle. It's not a hardened security chip. It'll cost a bit but it's probably possible for a company to do it for free to show off their services. This is how die shots usually happen.

I would guess most 32-bit microcontrollers and any SoCs capable of running Linux have some sort of boot ROM, at least for the flash programming mode (especially if it's over USB).

Pretty much, yeah. You'll usually see a boot ROM, but it will also have the ability to boot into flash so you can run your own bootloader. But the ROM is always there as a fallback and can't be destroyed.

Nothing about a mask ROM makes the data unrecoverable. It's still Memory that can be Read.

I'd be very surprised if there's not an exploit that will get the CPU to barf up the full ROM contents. That's if there isn't a more direct way to read it.

Even in the extremely unlikely case that it can't be read programmatically, you can always physically decode it with a microscope and a working eyeball.

From there, it's "just" a matter of decompiling the machine code into something readable. It's not trivial, but it can be done by a single person in a reasonable timeframe.

You can just read the ROM on ESP32 (or download it, it is somewhere on the website in the SDK)

This is very interesting. I'm keen to get involved but, while I'm very experienced with ESP32, I don't have experience with this type of reverse engineering.

How long did it take you to get the environment and tools set up, so you could start digging in?

Is time or money a more valuable investment at this stage? If it's not too forward, how much would be useful to your organisation? (I can email if preferred.)

Please contact me via email (at the bottom of the blog post)

Reply on news.ycombinator.com