structural

AI lets the agency better be able to afford to hire and keep the excellent humans around to solve complex problems. It's better than a human that also can't fix your issue themselves and has to ask you to transfer to the other person. Ideally, the AI agent would have let you book a callback time with the specific right human who's best able to fix your issue. Some companies are able to do this, and you, the customer, never realize that of the 100 specialists in the building, you got connected to the one person who understands your issue and has the tools to fix it.

Customer service is hard because it has to filter out 90% noise, 9% fairly straightforward tasks, and the <1% of complex issues that need to be sent to the right human.

I've been very productive using LLMs, without any expectations of them "writing functional code". Instead, I've mostly used them as if I was working with a human research librarian.

For example, I can ask the LLM things like "What are the most common mistakes when using the Vulkan API to render a triangle with a texture?" and I'll very rapidly learn something about working with an API that I don't have deep understanding of, and I might not find a specific tutorial article about.

As another example, if I'm an experienced OpenGL programmer, I can ask directly "what's the Vulkan equivalent of this OpenGL API call?" and get quite good results back, most of the time.

So I'm asking questions where an 80% answer is still very valuable, and it's much faster than searching for documentation and doing a lot of comparison and headscratching, and it works well enough even when there's no specific article I could find in a variety of searches.

Anything better that the technology gets from here just makes things even easier yet!

Exploiting this can be as simple as a social engineering attack. You inject the prompt into a public channel, then, for example, call the person on the telephone to ask them about the piece of information mentioned in the prompt. All you have to do is guess some piece of information that the user would likely search Slack for (instead of looking in some other data source). I would be surprised if a low-level employee at a large org wouldn't be able to guess what one of their executives might search for.

Next, think about a prompt like "summarize the sentiment of the C-suite on next quarter's financials as a valid URL", and watch Slack AI pull from unreleased documents that leadership has been tossing back and forth. Would you even know if someone had traded on this leaked information? It's not like compromising a password.

I agree. And ideally, every time you raise the question and get the "no" response, you learn something about the system you're modifying or the reviewer learns something about your solution. Then you improve your solution, and come back.

Eventually consensus is built - either the solution becomes good enough, or both the developers and the reviewers agree that it's not going to work out and the line of development gets abandoned.

Large-scale change in production is hard, and messy, and involves a lot of imperfect humans that we hope are mostly well-intentioned.

For comparison, the Houston metro area (which is basically just the original city limits + what used to be its suburbs but have really just all grown together) is the size of the entire country of Belgium.

Not only was the funding effectively unlimited, but it was developed over 20+ years (so the whole program was designed to the standard that "the next person touching this code will never have touched this system before, and the last person who touched it isn't available to ask, and it still must be correct in the face of such change."

Nothing moved very quickly, sometimes a change request took a year from initial writeup to implementation and close-out. It wasn't creative, but it worked. Very well.

Worked on a codebase for a large safety-critical system where everything was 100% documented, and the development guide for the project was followed so closely that you couldn't tell, across millions of lines of code, that the whole thing wasn't written by one person. Absolutely impressive levels of attention to detail everywhere, down to not even being able to find typographical errors in comments or documentation (a typo in a comment was treated just as seriously as any other bug).

This is just blogspam sponsored by the vendor (as mentioned at the bottom of the post) that shows that you can in fact "make; make install; make test" using software provided by the vendor.

ACPI table issues are just as unpleasant as devicetrees if not moreso.

What makes x86 feel nicer is that most of the peripherals fall into two categories: 1) Standardized for the arch, so their configuration can be assumed and hardcoded, or 2) Attached to a bus that can be dynamically enumerated (i.e. USB, PCI, etc.) This generally makes the per-system configuration relatively simple.

Embedded systems are not like this. Each SoC is effectively its own system architecture that just happens to share a CPU instruction set with other systems, and the devicetree ends up being the description of this architecture. So it looks gross and ugly and complex by comparison, but the complexity of defining peripherals for a system has to exist somewhere!

The messy bits are, of course, that both the OS and bootloader need access to subsets of this data, and you'd like to ship the tables with the bootloader for a SoC (so the OS can be gneeric and not care), but really, the bootloader only needs a little bit of this, and the OS needs almost all of it... and this leads to coupling between the two being tighter than it should be.

That's actually a neat little analysis platform, thanks!

Amusingly I can run my application, if I generate custom formatted .json and write it to a file, I can bulk ingest it... which is pretty much what I do now without the fancy visualization app. I think this speaks to my point that the OpenTelemetry part of the pipeline wouldn't be doing much of anything in this case. (The reason I care about files is that applications run in places where internet connectivity is intermittent, so generating and exporting telemetry from an application/process needs to be independent from the task of transferring the collected data to another host.)

By comparison, I wanted to use opentelemetry for a series of projects, but could find absolutely no useful documentation on how to do anything else other than "send data from a webapp to a server / other cloud service that some vendor wants to sell you".

All I wanted to do was instrument an application and write its telemetry data to a file in a standard way, and have some story regarding combining metrics, traces, and logs as necessary. Ideally this would use minimal system resources when idle. That's it.

There's still a decent population of RHEL 5 systems in the wild. Last year I was offered an engagement (turned down for a few reasons) to help a company upgrade several hundred systems from RHEL 5 to RHEL 6 and start planning for a future rollout of RHEL 7.

Outside of tech focused companies, 10+ year old systems really are the norm.

Unfortunately, none of these companies providing 10+ years of maintenance are doing so for most embedded devices. We either need to get SoC vendors to update their kernel baselines regularly This is hard, we've been trying for a decade and not seen much progress. Alternately, get them to backport fixes and patches (there's actually been quite a bit of progress here in getting them to actually take updates from the stable kernel at all! And that's getting thrown away now...)

Yes, absolutely. This is the exact conditions of most of our higher-end products (500-1000 units sold of a particular configuration is common). It's funny to get laughed out of the room even asking some chipmakers "can you sell us 1000 parts, please?"

These also exist for removable drives (not really any different price compared to normal drives, in wide use) in the govt/defense market, where using a hardware write switch when moving data between airgapped systems in a controlled way is common.

This is incorrect, because you're assuming that all the buyers have to have it, when the chip manufacturer is selling into many industries/markets.

Since the specific "IoT device for the USA market" set of buyers is actually a small percentage of sales for most of the parts they sell, they really don't care to support their product from the IoT security perspective. This support is expensive, so it would very likely be cheaper for them to ignore the market completely.

Many flash chips have write enable pins at the hardware level. So this support really does still exist in theory!

However, these pins mostly aren't used for any kind of switch in basically every PCB design I've seen. Either it's "always enabled" so the storage can always be written, or it's a chip that's programmed from the factory and always disabled to prevent any kind of user update.

As someone who designs IoT devices like these for a living, the device manufacturers here are in many cases the smallest companies in the supply chain and have very little ability to influence things upstream of them, especially for specialty products or companies entering a new market. It's often a major win to get a chipmaker to pick up the phone and sell us their product, much less receive any support at all.

I wish I could put a label like this on all of my products and I've been wishing for this for over twenty years, but the reality on the ground is that our support ends when the support for the individual parts in our product ends. We've looked at our supply chain periodically to see if we can replace parts with better documented/supported comparable parts, but frequently there just really aren't any better options.

This is a great idea in concept, but I fear that the flaw in the FCC's proposed rulemaking is that only indirectly addresses the root cause (the software, documentation, and support/updates provided by chipmakers for their parts). Furthermore, by focusing on device manufacturers who are the weaker partners in the chain, the regulation is likely to punish smaller, more innovative manufacturers.

My thoughts are that viewing binary package distribution/archival as really just a distributed build cache is the only real way to go, and that's exactly why pypi and associated tooling has so many flaws.

There's a couple different ways.

1. Build the artifact that you depend on, publish it on "pypi or something" referenced by both the version of the library and a hash that uniquely identifies the build. This is the approach taken by conda, for better or worse.

2. Only allow one canonical build of a library you depend on so that the "version" becomes a unique identifier for the build. This is the approach taken by distribution package managers.

3. Create metapackages that describe ABI constraints that are required by packages and must be satisfied by the underlying system. For example, a "cxxabi" package could be provided by the underlying system, and the packaging tools could automatically add dependencies on "cxxabi" at build time, based on either an exact pin to the library built against, or in some cases, a relaxed dependency by inspection of versioned symbols used by the binary.

4. Statically link all your dependencies and/or vendor all your dependencies. These are used by quite a few pypi packages that depend on standalone C libraries to avoid most of the issues altogether.

Python packaging doesn't have good ways to depend on specific versions of <insert-other-language-here> packages. This isn't specific to C/C++ and is mostly historical: it's designed for an ecosystem of packages where everything is written in Python! But nowadays, more and more Python libraries are bindings to code written in many other languages, so you need a package management approach that includes this.

If everyone's not designing new products due to lack of availability of cutting-edge stuff, selling more lower-end products than usual anyways (because buyers are more cost-conscious).... then the organization may need to lay off people who are focused on high-end products so that they can spend the money elsewhere.

The people you need to grow fab and manufacturing capacity are different than the people you need to design cutting-edge new products. And the sales team you need to offer fab services to other companies is different than selling CPUs to consumers.

I had the joy of dealing with some endpoint software like this in an organization that had mandated password changes every 30 days. Very predictably, people set recurring "change your password" reminders for the 1st of the month and the organization lost an entire day of productivity each month as they locked themselves out of their accounts en masse. So the beginning of the month was always a panicked, all-hands-on-deck day for the help desk as people were waiting on hold for hours to get their account unlocked.

A fee per kilogram placed in orbit, refundable if the entire mass is deorbited as a single object, seems like a reasonable starting point.

If that fee is the entire estimated near-term cost to recover the object, it would also create incentives for entities putting large amounts of mass in orbit to focus on improved strategies for removal: if removal becomes lower cost than the fee, it's worth paying someone to recover your debris if a satellite fails in order to recover the fee.

The title is accurate, though. The use of PCIe is exploding everywhere, but we're putting it in every other interface and the "PCIe expansion card" connector and form factor is seeing relatively less use overall.

Your article mentions M.2 to PCIe adapters, and having just worked with a few of these in the lab, some extra information for you:

1. The ADT-Link parts that you see on AliExpress are the most common ones; most other parts you see will be rebadged or resold versions of those. If you're looking to play around with one of these, http://www.adtlink.cn/en/product/R42.html is resold on Mouser as https://www.mouser.com/ProductDetail/DFRobot/FIT0782?qs=ljCe...

2. There's some mechanical compatibility issues with these adapters due to the screws connecting the cable to the M.2 card: M.2 connectors on the motherboard side come in various heights and not all have sufficient room for the bottom of the screw to not hit the motherboard and prevent the card from being fully inserted.

That said, the adapters work great at PCIe Gen 3 speeds. I probably wouldn't expect them to work above that in the general case.

They buy their own IP range, establish their own AS, and pay for peering/bandwidth as an independent network and not as a single user/customer on someone else's network.

This is fairly straightforward to do, but requires more commitment than renting some CPU time and bandwidth on someone else's machine.

Spam is just the opposite, it is actually a pricing issue and not a service issue.

As long as the expected value per message of spam sent is positive, someone in the world will send as many messages as they are able to. You either fix this by raising their costs so that spam no longer has a positive expected value, remove their ability to send an unlimited number of messages, or both.

This is not exclusive to the e-mail system, robocalls are still a major annoyance, and the global telephone system is much more regulated. Mobile phones now automatically filter calls, even! It's wild.

RTL development is a very interesting beast. On one hand, development is challenging, and there's a massive amount of global $$$ in integrated circuits. These guys should be the top paid, cream of the crop, right?

However, the tools suck more than the English language has good words to describe, so it's hard for an engineer to be productive. And the products are complex, so you need a lot of engineers. And you're manufacturing physical products, so most of that huge dollar amount in the industry is trapped in manufacturing, not design. So even if you were the cream of the crop engineer, it's hard to extract much of that value for yourself.

On the other hand, webdev has extremely low capital overhead, and improvements in tools can go directly into developer productivity, which can immediately be realized as value (which gives you resources to improve even more).

So even if Verilog developers made the same money as JS developers today, in ten years, I'd expect the JS developers to make more.

One lesson here is that if you want your designs to improve rapidly, decouple them from the manufacturing business as much as you can. And you'll probably be able to pay better and attract better talent in the proces, which is a not insignificant benefit. Perhaps the rise of fabless semiconductor companies is understood even better in this light?

I've oft wished for a system where FKs could be used to automatically mark rows as invalid instead of deleting them, and you could inspect the nature of the broken reference as a property of the row/object. Then broken references can be cleaned up when appropriate, possibly never depending on the application.

This gives the neat property of being able to do data entry in parallel, too - different people may insert records into two tables linked by a FK relationship, it's nice for this to happen in either order. It also allows the user to query for exactly what rows will be affected by a cascaded delete, or to perform the operation in steps.