pslam

I'm saddened by the author going to this trouble. Embedded cpu selection doesn't happen in isolation - or at least, it shouldn't. The available tools are a large part of why you would pick chip A over chip B. If you don't have a C++ compiler, and you end up having to re-invent C++ in C, then maybe you should have picked the chip with decent tools.

If this project really did require re-inventing C++ in C, it must be justified by being fairly large. In which case, a low-end ARM (Cortex M based) microcontroller would have been entirely suitable. All ARMs have multiple C++ toolchains which support them.

If this is a microcontroller you end up lumbered with, and didn't choose, then fair enough, but this effort should be labeled for the hack that it is. This is not best or even good practice.

The entire point of my system is that I don't need to care which copy is good. If it scrubs and detects any error, I'll mount disks independently as two filesystems, and diff the contents. I'll then restore corrupted data from backup.

Yes, a checksumming filesystem would be better, but at the time of creation (and even now?) none of the filesystem choices were mature, or proven to be sufficiently reliable. Ext4 and software RAID1 are both mature and proven reliable.

Yeah the RAID1 part is weak sauce. However, my specific setup mitigates these flaws:

* The intention is to detect at-rest bit flips before they progressively pollute everywhere. I don't mind if it's not instantaneously detected on-access. I perform a nightly full scan - so there's up to 24 hours where an at-rest bit-flip may lie undetected, but it won't progress past that.

* I use ECC at all cache hierarchy levels possible, and ECC DDR, with active background scrub. So bit flips here won't occur (with vanishingly small probability).

* I use software-RAID1 only. The path between DDR, thru CPU, and to storage controller is unlikely to have a bit flip. From there onwards, the data is essentially written twice, so there is vanishingly low probability of both having a bit-flip, except for systematic failure for that bit pattern in two attempts to two different disks and two different controllers.

So there's still some places in the stack where errors can be introduced, but the most common areas of fault are either duplicated or covered by detection mechanisms.

My goal is to detect, but not correct, transport and at-rest bit flips. I'll discard everything when an error is detected, and use backups.

For my next setup I'll probably switch to a filesystem with a better end-to-end error detection story, such as ZFS, and ditch RAID altogether.

I run a slightly crazy setup for about 200GB of data:

* raid1 on all data. This isn't a backup - it's for end-to-end detection of bit flips in (some of) the storage path (and at rest). This needs a better solution (zfs?)

* zbackup of all selected files to external hard disk, overnight. This handles the de-duplication.

* duplicity of the zbackup dataset to S3, immediately following. This is usually a small upload - zbackup diffs are tiny and it doesn't touch files it doesn't need to.

* Rate limited so full backup is about a week, incrementals usually only a few MB.

It seems to work well so far, and I'm prepared to do a fresh full backup every year, swapping disks periodically. This general idea is to keep data in 3 physical places: local, external and remote. Local can be recovered from external. External can be thrown away and recreated. Remote recreated.

Wish there was an all-in-one solution with all the checkboxes checked. It feels silly to have to get there with a bunch of scripts.

In a sense, this is true. The view that all bugs are static, determinable, and caused by a lack of care and attention, is old fashioned and demonstrably wrong.

In mature code bases (e.g core linux kernel), (almost) all the low hanging bugs have been picked clean. There remain only faults triggered by untested configurations, unconsidered usages, and by ordering non-determinism. The last one is particularly chaotic - you can have a system which stays up for years, but falls over due to an extremely unlikely combination of events.

Think about it this way: every statically compiled program with an unbounded loop, in which there are calls to malloc() with runtime-determined sizes, and differently ordered but balanced free() at later times, may crash. This is a possibility despite there being no leaks and worst case size fits in memory. The issue is the heap, with some combinations of allocation sizes, and order of malloc vs free, will fragment and effectively "leak", eventually exhausting resources. It is generally not determinable whether this will happen to a program. They all survive at the whim of probability, although the odds are usually very much in their favor.

Seeking a mandatory backdoor is equivalent to banning encryption.

This isn't just metaphorical - it is a practical, provable result of it. The various attempts throughout history have been abysmal failures of security.

The other practical result is that the current UK government appears to believe that citizens are their subjects, whose freedom is a privilege and not a right.

It's also very common in older ISAs, especially DSPs. As pointed out elsewhere, it's common in VLIWs (which is generally "older" ISAs).

A good example is the Motorola 56K DSP family. Instructions are 24 bit wide (24 bit wide memory!). They come in a few formats, which can be broken down into two major types: complex instruction, or simple instruction plus up to 2 memory moves and address updates.

It's basically designed around performing a single-cycle 24x24->56b multiple-accumulate with two memory reads and address increments, with zero loop overhead. The rest of the ISA really, really suffers for it :) But it's impressive when it's doing exactly what it was designed to do! It's a great example of a domain-specific ISA.

Also worth noting that in 56K, there is no score-boarding or other dependency tracking to automatically insert pipeline bubbles. That's the job of the hand-writer/assembler/compiler. If you get it wrong, it simply does something undefined. It's "fun" when this happen.

> > It is a gmail account.

> You're completely incorrect. You see how this actually is confusing!

This is historically inaccurate. I have never signed up for a Google Account and yet I have one.

Before Google+ and Google Apps existed as concepts, people had GMail accounts. Then along came other Google Apps, and GMail accounts were upgraded to Google Accounts which could do more than just GMail.

Fast forward a few years, and Google+ comes along, and confused the hell out of this even further.

Yes, and these are effectively 3 different instructions sharing the same mnenomic. If you assemble them:

        mov     0x1000(%rax,%rax), %eax
        mov     %eax, 0x1000(%rax,%rax)
        mov     $7, %eax

You get:

        0000000000000000	8b840000100000  	movl	0x1000(%rax,%rax), %eax
        0000000000000007	89840000100000  	movl	%eax, 0x1000(%rax,%rax)
        000000000000000e	b807000000      	movl	$0x7, %eax

Which is basically 0x8b->Load, 0x89->Store, 0xb8->Constant.

That's a fun program. Similar techniques to this are used to create "ROP" (return-oriented programming) exploits. You get enough building blocks to get general purpose programming, and it's "just" matter of mapping to those building blocks.

X86 "mov" is a bit of a cheat, though. It's a single mnemonic for what's really dozens of quite different instructions. For example, ARM has "mov" for register-register operations, but "ldr/str" for register-memory, while X86 just uses mov for everything. Even ARM mov optionally puts a barrel shifter into the path, so it's not that pure either.

I recall a long time back, when Linux was configured as standard for 100Hz ticks (aka "jiffies"), the counter was initialized close to wraparound instead of 0.

The result was you typically encountered "jiffy wraparound" after a few minutes of uptime. You learned whether your system was stable in this situation fairly quickly, rather than 248 (or 497) days later. Kernel developers typically don't have uptimes measured in days. Starting the counter close to wraparound increased the likelihood it was going to get code coverage.

Wow! Yes, I am interested in that kind of thing.

Also a coincidence: me and my brother used to do cooperative 2-player speeed-runs of Doom/Doom2 as part of a thing called "H2HMud". We had a setup which consisted of a fast 486, and a really slow 386. It ran really slowly - about 1/4 normal rate when you woke up a room full of entities. The 486, however, rendered every frame, which meant the person on the fast machine could happily pick every off pixel-perfect, while the slower machine player would do "support" stuff (flip switches).

I wondered what the "limit" would be if you could slow it down arbitrarily, and re-stitch, and I guess that's what it looks like in the video you posted. Thanks!

Indeed, and I should have said "pseudorandom". What I'm trying to say is that it's fully predictable.

Yes, they could have used rand() with an appropriate srand(seed), communicated between machines. They could even have used a DRBG built with an AES cipher, except it would have been too slow. However, even a bad implementation of rand() usually uses an algorithm which is essentially just "seed = seed * m + a", which is slower than a table lookup.

Anyway - the interesting thing is that a 256 entry table is sufficient for game purposes (not obvious to players), is fast, and makes the synchronous multiplayer setup work.

Even the "randomness" is deterministic. The fixed, repeating table is an optimization, but it's also completely necessary for multiplayer to work.

Doom does things very differently to how more recent multiplayer FPS multiplayer works. Every player's host machine has identical state. That's not as superficial as things like score, who is alive/dead, etc - it's exact positions or all entities, their metadata... all the way down to the next number to be picked by the random number generator.

This is updated every tick (30Hz? can't remember) and every machine needs to synchronize with every other within that tick, or the game slows down. If you've ever played Doom over a modem, you know what it's like when someone has bad settings which result in high latency - in this case, poor frame rate. The data sent between machines consists of the inputs - e.g rotation, movement, fire, talk - and not higher level constructions such as motion vectors.

So it's entirely necessary that the "random" number generator is not so random at all. It's fairly hilarious to see the results of messing with it, though :)

It does make me think that cheats (not that I ever cheated) were missing a trick. If they knew what the next number to be pulled from the random number generator was, they could, for example, delay firing a weapon by a frame or two, to get more damage. Probably other more interesting things I can't think of immediately. If there was ever a bot vs bot Doom competition, this is something you'd definitely want to consider.

I've been wondering recently how viable it would be to implement an FPGA-within-an-FPGA:

  * Create a model of a simple, open FPGA.
  * Create tools to support the open FPGA.
  * Synthesize the FPGA for an existing proprietary FPGA.
  * Install bitstream in SPI NOR.
  * Ignore the proprietary FPGA from now on - work with the open FPGA that's inside it.

Yes, it would be absurdly inefficient. However, you can buy pretty huge FPGAs for very little these days. If the ratio of host:target LUTs isn't too bad, and you can find constructs which synthesize efficiently, it might yield something usable.

Perhaps it could be a way of bootstrapping an "open" FPGA effort.

I bet somewhere in the source there is a line which looks like:

    if (inclusiveEnd + 1 > size) {
        return ERR_INVALID;
    }

HTTP ranges are inclusive, and most likely implemented here with unsigned 64 bit integers. My guess is the author converted to exclusive range, then compared with size, as a form of validation. It passes the check, because 18446744073709551615 + 1 results in wraparound to 0.

The general solution is instead to use something like:

    if (size < offset || start > size - offset) {
        ... // range violated
    }

But you hardly ever see people do that.

Thanks for posting this. I'm often surprised how few people know that almost all synchronization primitives can be implemented with each other. They are effectively equivalent.

However, as you say, some end up being hideously complicated. Some are "closer to the metal" than others - both in terms of the actual instruction set implementation, and kernel mechanisms used.

I've never been a fan of semaphores, because when used with count>1, they tend to be duplicating some other information in the system, which is easy to get out of sync. When used with just count 0 or 1, it's more intuitive (and closer to underlying implementation) to just use a mutex. And as you point out, the various mutex incarnations usually have an easier time of tracking ownership and debugging, by their nature.

Bingo :)

Actually, not just embedded systems. There's HSMs (hardware security modules) which also can't be updated to support new functions. Often this is because the underlying primitives have been implemented in fixed-function hardware to prevent timing, power and even RF analysis.

There are installations which don't support ECC, usually due to hardware limitations, but do support RSA-4k. Often this cert is the trust anchor for various other uses than just SSL, so the result is you end up using RSA-4k instead of ECC.

There's also folks who don't entirely trust ECC because it's "too new". To be fair, RSA isn't broken, and 4k is sufficiently big to be safe even with a partial break.

TCL is an incredibly useful language to quickly construct small code snippets in, and is especially good as a REPL language - as a command line interface.

On the other hand, it's the bane of my life. It's particularly bad (and verbose) at anything involving expressions. Unfortunately, that's mostly where I find it being used, because it's very popular in hardware. You'll find it being used for emulators, simulator, FPGAs, JTAG interfaces, and so on. There, it's shuffling a lot of data around, and in particular performing bitfield operations. It's notoriously verbose at doing all of that.

Hardware register definitions and the like are also usually expressed in a C-like fashion - that is to say they're usually using square brackets e.g "some_reg[2]" notation. This is annoying for TCL, because square brackets are its "string eval replace" construction, so you end up escaping stuff.

Its error traceback, usually "puts $errorInfo", is also awkward and often doesn't give you the right locality of where the error was, especially if it's syntax-related. One of the most infuriating aspects is that the comment character "#" doesn't obey the rules you see in other languages - it's not like a pre-processed character. Scope rules still apply within comments, e.g you can open a proc definition and it'll complain about a dangling brace because it still parses the comment line contents.

The everything-is-a-string design is awesome but also makes it inherently slow, to the extent that even on a modern system it can end up being the bottleneck in execution. It also results in really weird parameter passing rules between functions. I'm not a fan of having to occasionally add magic intermediate string evaluation steps, or the wildcard expansions necessary for passing lists. It's error-prone and sometimes feels like just hammering the interpreter until it Does What I Mean.

Everything-is-a-string is also damn dangerous, and can result in arbitrary code execution if you're not careful. Anyone who's ever run an IRC script or bot has probably experienced this, and knows what I mean.

That's not to say Tcl is crappy. It's just a language which was designed in another age, and it's really showing it. It's awesome as a command line REPL, but I highly recommend something else (Python or Lua) if you're doing anything even moderate scale behind the interface.

It is very expensive to hire an H1-B compared to a US resident. The only way for it to be cheaper would involve fraud: paying a worker less than prevailing wage, which would require cooking your books, lying to US Immigration, and getting into a lot of trouble. (As is the case for a few big out-sourcing companies recently)

Filing fees are high ($5k+ over 6 years), as is the cost in lawyers (probably similar), auditing, and time consumption. It is impossible to keep an employee on H1-B indefinitely, so generally they will also go through the Green Card process, which is also expensive and time consuming.

Generally companies will offer a relocation package, which is also very expensive when it's international, and have partnerships with banks (having zero US credit sucks) and other services to get employees settled.

The idea that hiring an H1-B is a cost-saver is utterly bunk, and needs to stop spreading. (Other than fraud)

It is illegal to pay an H1-B Visa Holder differently to the prevailing wage. Converting to a Green Card should have no effect on your pay, and if it does, the employers are committing fraud.

You may have a side-effect of increased pay due to having more positions open to you. Holding an H1-B restricts you to a specific job description. For example, if you were a plain ordinary "Software Engineer" with nobody reporting to you, then you are not normally allowed to move to a role where you would have reports. This can be overcome with due-process and lawyers etc, but the basic idea is an H1-B worker is coming to the US to fill a job role not just a job.

So, when you convert to holding a Green Card, you can get promoted into positions which weren't previously open to you. You get the pay increase by being a better engineer and getting the better job.

Thanks for making that!

I hadn't considered the rotation, because as you point out, it doesn't exactly match up. It's gaining distance from the center of image as it moves.

It could still be some kind of internal reflection artifact (even dust?) which varies in distance slightly, I guess. I'm pretty sure the team looking at these images would have gone over every object in depth, so it's unlikely I've spotted something they haven't :)

There's another object near the left edge of the video, moving from top to bottom. Does anyone know what that is?

Its movement per frame seems too consistent and predictable to be an artifact (noise) of the camera.

That's exactly the threat full disk encryption is designed for.

Sorry, I wasn't clear enough that my code was intended as sarcastic. It's obviously silly to zero variables because the compiler is free to ignore you. The point is, the underlying machine is going to do the same.

There are many ways to dig out stale memory if you're running at sufficient privilege, for example. Direct cache introspection, for example, or bypass. Zero-izing alone is not sufficiently strong to mitigate the threats people imagine it works against.

So you're essentially trying to protect against a hypervisor attack? That's never going to work unless you put the primitives in the hypervisor. Assuming you own it.

Thanks - that's a good example of what I was trying to convey.

The point is Rust already provides safety guarantees. If you don't trust the runtime, then why would you trust the built-in zero'ing? I get the "defense in depth" argument, but it feels a bit like doing this:

    {
      int a = secret;  // Get secret.
      assert(a == secret);  // Check "a" is actually that.
      a = 0;  // Ensure "a" is zero'd on exit.
      assert(a == 0);  // Just because.
    }

And yes, I get that you can build this into the language so it's not quite as ridiculous - you actually wipe tainted stack, for example.

But the point is: the runtime has an ABI and a machine model. Information is allowed to leak across function boundaries, beacuse it doesn't matter. Without using the "unsafe" keyword, there are no methods of getting around the machine model and dipping into the underlying actual machine.

Even if you don't have a "safe" language and runtime, it's still of limited value. It protects against threats involving data or control flow corruption after key usage, and where there isn't sufficient control of the program to perturb the secret-consuming functions. That's more of an annoyance than prevention. On the other hand, it gave the programmer a false sense that it was properly wiping secrets.

This keeps coming up, but I think it's a very, very bad idea. It's false security.

If you are running in an environment where you don't trust code running in the same compartment/sandbox/process, then it's futile to zero out memory. The caller could have prepared things such that the memset doesn't work, if the key material went somewhere else.

If you ever find yourself thinking you need to do this, what you instead need is a helper process who's only purpose is to do primitive operations with sensitive key material.

Particularly as Rust is already a "safe" language - it doesn't even make sense to zero memory which by definition another piece of code can't access. Unless there's declared "unsafe" code lying around, but you wouldn't put that in the same process, would you? At which point, what are you even protecting against? If an in-process threat is that advanced, then you're not achieving anything.

This is probably a bad idea and I would bet (and probably win) that less aggressive unrolling would be better in pretty much all cases except a micro-benchmark.

This kind of trick worked well on CPUs in the 1980s (as jvoorhis points out in another comment). It works wonder for micro-benchmarks. It is a net loss for any CPU and workload where instruction cache miss rate can severely impact performance. That is to say, pretty much everything made after 1990s and doing more than just shuffling memory around.

The problems are: 1) this is a large fraction of I-Cache size, 2) unrolling doesn't actually save much (or sometimes anything) on out-of-order multi-issue cores.

First off: extremely aggressive unrolling results in increasing I-cache thrashing. This is usually not apparent in micro-benchmarks, because the entire working code set fits in the I-cache. It becomes apparent if the unrolling is stupid enough to be larger than I-cache (or usually just close enough to it), because then you're spending just as much (usually more) time re-fetching instructions from memory as you are moving data around. That's why picking the best result from a micro-benchmark is misleading. In real use cases, you find that functions like this one tend to cause a "global slow down", with no specific function accounting for it, because they're greatly reducing the effectiveness of the I-cache.

Also, unrolling doesn't really do much these days. You can do better than REP MOVSD and friends - go read Intel's documentation for their recommended sequences (which has a nasty habit of changing every damn CPU generation). Modern out-of-order, multi-issue cores can very easily saturate their load/store units, while also executing all those pesky loop counters and branches. In fact, that's the whole point of being out-of-order and multi-issue: keeping units busy.

So, please can everybody stop using Duff's device, unless: 1) you're targeting a CPU which has no I-cache (e.g tightly coupled ROM/RAMs), 2) you really do have a working set where this fits well enough to be better, 3) you're trying to cheat on benchmarks, or 4) you're trying to obfuscate your binaries against reverse engineering.