How a robot got Super Mario 64 and Portal “running” on an SNES | Hacker News Reader

How a robot got Super Mario 64 and Portal “running” on an SNES

arstechnica.com

172 pointsminimaxir36 comments

I watched this live on Twitch, it was unbelievable. I forgot for a time that they were still on the SNES - the streaming of the video was that good. Low quality, but good enough that you forgot you were in the "TAS OS", and it wasn't just Twitch streaming artifacts.

The TAS blocks at the Games Done Quick fundraising events are always amazing, but this was a step above.

The way they did it was a bit confusing. They were playing Zelda, and then they cut the game off right as they walked through a door. Shortly after, Super Mario N64 popped up on the center of the screen. Twitch chat wondered if they were still playing Zelda and playing SM64 through the Zelda game. It wasn't clear what was going on, and they saved the explanation until the very end.

Yeah, I think the effect would have been better had they explained what was going on immediately rather than saving it until the end. Very cool tech, regardless :)

It helps that I've watched the previous 3 or 4 TAS blocks as well; the breaking out of a game and owning the system after a few moments of playing the game is a common and repeated theme.

Wow. Just wow.

Being unfamiliar with this area I can't tell if this is ridiculously good or just above average. Nevertheless, for _me_ this was an amazing display of both hardware and software hacking.

This kind of falls into the same category of "out of the box" thinking that is used by NASA ground crews to squeeze more and new functionality out of old and distant hardware that far exceeds its original design parameters.

Loved it.

This is pretty insane. https://arstechnica.com/gaming/2017/01/prepare-your-switch-t...

> > TASbot... Gods above and below, that's some mighty fine hacking to translate inhuman button presses into actual glitch code. It's like waving your hands madly in the air and conjuring up an airliner, Matrix-style.

> As TASBot's keeper I can tell you that what we have prepared this year is akin to gesticulating wildly and calling a space shuttle into existence out of nowhere. :)

I would say that this is what hacking is all about :)

> While the AGDQ SM64 and Portal TASBot runs were really just pre-recorded video and audio streaming through the SNES

phew! I was totally confused -- assumed they were somehow rendering and doing the actual gameplay.

Anyone know how / why they're able to send this much data over the controller ports? That seems like a sample rate that's way into the "insane" realm, irrationally beyond what any human could produce.

The live video includes a caveat that they "cheated" by networking directly into the console for several orders of magnitude more button inputs. (i.e more than the 8 controllers from the console)

The cheat was apparently controversial for the team.

For tasbot though, I wonder what that actually means. If they soldered onto the board to bypass a chip or something I can certainly see it being controversial - they're usually very clear about "this system is unmodified, we just have a bot pressing buttons".

It's less button presses at this point and more putting data into the memory where controller input is stored at an arbitrary rate.

Talk from Defcon: https://www.youtube.com/watch?v=9ZrNWiBzqUE

Aaaaaah. So that lets them update the value (up to) every time the memory is accessed - that I can definitely see running so quickly. And still "using controllers", arguably.

Still. Much amaze. Wow.

(too late to edit) hmm, no, that video talks about how they did the twitch chatroom thing. As I interpret it, they did that by tricking the snes to re-poll the buttons faster than it would normally do, and they said they could achieve every couple ms (instead of once per frame), which is wildly below what the streaming video demo needed.

The video is awesome though, thanks for the link! Now ya've got me queueing up tons of defcon videos....

Hm, I might be remembering wrong or mixing it up with details from various forums, but I thought it mentioned the FPGA version capable of pushing several mb/s through the controller ports.

Yeah, they mentioned it exists (2mb/s iirc) but made no attempt to describe how it works.

The system is unmodified. They effectively bypassed a chip inside the controller.

I interpreted it as meaning they removed the parallel-to-serial shift register in the controller (or at least stopped simulating it):

http://www.gamesx.com/controldata/nessnes.htm

The SNES has hardware to generate the P/S signal once per frame, but as the SNES CPU was originally intended to be backwards compatible with the NES it still has the old NES-style bit-banging mode ("old-style joypad registers"). If you remove the shift registers and drive the data line directly you can get higher bit rate. The SNES itself stays unmodified.

Relevant quote from the article:

"By polling four simulated multitap inputs at 300 times per frame using a method devised by p4plus2 (Jordan Potter), the TASBot team is able to effectively send data to the SNES at 1.15 Mbps"

1.15Mbps / (300 samples per frame * 60 frames per second) = 64 bits per sample, or 16 bits per input, which seems like a likely figure.

Out of work, high apm Korean professional StarCraft players using custom controllers built with low actuation force, short travel distance buttons. They could have had a slightly faster input rate, but they had to assign 0 to a button instead of no-button-press because the players were always instinctively pressing something. Some of the best players were worried that it would hurt their game because they had to slow down to synchronize with the slower players.

[1] http://www.nbcnews.com/technology/how-fast-fast-some-pro-gam...

[2] https://slingshotesports.com/2016/10/28/the-demise-of-korean...

10s of events per second is a far cry from tens or hundreds of thousands per second. They're pushing 1.15Mbps (see other comments) over only 4 controllers. That's something on the order of every pro StarCraft player at once on a single machine released in 1990.

It was a joke about the speed of StarCraft players... I was hoping that the fact it was obviously impossible would have tipped people off...

>They could have had a slightly faster input rate, but they had to assign 0 to a button instead of no-button-press because the players were always instinctively pressing something.

What? I can't parse that sentence and I can't find anything like that in the articles you linked.

I'm told that top StarCraft players keep up a steady click-stream even if there is no need to click, they just keep clicking the same place. I assume that's so that they get into a rhythm or something. So ideally having no buttons pressed is a zero, and have any combination of buttons is not zero. I think the SNES controller has six buttons, plus one of up/down/left/right, so you'd get 8 bits. But if they always have to be pressing something, then you'd have to assign one combination to 0, so only 7 bits. (Probably I did the math wrong or something there) It's a joke, anyway, so if I have to explain it, it's obviously not funny.

Oh, okay. I thought you were talking about starcraft players as background info for controller limitations, not literally saying they were hitting the buttons. The part about "hurting their game" made me think you were talking about playing starcraft.

And requiring that at least one button/direction be pressed doesn't meaningfully reduce the data rate. You don't lose an entire bit, you just avoid that specific combo.

> I'm told that top StarCraft players keep up a steady click-stream even if there is no need to click, they just keep clicking the same place. I assume that's so that they get into a rhythm or something.

Yeah, the reasoning is (basically) like the rotation speed of a disk drive. The faster your spin - the higher your response resolution. Instead of taking (up to) a second (for a 60 apm player) to respond you take 60/400 second (for a 400 apm player).

There are of course other factors like that 60 apm players don't simply play at a 1 action per second interval and have faster reaction times than that.

This is the Rube Goldberg Machine of programming.

I wonder if early programming projects weren't like this. Abusing low level components to do things they weren't really meant to do.

That accurately describes the Commodore 64 scene. Games, demos, disk speeders, and copy protection particularly abused the hardware. Many games couldn't run without hardware glitches from the video chip, and undefined CPU opcodes which end up executing some weird fusion of otherwise normal instructions were often exploited to save a few necessary cycles here or there.

True; but I meant even 50s computer projects. They were trying to make memory out of mercury acoustic waves .. feels so hackish in a way

Wait, was that real? I thought Stephenson made that up...

Unless there's some misunderstanding:

https://duckduckgo.com/?q=mercury+delay+line&ia=web

How far into the video does TASBot begin?

The actual play starts at 29:00 (https://youtu.be/Ukq29ePnTqI?t=29m00s), but the video streaming through the SNES starts at 55:50 (https://youtu.be/Ukq29ePnTqI?t=55m50s).

Thanks for the timestamps. That's incredible!

During this stream[0], the TASBot setup is reused for a remote-playable version of Stack-Up. There's an earlier stream where DwangoAC pitches how it would work, which I'm having some trouble finding now.

(I recommend that streamer, too. His main project for several years has been to play all NES games to completion.)

[0] https://www.twitch.tv/themexicanrunner/v/109380650

Reply on news.ycombinator.com