> You say conscious beings are obviously just finite state automata. That’s curious to me because it seems strange that I would essentially be a certain mathematical structure
That's a complex topic, touching on metaphysics, epistemology and the philosophy of mathematics. To summarize, explaining the unreasonable effectiveness of mathematics without simply adopting a Platonist-type ontology requires making more assumptions than simply accepting the existence of mathematical objects. At that point, it's also more parsimonious to then conclude that our world is simply one such object, rather than positing that both mathematical objects and matter exist.
> but why is it like something to be an automaton?
I think your steelman is pretty close, and you can see exactly this approach in mechanistic theories of subjective experience:
https://www.frontiersin.org/articles/10.3389/fpsyg.2015.0050...
Since this is Hacker News, an apt analogy might be how single CPU computers simulate concurrency via task switching. Our brains undergo similar signal switching between perceptual signals coming from our internal model of the world, and our direct perceptions of the external world, and this creates the illusion.
> Some kind of monism seems reasonable but I don’t see how experience can be assumed to be downstream from the mathematical laws of fundamental physics—that seems like an unproven assumption that’s philosophically tricky and extremely contentious
Absolutely. That's why Dennett has spent 50 years researching and writing on the topic. It's a tangled web that requires what he calls perceptual inversions to make it intuitive. But of all philosophers, he's probably spent the most time with neuroscientists, physicists and other philosophers. A lot of people don't like what he says, but he's definitely worth reading if this a question that interests you.