Is there anything in all this that can make my machine behind the router get the same address all the time? Do I manually assign it locally as with IPv4 and hope no one else in the building assigns the same IP?
Is there anything to assign IP-to-Hostname at least on the local (behind the gateway) level. Or is this handled via some other service I need to run (maybe some server that runs dns and allows machines to push their desired hostname to it).
On the machine I'm writing this reply on, when I boot it up (with either FreeBSD, Linux or Windows) it gets a router advertisement from the router (which is routing both IPv4 and IPv6). It gets a standard NATed DHCP lease for IPv4. For IPv6, it combines the /64 network prefix assigned by my ISP with the NIC MAC address to compute a unique (and stable) address.
There are of course alternatives. I can assign it statically, I can use privacy extensions to periodically change the address (seems to be the default on Windows for me) or I can use DHCPv6 to use DHCP-allocated addresses (my router doesn't support this though I have done it in the past with BIND).
one caveat: DHCP6 is younger than IPv6, so not all router firmwares will have gotten around to implementing it yet.