And no, by adding new stuff I don't mean a new standard.
It's called an addendum. Which, I'm surprised I overlooked, some were made especially to address the security issues: http://www.bacnet.org/Addenda/
If security really is a big issue, the LAST thing I want is give access to the secured network to my PHONE. Is your phone secure? Would you store your bitcoins on it?
The reason that home users have ethernet and IP is because of the hourglass model. The reason that most services just work is that they talk TCP/UDP - anything below that they can ignore - so if we use a new hardware layer, if we use FDDI etc we can still use TCP because everything talks in IP. BACnet kinda breaks that - you mentioned ZigBee adding BACnet support - ZigBee has supported IP for ages - had BACnet followed the hourglass model (as literally every other internet service I have ever seen has done) then support for BACnet would have been immediate and unnecessary to trumpet.
And by supporting IP, it was also (indirectly) supporting BACnet. You just explained how services using TCP/UDP could ignore everything below... I don't understand why you don't understand it's the same thing for BACnet. Those 'official' transport layers offer you alternatives. But guess what, by supporting Ethernet and IP, BACnet can already use pretty much all the network architecture around..
BACnet doesn't quite fit into that, mostly because it replaces IP for (as I understand it) no particularly good reason. Ok, this might be the source of the confusion. Yes, the BACnet network layer needed to be defined, because at the time IP wasn't everywhere like it is today. BUT, you can easily embed this into other protocols. That's exactly how you talk about your hourglass model: BACnet devices don't care if they are connected by MS/TP, Ethernet, IP, Arcnet... They just look at the BACnet level. Just like many services look at the TCP level, not at the IP or Ethernet level.
About IPv6 for new networks, if they become required, I see no real obstacle into extending the standard for IPv6. IPv6 is itself an example of a standard changing with time. Would you prefer to throw away IP a start from scratch because IPv4 devices might not be able to communicate with IPv6?
You want standards when you have a mature market and you want to prevent lockin. Standards in an emerging market can lead to stagnation and reduced innovation(...)
But why do you consider the consumer domain to be an emerging market? For all intended purposes, you still have lighting, HVAC, access... those all already exist in bigger organizations!
There's nothing new here!