As long as it's not connected to a network I guess. I wonder though is someone still scanning for Win95 machines? Maybe it's safer to use it these days than it ever was during it's active period.
I am curious to know what security holes you can exploit in a fully patched system with a firewall.
Depends on what you're letting past the firewall. Even ICMP requests could (not sure if this was ever patched) be used to DoS a Win95 machine. There are several open TCP/IP bugs with Win9x, the newest one being from 2008.