That is a factor, but if you use a clean install and don't connect to the internet outside of the VM then there's little chance of getting infected.

Whonix does has the possibility of keeping the gateway on another machine.

You could in theory keep the VM on a flash drive and boot a liveUSB/CD with virtualbox already installed, thus running Whonix in "live" mode.