There are several ways to do this. The best way would be to use a router that lets you route. Create the VPN tunnel as always on. Set routing policy based on destination. Gameserver.com goes out the ISP direct, everything else goes out the VPN. You can add additional tunnels and routs as needed but the building block is having a real router that lets you set routes. Personally, I have a static VPN to work, a tor tunnel, VPN to personal offsite infrastructure and an everything else tunnel. I set routes so my traffic goes the the right place automatically, for myself and anyone else in my house.
How do you figure out what to set routing policy for? You say gameserver.com, but how do I figure that out for CSGO vs Overwatch? What about games where you're connected to a friend's computer hosting a match, such as 7 days to die or potentially minecraft?
I run a custom made freebsd pfsense box, before that it was linux with iptables.
For the iptables box, I put a second 4 port intel pro 1000 nic in my normal host, ran a VM with PCI pass through so the VM controlled the nic and used that as a router.
I think there are purpose built routers out there that will do this but it will depend on the router on how you set this up. You'd have to do research on this. Pretty sure the ubiquity routers will let you do things like this. I've done similar on open-wrt.