You’d need an RDP-style protocol to do that, and at the cost of the increased latency this brings.

And now think about what this will mean when you’re actually doing RDP on top of it.