From memory, the Georgist belief is that whoever collects the tax will spend it, and the spending will distribute the value in a way which is...not necessarily equal, but is closer to equality than the status quo was.

The wealth transfer is a feature rather than a bug, in this case. Speculators sitting on unimproved or underutilised land are seen as literal rent-seekers, and taxing them is seen as reclaiming a natural monopoly at the same time as removing the leeches who're sucking up big chunks of a city's growth.