You are amazingly blasé about that fact that this would be using taxation to prevent people from using their own property as they see fit. This is utilitarianism run amok.
We prevent people from using their property as they see fit all the time via zoning. It is in fact mostly those with generational wealth that use this power.
Most people don't care about zoning, but they really do care about zoning changes that affect them.
Being told you can't live in your house anymore because it is now zoned commercial gets people riled up, and being told the same because it's now too expensive does the same.
It's a hot-button issue but you can defuse it by putting safeguards in, though they're not always successful - see eminent domain scandals.
Actually this entire construction is wrong: currently people who improve their land get penalized by higher taxes. If there are only land value and not property taxes, we are in fact increasing your freedom to make changes to your land without penalty.
Your argument ignores that changes to neighbors' property would then cause an increased burden to a land owner. This would not seem to be an improvement for freedom
It’s a finite resource. 100 acres of land being barely used should certainly come with a cost. Of all of the things that could be taxed, this one actually makes the most sense.
People aren't objecting to taxing 100 acres of Manhattan at rates that are associated with the value of the land there, people are upset about the idea that they could buy something, and then through no fault of their own, it becomes worth more and they have to sell because they can't afford to pay. Sure you get a ball of money as a consolation price, but you still had to move on.
But this is more of a straw man than a real objection; as there are various and sundry ways to resolve this (for fixed income, you can have property tax deferral until sale-after-death, you can have homestead exemptions, etc etc).
> buy something, and then through no fault of their own, it becomes worth more and they have to sell because they can't afford to pay
I fail to see why that’s a problem. Those taxes go to fund local services (like schools, police, fire, etc.) whose costs likely scale when the local cost of living and land values scale. Buying land doesn’t mean you should get to lock in the price you pay for these services for perpetuity.