Yes, I think is reflected into our situation. For our TF codebase, specifically, it would be greate to deduplicate modules (which is something that couldn't be done some time ago), but there is no simple way of, say, creating a new module, and slowly migrate resources into it.
In particular (AFAIK), there are no tools for moving stuff around, so in addition to the TF restructuring, one also need to write scripts to manually move the resources.
You're looking for 'terraform state mv'. After my first handful of these it's now as natural as refactoring and moving modules in any code-base, almost.
I'm aware one can rename the resources via mv. But when multiple self-standing modules with hundreds/thousands of resources in each have to be merged into an array of modules, it's a big work.
I'm not even sure that the new resource address can be figured out, and the list of resources can be search/replaced in order to produce a single renaming (mv'ing) script.
Even if this was possible, it would likely require:
- either each module to be moved monolithically, which is risky (e.g. data sources may break, since there's no referential integrity) and requires a fully designed and implemented destination module (carrying two different representations of the resources contained in each module).
- or, and I don't know if this works in real world, creating an structured but empty destination module, and slowly moving resources from the leaves down to the root. this is a lot of work, and probably requires a very large amount of references to be carried cross-modules.
Big refactorings are a difficult in any language/framework, but in TF are particularly so, because referencing between resources is rigid, so it's hard to move small parts and their references. Doing this is Chef is much simpler, since resource name and address matches and it's under control of the developer (but Chef has a different approach, of course).
In particular (AFAIK), there are no tools for moving stuff around, so in addition to the TF restructuring, one also need to write scripts to manually move the resources.