Yep. Most of the "complexity" that is justified by mantras like this is not necessary complexity (like the inherent baseline complexity in solving a certain problem space) but cruft/enterprise/poor coding style that winds up suffocating a project. That kind of complexity should be torn out with disregard for the notion that it's somehow necessary.
I would differentiate between intrinsic complexity of a problem and artifical complexity of a solution.
Certain knots can be solved without changing the topology! They would just add artifical complexity.
Also, some truths can have both very simple and very involved proofs. They are a perfect example how just the right approach can reduce much complexity.
Just formulating the problem in a different way can already reduce much artifical complexity.
