This joke is not just applicable to the ZX81, in fact the general idea of a RAM shared as video RAM and CPU RAM lives on today, just much less noticeable. And as an older example, the Commodore Amiga performed much worse CPU-wise if you used highres, you had "chip mem" and "fast mem" as a consequence etc.

The ZX80 and ZX81 just took a somewhat, ahem, less subtle (but efficient!) approach to get those cycles to the CPU :)