You are probably actually witnessing the reduction in performance of swap as your drive fills up. Check the memory pressure in activity manager. The fix is pretty easy (delete stuff).
Ack. It’s not that then. This has been the main issue for me on my m1 air. Still a great laptop for my needs, although the ui no longer feels lightening fast like it did when it was new.