The issue is that for some reason amp usually seems to block the content from being displayed until the javascript loads, which strikes me as the exact opposite of what an "accelerated mobile page" should do (almost like it was designed by an advertising company wanting to discourage ad blockers...). You can bypass it by activating reader mode in your browser.
Not only that, but AMP pages are only potentially perceived as 'fast' when you access them through google's search. When you do that all the AMP links' assets are pre-loaded in the background so it seems fast when you click through. But AMP pages themselves are just the same speed as anything else or slower when access without google's monopoly position pre-load.