Cloudflare et al do a lot of fingerprinting of the user agent. Any website that has 'high' anti-bot settings will return a 403 with anything but a browser. source: I've scraped lots of things.