I mean I'm biased since I did a physics undergrad _and_ took additional math classes, but from what I'm aware of, even top universities require less math from CS majors than they do for even the easiest Engineering degrees. Calc II (series, not multi-variate) is not "high level" in a modern context (maybe in a general social context) and + Linear Algebra is nowhere near "a lot" unless you compare to majors who do not even take calculus.
So pretty much: when comparing to other STEM majors, I'm given the impression that CS majors are near the bottom of math requirements. At both undergrad and graduate levels.
I went up through Calc III, and of course PHYS I and II w/ calc is a barebones requirement. Discrete, linear, stats, the basics there.
I think it varies per college. Calc II was the very first class I took on the very first day of my freshman year of college. It was really a decent bit of work but not unbearable.
Most CS programs are pretty math-intensive and this seems to be the general community consensus, I'm not sure where you're getting this impression from.
I mean specifically in the context of STEM degrees. I was curious about my intuition so I did a quick look. Used Stanford as a baseline and their sample plans if available. CS looks on par with many but slightly south of the median since several engineering degrees require more math. The ones that require less are "Management Science & Engineering" and "Materials Science and Engineering". But typically similar to chem, bio, and most engineering degrees, which all have more science courses which can teach additional mathematics (generally in calc and stats). Being close to the median I wouldn't call this "a lot," even if there is low variance in that distribution. Where I'd probably draw that line is beyond the general 3 calcs and linear algebra. Personally I'm generally unimpressed with most intro stats courses.
Additional note: I can at least attest that in my university Calc 3 (multi variate) is not a requirement and this does not appear to be out of the norm. Interestingly side note, same with ethics which is exceptionally common in other stem programs (I believe a federal requirement in engineering).
CS majors learn a lot of math?