In Holden's mind, yes. I'll assume you've read the book - if not, it really is a fun one to read, not stuffy in the least. But Holden is really nervous of women and of sex. There's the famous (in my mind) scene with the hooker, where he chickens out of having sex with her, because he reasons, perhaps rightly so, that sex with a hooker is unclean. But that's the attitude he has towards sex in general, not just towards hookers. (You could call it ironic that Holden's right in his particular case, but I haven't read the book in years so I couldn't make a strong argument for that.)

To Holden, being this pure, innocent child figure is the ultimate manner of living, and that's why he loves Phoebe so much - he sees her as this embodiment of what he wishes he was. The irony there is that Phoebe is the girl who reveals to Holden that he's got this whole catcher in the rye thing wrong, and that while he's trying to stop from growing up, this poem that resonates so much with him is about growing up. It just happens to use sex as its metaphor for growing old, which is a metaphor Catcher borrows for its plot.