I would never have my private keys leave a smart card. I'd assume your PGP private key on hard drive, located in your home directory no less, is as good as compromised.