The Privilege Of Passing
I would say that passing is a privilege because it enables those who can do it to avoid some of the impacts of their oppression. The fact that many people choose to pass means that having that option makes their lives better than they would be if they didn't have it -- which seems to be the definition of a privilege. (An interesting counterargument would be the claim that the ability to pass is an additional form of oppression, because those who can pass are then expected to pass.)
The sense in which passing is not a privilege is the sense in which we set up an opposition between a privileged group and an oppressed group. This is what I think Sylvia was getting at in describing passing as a "survival skill." White privilege, for example, accrues to the non-racially-oppressed parties. Passing, on the other hand, only exists insofar as the alleged privilege-holder is him/herself oppressed. And the ability to pass, while it may mitigate the worst effects of that oppression, doesn't negate the oppression. Passing is still costly, putting the passer in a worse situation than someone who actually is a member of the dominant group. Pointing to someone's ability to pass can show their relative advantage vis-a-vis someone similarly situated but unable to pass (e.g. a light-skinned versus a dark-skinned black person). But passing doesn't tell us anything about whether one form of oppression is worse or more deserving of remedy than another (e.g. more-easily-passed homophobia versus less-easily-passed sexism).