That's not something that's worried me since i started dipping my toes into podfic. But then, I grew up all over the US, and quickly developed the skill (it was a defense mechanism at the time) of being able to modulate my accent and speaking rhythms to match whoever I was talking to. I don't have a single 'voice' that is unambiguously mine, and the voice I use for podfic doesn't sound much like my speaking to friends voice or my professional voice.

And well, I have some internet friends who I've trusted with my legal name & mailing address, and some RL friends who I've trusted with my fandom identity (and some internet friend who live in my city, so I've invited them to my parties with college and work and church friends). So I'm already taking a much bigger risk than just having my voice online.