I just actually read your post after leaving my comment, and I'd add after that that it seems like you'd probably be more likely to enjoy podfics for stories you haven't already read. Many of the issues you pointed out with podfics seemed to come from the difference between how you interpreted a story when you read it vs. how the reader interpreted the story. So if your first experience of the story was hearing it, that might clear up some of that discomfort for you.

Maybe you have already tried this, but if not, and if you are really trying to enjoy podfics for whatever reason, it might be worth a shot.