Listen to pretty much any podcast today and you’ll hear the hosts asking you to leave a review on iTunes. It’s as ubiquitous as the “Don’t forget to like and subscribe!” tagline that every fledgling YouTuber tacks on to the end of their shows…But why do we do it? In Apple Podcasts, the largest and most mature podcast distribution service, rankings (for what they’re worth) are partially determined by the number of reviews a show has. And I, for one, think reviews are vital. You wouldn’t buy a new blender online, or book a hotel stay, without reading the reviews, and while a podcast is hardly of the same caliber, your reviews can make the difference between a potential listener saying “play” or “nay”.

There’s one little issue I have with Apple Podcasts, though, and it’s the five-star ratings system. So today, I’m going to tell you 5 reasons I think Apple should ditch it, and one good reason why it will probably stick around anyway. The reasons are: “podcast quality” is a poorly-defined metric, the star ratings can easily be manipulated upward, negative reviews are unlikely to be accurate, there’s no way to validate reviews, and the big one: there’s no way to tell if reviews are actually representative of listeners. Let’s look at these issues one by one.

First: “Podcast quality” is a poorly-defined metric. There are many, MANY factors that go into a good podcast: content, sound design, sound quality, the charisma of the host…And the relative weight of each of those factors is very personal to the listener, and to the show. I listen to some shows with terrible sound quality but unique, high-quality content, and I also listen to shows with average content whose production quality helps them stand out against the pack. And yes, I also have a show or two that I listen to just for the host. Each of those shows gets a five-star rating from me. Because what exactly justifies knocking off a star? Poorly-balanced voice levels? Intro voiceover that doesn’t match the theme of the show? Did they imply that they didn’t like my favorite movie of all time? In reality, pretty much any show I listen to is going to get five stars. Because if I’m a listener, those shows are my friends. And I don’t want to let my friends down. So I ask again: what are the other four stars for? What makes a three-star podcast? A two-star? Oh, just put it down for five.

Next: Positive reviews are easily manipulated. I’ve heard of contests, giveaways, review exchanges, and just plain begging. It’s easy to get a plethora of five-star reviews from your listeners if you ask the right way. It takes hardly any time, and next to no thought. After all, they’re already listening to you, so they must like you! BAM! Five stars! This goes back to (again) the fact that podcast quality is a poorly defined metric. Also, by the way, if you haven’t yet rated Use Your Voice in Apple Podcasts, then, um, *hint hint*…

Next: Negative reviews in Apple Podcasts are unlikely to be accurate. If you don’t like a show, do you keep listening to it to figure out exactly why you don’t like it, so you can write a thoughtful, well-reasoned one-star review? No, you don’t. No one does. They just stop listening. If you then take the time to write a very poor review, you must really have hated it. And in my experience, most one-star reviews are generally much more personal and tend toward trolling and/or baiting, rather than helpful. Add to that the fact that creators have no recourse to respond to negative reviews (like they do in the App Store), and well, negative reviews are just not helpful.

Next: There’s no way to verify or validate that reviewers are actual listeners. Have you ever seen on Amazon reviews, a tag next to a name that says “Verified Purchase”? There’s a reason they do that. So that you, as a potential customer, know that the person writing the review has, in fact, purchased the product before you. But in Apple Podcasts, there is no such assurance. The person writing the review, in fact, may have been paid to do so. There are plenty of people willing to do it, all it takes is a little bit of cash and voila! A quick search on the freelancing website Fiverr.com will yield plenty of people willing to review your podcast…the top search result will give you 16 reviews for just $10. Given that, how can you really trust that a five-star podcast is really a five-star podcast? Um, you can’t.

Finally: There’s a big assumption here with reviews, and this isn’t an issue just with Apple Podcasts. If you assume that reviews reflect what “people” think, then you must believe that the sample of people who have written reviews are a representative sample of the listening audience. And there’s absolutely no evidence to suggest that that’s the case. It’s a big logical leap to take, especially if you’re placing value on those five little gold stars.

But this leads me nicely into the one BIG reason that Apple Podcasts is unlikely to retire the five-star rating any time soon: it’s a recognized standard across multiple industries. Remember how I said that there’s no guarantee that reviewers are a representative sample of listeners? That’s true for the blender and hotel I talked about earlier too. And yet consumers still rely on reviews to help them decide on a purchase.

The only hope I hold out is that perhaps we might start to see a shift toward more meaningful metrics, as various distribution services converge on the “correct” way to measure what’s colloquially known as “listens”. For instance, if you could weight positively reviews from listeners who have consistently listened to at least 50% of episodes in the last year, you could be pretty sure they know what they’re talking about. Likewise, if you could reject reviews from listeners who’d never actually downloaded a show, that would be more accurate too. New stats in Apple Podcasts are due out later this year, so maybe that’s a positive to look forward to.