Beer-Rating Sites Have Some Serious Problems

beer ranking on smart phone
Screenshot via BeerAdvocate/Bon Appetit/Shutterstock (edited)

There's a decently good chance that you rely on the opinions of others to make a lot of your everyday decisions. Which show should you binge-watch next? Where should you get your lunchtime tacos? Which set of knives is the best one on Amazon? Now that beer has blown up to the point of giving you thousands of options, it’s also not unlikely that you've Googled reviews from your local beer store or bar about which beverage option is the best one in front of you.

The problem is that the sites that exist for us to review and rank beers have some major flaws with serious consequences.

This isn't a rant against online reviewers or people who make notes on beers. On the contrary: I think that kind of behavior is vital to the growth of understanding beer in this country! But I have pored through thousands of online reviews as someone who drinks, sells, and writes about beer, and as someone who regularly stands on both sides of the bar, it's time to have a conversation about what's going wrong and how we can try to fix the problems with a little understanding.

craft beers
Matthew Albanese/Thrillist

Ranking sites always heavily favor the bold (styles)

Probably the most repeated reason people debate the merits of the beer-rating system is that it simply is too predictable with what tops the charts. It's great news if you're a double IPA or imperial stout lover: your favorite beers are probably the highest rated in the world! Just take a look at any of the "best-ever" lists on the site and numbers won't lie.

Take RateBeer, for example: 41 of the 50 entries on its "best in the world" list are either imperial stouts, porters, or imperial IPAs; the remaining nine are made up of Belgian strong ales, incredibly rare Trappist beers, one saison, and a lambic.

Of course, some would argue that maybe heavy-hitting stouts and IPAs simply are just the best beers out there. But when you consider that the Beer Judge Certification Program Style Guidelines (they're a real thing!) cite over 120 unique styles, this representation becomes very troubling. It's why most novice beer drinkers assume there's something wrong with them for not being obsessed with hops or bold, boozy stouts.

There are no guidelines for critiques

It doesn't take an anthropologist to realize that the Internet is full of strong opinions with little information to back them up. If you're OK taking beer advice from a reviewer who writes his entire experience as "This smelled like a fart! One out of five," this lack of expertise is less of an issue. But it’s worrying that people will complain about beers exhibiting traits that are expected of their styles. "I hate this beer because it's smoky" isn't an acceptable critique when the beer in question is, in fact, a rauchbier (the German word for smoke beer).

The good news is that, just like an unreasonably harsh Yelp review, it can be easy to skip past the tasting notes of someone with a chip on his shoulder. The bad news is douchebags' number rankings count as much as the next person’s, and that gleaming final score at the top of the page is more influential than any brewer is willing to admit.

beer flight sampler

They're better for selling beer than discovering it

This is the point where the controversy over rating sites begins to share experiences with the wine world. Everyone gets a good feeling about reserving a table at a five-star restaurant or streaming a movie rated 9.8/10 on IMDB! That's also why it's easy for a retail salesperson (or a distributor selling to an uneducated beverage director) to sell a beer whose ranking is 99 or 100. Usually to people who didn't even know these beer-ranking sites existed in the first place.

The reality is that these reviews have very real consequences and the results affect livelihoods. To the person shopping for beer, standing in front of a cooler with their iPhone in hand looking for a recommendation, it may not be as apparent that something they love is right in front of them when positive rankings are skewed towards 3% of the available styles. And as the number of options for beer grows, this problem will only get worse.

Reviews are knee-jerk reactions

It seems like there's very little room for misunderstandings in the world of online beer ranking. Just glance through a page of reviews and you'll find plenty where the entire 400-word descriptions are based on a sample -- usually no more than a sip or two -- during a marathon day of tasting at a taproom or festival. Both BeerAdvocate and RateBeer have made it as easy as possible to review a beer from an app right on your phone, meaning that some are penned before the beer in question is even half finished.

Then there are the reviewers basing their first tastes of a beer from bars with dirty lines, or from well-expired bottles sold to them by an irresponsible shop. There are plenty of reviewers who know how to call this out, but there are also plenty who have a bad presentation of a beer and will never return to amend their slamming review.

craft beer tap
Dan Gentile/Thrillist

Older reviews don't always age well

A recent argument I've heard could be considered a predicament of the recent beer boom. Beers that have been around for a decade or more tend to carry more reviews than ones released more recently. This makes sense: when there was a field of 10 IPAs to choose from in your market, you were more likely to be impressed. But just like your teenage taste in movies, new experiences can make revisiting something feel less impressive (Garden State, anyone?), and it's very unlikely that your new views are being reflected in your old reviews.

On the plus side...

The reason why I won't tell anyone to up and delete their BeerAdvocate or RateBeer account is simple: if you're getting serious about beer, writing down tasting notes is the best way to remember the stuff you've tried. Both major rating sites ask you to note the appearance, aroma, and mouthfeel of your brew, which are important qualities that many drinkers leave out when compiling notes and are vital at rounding out your observations. It almost makes it like a beer LiveJournal! Except, you know, without the emo music links and dramatic emoji use (unless you're a really emotional beer reviewer, in which case, no judgment).

The sites can also tell you a lot about what's going on elsewhere in the beer world. Reviews and opinions aside, BeerAdvocate and RateBeer offer up-to-the-minute views of what's available in markets outside of your own. It's not uncommon for new beers to show up within minutes of their release. This is great if you're planning a trip across the country and want to set a mini bucket list for tasting new beers.

Despite all of this, it's very unlikely that anything will change in the way we review beers (or cars or movies or anything) online. Being part of an opinionated age can be overwhelming and frustrating, but if you choose to look at the bright side, it at least shows that the public feels very passionately about beer in a way that has never been felt in this country. And a high score often matters most to the brewers, who should be proud of earning a high score on a product they've devoted their time to perfecting in an industry that doesn't exactly crank out millionaires. Especially if they're making rauchbier and have the nerve to serve it to IPA-exclusive drinkers.

Sign up here for our daily Thrillist email, and get your fix of the best in food/drink/fun.

Zach Mack is Thrillist's contributing beer writer, the owner of Alphabet City Beer Co. in NYC, a newly minted Certified Cicerone®, and nothing else. Follow him: @zmack.