Whisky Advocate

The 100 point scale: maintaining some perspective

July 14th, 2011

Let me start by saying that I like the 100 point system. I use it. It’s the most commonly used system by the most highly regarded wine and spirits reviewers, and it’s the scale that most people are familiar with.

But, looking at some of the comments over the past dozen or so whisky reviews (by me and the other Malt Advocate reviewers) here on this blog, I noticed something I felt I need to bring up and discuss.

Some of you seem to be making a big deal over a whisky review that is, say, 3 or 4 points higher or lower than what you feel you would rate it. You’re acting like you and the reviewer are so divided in how you both feel about a whisky.

In reality, you have more in common with the reviewer than you think. For example, if a reviewer rates a whisky an 83 and you think it should be an 87, they would both get an 8 on a ten point scale. And if it were a five point scale (or five diamonds or five stars or whatever), which is used by some reviewers, a whisky rated by one person as a 91 and an 80 by another person on the 100 point scale might both get 4 points. Instead of discussing how much we disagree with each other, we would probably be talking about how much we agree with each other.

I guess what I am trying to say is that, while I really enjoy using the 100 point system and am glad we can discuss and debate about whisky based on the 100 point system, let’s keep the proper perspective when when you and I (or one of the other MA reviewers) are four or five points apart in our rating of a whisky.

In fact, I think someone mentioned in my Ardbeg Alligator review, that ratings of 88-89 and 92 are effectively the same. (I’m not a statistician, so I don’t know for certain, but I think you get my point.)

I’d rather we discuss what we like (or dislike) about a whisky, and why. That’s when we all learn, grow, and mature as whisky enthusiasts.

108 Responses to “The 100 point scale: maintaining some perspective”

  1. Hi John

    I don’t agree with you on your views on comparing scales

    I scale comparistions “calibrations”. Yes even two persons using the 100 point scale are not necesarily calibrated (if they should be is anotrher discussion!)

    I use the 5 point scale and I would say it calibrates to the way most uses the 100 point scale like this

    0 : 0-30
    1 : 40-60
    2 : 80-85
    3 : 88-90
    4: 90-95
    5 : 95+

    But then again, thats just me!

    Steffen

  2. Jewmalt says:

    Interestingly enough, whisky rating systems were the topic of this month’s Whisky Round Table discussion held by the following whisky bloggers:

    Guid Scotch Drink
    Jewish Single Malt Whisky Society
    Nonjatta
    The Casks
    Edinburgh Whisky Blog
    Caskstrength
    Whisky Emporium
    Whisky For Everyone
    Whiskyparty.net
    Whisky Israel
    Glasgow’s Whisky & Ale

    If interested, you can find the discussion here: http://jewishsinglemaltwhiskysociety.com/?p=4495

  3. WhiskyTimes says:

    Here Here!

    I am still undecided on the whole score thing at all, I can’t see how something so subjective can be objectively quantified… But I digress. If you are going to rate whiskies, I think its noble and right to share why you like enough to give it a high score, and its impossible to give a score up or down 2 or 3 points, because people like different things. I am with Ian Buxton on the wave length of “can you really tell a 82 from a 83 point whisky?”.

    Objectively I can say, “it takes a lot of skill and time to make a whisky so smooth and complex”, but I can’t say that, the nature of smooth and complex whiskies are intrinsically ‘Better’ or somehow more ‘Good’ compared to another. A lot of people like a fiery, ‘one trick’, whiskies… it depends on your mood more than anything else. Thats my 10 pence worth anyway.

    • End of the day we just want to know if a whisky a good or not, and how good it is. I never take ratings that seriously, but one of the problems with a finetuned scale is exactly to tell the difference between 82 and 83!

      Thats why I abandonded it myself as I wasn’t capabale to manage the scale. More skilled tasters might be able to, but I was not

      Steffen

      • Using the review scores as a rough quite is quite the right way to do it think.

        Usually the way to remove the random subjectivity of a quantifying review is to have it done by a large group of experts sticking to strict criteria which are clearly visible close to the review. Which is rare.

        There is no denying its difficult to score a whisky and the above is why I don’t. I just talk about the experience, the flavors and what it means to me personally. then i bounce that off my review partner @wt_james. At the end of the day we all know roughly what we like. If we read about a whiskys tastes and smells we can roughly figure out if we will like it or not.

        • Matt says:

          I agree with you wholeheartedly, Whiskey Times(dan).
          Of course, we all like different things and it’s no different when it comes to whiskey. Scaling is not an exact science. We (readers) trust the reviewer to give an honest review of the product being tasted. I have to assume it’s being looked at objectively and not just given a score based on it’s likability to the reviewer.

  4. Todd Hoyer says:

    I kind of agree with Steffen, but I think you have the scale calibrated too high. It would make sense to have some sort of standard by which the ratings should be read. If you do want to break down a 100 point scale to a 5 star scale I would think it’d be more like:

    0 stars – 0 – 50 (How often do you ever see a wine or spirit rated below 50, I don’t think I’ve ever seen that)

    1 star – 50 -65 (We’re still acknowledging these as poor, but not quite swill)

    2 stars – 65 – 75 (Still subpar, but may have differentiating qualities worth noting)

    3 stars – 75 – 85 (A quality product, here’s we’re the 100 point scale really helps, in it’s ability to really place a product within that “quality” range)

    4 stars – 85 – 95 (A superior product, again, here’s where the 100 point scale helps, but also kind of proves John’s point that anything in this range we can all agree is a superior product but based on personal tastes and preferences can result in the difference between say an 86 rating and a 90 rating by different people).

    5 stars – 95+ (Best of the Best).

    • yes its calibated very high. I want the scale to represent the whisky I drink

      Steffen

    • Red_Arremer says:

      Right, Todd. I was about to write something like what you just said. It’s really more like a 1/25 or 1/30 scale– so 1 point makes a bit more difference than john is saying…

  5. Andre Girard says:

    Nice dabate indeed. My starting point is always “did i love it or not” and I start from that. But the pointing system is all about personnal perspective. Means a good whisky for me is not necessary a good whisky for another person. So i always prefer to refer to the tasting notes itself (balance, taste etc) instead of the final score.

  6. Tim F says:

    Not a fan of the point scale personally. But I don’t have a problem with anyone who wants to use it. The real problem is the people who take it too seriously. Taste the whisky yourself and make your own mind up. Try and understand that a point score is just how that person was experiencing that whisky on that day. If you find yourself getting cross or angry about it, you really need to get out more.

    • David Bailey Jr. says:

      I concur, although I don’t have an issue with the rating system. It is what it is. One man’s malt favorite may receive disdain from another. Simply enjoy the whisky experience :-)

  7. Scott says:

    I think a couple of my comments some weeks ago may be among those referred to here. Responses to those comments, as well as this post, seem to have an unfortunately defensive quality, so I’d like to explain or explore a bit further the issue with ratings I was raising. In back-to-back mini-reviews by the same reviewer, the narrative reviews seemed markedly different in tone and word choice. Yet the whisky whose narrative review used language with the more consistently positive connotations was given a lower numerical rating. I wasn’t complaining, nor was I arguing with either the narrative reviews or the rating, but the results seemed counterintuitive and worth asking about for clarification.

    I mean, say I reviewed two cars thusly: “Car A: Sprightly, quick acceleration, responsive controls, tight cornering with minimal lean, score 83. Car B: Lumbering off the start, thick steering, soft brakes, decent interior build quality, score 89.” It’s easy to imagine all kinds of things that didn’t make the narrative review that could have influenced the final score, but one has to imagine them since the higher number for the latter review all but requires there to have been factors not mentioned in the review. And not necessarily factors of absolute or objective comparison between the cars; if the first car was a vintage MG with no safety equipment and half the time you couldn’t even start the thing without getting under the hood, you’d expect it to garner lots of positive adjectives about the experience of driving it but a low score when judged as a car. Whereas if the second car was an entry-level four-door sedan with high-end safety and convenience features as standard equipment, you’d expect a review focused on performance to look mediocre but a final score to be on the high end. And neither set of factors informing the final score would really apply to the other.

    Alternately, it’s possible that the reviewer was applying completely inconsistent standards from one review to the other, and the numbers are simply meaningless bits of arbitrary nonsense. We’ve all seen restaurant, movie, and even wine/spirit reviewers for whom this is true. (Example: The Washington Post‘s Tom Sietsema, whose narrative reviews are as informative and keen as any food critic in the nation, but whose star ratings consistently render great restaurants as two or two-and-a-half stars out of four but merely good restaurants as three-star affairs.

    So while it’s not particularly useful to simply say, “You rated that an 86? That’s crazy! It’s clearly a 91-point whisky!” it’s perfectly valid to seek clarification about the meaning of a score or its relationship with a narrative review. And while it’s true that an 86 and a 91 are effectively the same score between two different reviewers, any difference in score by the same reviewer should mean something. If a reviewer does not intend for the reader to understand that something scored an 86 is superior to something scored an 85, much less 91 versus 86, then he should not use a 100-point scale. “Because Robert Parker did it” is not a sufficient excuse for using a scoring system that implies a level of precision one does not intend.

    On the other hand, it’s worth reading Roger Ebert’s explanation of why he uses a four-star final score, not five stars, or ten, or 100 points: http://blogs.suntimes.com/ebert/2008/09/you_give_out_too_many_stars.html

    As long as the reviewer aims for integrity, explains himself, and intends whatever bottom-line score to mean something to the reader, even if it must mean less than the narrative review, it’s all good. As Ebert concludes, speaking to the hypothetical reader to argues with how many stars he gave a particular movie, “Have you considered actually reading the review?”

    • Lew Bryson says:

      Ah. Last point made is the best one. I’m using the same point system John is, obviously, but I don’t like to see the points divorced from the notes, or even from the context of other reviews. I’m reviewing a bunch of white whiskeys for the next issue, and each review informs the others, both for me and — I hope! — for the readers. It’s always an ongoing process for everyone. Even Parker.

      • Red_Arremer says:

        I like that point as well Lew. Along the same lines, I also I like it when that other magazine has two people review the same product.

      • two-bit cowboy says:

        Hey, Lew:

        Not sure what you mean “each review informs the others.” If you’re reviewing “blue” and “yellow,” and you say, “yellow’s nose was very similar to blue’s except more complex,” your review of yellow is wasted on me. Two years from now if I want to read what you said about yellow, I won’t want to be forced to go find what you said about blue to “get” your review of yellow. Too many writers do that sort of gang tasting writing and it doesn’t help the reader. But that’s me as the reader, and I don’t suggest that I speak for any other readers.

        • Red_Arremer says:

          The “two years from now” point is well taken, Two-bit. If notes are to be optimized for timelessness then bottles should not be reviewed in tandem.

    • JSJ says:

      “‘Because Robert Parker did it’ is not a sufficient excuse for using a scoring system that implies a level of precision one does not intend.”

      Exactly: we must remember to mind our Ps and Qs and significant figures!

      http://en.wikipedia.org/wiki/Significant_figures

  8. Scott says:

    Would it change anything, and if so what, to consider the 100-point scale from a sort of baseball “sabermetrics” perspective? That is to say, rather than adding up points out of 25 or 33 for particular attributes, what if the point of a 100-point scale were to communicate the odds that the reviewer would enjoy being served that whisky? There will almost always be times when for whatever reason – pairing, setting, time of day, whatever – one just won’t enjoy a particular whisky. So what if a score of 95 meant that the reviewer were estimating, he’d guess that if served that whisky 100 times, he’d enjoy it very much 95 times, and wish he’d been served something else 5 times. A score of 85 would mean that he’d wish for a different dram 15 times out of 100 random servings.

    Seen that way, a whisky scoring 95 isn’t necessarily a better whisky in any absolute sense than a whisky scoring an 85, or if it is “better,” it may be only very slightly so. But the lower-scoring whisky is significantly more likely to disappoint on any given serving than the higher-scoring whisky. Might that be a useful way for a critic or a reader to think of a 100-point scale?

    • MrTH says:

      Probably not…I’d have lots of 99- and 100-point bottles here! Not very useful, is it?

      • Scott says:

        Hmm, good point. Thinking about my own self, this 100-point method would make Buffalo Trace and almost anything by Balvenie as 100-point whiskies, whereas many whiskies that I actually think of as better-made spirits would score a lot lower. Never mind then!

  9. mongo says:

    you might say that there is no difference between 84 and 88 or between 88 and 92, but it seems harder to say there is no difference between 84 and 92. someone who would rate two whiskies in the 86-88 range is marking more similarity in those ratings than someone who is marking one 84 and another 92.

    to echo steffen above, the wider the scale you use use is the more it implies fine-grained distinctions, and thus the more people who take them seriously are inclined to pick them apart.

  10. Mr Manhattan says:

    Even if 100 points total are “available” for scoring, the scale becomes effectively compressed if all reported scores fall between 75 and 100, i.e. it becomes a 25 point scale for the class of products being evaluated and reported.

    This makes me curious about two things:

    1- Is the 100 point system being employed here based on assigning points for different attributes of a product (color, nose, mouth feel, initial palate, finish, etc) and then adding them up or does it get assigned as a single number by the reviewer after the evaluation is finished—i.e. “based on my experience and preferences, I’d call it an 89″?

    2- Are there products being evaluated by John & Co. which, receiving low scores (e.g. below 70 or 75), we don’t see reported here?

    Michael

    • Red_Arremer says:

      John has said that he doesn’t do/publish formal reviews, which he believes will be damaging to a products reputation unless he believes it is for the good of whisky, in general.

      • Mr Manhattan says:

        Makes sense to me.

        So then I might assume there are some scores we don’t see because they are below whatever threshold of quality John & Co would define as “not beneficial.” I’d be curious to know how many such scores are given and the distribution (all w/o knowing to whom they were given). That would help to round out the picture we have of how points are being assigned.

        [And BTW I am perfectly happy w/ how things are being done. I’m just curious.]

        Michael

  11. Vinny says:

    I’ve always wondered what would happen in a blind tasting where the same set of whiskies were reviewed a couple of weeks apart say 4 times. How much fluctuation would there be in the score?

    Perhaps 3-4 points is just the margin of error and the differences between say a score of 80 and 83 is not significant.

    Vinny

  12. Archaeology Carl says:

    I rate the whiskeys I taste on a 3 point scale. I do this for my own records, and I share it with friends. Its simple, but leads to some lively debates.
    3 – Love it. Great taste and finish. Well balanced. I will always want a bottle of this in my bunker (regardless of cost), will recommend it to friends and give it as a gift.
    2 – It was good, but either the taste/finish was just OK, it was not a style I prefer, or it cost too much for me to keep regularly buying it. Some one else may love this, but not me.
    1 – Hate it. Didn’t like the taste and finish. Wouldn’t waste my money on another bottle and wouldn’t recommended it to a friend.

    And I treat reviewers like I do movie critics. Some share my same preferences, but others don’t. While I do give their scores some weight, I follow their description of the flavor, finish, and balance more closely (ie. sweet, smokey, peppery, peat, etc). That is often what I base my purchasing decision on.

    Carl

    • Douglas Fischer says:

      My way of “rating” a whisky is pretty similar in that there are very few categories.

      – If I really like a whisky, then I will buy it, make sure to replace the bottle in my collection once it has been consumed, and I will most likely seek out other expressions of that particular distillery or label to try.

      – If a whisky is “so-so” I may or may not buy it (if I did not already do so) and it may or may not get replaced in the collection. Usually I will give another expression a try in a bar or tasting, but I will rarely buy another bottle blind.

      – If a whisky is subpar I won’t buy it and I will be cautious about trying other expressions of it. If it made it into my collection before I realized its subpar nature it will usually only be used with a mixer or some other way of just getting the bottle consumed – once in a rare while I have actually poured a bottle out rather than subject myself to its contents.

      I try to have a whisky at least twice before I categorize it (unless it is really that good or that bad), to account for any bias of mood, setting, or palate effector.

      Where I find the most benefit of reviews and ratings is when I can compare my own thoughts on a whisky to multiple reviewers, and find ones who happen to share my feelings. John, it so happens that your palate and mine match very nicely (at least in terms of what you find good and not – I do not have your range of tasting capability – yet…). Once I’ve found some well-matched reviewers, now I can see what they think about whiskies I haven’t tried yet to get an idea of ones I should try.

      This has actually helped me identify a high-quality expression for which I happened to have originally tasted a corked bottle. I was happy to discover the problem, and even more happy to sample the expression sans corking and found it to be absolutely delightful.

      So basically, the real benefit of reviewers doesn’t become clear until you’ve tried a number of reviewed whiskies yourself and calibrated your personal tastes with those reviewers. Then, you can use those reviewers as guides to those whiskies that may be worth your effort to seek out and try and/or buy.

      Cheers.

    • Simon Seaton says:

      I am right with you Carl. I give myself 4 rather than 3 options but effectively the same idea and I make my personal acid test…. if I was offered a free dram at a party or a friends house would I accept or decline. Taste is too subjective, and I know my palate and mood always change so to score objectively (or even consistently) is basically impossible for me.

      4 – One to buy, recommend to others and budget willing, keep around the house- same as Carls’ 3. If offered a dram would always accept and always go back for second.
      3 – Something about this that maybe be of interest / unique at the right time / right place. I wouldn’t keep a bottle and will usually buy something else given a choice. Would recommend it to others perhaps and would happily accept a dram but maybe just the one…
      2 – Good or OK. Might appeal to others and they should try for themselves – especially if it is a well established brand with a strong following (I am may just not get it or my palate was off that day) but I cant see myself ever buying it. If someone offered me a dram I would accept but might ask if they had anything else first.
      1 – Same as Chris wouldn’t buy it, wouldn’t recommended it to a friend and the acid test for me, wouldn’t accept a dram if offered – would ask for something else.

  13. Tim F says:

    From these comments it’s becoming increasingly obvious that one of the fatal drawbacks of the 100-point scale is that everyone is using it in a different way and there is widespread confusion on how people who care about the scores are supposed to interpret them. It seems that we need a different interpretation for each reviewer. I can’t see the point myself. It should be obvious from the reviewer’s description whether they liked the whisky or not.

    • Douglas Fischer says:

      I think to some extent the individual basis for the exact score is almost unavoidable. Everyone has a slightly different palate and is going to pick up on different nuances of the whisky. That said, they should fall within the same range from one reviewer to another, give or take a few points. I like what the Malt Maniacs do with their blind tastings and consensus scoring – this should even out some of those individual discrepancies and provide a more solid footing on which to rate a whisky.

      I don’t think we’re ever going to be at a point though where any random review of a whisky is going to be something that can be taken at face value – it’s always going to need to include the tasting notes and, honestly, the identity of the reviewer. And that’s okay. The best “advice” in whether or not a product would be to your personal liking is the review by someone who either knows you and your likes, or with whom you share something in common in terms of likes. Find some reviews of whiskies you’ve tried and see what those reviewers say. Find one or more reviewers who seem to have a similar taste profile to you and focus on their reviews of whiskies you haven’t tried. Chances are good they will steer you in a good direction.

      Cheers.

  14. Allen says:

    Ralfy posted a video on tasting notes yesterday that briefly touches on the topic being discussed. Well worth a look: http://www.whiskyreviews.blogspot.com/ Since no two people have the exact same taste buds (to my knowledge), all reviews should act as more of a reference and not taken as a concrete fact. I look to reviews to tell me if a whiskey has any strange flavors I might not like, flavors that I might enjoy, comparisons to other whiskey I may have already consumed, and if it is a good value. I don’t look to the review to tell me how much I should like this whiskey, that’s up to me to decide. I think people get too hung up on the score.

  15. Keith Sexton says:

    When I look at the score, I use it it only to see how john, or whoever else, rates the whisky. But even then, it’s not really necessary. They never write a mere sentence or two in a review. It’s always a lengthy paragraph or two that goes into detail about their opinion of the whisky. If you read the review, that’s really all you need.

  16. Paul M. says:

    I know that any rating scale is subjective. I do find it very useful when comparing how a particular reviewer rates an item with another item. It gives me a sense on what that reviewer thinks. For example, is this Ardbeg better that another Ardbeg.

    What I find more useful is the description. That will help me more in deciding whether or not I might want to buy an item. I have enjoyed many whiskies, that although may not have a very high number associated with it, because of the description and profile that it presents. After drinking, I may agree with a number that a reviewer may have given, it does not mean that I don’t like it. It just may not have enough balance or complexity to deserve a higher number. But again, taste is very subjective.

    If anyone’s purchase or drinking decisions are bases solely on the number scale, then they may be missing out.

  17. Colin says:

    When I tried to blog about whiskies I used a school grade style A+ through to D though other scores have been helpful in comparing my tastes against the other reviewers. My preference is serge’s on whiskyfun a 100 point system with an additional breakdown.

  18. Brian B. (brian47126) says:

    Oops–Sorry john, that was me who made the comment about Ardbeg Alligator. I think I should have added to that comment that I was trying to guess your score at that time so I was pleased to see that I was wrong and it was two points higher. I know the numbers are not really what it’s all about. I was simply trying to guess exactly what you would score it. I did not mean to smack the hornets nest–sorry John.

  19. Toby Belch says:

    In my view, the 100 point scale is a silly fantasy because there is no quantitative basis for such precision. Why use so many numbers if we believe that differences of 4 or even more points are nearly meaningless? The fact is that the 100 point numbers are just made up by the reviewers. The numbers are fantasy. I might as well look at a woman’s photo and rate her beauty on a 100 point scale. What would that rating mean? Almost nothing. Except to me. At that moment.

    As as for statistics, I am not a statistician either. But I know that statistics have no meaning in a measurement if the error in the measurement is unknown.

    The only sensible scale for spirits ratings, I believe, is the 5 star scale. One star means the spirit sucks, badly. Two stars means it is drinkable but not more. Three stars is quite acceptable; 4 stars, very good; and 5 stars is really outstanding. Much more precision than these five levels is silly.

    When Paul Pacult began his Spirit Journal in the 1990s, he used a 100 point scale. Soon, however, he realized how silly it was. So he converted to the 5 star system. I haven’t found his reviews any less useful since he made this sensible change many years ago.

    Of course, Jim Murray gets the award for carrying this silliness to the extreme. He uses half points on a 100 point scale. Apparently he believes his palate is like a thermometer poked into a baby’s bottom – you get a reading to the nearest half degree.

    Fortunately, spirits reviews are completely harmless, no matter what scale is used. They are entertainment. Shakespeare made up a lot of stuff, and we enjoy reading it, nonetheless. In the same way, we can enjoy numbers that the whiskey reviewers make up. They have fun making up the numbers, we have fun reading them. What’s so wrong with that?

    • In reality only 75-95 is used on the 100 point scale and when you say a difference of 4 points are nearly meaningless , you also say that whats used is bassically a 5 point scale :-)

      Steffen

      • Toby Belch says:

        Steffen – Excellent point. Since the 100 point scale is close to a five point scale, why not abandon the fantasy of a 100 point scale? Especially since the human olfactory system is notoriously unreliable and time variable.

        • Scott says:

          Or, and I really think we’re on to something here, don’t correct the scoring method to the unreliability and variability of the human nose, but correct the nose to match the precision of the 100-point scale. From now on, all whisky judging panels must include a trained sniffing dog, who will be solely responsible for scoring the nose component. And also maybe a bald eagle to judge the color. From a mile away.

          Humans can then focus on what we’re good at: taste, finish, and inventing fancy adjectives and metaphors for the writeup.

          Kidding! One thing I value about John, and actually most whisky reviewers, is the degree to which their nosing/tasting notes tend to be objective, in that they focus on nouns, and are largely free of the subjectivity and cant inherent in adjectives and fancy metaphors like you see in too many wine reviews. “Notes of vanilla, bergamot, and clove on the nose” I count as reasonably objective, in that those are actual things with actual odors that even if no two people will experience identically, they’ll both at least experience if the chemical components are actually present. As opposed to any result from the Silly Tasting Notes Generator: http://www.gmon.com/cgi-bin/sillytng.pl

          • “in that those are actual things with actual odors that even if no two people will experience identically, they’ll both at least experience if the chemical components are actually present.”

            The Institute of Brewing and Distilling published a paper that evaluated flavor chemicals by their aroma family (ie., were they peaty, grainy, estery, etc). The authors’ goal was to develop a flavor wheel suited to professional and industry evaluation. The interesting aside was that many professionals did not choose the same terminology to describe the same flavor compound. In the paper, furfural was described as marzipan (54%), sweet (26%), oily (15%), grainy (<10%) at high thresholds, while at the low end of detection it's described as grainy. So even though this is the same flavor chemical, reviewers often didn't describe i the same way. Assuming that they detected it. In the study, the authors made sure the participants could detect it, but that's not always true in real life. And even if the reviewer detects the flavor chemical, there's no guarantee that someone else, a reviewer or taster, will come up with the same descriptor.

            The paper is K-Y. Monica Lee, Alistair Paterson and John R. Piggott, "Origins of Flavour in Whiskies and a Revised Flavour Wheel: a Review", J. Inst. Brewing, Vol. 107, No. 107, pp 287-313.

          • Mr Manhattan says:

            That’s a study between different people…how about the same taster on different days or even different times on the same day? I bet the results are almost as variable. ;->

          • Such a study would be interesting and would validate the common view and to what degree. My personal experience is that there can be significant variation in taste often coming from interaction with food flavors leftover from a meal or other beverage (including beer). Let’s discount effects due to illness.

            But I suspect that the professional tasters may not have less of a bias, but greater experience in avoiding those problems. Think of the tasters for commercial whisky blends that have to consistently approve a whisky as matching the flavor profile of the product under consideration. On the production side, people need to be able to test mash by taste and smell, as well as distillate, and recognize problems early so they can be identified and corrected. If they didn’t do that job consistently, there’d be a lot more variation in whiskey than there is. (I understand that the large houses often have a committee of three to five tasters in order to minimize individual variation. Plus it’s lots more fun drinking in company.)

        • Douglas Fischer says:

          I think in some ways many people find the concept of the 100-point scale comfortable because of other areas in life in which we are exposed to it. Think about school – grades (at least in the US) are given on a 100 point scale, we have the Celsius temperature scale which is calibrated on the freezing (0) and boiling (100) points of water, and of course any time you talk about percentages this is a 100-point scale by name. So people are very familiar with it and so can quickly acclimate themselves to its use with reviews.

          I agree, if you look at most any review you’re not going to find something sub-60 (or even sub-70), so in effect it is a 30 or 40-point scale with a 60 or 70-point offset.

          To me any question of whether any particular scale is the “right” one comes down to what is the best – and by that I mean the most clear and comprehensible – manner of relaying the desired information to the desired audience, that is to relay the quality of a whisky to consumers. Of course the problem is “consumers” encompasses a wide range of knowledge, expertise, and interest. Some consumers would be more than happy with a simple buy/no-buy review, while others would desire a long, multipage narrative detailing every possible aspect of a whisky, with most consumers falling somewhere in between. So, reviewers are tasked with devising a scoring mechanism to be able to suit all of these people – and in order to at least try and standardize reviews across reviewers to get others to adopt the same mechanism.

          I would say that while the 100-point scale may or may not be the “ideal” mechanism, it works pretty well, it is widespread in the field, and while it is not necessarily applied in the exact same way by all reviewers, it is applied with sufficient consistency to meet the needs of the majority of consumers.

          Cheers.

  20. Danny Maguire says:

    lots of very interesting comments, but nobody explaining exactly how the 100 point scale is worked out, are their 10 different areas that are scored out of 10, 5 out of 20, or what? I know that the higher the score the reviewer gives a whisky the more they like it but unless every reviewer is using exactly the same base then the scores are meaningless, it’s just one persons view against anothers. If the reviewers are using the same system then their reviews for the same whisky should come out about the same, even if it’s not to the personal taste of one, or more, of them. They should still be able to recognise the quality of the produce.

  21. Mats says:

    Steffen Brauner and Todd Hoyer bring up a good point (which is also incidentally often overlooked in market research), and that is that a scale is not always symmetrical (balanced around the middle score). When you rate “liking” in this way an acceptable score (“average result”) is not in the middle of the scale. Hence, it is not really a 100-point scale, but somewhat more like a 30-point scale, as scores below 70 seems to be very rare indeed (as basically anything below 70 would be akin to “avoid at all cost” anyway).

    This is a similar argument to the notion that familiarity scores on a 10-point scale should have degrees of familiarity from 3 and up (with 1= totally unfamiliar and 2=almost unfamiliar), as there are more degrees of familiar than unfamiliar, or that customer satisfaction scores on any scale should lie above at least 75-80% of the maximum score (i.e. an average of 3.5 on 5 point scale actually means that customers are fairly dissatisfied as this is below an “average” satisfaction level).

    What I would really like to see is a two dimensional scale, where one dimension is “pure” quality and the other is “value for money”. The quality scale is not symmetrical, but the value for money probably should be.

  22. JohnM says:

    Actually… I’m going back to college to do a post grad in statistics in September, at the age of 40. My degree is in physics and maths, originally, but am interested in this kind of stuff too. I’d love to do some kind of test as my thesis, as was done with some wine raters to see how accurately humans can rate things as subjective as taste. I know I couldn’t mark out of 100, but there’s no doubt others are far more skillful than me at rating things. I could taste the same whisky several times and could love it one day and not like it at all the next.

    When I read a 100-point rating, I always view it as a five-point or ten-point scale. The difference of one in a hundred between two whiskies, while it might mean something to the rater, means nothing at all to me.

    When wine raters were statistically analysed, there was a +/-4 point spread, which would suggest a 10-point scale to be more realistic.

    Something else I think I’ve noticed… in blind taste tests, a lot of older or peatier or sherried whiskies do well. Maybe this is not the case, but it seems like it to me. My guess would be because these properties are more easily identified when the judge has little else to go on. Look at the Malt Maniacs’ yearly competition. Of course, maybe it’s just that these are the best whiskies…

  23. David D says:

    The problem with points is that – in this ADHD-driven society we now live in – no one reads the review, they just quote the points. I know this because every single producer that comes into the store tells me how many points their wine or whisk(e)y received from so and so, leaving out any other important details. When the pros use points, the customers and enthusiasts feel that they should use the points as well. One can of course claim that points should not be separated from the review or the notes, but that’s not the way people function. Everyone wants the fastest, quickest way to summarize everything because it’s more effective.

    As far as agreeing more often, that’s easy. If people would just talk about what they liked (flavors, textures, etc) then there would be fewer arguments. If you quantify a review, you’re asking for disagreement because people like to argue by nature. Everyone wants to have their opinion (hence why we’re all typing our comments in). If you say the sky is blue, someone will inevitably say that it’s more greyish than blue. Read any message board or the comments under a newspaper article and you’ll be shocked by the lack of humanity. Most don’t want to agree, they want to prove they know more than everyone else by nitpicking every little detail. It’s a contest and they want to win.

    But that’s just my pessimism talking.

  24. JohnM says:

    …just one more thing… sorry if I’m being boring, but if there was a +/-4 point spread in such tastings, that’s 8 points in 100. However, if the rater never considers the bottom part of the scale, marking, say, from 50 upwards, then it’s an 8-point spread in 50 – quite a significant margin of error… That’s not to say, however, that the rater isn’t marking a one in a hundred increment at the top of the scale even if they aren’t considering the bottom of the scale.

  25. Louis says:

    To me, the 100 point scale reminds me of school, at least in most of the US. So 90’s scores are an A, 80’s B, 70’s C, 65-69 D, and the rest failing. The difference between 88 and 92 is a B+ to A-, it means Deans List/Honor Roll, and possibly a scholarship or job offer at the college level. Then again nobody cares about even a 10 pont difference between 45 and 55.

    The star system might be a bit better, but then half stars come into play. And assuming that nobody rates less than one full star, we only have nine rating slots, down from 25 or so. The slotting aspect of ratings is why I stopped caring about ratings some years back. And there is the question of whether a peat monster that scores 89 is ‘better’ than a Speyside that only got an 88 (hint, it’s whatever you’re more in the mood for that matters).

    So in the end, ratings are good for determining the GENERAL worthiness of a particular dram, and only with respect to whiskies of the same style. In the past, I would have added from the same reviewer. But now that the Malt Advocate has spread things around, one would hope there is general agreement among the review team as to what the scale encompasses.

    Slainte.

    Louis

  26. sku says:

    I actually think the differences between whiskey are vaster than can be contained in a mere hundred points which is why I use a thousand point scale. Even this, though, is challenging, and I may soon move to a million point scale. I also think it will be helpful for getting me on shelftalkers: “Rated 1,000,000 points by Sku!!!!”

    • sam k says:

      Sku, why do you insist on making fun of this intensely serious subject? I mean really, what can be more worthy of such astonishing scrutiny than the way a reviewer judges their whiskey? Oh, maybe the gridlock on the national debt ceiling, sure, but this is amazingly important stuff, dammit! Don’t diminish the discourse with your trendy SoCal humor.

      Actually, you know I love it. After trudging through the geekily serious responses, I almost spewed beer (no whiskey tonight, sorry) on my computer screen after reading yours. Keep it light, my friends…none of this really matters! Five stars = 100 points = a thousand points of light = seven gables.

      Taste it your own self and decide.

    • Ryan says:

      Sku, there’s a shade more flux than I’m cozy with in your 1,000,000 point scale. You should have no idea what you have smelled or tasted, nor how you arrived at your scores, by conducting triple-blind tastings while semi-anesthetized in partial vacuum. Oh, and any particular reason for limiting yourself to positive value scores? I’d like to see a -10,000 (or worse) throw-in for the sake of comparison.

    • Red_Arremer says:

      Numbers from a reviewer like John are serious enough. They have an impact on the prestige of brands and bottlings. They motivate consumer interest or lack thereof.

      They are also part of how John makes a living. He wouldn’t be where is today if people didn’t take the rating system he uses seriously. So Ratings are worth discussing now and again– plus they raises plenty of interesting questions that pleasantly transcend ordinary whisky talk.

      • smsmmns says:

        Red, John isn’t here because people take his ratings seriously. John is here because of his research into the sources of happiness, his rejection of evolutionary theory, and his staunch stand against the sometimes controversial 1,000,000 point scale

        • Red_Arremer says:

          Of course you’re right Sam– Sometimes I oversimplify things for the point of argument ;)

        • Douglas Fischer says:

          One of the things that makes me glad John is here is that he has a passion for whisky (and beer!), a fairly stable and consistent palate, and access to a very wide range of whiskies (which can also be a source of jealously – in a friendly way!). Taken together, I think his reviews are a good source of information on whiskies I may want to try but haven’t had access to. With the large number of whiskies reviewed even if someone doesn’t agree with his ratings they can get a feel for how his ratings rank relative to their own and extrapolate from there.

          Cheers.

  27. The Leveller says:

    I always feel that the best way to use the 100-point scale is to take 75 points off – then the relative differences really open up. Like it or not, taking Robert Parker who was probably the leading driver here, a 96 rating is meant to be taken very differently from a 93.

  28. Jerome says:

    I use the 100 point system, but in reality it is more of a 5 star system (1-69, 70-79, 80-85, 86-89, 90-95). I suppose a whisky could, in theory, get more than 95 points but really anything over 90 points means it blew me away. In my system a 90 is notably different in quality than an 88, as is an 86 to an 84. For this reason, I don’t give out scores because they would be meaningless to others. It is also why a tend to ignore the marks of others and look instead at the level of enthusiasm in their descriptions.

    • Hi Jerome, that makes your system calibration very similar to mine. My ratings being very meaningless to tohers haven’t refrained me from making (some of) them public :-)

      Steffen

  29. I went too far and made a graph of how I see review scores: http://tinyurl.com/6znqu2x

  30. bj reed says:

    To add a little statistics to the discussion – what we have here is an attempt to do interval scaling on a subjective assessment and this is fraught with problems statistically because you are not using an objective measurement.

    For instance, most interval scales are used for things you can actually measure objectively (e.g. household income, selling price for a house, a person’s age), things you can rank objectively from highest to lowest (referred to as ordinal scaling) is the next best statistical tool but it assumes you can measure the distance between scales and is often used to measure preceptions (e.g. whisky is great, pretty good, fair, poor) but statistically you should be able to measure the distance between great and pretty good and of course we often cannot do that when we measure perceptions.

    The weakest statistical scaling is nominal which simply tells you that two things are different (e.g. whisky is chill filtered or its not chill-filtered) – In most cases perceptual rankings should be nominal not interval and we should you statistical tests of differences but not tests assuming objective measurement.

    This would make the case for ranking whisky by basic differences which doesn’t necessarily assume you can measure differences objectively. The more you use objective scaling (e.g. interval or ordinal) to measure subjective assessments, the weaker the statistical validity. Doesn’t mean you don’t do it, just means you understand its weaknesses.

  31. Joshie says:

    At my little dog & pony show, I don’t use any sort of scaled rating system. At the end of most reviews I will just say recommended or not recommended. Tastes vary so widely even among connoisseurs that I find a description to be more useful than a number or a grade. Another factor is price. Some believe that they don’t ever think about price when they are tasting, but I think they’re just deluding themselves. It’s human nature to expect more from something that cost more and expect less from something that costs less. As a wise man once said “From everyone to whom much has been given, much will be required.”

    This is what the job of a critic is, imo. A good critic is not always someone with the best palate or most extensive knowlege. A good palate and extensive knowledge are certainly helpful, but a good critic is someone who is able to translate the experience of drinking a particular beverage (or hearing music or seeing a film or reading a book) into words on the page. The reader can then take those words and make a choice about what to buy or not buy or just enjoy the words as they stand on the page.

    That said, the 100 pt scale has benefits. The benefit of a magazine like MA using the 100 pt scale is that it is close to universal at this point across the beveage world, for better or worse, so it’s easier for readers and marketers to use ratings in that scale. I will admit that when I read MA reviews I do skim the numbers first before I go back and read reviews that interest me.

    Anyway, another thought-provoking post, John. Highly recommended. :)

  32. David OG says:

    An old friend turned me on to the most powerful rating system available. Chuck it, drink it, buy it. Chuck it: don’t drink this whisky for any reason. Drink it: if you’re buying, I’m drinking. Buy it: worth the cost and a necessary addition to the bar. I’m sure you’re all thinking that this lacks nuance and descriptive power, but what else do people really want to know? Succinct, practical, and ALWAYS statistically relevant.

    • sam k says:

      I LOVE IT!!

      • Red_Arremer says:

        Do you own a liquor store sam?

        • sam k says:

          I wish!

          • TheMandarin says:

            Drink it, buy it or chuck it is undoubtedly, one of the funniest pieces of folk wisdom I’ve read in a while, but Sam, why am I detecting a fear of connoisseurship? Also, why spend so much time on WDJK, a website known for it’s serious and thoughtful stance, if you want to see whiskey taken less seriously? (You were, if my memory serves me right, the second most frequent poster on this entire board.) My apologies if I’m exaggerating your case but maybe you can help us move the discussion forward by articulating your views.

            Also, interesting post John. Nicely done.

  33. Brendan says:

    Wow, OK, a very cool conversation.

    John’s initial post got me to thinking about the standard deviation on standard scores, as I use at work, and the fact that two scores within a given number of points of each other essentially represent the same quality.

    However, with standard scores, 100 represents the mean. On a whisky rating scale, the mean is probably somewhere around 90, depending on the rater, and the median is 50. I would not buy a whisky rated anywhere near 50, and so we end up in Lake Woebegone, where all of the children are above average.

    Can’t fix that, though, so let’s instead aim for inter-rater reliability, meaning that your 85-rated whisky would also rate about 85 for me. That’s really the purpose of whisky ratings anyway, that I spend the money on a bottle that John really likes because I expect to really like it, too, or I skip a given bottle for the same reason.

    Some of that comes down to finding a critic whose tastes are similar to yours, or being able to gauge your tastes against those of a predictable critic with a different perspective. In that function, then, the reviewer and the distiller should have the same goal, which is consistency. I’ll buy my second bottle of a given whisky because I liked the first bottle and if it’s really different, especially in a way that I don’t like, then I won’t buy a third, because of the second bottle.

    The other thing here, as some folks have noted, is that an 85 does not mean that there are 85 of anything associated with a particular whisky. The number is a symbol of how that whisky compares to others, which basically gets you back to brackets and stars or whatever.

  34. Garrett says:

    I would like to say — though it might already have been, I didn’t read all 52 comments — that almost all whiskies I see reviewed are in the 80-95 range. Since to a certain extent we are already looking at a shortened scale, 89 vs 92 would actually constitute a fairly major shift in quality. If the bottom end of the scale (0 – ~70) is rarely/never utilized it doesn’t do too much good to use it to justify ambiguity in the system.

    Or we could just drink the stuff.

  35. Gregg H says:

    Last night, I was at a whiskey tasting and realized something important to me, I might rate the same whiskey, differently, at 2 different times. Depending on the food I’m eating, or the mood I’m in. People often ask “what is your favorite whisky?” How do you answer? Most people say that it depends on their mood. If your favorite whisky was always the one with the best score, your favorite would never change.

    I use the point scale as a basic guideline, realizing that it is completely subjective. I like using 100 points, because it allows for subtle differences. We could use a 1000, or even a 10,000 point scale, but that would be ridiculous.

  36. Jason Pyle says:

    Ahh the rating system. The imperfect “science”. John, you make some good points and it’s important to remember this is also the opinion of the reviewer and nothing to get full of ire over if someone disagrees. One man’s 80 is another mans 90, and that’s ok.

    Also, if you don’t mind I’d like to add a pet peeve. I’m not a fan of giving each category a point value, that when totaled equal the final #. When a reviewer takes something that is subjective as it is, and then makes it further so by attributing max points for Nose, Palate, Finish, and Balance/Other, the reviewer is not injecting “science” or process in the mix like they may think. How many times have you tasted a whiskey, were blown away by the nose, let down a bit by the flavors on the palate, and further impressed with the finish? Should some of these categories compensate for lapses in others? I think so. I’ve had whiskeys I’d nose for hours and were rather simpleton on the palate, and vice verse. Certainly there must be consistency, but there are other ways to accomplish that in my humble opinion.

    If I’m totally honest, I use a 10 point system primarily to keep it from being confused with other systems. It just gives the ability to reclassify what I think vs. being compared with what another reviewers rating systems.

  37. The problems with the 100 point scale from the obvious one that it’s not really a 100 point scale (only 25 or 40 point range) are

    • it gives the impression of a precision that it does not have
    • it implies a linear measure of quality/goodness that it does not have

    So you get into these arguments/discussions over a single point or two. And as the numbers go up it gets harder to get to the next number. So the scale really isn’t linear.

    I’m glad there aren’t little shelf talkers on whiskey in the shop trying to entice me with their “90+” points from Malt Advocate! And to the person looking to buy something who knows only a little, and there’s no salesperson to assist them (and they might not be knowledgeable about whiskey), there is a big difference between 88 and 90.

  38. MrTH says:

    A quibble, John:

    “For example, if a reviewer rates a whisky an 83 and you think it should be an 87, they would both get an 8 on a ten point scale.”

    No, the first would be an 8, and the second rounded to a 9. And this points up the precise [sic] usefulness of a 100-point scale: its inherent imprecision. Once you get over the idea that you can objectively tell the difference between an 84-point whisky and an 86-point one, you realize that there is a margin of error in every score. Far from being a precise point on the scale, each score is more of a fuzzy cloud. On a ten-point scale, scores of 8 and 9 would imply a relatively large gulf between these two whiskies; on the 100-point scale, it’s easy to see that their fuzzy clouds would overlap to a greater degree than they are separated.

    That said, I have no interest whatever in attaching numerical scores to whisky, or beer, or restaurant meals, or movies, or cars, or women’s beauty (haven’t attempted the latter since I was 17). I can’t see why I would want to. I do look at reviewers’ scores, but agree that Roger Ebert has it right: read the review. The score is just a shorthand way of rating, and shouldn’t be taken all that seriously. (And it seems obvious to me that the people who take it too seriously are the ones most vehemently opposed to using a 100-point scale.)

    And now, I’m going to pop open my newly acquired bottle of Spinal Tap Single Malt. It scored 110.

  39. Brendan says:

    Cheryl’s point is an interesting one. It probably is harder to go from 88 to 90 than it is to go from, say, 48 to 50, but the difference between 48 and 50 is irrelevant because I wouldn’t buy something with either rating even though, on a 100-point scale, both whiskies are in the average range. However, I would be more likely to buy a 90 than an 88 if I’d actually read the reviews and found both whiskies to be of a style that I like.

    On a normal curve, a few points +/- of average makes a much bigger difference than a few points +/- at either extreme, but again, the ratings are skewed. Almost everyone is above average.

    • MrTH says:

      Depends on your definition of average. You seem to think that it’s 50, by definition; I’d say it’s the average of actual scores given. At exam time, average is C+/B-, or about 78. Or at least it was before grade inflation…it’s probably more like B+ now.

      Just because very few whiskies released are truly bad–below, say, 60–doesn’t mean that end of the scale is meaningless. It’s just sparsely populated (thankfully).

  40. Brendan says:

    Hi, MrTH

    On a scale of 0-100, with a normal curve, the 50th percentile is 50 points. It’s not really my opinion. I’ve also noted a few times that we’re not on a normal curve and that, for most raters, the average is closer to 90 – we can read each others’ posts as well as reading complete product reviews.

    My definition of average is the sum of the scores divided by the number of scores. It’s been an interesting conversation so far but it’s also a nice day here.

  41. two-bit cowboy says:

    Where any rating scale fails–and reviews succeed–becomes obvious in style extremes. A sherry bomb that “earns” a 95 is still a heavily sherried whisky that some folks can’t stomach. The same can be said for smoke-heavy whiskies. And the same might be true for whiskies matured solely in first-fill ex-bourbon casks. Etc., etc., etc. So the point of the numerical scale is a waste in these cases, making it more a distraction than a valuable tool.

  42. Michael says:

    Rating wine and whisky may be completely different exercise: 96 pints and higher from Parker influences wine sales much more than any rating system that exists for whisky. There is a huge difference for wine producer to get 90 or 95 from Parker. Some would not even buy wine that does not score at least 97+ (on Parker’s scale). As many of you know, his ratings for Bordeaux futures practically set the prices for wines in a given vintage.

    • One of the reasons for this is that there is not 1 whisky reviewer seen as above everybody else (everybody might have their own favourite or dislikes). Personally I have 1 I can relate to better than others but that doesnt stop me from reading other ratings and reviews for inspiration

      • Michael says:

        Not exactly. There are at least 5 reviewers which scores are included in any decent Bordeaux Futures list. As you know, you are buying (and paying) for wine that you may see in two years. How do you want to decide what to buy if you do not trust at least one reviewer? They have access to wines at en primeur testings. By the way, I pay more attention to Jancis Robinson’s scores (and testing notes) than Parker’s but I do buy wine that I would not be drinking for at least 10 years (no, you do not open those wines after getting them).

      • MrTH says:

        Well, there is one reviewer who aspires to be the Parker of whisky. Long may he fail.

  43. Michael says:

    I would also say that I am influenced by whisky ratings, by some reviewers. I may not be as certain as some here, as far as my own experience and knowledge are concerned. I definitely get interested if certain whisky get a score of 95 and higher. 5 point scale is meaningless for me. 20 point system is a minimum to give any indication of the quality of whisky.

    • Well, I would say that a 4, 4½ or 5 on a 5 point scale or 18-20 on a 20 point scale would give you a good idea :-)

      I still believe the scale is irrelevant, you just want to know if the whisky is good or not, and for that a rating is good.

      Tasting notes can be meaningless as well. I often see tasting notes where I wonder if we had the same whisky…

      Steffen

  44. Patrick says:

    I wrote a short essay about tasting notes and ratings some months ago (http://www.whisky-news.com/En/reports/Tasting_note.html).

    when someone is reading a tasting note, this is subjective to the taster appreciation. The same whisky will probably be scored diffently between John Hansell and Dave Broom and indeed, between a 82 and 83, there is a difference, it tasted by the same taster.

    I am always impressed by all these debates about rating: eg., this Ardbeg deserves a 92 pts and not 88 pts, etc. This is an ongoing discussion and most whisky enthusiasts are considering these values as hard “scientific” endpoints.
    At the end, the only thing that matters is that a rating is there to express OUR appreciation of a given whisky at a given time and moment and at the end. If you love a badly rated whisky, what does it matter as long as you enjoyed it!

  45. mashbill says:

    Of course John loves the 100 point system. It allowed him to sell to Shanken, who loves it even more.

    • John Hansell says:

      Yes, this isn’t the first time you’ve expressed your love affair with the Shanken group. (In case you havn’t noticed, the magazine has only gotten better since we joined the group, and it will continue to do so.)

      • sam k says:

        You’re assuming he’s a reader! :>)

      • Douglas Fischer says:

        I actually have noticed this – it has also gotten longer (and therefore needed a different binding). The number of articles and reviews now makes it a two-night read for me instead of one. I like.

        The only minor quibble I would make is the addition of the tobacco ads, but I don’t really consider it to be a real issue, especially since it’s not surprising given Shankan’s other holdings (and the obvious business benefit of cross-market advertising – no doubt there are now whisky ads or even MA ads in Cigar Aficionado), and the fact they are done tastefully and kept to a minimum.

        I also have no doubt the record speed with which the WhiskyFest NY VIP tickets sold out this year is due in part to marketing efforts by Shankan (ensuring my wife and I will be buying our WhiskyFest Chicago 2012 VIP tickets the day they go on sale). It’s clear they are doing right by MA, and that’s good for all of us.

        Cheers.

  46. Danny Maguire says:

    But it, drink it, chuck it. A very simple system and that is its charm. I’ve tried whisky’s in all 3 catagories but I can honestly say the worst I’ve ever tried is a German one called Slyrs. Have you ever reviewed it John? If so, how did you rate it? Its colour was quite good, a nice dark straw, nose a bit metalic, palate and finish both very metalic. I’d like to visit the distillery and see what kind of stills they’re using. How would I rate it? It’ll have to be in the bottom quartile.

  47. Brendan Sophocleous-Jones says:

    I cannot speak to whisky scoring but, as a wine collector and as some commentators above have referred to Parker’s 100-point system, I’d like to clear up any misunderstandings here about the 100-point system. It is not a 100-point scale; it is a 100 point *system* with a 50 point scale and the scoring starts at 50.

    See http://en.wikipedia.org/wiki/Robert_M._Parker,_Jr.#Parker.27s_100-point_rating_system

    Although both Parker (Wine Advocate) and Wine Spectator simply allocate a score to a wine (the benefit of experience, one presumes), other reviewers sum individual qualities as follows: 5 points for color, 10 for nose, 15 for taste, 10 for finish, and 10 for overall impression.

    As you can see, dropping 10, or even 5, points in one quality alone is a significant difference.

  48. Mike B. says:

    I ‘m not a fan of a point system, but use it just to understand what a expert thinks. I find it all comes down to your perticular taste. Parker and others very rarely do blind tasting, that why a 9 dollar bottle of wine never cracks 90 pts. and in some case they really should. John, if you were to do a blind tastings, do you think that some lower rated whisky would rate better? Case in point , Amurt fusion, Price pt to dollar value is fantastic. I think some brands stack up, and would rank higher in a blind tasting. For me rating systems are subjective guide. The taste description, is what i really look for, flavors help me pick my next bottle. Its just comes down to a personal taste.

    • Brendan Sophocleous-Jones says:

      To: Mike B.

      >Parker and others very rarely do blind tasting,

      This is incorrect. From Parker’s own site, http://www.erobertparker.com/info/legend.asp:

      “When possible all of my tastings are done in peer-group, single-blind conditions.”

      Wine Spectator also taste blind. See: http://www.winespectator.com/display/show/id/about-our-tastings.

      > that why a 9 dollar bottle of wine never cracks 90 pts.

      This is almost true but it is misleading. Parker has given 90+ scores to many very reasonably priced wines. He even produces a book, “Parker’s Wine Bargains”, dedicated to bargain-priced but quality-drinking wines.

      If you search for “cheapest 90+ parker wine” on Google you will find a number of wines around the $9 mark that have scored 90 points (or more) from preeminent reviewers. It is true that there aren’t very many of them at that price, but that is because wines around that price (or lower) tend to be not particularly good. Around the $15 dollar mark, however, there are—literally—hundreds of good wines with good scores available.

      The 100 point system has its faults and Parker isn’t perfect, but your post is inaccurate.

      • Michael says:

        Very good points. As a matter of fact, I do not understand where this desire to see $9 wine get higher rating than a $900 bottle comes from. By the way, I have never had a good wine that was cheaper than $20. In reality is is usually around $30-40 and never French (for this price).

  49. Jeffrey Woolley says:

    The real issue is not merely the sum of possible points, but how one awards those points. Most reviewers use a holistic rubric (whether they know it or not). In my opinion, an analytical rubric encourages more discussion and though it remains subjective at least reveals its specific criteria.

© Copyright 2014. Whisky Advocate. All rights reserved.