How reliable are wine judges? When I saw the headline, I assumed the worst. Newspapers like The Daily Mail like nothing more than printing stories about professionals who can’t blind taste the difference between a classed growth claret and a cheap Chilean Merlot. Why spend £100, they snigger, when the stuff that sells for £5.99 tastes just as good to “so-called experts”?
But this piece was different. For a start, it appeared in the excellent German trade magazine, Meininger’s Wine Business International, rather than a net curtain-twitching tabloid. And it also came to a very different conclusion, one that underscored the value of experience and expertise and questioned the idea that blind tastings are random, scattergun exercises.
The article was about Christian Wolf, the tasting director of Mundus Vini, one of Europe’s largest wine competitions. You could say that he has an interest in promoting the usefulness of blind tastings – they pay his salary, after all – and, while we’re about it, I should declare an interest as a co-chair of the International Wine Challenge. But Wolf’s findings were still interesting.
His novel idea was to divide his judges into eight groups and make them taste together – separate tables of men, women, Germans, older and younger judges, sommeliers, wine writers and members of the wine trade (importers, retailers and producers, I assume). He doubled up on flights and individual wines to see if the groups came to different conclusions about the same entries. They didn’t. Most of the time, the scores were “very close”, apparently. “Overall,” he said, “it was interesting to see that the higher the professionalism of the people you are inviting, the less important is their age, sex, origin or profession”.
The one thing that groups disagreed about was Brettanomyces, but not in the way you might imagine. I assumed that older judges, raised on the idea that Aussie Shiraz could legitimately pong of sweaty saddles and that “great Burgundy smells of shit” would be more indulgent towards the fault. But the opposite was true. They were more critical than the younger tasters. Perhaps the natural wine movement has made the latter more accepting of a bit of funk.
Wolf ascribed the close correlation on most wines to the “modified version of the OIV system” that Mundus Vini employs. This, he claimed, was “the key to consistent results”. I strongly beg to differ here. Modified or not, the OIV system is a joke, a pointless exercise in form filling that detracts from the meaningful analysis of wine. Like several other prominent international judges, I refuse to participate in any competition that uses it. Sorry, Mundus Vini.
I’d argue that the results were a tribute to the quality of the judges rather than the OIV system. We all know that there’s a large degree of subjectivity about wine – one person’s sumptuous, full-bodied 16.5% red is another’s overblown rocket fuel – but I find that experienced judges generally agree on a wine’s merits or shortcomings, even if it’s something that they wouldn’t choose to drink themselves. Incidentally, that’s why discussion – something that the original OIV system actively discourages – is so important among panel members.
No one is born a great taster. Some people have a natural aptitude for it – Oz Clarke is the best example I know – but you don’t learn to taste well without putting in what Malcolm Gladwell in his book Outliers called the requisite 10,000 hours of practice. To do so, you need a thick skin, determination, stamina and the ability to listen and learn from people who know more than you do. One of the greatest things about the wine business is its generosity of spirit.
Many people don’t regard wine tasting as a serious profession. A friend of mine calls it “the most unjob-like job in the world” and she has a point. No one would deny that swirling, sniffing, gargling and spitting is a lot more fun than working in an Amazon warehouse or an abattoir. But it’s still a job that requires dedication, patience, knowledge and the acquisition of a few knocks along the way. Just ask the seven new Masters of Wine who were admitted to the Institute last week, all of whom would have put in plenty of hours at the tasting bench.
The next time you read an article in The Daily Mail laughing at wine judges and deriding our expertise, throw it in the bin. All tasters make mistakes, but more often than not the good ones reward the right wines.