What happens if you take two wine critics with opposing, sometimes polarised views on natural wine, throw them into London’s biggest celebration of the sector and force them to taste the same wines together?

The seeds of this “challenge” were sown after the 2013 RAW Fair, an event that struck Robert Joseph and Simon Woolf in very different ways. Simon’s experience was overridingly positive, while Robert’s was scarred by some truly horrible flawed wines. When we discussed our differing reactions, we discovered that Robert had focused his attention on French wines, while Simon had been busy exploring other countries. Wouldn’t it be interesting to taste the same wines and to compare notes?

To give the exercise a modicum of scientific credibility, we followed a randomised path around the hall, tasting one wine from each of 50 or so producers. We also agreed:

• We’d taste at least one wine from each country represented

• We’d try to get the producer to select which of their wines we taste

• We’d mark all of the wines out of 10, with 5/10 being the borderline between a sample being basically drinkable and unpalatable.

The Tasters

Robert: I have a reputation as being unashamedly “anti-natural”. In fact, I love innovation and experimentation and applaud the continued efforts to reduce the doses of SO2 used in winemaking. I also like the complexity I find in white wines, in particular, that are fermented with natural, indigenous, yeast. On the other hand, I don’t like vinegary – volatile acid – nail varnish – acetone – mousey, stable-floor, Elastoplast, cardboard flavours that are the mark of oxidation and/or bacterial spoilage that might have been countered by the judicious use of SO2.

As natural wine fans like to point out, these faults are not the exclusive preserve of wines made with little or no SO2; they’re just more closely related – like automobile accidents and drunk drivers. Some low SO2 wines are both stable and delicious, but the definition of “low” is open to debate and the stability of zero-SO2 wines has yet to be proved.

My main argument is with some of the more vocal natural wine lobbyists. As an agnostic over most things, I react very badly to Christian evangelists, Muslim fundamentalists, political activists and anybody else who seeks to persuade me that theirs is the one true path that I and everyone else should follow. And as a writer, I care a lot about language. So, I resent the misappropriation of the term “natural” and the implication that thousands of wines I have enjoyed throughout my life are un-natural.

Simon: It’s partly accidental, but I do have a reputation for being enthusiastic about natural wines. However, I’m no apologist for bad or faulty examples, and in many ways I’m just as sceptical as Robert. The difference is maybe that I take more of a “glass half-full” attitude to new discoveries, whereas Robert is quicker to judge.

I don’t believe in pigeon-holing, and for me the term “natural wine” is also problematic. I like the fact that Raw Fair terms itself “the artisan wine fair”. I hate evangelising sommeliers or retailers who preach the gospel and forget that evaluating, selling and enjoying wine in an honest and understandable way is much more important.

Perhaps we’re not so different after all Robert?

What were we looking for?

Simon: Simple – I wanted to discover delicious wines, whilst at the same time empirically evaluating the overall standard . I was prepared for some ups and downs, but the question was, would we agree on which was which?

Robert: Like Simon, I was looking for quality, but I also wanted to discover whether natural wines tasted identifiably different from conventional ones – and if so, how one might describe that difference, and whether it related to the low levels of SO2 used. I was interested – and pleased – to see Isabelle Legeron’s decision to establish 70mg/litre of total SO2 as the ceiling beneath which a wine could be defined as natural. Would we find appreciable differences between wines at specific sulphur levels between zero and this figure?

Revelations/surprises

Simon: I’d assumed that Robert and I might disagree on whether some of the wines were acceptable, or consumer friendly – and that we might not even agree on whether a wine was faulty or not. It turned out that we almost always agreed on the fundamentals – and Robert showed impressively catholic taste, for example praising a heavily tannic qvevri (orange) wine from Georgia’s Mstvane variety that I felt was too young and aggressive.

From my perspective, personal taste turned out to be the biggest source of disagreements. I am clearly more comfortable with cool climate wines that have “hard” verging on green tannins, whereas Robert prefers more seductive flavours and textures.

I was impressed with some unfamiliar skin-contact and aged whites from Northern and central Italian regions like Lombardy and the Veneto – wines I may have missed without our random sampling approach.

Robert: I certainly enjoyed tasting with Simon and, not entirely coincidentally, had a much better time than 12 months ago. None of the wines we tasted was remotely as unpalatable as the worst efforts in 2013 and some were truly enjoyable. As usual, I tried to look beyond my personal preferences: were these wines I could recommend to other people? There were certainly wines in this tasting that I’d buy and some that I might not choose to drink myself, but would happily set before people seeking a low-SO2 wine.

This last category would not include Frank Cornelissen’s Etna wines, despite my having a rush of blood to the head and giving two of them 7/10 (the others got 6 and 6.5). These are highly-priced rustic wines I’d be very happy to leave firmly uninvited to my dinner table. I cannot imagine how anyone who really enjoys these would ever relish my kind of classic, elegant Burgundy, or excitingly peppery northern Rhones.

Our marks

Marking out of 10 – as we agreed – was a relatively blunt tool, even with the occasional half or quarter point. As these charts reveal, while Simon (red) is a marginally more generous marker than me (blue), there were very few major differences in opinion between us. And certainly far fewer than anyone wanting to watch a punch-up might have hoped for. There were just seven wines where our opinions diverged significantly.

Comparing marks to the declared SO2 levels was instructive, but not surprising. Robert very evidently prefers wines with at least moderate levels of SO2: nearly half of his top 25 had at least 50g of added SO2, according to the information we were provided. On the other hand, he gave good marks to five that apparently had no added SO2 at all.

It was his most poorly-rated wines that told the story, however. All but three had 25g or less of added SO2.

Robert: When I looked at my notes, there were consistent comments about these wines. Apart from the ones that were frankly ‘grubby’ and ‘volatile’ – vinegary – there was a litany of ‘cardboard’ – a characteristic I associate with oxidation – and ‘hard’. The wines with the highest – a relative term in this context – levels of SO2, were the ones that were more likely to described as ‘rich’ in texture.

There were more wines with under 25g in Simon’s top 25 than Roberts: 11 versus seven. But the chart of his most poorly-marked wines also tells a story:

Roughly three quarters of these wines were the ones with the lowest SO2 levels. Now, to be fair, 11 of these wines scored a very decent seven or seven and a half out of 10, so it would be wrong to say that Simon disliked these wines. But he was marking more generously on the day, and there is no getting away from the fact that these were evidently not his favourites.

Disappointments

Robert: Ultimately, this tasting confirmed my opinion that “nature” doesn’t make wine any more than it makes Cheddar cheese or chocolate cakes. Isabelle Legeron’s statement (in the FT) that “You need to trust that nature does the best job possible.” is plainly ridiculous.

The best winemakers – avowedly “natural” or otherwise – are like the best sailors: people who respect and understand the environment in which they work. They neither allow it to run the show, nor seek to write it out of the script. The low-SO2 wines I rated well were evidently made by people who knew what they were doing. I believe that many of the others were produced by those who followed Ms Legeron’s advice to “trust nature”.

Simon: I was concerned about whether the standard was going to be lower once we got to France – the largest group of producers by far, and also the last country we tasted. My concerns were valid – the only two outright faulty wines of the day were French, a Loire red with a serious VA problem and a clearly corked Bandol. The producer gave a Gallic shrug rather than opening another bottle. These two wines did of course score 0/10.

It’s a shame that the French still appear to be complacent – if not downright careless.

We paid close attention to production methods, and I would have to admit that the no-sulphur white wines faired less well than those with a modest dose at bottling. Wines like Angiolino Maule’s otherwise wonderful Sassia were marred by dirty finishes. I’d like to believe Legeron when she says this “mousiness” disappears after more time in bottle – had we tried the 2012 rather than the 2013, maybe this conclusion would read differently.

I’m neither ardently pro or anti the no-sulphur approach, but the risks are all too plain, at least for white wines made without skin-contact.

I did have a concern that Robert paid such close attention to the SO2 levels of the wines that we tasted that he was almost predisposed to dislike those wines which were stated as “no added sulphur” in the booklet. Even someone as experienced as Robert is not above occasional auto-suggestion, however neutral they try to be.

I wasn’t out to “convert” Robert, as some implied, but it was heartening that we agreed on a generally high standard – very few bad wines, and only 2/50 faulty ones – 4%. In many ways, our experience reinforced the obvious – personal taste is everything, far more important than dogma or labels.

Favourite wines

Robert

Cullen Vineyards Mangan Vineyard Merlot, Malbec, Petit Verdot 2012, Margaret River, Western Australia (69mg/l SO2)

Lovely, pure wine with ripe but subtle fruit and some peppery spice from the Malbec. Really stylish, complex Western Australian wine with a sense of place.

Seresin Leah Pinot Noir 2010, Marlborough, New Zealand (30mg/l SO2) Attractive subtly perfumed New Zealand Pinot Noir with lovely pure wild raspberry fruit. Very stylish.

Das Hirschhorner Riesling Buntsandstein 2008, Pfalz, Germany. (69mg/l SO2)

Classy, appley-limey Riesling with lovely ripe, dry purity.

Peter Wetzer Kekfrankos Spern Steiner 2012, Sopron, Hungary. (68mg/l SO2)

Blueberry, cherry stone and real minerality are all keynotes here, plus a note of savoury, ripe olive character, Delicious.

Les Clos Perdus L’Extreme Blanc 2012, Languedoc, Frace. (40mg/l SO2)

A terrific blend of Grenache Gris and Blanc from old vineyards. Mineral and fresh with some fascinating smoky, dry peppery character. Will be great with food.

Simon

Podere Scurtarola Vermentino Bianco 2008, Tuscany, Italy

2 years in barrel has given this wine a voluptuous creamy texture, without any sacrifice of freshness. Still plenty of ripe peach fruit, with more herbal, savoury character on the finish. Impressive.

Fattoria Mondo Antico Sinodo 2009, Lombardy, Italy – No added sulphur, Blend of Barbera, Croatina, Uva Rara, Uva della Cascina, Moradella

Crunchy red berry fruit, with a hint of cacao. Typical Barbera nose and weight, beautifully balanced and elegant, with the Croatina giving just a bit more heft than otherwise expected.

Vina Čotar Vitovska 2011, Kras, Slovenia

Textbook Vitovska, slightly wild, peach and pear aromas, quite grippy in a sophisticated sort of way.

Andreas Tscheppe Salamander Chardonnay 2012, Styria, Austria – no added sulphur

Intense perfumed nose (peach blossom), lifted acidity, quite full bodied, super finesse and minerality on the finish

Seresin Rachel Pinot Noir 2010, Marlborough, New Zealand

Restrained yet ripe, sweet fruit. Featherlite tannins, everything in perfect balance. Burgundian elegance, but with its own exotic character.