In a pickle over the peer review process

So much is forfeited by those who demand statistical certainty, says Dr Les Sandles. Photo by Atitaya Pimpa

Where did it come from, this misinformed obsession with ‘peer-reviewed’?

These days, I rarely have a technical conversation with a farmer, service provider or researcher without immediately being confronted (and affronted) with a version of “I don’t do anything without (“Where is the ...) peer-reviewed data?”.

If I have it, the next question: “Is it Australian?” Well, yes. “Oh, is it from this region?” Yes. “Well, hmmm, was it conducted next door?” Okay, that last one I made up but it is often that ridiculous. It drives me insane!

And this is because, as a field scientist, innovator, disruptor and consultant, my small fraternity makes its living (and reputation) on contributing new useful stuff, often transformational stuff and, if what we contribute doesn’t stack up financially, we are soon out of work.

On the other hand, it frequently takes years for institutional science to catch up.

Sure, not all ideas work, but then, not all research is correct, and the peer review process itself is inherently flawed.

By the time maverick consultants peddle ideas about as part of our professional offering, we have a high confidence level based on experience, observation, need or fit, and the bravery of our best mate who has his adventures trying things out and is okay if here and there, the ideas turns out to be s**t.

A few bright ideas particularly close to my heart (and business) include:

  • Lead-feeding: Introduced in the late 1980s and ‘proven’ in 2007. (Farmers sent the researchers back to repeat the research three times before they got the right result.)
  • Canola meal: Introduced in 1989 and ‘proven’ in 2013. (Spruiked as an exciting development in 2023.)
  • NDF as the key constraint to feed intake in Australian dairy herds: Introduced in 1987 and actively contested by our scientists until 2008.

The futility of waiting for peer-reviewed research is clear: about 15 years of navel gazing while the doers are implementing!

Why 15 years? Well, because it takes six or seven for the scientists to be confident there is enough ‘anecdotal’ evidence that they will get a publishable result, two years to go through the funding/approval process, three years to conduct the research program, and two years between the research and write-up, then the best part of another for the peer review process and publication.

Surely, at some point there is merit in ‘anecdotal’ evidence. Is it 100,000 cows or 200,000? A million? Surely such numbers getting the same result must provide some confidence.

Peer-reviewed desperados — heed this.

Economic opportunity exists in the time and space between the statistical and economic significances. The former relies on variance — the response ranges within a data set. The latter (for us) is simply milk in the vat.

Take canola meal. The typical response to feeding just 1kg in early lactation is 5-7 litres — but we expect and accept responses ranging from zip to 20 litres. This is great business.

In contrast, gaining the requisite statistical probability of repeatability (the P value) necessitates large numbers of animals (n) to cope with the variability, and is therefore expensive and difficult.

So much is forfeited by those who demand statistical certainty.

Institutional thinking is, and must be, iterative — slowly, carefully, building on what was proven before, usually in the shadow of an academic tyrant (otherwise known as project leader, supervisor, or professor).

Institutional research discourages transformational thinking because it relies on agreement rather than adventure (this is why I have always been unemployable), and necessary experiments with poor probability of confirming known outcomes are unlikely to be conducted.

The big breakthroughs — the stuff that is potentially business, life, or planet changing — are those who constantly ask “What if ...?”

Dr Les Sandles is a renowned thought leader and provocateur in the dairy industry. Best known for his role in revolutionising nutritional and pasture management practices, Les has turned his attention to the ‘last frontier’ — transmogrifying the forage production system into a C-munching machine. Contact him at: info@4sight.bioif