Making sense of strip-trial data doesn’t have to be complicated, said Emerson Nafziger, University of Illinois Professor and Extension Agronomist at the 2010 AGMasters Conference in December.
“We can think of crops grown in fields as a ‘population’ of plants,” he said. “Basically when we apply some sort of treatment, we need to know if this forms a new population or not? Statistics, which might be considered the science of describing variability in a population, can help us figure out what is really happening.”
Nafziger provided conference participants with a short course in statistics using actual strip-trial data from on-farm trials to show how strip-to-strip variability affects results, and to explore what it means to be “significant.”
Using an Excel spreadsheet, Nafziger analyzed data in trials to explain statistical terms and how to use them to interpret results.
“The ‘truth’ – did a treatment cause a response or not – always exists, it’s just our job to find it,” he said. “We aren’t in the business of doing ‘nice’ trials; rather, applied research is the business of trying to say something when we are done.”
Because on-farm trials often have a great amount of variability, he said it’s important to do random treatment assignments.
“With ‘yes-no’ type inputs, for example to use a fungicide or not, assign treatment randomly to one strip of paired strips,” he said. “This should be done before planting or right after, in order to prevent bias in keeping or dropping data.”
When designing on-farm research trials, Nafziger said to keep it simple.
He recommends using a strip size wide enough to allow borders. He also encourages growers to randomize within each repetition, use 4 to 8 pairs of repetitions per location, keep accurate records of where things are planted, measure yields accordingly, and convert to standard moisture in a standard way.
Nafziger warns growers not to discard data unless they know for sure what happened to cause the data to be untrusted. When the study’s completed, stop and get your answer, he said.
A “significant” effect or difference means that the treatment was likely to have caused an effect, but it does not mean that the treatment is useful or that it will pay. He said we can also be fooled, and get “significant” responses due to the “luck of the draw” when we assign treatments to strips. The chances of this diminish quickly as we use more fields to do such comparisons.
“Non-significant results can be obtained from no effect of the treatment or from so much variability among strips that we can’t separate a treatment effect from the ‘background noise’ of variability,” he said.
While there is always the choice to not accept such a conclusion as final, Nafziger cautions against thinking that more work will produce an outcome that we like better.
“Don’t cherry pick your data to give the answer you want; if you need to get a certain answer, why bother to go to all this work?” he said. “Our point is not to find significance, but rather to figure out what happened and where we go from there.”
No matter what, getting predictive answers to applied research questions takes a great deal of work, and getting good answers takes honesty and even more work, he said.
“There really aren’t any shortcuts,” he said. “Statistics do not substitute for the large amount of data and keen observation that good on-farm research always requires.”