Augmented Reality Farming is Here
We're close to the augmented reality farming portrayed in Meta’s recent commercial, but will it become a mainstay of modern agriculture?
Welcome to AgPunk, a newsletter about agriculture, sustainability and technology as seen through the lenses of science fiction and solarpunk. Today we’re exploring the feasibility and relevance of augmented reality (AR) farming.
“The impact will be real.”
At least that’s what Meta proclaims in its recent campaign showcasing a future in which AR powers multiple professions, including farming.
One commercial depicts a farmer wearing AR glasses as he analyzes the current soil moisture conditions of a fruit-bearing tree. The glasses forecast how additional soil moisture might impact yield, presumably so that the farmer can immediately adjust the tree’s water supply for better results.
In another scenario, we might envision a farmer jumping down from her emission-free, electric farm truck and gazing out across ribbons of grain and legume crops alternating along contoured lines. A hive of automated, autonomous cultivation machines target nutrients based on real-time data from digital soil probes and leaf sensing capabilities, delivering significant reductions in nutrient and pest management input costs.
She moves on to the neighboring field. This field is devoted to a diverse, regenerative agroforestry system that integrates two stories of perennial, high-value crops with a flock of sheep. She dons her AR-enabled, artificially intelligent glasses for daily metrics on soil conditions across this field, growth progress of specific plants, and whether any plants are nearing harvest. It’s a future where AR drives farm productivity and nurtures sustainability.
Mainstay or mirage? Probably a little of both.
Already, AR-enabled farming is technically viable, but its accessibility and relevance is limited to smaller, specialty growers, agronomists and educational settings. But will AR change the future of farming and pay off – particularly with rising skepticism around whether digital will live up to its potential?
If you consider the following three key assumptions, I believe a future of AR-enabled farming is attainable:
Farmers want and need access to real-time agronomic data.
We can predict crop yields with some degree of confidence.
AR will enhance, but not usurp the role of the advisor
Assumption 1: Farmers want and need access to real-time agronomic data
Capturing real-time agronomic data about soil and plant health can be a gamechanger when it comes to making smart, targeted decisions that save a farmer money and preserve resources – but it’s also the biggest barrier to moving the needle on precision agriculture and soil health practice adoption.
Where We Are Today
Plant / Tissue Sampling: Farmers can send plant tissue samples to a lab – either as part of an ongoing fertility management program or at the first signs of nutrient deficiency. Lab analyses can be performed within days of the sample being received by the lab, but often are challenged by user error in collecting and shipping the samples.
Soil Sampling: Commercial farmers typically take soil core samples once a year from multiple locations across different fields, and then send them to soil analysis labs. The results include a range of soil condition data, including pH, nitrogen, phosphorus, potassium, soil organic matter (which can be used to estimate soil carbon), and other conditions upon request.
Yet current soil sampling accuracy is limited by three factors: depth, timing and location.
While the roots of a corn crop can grow up to 60-inches deep, soil labs recommend sampling only to the top 6 to 12 inches to assess fertility, and researchers, such as Blanco-Canqui et al. (2021), recommend that soil samples be taken from depths of up to 40 inches to read soil carbon levels.
There’s often a two-week gap between sampling and analysis, which presents a challenge to farmers as weather events don’t wait.
Random samples can’t deliver spatial specificity. “Composite samples” can only provide a generalized view of soil conditions across a field.
“Grid” Sampling: Grid sampling can offer greater locational precision, but the practice is only adopted on 10 to 40% of farms. Sampling locations are selected intentionally to determine how soil conditions might be impacted by crops, slope, and other variables. Grid samples can also help a farmer target nutrient and pesticide applications to produce less waste.
What the Future Holds
While grid sampling gets us closer to precision insights, farmers still face notable depth, temporal and spatial challenges with capturing a real-time picture of agronomic conditions. Fortunately, the near-term innovation pipeline holds promise for more precise, real-time soil and plant measurements, including:
Commercially available tech-powered probes that provide near-instant readings, including ChrysaLabs, Teralytics and Stenon (there are still kinks being worked out with Teralytics, it seems).
Aerial imagery - It’s appropriate that Meta used “soil moisture” as the data read by its AR glasses, since soil moisture is one soil condition that scientists have been able to reliably read using alternative methods, such as aerial imagery or machine learning.
Embeddable sensors, or “plant tattoos”: Sensors that can be adhered onto or embedded directly into the stalk of a plant and give a read on the plant’s nitrate levels or water status.
Leaf scanning technology, like that provided by Leaf Spec Ag Technologies, “provides a digital lab solution for growers to identify in real time the nutrient and chemical compositions of plants.” According to Zhang et al. (2019), leaf scanning technology can analyze the chlorophyll content in a leaf, and correlate that with plant nutrition.
How close are we to achieving the functionality of Meta’s AR headset – having insights powered by machine learning models fed by data from digital soil probes, plant tattoos, remote leaf scanners and aerial imagery? Probably 5 to 10 years, based on the current tech pipeline and the reality that this technology doesn’t need to be hyper-specific to the conditions around or in a given plant (particularly in larger, commercial field crop operations).
And new technology could always disrupt the above. For example: what if we created nanobots “shepherds” that live in the soil and drive around capturing data that is then mapped back to the specific location in the field? Or might we somehow breed plants with sensing technology embedded within the genome that we could access via the Internet of Things? The beautiful thing about technology is that ideas that might seem fantastical today could power potent innovations to come.
Assumption 2: We can predict crop yields with some degree of confidence.
Where We Are Today
Seed genetics can dictate maximum potential, but yield is also shaped by the environment in which a seed grows (i.e., water, nutrients, soil structure and biology, weather).
Farmers have long relied on a field’s historical performance to predict crop yield. Crop simulation models that synthesize experimental data and user calibration can also estimate yield.
To assess whether these existing methods work for farmers, I asked three established corn and soybean farmers how they relate to yield prediction accuracy today. (Working with farmers over the last decade, few have indicated yield prediction capabilities as a major pain point – so I was curious to hear their thoughts.)
In general, yield predictions for corn and soybeans aren’t accurate enough to take seriously, unless they’re made late in the season.
Most farmers still rely on historical performance for yield estimates.
Better soil health can deliver more stable yields, but weather and other uncontrollable factors still have the biggest influence on yield.
Many farmers use and are satisfied with tools from Climate Corp and Granular to predict yields based on management decisions.
For specialty crop growers who produce nut and fruit crops, the ability to predict harvest time and plant yield is more critical. Many of these growers structure their contracts around certain harvest targets, so having accurate insights on harvest dates and yield will help these growers capture better economic outcomes.
What the Future Holds
The frontier of yield prediction technology lies in machine learning models.
A recent study showed that pairing machine learning models with crop simulation models leads to better accuracy in yield prediction.
Crop simulation models excel at taking in environmental data. By pushing this data into a machine learning algorithm, the yield predictions become more accurate. Rashid et al. (2021) analyzed existing studies of yield prediction methods and found accuracies of 10 to 40% across various models, with machine learning algorithms leading to better outcomes.
Remote sensing, either through satellite sensing of Earth conditions or computer vision at the ground level, enable the capture of more real-time data to inform crop systems and machine learning models. For example, there’s been significant investment by Google and Driscoll in computer vision-powered crop counting technology.
Pairing these technologies offers the greatest accuracy in predictions. For example, NASA has created a new modeling framework called GEO-CropSim that integrates remote sensing (”Earth observation”) with crop system models. Geo-CropSim has been shown to produce more highly accurate results than a crop systems model alone.
Currently, farmers seem satisfied enough using historical field data to set yield targets. And while there’s no way to predict yield with 100% accuracy given the influence of factors beyond a farmer’s control, by combining historical data, advanced machine learning models and remote sensing, we can get pretty close to future AR aspirations, today.
Assumption 3: AR will enhance, but not usurp the role of the advisor
Where We Are Today
Farmers today rely on a trusted team (agronomists, certified crop advisors, pest protection professionals) to make decisions on the farm (see Houser et al. (2019)). Agronomists and certified crop advisors, however, are becoming more tech-enabled to stay relevant, and more farmers are relying on software and peer networks to drive field decisions.
Whether AR technology will replace advisors (or at least change the way farmers and advisors interact) is contingent upon how the incumbents in the industry invest their time and money. Input sales are inextricably tied to agronomic advice in the current system. As input companies emphasize margins over sales, I believe we’ll see a significant shift from human-driven to tech-driven agronomic advice.
That shift is already happening today:
Companies like AgVend are helping agricultural retailers offer their services digitally to their field teams and farmers.
A partnership between Winfield United and AgVend enables “participating sales teams and growers [to] have access to WinField United programs – including loyalty and incentive programs, financing options, customized agronomic prescription plans, risk mitigation warranties and digital marketing campaigns – through the retailer’s AgVend-built digital platform.”
What the Future Holds
It’s feasible to expect that AR-enabled agronomy could provide the insights and advice large-scale farmers need to make critical decisions, but the likelihood of AR fully replacing that human touch is less certain. We’ve seen this play out with digital technology like Granular and ClimateView, where farmers have shown less interest in investing in the tools themselves, and more willingness to having their advisors use these tools to provide agronomic advice.
There are scenarios where AR-enabled agronomy makes more sense.
Specialty growers with smaller plant densities.
Larger commodity farmers using drones and computer vision to collect data over large plots of land.
In the classroom, helping agricultural students level up their understanding of in-field dynamics.
Summing It Up
AR-enabled farming is likely within reach in the next decade, should we choose to pursue it. It will hinge upon critical advances in agronomic data collection and yield prediction, and it will be enabled by an already-underway cultural shift in how farmers seek agronomic analysis and advice. If we pursue it, AR-enabled farming might have the greatest relevance for small farmers, specialty crop growers, and agricultural education initiatives.