Skip to content
PenzaıHUMAN TECH
The Penzai mascot inspects a chili pepper leaf with a magnifier
Phase 5 of 8 · In development

Detecting plant diseases with AI: the AgriAI agent

The AgriAI Crop protection agent analyzes photos and climate to spot stress, deficiencies, disease and insects early, and proposes action.

Crop protection agent

Spots problems before they become damage.

What it does

  • Analyzes photos from fixed cameras or your phone
  • Recognizes early signs of stress, deficiencies and disease
  • Counts insects on photographed traps
  • Estimates disease risk from climate conditions
  • Proposes the treatment and logs it after your go-ahead
  • Alerts you or your agronomist with photos and data

How it works, step by step

  1. Fixed cameras, traps and phone photos collect images
    Fixed cameras, traps and phone photos collect images
  2. A computer vision model looks for signs of stress, deficiencies and disease
    A computer vision model looks for signs of stress, deficiencies and disease
  3. The agent combines images with climate data to estimate risk
    The agent combines images with climate data to estimate risk
  4. It alerts you with photos and a proposal: treatment only starts with your go-ahead
    It alerts you with photos and a proposal: treatment only starts with your go-ahead

Agronomic deep dive

The science behind the agent

In crop protection timing is everything: the same disease that affects three plants today can affect a whole zone in ten days. The goal is not to spray more but to see earlier and act only where needed, which is the core of integrated pest management.

Climate drives the risk

Many fungal diseases need specific conditions to develop. Gray mold (Botrytis) is favored by very high humidity and wet leaves, typical of cool nights with condensation; pepper powdery mildew instead benefits from drier conditions and large temperature swings. Knowing these rules lets you estimate risk before symptoms appear.

Insects: count to decide

For thrips, aphids and whiteflies, the most common pests of greenhouse pepper, colored sticky traps are used: yellow and blue traps catch adults and their weekly count shows whether the population is growing. It is also the basis for deciding when to release beneficial insects, the predators and parasitoids of biological control.

What computer vision can and cannot do

A vision model recognizes visible, repetitive symptoms well, such as spots, yellowing or insects on a trap, and above all never tires of looking. It does not replace diagnosis: different symptoms can look alike, and nutrient deficiencies, water stress and viruses are sometimes confused. That is why the agent reports with photos, probability and climate data, and the treatment decision stays with the agronomist.

Integrated and biological control in practice

Integrated pest management follows a set order: prevention (varieties, hygiene, insect nets, correct climate), monitoring, action thresholds and only at the end chemical products, chosen among authorized ones and respecting pre-harvest intervals.

In greenhouse pepper biological control is very common: predatory insects such as Orius and mites such as Amblyseius swirskii against thrips and whiteflies, parasitoids such as Aphidius against aphids. It works if beneficials are released early, while pests are still few: that is why trap counts matter so much.

Viruses and insect vectors

Viruses cannot be cured: they are prevented. In pepper one of the most feared is tomato spotted wilt virus (TSWV), transmitted by thrips. Against viruses the only strategy is to control vectors, use tolerant varieties where they exist and remove affected plants at once. That is why the agent treats a rise in thrips as an alarm even when plants look healthy.

How we train and check the model

A vision model is only as good as the images it was trained on. That is why we check it on photos taken at the Farm Tech and labeled by agronomists, and we measure two different things: how many real problems it finds and how many false alarms it raises. A model that never raises a false alarm but misses half the problems is useless, and so is one that goes off for every dusty leaf.

Treatment records and rules

In Italy anyone using plant protection products must keep a treatment record, the so-called field notebook, and integrated pest management is the reference of the National Action Plan on the sustainable use of plant protection products. The agent fills in the record with date, zone, product and reason, so the data serves both inspections and learning from season to season.

How it works with the other agents

From the Climate agent it receives humidity and leaf wetness, the basis of the risk calculation. To the Fertigation agent it flags symptoms that may come from the roots or from nutrient deficiencies. To the Forecasting agent it reports expected losses, so the harvest estimate accounts for affected plants.

The parameters that matter

ParameterReferenceWhy it matters
Leaf wetnessHours with wet leaves or humidity near saturationIt is the key factor for gray mold and many other fungal diseases
Trap catchesWeekly count per trap and per zoneShows whether and where insect populations grow
Plant imagesRepeated photos at the same spots, in similar lightAllow comparing the same plant over time
Night temperature and humidityWatched for the risk of condensation on leavesNights often decide when infections start

Reference values from the agronomic literature for soilless greenhouse vegetables: always calibrate them to crop, variety, stage and local climate.

The sensors it uses

What each one measures, where it goes and the detail that makes the difference. “Core” is needed to start, “Extra” adds precision. See the full list for all agents

  • Lab analysis of water and drainageCoreIndividual elements (nitrate, potassium, calcium, magnesium, sodium, chloride…) that EC cannot tell apartSamples sent to a lab every few weeksIt isn't a sensor, but without it EC tells you how much salt there is, not which.
  • Air temperature and humidity (aspirated box)CoreTemperature and relative humidity; VPD and dew point are worked out from themAt canopy height, in a shielded, aspirated box, at least one per compartmentWithout a shield and fan, sunlight can throw the reading off by several degrees.
  • Weather stationCoreTemperature, humidity, solar radiation, wind and rain outside the greenhouseOn top of or next to the greenhouse, clear of obstaclesWind and rain also matter for safety: closing vents before a storm.
  • Weather forecastCoreExpected temperature, radiation and rain over the coming daysAn online data service: nothing to installThey let you act ahead instead of chasing, for example heating less before a sunny day.
  • Fixed or trolley-mounted camerasCoreImages of the plants to spot symptoms and count flowers and fruitsOn a trolley running on the heating pipes, or at fixed pointsImages taken from the same spot at the same time are the easiest to compare.
  • Sticky traps with a cameraCoreInsects caught each day: thrips, whiteflies, winged aphids and othersBetween rows, at the height of the plant topsThe daily count shows when a population starts to grow, before the damage.
  • Field photos and notes (smartphone)CoreObservations by the people working in the greenhouse: symptoms, fruit counts, crop stagesFrom the AgriAI app, during the greenhouse walkThe grower's eye is still the most valuable sensor: the app turns it into data.
  • Infrared leaf thermometerExtraLeaf temperature, without contactAimed at the canopy, from above or the sideLeaves warmer than the air point to plants transpiring little: possible water stress.
  • Leaf wetness sensorExtraWhether leaves are wet, and for how many hoursIn the canopy, where condensation forms firstHours of wetness are among the main indicators of botrytis and other fungal risk.
  • Satellite imagery (Sentinel-2)ExtraVegetation indices such as NDVI, mostly useful in open fieldsFree data from the EU Copernicus programme, updated every few days when skies are clearThey can't see inside a greenhouse; in open fields they show where the crop grows less.
  • Multispectral or thermal cameraExtraLight reflected beyond the visible range and canopy heat, to see stress before symptomsOn a trolley, at a fixed point or, in open fields, on a droneIt costs more: we add it only where an ordinary camera isn't enough.

The most common mistakes

  • Spraying on a calendar instead of on actual risk
  • Noticing the problem only when it is visible from afar
  • Relying on automatic diagnosis without a technician's confirmation
  • Not recording what was done: without history nothing is learned from season to season
Modern crop protection is continuous surveillance plus expert judgment. The agent provides the surveillance, day and night; the judgment stays with those who know the crop.

Frequently asked questions

25 questions, two answers each: the agronomist's and the developer's. When we cite an author, the link goes to the original work.

How does it recognise a disease from a photo?

The agronomist

It recognises symptoms: spots, moulds, yellowing, deformation. But the same symptom can have different causes, and sometimes only a lab test gives a certain diagnosis.

George N. Agrios, Plant Pathology, Elsevier, 5th ed., 2005

The developer

It uses convolutional neural networks, the family of models made famous by Yann LeCun, trained on thousands of images labelled by experts. The model returns a probability, not a verdict.

Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE, 1998
How reliable is the recognition?

The agronomist

Mohanty, Hughes and Salathé's 2016 study is a warning: models nearly perfect on lab images lost much accuracy on photos taken in different conditions. In a greenhouse light, background and angles always change.

Sharada P. Mohanty, David P. Hughes, Marcel Salathé, Using Deep Learning for Image-Based Plant Disease Detection, Frontiers in Plant Science, 2016

The developer

That is why we measure reliability on images from the pilot greenhouse, not public archives, and check that the probability the model states matches reality: the calibration studied by Guo and colleagues.

Chuan Guo, Geoff Pleiss, Yu Sun, Kilian Q. Weinberger, On Calibration of Modern Neural Networks, ICML, 2017
What is the disease triangle?

The agronomist

It is a classic principle of plant pathology: disease only develops when a susceptible plant, a pathogen and a favourable environment come together. Remove one side and disease doesn't start.

George N. Agrios, Plant Pathology, Elsevier, 5th ed., 2005

The developer

The agent reasons exactly this way: variety and stage for the plant, images and traps for the pathogen, climate for the environment. Risk comes from all three inputs together, not from one.

Why combine images with climate data?

The agronomist

Because climate tells you what is about to happen, images what has already happened. Hours of leaf wetness and high humidity come days before visible symptoms.

Hamlyn G. Jones, Plants and Microclimate, Cambridge University Press, 2013

The developer

It is a multimodal model: it merges different kinds of data, images and time series, into a single estimate. The theory is in Goodfellow, Bengio and Courville's textbook.

Ian Goodfellow, Yoshua Bengio, Aaron Courville, Deep Learning, MIT Press, 2016
Does it replace the agronomist?

The agronomist

No. Diagnosis and the choice of treatment stay with the agronomist. The agent saves them scouting time and points out where to look first.

The developer

Technically it is a triage tool: it ranks zones by likelihood of a problem. That is where machine learning adds most value and does least harm when wrong.

Cynthia Rudin, Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, Nature Machine Intelligence, 2019
What is integrated pest management?

The agronomist

The concept was born in 1959 with Stern, Smith, van den Bosch and Hagen: combine biological control, cultural practices and, only when needed, chemicals. In Europe it is now a legal requirement.

Vernon M. Stern, Ray F. Smith, Robert van den Bosch, Kenneth S. Hagen, The integrated control concept, Hilgardia, 1959

The developer

The agent makes IPM measurable: action thresholds configurable per zone, catch history and a log of every treatment, so you can see whether decisions worked.

What is an action threshold?

The agronomist

It is the pest density above which treatment pays, because expected damage exceeds the cost of treating. Below it you monitor and let beneficial insects work.

Vernon M. Stern, Ray F. Smith, Robert van den Bosch, Kenneth S. Hagen, The integrated control concept, Hilgardia, 1959

The developer

The agent doesn't wait for the threshold to be crossed: from the catch trend it estimates when it will be reached and warns you first. It is a time-series forecast with standard methods.

Rob J. Hyndman, George Athanasopoulos, Forecasting: Principles and Practice, OTexts, 3rd ed., 2021
How does it help biological control?

The agronomist

In pepper, predators such as Orius laevigatus and mites such as Amblyseius swirskii are used against thrips and whiteflies. IOBC studies show they also work against new species, if released at the right time.

IOBC-WPRS, Orius laevigatus and phytoseiid mites to control Thrips parvispinus in sweet pepper, IOBC-WPRS

The developer

The agent logs releases and observations and tracks the predator-to-prey ratio over time. When the ratio falls, it suggests a new release before pests take over.

How do camera traps work?

The agronomist

They are sticky colour traps, yellow or blue: blue mainly attracts thrips, yellow attracts whiteflies and winged aphids. Counting them by hand takes hours every week.

The developer

A camera takes a photo every day and an object-detection model from the YOLO family finds and counts the insects. New insects are told apart from those already counted by comparing with earlier photos.

Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi, You Only Look Once: Unified, Real-Time Object Detection, CVPR, 2016
Thrips and TSWV: why are they so feared?

The agronomist

Thrips transmit tomato spotted wilt virus (TSWV), which also hits pepper. There is no cure for viruses: you can only prevent them by controlling the vector.

Government of Alberta, Pests of greenhouse sweet bell peppers and their biological control, Alberta.ca

The developer

So the agent gives thrips priority: rising catches trigger an alert even below the normal threshold, and plants with suspect symptoms are flagged with photos for checking.

Botrytis and powdery mildew: how does it tell them apart?

The agronomist

Botrytis loves high humidity and wet leaves; pepper powdery mildew can develop at lower humidity and attacks from inside the leaf. Different conditions, different prevention.

George N. Agrios, Plant Pathology, Elsevier, 5th ed., 2005

The developer

The agent uses a separate risk model for each disease, with its own climate conditions. A single generic risk index would hide exactly the differences that matter.

How many images are needed to train the model?

The agronomist

You need images of every disease stage, across several varieties and seasons: an early symptom looks very different from an advanced one.

The developer

Usually thousands per class. With transfer learning you start from a model already trained on millions of generic images and specialise it, greatly reducing the data needed. Kamilaris and Prenafeta-Boldú's review documents this in agriculture.

Andreas Kamilaris, Francesc X. Prenafeta-Boldú, Deep learning in agriculture: A survey, Computers and Electronics in Agriculture, 2018
Who labels the images?

The agronomist

Expert technicians and agronomists. A wrong label teaches the model a mistake: the expert's quality matters as much as the number of images.

The developer

Doubtful images are labelled independently by two people and agreement is measured. Where they disagree, a third person decides: clean data is worth more than a bigger model.

D. Sculley et al. (Google), Hidden Technical Debt in Machine Learning Systems, NeurIPS, 2015
How does it avoid false alarms?

The agronomist

Too many false alarms are as dangerous as none: after a week of useless alerts, greenhouse staff stop reading them.

The developer

The alert threshold is tuned for the right balance between problems found and false alarms, measured on real greenhouse data. A well-calibrated model lets you choose that balance deliberately.

Chuan Guo, Geoff Pleiss, Yu Sun, Kilian Q. Weinberger, On Calibration of Modern Neural Networks, ICML, 2017
Does it recognise nutrient deficiencies too?

The agronomist

Deficiencies can look like diseases: yellowing can be iron, magnesium or a virus. It needs confirming with a solution or leaf analysis.

Cees Sonneveld, Wim Voogt, Plant Nutrition of Greenhouse Crops, Springer, 2009

The developer

Here the agents work together: if the Crop protection agent sees yellowing and the Fertigation agent logs high pH, the likeliest cause is a deficiency, not a disease.

Do you use multispectral cameras?

The agronomist

They see stress before visible symptoms, because they measure light the human eye can't see and leaf temperature. Valuable, but costly.

Hamlyn G. Jones, Plants and Microclimate, Cambridge University Press, 2013

The developer

We add them only where an ordinary camera isn't enough. First we measure how much they actually improve diagnosis in the pilot greenhouse, then decide whether the cost is worth it.

Does the agent apply treatments on its own?

The agronomist

No, never. Plant protection products are regulated: they require certification and compliance with labels, pre-harvest intervals and the national action plan.

The developer

It is a design choice: the decision to treat is always human. The agent prepares the information and logs what was done, in line with the human oversight required by the EU AI Act.

European Union, Artificial Intelligence Act (EU) 2024/1689, EUR-Lex
Does it help with the field register?

The agronomist

Yes: every treatment must be recorded with product, dose, date, crop and reason. Doing it properly is mandatory and protects the farm in inspections and with buyers.

The developer

The register is filled in from data already collected: who confirmed the treatment, where and when. It exports in the required formats, with nothing rewritten by hand.

Does it also measure how severe a disease is?

The agronomist

Severity is classically measured as the percentage of area affected. That decides whether an outbreak is under control or getting worse.

George N. Agrios, Plant Pathology, Elsevier, 5th ed., 2005

The developer

Affected area is measured with image segmentation, for example U-Net, an architecture born for medical imaging that separates healthy from diseased tissue pixel by pixel.

Olaf Ronneberger, Philipp Fischer, Thomas Brox, U-Net: Convolutional Networks for Biomedical Image Segmentation, MICCAI, 2015
Can I use photos from my phone?

The agronomist

Yes, and they are valuable: whoever walks the greenhouse notices things no fixed camera sees. A photo with a note beats a memory.

The developer

Phone photos vary a lot in light and framing, the weak spot shown by Mohanty's study. The app guides the shot and the model is also trained on imperfect photos.

Sharada P. Mohanty, David P. Hughes, Marcel Salathé, Using Deep Learning for Image-Based Plant Disease Detection, Frontiers in Plant Science, 2016
What if a never-seen disease appears?

The agronomist

It happens: new species such as Thrips parvispinus are spreading in Europe. In those cases what counts is how quickly an expert is called.

IOBC-WPRS, Orius laevigatus and phytoseiid mites to control Thrips parvispinus in sweet pepper, IOBC-WPRS

The developer

The model must be able to say "I don't know". Ensembles of voting models estimate uncertainty: if the models disagree, the case goes to a person instead of getting a wrong label.

Balaji Lakshminarayanan, Alexander Pritzel, Charles Blundell (DeepMind), Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles, NeurIPS, 2017
Are pesticides really reduced?

The agronomist

With monitoring you move from calendar spraying to treating only when needed, the core idea of integrated pest management since 1959.

Vernon M. Stern, Ray F. Smith, Robert van den Bosch, Kenneth S. Hagen, The integrated control concept, Hilgardia, 1959

The developer

We don't promise percentages: we count treatments and product quantities per cycle before and after, on the same crop, and publish the result measured at the Farm Tech.

Do the cameras film people too?

The agronomist

People work in the greenhouse, and their privacy deserves as much respect as plant health.

The developer

Cameras point at the plants; if a person appears, the face is blurred automatically before saving. Processing follows the GDPR and workers are informed.

European Union, General Data Protection Regulation (EU) 2016/679, EUR-Lex
Does it show why it suspects a disease?

The agronomist

An agronomist wants to see the symptom, not just read a name. Without the photo and the climate data, an alert can't be checked.

The developer

Every alert highlights on the photo the areas that weighed most in the decision, using attribution methods such as SHAP, and shows the climate conditions of the days before.

Scott M. Lundberg, Su-In Lee, A Unified Approach to Interpreting Model Predictions (SHAP), NeurIPS, 2017
Where does AI for plant health stand today?

The agronomist

Research is very advanced in the lab; in the field challenges remain with real conditions, local varieties and rare diseases. Hughes and Salathé's open PlantVillage archive kicked off much of this work.

Sharada P. Mohanty, David P. Hughes, Marcel Salathé, Using Deep Learning for Image-Based Plant Disease Detection, Frontiers in Plant Science, 2016

The developer

Kamilaris and Prenafeta-Boldú's review shows deep learning often beats traditional methods on agricultural images, but needs quality data. That is why we start with data collection, not the model.

Andreas Kamilaris, Francesc X. Prenafeta-Boldú, Deep learning in agriculture: A survey, Computers and Electronics in Agriculture, 2018

Become a pilot farm

Free digital check-up: we look at systems, consumption and problems and tell you whether and where AgriAI can help.

Apply as a pilot farm / I have land to bring back / I'm an agronomist: join the scientific team