Here goes yet another post for the R9700 giveaway.
Background
I work as a data engineer in my daily job, but I’m also a machine learning academic researcher from Brazil, where agriculture is a core part of our economy and many small family farms still struggle with access to crop-health expertise. Since my undergraduate program, I’ve been focused on developing digital tools that reduce that gap. The challenges are significant: pests and diseases result in major losses every year, and misapplied pesticides can make the situation worse.
To address this, I’ve been building a full computer vision and machine learning system for disease detection using images of plant leaves. The system is designed to help farmers and technicians get quick guidance even if they’re hours away from a phytopathology specialist.
What the system does
There are plenty of plant disease classifiers out there, but most only predict a single label. In practice, diagnosing crop disease is a multi-step process. To better reflect the real workflow, I use a multi-task learning model that takes a single leaf image and performs several tasks at once:
- Detects whether a leaf is in the image
- Identifies the plant species
- Determines if the leaf is healthy or sick
- Predicts the disease agent (a fungus, a virus, a bacteria, etc.)
- Detects macro-symptoms (yellowing, wilting, etc.)
- Classifies the specific disease when present
The core idea here is NOT to replace the human expert, but to support them in their work and speed up the diagnosis process, and thus the idea is to kinda mimic the workflow of a phytopathologist. Such automated diagnosis is actually illegal here; you need an official diagnosis signed by a specialist to apply pesticides or any other intervention.
That’s also why explainability is such an important thing for this project, trying to provide some insights into the model like so:
The above image is a Grad-CAM visualization of feature maps for a given layer, trying to showcase what exactly in a picture made the model give a certain output.
To support real usage in the field, the model is part of a larger system:
- Leaf segmentation (SAM and SAM2)
- A FastAPI-based inference backend
- A web interface for specialists and farmers
- A local LLM layer and a vector database (pgvector) used for retrieval-augmented conversation, based on Brazilian agricultural reference materials
We have been testing it with specialists from the phytopathology lab in my city, a public crop-health lab that provides free support to small farmers. This has already been used in the field for cassava and grape crops, reducing the need for long travel and allowing faster triage and data collection.
Below is a screenshot of a simple front-end I’ve built on top of the above-mentioned inference API for users to quickly test out:
(I’m not really confident in publicly sharing the link for it at the moment, sorry for that)
Current hardware setup
I like doing most of my work locally whenever possible, and thus I’ve been using my own hardware for training and experimentation.
- Main workstation: Ryzen 9950X, 256 GB RAM, 2x RTX 3090 - Model training, segmentation, LLM/RAG. PC guts post. Neoflex post.
- Secondary workstation: 5950X, 128 GB RAM, RTX 3060 - CNN inference and offloading experiments. PC guts post.
My original setup was the AM4 one, but as I tried to create more efficient training pipelines by doing the data loading process ahead of time and keeping many things in flight in memory, as well as many other background tasks, I ended up upgrading to the AM5 setup and leaving my previous one as a headless inference station to alleviate my RAM and VRAM usage.
Such local setup makes it easy for me to quickly iterate and try out stuff with notebooks without having to wait for cloud resources or see a hefty bill in my credit card.
With that said, I do use OpenAI’s APIs every now and then since it’s provides way more quality than any other local model I’d be able to run locally, and is cheap enough to not be a burden. It was specially useful when I was doing RAG for augmenting my training datasets. My university also provides a cluster with A100s and 3090s, but having to deal with slurm is kinda annoying and two 3090s often end up beating an A100, specially when I can just use tools of my liking without having to submit jobs.
Research direction
I’ll be starting my PhD next month where I will expand this system into several new areas:
- Replacing CNNs with transformer-based vision models
- Testing smaller and more efficient segmentation networks (SAM3 and others)
- Ablation studies to measure the practical benefits of multi-task learning in those vision models
- Explainable AI work so specialists can understand model reasoning
- Quantization and distillation for lightweight deployments
- Running inference at the edge on Jetson boards, Raspberry Pis, and eventually drone-captured image streams
Why the R9700 would help
Having 32 GB of VRAM would allow me to:
- Train larger vision transformer (ViT) models and experiment with higher-resolution segmentation
- Run heavier LLMs in parallel with the multi-task learning (MTL) pipeline as part of the advisory layer
- Move a 3090 to the secondary workstation, giving more headroom for experimentation with my main desktop without interrupting my inference API used for the web interface
- Actively contribute to ROCm support and performance when it comes to LLMs and Pytorch usage
Academic output and resources
Here are some of the “publicly” (journals are annoying, DM me if you want to read my copies of those) available outcomes from this project so far:
- Paper where I was a co-author during my undergrad, which basically served as the starting point for this project.
- My undergrad thesis, which is a more detailed look at the project and the results.
- A paper where I was the first author, going more in depth into the multi-task learning approach and the results, as well as a quick glance into explainability for vision models.
- My master’s thesis, which is a more detailed look at the project and the results, actually explaining the segmentation strategies I’ve tried, the RAG pipeline I used to augment my training dataset, and the overall system architecture.
- A kind of unrelated paper where I was a co-author in the automotive domain, where I helped with lots of work in the quantization and distillation steps to speed up model inference in SBCs, which is something that I plan to work on top of my own models during my PhD.
I also plan to open more of the code as I formalize deployment pipelines during the PhD. Once I clean up the current webpage which provides the inference API, I’ll post it here as well.
With all of that said, feel free to give any feedback into my project or just ask any questions ![]()

