Autonomous UAS Image Processing and Mission Software
Problem
RoboNation SUAS requires the aircraft to fly autonomously at 20-40 m altitude, detect and classify targets on the ground and act on them according to the mission. The difficulty: at that altitude targets occupy only 80-120 pixels in the frame, and detection has to run at near real time on an embedded computer with a tight power budget.
My responsibility
- Architecting the target detection/classification, autonomous navigation and mission planning software
- Leading the five-engineer software team
- Defining the data preparation pipeline and the training hyperparameters
- The Technical Design Report and the rules-compliant team website
Target detection from 20-40 m altitude
1 · Data preparation
To bring raw 4K drone frames down to YOLO's fixed input size I designed a pipeline that splits them into 1280×1280 tiles with 40% overlap. The overlap matters: when a target falls on a tile boundary it guarantees the target appears whole in at least one tile. I made the train/validation split at image level: if tiles from the same scene end up on both sides, the model learns through leakage and the validation metric lies.
2 · Class balancing
The raw data was class-imbalanced. Using offline augmentation I applied a per-class multiplier: 8× for the under-represented class, 3× for the over-represented one. That turned ~2,100 raw frames into a balanced ~15,000-image training set at a 1:1.2 ratio. I deliberately left colour augmentation to training rather than this step: applied in both layers it becomes double augmentation.
From dataset to embedded inference
3 · Training
Transfer learning on YOLOv11m. The model converged at epoch 33 and early stopping fired at 48, which means the data ceiling was reached: what is needed is more varied data, not more epochs. Having made that diagnosis I wrote a 13-item improvement plan: higher input resolution, collecting data across more altitudes and lighting conditions, and partial-occlusion augmentation.
4 · Embedded deployment
The model was positioned to run in real time on NVIDIA Jetson via TensorRT FP16. Training and export resolution have to match here; left different, the model behaves unpredictably in the field.
Avionics and compute architecture
Companion vision computer
Flight controller
Gimballed camera
Propulsion
A software architecture decision
We split image processing and flight control across separate processors: the flight controller does nothing but fly, the companion computer carries the vision workload, and the two talk over ROS 2. That separation keeps flight safety unaffected when the vision load spikes, which is one of the most critical design decisions in an autonomous system.
Flight testing and project documentation
Additional deliverables
Alongside the software work I produced the team's Technical Design Report and the team website that satisfies the competition rules' web requirements. The site was designed in Framer and then migrated to its own codebase; it holds 51 pages, 28 technical blog posts and five technical documents.
def karola(goruntu, boy=1280, ortusme=0.2):
"""4K kareyi sabit YOLO girdisine, ortusmeli karolarla indir.
Ortusme olmazsa karo sinirina denk gelen hedefler ikiye bolunur
ve iki yarim dusuk guvenle kaybolur."""
adim = int(boy * (1 - ortusme))
h, w = goruntu.shape[:2]
for y in range(0, max(h - boy, 0) + 1, adim):
for x in range(0, max(w - boy, 0) + 1, adim):
evet = goruntu[y:y + boy, x:x + boy]
yield (x, y), evet # ofset, geri projeksiyon icin sart