Projects
A personal project: teach a computer to fly a drone from scratch. First hold a hover while random gusts push it around, then fly to points tens of metres away, then see and avoid obstacles through its own camera, and eventually fly my real Mavic Mini the same way. Everything starts in a simulator; real flights get rehearsed in the simulator first, and I am always at the sticks for anything real.
Where it stands (2026- 11- 13): hovering is solved (0.33 m average error in gusts, beating a hand-written controller). Goal reaching at 10 to 50 m is adopted: the current policy lands within about 0.3 m, on par with a hand-written controller. The stage in progress is obstacle avoidance through the camera; the first attempt dodged obstacles but would not commit to the goal, and the honest post-mortem is in the journal below.
This page is a build journal: what worked, what did not, what I stopped and why. Newest first. Every video here is from the simulator.
Footage
Training montage: the failed scoring, the fix, the training curve, and what comes next. Simulator footage only.
First attempt at holding a position (hover_v1). It logged a 3 cm error, but that was a measuring mistake: the target was set at landing-pad height, so the drone was mostly sitting near the ground. Not a real hover, and my first lesson in checking the scoreboard, not just the score.
Side by side under the same random wind gusts: a failed hover policy and a passing one. The pass mark was set in advance: under 1 m average error, never drifting out, and better than both doing nothing and a hand-written controller.
The long-range stage: fly to points 10 to 50 m away. This is the adopted policy (LR-2b): it settles 0.30 to 0.35 m from the goal, about as close as a hand-written controller while handling more cases. Getting here took three failed retrains, one of which I had originally graded on the wrong test (see the journal).
The same obstacle course, untrained on the left and trained on the right. The trained policy (obstacle_v2b) passes the short-leg course; making it avoid while also committing to far goals is the stage in progress now.
For contrast: a scripted (not learned) flight, takeoff, orbit and landing, so you can see the simulator and the camera the learned policies work with.
Build journal
Didn't work · 2026-11-13
Camera avoidance first attempt: it dodges, but stops short of the goal
Trained the camera pipeline for obstacle avoidance over 72k steps, then ran paired test flights: goals placed behind obstacles, 32 episodes, same starts for every policy. Dodging genuinely worked: collisions 3 versus 30 for the blind policy, and blocking the camera input made it crash more (7 without the camera stream), so it really flies from the camera. But it arrived 0 times out of 32: it advances about 2 m then parks about 7 m short. Same camping signature as the long-range failures, and avoidance makes camping look attractive because detouring risks a penalty while hovering barely bleeds. The known fix: the long-range policy only stopped camping after nearly double the training. Retrying as v2: warm start from the adopted long-range pilot and train to at least twice the length. Simulator, honest numbers.
Fixed · 2026-11-13
M4 first AirSim flights: partial, rig fixed
Real-physics go-to-object run landed 1 arrival of 4 (red mailbox 1.00 m, but re-detect never engaged), 1 timeout 3.47 m, and 2 deaths: the runner's kinematics fallback called get_kinematics, an API this simulator fork does not have. Fixed the fallback to the proven estimated-kinematics call and redeployed sha-matched; one clean re-run queued for the next sim window.
Worked · 2026-11-13
Blind-policy crash footage captured
The RL policy that passed the obstacle course has no camera, so we put a 3 m cube 6 m ahead on its straight point-A-to-B line and filmed what it does: nothing sees it, it flies straight in and holds against the face. Captured in AirSim with the raw policy, collision logged at step 16 of 28 after three geometry fixes (a flight corridor that was actually 2.2 m below ground level was costing us three failed attempts). Two of three takes collided; the clip ends on 'check what your model can actually see'. For review only, nothing posted.
Worked · 2026-11-12
Obstacle training attempt 2 running clean
Past 60,000 of 72,000 steps by 22:00 at about 235 steps a minute, no stalls. Its test runs automatically when training ends.
Didn't work · 2026-11-12
Obstacle training stalled at step 1,002
Attempt 1 froze 9 minutes in when the simulator stopped answering; the watchdog restarted it automatically. Its log had been overwritten, so the cause is unknown. Each attempt now keeps its own log, and the rule is two strikes: a second stall stops the run for diagnosis instead of looping.
Fixed · 2026-11-12
The poles were lying on their sides
One probe, one variable at a time, found it in minutes: the 3D file format treats a different axis as 'up', so every pole we spawned lay flat across the flight path like a wall. Rebuilt upright, then fixed four smaller bugs the first one had hidden (contact margin too small, test controller flying a diagonal, crashes masked by a guard, short crashed episodes treated as fatal). Smoke test passed four times in a row.
Didn't work · 2026-11-12
Obstacle training would not start: the drone stopped in open air
Every smoke test of obstacle avoidance failed the same way: the drone stopped about 0.66 m short of a pole no matter how far sideways the pole was. Five rounds of guessing (scale, the collider, spawning anything at all) were wrong.
Worked · 2026-11-12
Long-range pilot adopted
LR-2b is now the long-range pilot (10-50 m legs). The short-range obstacle policy v2b stays the shipping short-range pilot.
Fixed · 2026-11-12
The 'failed' long-range run had been graded on the wrong test
Run LR-2b had been scored on where it crossed the finish gate, while the simple controller was scored on where it settled. Re-tested on the same hold-still test as the controller, it settles 0.30-0.35 m from the target at every distance from 10 to 50 m, against the controller's 0.34-0.56 m: 16 of 16 arrivals, zero hits, 4 of 4 routes. Overnight FAIL reversed to PASS. Lesson: compare on the same test or the comparison means nothing.
Didn't work · 2026-11-12
The handoff idea failed too
Test: the learned policy flies the long leg, then the simple controller takes over for the last 1.5 m. It arrived 13 of 16 times and finished 0.50-0.60 m off, 1.1-1.5x worse than the simple controller alone, and no faster. Handing over at 3 m was closer (0.40-0.44 m) but still not better. Two pilots in a row did worse than either one.
Didn't work · 2026-11-12
The drone parks at the gate: four reward designs later, we stopped tuning the reward
Four reward variants in one night all taught the drone to fly 10 to 50 m legs and arrive, best run 16 of 16 with zero collisions. None beat the hand-written controller at dead-center precision, about 0.6 m vs 0.3 to 0.5 m. The last variant paid the drone for holding still near the target after arrival; it settled closer, best bin 0.48 m, but started missing arrivals, 11 of 16 then 8 of 16, so each fix opened a new failure. We stopped reward tuning on a pre-agreed rule and will test a handoff next: the learned policy flies the long legs and the simple controller takes the last metre, same seeds, same gusts, same bar, decided before training started.
Didn't work · 2026-11-12
Second precision retry still misses the hand controller
LR-2b added a bonus for how deep the drone crosses the 0.75 m arrival gate (20 times depth), resumed from the 72k checkpoint with its replay relabeled, trained to 114k. Arrivals stayed 16/16 and the mean final error improved from 0.683 to 0.604 m, and the 10 m bin reached parity with the scripted controller (1.02x). But bins 20/35/50 m still land 1.21-1.51x the controller error, so the fixed bar (must be within 1.1x) still says fail. Honest read: a bonus paid at the arrival instant cannot make the drone settle on the point, because the episode ends the instant it crosses the gate. The next run changes the rule: after arrival the drone holds a fixed 4 s window and is paid steeply for staying near the centre, scored on where it settles, and the scripted baseline gets the same hold.
Didn't work · 2026-11-11
Long-range reward v6 in AirSim: arrival solved, precision not
Third attempt at long 10-50 m legs, this time with the research-backed reward (fly-closer progress + time penalty, no survival bonus). It fixed the failure that killed the last two tries: the old ones parked short of the goal and idled (1/16 then 0/16 arrivals); this policy arrived 16 of 16 single legs and all 4 multi-leg chains with zero collisions, and every distance bin averaged under 0.7 m. The short-range suite also tied the best policy we had (0.566 m, 4 of 4). But our pass bar was fixed before training and it also requires matching a hand-written controller's final precision within 1.1x, and the policy lands at 1.26-1.73x of it (about 0.68 m vs 0.47 m). Reason: the reward pays the moment the drone crosses the 0.75 m arrival gate and nothing below it, so it learned to stop just inside the gate while the hand-written controller keeps pushing in. Verdict stands as FAIL and the old short-range policy stays shipping. Next move is Ryan's call: add a small bonus tier inside the gate and retrain warm from this policy, ship it on arrivals anyway, or keep the old one.
Worked · 2026-11-11
Ground-truth worlds for missions
Built three seeded scene presets for the sim - a forest with real tree clusters, a mixed parking lot, an open field - each with a ground-truth map that says exactly what is where, so mission conditions like 'an area with lots of trees' or 'the blue vehicle at grid cell 2,2' can be checked against the layout file itself instead of trusting the detector. Caught and fixed six vehicle color mislabels by re-checking the model textures by eye before anything shipped.
Didn't work · 2026-11-11
First follow-the-target policy: the hand-written baseline still wins
Next capability on the list: the drone follows a moving person or bicycle, keeping it centered in the camera while getting around obstacles. Built a sim course where a cube blocks the straight line to the target, plus a hand-written tracking controller as the yardstick, then trained a reinforcement-learning pilot for 30k steps with the same detector noise and link latency the real aircraft will have. Honest result: the learned pilot clearly works (much better than doing nothing, zero collisions, target kept in frame) but came in about 1.5% WORSE at camera centering than the hand-written controller, so it did not earn the pass mark. One short run of a hard problem: 30k steps is small and the reward was only just re-shaped to remove a lingering trap. Next iteration gets a longer schedule, easier-target curriculum, and the banked baseline numbers as the fixed bar to beat.
Worked · 2026-11-11
Long-range reward v6 proven on the physics twin
After two long-range training failures we changed the incentive: the new reward pays for closing distance to the waypoint and charges a small cost per second, so hovering near the goal no longer pays. A pure-math checker confirms arriving beats camping at any distance (worst ratio 1.22, 1.30 after tuning). Trained 3 seeds in about 200 s each on the fitted physics twin: all 3 seeds now reach every 10 to 50 m test goal inside the 0.75 m gate, all four three-leg chains, short legs under 0.7 m. Honest limit: the old reward also passed 2 of 3 twin seeds, so this proves the new incentive teaches long legs without a camping trap; the real simulator run decides next.
Didn't work · 2026-11-11
Long-range retrain fails with the opposite signature
The v3b retrain fixed the parking bug (the drone no longer hovers just outside the 0.75 m gate) but now every long leg ends about 3 m short with the time budget spent: the new approach reward made the policy fly slower and longer around the goal. Arrivals 0 of 16 legs; the scripted hand-written controller still arrived 16 of 16. Two attempts at long-range flight failed in two different ways, so the series stops here for a design review instead of a third guess. The short-range shipping policy is unchanged.
Didn't work · 2026-11-11
Long-range waypoint training failed: the drone parks a metre short of the goal
Trained the quad policy overnight to fly waypoint legs up to 50 m instead of the 7 m it knew. It learned the direction but stopped about a metre outside the arrival gate and hovered until the time budget ran out: 1 arrival out of 16 test legs, versus the hand-written controller arriving every time at half a metre. The setup audit found why: the reward that pulls the drone toward the goal flattens out past 10 m, so the last metre before the gate carried almost no signal, and the hardest difficulty stage only got a third of the training time. Fix designed: a stronger final-approach reward inside the last 2 m and a curriculum that spends most of training on the far legs. The short-range policy stays the shipping version; nothing published regressed.
Worked · 2026-11-11
The pilot now runs on the phone and matches
Two checks passed the same night. The hover policy exported to ONNX/LiteRT agrees with the PyTorch trainer to 4.5e-7 over 20,000 recorded observations, and a loopback flight in the simulator driven by the phone-side pilot code hovered at 0.334 m mean error against 0.332 m from the Python path, with zero safety-gate violations and 37 green unit tests behind the safety layer. The earlier adapter gap (0.476 m against the 0.335 m bar) turned out to be test-harness timing, one step stale, and closed at 0.342 m inside the bar once the harness waited for a fresh answer before each step.
Worked · 2026-11-10
Offline 3D map from drone video works in sim
A waypoint-route flight in the sim (221 FPV frames) plus ground-truth poses went through COLMAP on the laptop CPU. Injecting the telemetry poses (point_triangulator) registered 221 of 221 frames and put map points a median 0.36 m from sim ground-truth depth. Blind SfM alone registered only 5 of 221 frames on the textureless Blocks scene, so the mapping pipeline must take poses from telemetry and not rely on feature matching. Whole pipeline runs in about 6 minutes on 8 CPU cores. Sim footage only; numbers and map files are banked privately.
Didn't work · 2026-11-10
S2 loopback re-run: real gap found, 0.476 m vs the 0.335 m bar
Ran the staged S2 adapter loopback on a free sim at 18:03 CT with the home-offset fix in place. The fix worked: adapter loopback error dropped from 1.505 m to 0.476 m, zero landings, all 8 episodes ran full length. But 0.476 is still 42% above the banked 0.335 m hover result, outside the pre-stated 10% band, so the verdict is FAIL and we report it as-is rather than retuning to make it fit. The gap is systematic (same in both test rounds and every episode), which points at command-path timing or the angular-velocity observation channel, not episode noise. Numbers, logs and a candidate one-variable experiment are banked for the adapter lane to root-cause.
Worked · 2026-11-10
Obstacle course with turning passed
The obstacle course now includes turning (yaw), which the real drone needs, and the trained policy passed: 0.57 m from the target on average against 0.64 m for the hand-written controller, 4 of 4 arrivals, zero collisions. This run restarted from weights saved when an earlier attempt crashed at 25,700 steps and checkpointed every 3,000 steps, so nothing was lost. That finishes the scheduled flight training runs. Next up is an untrained-vs-trained side by side video.
Didn't work · 2026-11-10
Two bad test setups, and a training run lost to a pause
Two of my first three tests of the waypoint pilot were wrong: the test script quietly wrapped the pilot in the wrong control mode, so the numbers meant nothing. I caught it from the log, fixed it and kept the bad result on file. Then the nightly laptop pause froze a training run at 25,700 steps and it crashed on resume, losing about 80 minutes. The fix: training now saves every 10 minutes and pauses by saving instead of freezing.
Worked · 2026-11-10
A fast physics copy of the simulator works
A small physics copy of the simulator, fitted to real simulator recordings and my first real flight, trained a waypoint pilot in 88 seconds. In the real simulator it matched the hand-written controller over 16 test flights (0.56 m vs 0.53 m, 16 of 16 arrivals). It did not beat the controller, so the waypoint step is closed as a tie, but it proves the shortcut transfers.
Worked · 2026-11-10
Obstacle course passed on the first try
The drone flew the obstacle course in the simulator with zero collisions: 0.57 m from the target on average, against 0.62 m for the simple hand-written controller, 4 of 4 arrivals. It took 3.6 hours to train. The margin over the controller is small and the test is only 4 flights, so I count the zero collisions as the real result, since the hand-written controller cannot avoid anything.
Worked · 2026-11-10
Obstacle course: learned policy beats the scripted controller
Stage 3 of the training ladder passed on its first try. A SAC policy trained 40000 steps across six parallel simulators (about 5.5 hours) now flies the obstacle course: it reaches the waypoint in 4 of 4 test runs, finishes on average 0.57 m from the target versus 0.62 m for the scripted P controller and 8.85 m for doing nothing, with zero collisions in all episodes. Training curve: average reward climbed from 24 to a plateau near 64, episode length fell from about 96 to about 15 steps, so it learned to arrive fast, not to hover near the goal. Honest caveat: the arrival margin over the controller is about 5 cm on a 4 episode test, so think parity plus clean collision behavior, not a landslide. Deviation recorded: yaw was not part of the action space for this run; a 4th yaw action is planned for obstacle_v2.
Fixed · 2026-11-09
Stopped the resumed waypoint run at its 48,000 step target
The resumed waypoint_v2 run hit its 48,000 step target at 20:53 with checkpoints saving normally again. But a bug in the resume script's stop condition made it keep training past target (the stop point was computed as 48,000 steps ADDED to the resume point instead of an absolute limit). We stopped it at 50,502 and banked the 48,000 step checkpoint as the policy for evaluation. Mid-run numbers only: mean reward held around 64 since the resume, episodes averaging 15 to 18 steps. No verdict yet, the clean eval against baselines comes next.
Didn't work · 2026-11-09
waypoint_v1 did not beat the simple controller
Verdict WAYPOINT_NOT_LEARNED: mean final distance 0.62 m against 0.60 m for the hand-written controller. v2 starts from that controller and learns corrections on top of it. One run was also killed at 30k steps by a 4-hour timeout in my wrapper and resumed from its checkpoint.
Decided · 2026-11-09
Training moves off the laptop: a physics copy of the simulator
The Windows VM cannot run the simulator, so a small NumPy physics copy is being built (about 7,000 steps a second on one core, against 2 for the real simulator). It will be fitted to real simulator captures and to real Mini flights. The real simulator stays the judge: a policy only counts when it beats the baselines there.
Worked · 2026-11-09
Phone app talks to the real Mavic Mini
The Drone Relay phone app now registers with the drone maker's SDK, connects to the Mavic Mini through its remote, shows live video with detection boxes, records telemetry and has a STOP button that hands control back to the remote. First hand-flown flight recorded (about 5 minutes); it gave rough real response numbers: roughly 0.3 to 0.5 s lag between stick and velocity.
Decided · 2026-11-08
Pausing the explanations, starting waypoint training
Two plan changes tonight. I stopped the CCW-Net explanation work partway: 65,000 recorded pilot decisions and the first model fits are kept, but the laptop goes to training for now and the explanations are more useful run against the waypoint policy at the end. Second stage kicked off: the network now learns to fly between waypoints instead of only holding position. About 3,200 steps in, roughly a quarter of the way (hover took about 12,000). Weight spread is tight so far (std 0.104); mid-run numbers, nothing is claimed as passing until the eval and the explanation check run.
Worked · 2026-11-08
Six simulators at once, zero slowdown
We wanted to run several drone simulators side by side on the laptop so data collection and training go faster. Measured it end to end: one simulator takes 0.547 s of wall clock per 0.5 s sim step, and that stays the same through 6 simulators at once, so 6 sims means roughly 6x the work for free. The real limit turned out to be graphics memory, not speed: 6 copies fit (4.4 of 6 GB), 8 copies overflow (5.7 GB). Bonus find: the GPU busy gauge is pinned near 99% even with one simulator at low settings, so it tells you nothing; running windows small and off-screen is what actually frees memory (about 0.2 GB per copy saved). Every automation script now parks only its own simulator instead of killing any one it finds, proven by running an eval beside a foreign simulator without a scratch. Training plan can now use 6 parallel environments.
Worked · 2026-11-08
Media kit: the give-up policy and the fix, side by side
Built the footage for the website post. Same gusts, same camera, two reward functions: the old scoring policy drifts out and dies past 8 m in about 13 seconds, the fixed one holds its spot with mean error 0.34 m (baseline doing nothing: 1.87 m) and rides out a doubled gust, worst deviation 2.4 m. Also made a watch-it-think clip: the policy's live state, hidden activations and thrust commands next to the drone's own camera, in the spirit of my friend's CCW-Net paper. Plus training-curve and network time-lapses and the three honest failure clips. Nothing published yet, Ryan picks and ships.
Worked · 2026-11-08
Real Mavic Mini reached through its remote
A Raspberry Pi pretends to be the phone, so the remote opens its real link. The aircraft streams live data to the lab and the camera tilt answered a command (down 90 degrees, then level). Only the camera was commanded; no flight commands yet. Earlier dead ends: the PC cannot talk to the aircraft through the remote directly, and the Steam Deck crashed when we tried to make it the phone.
Worked · 2026-11-08
Six recorders at once for the paper step
Recording the pilot's decisions for the paper was running on one simulator, about 1.5 decisions a second. Three simulators ran beside it with no slowdown, so I added three more recorders on their own simulators, then two more: six at once, all with fresh random starts. Stopping and restarting the first one would have thrown away two hours of data, because it only saves when it finishes, so it kept running. The paper step now gets about 65,000 decisions instead of 21,000, and finishes at the same time. Six simulators use about 4.5 of the laptop graphics card's 6 GB; memory and the processor are the next limits.
Worked · 2026-11-08
Three simulators at once on one laptop
I tested whether the laptop could run more than one drone simulator at the same time. It can: three ran together, each flying at full real-time speed, while a fourth job kept recording data at its normal pace. Graphics memory went from 1.4 to about 2.6 of 6 GB. The catch I found: my own job scripts stop a simulator by name, so one finishing would shut down the others. They need to track which simulator is theirs before I use this for training.
Fixed · 2026-11-08
Checking the explanation step against the paper
Before training the explanation model, I compared the setup with my friend's paper line by line. Two things would have made it a weaker test: the concepts didn't say which direction the drone had drifted (the main thing that drives its corrections), and they weren't grouped into either-or pairs like the paper's aircraft example (lead or lag, climb or dive), which the paper's main measurement needs. The concepts are now drifted north or south, east or west, above or below, climbing or sinking, measured the paper's way over ten training seeds.
Decided · 2026-11-08
Next pieces planned
A phone scan of a real outdoor space can now be loaded straight into the simulator to rehearse flights there first. Camera-based obstacle avoidance was researched (distance estimates from one camera feeding a small policy), and the real-drone link was designed with a stop button and a link-loss failsafe.
Worked · 2026-11-08
Version 7 stays
Fixed scoring plus a different learning method (SAC, better when every simulator step is slow). Within an hour rounds lasted about 98 of 100 steps instead of 25, and the score was climbing. Pass or fail is decided by the test against the baselines when training ends.
Fixed · 2026-11-08
The real problem: the scoring rewarded giving up
Every step cost points for being away from the spot, and drifting far enough simply ended the round with no extra penalty. Giving up quickly lost fewer points than trying and being slightly off for the whole round, so every version learned to leave. Now staying near the spot earns points every step and drifting out costs a fixed penalty, so staying always wins. Lesson: before training, check whether the score can be gamed by quitting.
Stopped · 2026-11-08
Stopped version 6 early
Version 6 changed how the drone explored. A review of the setup showed it could not work either, so it was stopped at about 3,500 steps instead of wasting three more hours.
Didn't work · 2026-11-08
Versions 4 and 5 failed
Version 4 averaged 1.56 m from its spot (the bar is under 1 m). Version 5 did worse than doing nothing. A separate automation bug also overwrote version 4's training log, which cost some evidence.
Fixed · 2026-11-08
The drift was ours
A careful probe showed the simulator holds a hover to about 3 cm on its own. The 0.7 m/s drift I had blamed on the simulator came from my own wind-gust injector. Gusts were toned down to a realistic level.
Didn't work · 2026-11-08
Versions 2 and 3 failed
With a proper target, both policies did no better than doing nothing. I first blamed a drift in the simulator's controller.
Didn't work · 2026-11-08
The 3 cm hover that wasn't
The first trained policy reported a 3 cm average error. Watching the video showed why: the hover target had been set at landing-pad height, so the drone was mostly sitting near the ground. The number was real; the test was wrong. The environment now settles the drone at least 2 m up before setting the target.
Fixed · 2026-11-08
First flight, after one stubborn wall
The simulator crashed every time it was launched remotely. It only runs inside a real desktop session, so every sim job now runs as a scheduled task. After that: takeoff, orbit, landing, filmed from the drone's own camera.
Stopped · 2026-11-08
Dropped the warm-up simulator
The plan was to prove hover learning in a small physics simulator first. It ships no Windows builds, so I stopped and went straight to the real simulator instead of losing a day to it.
Decided · 2026-11-07
Picked the simulator
The original Microsoft AirSim and its forks are dormant or archived, so I chose Project AirSim (MIT licensed, actively maintained by the original team) on my laptop's RTX 4050.