Ideas on how to train a NN
The Quadcopter forest project: augmenting inav with a vision based obstacle avoidance. Since I'm surrounded by forests, it'll be trained in forests. The idea is to have a solid RTH that'll keep going home while avoiding obstacles.
Attempt #1 using a mobilenet classifier to find what is left, front and right of the route of least obstacles. Mobilenet handles thousands of categories, here we'll see how it handles just 3. The idea is borrowed from University of Zurich making a drone that follows mountain trails.
It's suitable for mountain trail following since there is a clear visual formed by footpath and cleared bush walls but from my early test it's not very good at differentiating obstacles from gaps. I think it's because the left and right cameras offer images that are too similar to front cameras. So I just tried with a single camera walking between trees in the widest gaps, clearly avoiding trees and then a second pass looking at trees and bush obstacles from many angles. Maybe the classifier will figure out obstacles and I'll then get move forward when clear, move to the right when front has an obstacle, gambling that in a forest there is always a gap wherever you look and if trees have fallen to make a wall then keep going right until it's clear, then nudge left back to the target direction.
In case the pedestrian camera method doesn't work I reinstalled unity to use it as a NN gym. For a newcomer to AI, Unity-ML is pretty awesome. Just seeing paddles learn to balance a ball on a 600$ desktop computer is amazing.
- In unity, toggle Control in Academy, put in the learning brain
- In Conda
activate ml-agentcd ml-agents-0.8.0mlagents-learn config/trainer_config.yaml --run-id=firstRun --train - In Unity press play and it'll train the NN, once it's finished, copy the trained model inside unity /TFModels and connect that in the learning brain of the agent then toggle off Control in Academy
Then I noticed that microsoft has a drone sim autopilot project here https://github.com/Microsoft/AirSim/blob/master/Unity/README.md
I'll dig into how the NN are trained and what they look like inside, if they can be accelerated by the K210 and I figure out a pipeline between Unity-ML and the K210 then we're in business, otherwise I'll revert back to a taking videos inside the CG forest of Fontainebleau made by the Paris team and training the mobilenet classifier above.
