Table of Contents
Aerial Manipulation
This project aims to develop techniques for fast and reliable pick and place applications using an aerial vehicle equipped with a multi-DOF arm. The applications for such a task would be to place a sensor on difficult to access spaces such as a vertical wall of a high-rise building or power-line inspection, industrial package sorting, environmental sampling, fruit picking
We developed a suite of control techniques, high-level framework, and hardware modifications to achieve a reliable aerial manipulator. The aerial manipulator is able to achieve an overall pick accuracy of 90 % in a test of 100 trials. Of the 10% failures
Control Techniques:
We applied a predictive control technique known as MPC to design high-level trajectories that can be tracked using a nonlinear controller. Further, we designed an optimal trajectory at the end of the picking phase to maximize the picking of an object.
State Machine Framework
To augment the control techniques and handle different hardware and software failures, we designed a finite state machine framework that combines different controller components to perform
State machine framework 
State machine Graphical Interface
Results
The framework was tested on a DJI Matrice to pick objects in both indoor and outdoor settings.
Project Url: https://github.com/jhu-asco/aerial_autonomy
Robust Obstacle Avoidance using Adaptive Model Predictive Control for a Quadrotor
This work focuses on developing control techniques to fly a quadrotor aggressively around obstacles. Although, this can be achieved trivially by staying away from obstacles, the exact distance to stay away from is usually not known. If we just stay very far from obstacles, the behavior becomes very conservative and we may end up not reaching the goal. If we do not allow for any safety buffer, a small deviation from the nominal trajectory can cause collisions.
We propose a control technique that combines online system identification with MPC to navigate obstacles safely. The system identification part finds the mean and covariance of model parameters from system trajectories. The distribution in parameter space is then propagated into state space using Unscented Kalman Filter (UKF). The trajectories are then optimized to avoid obstacles by a distance based on the buffer around the nominal trajectory and the obstacle size. This allows the quadrotor to go through tight obstacles at high speeds.
navigating obstacles
NonLinear Model Predictive Control (NMPC) techniques are used to predict safe trajectories around obstacles. These trajectories explicitly take into account the uncertainty in the state space while planning for trajectories around obstacles. NMPC also achieves a user-specified desired goal state and minimizes control effort during planning.
Closed loop obstacle avoidance has been proposed by combining nominal trajectory optimization with LQR feedback law. The video shows a simulation of an RC car going around a track while avoiding obstacles. The obstacles on the track are detected on the fly and the optimization adjusts the trajectories to ensure the propagated ellipsoids are not intersecting with the obstacles. We use the LQR feedback law to propagate the ellipsoids to ensure the size of the ellipsoids do not grow with time.
Neural Network Modeling for Steering Control of an Autonomous Vehicle
This work proposes modeling the steering dynamics of an autonomous vehicle using a Recurrent neural network. The developed dynamic model is utilized with a MPC controller to track a desired steering reference trajectory accurately.
The control architecture used to control the steering on the autonomous vehicle is shown below:
The predictive performance of the RNN model as compared to first principles models is shown below
Overall, we have shown that using
Robust Obstacle Avoidance using Tube NMPC
This project extends the previous work on robust obstacle avoidance using closed loop propagation of the uncertainty. Instead of parametric uncertainty in the dynamics, we consider noise in the state space of the system. This noise is propagated using an optimization method that finds a region around a nominal trajectory that the robot is guaranteed to stay within even under bounded disturbances applied on the robot.
The propagation algorithm is combined with nominal trajectory optimization to find trajectories that avoid obstacles using a safety buffer that is based on the size of the disturbance region around the nominal trajectory. We applied the algorithm to both a simple unicycle model and a quadrotor model to show the scalability of the algorithm to higher dimensions.
The picture below shows robust obstacle avoidance trajectories for a unicycle model and a quadrotor model that reach the desired goal while avoiding obstacles using a buffer based on the disturbance regions around the nominal trajectory.
Robust obstacle avoidance trajectory for a unicycle model (The red lines show sample trajectories under model disturbances). 
Robust obstacle avoidance trajectory for a quadrotor model
Autonomous Control of Unmanned Ground Vehicles (UGV) on unstructured 3D terrain
This work considers local optimal control and planning of UGV using high fidelity physics simulation. When navigating an offroad terrain, it is often hard to determine the navigability of any patch. In this work, we propose a
Unlike traditional planning schemes, we do not assume we have any prior information about the terrain. For example, if there is a brick wall ahead of us, the physics based samples will collide with the brick wall and automatically incur a high cost. My minimizing the trajectory cost, we automatically find trajectories that minimize control effort, avoid obstacles and reach goal position just based on known terrain properties and a high quality 3D map of the surface.
In this work, we use a
