Master's Projects Behavior Cloning for Self Driving Car Behavior Cloning can be seen in Tesla AutoPilot, where the algorithm learns from human driving. Here data of driving a car in simulator is used to train a CNN (modified LeNet Architecture), that could drive the car autonomously. (Input front camera, output steering angles) Git Repo | Video Learning from Demonstration (LfD) for Baxter using Vicon Teleoperating a humanoid has wide variety of applications both like nursing, space exploration, etc. Vicon Motion Capture system in animation design and movie making. It is used to track the motion of human and map it onto Baxter robot to perform take like grasping. Virtual headset is also used to get live camera feed from Baxter head camera. Learning from Demonstration for manipulation tasks can be widely useful for person robots. We collected several demonstrations using Vicon and VR Headset. Hidden Markov Model is trained with reduced states (with K-Means clustering). These features are extracted in Configuration space. Bi-RRT* is also used in execution phase to tackle with dynamic obstacles in workspace. Git Repo Teleop Interface | Git Repo HMM Training | Video Sensor Fusion for Autonomous Cars Extended Kalman Filter and Unscented Kalman Filter based fusion of sensor data (Lidar and Radar) has been implemented on Udacity Simulator for Autonomous Cars. EKF linearizes the non-linear (radar observations to state space transfer) function by first order approximation. But since this approximation is not a good idea, UKF overcomes the problem by generating sigma points and transfering them and fitting gaussian. This procedure is repeated and the non linear function is used as it is without approximations. Git Repo EKF | Git Repo UKF Particle Filter based Localization for Autonomous Cars Particle Filter can fit multi model estimate instead of Gaussian. Since if there are two similar rooms, gaussian might have to choose between one of the them but particle filter assumes it is in both (if there is no prior as of then). This is more robust, but computationally intensive with more particles. Git Repo Inverse Dynamics and Passivity based Controller implementation on Kuka Youbot Arm Robotic Arms are very important for Industrial manufacturing sector. Precision of trajectory tracking and fast convergence defines how good the control algorithm is performing. In this project comparison of Inverse Dynamics controller (which uses feedback linearization) and Passivity Control (energy based controller) for trajectory tracking of 5DOF kuka Youbot arm. Initially the performance of algorithms are tested in Matlab Simulation using Sim Mechanics Toolbox, then Implemented on Hardware using ROS. Git Repo Policy Gradients (REINFORCE) on Cart-Pole Tradition control techniques need system dynamics to design a control. But Artificial Intelligence based Reinforcement Learning has model free control learning technique. It learns by interacting with the environment and getting reward from environment. A Policy Gradient based Reinforcement Learning using Actor Critic technique is simulated on a Cart Pole Environment in OpenAI gym. Git Repo | Video CNN based Traffic sign Classifier Traffic sign classification is one of the important perception module in Self Driving Car Domain. Basic LeNet, traffic sign classifier is built using Tensorflow, providing a validation accuracy of 95.4% and test accuracy of 94% with learning rate = 0.001 for just 100 epochs of training. The model is trained on German Traffic signs Dataset which has 54 traffic signs. Git Repo Lane Line Tracking Tracking Lane Lines is very important in perception module of Self Driving Car Domain. To perform this the algorithm is tested on several videos of road taken from forward facing camera of a car. Gaussian noise is added for blur effect to the videos. The algorithm uses canny edge detector to detect edges and extra noise like trees from image are removed by selecting Region of Interest (ROI). Hough Transform is used to find lines and those lines are extrapolated to get left and right lane lines. In advance lane line tracking better gradient (Laplacian, sobel) and thresholding is done in different color spaces (HLS). Perspective transform is done and polynomial is fitted and warped back which gives better results detecting curved road than just straight lines. Git Repo Lane Line Tracking | Git Repo Advance Lane Line Tracking | Video Vehicle Detection The project aim is to detect cars. The pipeline works like this: 1. Perform a Histogram of Oriented Gradients (HOG) feature extraction on a labeled training set of images and train a classifier Linear SVM classifier 2. Implement a sliding-window technique and use trained classifier to search for vehicles in images. 3. create a heat map of recurring detections frame by frame to reject outliers and follow detected vehicles. 4. Estimate a bounding box for vehicles detected. Git Repo | Video Inertial Odometry using LSTM Estimation of egomotion is very important for Self Driving Cars and UAVs. Existing inertial odometry calculation were not good because they couldn't capture exact noise model. So, Recurrent Neural Network (LSTM) is used to predict relative motion between two camera frames, as some times huge motions couldn't be captured by visual odometry (because of no correspondences in images), then extra sensor like Inertial Measurement Unit (IMU) is used to capture that motion, this gives advantage of predicting better ego motion and could be fused be visual odometry. Git Repo Bachelor's Projects Autonomous Infrastructure Inspection Industrial Inspection (like examining cracks in industrial boilers) takes lot of time and money. Using UAVs both of those factors could be cut down effectively. The aim of this project is automating an UAV to do this task. It is equipped with Hukoyu laser scanner (LIDAR), onboard sensors (camera, IMU), controller (DJI Naza M) and computer (Odroid). Mesh is generated automatically from 3D pointcloud map and a initial 3D coverage plan (path and velocities) is generated considering the noise model (empirically calculated controller uncertainty). Raycasting is done to track the covered regions and realtime coverage planner adjusts its path optimizing flight time. Video Inventory management robot UGVs are widely used in warehouse management by Amazon and other companies. But they are mostly limited to moving the platforms but do not perform intelligent pick and place operations. Inspired by this idea an Autonomous Ground Robot equipped with a 3DOF robotic arm to manipulate things is built from scratch using 3D Modeling and Designing. For 3D mapping and Localization with Kinect camera RTABMap is used. A simple PID control is used for 3DOF robotic arm, and also for an extra application of person following in markets. Video Gesture controlled quadrotor using kinect sensor Controlling robots with gestures could be easier for human if there are many tasks. Kinect Camera is used for Video Games to play with gestures. An interface for ARDrone UAV and Kinect camera is created on ROS Platform for gesture control. Initially tested with 6 different gestures. As number of gestures increases overlap of gesture features (in task space) occurs. To efficiently classify them Artificial Neural Network is trained. Video Controlling quadrotor using Leap sensor Controlling robots with gestures could be easier for human if there are many tasks. LEAP Sensor is used for Video Games to play with gestures. An interface for ARDrone UAV and LEAP Sensor is created on ROS Platform for gesture control. Initially tested with 6 different gestures. As number of gestures increases overlap of gesture features (in task space) occurs. To efficiently classify them Artificial Neural Network is trained. Video Person following quadrotor UAVs are widely used in military and security applications. Tracking a car or person at lower heights with camera can be very hard sometimes as the target goes out of view more often and very fast. To address this problem we utilize a GPS sensor from person or car and track it along with providing live camera feed. Video PID tuning of quadrotor Even though quadrotor is a non-linear system, it can be approximated as a linear system locally and can be controlled with simple PID controllers for stabilization. Manual PID tuning of Quadrotor is done using Arduino Due, as part of control systems course to understand the concepts of feedback systems, underdamping, overdamping. Video Sphero A mobile robot that can work both on Land and water is expensive to built because of electronics on the robot. A simple spherical shell is 3D printed and all the electronics and motors (which control the motion) are kept inside safeguarding from water by shell. Also, it is wirelessly controlled with an android application from smartphone Video Fixed Wing UAV Fixed Wing UAV are more efficient in flight time (consuming less battery) compared to quadrotors. A fixed wing UAV is made with foam. Pixhawk is used along with mission planner for auto surveillance and 3D mapping application (using agisoft) Video Ground Robot using Leap gestures Long range ground robot control has wide variety of applications like remote sensing application in mines, nuclear sites. As a prototype for this application, zigbee communication is used for long range. Also, with gesture control with a LEAP Sensor. Video Path Tracing Robot In fire fighting applications, robots that record it's path travelled and coming back when visual feeback is not clear will be useful. For this application a mobile robot is developed that stores the path travelled (using servo motors, wheel odometry is stored) and traces its path back to intial position. Video Android based Digital Signal Oscilloscope Digital Oscilloscopes are costly and are not portable. Smart phones has capability of data visualization and computation. A simple module that measure analog signal with in a range and digitized using ADC. Bluetooth communication is used to send the data to a phone. It has limitations because of ADC conversion and Bluetooth communication frequency.