so-101 Robotic Arm

I have finished the first working version of a system to autonomously gather data. This system is built on two policies that each resets the other's environment allowing for a closed loop that gathers data without intervention. A separate YOLO vision model verifies the success or failure of each attempted episode to ensure all data collected is valid. I am now gathering tons of data from this system and experimenting on training policies partially and fully on this autonomously gathered data. I also plan on extending this system to other tasks as currently the task (taking an object in and out of a box) is not a useful or difficult task, and was chosen as a first test for ease of implementation.

I have finished building a 2nd SO-101 to serve as the leader arm for teleoperation. Additionally I discovered that my on arm camera was having visual glitches, and have since swapped out the cameras. Currently with those issues fixed I am training the arm to place an object in the box, then take it out as to have a fully closed loop where it can both perform an action and reset its environment.

I am now using phone teleoperation to collect training data for placing various objects into a cup. I am also working on a program to collect training data from the robot's autonomous operation, aiming to train a model to perform the task more reliably using only autonomously collected data.

I built a small program for my Music and Coding class that allows the robot to play any melody in the C major scale. The video above shows the robot playing Twinkle Twinkle Little Star. This was a simple project to familiarize myself with the basics of robot control for my SO-101.

I built this SO-101 robotic arm using servos and 3D-printed parts and am currently setting it up using LeRobot. I plan to research how to collect meaningful data using self-supervised play and data collection. I will then use this data to train the robot to perform tasks.