Mobile robots

Why mobile robots? They have many applications, programming them is a challenging task, especially when get to the part of objects detection with a camera and mapping. The robot has many peripheral devices, such as motors and sensors, and can be expanded with an additional arm for lifting objects.

I became interested in building robots in 2008. These were completely different times, almost no one had heard of Arduino, and Raspberry Pi had not yet been born. Building a mobile robot required more work than today. For the first project, I decided to buy a development board with an Atmel chip. I had got online a books for programming microcontrollers in BASCOM. After a few weeks, I built a six-wheeled vehicle. It had two DC motors and four servos to turn the wheels. There is a proximity sensor installed at the front of the vehicle, which I built from scratch.

6 wheel vehicle (BASCOM)

Design mobile robot

In 2009 bought an Arduino Duemilanove which had an ATmega328p on board, a chip with more outputs and inputs than the ATmega8 that was used in six wheeled vehicle. Experimented a bit with diodes and buttons and started building a line follower. Decided that would postpone the six-wheeler project as it required more work and money(let's not forget the crisis). The driving direction is changed by adjusting the speed of the driven wheels. On the front is a sensor module for color recognition. On the back is an arduino and a motor controller built and a color sensor controller.

Design Line follower

Line follower (Arduino)

Line Follower (Arduino)

Until 2011, in my spare time, experimented with sonar and obstacle sensors and a sumo robot, but due to lack of time, stopped doing mobile robot and returned to the topic in 2020. Started building a mobile robot, this time 4 wheeled with Raspberry Pi.

I realized very quickly that in order to continue this project, I had to learn Python. In two month did "Complete Python Developer: Zero to Mastery" and then "Master Computer Vision System and OpenCV3" on Udemy. These courses allowed me to start playing with Raspberry and building a robot that recognizes objects.

4 wheels (Raspberry Pi, Google Coral)

4 wheels with camera

As it turned out, the Raspberry Pi 3 has a weak processor and not enough RAM, which makes object recognition very slow. Started delving into the field of object recognition and found an interesting topic like Tensorflow. Decided to use Tensorflow Lite and the built-in model on a Raspberry Pi, but the image was still analyzed at 2 frames per second. That's why I bought a Google Coral accelerator which accelerated object recognition to 8 frames per second.

Tensorflow Object Recognition

Tensorflow Driving Car

In 2020, I decided to take the Software Developer course, so I didn't have time to continue with the project until summer 2021.

During the summer break, I used the new skills I acquired in the course and created a web application in Python and Flask

This application allows me to remotely control the robot using a laptop or smartphone.

Web App (Python + Flask)

In 2022, during the Christmas break, I had some time to teach the Raspberry pi to recognize objects.

I trained my system to recognize the remote control, cup, baubles and bin. To do this, I followed instructions from the official tensorflow manual. Prepared 300 photos of objects that I want to recognize. The problem is that for training purposes the size of the photos should be smaller than phone's camera. Photos should not be larger than 800x600 pixels, otherwise more computational time is needed for training. I used the Photoshop to do this, but unfortunately after some time realized it could take hours so had to find another way.

Decided to make my own mini application that changes the resolution of photos. A program that allows me to change the resolution of photos in seconds is available on GitHub. Application allow to reduce size of the photos and videos. You can just download this app, no need to install and add the photos and videos that you want to convert to smaller size and run application.

In the following videos is vision system with Raspberry pi4 (Tensorflow). Using the Coral accelerator allows increase to 14 frames per second.

Tensorflow Object Recognition

Object Recognition with Google Coral

In July 2023, I taught my robot to recognize objects like signs Start, Go, Stop, Turn Left, Turn Right and Turn back.

I implemented a control system for my robot depending on what it recognized. Below you can see that when the robot recognizes GO, it starts to go forward when it encounters a sign, turn right, then it goes right and stops at the end when it sees "stop".

I used program on Google Colab Notebook to train a model. This code allows my robot to "make a decision" based on what it sees.

Using the Raspberry Pi alone, I can only get 2 FPS, but the Coral accelerator allows me to increase that speed to 9 FPS. As you can see, the raspberry pi allows me to create an autonomous robot with near real-time object detection, but when the robot is driving and using other sensors, the frame rate drops dramatically. In my opinion, the Raspberry Pi is a very good tool for basic task, but for a more complex task like ROS, it's probably not good enough. I think that something from the Jetson range would be a better option because has better GPU.

GO, Turn Right and Stop

Object Recognition with Google Coral