We’ll be expanding on the code that we built in Part — 1 of this series. If you haven’t tried that out, make sure you go back, try, and come back here!

Link to Part — 1 of this series

Using OpenCV with Gazebo in Robot Operating System (ROS) — Part 1 — Display real-time video feed…

Link to GitHub repo

It contains all the scripts, models and world files that are used in this tutorial series.

dhanuzch/pkg_cv_ros_tutorial_by_dhanuzch

Launching the gazebo world

Before running the next command make sure you have run git pull and sourced the setup.bash file that’s in {your_workspace_name}/devel/ folder.

Then, to launch the gazebo world, run this command in terminal.

$ roslaunch pkg_cv_ros_tutorial_by_dhanuzch 2_world.launch

This should launch a gazebo world that has a wall with QR code on it and a camera. It should look something like this…

If you can see this, Congrats! You have set everything correctly as of now.

Alternatively, if you want to launch gazebo world and start running the script out of the box. Then run

$ roslaunch pkg_cv_ros_tutorial_by_dhanuzch 2_world_and_script.launch

If you wanna know how the code works, then continue reading…

Image preprocessing

Before jumping on to the code, you gotta know about this thing called image preprocessing. It is something that we have to do to get meaningful data out of the raw camera output. Here’s an article on ‘Image pre-processing’ that I personally found useful. We’d be using some very basic preprocessing techniques in this tutorial. Below, I’ve discussed one of many ways to preprocess images to decode QR codes.

1. Convert the image to a grayscale image with COLOR_BGR2GRAY function that’s available in cv2
2. Since QR Codes are generally made up of two colors, we can make use of the threshold function. With that function, we can convert grayscale image that we obtained above, to a binary image that only has two colours, black and white.

Notice how clear the image is, after preprocessing. Now let’s move on to the code

The Code — Explained.

Since we are building up on the code that we discussed in Part — 1, I’ll be focusing only on the changes that were made.

This code is also available as a part of a package in my GitHub repo

pyzbar library

This library is is used to decode QR codes and barcodes, To learn more about this library, take a look at this article.

line 8imports the decode function that is available in pyzbar library.

Image preprocessing

line 29is to convert the resized_image to a grayscale image.

line 30 is to set a value for threshold. Note that threshold value depends on a lot of factors. And I’ve set it to 40. To learn about it in depth, take a look at the official guide.

line 31 converts the grayscale image to a binary image. Here, thresh,255means all pixel values above 40, will be set to 255(white).

Since the lighting conditions and the QR code is stable in this tutorial, we are taking this approach. But in real life using adaptive thresholding or other techniques might be the best bet.

Decoding

line 36 is the simplest 😛. All you have to do is call decode function with the image as an argument and the rest will be taken care by the pyzbar library. This will return decoded results.

line 38prints the decoded results.

The decoded result is as follows…

[Decoded(data="You're breathtaking!", type='QRCODE', rect=Rect(left=100, top=278, width=216, height=214), polygon=[Point(x=100, y=278), Point(x=100, y=490), Point(x=316, y=492), Point(x=315, y=278)])]

Doesn’t our output contain too many information?

To print just the decoded data. Let’s insert the following lines in our script.

qr_data = qr_result[0].data
print qr_data

Alright, this gives us the desired output. Which is… “You’re breathtaking” :) Don’t believe me? Here’s the output xD

Isn’t it slightly inconvenient to get the desired output in the terminal?

So, next we will try to print the data in Camera output window.

Drawing and writing meaningful stuff in cv window

Next, our goal is to draw a box over the detected QR Code and print the message on top of the box.

line 43 gets the position of QR code in window from qr_result

With the rectangle function available in cv2 and data obtained from line 43, line 45draws a rectangle around the QR code of colour red (0, 0, 255) and thickness 4

line 47 defines the format to print the data of QR Code

line 48 prints the data of QR code 10 pixels above the box. This is defined with (x, y-10), Font type is Hershey simplex. Colour is red (0, 0, 255) and thickness is 2. All these details will be printed in the window that displays resized_image

Now, try it with other QR codes. And see if the code works…

Lemme know in comments if you have any doubts or face difficulties. And also make sure to comment, what my next tutorial should be about!

That’s it for this series guys!

Bye!

Follow us for Robotics Memes — News — Tips — Projects & More

https://www.instagram.com/everythingrobotics/

• dhanuzch - Overview
• Everything Robotics

I hope you have everything set up. If you haven’t, follow Part — 0 of this series to set it up, and come back here.

Link to Part — 0 of this series

Using OpenCV with Gazebo in Robot Operating System (ROS) — Part 0 — Getting everything set up

Link to GitHub repo

It contains all the scripts, models and world files that are used in this tutorial series.

dhanuzch/pkg_cv_ros_tutorial_by_dhanuzch

Launching the gazebo world

Before running the next command make sure you have run git pull and sourced the setup.bash file that’s in {your_workspace_name}/devel/ folder.

Then, to launch the gazebo world, run this command in terminal.

$ roslaunch pkg_cv_ros_tutorial_by_dhanuzch 1_world.launch

This should launch gazebo world that has a pine tree, a camera and a turtlebot. It should look something like this…

If you can see this, Congrats! You have set everything correctly as of now.

Alternatively, if you want to launch gazebo world and start running the script out of the box. Then run

$ roslaunch pkg_cv_ros_tutorial_by_dhanuzch 1_world_and_script.launch

If you wanna know how the code works, then continue reading…

The Code — Explained.

This code is available as a part of a package in my GitHub repo

To run this script, run this command in terminal…

$ rosrun pkg_cv_ros_tutorial_by_dhanuzch camera_read.py

Importing the libraries

line 1 It’s called shebang (or) hashbang, and with it you’re telling your machine to use a specific interpreter. This is required in ROS because our script will be run as an executable. Without this your script will not be executed properly.

line 3&4 rospy allows us to interface ROS features with python. And cv2 imports the OpenCV library, with which we’ll be doing image processing.

line 6 It is ROS package that defines messages for most commonly used sensors. Visit this link to check out the message definition.

line 7 cv_bridge is a package that converts ROS image messages to openCV images and vice versa. Keep reading to know how we implement it.

main() function and __init__ function

line 42 initializes a ROS node with the name camera_read . When anonymous = True a random number is assigned to the end of the node name to make it unique. Since we just have one node with the same name, anonymous is set to False.

line 43 calls the main function.

When the main function is called…

line 32 invokes the __init__ function of class camera_1, this function creates an instance variable called image_sub, and it subscribes to /camera_1/image_raw topic.

That topic has message of type Image , and every time a new message arrives it calls the callback() function.

line 35 keeps the ROS node from shutting down, and it will yield activity to other threads.

line 39destroys all the openCV windows that were created.

callback() function

Finally we will look at the function that gets the job done.

line 18 converts the ROS image message to cv2 image with bgr8 encoding.

For example, let’s say if you want to convert a RGB image to a grayscale image, you may use mono8 instead of bgr8. If you’re unsure of this you may set the value as desired_encoding= “passthrough”

line 24 the image output is resized to 360 X 640 pixels. If we try to display the output of camera without downscaling, chances are it will be too big for the display and you might not be able to see the full window.

If you see in the above image, the bottom part of the output is cut off. So it is advisable to downscale the image.

line 27 the resized image is displayed, and the name of the window is set to "Camera output resized"

Follow us for Robotics Memes — News — Tips — Projects & More

https://www.instagram.com/everythingrobotics/

• dhanuzch - Overview
• Everything Robotics

What’s next?

In the Final part of this series, we will be scanning a QR code that’s in a Gazebo world.

Using OpenCV with Gazebo in Robot Operating System (ROS) — Part 2 — Read and decode a QR Code…

Ask your questions in the comments section! Bye :)

Intro

This will be a three part series, in which I’ll help you to use OpenCV with Gazebo in ROS (Robot Operating System)

• Part 0 — Getting everything set up
• Part 1 — Display real-time video feed of 2D camera in gazebo
• Part 2 — Read and decode a QR Code present in Gazebo with pyzbar library

Alrighty! Let’s get started with Part 0. Oh wait. Why did I start with Part 0 instead of Part 1 tho? Coz I wanted to flex on ’em non-CS folks and confuse them xD

Getting things in order

1. Installing Gazebo

Assuming you have ROS and Gazebo installed, I’ll move forward with this tutorial. If you’re not sure if Gazebo is installed, type gazebo in terminal. It should open a window like this

If you don’t get a screen like this, follow the official tutorial in this link to install gazebo on your machine

2. Installing OpenCV

Once you’re done with that, you gotta install OpenCV. To do that run the following command in terminal

$ sudo apt-get install libopencv-dev python-opencv

3. Installing pyzbar

Up next, we have to install a python library called pyzbar that allows us to scan barcodes and QR codes using 2D camera. We will be using pyzbar in part 2 of this series.

To install, run this command in terminal.

$ pip install pyzbar

If you haven’t installed pip on you machine, run this command in terminal.

$ sudo apt install python-pip

4. Installing cv_bridge

Finally, to install cv_bridge run these commands separately in terminal.

$ sudo apt-get install ros-$ROS_DISTRO-vision-opencv

$ sudo apt-get install ros-$ROS_DISTRO-cv-bridge

We’ll see more about this package in the upcoming section.

5. Using my customized ROS package

All the scripts, models and world files used in this series are available in my repository as a ROS Package.

Link to GitHub Repo

dhanuzch/pkg_cv_ros_tutorial_by_dhanuzch

Alternatively, you can clone it to your PC by running this command in src folder of your workspace(for example, catkin_ws/src)

$ git clone https://github.com/dhanuzch/pkg_cv_ros_tutorial_by_dhanuzch.git

And then compile the package by running catkin build or catkin_make , This depends on which build system you use. I personally prefercatkin build over catkin_make because it has additional functionalities and really useful tools that catkin_makedoesn’t offer.

To learn more about the differences between these two

Intro to cv_bridge

Most of you reading this might know what ROS is and how fast the number of ROS powered robots is growing day by day. Nowadays, ROS is being used in NASA’s Rovers, vaccuum cleaners and even in boring industrial robots :p

And robot vision is a key aspect in Robotics, Many robots use 3D depth sensors and other sensors such as LIDARs and RADARs to perceive the world. Similiarly 2D cameras are used in robots for various objectives and tasks. Using 2D cameras has its own set of advantages that other sensors simply can’t provide.

OpenCV is a tool that allows us to process images from those cameras and it enables us to recognize faces, objects and texts and finally what we will be doing in this tutorial series — decode QR Codes. There are so many applications to openCV, just a simple google search will give you an idea of the features and functionalities of openCV.

cv_bridge allows us to use those functionalities and features in our robots that run on ROS. In other words, it bridges the gap between ROS and openCV, allowing us to use and explore a whole array of features related to robot vision.

Follow us for Robotics Memes — News — Tips — Projects & More

https://www.instagram.com/everythingrobotics/

• dhanuzch - Overview
• Everything Robotics

What’s next?

In the next part of this series, we will be using cv_bridge to display the output of a camera placed in gazebo world.

Using OpenCV with Gazebo in Robot Operating System (ROS) — Part 1 — Display real-time video feed…

Keep flexing on ’em non CS peeps :)

Bye!