Process of achieving color and image recognition by myPalletizer AI Kit



  • Based on the Linux system and a 1:1 simulation model in ROS, the AI Kit composes of the vision, positioned gripping, and automatic sorting modules. Featuring computer vision, an equipped camera can recognize and locate the cubes of different colors or images through OpenCV, and then the core processor of the a robotic arm can calculate their current and targeted spatial coordinate positions, and finally grip a cube into the corresponding barrels.

    Now myPalletizer 260 is capitable with AI Kit, and here is the detailed process of achieving color and image recognition by myPalletizer AI Kit.
    0_1657091026425_微信图片_20220705135328.png
    Color recognition

    1. Press "Ctrl"+"Alt"+"T" on the keyboard to open a command window and run the following command to start the master node:
      roscore
    2. Press "Ctrl"+"Shift"+"T" in the command terminal to open another command window in the same directory, and then type the command to launch the "vision.launch" file:
      roslaunch myPalletizer_260 vision.launch
    3. Press "Ctrl"+"Alt"+"T" to open another command window and enter the command to go to the file directory to be operated
      cd catkin_ws/src/mycobot_ros/mycobot_ai/myPalletizer_260/
    4. Run the program "detect_obj_color.py", then we can acheive the color recognition and capture
      python scripts/detect_obj_color.py
      0_1657091120960_图片1.png

    Image recognition

    1. Before running "detect_obj_IMG_FOLder_opt. py" program, we need to ensure that steps 1, 2 and 3 in color recognition have been performed
    2. Run the program "add_img.py" and add the images we want to recognize to the specified folder ('red', 'blue', 'green', 'gray').
      python scripts/add_img.py
      0_1657091152360_微信图片_20220705135130.jpg

    According to the prompts input by the terminal, we capture the image in the second image box.

    1. After the image area capturing is completed, press "Enter" and enter the name of the folder to save the image (' Red ', 'Blue ',' Green ', 'Gray ') as prompted by the terminal. Press "Enter" again to save the image to the corresponding folder.
    2. Start the image recognition program "detect_obj_img_folder.py", and verify that the robotic arm can correctly pick the pictures added in step 2 and 3, and place them into the barrel with the corresponding color.
      python scripts/detect_obj_img_folder_opt.py
      0_1657091186831_图片3.png

    This is designed for individuals developers as well as STEM educators who are interested in applications combining AI with robotics. Unless color and image recognition, what application scenarios do you want to achieve if you get the AI Kit? Please share with us in the comment!