Developing an Object Detection and Gripping Mechanism Algorithm using Machine Learning
DOI:
https://doi.org/10.59097/jasae.v1i2.15Keywords:
Object detection , Localizing , YOLO , Gripping, RobotAbstract
Localizing and recognition of objects are critical problems for indoor manipulation tasks. This paper describes an algorithm based on computer vision and machine learning that does object detection and gripping tasks. Detection of objects is carried out using a combination of a camera and depth sensor using a Kinect v1 depth sensor. Moreover, machine learning algorithms (YOLO) are used for computer vision. The project presents a method that allows the Kinect sensor to detect objects' 3D location. At the same time, it is attached to any robotic arm base, allowing for a more versatile and compact solution to be used in stationary places using industrial robot arms or mobile robots. The results show an error of locating an object to be 5 mm. and more than 70% confidence in detecting objects correctly. There are many possibilities in which this project can be used, such as in industrial fields, to sort, load, and unload different kinds of objects based on their type, size, and shape. In agriculture fields, to collect or sort different kinds of fruits, in kitchens and cafes where sorting objects like cups, bottles, and cans can occur. Also, this project can be added to mobile robots to do indoor human services or collect trash from different places.
References
Zhang G, Jia S, Zeng D, Zheng Z. Object detection and grabbing based on machine vision for Service Robot. 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON).2018; https://doi.org/10.1109/iemcon.2018.8615062.
Jia S, Ju Z, Xu T, Zhang H, Li X. Mechanical arm grasping and target recognition strategy based on the coarse-to-fine algorithm. 2016 IEEE International Conference on Information and Automation (ICIA), 2016;1812-1817. https://doi.org/10.1109/ICInfA.2016.7832112.
Han M, Wang P, Wang X, Li X. Mobile Grabbing Robot with Target Detection Using Quadric Fitting. 2021 IEEE International Conference on Integrated Circuits, Technologies and Applications (ICTA). 2021; 199-200, https://doi.org/10.1109/ICTA53157.2021.9661895.
Ouyang W, Huang W, Min H. Robot Grasp with Multi-object Detection based on RGB-D Image. 2021 China Automation Congress (CAC). 2021;6543-6548. https://doi.org/10.1109/CAC53003.2021.9728678.
Chi W, Meng MQ, Chen X. Robot aided object segmentation based on kinect without prior knowledge. 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO). 2012;1784-1788. https://doi.org/10.1109/ROBIO.2012.6491226.
Ali MM, Liu H, Stoll R, Thurow K. Arm grasping for mobile robot transportation using Kinect sensor and kinematic analysis. 2015 IEEE International Instrumentation and Measurement Technology Conference (I2MTC) Proceedings 2015; 516-521. https://doi.org/10.1109/I2MTC.2015.7151321.
Ali MM, Liu H, Stoll N, Thurow K. Intelligent arm manipulation system in life science labs using H20 mobile robot and Kinect sensor. 2016 IEEE 8th International Conference on Intelligent Systems (IS). 2016;382-387. https://doi.org/10.1109/IS.2016.7737449.
Behzadi-Khormouji H, Derhami V, Rezaeian H. Adaptive Visual Servoing Control of robot Manipulator for Trajectory Tracking tasks in 3D Space. 2017 5th RSI International Conference on Robotics and Mechatronics (ICRoM).2017;376-382, https://doi.org/10.1109/ICRoM.2017.8466231.
Shim KH, Jeong JH, Kwon BH, Lee BH, Lee SW. Assistive Robotic Arm Control based on Brain-Machine Interface with Vision Guidance using Convolution Neural Network. 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), 2019;2785-2790. https://doi.org/10.1109/SMC.2019.8914058.
Ma Q, Niu J, Ouyang Z, Li M, Ren T, Li Q. Edge Computing-based 3D Pose Estimation and Calibration for Robot Arms. 2020 7th IEEE International Conference on Cyber Security and Cloud Computing (CSCloud)/2020 6th IEEE International Conference on Edge Computing and Scalable Cloud (EdgeCom). 2020; 246-251. https://doi.org/10.1109/CSCloud-EdgeCom49738.2020.00050.
Farag M, Abd Ghafar AN, Alsibai MH. Grasping and Positioning Tasks for Selective Compliant Articulated Robotic Arm Using Object Detection and Localization: Preliminary Results. 2019 6th International Conference on Electrical and Electronics Engineering (ICEEE), 2019;284-288. https://doi.org/10.1109/ICEEE2019.2019.00061.
Mars 2020 Perseverance Rover Wallpaper [1080x2297] : r/MobileWallpaper n.d. https://www.reddit.com/r/MobileWallpaper/comments/ltjuio/mars_2020_perseverance_rover_wallpaper_1080x2297/
ros-visualization/rviz. GitHub. https://github.com/ros-visualization/rviz (accessed Feb. 02, 2022).
AmandaDattalo. ROS/Introduction - ROS Wiki 2019. http://wiki.ros.org/ROS/Introduction (accessed February 2, 2022).
Karl-Bridge-Microsoft. Kinect for Windows - Windows apps 2022. https://learn.microsoft.com/en-us/windows/apps/design/devices/kinect-for-windows (accessed February 27, 2023).
Keras Team. Keras documentation: Getting started n.d. https://keras.io/getting_started/ (accessed February 2, 2022).
TensorFlow Developers. TensorFlow 2023. https://doi.org/10.5281/ZENODO.4724125.
MarkoBjelonic. darknet_ros - ROS Wiki 2018. https://wiki.ros.org/darknet_ros (accessed February 2, 2022).
Tossy. YOLO V4 for darknet_ros 2023.
Bjelonic M. YOLO ROS: Real-Time Object Detection for ROS 2018.
Nvidia. CUDA Instalation Guide Windows. CUDA Instalation Guide Windows 2022. https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html (accessed February 2, 2022).
Rico FM, Ramos FG. gb_visual_detection_3d 2023.
Rico FM. Personal Blog Francisco Martín Rico 2022. https://gsyc.urjc.es/~fmartin/ (accessed February 2, 2022).
Epson America, Inc. 6-Axis Robots | Explore by Product Series | Epson US n.d. https://epson.com/6-axis-robots-product-family (accessed February 2, 2022).
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Hazem Mohammad, Sameer Kishore, Judhi Prasetyo
This work is licensed under a Creative Commons Attribution 4.0 International License.
