IFRA (Intelligent Flexible Robotics and Assembly) Group
Centre for Robotics and Assembly
Cranfield University
Table of Contents
The IFRA (Intelligent Flexible Robotics and Assembly) Group is part of the Centre for Robotics and Assembly at Cranfield University.
IFRA Group pushes technical boundaries. At IFRA we provide high tech automation & assembly solutions, and we support smart manufacturing with Smart Industry technologies and solutions. Flexible Manufacturing Systems (FMS) are a clear example. They can improve overall operations and throughput quality by adapting to real-time changes and situations, supporting and pushing the transition towards flexible, intelligent and responsive automation, which we continuously seek and support.
The IFRA Group undertakes innovative research to design, create and improve Intelligent, Responsive and Flexible automation & assembly solutions, and this series of GitHub repositories provide background information and resources of how these developments are supported.
SOCIAL MEDIA:
IFRA-Cranfield:
- YouTube: https://www.youtube.com/@IFRACranfield
- LinkedIn: https://www.linkedin.com/in/ifra-cranfield/
Centre for Robotics and Assembly:
- Instagram: https://www.instagram.com/cranfieldrobotics/
- Facebook: https://www.facebook.com/cranfieldunirobotics/
- YouTube: https://www.youtube.com/@CranfieldRobotics
- LinkedIn: https://www.linkedin.com/company/cranfieldrobotics/
- Website: https://www.cranfield.ac.uk/centres/centre-for-robotics-and-assembly
Welcome to the irb120_PoseEstimation repository! This repository contains a comprehensive solution for executing a cube pick-and-place task with the ABB IRB-120 industrial robot and the Schunk EGP-64 parallel gripper. The ROS 2 packages provided here enable seamless execution in both simulated and real environments. Leveraging ROS 2 Humble on an Ubuntu 22.04 PC, this repository furnishes all necessary source code, ensuring compatibility and ease of deployment.
To enhance the perception capabilities of the ABB IRB120 robot for the application, a combination of YOLOv8, OpenCV, and ROS 2 has been employed. This integration empowers the system with advanced object detection and recognition capabilities, allowing for efficient and reliable execution of the cube pick-and-place task.
Cube Pick-and-Place Task:
The system, equipped with a standard web camera, adeptly detects the randomly spawned cube within the workspace and proceeds to classify it based on its distinct colored feature. The intricate task of manipulating the cube involves precise pose estimation, ensuring its placement within the designated tray and alignment with the corresponding slot, with the identified feature positioned upwards.
Explore the packages and get started with your cube pick-and-place tasks today!
Video Demonstration: Program Execution (Simulation + Real Robot)
The installation and execution of the irb120_PoseEstimation ROS2 packages requires the previous installation and set-up of the following components:
- Ubuntu 22.04 machine with ROS2 Humble.
- ROS2 Packages for the Simulation and Control of Robots using Gazebo and MoveIt!2.
- YOLOv8 for the object detection feature.
- OpenCV for the Image Processing feature.
The packages used to run the Simulation and Control of the ABB IRB-120 Robot in irb120_PoseEstimation are based on the IFRA-Cranfield/ros2_SimRealRobotControl repository. Therefore, it is recommended to follow the installation steps defined in that repository in order to properly set-up a ROS2 Humble machine for Robot Simulation and Control. To facilitate your work, those steps have been summarized and outlined in the INSTALLATION.md document here.
The latest version (v8) of YOLO (You-Only-Look-Once) can be installed using the following pip command, which installs YOLOv8 along with the requirements for a Python>=3.8 environment with PyTorch>=1.8:
pip install ultralyticsOpenCV-Python is needed for this repo, and it can be installed with the following command:
sudo apt-get install python3-opencvOnce the ROS2 Humble environment has been properly set up, and all the required packages have been installed, the ROS2 Packages of the irb120_PoseEstimation repository can be installed by executing the following commands in the terminal:
cd ~/dev_ws/src
git clone https://github.com/IFRA-Cranfield/irb120_PoseEstimation.git
cd ~/dev_ws
colcon buildWithin this repository, you'll find a comprehensive integration of ROS 2 Robot Simulation and Control packages, including Gazebo, MoveIt!2, and Robot Bringup packages, meticulously combined with a custom ROS 2 package housing the YOLOv8 and OpenCV modules, aptly named the detection package. This amalgamation provides a robust foundation for executing the cube pick-and-place task seamlessly across both simulated and real environments. The ROS 2 Robot Simulation and Control packages facilitate simulation and control of the ABB IRB-120 industrial robot, while the custom detection package empowers the system with advanced object detection and recognition capabilities, essential for precise manipulation and pose estimation of the target cube. Explore these packages to delve into the intricacies of robotic perception and control, accelerating the development and deployment of robotic applications.
The Gazebo package within this repository is a pivotal component, encompassing crucial data related to the robot's representation in the Unified Robot Description Format (.urdf). Beyond merely describing the robot's physical structure, this package also houses essential parameters necessary for fine-tuning ROS 2 controllers, ensuring precise control over the ABB IRB-120 industrial robot's movements. Furthermore, it includes detailed CAD files offering insight into the robot's design, alongside pertinent information concerning the Gazebo environment. This comprehensive package serves as the cornerstone for simulating and accurately replicating the robot's behavior within the simulated environment, facilitating seamless development and testing of robotic applications.
You can access all the Gazebo ROS 2 Package source code here.
The MoveIt!2 package within this repository is essential for motion control of the ABB IRB-120 industrial robot. It serves as a foundational element for motion planning and execution, offering comprehensive information required by the MoveIt!2 tool. This package includes crucial data such as robot specifications, obtained from the Gazebo package, ensuring consistency and accuracy in robot representation. Moreover, it encapsulates controller parameters necessary for fine-tuning and optimizing the robot's movements, enhancing its precision and efficiency. Additionally, the package provides a seamless Motion Planning interface, empowering users to effortlessly generate and execute motion plans, thereby streamlining the development and deployment of complex robotic applications.
You can access all the MoveIt!2 ROS 2 Package source code here.
The Robot Bringup package within this repository acts as the main link between the ROS 2 system and the real robot arm. It facilitates seamless integration of the robot arm control with ROS 2, requiring the presence of a robust ROS 2 driver. This driver operates concurrently within both the robot controller and ROS 2 environment, enabling comprehensive control over the robot's motion and state through various ROS 2 nodes. By establishing this vital connection, the Robot Bringup package enables seamless communication and coordination between the ROS 2 system and the real robot arm, laying the groundwork for efficient and reliable execution of robotic tasks in real-world environments.
You can access all the Robot Bringup ROS 2 Package source code here.
The detection package within this repository encompasses essential functionalities crucial for effective cube detection, classification, pose estimation, and pick-and-place tasks. It incorporates the YOLOv8 module for robust cube detection and classification, enabling the system to accurately identify the target object within the workspace. Additionally, the package integrates the OpenCV module for precise 6DoF Pose Estimation of the cube, ensuring accurate placement and alignment during manipulation tasks. Furthermore, the robot and gripper modules housed within this package contain ROS 2 Action Clients responsible for communicating with MoveIt!2, facilitating seamless operation of the robot movements. Lastly, the detection package encompasses the entire program logic for the application, orchestrating the coordination and interaction of various modules for efficient cube detection, pose estimation, and pick-and-place tasks, thereby enhancing the overall functionality and performance of the system.
You can access all the Robot Bringup ROS 2 Package source code here.
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated. If you have a suggestion that would make this better, or you find a solution to any of the issues/improvements presented above, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks you very much!
Intelligent Flexible Robotics and Assembly Group
Created on behalf of the IFRA Group at Cranfield University, United Kingdom
E-mail: IFRA@cranfield.ac.uk
Licensed under the Apache-2.0 License.
You may obtain a copy of the License at: http://www.apache.org/licenses/LICENSE-2.0
Cranfield University
School of Aerospace, Transport and Manufacturing (SATM)
Centre for Robotics and Assembly
College Road, Cranfield
MK43 0AL, Bedfordshire, UK
You can cite our work with the following statement:
IFRA-Cranfield (2023). Object Detection and Pose Estimation within a Robot Cell. URL: https://github.com/IFRA-Cranfield/irb120_PoseEstimation
Reference to CONFERENCE PAPER (if published) to be added here.
Submitted to IEEE-CASE2024 conference.
Mikel Bueno Viso - Research Assistant in Intelligent Automation at Cranfield University
E-mail: Mikel.Bueno-Viso@cranfield.ac.uk
LinkedIn: https://www.linkedin.com/in/mikel-bueno-viso/
Profile: https://www.cranfield.ac.uk/people/mikel-bueno-viso-32884399
Irene Bernardino Sanchez - Robotics MSc Student
E-mail: i.bernardinosanchez.854@cranfield.ac.uk
LinkedIn: https://www.linkedin.com/in/irene-bernardino-sanchez-08bbb1195/
Dr. Seemal Asif - Lecturer in Artificial Intelligence and Robotics at Cranfield University
E-mail: s.asif@cranfield.ac.uk
LinkedIn: https://www.linkedin.com/in/dr-seemal-asif-ceng-fhea-miet-9370515a/
Profile: https://www.cranfield.ac.uk/people/dr-seemal-asif-695915
Professor Phil Webb - Professor of Aero-Structure Design and Assembly at Cranfield University
E-mail: p.f.webb@cranfield.ac.uk
LinkedIn: https://www.linkedin.com/in/phil-webb-64283223/
Profile: https://www.cranfield.ac.uk/people/professor-phil-webb-746415