Skip to main content

Advertisement

Springer Nature Link
Account
Menu
Find a journal Publish with us Track your research
Search
Cart
  1. Home
  2. Journal of Intelligent & Robotic Systems
  3. Article

Integration of Artificial Vision and Image Processing into a Pick and Place Collaborative Robotic System

  • Regular paper
  • Open access
  • Published: 08 November 2024
  • Volume 110, article number 159, (2024)
  • Cite this article
Download PDF

You have full access to this open access article

Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript
Integration of Artificial Vision and Image Processing into a Pick and Place Collaborative Robotic System
Download PDF
  • Adriano A. Santos  ORCID: orcid.org/0000-0001-8991-67681,2,
  • Cas Schreurs1,
  • António Ferreira da Silva  ORCID: orcid.org/0000-0003-2067-76011,2,
  • Filipe Pereira  ORCID: orcid.org/0000-0001-7809-55543,
  • Carlos Felgueiras  ORCID: orcid.org/0000-0002-4202-55514,
  • António M. Lopes  ORCID: orcid.org/0000-0001-7359-43702 &
  • …
  • José Machado  ORCID: orcid.org/0000-0002-4917-24743 

  • 719 Accesses

  • Explore all metrics

Abstract

In the field of robotics, pick and place applications are becoming increasingly popular due to their ability to automate repetitive tasks that can create temporary or permanent injuries. To enhance the efficiency of these applications, object recognition using a fixed camera or one mounted on a robotic hand has been employed. This paper explores the possibilities of implementing a low-cost camera into a collaborative robotic system. A software architecture has been developed, including modules for perception, pick and place, and part transfer. A comprehensive overview of various intuitive drag-and-drop image processing technologies and their suitability for object recognition in a robotic context is provided. The challenges related to lighting and the effect of shadows in object recognition are discussed. A critical assessment is made of the architecture development platform as well as the study and the results are performed, and the effectiveness of the proposed solution based on the Niop architecture is verified.

Article PDF

Download to read the full article text

Similar content being viewed by others

A Review of Pick and Place Operation Using Computer Vision and ROS

Chapter © 2022

Industry 4.0, Intelligent Visual Assisted Picking Approach

Chapter © 2018

Industrial workspace detection of a robotic arm using combined 2D and 3D vision processing

Article Open access 20 December 2024

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.
  • Artificial Intelligence
Use our pre-submission checklist

Avoid common mistakes on your manuscript.

References

  1. Patil, S., Vasu, V., Srinadh, K.V.S.: Advances and perspectives in collaborative robotics: a review of key technologies and emerging trends. Discov. Mech. Eng. 2, 13 (2023). https://doi.org/10.1007/s44245-023-00021-8

    Article  MATH  Google Scholar 

  2. Bormann, R., de Brito, B.F., Lindermayr, J., Omainska, M., Patel, M. Towards Automated Order Picking Robots for Warehouses and Retail. In: Tzovaras, D., Giakoumis, D., Vincze, M., Argyros, A. (eds).: Computer Vision Systems. ICVS 2019. Lecture Notes in Computer Science, 11754, 185–198. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-34995-0_18.

  3. Silveira, M., Santos, A.A., Pereira, F., da FerreiraSilva, A., FelgueirasRamosMachado, C.A.J.: 3D Vision Object Identification Using Yolov8. Int. J. Mechatron. Appl. Mech. 17, 7–15 (2024). https://doi.org/10.17683/ijomam/issue17.1

    Article  Google Scholar 

  4. Keshvarparast, A., Battini, D., Battaia, O., et al.: Collaborative robots in manufacturing and assembly systems: literature review and future research agenda. J. Intell. Manuf. 35, 2065–2118 (2024). https://doi.org/10.1007/s10845-023-02137-w

    Article  Google Scholar 

  5. Javaid, M., Haleem, A., Singh, R.P., Rab, S., Suman, R.: Significant applications of Cobots in the field of manufacturing. Cogn. Robot. 2, 222–233 (2022). (ISSN 2667-2413)

    Article  Google Scholar 

  6. Borboni, A., Reddy, K.V.V., Elamvazuthi, I., AL-QuraishiNatarajanAzhar Ali, M.S.E.S.S.: The Expanding Role of Artificial Intelligence in Collaborative Robots for Industrial Applications: A Systematic Review of Recent Works. Machines 11, 111 (2023). https://doi.org/10.3390/machines11010111

    Article  Google Scholar 

  7. Kakade, S., Patle, B., Umbarkar, A.: Applications of collaborative robots in agile manufacturing: a review. Robot. Syst. Appl. 3(1), 59–83 (2023). https://doi.org/10.21595/rsa.2023.23238

    Article  MATH  Google Scholar 

  8. Asaad, H., Askar, S., Kakamin, A., Nayla, F.A.: Exploring the Impact of Artificial Intelligence on Humanrobot Cooperation in the Context of Industry 4.0. Appl. Comput. Sci. 20, 138–155 (2024). https://doi.org/10.35784/acs-2024-21

    Article  Google Scholar 

  9. Montini, E., Daniele, F., Agbomemewa, L., et al.: Collaborative Robotics: A Survey From Literature and Practitioners Perspectives. J. Intell. Robot. Syst. 110, 117 (2024). https://doi.org/10.1007/s10846-024-02141-z

    Article  Google Scholar 

  10. Jørgensen, T.B., Jensen, S.H.N., Aanæs, H., et al.: An Adaptive Robotic System for Doing Pick and Place Operations with Deformable Objects. J. Intell. Robot. Syst. 94, 81–100 (2019). https://doi.org/10.1007/s10846-018-0958-6

    Article  MATH  Google Scholar 

  11. Sezer, A., Altan, A.: Detection of solder paste defects with an optimization-based deep learning model using image processing techniques. Soldering Surface Mount Technol. 33(5), 291–298 (2021). https://doi.org/10.1108/SSMT-04-2021-0013

    Article  MATH  Google Scholar 

  12. Yağ, İ, Altan, A.: Artificial Intelligence-Based Robust Hybrid Algorithm Design and Implementation for Real-Time Detection of Plant Diseases in Agricultural Environments. Biology (Basel) 11(12), 1732 (2022). https://doi.org/10.3390/biology11121732

    Article  MATH  Google Scholar 

  13. Anjum, M.U., Khan, U.S., Qureshi, W.S., Hamza, A., Khan. W.A.: Vision-Based Hybrid Detection For Pick And Place Application In Robotic Manipulators. 2023 International Conference on Robotics and Automation in Industry (ICRAI), Peshawar, Pakistan, 1–5 (2023). https://doi.org/10.1109/ICRAI57502.2023.10089602.

  14. Borrell, J., Perez-Vidal, C., Segura, J.V.: Optimization of the pick-and-place sequence of a bimanual collaborative robot in an industrial production line. Int. J. Adv. Manuf. Technol. 130, 4221–4234 (2024). https://doi.org/10.1007/s00170-023-12922-9

    Article  MATH  Google Scholar 

  15. Gomes, N.M., Martins, F.N., Lima, J., Wörtche, H.: Reinforcement Learning for Collaborative Robots Pick-and-Place Applications: A Case Study. Automation 3, 223–241 (2022). https://doi.org/10.3390/automation3010011

    Article  MATH  Google Scholar 

  16. Javaid, M., Haleem, A., Singh, R.P., Rab, S., Suman, R.: Significant applications of Cobots in the field of manufacturing. Cogn. Robot. 2, 222–233 (2022). https://doi.org/10.1016/j.cogr.2022.10.001. (ISSN 2667-2413)

    Article  Google Scholar 

  17. Pereira, F., Pinto, L., Soares, F., Vasconcelos, R., Machado, J., Carvalho, V.: Online yarn hairiness– Loop & protruding fibers dataset. Data Brief 54, 110355 (2024). https://doi.org/10.1016/j.dib.2024.110355. (ISSN 2352-3409)

    Article  Google Scholar 

  18. Patil, S., Vasu, V., Srinadh, K.V.S.: Advances and perspectives in collaborative robotics: a review of key technologies and emerging trends. Discov. Mech. Eng. 2, 13 (2023). https://doi.org/10.1007/s44245-023-00021-8

    Article  MATH  Google Scholar 

  19. Universal Robots. https://www.universal-robots.com/about-universal-robots (2024). Accessed on 18 April 2024.

  20. Zimmer group. HRC-03 2-jaw parallel gripper. https://www.zimmer-group.com/en-us/technologies-components/components/handling-technology/grippers/hrc/collaborative/2-jaw-parallel-grippers/hrc-03 (2024). Accessed on 18 April 2024.

  21. Intel RealSense. https://www.intelrealsense.com/depth-camerad435i (2024). Accessed on 18 April 2024.

  22. Neadvance. Available online: https://neadvance.com (2024). Accessed on 18 April 2024.

  23. Niop. https://neadvance.com/products/niop (2024). Accessed on 18 April 2024.

  24. Szeliski, Richard. Introduction. In Computer vision: algorithms and applications, 2nd ed., Springer Nature, Seattle, WA, USA, pp. 1–26 (2022).

  25. P. Ongsulee. Artificial intelligence, machine learning and deep learning. In 15th International Conference on ICT and Knowledge Engineering (ICT&KE), Bangkok, Thailand (2017). https://doi.org/10.1109/ICTKE.2017.8259629.

  26. Dziubek, M., Rysiński, J., Jancarczyk, D.: Exploring the ViDiDetect Tool for Automated Defect Detection in Manufacturing with Machine Vision. Appl. Sci. 13, 11098 (2023). https://doi.org/10.3390/app131911098

    Article  MATH  Google Scholar 

  27. Haffner, O., Kučera, E., Rosinová, D.: Applications of Machine Learning and Computer Vision in Industry 4.0. Appl. Sci. 14, 2431 (2024). https://doi.org/10.3390/app14062431

    Article  MATH  Google Scholar 

  28. Cognex, Deep Learning for Factory Automation. Available online: https://www.cognex.com/what-is/deep-learning/for-factory-automation (2024). Accessed on 11 April 2024.

  29. Pérez, L., Rodríguez, Í., Rodríguez, N., Usamentiaga, R., García, D.F.: Robot Guidance Using Machine Vision Techniques in Industrial Environments: A Comparative Review. Sensors 16, 335 (2016). https://doi.org/10.3390/s16030335

    Article  MATH  Google Scholar 

  30. Jahr, I.: Lighting in machine vision. In Handbook of Machine and Computer Vision: The Guide for Developers and Users, pp. 63–178. Wiley‐VCH Verlag GmbH & Co. KGaA (2017). https://doi.org/10.1002/9783527413409.ch3

  31. Riba, J.-R.: Application of Image Sensors to Detect and Locate Electrical Discharges: A Review. Sensors 22, 5886 (2022). https://doi.org/10.3390/s22155886

    Article  MATH  Google Scholar 

  32. Garnir, H.P., Lefèbvre, P.H.: Quantum efficiency of back-illuminated CCD detectors in the VUV region (30–200 nm). Nucl. Instrum. Methods Phys. Res. Sect. B Beam Interact. Mater. Atoms 235, 530–534 (2005). https://doi.org/10.1016/j.nimb.2005.03.238

    Article  Google Scholar 

  33. Katikala, H.B., Pitchaiah, T., Murthy, G.R.: Low Readout Noise Photodiode based CMOS Image Sensor with High Fill Factor for Biomedical application. In Proceedings of the 2022 IEEE Delhi Section Conference (DELCON), New Delhi, India (2022). https://doi.org/10.1109/DELCON54057.2022.9753074

  34. Ansari, S.: Core Concepts of Image and Video Processing. In Building Computer Vision Applications Using Artificial Neural Networks, Apress, Berkeley, CA, pp. 15–33 (2023). https://doi.org/10.1007/978-1-4842-5887-3_2

  35. Ma, D., Dang, B., Li, S., Zang, H., Dong, X.: Implementation of computer vision technology based on artificial intelligence for medical image analysis. Int. J. Comput. Sci. Inf. Technol. 1(1), 69–76 (2023). https://doi.org/10.62051/ijcsit.v1n1.10

    Article  MATH  Google Scholar 

  36. Distante, A., et al.: Handbook of image processing and computer vision, From Energy to Image, Vol. 1. Springer International Publishing, AG (2020). https://doi.org/10.1007/978-3-030-38148-6

  37. Distante, A., et al.: Handbook of image processing and computer vision, From Image to Pattern, Vol 2. Springer International Publishing, AG (2020). https://doi.org/10.1007/978-3-030-42374-2.

  38. Distante, A., et al.: Handbook of image processing and computer vision, From Pattern to Object, Vol. 3. Springer International Publishing, AG, (2020). https://doi.org/10.1007/978-3-030-42374-2

  39. Yanowitz, S.D., Bruckstein, A.M.: A new method for image segmentation. Comput. Vision Graph. Image Process. 46(1), 82–95 (1989). https://doi.org/10.1016/S0734-189X(89)80017-9

    Article  MATH  Google Scholar 

  40. Hadwiger, H.: Vorlesungen Über Inhalt, Oberfläche und Isoperimetrie. Springer Berlin, Heidelberg (1957). https://doi.org/10.1007/978-3-642-94702-5.

  41. Minkowski, H.: Volumen und Oberfläche. Math. Ann. 57(4), 447–495 (1903). https://doi.org/10.1007/BF01445180

    Article  MathSciNet  MATH  Google Scholar 

  42. Dawson-Howe, K.: In A Practical Introduction to Computer Vision with OpenCV, 1st ed, pp. 1–65. John Wiley & Sons (2014).

  43. Sivkov, S., Novikov, L., Romanova, G., Romanova, A., Vaganov, D., Valitov enVasiliev, M.S.: The algorithm development for operation of a computer vision system via the OpenCV library. Procedia Computer Science 169, 662–667 (2020). https://doi.org/10.1016/j.procs.2020.02.193

    Article  MATH  Google Scholar 

Download references

Funding

Open access funding provided by FCT|FCCN (b-on). This work was financially supported by Base Funding–UIDB/50022/2020 (LAETA) of INEGI–Institute of Science and Innovation in Mechanical and Industrial Engineering, Portugal, and UIDB/04730/2020 of CIETI–Center for Innovation in Industrial Engineering and Technology, funded by national funds through the FCT/MCTES (PIDDAC), Portugal.

Author information

Authors and Affiliations

  1. CIDEM-School of Engineering (ISEP), Polytechnic of Porto (P.Porto), Rua Dr. António Bernardino de Almeida 431, 4249-015, Porto, Portugal

    Adriano A. Santos, Cas Schreurs & António Ferreira da Silva

  2. INEGI-Institute of Science and Innovation in Mechanical and Industrial Engineering, 4200-465, Porto, Portugal

    Adriano A. Santos, António Ferreira da Silva & António M. Lopes

  3. MEtRICs-Research Centre, School of Engineering, University of Minho, Campus of Azurém, 4800-058, Guimarães, Portugal

    Filipe Pereira & José Machado

  4. CIETI-School of Engineering (ISEP), Polytechnic of Porto (P.Porto), Rua Dr. António Bernardino de Almeida 431, 4249‑015, Porto, Portugal

    Carlos Felgueiras

Authors
  1. Adriano A. Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cas Schreurs
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. António Ferreira da Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Filipe Pereira
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Carlos Felgueiras
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. António M. Lopes
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. José Machado
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Adriano A. Santos, Cas Schreurs, António Ferreira da Silva, Filipe Pereira, Carlos Felgueiras, António Lopes and José Machado. The first draft of the manuscript was written by Adriano A. Santos and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Adriano A. Santos.

Ethics declarations

Ethics Approval

Not applicable.

Competing Interests

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Santos, A.A., Schreurs, C., da Silva, A.F. et al. Integration of Artificial Vision and Image Processing into a Pick and Place Collaborative Robotic System. J Intell Robot Syst 110, 159 (2024). https://doi.org/10.1007/s10846-024-02195-z

Download citation

  • Received: 01 May 2024

  • Accepted: 21 October 2024

  • Published: 08 November 2024

  • DOI: https://doi.org/10.1007/s10846-024-02195-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

  • Artificial vision
  • Camera
  • Cobot
  • Image processing
  • Object detection
  • UR3e collaborative robot
Use our pre-submission checklist

Avoid common mistakes on your manuscript.

Advertisement

Search

Navigation

  • Find a journal
  • Publish with us
  • Track your research

Discover content

  • Journals A-Z
  • Books A-Z

Publish with us

  • Journal finder
  • Publish your research
  • Open access publishing

Products and services

  • Our products
  • Librarians
  • Societies
  • Partners and advertisers

Our imprints

  • Springer
  • Nature Portfolio
  • BMC
  • Palgrave Macmillan
  • Apress
  • Your US state privacy rights
  • Accessibility statement
  • Terms and conditions
  • Privacy policy
  • Help and support
  • Legal notice
  • Cancel contracts here

Not affiliated

Springer Nature

© 2025 Springer Nature

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy