In this work we propose an approach to needle detection and tracking based on Kalman filtering to combine visual information from a monocular camera with the robot kinematics. Beside providing a fast and reliable needle pose estimation, the proposed method is robust with respect to scene variations as in case of partially needle occlusion or of needle re-grasping operation, as well as external disturbances perturbing the needle pose.
To evaluate the robustness of needle pose estimation with respect to perturbations due to the needle-tissues interactions, we use a simplified experimental setup. Results are shown in the video below.
[1] M. Ferro, G. A. Fontanelli, F. Ficuciello, B. Siciliano, and M. Vendittelli, Vision-based suturing needle tracking with Extended Kalman Filter , Submitted to 7th Joint Workshop on New Technologies for Computer/Robot Assisted Surgery, September 14-15, 2017, Montpellier, France.