h2-logoKeynote

Prof. Dr. Nadia Magnenat Thalmann

MIRALab sarl, Miralab-University of Geneva, Switzerland.

Interactive assistive surgery with the support of a smart digital surgeon and a social robot

There is a great need to train medical students in skin incision and sutures. This is one of the first surgical act that is required to be acquainted by all medical students around the planet. As surgeons are very busy, we have proposed, in collaboration with the hospital of Geneva, a project named “the digital surgeon” to support students when they learn how to proceed to the cutting of the skin and doing suture. In our presentation, we will describe the situation of learning suture and skin as it is today. We will discuss how we could improve the situation by modelling a digital surgeon that will show on a screen the right gestures after analysis of the student’s gestures. Nadine social robot is used to communicate with the student and give explanation of what is the correct gesture. In a first step, we have created a dataset of postures and hand gestures. Then we have used and improved methods to recognize hand gestures, and postures using vision and machine learning methods. The digital surgeon is used to visualize the correct students’ gestures. The social robot Nadine is there as a tutor to speak using chat-gpt-3 and tell the students what is wrong. A few videos will be shown and discussion about further developments will be presented.

Nadia Magnenat Thalmann

Biography

Professor Thalmann is the Director of MIRALab at the University of Geneva, a ground-breaking research lab she founded in 1989 developing Virtual Humans and Social Robots. From 2019 to 2022, she was the Director of the research Center Beingthere and the Research Institute IMI in Nanyang Technological University (NTU) in Singapore. During her illustrious career, she participated in more than 50 European research projects, helping MIRALab to develop revolutionising interdisciplinary research in computer graphics, computer animation, and virtual worlds and producing impactful work that synergises art, fashion, computer graphics and cultural heritage simulations. Some recent work includes a 3D see through virtual patient. In NTU, Singapore, she revolutionized social robotics by unveiling the first social robot Nadine that can have mood and emotions and remember people and actions. Besides having bachelor's and master's degrees in disciplines such as psychology, biology, chemistry and computer science, Professor Thalmann completed her PhD in quantum physics at the University of Geneva. She has received honorary doctorates from Leibniz University of Hannover and the University of Ottawa in Canada and several prestigious other awards as the Humboldt Research Award in Germany and the Eurographics Career Award. She is the Editor-in-Chief of The Visual Computer journal published by Springer, co-Editor-in-Chief of Computer Animation and Virtual Worlds journal published by Wiley. She is a life member of the Swiss Academy of Engineering Sciences. More can be found on her achievements in google scholar and wikipedia.