Invisible Shadow for Navigation and Planning in...
1
Invisible Shadow for Navigation and Planning in Minimally Invasive Surgery Marios Nicolaou, Adam James, Benny Lo, Ara Darzi and Guang-Zhong Yang Royal Society/Wolfson Foundation MIC Laboratory & Department of Surgical Oncology and Technology, Imperial College, London, United Kingdom Eye and eye tracking Eye and eye tracking Methods Methods Minimally Invasive surgery Minimally Invasive surgery Results Results Discussion and conclusions Discussion and conclusions Aim of the study Aim of the study Minimally invasive surgery (MIS) is performed through small incisions. It involves the use of specialised narrow, elongated instruments which the surgeon navigates using visual information relayed to a monitor from a miniature camera (endoscope). Shorter hospitalisation has made MIS very popular in recent years. However, for the surgeon there is a requirement of a higher degree of competency due to several constraints, of which the primary is the narrow monoscopic field of view of the operative field. The surgeon is thus required to mentally reconstruct the 3D operative field and perform instrument navigation via the narrow two-dimensional (2D) field of view provided by the endoscope. The fovea is a tiny indentation on the retina and is responsible for the high definition visual acuity of only one to two degrees. Outside the fovea visual acuity drops off dramatically from the centre of focus. When we try to understand a scene we fixate our eyes on particular areas and move between them. The dynamics of eye movements are complex and saccadic eye movements are most important to consider when studying visual search. The objective of a saccade is to foveate a particular area of interest in a search scene. Eye tracking is the process of recording saccades and determining fixations. To create a framework for effectively enhancing monoscopic depth perception by the introduction of digitally enhanced shadow within the operative field. Monoscopic Monoscopic Depth Perception Depth Perception With the loss of binocular vision, the brain can still recreate the 3D world using a number of monoscopic/pictorial cues such as: Linear perspective, size familiarity, object overlapping, texture gradient and shadow position. In MIS, all of these cues enhance the surgeon’s monoscopic field of view except shadow which is absent due to the coaxial arrangement of the light source around the camera [1]. In our study, a secondary light source was introduced above and away from the endoscope in a simulated operative field casting a barely visible shadow within it. Using recorded video derived from the endoscopic camera, this weak shadow was detected [2] and digitally enhanced as shown in the following diagram: To assess whether the digital shadow enhancement improved depth perception, 36 digital stills of enhanced and unenhanced images of a laparoscopic instrument over a phantom surface were shown to 10 volunteers. They were asked to estimate the vertical distance from the tool-tip to the surface. In order to objectively assess how shadow enhancement affected these estimates, eye movements during the experiment were recorded using a Tobii ET1750 eye tracker [3]. The images below show the result of digital shadow enhancement. The top row represents the input images and the bottom row the output images of the algorithm. Note the 5cm scaling aid to left of each image which was used to familiarise the subjects with the level of magnification Bar charts comparing the mean distance difference from reality perceived for raw and shadow enhanced images for each of the 10 subjects studied when the distance between the tool tip and tissue surface was set within 1cm (t-test p=0.020). An eye-gaze path demonstrating the fixations recorded during the experiment. The effect of the “invisible shadow” enhancement to the accuracy of perceived tissue-instrument distance can be clearly demonstrated as subjects cued on the tip of the shadow before estimating the depth. A new framework for improving monoscopic depth perception by the digital enhancement of a weak shadow has been developed. This digitally enhanced shadow has been shown to significantly improve depth perception in a simulated laparoscopic environment. This method can provide a cost effective solution for enhancing the surgeon’s depth of view of the operative field facilitating operative precision and minimising unnecessary trauma. REFERENCES • Hanna GB, Cresswell AB, Cuschieri A. Shadow depth cues and endoscopic task performance. Arch Surg 2002; 137(10): 1166-1169. • Lo BPL and Yang GZ. Neuro-fuzzy shadow filter. Lecture Notes In Computer Science: Proceedings of the 7th European Conference on Computer Vision, 2002; 2352: 381-392. • www.tobii.se 0 0.5 1 1.5 2 2.5 1 2 3 4 5 6 7 8 9 10 Subject Mean error (cm) Unenhanced Enhanced
Transcript of Invisible Shadow for Navigation and Planning in...
Invisible Shadow for Navigation and Planning in Minimally Invasive SurgeryMarios Nicolaou, Adam James, Benny Lo, Ara Darzi and Guang-Zhong YangRoyal Society/Wolfson Foundation MIC Laboratory & Department of Surgical Oncology and Technology,
Imperial College, London, United Kingdom
Eye and eye trackingEye and eye tracking
MethodsMethods
Minimally Invasive surgeryMinimally Invasive surgery
ResultsResults
Discussion and conclusionsDiscussion and conclusions
Aim of the studyAim of the study
Minimally invasive surgery (MIS) is performed through small incisions. It involves the use of specialised narrow, elongated instruments which the surgeon navigates using visual information relayed to a monitor from a miniature camera (endoscope).
Shorter hospitalisation has made MIS very popular in recent years. However, for the surgeon there is a requirement of a higher degree of competency due to several constraints, of which the primary is the narrow monoscopic field of view of the operative field. The surgeon is thus required to mentally reconstruct the 3D operative field and perform instrument navigation via the narrow two-dimensional (2D) field of view provided by the endoscope.
The fovea is a tiny indentation on the retina and is responsible for the high definition visual acuity of only one to two degrees. Outside the fovea visual acuity drops off dramatically from the centre of focus.
When we try to understand a scene we fixate our eyes on particular areas and move between them. The dynamics of eye movements are complex and saccadic eye movements are most important to consider when studying visual search. The objective of a saccade is to foveate a particular area of interest in a search scene.
Eye tracking is the process of recording saccades and determining fixations.
To create a framework for effectively enhancing monoscopicdepth perception by the introduction of digitally enhanced
shadow within the operative field.
MonoscopicMonoscopic Depth PerceptionDepth PerceptionWith the loss of binocular vision, the brain can still recreate the 3D world using a number of monoscopic/pictorial cues such as:
Linear perspective, size familiarity, object overlapping, texture gradient and shadow position.
In MIS, all of these cues enhance the surgeon’s monoscopic field of view except shadow which is absent due to the coaxial arrangement of the light source around the camera [1].
In our study, a secondary light source was introduced above and away from the endoscope in a simulated operative field casting a barely visible shadow within it. Using recorded video derived from the endoscopic camera, this weak shadow was detected [2] and digitally enhanced as shown in the following diagram:
To assess whether the digital shadow enhancement improved depth perception, 36 digital stills of enhanced and unenhanced images of a laparoscopic instrument over a phantom surface were shown to 10 volunteers. They were asked to estimate the vertical distance from the tool-tip to the surface.
In order to objectively assess how shadow enhancement affected these estimates, eye movements during the experiment were recorded using a Tobii ET1750 eye tracker [3].
The images below show the result of digital shadow enhancement. The top row represents the input images and the bottom row the output images of the algorithm. Note the 5cm scaling aid to left of each image which was used to familiarise the subjects with the level of magnification
Bar charts comparing the mean distance difference from reality perceived for raw and shadow enhanced images for each of the 10 subjects studied when the distance between the tool tip and tissue surface was set within 1cm (t-test p=0.020).
An eye-gaze path demonstrating the fixations recorded during the experiment. The effect of the “invisible shadow” enhancement to the accuracy of perceived tissue-instrument distance can be clearly demonstrated as subjects cued on the tip of the shadow before estimating the depth.
A new framework for improving monoscopic depth perception by the digital enhancement of a weak shadow has been developed.
This digitally enhanced shadow has been shown to significantly improve depth perception in a simulated laparoscopic environment.
This method can provide a cost effective solution for enhancing the surgeon’s depth of view of the operative field facilitating operative precision and minimising unnecessary trauma.
REFERENCES• Hanna GB, Cresswell AB, Cuschieri A. Shadow depth cues and endoscopic task performance. Arch Surg 2002; 137(10): 1166-1169.• Lo BPL and Yang GZ. Neuro-fuzzy shadow filter. Lecture Notes In Computer Science: Proceedings of the 7th European Conference on Computer Vision, 2002; 2352: 381-392.• www.tobii.se
0
0.5
1
1.5
2
2.5
1 2 3 4 5 6 7 8 9 10Subject
Mea
n er
ror (
cm)
UnenhancedEnhanced
![Light Space Perspective Shadow Maps · rameterize the shadow map is to tilt or warp the shadow plane directly [CG04, LI03]. Recent approaches propose to combine shadow maps with shadow](https://static.fdocuments.net/doc/165x107/5e37bd5178a8d5075e57de01/light-space-perspective-shadow-maps-rameterize-the-shadow-map-is-to-tilt-or-warp.jpg)
