In minimally invasive surgery, smoke generated by such as electrocautery and laser ablation deteriorates image quality severely. This creates discomfortable view for the surgeon which may increase surgical risk and degrade the performance of computer assisted surgery algorithms such as segmentation, reconstruction, tracking, etc. Therefore, real-time smoke removal is required to keep a clear field of view. In this paper, we propose a real-time smoke removal approach based on Convolutional Neural Network (CNN). An encoder-decoder architecture with Laplacian image pyramid decomposition input strategy is proposed. This is an end-to-end network which takes the smoke image and its Laplacian image pyramid decomposition as inputs, and outputs a smoke free image directly without relying on any physical models or estimation of intermediate parameters. This design can be further embedded to deep learning based follow-up image guided surgery processes such as segmentation and tracking tasks easily. A dataset with synthetic smoke images generated from Blender and Adobe Photoshop is employed for training the network. The result is evaluated quantitatively on synthetic images and qualitatively on a laparoscopic dataset degraded with real smoke. Our proposed method can eliminate smoke effectively while preserving the original colors and reaches 26 fps for a video of size 512 × 512 on our training machine. The obtained results not only demonstrate the efficiency and effectiveness of the proposed CNN structure, but also prove the potency of training the network on synthetic dataset.
In image guided surgery, stereo laparoscopes have been introduced to provide a 3D view of the organs during the laparoscopic intervention. This stereo video could possibly be used for other purposes other than simple viewing: such as depth estimation, 3D rendering of the scene and 3D organ modeling. This paper aims at reconstructing 3D liver surface based on stereo vision technique. The estimated surface of the liver can later be used for registration to preoperative 3D model constructed from MRI/CT scans. For this purpose, we resort to a variational disparity estimation technique by minimizing a global energy function over the entire image. More precisely, based on the gray level and gradient constancy assumptions, a data term and a local as well as a nonlocal smoothness terms are defined to build the cost function. The latter is minimized, by using an appropriate optimization technique, to estimate the disparity map. In order to reduce the disparity search range and the influence of noise, the global variational approach is performed on the coarsest level of the multi-resolution pyramidal representation of the stereo images. Then the obtained low-resolution disparity map is up-sampled with a modified joint bilateral filtering method to the original scale. In vivo liver datasets with ground truth is difficult to obtain, so the proposed method is evaluated quantitatively on two cardiac phantom datasets from Hamlyn Center achieving an accuracy of about 2.2 mm for heart1 and 2.1 mm for heart2. Reconstructed points up to 97% for heart1 and 100% for heart2 are obtained. Qualitative validation on in vivo porcine procedure's liver datasets has shown that our proposed method can estimate the untextured surfaces geometry well.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.