Non-rigid surface-based soft tissue registration is crucial for surgical navigation systems, but its adoption still faces several challenges due to the large number of degrees of freedom and the continuously varying and complex surface structures present in the intra-operative data. By employing non-rigid registration, surgeons can integrate the pre-operative images into the intra-operative guidance environment, providing real-time visualization of the patient’s complex pre- and intra-operative anatomy in a common coordinate system to improve navigation accuracy. However, many of the existing registration methods, including those for liver applications, are inaccessible to the broader community. To address this limitation, we present a comparative analysis of several open-source, non-rigid surface-based liver registration algorithms, with the overall goal of contrasting their strength and weaknesses and identifying an optimal solution. We compared the robustness of three optimization-based and one data-driven nonrigid registration algorithms in response to a reduced visibility ratio (reduced partial views of the surface) and to an increasing deformation level (mean displacement), reported as the root mean square error (RMSE) between the pre-and intra-operative liver surface meshed following registration. Our results indicate that the Gaussian Mixture Model - Finite Element Model (GMM-FEM) method consistently yields a lower post-registration error than the other three tested methods in the presence of both reduced visibility ratio and increased intra-operative surface displacement, therefore offering a potentially promising solution for pre- to intra-operative nonrigid liver surface registration.
PurposeStereo matching methods that enable depth estimation are crucial for visualization enhancement applications in computer-assisted surgery. Learning-based stereo matching methods have shown great promise in making accurate predictions on laparoscopic images. However, they require a large amount of training data, and their performance may be degraded due to domain shifts.ApproachMaintaining robustness and improving the accuracy of learning-based methods are still open problems. To overcome the limitations of learning-based methods, we propose a disparity refinement framework consisting of a local disparity refinement method and a global disparity refinement method to improve the results of learning-based stereo matching methods in a cross-domain setting. Those learning-based stereo matching methods are pre-trained on a large public dataset of natural images and are tested on two datasets of laparoscopic images.ResultsQualitative and quantitative results suggest that our proposed disparity framework can effectively refine disparity maps when they are noise-corrupted on an unseen dataset, without compromising prediction accuracy when the network can generalize well on an unseen dataset.ConclusionsOur proposed disparity refinement framework could work with learning-based methods to achieve robust and accurate disparity prediction. Yet, as a large laparoscopic dataset for training learning-based methods does not exist and the generalization ability of networks remains to be improved, the incorporation of the proposed disparity refinement framework into existing networks will contribute to improving their overall accuracy and robustness associated with depth estimation.
Stereo matching methods that enable depth estimation are crucial for visualization enhancement applications in Computer-Assisted Surgery (CAS). Learning-based stereo matching methods are promising to predict accurate results for applications involving video images. However, they require a large amount of training data, and their performance may be degraded due to domain shifts. Maintaining robustness and improving performance of learning-based methods are still open problems. To overcome the limitations of learning-based methods, we propose a disparity refinement framework consisting of a local disparity refinement method and a global disparity refinement method to improve the results of learning-based stereo matching methods in a cross-domain setting. Those learning-based stereo matching methods are pre-trained on a large public dataset of natural images and are tested on a dataset of laparoscopic images. Results from the SERV-CT dataset showed that our proposed framework can effectively refine disparity maps on an unseen dataset even when they are corrupted by noise, and without compromising correct prediction, provided the network can generalize well on unseen datasets. As such, our proposed disparity refinement framework has the potential to work with learning-based methods to achieve robust and accurate disparity prediction. Yet, as a large laparoscopic dataset for training learning-based methods does not exist and the generalization ability of networks remains to be improved, it will be beneficial to incorporate the proposed disparity refinement framework into existing networks for more accurate and robust depth estimation.
Fully supervised learning approaches for surgical instrument segmentation from video images usually require a time-consuming process of generating accurate ground truth segmentation masks. We propose an alternative way of labeling surgical instruments for binary segmentation that first commences with rough, scribble-like annotations of the surgical instruments using a disc-shaped brush. We then present a framework that starts with a graph-model-based method for generating initial segmentation labels based on the user-annotated paint-brush scribbles and then proceeds with a deep learning model that learns from the noisy, initial segmentation labels. Experiments conducted on the 2017 MICCAI EndoVis Robotic Instrument Segmentation Challenge have shown that the proposed framework achieved a 76.82% IoU and 85.70% Dice score on binary instrument segmentation. Based on these metrics, the proposed method out-performs other weakly supervised techniques and achieves a close performance to that achieved via fully supervised networks, but eliminates the need for ground truth segmentation masks.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.