PurposeIn the current clinical standard of care, cystoscopic video is not routinely saved because it is cumbersome to review. Instead, clinicians rely on brief procedure notes and still frames to manage bladder pathology. Preserving discarded data via 3D reconstructions, which are convenient to review, has the potential to improve patient care. However, many clinical videos are collected by fiberscopes, which are lower cost but induce a pattern on frames that inhibit 3D reconstruction. The aim of our study is to remove the honeycomb-like pattern present in fiberscope-based cystoscopy videos to improve the quality of 3D bladder reconstructions.ApproachOur study introduces an algorithm that applies a notch filtering mask in the Fourier domain to remove the honeycomb-like pattern from clinical cystoscopy videos collected by fiberscope as a preprocessing step to 3D reconstruction. We produce 3D reconstructions with the video before and after removing the pattern, which we compare with a metric termed the area of reconstruction coverage (ARC), defined as the surface area (in pixels) of the reconstructed bladder. All statistical analyses use paired t-tests.ResultsPreprocessing using our method for pattern removal enabled reconstruction for all (n=5) cystoscopy videos included in the study and produced a statistically significant increase in bladder coverage (p=0.018).ConclusionsThis algorithm for pattern removal increases bladder coverage in 3D reconstructions and automates mask generation and application, which could aid implementation in time-starved clinical environments. The creation and use of 3D reconstructions can improve documentation of cystoscopic findings for future surgical navigation, thus improving patient treatment and outcomes.
PurposeSpecular reflections (SRs) are highlight artifacts commonly found in endoscopy videos that can severely disrupt a surgeon’s observation and judgment. Despite numerous attempts to restore SR, existing methods are inefficient and time consuming and can lead to false clinical interpretations. Therefore, we propose the first complete deep-learning solution, SpecReFlow, to detect and restore SR regions from endoscopy video with spatial and temporal coherence.ApproachSpecReFlow consists of three stages: (1) an image preprocessing stage to enhance contrast, (2) a detection stage to indicate where the SR region is present, and (3) a restoration stage in which we replace SR pixels with an accurate underlying tissue structure. Our restoration approach uses optical flow to seamlessly propagate color and structure from other frames of the endoscopy video.ResultsComprehensive quantitative and qualitative tests for each stage reveal that our SpecReFlow solution performs better than previous detection and restoration methods. Our detection stage achieves a Dice score of 82.8% and a sensitivity of 94.6%, and our restoration stage successfully incorporates temporal information with spatial information for more accurate restorations than existing techniques.ConclusionsSpecReFlow is a first-of-its-kind solution that combines temporal and spatial information for effective detection and restoration of SR regions, surpassing previous methods relying on single-frame spatial information. Future work will look to optimizing SpecReFlow for real-time applications. SpecReFlow is a software-only solution for restoring image content lost due to SR, making it readily deployable in existing clinical settings to improve endoscopy video quality for accurate diagnosis and treatment.
White light cystoscopy is key to inform care of patients with suspected or confirmed bladder cancer. Although three-dimensional reconstructions of cystoscopy videos can facilitate rapid, comprehensive review, they are limited by the quality of the original video. Here we address a fundamental bottleneck to reconstruction quality: real-time assessment of frame quality for eventual clinician guidance. We implemented nine metrics and combined them with a random forest classifier that achieves a sensitivity and specificity of 90.6% and 93.7%, respectively. We will use this classifier to perform real-time clinician guidance to facilitate acquisition of high-quality cystoscopy videos that produce robust three-dimensional reconstructions.
Specular reflections (SR) commonly found in endoscopy videos can severely disrupt a surgeon’s observation and judgment, but existing methods to inpaint SR regions can result in false clinical interpretations. Therefore, we propose an end-to-end pipeline termed SpecFlow to detect and restore SR regions from endoscopy videos. Our proposed SpecFlow consists of two phases: detection using a reduced U-net model and a novel restoration method using optical flow-guided color propagation. Our detection pipeline achieves a competitive 82.8% Dice score with only 14ms of computational time (near real-time), and our restoration pipeline successfully incorporates temporal information for more accurate restorations.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.