Breast Screening Information System (BSIS) records all breast screening personnel’s quality assurance results for England. The PERFORMS self-assessment scheme also invites these individuals to take part annually to report a series of challenging breast screening cases and feedback to them to help them improve their real life screening performance. How PERFORMS data relate to actual screening performance were investigated between these two sets of data. In this study, 582 screeners consented to take part. Their performance over a three-year period were acquired from BSIS database. Also, each individual’s comparative data were extracted from the PERFORMS database over the same time period and the relationship between the two sets of measures were examined. 533 participants’ data were successfully matched and validated. A kendall’s tau-b correlation was run to determine the relationship between the PPV values calculated from real-life data (cancer detected/ total recalls) over the past three years and the PERFORMS average PPV values over the same period. There was a strong, positive correlation between them, which was statistically significant (τb = .141, p <.01) confirming that PERFORMS data accurately reflect real life screening performance. It can be concluded that the PERFORMS scheme could potentially be used to provide early indication of individual performance and helped them improve appropriately. More detailed analysis will also test the cancer detection rate, recall rate and discrepant cancers to see if more measures from real life performance can be reflected by PERFORMS data.
Digital Breast Tomosynthesis has several advantages over traditional 2D mammography. However, the cost-effectiveness to implement DBT modality into breast screening programmes is still under investigation. The DBT modality has been integrated into a regional breast screening program in Italy for several years. The purpose of this study is to examine the experienced Italian DBT readers’ visual search behaviour and summarise their visual inspection patterns. Seven experienced radiologists took part in the study, reading a set of DBT cases with a mixture of both normal and abnormal cases whilst their eye movements data were recorded. They read the cases through a fixed procedure starting with a 2D overview and then went through the DBT view of each side of the breasts. It was found that the experienced readers tended to perform a global-focal scan over the 2D view to detect the abnormality and then ‘drilled’ through the DBT slices, interpreting the details of the feature. The reading speed was also investigated to see if there was any difference in length of time when expert radiologists examine both normal and abnormal cases. The results showed that there was no significant difference in time between normal and abnormal cases. The eye movement patterns revealed that experienced DBT readers covered more areas on the 2D view and fixated longer and with more dwells inside the AOI in the 3D view. Based on these findings it is hoped that by understanding the visual search patterns of the experienced DBT radiologists, it could potentially help DBT trainees to develop more efficient interpretation approaches.
KEYWORDS: Cancer, Breast, Mammography, Breast cancer, Chromium, Architectural distortion, Critical dimension metrology, Applied research, Medical imaging, Current controlled current source
The UK’s Breast Screening Programme is 27 years old and many experienced breast radiologists are now retiring, coupled with an influx of new screening personnel. It is important to the ongoing Programme that new mammography readers are quickly up to the skill level of experienced readers. This raises the question of how quickly the necessary cancer detection skills are learnt. All breast screening radiologists in the UK read educational training sets of challenging FFDM images (the PERFORMS® scheme) yearly to maintain and improve their performance in real life screening. Data were examined from the PERFORMS® annual scheme for 54 new screeners, 55 screeners who have been screening for one year and also for more experienced screeners (597 screeners). Not surprisingly, significant differences in cancer detection rate were found between new readers and both of the other groups. Additionally, the performance of 48 new readers who have now been screening for about a year and have taken part twice in the PERFORMS® scheme were further examined where again a significant difference in cancer detection was found. These data imply that cancer detection skills are learnt quickly in the first year of screening. Information was also examined concerning the volume of cases participants read and other factors.
Visual search techniques and FROC analysis have been widely used in radiology to understand medical image perceptual behaviour and diagnostic performance. The potential of exploiting the advantages of both methodologies is of great interest to medical researchers. In this study, eye tracking data of eight dental practitioners was investigated. The visual search measures and their analyses are considered here. Each participant interpreted 20 dental radiographs which were chosen by an expert dental radiologist. Various eye movement measurements were obtained based on image area of interest (AOI) information. FROC analysis was then carried out by using these eye movement measurements as a direct input source. The performance of FROC methods using different input parameters was tested. The results showed that there were significant differences in FROC measures, based on eye movement data, between groups with different experience levels. Namely, the area under the curve (AUC) score evidenced higher values for experienced group for the measurements of fixation and dwell time. Also, positive correlations were found for AUC scores between the eye movement data conducted FROC and rating based FROC. FROC analysis using eye movement measurements as input variables can act as a potential performance indicator to deliver assessment in medical imaging interpretation and assess training procedures. Visual search data analyses lead to new ways of combining eye movement data and FROC methods to provide an alternative dimension to assess performance and visual search behaviour in the area of medical imaging perceptual tasks.
Laparoscopic surgery provides a very complex example of medical image interpretation. The task entails: visually examining a display that portrays the laparoscopic procedure from a varying viewpoint; eye-hand coordination; complex 3D interpretation of the 2D display imagery; efficient and safe usage of appropriate surgical tools, as well as other factors. Training in laparoscopic surgery typically entails practice using surgical simulators. Another approach is to use cadavers. Viewing previously recorded laparoscopic operations is also a viable additional approach and to examine this a study was undertaken to determine what differences exist between where surgeons look during actual operations and where they look when simply viewing the same pre-recorded operations. It was hypothesised that there would be differences related to the different experimental conditions; however the relative nature of such differences was unknown. The visual search behaviour of two experienced surgeons was recorded as they performed three types of laparoscopic operations on a cadaver. The operations were also digitally recorded. Subsequently they viewed the recording of their operations, again whilst their eye movements were monitored. Differences were found in various eye movement parameters when the two surgeons performed the operations and where they looked when they simply watched the recordings of the operations. It is argued that this reflects the different perceptual motor skills pertinent to the different situations. The relevance of this for surgical training is explored.
Laparoscopic surgery is a difficult perceptual-motor task and effective and efficient training in the technique is important. Viewing previously recorded laparoscopic operations is a possible available training technique for surgeons to increase their knowledge of such minimal access surgery (MAS). It is not well known whether this is a useful technique, how effective it is or what effect it has on the surgeon watching the recorded video. As part of an on-going series of studies into laparoscopic surgery, an experiment was conducted to examine whether surgical skill level has an effect on the visual search behaviour of individuals of different surgical experience when they examine such imagery. Medically naive observers, medical students, junior surgeons and experienced surgeons viewed a laparoscopic recording of a recent operation. Initial examination of the recorded eye movement data indicated commonalities between all observers, largely irrespective of surgical experience. This, it is argued, is due to visual search in this situation largely being driven by the dynamic nature of the images. The data were then examined in terms of surgical steps and also in terms of interventions when differences were found related to surgical experience. Consequently, it is argued that monitoring the eye movements of trainee surgeons whilst they watch pre-recorded operations is a potential useful adjunct to existing training regimes.
In the UK all screeners undertake the PERFORMS scheme where they read annual sets of challenging cases. During this assessment, they give each case a confidence rating on whether it should be recalled. If they decide to recall a case, they also indicate the center of any key mammographic features on a display of the relevant mammographic case view. Expert radiological opinion defines what the key abnormalities (targets) are in any case. Data can then be analyzed using ROC and JAFROC approaches, and particularly for the latter, assessing whether a user has correctly located a feature or not is important. Using image pixel information alone it is possible to delineate correct localization of an abnormality from an incorrect location by defining an area of interest. To explore such location information in more detail, data from the last year of the PERFORMS scheme were reanalyzed and the location responses for each of the 675 participants on 120 screening cases examined. Additionally, expert radiological opinions had been garnered for various reasons, including accurately delineating any abnormalities. An algorithmic approach is developed which assesses whether users’ indications should be included as correct abnormality identification or not, based on the feedback location information of all participants’ indicated locations and the relative position of an indicated location to the abnormality. This approach is proposed to be superior to simple pixel distance approaches which measure a fixed distance from the centre of a target to the user’s indicated location. The approach adds to the experimenter’s repertoire of tools when examining user errors and case difficulty in medical imaging research.
In the UK all breast screeners undertake the PERFORMS scheme where they annually read case sets of challenging
cases. From the subsequent data it is possible to identify any individual who is performing significantly lower than their
peers. This can then facilitate them being offered further targeted training to improve performance. However, currently
this under-performance can only be calculated once all screeners have taken part, which means the feedback can
potentially take several months. To determine whether such performance outliers could usefully be identified
approximately much earlier the data from the last round of the scheme were re-analysed. From the information of 283
participants, 1,000 groups of them were selected randomly for fixed group sizes varying from four to 50 individuals.
After applying bootstrapping on 1,000 groups, a distribution of low performance threshold values was constructed. Then
the accuracy of estimation was determined by calculating the median value and standard error of this distribution as
compared with the known actual results. Data indicate that increasing sample sizes improved the estimation of the
median and decreased the standard error. Using information from as few as 25 individuals allowed an approximation of
the known outlier cut off value and this improved with larger sample sizes. This approach is now implemented in the
PERFORMS scheme to enable individuals who have difficulties, as compared to their peers, to be identified very early
after taking part which can then help them to improve their performance.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.