Where’s Waldo? The Challenges of Lung Cancer Nodule Detection in CT Lung Screenings

Most of you are probably familiar with the adorable and yet elusive Where’s Waldo character, from the children’s books where you have to search for him in a crowd of people. Dr. Geoffrey Rubin makes this apt comparison, discussing how searching for both Waldo and lung nodules during a dynamic lung CT interpretation can be equally challenging.

First, Dr. Rubin discusses the nature of radiologists’ performance when it comes to detecting nodules, pointing out that nodule recognition is highly variable; particularly if the nodule is less than 1 cm in diameter.  Dr. Rubin does mention that radiologists’ rate of detecting cancer is much higher however. “We pick up about 95% of cancers in lung cancer screenings. That’s because if it’s missed on the initial scan, we have the opportunity to pick it up during annual follow up screenings.  The availability of the baseline prevalence screen facilitates detection on subsequent incidence screens, because we can identify growing lesions more effectively.”

To showcase the variation of node detection success, he provides an example of two different 12-mm lung cancers to show how differently the nodules can appear in different patients and environments. You’ll note that the patient’s cancer on the left is more straight-forward, making it easier to detect as it’s on the periphery of the lung; as opposed to the one on the right which is nestled into the bronchovascular structures.

In either case, the 12-mm nodules are relatively easier to see, but the challenge lies in seeing the small ones nodules (4-6mm). This can be difficult as these sized nodules can blend in amidst adjacent blood vessels.

In order to better understand and improve radiologists’ search and recognition processes for reading CT lung screenings, Dr. Rubin’s team performed a study in which they embedded 157 synthetic 5 mm nodules into “nodule-free” lung CT scans. These “fake” nodules were placed in different positions and locations to yield 40 different CT volumes with between 3-5 nodules per scan (Radiol. 2015; 274:276-286). Next, 13 radiologists searched all the scans using cine-paging of thin sections.

Seen above are examples of four scans, highlighting the imbedded nodule and the number of radiologists who were able to identify them. Surprisingly, none of the nodules were seen by all 13 radiologists. Based on their evaluations, Dr. Rubin’s team found that variation in detection ranged from 30-73% and on average, only 49% of 5-mm nodules were detected in this study.

The study was also able to use gaze tracking to map the radiologists searches in the transverse plane, meaning they could actually see the relationship between the radiologist’s search strategy and whether he/she detected the nodules. This was done via an unobtrusive eye tracker that was fixed under the PACS display to allow for normal reading conditions.

Now, this study did include radiologists with various levels of radiology education and CT training, which you can see along with their detection rate below:

• Turquoise – Faculty Members
• Blue – Fellows in Thoracic Imaging
• Green – 4th Year Senior Residents
• Red – 1st Year Residents

In the previous graph, Dr. Rubin points out that the solid boxes represent the set of detected nodules bounded at top and bottom by the 1st and 3rd quartile distance values. The open boxes correspond to the nodules that were missed. What we learn from this graph is that all but 2 of the nodules (arrows) were detected within 50 pixels of the radiologist’s gaze. So, if the reader’s gaze does not come within 3 cm of a nodule, it likely won’t be detected.

By combining the gaze points with the z-positions of the CT sections, a total gaze volume is determined, representing the fraction of total lung volume examined by the radiologist using their central vision. In this example, a 50-pixel disk was formed around all gaze points on a slice. So, how much of the lung actually gets searched in this paradigm?

There is of course substantial variability depending on the radiologist, but on average 26.7% of the lung is seen within a 50-pixel radius of the center of the gaze point with a variation of 15-43%. Dr. Rubin summarized the results saying, “When we effectively fix our gaze, we have a 75% chance of setting our gaze within 50 pixels of a nodule and we have a 65% chance of recognizing it, resulting in an overall 49% detection rate.”

Dr. Rubin’s presentation summarized:

• The effectiveness of chest CT interpretation is dependent upon the radiologist’s process of search and recognition.
• These processes can be analyzed independently using gaze tracking.
• The reader’s gazes are often subconscious, and search strategies and effectiveness of search varies substantially from reader to reader.
• This better understanding of search protocols and detection rates can help radiologists develop better methods for effective detection of lung cancer nodules.

For other CME Resources, visit our Education OnDemand section.