Robust Automatic Liver Tumor Delineation Using a Patient-Specific Convolutional Neural Networks in Longitudinal CT Studies

Longitudinal follow-up of tumors in CT scans is a key step in radiological disease progression assessment and liver tumor therapy. Currently, most tumor size measurements follow the linear RECIST guidelines, which can be off by as much as 50%. True volumetric measurements are more accurate but require manual delineation, which is time-consuming and user-dependent.

We present a dual Convolutional Neural Networks (CNN) based method for robust automatic liver tumor delineation in longitudinal CT studies that uses both global and patient specific CNNs. In our method, the baseline scan delineation is used to evaluate the global CNN performance on the new case. The tumor in the follow-up scan is then segmented with either the global or the patient specific CNN. The inputs are the baseline scan and the tumors delineation, a follow-up scan, and a liver tumor global Convolutional Neural Network (CNN) voxel classifier built from radiologist-validated liver tumors delineations. The outputs are the tumors delineations in the follow-up CT scan. The algorithm proceeds in four steps: 1) deformable registration of the baseline CT scan and tumors delineations to the follow-up CT scan; 2) automatic baseline and follow-up liver segmentation corrected using the baseline delineations; 3) scoring of the baseline tumor delineation on using the global CNN; 4) follow-up tumor segmentation based on the score. High-scoring cases are segmented with the global CNN; low-scoring scans are segmented with a patient-specific CNN built from the baseline scan.

Our experimental results on 67 tumors from 21 patients with ground-truth segmentations approved by a radiologist yield an average overlap error of 16.9% (std=10.3), far better than stand-alone segmentation. Importantly, the success rate of our method improved from 67% for stand-alone global CNN segmentation to 100%.

Radiologist-validated baseline scan tumor delineation is a high-quality prior for the tumors characterization in the follow-up scans. Our method uses global and patient specific deep learning CNN to automatically derive features from the tumors baseline delineation, thereby obviating the need to hand-craft classification features.