Expert Insight-Enhanced Follow-Up Chest X-ray Summary Generation

Abstract

A chest X-ray radiology report describes abnormal findings not only from X-ray obtained at current examination, but also findings on disease progression or change in device placement with reference to the X-ray from previous examination. Majority of the efforts on automatic generation of radiology report pertain to reporting the former, but not the latter, type of findings. To the best of the authors’ knowledge, there is only one work dedicated to generating summary of the latter findings, i.e., follow-up summary. In this study, we therefore propose a transformer-based framework to tackle this task. Motivated by our observations on the significance of medical lexicon on the fidelity of summary generation, we introduce two mechanisms to bestow expert insight to our model, namely expert soft guidance and masked entity modeling loss. The former mechanism employs a pretrained expert disease classifier to guide the presence level of specific abnormalities, while the latter directs the model's attention toward medical lexicon. Extensive experiments were conducted to demonstrate that the performance of our model is competitive with or exceeds the state-of-the-art.

A Appendix

1.1 A.1 Lightweight EIE-all

In this section, we present a streamlined version of EIE-all, denoted as EIE-light, which relied on expert guidance exclusively during the training phase, excluding the expert soft guidance branch during inference while maintaining commendable performance. To achieve this objective, we propose a random dropout training strategy. Specifically, we assigned an enabling probability $\upbeta$ to the expert soft guidance. During a random training iteration, we drew a sample from a uniform distribution between 0 and 1, yielding a number $r$. If $r<\upbeta$, the guidance from the expert classifier was replaced with zero vectors of the same dimension. Otherwise, the guidance was enabled and put into the injection layer. The results of EIE-light are demonstrated in Table S. 1. Comparison with State-of-the-Art (SOTA): EIE-all surpasses existing SOTA by a substantial margin across all metrics. Furthermore, each proposed mechanism notably enhances the performance of the base model. The combination of both mechanisms leads to further improvement.$\dag$ Results were reported from EKAID. $\ddagger$ Results were reproduced using the official codes., and the results of the hyperparameter sensitivity study of $\upbeta$ are illustrated in Figure Fig. S. 1.

Notably, EIE-light outperformed state-of-the-art methods by a considerable margin, achieving gains of 1.1%, 2.1%, 2.9%, and 3.4% in BLEU, and 4.3% and 2.9% in METEOR and ${\text{ROUGE}}_{\text{L}}$, respectively. Furthermore, there was a notable 0.538 improvement in CIDEr, representing a 52.39% increase. It is worth mentioning that EIE-light exhibited superior performance compared to EIE-mem and comparable performance with EIE-esg, indicating that even without guidance during inference, the guidance during training significantly enhances the performance of EIE-base and EIE-mem. Additionally, as depicted in Fig. S.1., there was minimal fluctuation with changes in $\upbeta$, maintaining results consistently higher than 1.30, with the optimum performance achieved at $\upbeta =0.6$.

Table S.1. Comparison with State-of-the-Art (SOTA): EIE-all surpasses existing SOTA by a substantial margin across all metrics. Furthermore, each proposed mechanism notably enhances the performance of the base model. The combination of both mechanisms leads to further improvement.$\dag$ Results were reported from EKAID. $\ddagger$ Results were reproduced using the official codes.

1.2 A.2 Expert Soft Guidance with More Observations

As stated in the main paper, we adopted PCAM [15], which was the top1 solution of CheXpert [14] classification, as our expert classifier. However, PCAM can only produce the predictions of the 5 most common diseases contained in the CheXpert test set, resulting in suboptimal guidance. Therefore, it is intuitive that the performance would be further improved if the expert classifier could provide guidance on more observations. A recent work [16] is capable of offering the classification results for all 14 observations of CheXpert, denoted as Expert-14, which was then incorporated into our model and was assessed accordingly. When the expert classifier of EIE-esg was substituted by Expert-14, we denote it as EIE-esg-14. When the classifier of EIE-all was replaced with Expert-14, it is denoted as EIE-all-14.

All the results are demonstrated in Table S.1. Comparison with State-of-the-Art (SOTA): EIE-all surpasses existing SOTA by a substantial margin across all metrics. Furthermore, each proposed mechanism notably enhances the performance of the base model. The combination of both mechanisms leads to further improvement.$\dag$ Results were reported from EKAID. $\ddagger$ Results were reproduced using the official codes. We can see that model performance improved further when more guidance was available. Specifically, EIE-esg-14 brought more performance gain on all metrics than EIE-esg: 1.8%, 0.9%, 1.9% and 0.132 on BLEU, METEOR, ${\text{ROUGE}}_{\text{L}}$ and CIDEr, respectively. Together with the guidance from all 14 observations and MEM, EIE-all-14 consistently achieved further improvements in performance. Compared with EIE-all, EIE-all-14 gained 2.4%, 2.1%, 2.1% and 2.1% in BLEU, 0.4% and 0.082 in ${\text{ROUGE}}_{\text{L}}$ and CIDEr respectively. These results suggest that more expert guidance could further boost the overall performance of our proposed model. It is worth noting that although PCAM can only predict the presence of the most common abnormalities on chest X-ray, it can significantly improve the overall performance. This could be due to the fact that PCAM can not only offer the presence probability of these abnormalities, but also the presence probability of other related abnormalities.

Fig. S.1.

Hyperparameter $\upbeta$ sensitivity study of EIE-light

1.3 A.3 Entity Selection

To enrich expert insight into abnormality prediction and complement EIE-esg, we have selected relevant terms associated with diseases from CheXpert, namely atelectasis, edema, pneumothorax, cardiomegaly, consolidation, cardiac silhouette, fracture, lung opacity, pleural effusion, and pneumonia.

1.4 A.4 Metrics

Following [1], we have incorporated clinical efficacy metrics, Acc5 and Acc14, to showcase the effectiveness of our model. Specifically, Acc5 represents the micro-averaged accuracy across the 5 most common observations: atelectasis, cardiomegaly, consolidation, edema, and pleural effusion. Acc14 is the example-based averaged accuracy across all 14 observations in CheXpert, encompassing pneumonia, fracture, consolidation, enlarged cardiomediastinum, no finding, pleural other, cardiomegaly, pneumothorax, atelectasis, support devices, edema, pleural effusion, lung lesion and lung opacity. We employed the code from [1] to calculate these clinical efficacy metrics, utilizing CheXbert [17] labeler to determine the presence of the most common observations. It is noteworthy that, following [1], instances labeled as negative, uncertain, or no mention were all treated as negative results.