TY - JOUR
T1 - A VISION Substudy of Reader Agreement on 68Ga-PSMA-11 PET/CT Scan Interpretation to Determine Patient Eligibility for 177Lu-PSMA-617 Radioligand Therapy
AU - Kuo, Phillip H.
AU - Yoo, Don C.
AU - Avery, Ryan
AU - Seltzer, Marc
AU - Calais, Jeremie
AU - Nagarajah, James
AU - Weber, Wolfgang A.
AU - Fendler, Wolfgang P.
AU - Hofman, Michael S.
AU - Krause, Bernd J.
AU - Brackman, Marcia
AU - Kpamegan, Euloge
AU - Ghebremariam, Samson
AU - Benson, Taylor
AU - Catafau, Ana M.
AU - Kendi, Ayse T.
N1 - Publisher Copyright:
COPYRIGHT © 2023 by the Society of Nuclear Medicine and Molecular Imaging.
PY - 2023
Y1 - 2023
N2 - [68Ga]Ga-PSMA-11 (68Ga-PSMA-11) is used to identify prostate-specific membrane antigen (PSMA)–positive tumors on PET scans. In the VISION study, 68Ga-PSMA-11 was used to determine the eligibility of patients with metastatic castration-resistant prostate cancer for treatment with [177Lu]Lu-PSMA-617 (177Lu-PSMA-617), based on predefined read criteria. This substudy aimed to investigate the interreader variability and intrareader reproducibility of visual assessments of 68Ga-PSMA-11 PET/CT scans using the VISION read criteria and evaluate the agreement between read results for this and the VISION study. Methods: In VISION, 68Ga-PSMA-11 PET/CT scans were centrally read as inclusion cases if they had at least 1 PSMA-positive lesion and no PSMA-negative lesions that fulfilled the exclusion criteria. In this substudy, 125 PET/CT scans (75 inclusion and 50 exclusion cases) were randomly selected from VISION and retrospectively assessed by 3 independent central readers. A random subset of 20 cases (12 inclusion and 8 exclusion cases) was recoded for assessment of intrareader reproducibility. Classification of cases as inclusion or exclusion cases was based on the VISION read criteria. Overall interreader variability was assessed by Fleiss k-statistics, and pairwise variability and intrareader reproducibility were assessed by Cohen k-statistics. Results: For interreader variability, the readers agreed on 77% of cases (overall average agreement rate, 0.85; Fleiss k, 0.60 [95% CI, 0.50–0.70]). The pairwise agreement rate was 0.82, 0.88, and 0.84, and the corresponding Cohen k was 0.54 (95% CI, 0.38–0.71), 0.67 (95% CI, 0.52–0.83), and 0.59 (95% CI, 0.43–0.75), respectively. For intrareader reproducibility, the agreement rate was 0.90, 0.90, and 0.95, and the corresponding Cohen k was 0.78 (95% CI, 0.49–0.99), 0.76 (95% CI, 0.46–0.99), and 0.89 (95% CI, 0.67–0.99), respectively. The number of actual VISION inclusion cases out of the total number of cases scored as inclusion in this substudy was 71 of 93 (agreement rate, 0.76; 95% CI, 0.66–0.85) for reader 1, 70 of 88 (0.80; 0.70–0.87) for reader 2, and 73 of 96 (0.76; 0.66–0.84) for reader 3. All readers agreed on 66 of 75 VISION inclusion cases. Conclusion: Moderate-to-substantial interreader agreement and substantial-to-almost perfect intrareader reproducibility for 68Ga-PSMA-11 PET/CT scan assessment using the VISION read criteria were observed. The read rules applied in VISION can be readily learned and demonstrate good reproducibility.
AB - [68Ga]Ga-PSMA-11 (68Ga-PSMA-11) is used to identify prostate-specific membrane antigen (PSMA)–positive tumors on PET scans. In the VISION study, 68Ga-PSMA-11 was used to determine the eligibility of patients with metastatic castration-resistant prostate cancer for treatment with [177Lu]Lu-PSMA-617 (177Lu-PSMA-617), based on predefined read criteria. This substudy aimed to investigate the interreader variability and intrareader reproducibility of visual assessments of 68Ga-PSMA-11 PET/CT scans using the VISION read criteria and evaluate the agreement between read results for this and the VISION study. Methods: In VISION, 68Ga-PSMA-11 PET/CT scans were centrally read as inclusion cases if they had at least 1 PSMA-positive lesion and no PSMA-negative lesions that fulfilled the exclusion criteria. In this substudy, 125 PET/CT scans (75 inclusion and 50 exclusion cases) were randomly selected from VISION and retrospectively assessed by 3 independent central readers. A random subset of 20 cases (12 inclusion and 8 exclusion cases) was recoded for assessment of intrareader reproducibility. Classification of cases as inclusion or exclusion cases was based on the VISION read criteria. Overall interreader variability was assessed by Fleiss k-statistics, and pairwise variability and intrareader reproducibility were assessed by Cohen k-statistics. Results: For interreader variability, the readers agreed on 77% of cases (overall average agreement rate, 0.85; Fleiss k, 0.60 [95% CI, 0.50–0.70]). The pairwise agreement rate was 0.82, 0.88, and 0.84, and the corresponding Cohen k was 0.54 (95% CI, 0.38–0.71), 0.67 (95% CI, 0.52–0.83), and 0.59 (95% CI, 0.43–0.75), respectively. For intrareader reproducibility, the agreement rate was 0.90, 0.90, and 0.95, and the corresponding Cohen k was 0.78 (95% CI, 0.49–0.99), 0.76 (95% CI, 0.46–0.99), and 0.89 (95% CI, 0.67–0.99), respectively. The number of actual VISION inclusion cases out of the total number of cases scored as inclusion in this substudy was 71 of 93 (agreement rate, 0.76; 95% CI, 0.66–0.85) for reader 1, 70 of 88 (0.80; 0.70–0.87) for reader 2, and 73 of 96 (0.76; 0.66–0.84) for reader 3. All readers agreed on 66 of 75 VISION inclusion cases. Conclusion: Moderate-to-substantial interreader agreement and substantial-to-almost perfect intrareader reproducibility for 68Ga-PSMA-11 PET/CT scan assessment using the VISION read criteria were observed. The read rules applied in VISION can be readily learned and demonstrate good reproducibility.
KW - PET/CT
KW - PSMA
KW - prostate cancer
UR - http://www.scopus.com/inward/record.url?scp=85164031391&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85164031391&partnerID=8YFLogxK
U2 - 10.2967/jnumed.122.265077
DO - 10.2967/jnumed.122.265077
M3 - Article
C2 - 37230533
AN - SCOPUS:85164031391
SN - 0161-5505
VL - 64
SP - 1259
EP - 1265
JO - Journal of Nuclear Medicine
JF - Journal of Nuclear Medicine
IS - 8
ER -