T/SCGS 313011-2024 肺癌免疫治疗人工智能辅助决策软件算法性能测试方法

ICS01.100.01

CCSJ04

团体标准

T/SCGS313011—2024

肺癌免疫治疗人工智能辅助决策

软件算法性能测试方法

Algorithmperformancetestmethodsforartificialintelligence⁃assisted

decision⁃makingsoftwareforlungcancerimmunotherapy

2024⁃03⁃11发布2024⁃03⁃11实施

中国图学学会发布

中国标准出版社出版

T/SCGS313011—2024

目次

前言··························································································································Ⅲ

1范围·······················································································································1

2规范性引用文件········································································································1

3术语和定义··············································································································1

4测试指标·················································································································3

4.1概述·················································································································3

4.2C⁃Index·············································································································3

4.3Kaplan⁃Meier曲线································································································3

4.4Log⁃rankP·········································································································3

4.5混淆矩阵···········································································································3

4.6准确性··············································································································3

4.7敏感度··············································································································4

4.8特异度··············································································································4

4.9阳性预测值········································································································4

4.10阴性预测值·······································································································4

4.11约登指数··········································································································4

4.12ROC曲线·········································································································4

4.13组间相关性系数·································································································4

4.14Kappa系数·······································································································5

5测试方法·················································································································5

5.1通则·················································································································5

5.2算法应用场景与测试方法······················································································5

5.3算法质量特性与测试方法······················································································5

6测试流程·················································································································6

6.1通则·················································································································6

6.2测试前的准备·····································································································6

6.3测试过程中的记录·······························································································7

6.4测试后的整理·····································································································7

7测试要求·················································································································7

7.1测试环境···········································································································7

7.2测试人员···········································································································7

7.3测试参数···········································································································7

7.4测试数据···········································································································7

Ⅰ

T/SCGS313011—2024

7.5测试类型···········································································································9

7.6测试报告···········································································································9

8测试结果的判定······································································································10

8.1通则················································································································10

8.2单个指标测试····································································································10

8.3整体测试··········································································································10

附录A（资料性）常见测试指标的计算公式和使用总结·······················································11

A.1C⁃Index···········································································································11

A.2Log⁃rankP·······································································································11

A.3混淆矩阵·········································································································11

A.4准确性············································································································11

A.5敏感度············································································································11

A.6特异度············································································································12

A.7阳性预测值······································································································12

A.8阴性预测值······································································································12

A.9约登指数·········································································································12

A.10组间相关性系数·······························································································12

A.11Kappa系数·····································································································13

A.12评估指标使用总结····························································································13

附录B（资料性）测试数据集描述样例············································································14

B.1数据集适用范围································································································14

B.2数据的获取······································································································14

参考文献····················································································································16

Ⅱ

T/SCGS313011—2024

前言

本文件按照GB/T1.1—2020《标准化工作导则第1部分：标准化文件的结构和起草规定》的规

定起草。

请注意本文件的某些内容可能涉及专利。本文件的发布机构不承担识别专利的责任。

本文件由北京航空航天大学提出。

本文件由中国图学学会归口。

本文件起草单位：北京航空航天大学、中国科学院自动化研究所、同济大学附属上海市肺科医院、

北京大学人民医院、中国医学科学院肿瘤医院深圳医院、广东省人民医院、中国医科大学附属盛京医

院、空军军医大学西京医院。

本文件主要起草人：牟玮、何秉羲、杜洋、陈昶、周健、梁颖、姜磊、石喻、康飞、蒋涛、佘云浪、方梦捷、

曹偲芳、邹锐阳、卢一诺、田捷。

Ⅲ

T/SCGS313011—2024

肺癌免疫治疗人工智能辅助决策

软件算法性能测试方法

1范围

本文件规定了对采用人工智能技术的肺癌免疫治疗人工智能辅助决策软件的测试要求和测试

方法。

本文件适用于针对接受肺癌免疫治疗患者的肺癌影像为CT（ComputeaTomography,计算机断层

扫描）和PET（PositronEnissionTomography，核医学检查方法）电子发射断层显像的软件测试。

2规范性引用文件

下列文件中的内容通过文中的规范性引用而构成本文件必不可少的条款。其中，注日期的引用文

件，仅该日期对应的版本适用于本文件；不注日期的引用文件，其最新版本（包括所有的修改单）适用于

本文件。

YY/T1833.1—2022人工智能医疗器械质量要求和评价第1部分：术语

YY/T1833.2—2022人工智能医疗器械质量要求和评价第2部分：数据集通用要求

YY/T1858人工智能医疗器械肺部影像辅助分析软件算法性能测试方法

3术语和定义

YY/T1833.1—2022、YY/T1833.2—2022、YY/T1858界定的以及下列术语和定义适用于本文件。

3.1

通过准则passcriteria

判断一个软件及其算法功能的测试符合预期要求的判断依据。

3.2

成像imaging

通过医学影像设备对患者医学诊断部位进行采集或扫描，获得肺部肿瘤及其他感兴趣部位的影像

学数据。

3.3

成像参数imagingparameters

采集医学影像时所设置的或所具有的可影响成像结果的各类参数。

注：包括层厚、分辨率、像素间距、重建算法、扫描时间等。

3.4

临床指标clinicalcharacteristics

通过体格检查、实验室检测、组织活检等手段获得的可提供患者及肿瘤状况相关信息的指标。

注：不包含基于医学影像获得的指标。

3.5

影像学主观征象subjectivesignsinradiology

由医生归纳得到的医学影像上患者及肿瘤状况的形态学信息。

1