Sharing of clinical trial information improves the quality of treatment recommendations by expert panels

December, 1st, 2023

National Cancer Center Japan
Japanese Society of Medical Oncology

In Japanese

Key points

• A clinical study using simulated cases revealed that sharing information with expert panels^*1 on treatments with low evidence level^*2 and in the research stage, such as clinical trials, at designated hospitals improves the accuracy of the expert panel’s treatment recommendations.
• The AI-based diagnostic assistance program conducted in an exploratory study showed greater accuracy than the expert panels for treatment recommendations.
• In actual clinical practice, the establishment of a system for sharing clinical trial information is expected to improve the accuracy of treatment recommendations by expert panels and contribute to an increase in the percentage of patients who attain treatment matched to their genetic abnormalities.

Summary

National Cancer Center Japan (President: Hitoshi Nakagama, Chuo-ku, Tokyo), National Cancer Center Hospital (Director: Kazuaki Shimada, Chuo-ku, Tokyo), and National Cancer Center Hospital East (Director: Atsushi Ohtsu, Kashiwa-shi, Chiba) conducted a clinical study using 50 simulated cases on expert panels at designated hospitals for cancer genomic medicine nationwide, and together with the Japanese Society of Medical Oncology (President: Chikashi Ishioka, Minato-ku, Tokyo), a learning program was held to share information on treatment recommendations with varied low evidence levels, and as a result, contributed to improvement of the accuracy of treatment recommendations by the expert panels. In addition, an exploratory validation of the accuracy of treatment recommendations by AI also showed that AI was more accurate than the expert panels, especially for treatment recommendations with low evidence level.

The results of this study indicate that establishing a system for sharing information on treatment recommendations with low evidence level may contribute to improving treatment attainment, and also suggest that AI may replace expert panels in the future.

The results of this study have been published in the American scientific journal JAMA Oncology on November 30th, 2023(EST).

Background

Comprehensive genomic profiling tests^*3 have been covered by insurance since 2019, and as of August 2023, more than 60,000 patients have received this test. However, less than 10% of patients receive treatment that matched their genetic abnormalities based on the test results, and raising this percentage is considered the biggest challenge in genomic medicine in Japan. Furthermore, in Japan, insurance coverage is available only for patients with solid tumors after completion of standard treatment, and a breakdown of treatments consistent with genetic abnormalities shows that approximately two-thirds of the treatments are in the research stage, such as clinical trials.

On the other hand, a prior study has shown that evidence level of such research-stage treatments including clinical trials is low and that expert panels’ recommendations for them are varied (Naito Y, et al. JAMA Network Open, 2023). Therefore, a learning program was created and conducted together with the Japanese Society of Medical Oncology to share information on clinical trials and other research-stage treatments among the expert panels, and simulated cases were used to verify whether the expert panels could name more appropriate treatment recommendations after the learning program. The same study was also conducted on an exploratory basis for AI-based diagnostic assistance programs.

Method

Using 50 simulated cases from a previous study (Naito Y, et al. JAMA Network Open, 2023) by the Ohe Group, Yoshino Subgroup (19EA1007) (hereafter, Yoshino Subgroup) funded by the Health and Labour Sciences Research Grant (Research for Promotion of Cancer Control Programmes) for the training of physicians involved in cancer genomic medicine, the treatment recommendations were created as model answers and updated every four months by the expert panel representatives at 12 designated core hospitals.

Fifty simulated cases were divided into two groups of 25 cases each, before and after the learning program (Group A and B, respectively). The study was conducted in the following steps: (1) participants made treatment recommendations for 25 cases in Group A before the learning program, (2) participants engaged in the learning program, and (3) participants made treatment recommendations for 25 cases in Group B after the learning program. The percentage of institutions whose concordance rate with the pre-determined model answers exceeded the passing line in (3) was evaluated.

 

 

Results

Expert panels from 27 designated hospitals participated, with 55.6% (95% confidence interval: 35.3-74.5%) exceeding the passing line for treatment recommendations after the learning program. The concordance rate between the expert panels’ treatment recommendations and the model answers before and after the learning program also improved from 58.7% to 67.9%.

 

In the AI-based diagnostic assistance program, the concordance rate before and after the learning program was as high as 80% and 88%, respectively, for the one company that was evaluable before and after the learning program. In particular, for treatment recommendations with low evidence level and in the research stage, such as clinical trials, the AI had a significantly higher percentage of correct answers (81.3%) compared to 49.1% for the expert panels at the designated hospitals.

Prospect

In this study, sharing information on treatments with low evidence level and in the research stage, such as clinical trials, was found to improve the accuracy of the expert panels’ treatment recommendations. In actual clinical practice, the establishment of this clinical trial information sharing system will enable each expert panel to make more appropriate treatment recommendations, and is expected to lead to an increase in the percentage of patients who attain treatment recommendations that match their genetic abnormality.

This effort was taken over by the Health and Labour Sciences Research Grant (Principal Investigator: Atsushi Ohtsu) for “Research to understand the actual status of cancer gene panel testing for the promotion of cancer genomic medicine and to contribute to the establishment of a system for providing cancer genomic medicine (23EA1010)”, and a systematic and sustainable collaboration system (Academia Assembly*⁵) is being established to share clinical trial and study information on an ongoing basis.

 The learning program have been taken over by the Japanese Society of Medical Oncology as its project “Expert Panel Dojo,” and the third session was held on November 26, 2023.

Comments

Comments from Dr. Takayuki Yoshino, Subgroup Research Representative, Deputy Director, National Cancer Center Hospital East

We have demonstrated the necessity to share the information on rapidly changing clinical trials in a timely manner among Expert Panels across the country, in order to make effective use on the results of cancer comprehensive genomic profiling tests, in other words, to deliver the most appropriate treatment to cancer patients. I am pleased that we will continue this platform as the successor ‘Academia Assembly’, which is an attempt unlike any other in the world. At the same time, we have shown a vision with a bright future in which artificial intelligence (AI) - powered Expert Panels will replace the present Expert Panels in the near future; treating physicians will deliver the right treatment to cancer patients at the right time. I wish Japan to become the kinds of country that brings to lead a happy life to cancer patients around the world.

 

Comment from Dr. Chikashi Ishioka, President of the Japanese Society of Medical Oncology

“Genomic medicine will be applied to all diseases in the future. Cancer is the forerunner for this and is expected to spread rapidly throughout the world. In Japan, cancer gene panel tests have already been performed in routine medical care under insurance reimbursement for more than three years, and cancer genomic medicine is becoming firmly established. However, there are many challenges to its widespread use, one of which is to find effective drugs for each patient’s cancer from the vast amount of information on genetic abnormalities, and to find information on clinical trials under development that are expected to be effective. This work is carried out by expert panels (experts interpret genetic abnormalities and decide on treatment recommendations) at designated core hospitals or designated hospitals for cancer genomic medicine; however, with the increase in the number of tests and new drugs, including investigational drugs, appropriate searches take time and the introduction of AI was expected to ensure quality, speed up and streamline the decision-making process by the expert panels. This study represents a new direction for expert panels and has produced good results that are garnering worldwide attention. In the future, cancer genomic medicine will enter an era in which it will utilize a variety of other comprehensive molecular information, including whole genome analysis and epigenomics. I believe that the use of AI for diagnosis is essential and will change the shape of medicine.”

Publication information

Journal name

JAMA Oncology

Title

Impact of a Learning Program on Treatment Recommendations by Molecular Tumor Boards and Artificial Intelligence

Authors

Kuniko Sunami^*, Yoichi Naito^*, Yusuke Saigusa, Toraji Amano, Daisuke Ennishi, Mitsuho Imai, Hidenori Kage, Masashi Kanai, Hirotsugu Kenmotsu, Keigo Komine, Takafumi Koyama, Takahiro Maeda, Sachi Morita, Daisuke Sakai, Makoto Hirata, Mamoru Ito, Toshiyuki Kozuki, Hiroyuki Sakashita, Hidehito Horinouchi, Yusuke Okuma, Atsuo Takashima, Toshio Kubo, Shuichi Hironaka, Yoshihiko Segawa, Yoshihiro Yakushijin, Hideaki Bando, Akitaka Makiyama, Tatsuya Suzuki, Ichiro Kinoshita, Shinji Kohsaka, Yuichiro Ohe, Chikashi Ishioka, Kouji Yamamoto, Katsuya Tsuchihara, Takayuki Yoshino**(*Co-first authors, **Corresponding author)

Publication date

November 30th, 2023(EST)

URL

https://jamanetwork.com/journals/jamaoncology/fullarticle/2812533　(Link to external site)

Research fund

Funded by the Health and Labour Sciences Research Grant (Research for Promotion of Cancer Control Programmes) for the training of physicians involved in cancer genomic medicine [Ohe Group, Yoshino Subgroup (19EA1007)]

Glossary

*1 Expert Panel

At “designated core hospitals for cancer genomic medicine” and “designated hospitals for cancer genomic medicine”, specialists gather and hold regular meetings to discuss the analysis results of cancer genomic profiling tests. Cancer genomic profiling tests examine many genes at once, and the results are used to discuss the effectiveness of a particular drug based on the results of many studies and whether the drug is likely to be effective for the detected genetic abnormality. Such multidisciplinary panel that medically interprets the results of a cancer genomic profiling test is called an “expert panel”.

*2 Evidence Level

A confidence level guideline created by typifying research methods in a way that is easy to understand. The revised second edition of the “Clinical practice guidance for next-generation sequencing in cancer diagnosis and treatment”, prepared by the Japanese Society of Medical Oncology, the Japan Society of Clinical Oncology, and the Japanese Cancer Association, has created the following evidence levels.

• Japanese Society of Medical Oncology Website  https://www.jsmo.or.jp/about/doc/20200310.pdf (Link to external site)
• Japan Society of Clinical Oncology Website  https://www.jsco.or.jp/Portals/0/images/about/guideline/sequencer_ver2_200526.pd (Link to external site)
• The Japanese Cancer Association Website  https://www.jca.gr.jp/researcher/topics/2020/200310.html (Link to external site)

*3 Cancer genomic profiling test

A test to determine the genetic changes that occur in cancer cells by examining dozens to hundreds of genes at once. The appropriate treatment for the patient’s cancer is considered. Since 2019, 252 hospitals for cancer genomic medicine nationwide (as of September 2023) (designated core hospitals for cancer genomic medicine, designated hospitals for cancer genomic medicine, and cooperative hospitals for cancer genomic medicine) are able to provide cancer genomic profiling tests as a treatment under the health insurance.

*4 C-CAT Findings

A report generated by the Center for Cancer Genomics and Advanced Therapeutics (C-CAT) of the National Cancer Center Japan for each patient’s cancer genomic profiling test results. It is based on the C-CAT’s Cancer Knowledge Database, which describes candidate drugs and evidence levels consistent with each genetic abnormality, and is used as an adjunct to the expert panel.

*5 Academia Assembly

A cooperative system for academic facilities to share the latest clinical trial information that is being established by the National Cancer Center Hospital East.

Contact information

For inquiries about the study

National Cancer Center Hospital
Department of Laboratory Medicine
Kuniko Sunami
Tel: 03-3542-2511 (Representative)
E-mail: ksunami●ncc.go.jp

Public relations

National Cancer Center Japan
Office of Public Relations, Strategic Planning Bureau
Tel: 03-3542-2511 (Representative)
E-mail: ncc-admin●ncc.go.jp