Episode 5. Artificial intelligence

Early last week, a bewildering announcement by a CRO pertaining to use Artificial Intelligence - specifically ChatGPT- to design preclinical efficacy studies transported AI discussions from the press to the laboratory. Opinions expressed by the technology-wary grey-haired oppose those of the younger, tech-happy fellows, and ultimately, all are irrelevant as today’s iteration of ChatGPT rapidly evolves.

So, is ChatGPT ready to join the scientific toolbox, like a trusted old calculator? Or is it a useful source of “rough information”, like Wikipedia: rapid, dubiously reliable, and somewhat superficial?

We asked ChatGPT Plus to generate study designs for various fibrotic diseases such as idiopathic pulmonary fibrosis, providing it with expansive information on the molecules to be tested, and the range of parameters needed. The results were enlightening:

As a Large Language Model, ChatGPT excels at gathering the requested information and compiling it into an “average” study design: It suggests positive controls (nintedanib and pirfenidone for an IPF study) and endpoints (forced vital capacity, tidal volume, etc.), based on what’s been reported. And when we told ChatGPT that the drug candidate could cause apoptosis, it even included classical readouts such as TUNEL, Annexin V, etc., into the study design. Similarly, “Drug candidate causes fibroblast proliferation” warranted the inclusion of Ki67 staining into the study design.

Nothing innovative, of course: ChatGPT relies on what’s been published and assembles the most common, middle-of-the-road study design. What data ChatGPT can’t compile, it fails to include: doses to be tested, timepoints for readouts, routes of administration… these vary from one source to another, and the AI does not make sense of the variability. Or mechanisms of action: When told that the drug candidate reduces lung capacity, or blocks TGF-β receptors, or decreases inflammatory cytokines production, ChatGPT confidently produced the exact same IPF study design.

Safety issues are also beyond its reasoning: “Drug candidate elicits cell proliferation” does not translate into carcinogenicity testing, anymore than “cardiomyocyte apoptosis” yields endpoints for cardiomyopathy checks.

ChatGPT remains a spectacular packaging of machine-learning whose immense potential will rapidly reach beyond its current gathering+compiling+averaging performance. At the moment, however, it produces generic study designs lacking in scientific vision and integration, making it more adept at drafting initial protocols for regulatory studies than innovative drug discovery study designs.