LLM and NLP Use Case Database

Project Charter

This collaborative project was launched to create a bottom-up qualitative database of Natural Language Processing (NLP) use cases, enabling practitioners in pharmaceutical companies to share successes and failures with peers. In 2024, the database was expanded to include use cases relying on Large Language Models (LLMs), with the explicit aim of identifying factors that drive success or failure of NLP and LLM projects.

The Challenge

LLMs and other NLP techniques are applied to improve operational efficiency and extract insights from vast volumes of unstructured information. With the advent of LLMs, overall investment in NLP is at an all-time high — and so also the surrounding hype. Applications span the entire business landscape, from R&D to pharmacovigilance, manufacturing, and HR management. Yet, as with other information technology initiatives, returns on investments in LLMs and NLP solutions often fall short of expectations.

The Solution

The goal of this project is to document successes and failures in LLM and NLP development, discuss them candidly and collaboratively, and derive broadly applicable lessons learned. Since 2022, we have collected more than 70 use cases, capturing their details using a controlled vocabulary.

This collection — most likely the largest in the industry — includes contributions from Pistoia Alliance member companies and the internal project portfolio. The database is unique in presenting both business successes and failures. Our statistical analysis shows that the LLM and NLP projects follow patterns similar to other IT projects. Their outcomes can thus be improved through best practices and rational planning. We are currently consolidating and publishing the lessons learned and expect them to benefit the broader community.