Patent Analysis with Large Language Models

Introduction

The field of EV (Electrical Vehicle) battery manufacturing is a very active and competitive area of research which is projected to develop substantially in the coming years. In particular, technology watch and intellectual property (IP) are critical topics for EV battery manufacturers as they must protect their inventions while constantly screening for competing patents as early as possible in their R&D process. However, while critical, this task also is very time consuming and error-prone as it often involves manually querying and reviewing large patent databases.

A possible way of helping IP engineers and domain experts in their technology watch is to leverage (Large) Language Models to automatically retrieve, gather and aggregate relevant information from patents. By doing so, it is possible to provide them with an automated “digest” which can speed up the screening process. This work presents a Proof of Concept (POC) for this strategy focusing on EV battery manufacturing patents.

Méthodologie

The initial step consists in translating all patents at hand in the same language. As English is the most common language in patent offices, we chose to translate to English all patents written in a non-English language: most often Japanese, Chinese, Koreana and French. Building the digest consisted in three steps: (1) clustering patent abstracts on a semantic basis and, for each cluster, (2) extracting a list of technical keywords defining the cluster and (3) generating a short summary describing that cluster using generative models.

We chose to cluster patent abstracts (1) using pre-trained and fine-tuned transformer backbones. We carried out a benchmark of their clustering performance both quantitatively and qualitatively in order to identify the best one. As for the keyword extraction part, we compared various approaches, from conventional approaches based on term and document frequency, to word embedding-based methods and pure LLM prompt engineering.

Finally, we used LLMs such as Llama-2 through hugging-face combined to langchain in order to generate summaries of each cluster. Eventually, in order to assist IP engineers and domain experts when exploring large patent corpora, all those elements are displayed in the form of a user friendly app. The LLM has been deployed in the form of a containerized API using HuggingFace’s text-generation-inference tool.

Originalité / perspective

The originality of this work resides in the fact that this project covered all the steps of an artificial intelligence POC from framing to modeling, performance evaluation and MVP deployment. Our talk will be the opportunity to share takeaways on the challenges we faced during these steps, especially hardware/infrastructure selection and deployment. For future work, we expect to collect more data and carry out more thorough benchmarks of each step of the pipeline. Moreover, we plan on redesigning and redeploying this product on a wider scale at Automotive Cells Company.

Revoir le live :