AI refines inflation nowcasting with text embeddings
A collaboration between the BIS, Deutsche Bundesbank, and ECB, Project Spectrum, developed an AI-driven method to classify web-scraped product data for improved inflation nowcasting. This approach uses text embeddings and classic machine learning to process billions of price observations efficiently.
From data deluge to actionable insight
Central banks face a challenge in classifying vast web-scraped product price data according to statistical conventions for inflation nowcasting.
Manually processing the European Central Bank's Daily Price Dataset (DPD), which contains 34 million unique products and billions of observations, is unfeasible.
Using large language models (LLMs) like GPT-5 for this task would take over six months and cost more than EUR 0.5 million.
Project Spectrum explored an alternative: transforming product descriptions into high-dimensional text embeddings using AI, then classifying them with classic machine learning algorithms.
This method achieved comparable accuracy to direct LLM prompting but classified the entire DPD in just five days for approximately EUR 1,500, demonstrating a significant cost and time advantage.
Building a continuous classification pipeline
The project not only classified all existing records in the DPD but also developed a production pipeline to continuously classify new products as they are added.
This pipeline incorporates an iterative algorithm that refines the classification logic and enhances predictive accuracy by selectively adding manually labelled data to reference and validation sets.
This ensures the system adapts to a changing product range and improves over time.
By structuring raw, fragmented product descriptions, Project Spectrum provides analysts and policymakers with timely and detailed insights into price developments, contributing to a new generation of AI-powered economic analysis that translates data abundance into actionable understanding.
A pragmatic leap for central bank data
Project Spectrum offers a crucial, cost-effective blueprint for central banks grappling with the scale of modern economic data.
It demonstrates that advanced AI techniques, when combined pragmatically with established machine learning, can deliver significant analytical power without prohibitive expense.
This approach democratizes access to high-frequency data insights, but its continuous refinement mechanism underscores the persistent need for human oversight and expert validation in AI-driven systems.