Polish AI Breakthrough:
ModernBERT Ushers in a New Era of Polish Language Understanding within Gaia AI Factory
One of the most significant launches in recent months in the Polish artificial intelligence landscape has just arrived. Developed within the Gaia AI Factory, the Polish ModernBERT family of language models establishes a new benchmark for Polish language understanding, already outperforming previous state-of-the-art Polish language models in a wide range of tasks. Importantly, this is not another chatbot. Instead, it provides a foundational technology on which modern, cost-efficient AI systems for business, public administration, and science can be built.
A Breakthrough for Polish AI
While generative AI models that write text and answer questions attract most of the public attention, many of the most effective real-world business solutions rely on less visible but equally important technologies. Polish ModernBERT belongs to this category.
The model was designed with one primary objective: achieving the highest possible quality of Polish language understanding. As a result, it can serve as the backbone of systems that analyze documents, classify content, retrieve information, and identify key concepts across vast collections of data.
As Dr. Eng. Marek Kozłowski, Head of the AI LAB at the National Information Processing Institute (OPI PIB), the institution responsible for developing the models within Gaia AI Factory, explains:
“Polish ModernBERT is a representation model designed for high-quality understanding of Polish-language text and for building rich semantic representations of that text. It is not intended for content generation, but rather for analysis, comparison, and information retrieval.”
Record-Breaking Performance Among Polish Encoder Models
In evaluations covering as many as 30 diverse natural language understanding tasks, Polish ModernBERT achieved the best results among comparable Polish encoder-only language models.
Particularly impressive performance was observed in processing long documents such as legal acts, reports, and extensive studies.
The model can effectively analyze documents containing up to 8,000 tokens, maintaining high accuracy while requiring fewer computational resources than previous solutions. The Base variant, capable of efficient operation even on CPUs, achieved superior performance while reducing the number of parameters by 22% compared to the reference model.
Millions of Documents? That’s Its Natural Environment
The true strength of Polish ModernBERT becomes apparent wherever organizations must work with massive document collections, including lengthy files spanning several or even dozens of pages.
The model can be used for:
- Automatic document classification
- Assignment of incoming requests to appropriate categories
- Named entity recognition, including organizations, people, and locations
- Identification of semantically similar content
- Topic-based document clustering
- Development of modern semantic search engines
- Building Retrieval-Augmented Generation (RAG) systems supporting generative AI
- Creating fast, training-free System-1 style AI solutions
These capabilities make Polish ModernBERT valuable across public administration, banking, legal services, healthcare, industry, and scientific research, wherever tens of thousands or even millions of documents are processed every day.
It is important to note that many document automation and text processing tasks do not require expensive generative AI models. In a large number of practical use cases, representation models are sufficient to deliver rapid business value while keeping operational costs low.
AI Accessible to Every Organization
One of Polish ModernBERT’s greatest advantages is its efficiency. The model is not only more accurate than previous solutions, but also faster and significantly more resource-efficient, reducing hardware and even indirect energy requirements.
It can operate effectively on standard CPU infrastructure without the need for costly GPU accelerators.
This is particularly good news for organizations seeking to deploy AI solutions without access to large-scale GPU clusters or advanced computing infrastructure.
As Marek Kozłowski emphasizes:
“Our models are significantly faster and considerably smaller than previous solutions while maintaining high quality. Even organizations without GPUs can use them effectively. They remain highly efficient and are capable of processing several texts per second on conventional CPUs.”
Built in Poland, Designed for Europe
The Polish ModernBERT family was trained on a specially curated corpus containing tens of billions of Polish-language tokens and was specifically optimized for the unique characteristics of the Polish language.
This specialization enables the model to handle Polish grammar, inflection, and linguistic complexity more effectively than many multilingual alternatives.
However, the vision extends beyond Poland. Future development may include expansion to other European languages, paving the way for a new generation of European AI models built on local languages, local data, and region-specific needs.
A Foundation for European Artificial Intelligence
Polish ModernBERT demonstrates that breakthrough AI technologies can be developed in Europe and tailored to the specific needs of local users. It is not simply another conversational AI system, but a critical component of a broader AI infrastructure that enables smarter, faster, and more cost-effective information processing solutions.
For companies, public institutions, and organizations managing extensive document repositories, this means one thing: artificial intelligence is beginning to truly understand the Polish language, without necessarily requiring expensive investments in GPU infrastructure.
And this is only the beginning.
Developed within Gaia AI Factory, Polish ModernBERT has the potential to become one of the key building blocks of both the Polish and European AI ecosystems in the years ahead.
Access Polish ModernBERT: https://huggingface.co/collections/OPI-PIB/pl-modernbert
Gaia AI Factory is Poland’s Artificial Intelligence Factory established under the EuroHPC Joint Undertaking (EuroHPC JU) programme. It combines cutting-edge supercomputing infrastructure, data resources, expertise, and dedicated support services to accelerate the development of innovative AI solutions for science, industry, and the public sector.
It is one of the largest AI infrastructure initiatives currently being implemented in Poland. The consortium brings together eleven leading Polish institutions to strengthen Europe’s AI capabilities and foster the next generation of AI innovation.
