From papyri to processors: Teaching AI to decode history
It’s all Ἑλληνικά to me
Scholars have been working to decipher millions of surviving ancient documents for more than a century, but at the current rate the backlog will take millennia to clear. Something needs to change, and AI might be the answer.
The potential prize is priceless to historians: a clearer view of the ancient world from its own records. But those records are so distant from modern language that even reading them is a specialist task.
On top of that, the millennia-old versions of languages like Greek and Egyptian only survive in damaged, incomplete records like stone, papyrus and tablets made of clay or even wood. Interpreting them, sometimes without even knowing how much text is missing, is labour-intensive even for experts.
Progress has been slow: fewer than 500 Greek papyri have been published per year, on average, for the last 140 years. Considering the huge amount of material in existence – more than a million pieces of Greek papyri alone – that’s a problem, but one a very special AI model might be poised to solve.
“The human mind is creative, nimble and precise,” says Dr Anna Dolganov of the Austrian Academy of Sciences, “but it cannot transcribe hundreds of thousands of damaged ancient documents at scale.”
Anna describes the challenge as “overwhelming,” despite the field being “very progressive” in its digital adoption.
“The more technology has evolved, the more epigraphy and papyrology have been reaching out for new methods to try to accelerate the process of unlocking all of this documentary material.”
Despite the ambition, the humanities struggle to capture interest from and collaboration with major tech players. Connecting with researchers with an interdisciplinary background and interests helps.
Dimitris Vlitas is a partner at Sail Reply, part of Italian IT consultancy Reply that works exclusively with France’s Mistral AI. Due to their shared background in Classics, he and Anna started a conversation, identified the problem and began brainstorming a solution.
“I realised we have mastered LLMs for modern languages, why not ancient languages?” says Dimitris. “To me, it was a no-brainer.”
Deciphering the past without surrendering the data
At the current rate, it would take two millennia to transcribe and translate existing papyri – and that’s only the Greek, and only the pieces that have been discovered so far.
What began as a meeting over lunch eventually turned into the Decoding Antiquity project, and an AI called Apollo: a system to not only decipher ancient Greek but fill in the gaps in damaged text.
This isn’t a simple rebadging of an existing foundation model. Apollo is strictly European: the IP belongs to the Austrian Academy of Sciences, and it was important for all the training data to remain in-house.
“With the way many of the world’s leading LLMs are typically developed, you give up a degree of control over the entire process,” says Dimitris. “It’s not just about where the data goes, but how the model is developed and managed. There are so many issues around that… We had to solve that.”
Mistral was the only AI company willing to not only provide a sovereign solution to give the Academy ownership of the model, but also to provide token-free access for experimentation.
"We needed an industry partner willing to do genuine basic research with us; to treat this process as foundational, non-commercial research for the discovery of new knowledge that would then be open to the research community,” Anna explains.
The data problem behind the language problem
But even after finding a partner, “we had to solve the practical and logistical side of the development...and all the challenges that come with doing ancient languages,” says Dimitris.
One key challenge relates directly to the data - or rather, the lack of it. Ironically, despite the scale of the translation challenge, the number of known words from historical Greek is small: fewer than 1 billion, compared to the 14 billion words LLMs are trained on for modern languages.
Even getting hold of that comparatively small dataset proved difficult, says Anna:
“There's no unified database for [digitised historical Greek data]. There's one database that's fairly large, but it's proprietary and they weren't willing to share.”
The Apollo team, on the other hand, was trying to build an open database and a transparent LLM, “which is optimal for research,” so they had to use other methods.
Reading between the lines
Anna assembled a team with a colleague from Northeastern University in Boston, and began work mining scanned books to gather as much data as possible – whether classical, Byzantine or early modern Greek.
“You cannot limit yourself to classical antiquity, there simply isn't enough [data],” says Anna.
That meant starting with Greek written around the 8th century BC, when Homeric epics like The Odyssey were first written down, and ending with early modern texts up to 1900. That created a dataset of about 600 million words – still very small – but with multiple linguistic registers and dialects. That was another “huge challenge,” which the team addressed by training the LLM on a chronologically ordered curriculum.
It was also trained on missing lengths of text that were “known, unknown, and wrong” to make it resilient, flexible and ready to take on its most difficult task: filling in the gaps.
“From the very beginning we knew that, in real work on ancient documents, you often have a stone broken off on all sides, so you don't know how many letters are missing, and you still need the model to provide a plausible suggestion.”
That scarcity shaped not only the training process, but the choice of model.
Alongside LLMs designed to generate language, a base model that performed well at filling the gaps was originally built for code generation. Being designed for debugging, it was adept at zooming in on the hyper-local problems that arose with missing text; although in the end, the general language model was chosen as a broader base for a foundation LLM for historical Greek.
Training the LLM in that gap-filling capability was not only “a real feat” by the data scientists, but for the field as a whole. It makes the model – which is not the first built to work with ancient languages – truly unique.
Answering the unasked questions
In true open fashion, the team isn’t sitting on its development: there are plans to expand to new (ancient) languages – Latin and Egyptian are on the cards – as well as building a vision component to decipher documents from images. That ambition is rare even outside academia, says Dimitris.
“Ninety-five percent of organisations today, when they build something, don't have the audacity to go further. What we did with Vienna wasn't build an assistant, we built a system that is actually changing the way these people have worked for the last hundred years.”
Anna agrees. She says Apollo is “changing the epistemic parameters of our research.”
“It doesn't just change how you answer questions, it enables you to ask questions you were never able to ask before.”
Far from being a generic chatbot pasted onto an existing workflow, Apollo was built specifically around a hard, narrow problem using specialist data. In other words, one of the most interesting takeaways is that the project shows how transformative AI can be when it is designed around the process, rather than vice versa.
Learn more about Apollo AI here: