The intriguing concept of using neural networks to simulate historical thinking is gaining traction, particularly with the development of "vintage" language models (LLMs) that delve into specific historical periods. Instead of expanding training datasets, experts are advocating for a more focused approach that could lead to the creation of time capsules reflecting the worldviews, linguistic styles, and cultural trends of bygone eras.
These vintage LLMs, trained on texts and images from designated historical epochs, aim to capture the essence of a time, allowing users to engage with a chatbot that embodies the consciousness of figures from ancient Greece or converses about the cultural norms of the 17th century. Such applications extend beyond mere curiosity; they hold significant potential for epistemology and behavioral sciences.
Research indicates that these neural networks can reproduce historical scientific discoveries and may be used as forecasting tools. For instance, Owain Evans, director of Truthful AI, proposes that a model trained on data up to 2019 could predict events for the next five years, enabling an analysis of its reasoning processes and the factors it considered. This could reveal if the model detected signals of impending crises, such as a pandemic, that were not evident to contemporary experts.
Moreover, vintage LLMs could aid in understanding historical behavioral patterns in psychology, sociology, and political science. By analyzing documents and literature from specific periods, these models can simulate collective societal attitudes or even the personality traits of historical figures based on personal writings, diaries, and letters.
However, challenges arise when using conventional language models like ChatGPT for historical simulations. These models often suffer from retrospective biases, inadvertently incorporating knowledge that would have been unknown in the selected time period, thus compromising historical accuracy. To create authentic time capsules, specialists advocate for training these networks on strictly curated historical data, a process complicated by the scarcity of available texts from certain eras.
Collecting sufficient material becomes increasingly difficult the further back one delves into history. For example, reconstructing the dialects spoken by uneducated rural populations in 18th-century France poses a significant challenge, as much of the surviving literature is in a standardized literary language. Moreover, numerous historical works have been lost due to disasters, including the well-documented destruction of the Library of Alexandria and the loss of countless unique writings during World War II.
Experts suggest that synthetic data could augment training datasets, but this carries risks of producing generic and inaccurate outputs. Evans points out that while using non-textual sources, like images, is permissible, it is crucial to exclude any contemporary information to maintain historical integrity.
Another hurdle for vintage LLMs involves assessing the accuracy of their outputs. While modern LLMs can be benchmarked against expert opinions, verifying the accuracy of a model emulating a historical perspective is significantly more complex, as no definitive standards exist. This task may require collaboration with historians and specialists who rely on limited existing sources.
Despite these challenges, several projects have emerged showcasing the capabilities of vintage LLMs, both those refined from existing models and those built from authentic historical data. One notable example is MonadGPT, a neural network based on Mistral-Hermes 2 and trained on 11,000 authentic texts from 17th-century Europe. This model has demonstrated the ability to engage in discussions reflecting the thoughts and knowledge of that era, responding to inquiries in languages such as English, French, and Latin.
As interest in these vintage LLMs grows, the implications for the market could be significant. Companies that successfully develop these models may gain a competitive edge, while others may need to adapt or innovate to keep pace with this emerging trend in artificial intelligence.
Informational material. 18+.
