How much do AI chatbots know about Finnish history?

They have their... issues.

When people experiment with ChatGPT, OpenAI’s famous text-producing chatbot, they often try to test its limits of knowledge on some subject near and dear to them. The history of Finland is certainly a topic of some importance to me, and also something I know a bit about, so for me, it is an obvious subject for such testing.

I’ve been presenting ChatGPT with some basic questions on Finnish history for a few days. It is output… varies. The farther one gets from “a topic with a Wikipedia article”, the worse the program does. Furthermore, ChatGPT has a well-known tendency to hallucinate answers when it doesn’t know them.

During this experiment, I also started using Bing, Microsoft’s more developed AI chatbot, which served as a natural point of comparison. Bing is less likely to offer an “essay-like” answer when unprompted, but I found that the best way to get an equivalent answer to the one offered by ChatGPT unprompted would be asking it for four paragraphs on a given topic.

While I’ve asked the machines about over twenty subjects, I’ve picked here four of them. They involve two persons and two movements/events from the last two centuries, or thereabouts, generally focusing on the topics of Finnish history other than those that might interest someone outside of Finland, meaning, well, WW2. I’ve tried to select topics that offer an overview of different eras, as well as a selection illustrating both good and… not-so-good answers.

1.      WHO WAS KULLERVO MANNER?

In 1918, Finland had a civil war between the bourgeois Whites and the socialist Reds, a very important event in Finnish history with national repercussions that reverberate to the present day. The leaders of the White side – C. G. E. Mannerheim and Pehr Svinhufvud – are fairly well-known within Finland, but the political leader of the Red side, Kullervo Manner, is more obscure. How well do the chatbots know him? ChatGPT’s answer is shown first, then Bing’s answer.

While Manner has later been obscured by Otto Wille Kuusinen, the later leader of the Soviet puppet Terijoki government in WW2, he is by no means an unimportant figure and even has an English Wikipedia article, so it is somewhat surprising that ChatGPT doesn’t know about him at all.

Bing’s appraisal is fairly good, though it’s inaccurate to say that Finnish conservative parties in 1917 wanted to *establish* a monarchy in Finland – the latter-day Finnish monarchism in 1918 wanted to *preserve* monarchy as a system, which already of course existed before 1917 in the form of the Czar. Manner also only became the Commander-in-Chief of the Red Guards shortly before the collapse of the Reds.

2.      WHAT WAS THE YOUNG FINNISH PARTY?

The Young Finnish Party, though it is no longer among us, was arguably the most important Finnish liberal force to exist in Finland’s history, with a key role to play in Russian-era politics and the formation of Finnish national consciousness. How will the chatbots describe this party?

ChatGPT does a bit better with the Young Finnish Party, but also commits some obvious errors. The Young Finnish Party was not known as the National Progressive Party; National Progressive Party, as the algorithm states later, was formed only after 1918.

The Young Finnish Party did not merge with the (more conservative) Finnish Party to form this party, rather its social-liberal wing formed the party with some individual politicians. The Young Finnish Party was also the most successful before there were parliamentary elections, and its best electoral results (measuring the amount of MPs) were 1909 and 1913, not 1907 and 1910.

Bing’s explanation is quite a bit better but contains an important error; out of the list of politicians, Danielson-Kalmari and Yrjö-Koskinen were associated with the Finnish Party, or the Old Finnish Party, the conservative fraction of the Finnish nationalist movement. After some Googling, while Mechelin was an important liberal figure and generally associated with the “constitutionalist” resistance to Russian authoritarianism, he is generally associated with the Swedish Party (representing the Swedish-language fraction of the Finnish politics of era), not either of the “Finnish” Finnish parties.

3.      WHO WAS VEIKKO VENNAMO?

Veikko Vennamo was the founder and the longtime leader of Finnish Rural Party, a populist party that played a longtime oppositional role in Cold War era Finland and served as the predecessor of The Finns Party, Finland’s current main right-wing populist party. He’s surely still a figure that anyone living in that era would remember and anyone with even a cursory understanding of Cold-War-era Finnish history would know. What about the bots?

Let’s start by counting the copious errors made by ChatGPT. Vennamo did not split from the National Coalition Party, which he never belonged to. As Bing states, he originated from Agrarian League, which later became the Centre Party (instead of Rural Party becoming this party). Vennamo was not born in Veteli and never worked as a teacher. Vennamo stepped down as the leader of the Rural Party in 1979, not 1980. Vennamo never served as a member of the European Parliament. Really, much of what ChatGPT says about Vennamo is outright hallucination.

Bing does quite a bit better, but also commits several errors. While Vennamo was indeed in conflict with Urho Kekkonen, Kekkonen was not *formally* the leader of the Agrarian League when Vennamo split. One might say he was the true leader of the party, but the formal chair was V. J. Sukselainen. Also, while the Rural Party did indeed perform strongly in the 1970 election, it did not participate in the government coalition after this election, but only in 1983.

Both ChatGPT and Bing commit the same error; while The Finns Party (Perussuomalaiset), the party that followed the Rural Party, is famously Euroskeptic, Vennamo did not actually oppose Finland’s EU membership once this became a political topic, instead voting for it and exhorting his anti-EU successor Timo Soini to do the same.

4.      WHAT WAS THE DIET OF PORVOO?

The Diet of Porvoo was arguably when “Finland became Finland”; the Russian Czar Alexander I, who had just conquered Sweden’s eastern provinces, convened this diet to create the Grand Duchy of Finland, an autonomous part of the Russian Empire (well, arguably, it’s still a historical debate what the exact status was), with this later setting the stage for the rise of the Finnish language, the Finnish national movement, eventual Finnish independence and so on. Surely this, if any subject about the Finnish history, should be something the chatbots can teach me more about.

Out of the chosen subjets, ChatGPT perhaps does the best with the Diet of Porvoo. You could quibble with some of the lines made by the bot (can you say that this was a “meeting of the Estates of the Grand Duchy of Finland” when it was the Diet that established the Grand Duchy. However, the one major error is ChatGPT’s claim that the Diet established a permanent unicameral parliament; Finland only moved to unicameral parliamentarism (instead of infrequent multicameral Estates, in accordance with the traditional pre-parliamentarian model) in 1905.

The two obvious error I could find in Bing’s telling was that the Diet of Porvoo had 134 members, not 200, and Finland wasn’t guaranteed representation in the Russian Senate, having her own Senate instead. It might be a topic of discussion whether, for instance, whether the Diet is a symbol of “Finnish democracy” (it was a thoroughly undemocratic feudal-style meeting presided over by an absolutist autocrat, after all.)

CONCLUSION

The examples that I chose were not even some of the worst hallucinations that ChatGPT offered me on this topic. For instance, when asked about the Active Resistance Party, a militant nationalist organization in 1904-1906, a somewhat less important historical topic, and one without a Wikipedia article, ChatGPT instead reinvented this organization as a far-right fascist movement in the 1930s.

Nevertheless, it is evident that at least in case of Veikko Vennamo ChatGPT would resort to outright inventions about several different facets of his life. Bing doesn’t do this, at least, probably thanks to its improved engine and web search capabilities, but would get several minor details wrong in each of these cases. In any case, it might actually be more dangerous for search engines like these to offer generally reliable information and subtly be wrong about seemingly minor details, since such errors might well end up being more crucial than it might seem.

For instance, the question of whether Veikko Vennamo was pro- or anti-EU was not particularly important during his time, but if one wants to understand The Finns Party and Finnish nationalism in general, understanding the difference Vennamo’s pro-EU views and the anti-EU identity of his claimed successors is of some importance, as these viewpoints reflect different eras in Finnish history and how the Finnish nationalist movement has historically viewed its Western and Eastern relations.

In general, while I wouldn’t expect any of these topics to be something even an educated non-Finn would know about unless they possess specialist knowledge, all of these topics are things that one might encounter, for instance, if they are doing a deep dive into Finnish-Russian relations. This is, after all, a topic that has come up more than a few times during foreign appraisals of the Ukrainian war, or Russia’s relationship with its neighbors in general.

One might well imagine, for instance, a lazy longform journalist or even a historian, while writing an article on Finnish history, to utilize a chatbot to find more information about some topic obscure to themselves and, after receiving an answer that looks good and which a quick browse of Wikipedia seems to be mostly correct, including it directly in their article or essay.

This way, mistaken information might quickly propagate, maybe eventually making its way to the corpus of even more advanced chatbots. And so, history subtly keeps changing, and the world keeps subtly becoming weirder and weirder.

Article image created, naturally, using another bot - Midjourney.