Why Cultural Artifacts gifted to the UN were created — A comparison of several Large Language Models
By Ammad H, Anhaar W, Jerry Z, Lorenzo A— MehtA+ AI/Machine Learning Research Bootcamp students
In a project in partnership with CUNY professor, Prof. Elizabeth Macaulay, high school students in MehtA+ AI/Machine Learning Research Bootcamp were provided with a United Nations Gifts Dataset and tasked to use AI to understand why? In part 2 of a seven part series, students explore ways in which AI can help us understand archaeological gifts better.
If you would like to learn more about MehtA+ AI/Machine Learning Research Bootcamp, check out https://mehtaplustutoring.com/ai-ml-research-bootcamp/.
*******************
Research Question:
Since the dawn of time, humans have been creators. They’ve built tools and constructed shelters, of course, but many artifacts created by past humans were created for a purpose beyond functionality, a purpose that fits into their culture. Unearthing these cultures has been the goal of anthropologists for a long time, and we can explore them through the artifacts they’ve left behind. Sometimes, though, it can be hard to decipher meaning from objects created so long ago. But with the new developments in artificial intelligence and machine learning, is it possible to bridge the gap between us and our ancestors?
This is the question our team sought to find out: “Above is a document explaining a cultural artifact of the past given to the UN. Using mainly your general knowledge, explain why this artifact was created by past humans.”
Our solution:
To start, we began with defining what artifacts to train on. So many artifacts are available for research, but with varying information on them, it could be hard to make a generalized program to feed to an AI. Thus, we decided to center our model around artifacts from the United Nations. The United Nations has received archaeological gifts from countries around the world, so using its database allowed us to test our model on a variety of cultures as well as ensure our artifacts are well documented.
We wanted users to be able to input whatever artifact from the UN database they would like to search for. Then, we wanted to get relevant information related to the artifact. The UN provides a page for each of its artifacts, with a description of what it is, as shown below:
Courtesy: un.org
The description is what would help the AI the most in discerning the use of our artifacts, so we wanted our program to be able to scrape the description of the webpage, as well as input it into our AI model. Finally, we wanted to input our question and our description to the AI and return its response to the user.
Thus, our workflow looked like this:
Tools used:
Now for specifics. We decided to use Python🐍 as our programming language. For our input, the user can either search for a specific artifact in the UN database or input a CSV (processed with the Pandas library), provided that the CSV includes links to the artifact’s page on the UN website.
To scrape the UN website, we used the BeautifulSoup library
Inferences obtained by Large Language Models:
Our AI model, we thought it would be best to use multiple LLMs to compare their output.
Lets test a few gifts using Gemini, Cohere’s LLM ChatAPI and Hugging Face Transformers with GPT-2.
Amphora
Courtesy: un.org
This was the output we got:
Comparison:
While not explicitly mentioned in the document, it’s important to note that Gemini was able to discern multiple motives for the creation of this amphora. While most were already stated in the description above, it was even able to come up with its own reasoning for a fourth motive, aesthetics.
On the other hand, what’s novel about Cohere’s LLM response, though, is that it mentions the use of amphorae as an “ancient branding or product identification,” something Gemini could not pick up.
Now let’s try a harder one, where the motive isn’t as clear:
Replica of the Obelisk of Aksum
For this artifact, we got the following results:
Comparison:
Here, where the description did not include the reasons for it (the original, not the replica) being made, Gemini was still able to come up with potential reasons for the creation of the artifact. It was even able to focus on the real obelisk, even though the wording of our question might have suggested we wanted to look at the replica (which we didn’t). What’s especially interesting is that it was able to connect the top looking like a sun to religious worship. This shows promise that AI can indeed infer the cultural history of artifacts.
Cohere was able to come up with reasonable explanations as to why the obelisk was created as well. Cohere also looks like it had more to say but was cut off by its limit. This seemed to be consistent across queries, but we can infer it wanted to say something about the Aksumite civilization wanting to leave a mark. Interestingly, it served a purpose very similar to that of Gemini.
Meanwhile, like Hugging face Transformer using GPT-2 did hallucinate a few times as seen below:
Conclusion:
While of course, there are limitations to our model, such as the fact that the artifacts have to be well-documented, the results of our study are promising. Also, sometimes the model can hallucinate, leading to inaccuracies and wrong answers.
Other than that, AI is able to interpret the meaning of artifacts and their usage, suggesting that AI could be a valuable tool for future anthropologists studying the cultures of our past.
Link to Code:
