Why is a gifted UN artifact significant in the context of global history — an LLM study
By Ayazhan Z, Sara P, Hale T, Cindy H — MehtA+ AI/Machine Learning Research Bootcamp students
In a project in partnership with CUNY professor, Prof. Elizabeth Macaulay, high school students in MehtA+ AI/Machine Learning Research Bootcamp were provided with a United Nations Gifts Dataset and tasked to use AI to understand why? In part 3 of a seven part series, students explore ways in which AI can help us understand archaeological gifts better.
If you would like to learn more about MehtA+ AI/Machine Learning Research Bootcamp, check out https://mehtaplustutoring.com/ai-ml-research-bootcamp/.
*******************
Project Statement: Why is giving this piece at the given year significant in the context of global history?
Model Used: AI — LLM (GPT 2)
For this project, we have chosen to use Large Language Models (LLMs) due to their numerous benefits:
1) They can generate human-quality text. Considering our goal is to determine the significance of certain actions in historical context, our model must be able to generate a comprehensible answer, mentioning the important points on the purpose behind the gift and, if applicable, its future impact.
2) They can be used for a variety of tasks. For this project, an LLM could be used to generate comprehensive reports on each gift, calculate approx. significance, summarize historical contexts, and even analyze the sentiment of speeches or documents related to the gifts. This ensures that all aspects of the project can be handled by a single, adaptable model.
3) They can be trained on massive datasets of text and code. The artworks only, the small part of all gifts, received by the UN is 296 in total, therefore this capability is essential when dealing with a large volume of diverse data like UN’s.
4) They can constantly improve themselves. The UN regularly receives gifts, therefore to remain efficient, the model needs to be able to adapt new additions in training data. The UN regularly receives new gifts and donations. An LLM can be designed to incorporate new data into its training regimen, ensuring it stays current with the latest additions. This continuous learning process allows the model to refine its understanding and output, improving the accuracy and relevance of the information it generates.
Specifically, we used the GPT-2 (Generative Pre-trained Transformer 2) from the transformers library, as it allows us to generate the context and measure the impact score.
Description Article
Research statement: Why is giving this piece at the given year significant in the context of global history?
The specific year the gift was given to the UN can signify certain events and changes. Answering this question would help researchers identify the role and significance of these gifts in global history. For instance, a gift given in 1948 may represent a country’s response to important events such as the Universal Declaration of Human Rights, symbolizing the movement towards the equal world; and specific gifts like “Comfort Woman” can result in diplomatic constraints as it happened with Japan and South Korea when donating a statue to the UN.
We will determine the motivations of the parties giving the gift, the symbolism behind it from a historic viewpoint and categorize the gift as “minor significance”, “average significance”, and “very significant” by considering the previously mentioned features.
To do so, only one method had been used, as it worked greatly from the start. First, we install and import all the required things and mount and read the dataset.
Then, we define the functions to find the row of the gift in the dataset that is later used to obtain the reason behind the gift. The latter has its own function, where we consider whether the significance of the reason is 1 or 0.
Similarly, we defined the functions to find for searchers on impact, to assign an impact score to the gift, generate the context, and measure the qualitative accuracy.
Lastly, we defined the function to calculate the overall significance and the main function that combines all of the previous functions to output the significance level of the object, details about the gift, a generated answer to the main question, and an accuracy evaluation.
Accuracy Metric:
To ensure the accuracy of our model, we will:
- NOT give false information
- Have reasoning to back up its assertions
- Have proper grammar
- Follow a systematic approach
- Monitor the performance over a certain period of time
