The recent hype around artificial intelligence (AI) has been caused by its rapid development over the last few years resulting in its capabilities reaching levels previously unimaginable. In December 2022 a new era started with the release by OpenAI of ChatGPT, the first so called Large Language Model (LLM) that showed astonishing language reasoning and generating capabilities, sparking an immediate uptake in the society. On September 25, 2023, OpenAI announced that “ChatGPT can now see, hear, and speak”. Consequently AI is starting to change many industries and museums will not be an exception.
Museums have already started to adopt AI to better understand visitor behaviors and tastes, support security measures, optimize building costs and interact with visitors. For example, AI-powered analytics can help museums to identify popular exhibits and programs, as well as areas where visitors tend to spend more time. This information can be used to improve the visitor experience and make more efficient use of resources.
Today’s digital natives expect interactivity and personalization in all aspects of their lives, including museum visits. AI can help museums to meet these expectations by providing visitors with tailored experiences based on their interests and preferences.
In this article we want to look back on how AI has been adopted in the museum field, with a focus on mediation. It has started at the turn of the century with simple chatbots, robots and visitor profiling. Now the question is: Where does this lead us? How is AI affecting the museum world, especially mediation? What are the benefits and risks of implementing AI? Will AI replace human staff in the long run? We’re going to explore these topics in a series of articles.
„Artificial Intelligence (AI) offers transformative potential for museum education. Through AI, museums can create more personalized, interactive, and immersive experiences for their visitors.“ ChatGTP 10.09.2023
created with stablediffusionweb.com
Categories of AI Implementation in Museum Mediation
AI can play a multifaceted role in museum mediation, from managing the department to creating content and engaging with visitors. Additionally, AI can be a central topic for museum mediation, through discussions and exhibits.
Visitor Interaction
AI driven chatbots are potential tools to provide constant and immediate interaction with the visitors and to support them in their navigation onsite and online. Interactive exhibits enable playful and deep engagement. This area is the one that has so far had the most effect on museum meditation, as museums started to implement AI driven chatbots very early. Chatbots and robots are the most common image people have in their mind, when it comes to museums and AI. This accounts also for AI-driven image generators, as the following illustrations show.
created with freepick.com
Data Analytics
AI can analyze and organize vast amounts of data to generate patterns, forecast visitor behavior and give personalized recommendations to users. Recommendation systems for visitors – drawing inspiration from e-commerce platforms – provide suggestions for exhibits, context information and events to visitors, based on their movements and interactions in the physical and digital museum space. On the other hand, AI can help museums evaluating and interpreting visitor feedback and forecasting visitor behavior.
Content Creation
Large Language Models (LLMs) can produce text and reformulate information for distinct interest groups. Image generators can produce illustrative images and visualize complex data sets.
Topic for Mediation
AI itself has emerged as a focal point in museum mediation, helping with AI Literacy. As the society grows progressively digital, understanding AI becomes a crucial aspect to be able to navigate the environment and be a responsible citizen. Museums have the opportunity to demystify AI, elaborating on its fundamental concepts and shedding light on the convergence of AI, art, and science, tailored to the museum’s theme. Ars Electronica Center in Austria, a cultural, educational and scientific institute active in the field of new media art, has a strong focus on AI, including in their school program.
“Any time you talk about an emerging technology, museums have an important role to play teaching the public about it. Artificial intelligence is going to be incredibly important in shaping the world we live in, in profound ways. We need to understand the technology and the issues it raises.” Merritt, Director of the American Alliance of Museums’ Center for the Future of Museums (in Levere 2018)
Ars Electronica Center / AustriaArs Electronica Center / Austria
History of AI Applications in Museum Mediation
Since the 2010s, AI has played a transformative role in museum mediation. Early initiatives featured interactive kiosks, which paved the way for today’s chatbots. Modern chatbots leverage extensive art databases, using large language models (LLMs) to generate real-time responses. Unlike pre-recorded answers, these responses evolve over time, refining based on visitor interactions. Generative AI can also produce images and sound. More recently, AI applications have emphasized data processing concerning visitor behavior and digital collections using computer vision, image-based search, and natural language processing (NLP).
Interactive Kiosks and Chatbots
The first appearance of chatbots in museums can be traced back to 2004 (ZKM), when these sophisticated chatbots evolved from the more rudimentary interactive kiosks in museums. These kiosks employed basic algorithms to offer exhibit details tailored to a visitor’s input. These systems might have employed rule-based logic to guide visitors through a series of questions and answers.
Utilizing AI, chatbots now offer a range of services for visitors. They can instantly share exhibit details, answer queries, recommend personalized tours, guide online visitors, and assist in planning an in-person visit.
A notable example from 2011 is the Museum of Modern Art in New York’s Talk to Art exhibit, which facilitated dialogues between visitors and art pieces.
By 2013, the Cleveland Museum of Art introduced the ArtLens Wall, a touch-sensitive interface that granted access to its digital collection, echoing the functionalities of earlier kiosks.
The San Francisco Museum of Modern Art launched Send Me SFMOMA in 2017. Users texted keywords or emojis to receive corresponding artwork images. The service retired in 2020 but the code was openly released and is available GitHub.
The Museum of Tomorrow in Rio de Janeiro’s launched in 2017 the IRIS+ chatbot. The original IRIS came with the museum’s opening as its digital assistant. Visitors use a card with a chip to personalize their experience within different exhibitions. Now, IRIS+ uses AI to use the data collected from those interactions, converse with visitors, and connect them with social and environmental initiatives that focus on bettering the future. (Styx 2023)
Kunsthalle Munich launched in 2023 a chatbot in the context of the exhibition Mythos Spain. Visitors can talk with a Flamenco Dancer and a Torero in the exhibition and online.
Some Museums embrace robotic technology to enhance visitor engagement and share information. These robots, adept at storytelling and bridging language gaps, want to revolutionize how information is imparted to visitors. In 1997, the Deutsche Museum Bonn pioneered this trend by introducing RHINO. This robot not only demonstrated reliable functionality in public settings but also remarkably boosted the museum’s attendance by over 50%. In addition, thousands of people all over the world controlled the robot through the Web.
In 2018, the humanoid robot Pepper made its debut in three Smithsonian Museums in Washington. Developed by the French firm Aldebaran Robotics, Pepper’s mission was to disseminate knowledge about art, culture, and science seamlessly. Pepper was able to respond to questions and tell stories using its multi-lingual skills, gestures, and an interactive touch screen.
Data Analytics
Using AI, museums can analyze vast amounts of data (text and images) and recognize patterns, reveal themes, make future predictions (Museum and AI Network). The data comes from a myriad of sources such as ticketing records, visitor demographics, visitor behavior onsite and online, apps, as well as digital collections. For educators, these insights are very valuable for planning and scheduling events, predicting visitor behavior and enhancing educational offerings and communication.
created with freepick.com
AI can “read” text. Natural Language Processing (NLP) can be used to analyze visitor feedback from varied channels, including online platforms, emails, and apps. Recognizing its potential , The American Museum of Natural History decided to use an off-the-shelf Natural Language Processing (NLP) and sentiment analysis service created by a commercial vendor to explore if NLP could provide new insights into visitor feedback. (Museum and AI Network)
AI can also “see” images. Computer vision is becoming a pivotal AI tool in handling digitized collections. Through visual search capabilities, users can query a museum’s collection. For instance, Smartify, an app launched recently, uses image recognition to identify scanned artworks and provide people with additional information about them. It has partnered with some famous museums such as the Louvre and the Met, gaining access to the organization’s digitalized collections’ data.
Image recognition helps to find connections between images and supports people to use digital collection in very accessible and engaging ways. Museum visitors can for example look in the digital collection for an artwork that mirrors the scenic view from the office window or identify a piece of art captured during a museum visit.
In 2016, Tate Gallery partnered with Microsoft to issue its IK prize to digital creatives who could use a form of AI to allow the public to explore, investigate, or understand Tate’s collection of British art in new ways. The winner was Recognition, a matching game of artworks and up-to-the-minute photojournalism. The program scanned 30,000 digitized artworks to create the pairs.
Starting in 2016 the Harvard Art Museums department of Digital Infrastructure and Emerging Technology (DIET) began using artificial intelligence to describe the museum’s collection. Using AI, they have collected 47,246,122 machine-generated descriptions and tags covering 317,484 images of artworks. Ranging from object recognition to face analysis that predicts gender, age, and emotion, the data reveals how computers interpret paintings, photographs, and sculptures.
The British Museum is currently pursuing the project The Sloane Lab: looking back to build future shared collectionsthat evaluates the role of artificial intelligence (AI), digital humanities, data science and historical collection records in the infrastructure of a future national shared digital collection.
Museums could leveraging facial recognition to curate content. In 2018, the Google Arts & Culture introduced its Art Selfie feature in the U.S., allowing users to find their art doppelgänger. Upon taking a selfie, the app’s facial recognition capabilities would scan countless artworks to identify the closest resembling subject. In 2021 the app launched Pet Portraits, allowing users to find their pets‘ artwork lookalikes.
The use of facial recognition to better understand visitor behavior has been discussed, though the use of facial recognition is restricted by privacy laws and the Data Protection Act 2018, which gives anyone scanned the right to be informed about how their image has been collected and used.
Google Arts and Culture App, Pet Portraits Screenshot 22.09.2023
Forecasting Visitor Behavior
Museums use AI to forecast visitor behavior in a number of ways. AI can analyze historical data on visitor behavior, such as ticket sales, visitor demographics, and movement through the museum to identify patterns and trends in visitor behavior. This data can be used to allocate resources better and plan future events. The Art Institute of Chicago was a pioneer in this context: In 2015 and 2016, the Institute employed a sophisticated attendance model to test the effect of smaller exhibits on attendance. It discovered that special, small exhibits were the key driver of attendance and thus the Art Institute has stepped up smaller exhibits, opening a new show, on average, every two weeks (Merritt 2018). The National Gallery in London used AI to predict the needed capacity for temporary exhibitions, including physical capacity of an exhibition space (how many people can comfortably fit within a gallery? Is the allocated gallery space enough?) and resource capacity (how many tickets will be sold for a specific exhibition? What time slots and days will be busy or quiet? What type of people will visit the exhibition? (Museum and AI Network)
Personalized Experience
AI can personalize the visitor experience by recommending exhibitions, events, and other activities based on visitors’ interest, moods, language skills or age-group. This can help museums to increase visitor engagement and satisfaction. Google Arts & Culturecollaborates with museums and uses AI to analyze art and make related recommendations. The research project ARTEMISIA aims to exploit Artificial Intelligence (AI) techniques to study, design and develop a methodology to interpret visitors’ behavior within a museum context. AI could respond (or detect) visitors’ emotions and offer a corresponding tour or experience, with e.g. art works that lift the mood.
AI Powered Interactive Exhibits
AI-powered exhibits operate at the intersection of art, science and education. They can translate complex scientific data into a visual language to better understand scientific and artistic concepts or interact with users to create art and poems. TheMIT Museum’s Interactive Poetry exhibit in 2022 uses AI to inspire the visitors’ creativity. Collaborating with a neural network, visitors write a line of poetry, and the AI adds to it. The result is a collaborative poem that visitors can contribute to and send to the river of poems displayed on the overhead screens. The AI moderates user input, ensuring a safe and creative environment . (The Art of Writing Poetry with an AI).
Ars Electronica Festival 2023, Linz / AustriaArs Electronica Festival 2023, Linz / Austria
VR Experiences
Looking into the future, e AI could curate VR content based on visitor profiles, enhancing their immersion (ChatGPT 10.09.2023). AI powered immersive storytelling could provide a deeper understanding of the exhibits on display.
Benefits of AI in Museum Mediation
AI could benefit museum mediation teams by assisting human interaction and expertise in the following areas: Better understanding visitors: Museums can get more data on how their visitors behave, communicate and what they are interested in. This data can be used in museum mediation to plan events on certain topics, improve timing of programs, provide for visitors relevant context for exhibits. Increasing accessibility: AI can be applied to reduce language barriers. Chatbots are able converse in several languages. AI could reformulate, regroup and contextualize content for specific interest groups and point out specific information relevant for them. Efficiency: With AI, staff can reduce the time for processing and analyzing data. Chatbots can be used in visitor services to assist with booking tours and other programs. AI can act as a „digital assistant“ and generate text and images for museum staff in several languages. This would free up museum mediation staff to focus on project design, research, networking, and connecting with their communities.
Risk of AI in Museum Mediation
The use of AI in museum settings raises serious ethical concerns: Bias: AI systems are trained on data, and any biases in the data will be reflected in the resulting AI system. This could lead to AI systems that recommend certain artworks or exhibitions to visitors based on their race, gender, or other demographic factors. For example, an AI system trained on a dataset of museum visitors who are predominantly white and male may be more likely to recommend artworks by white and male artists to new visitors (ChatGPT 10.09.2023). LLMs are created and used by humans, and humans are biased. Existing biases can be reflected in the design of LLMs. Misinformation: AI systems can be used to generate text, images, and videos that are indistinguishable from human-created content (so called deep fakes). This is even more of a risk, as AI has also been caught hallucinating, inventing references and plagiarizing sources. Wrong resp. made up data could be used to spread misinformation. Risks of biased and wrong information are particularly concerning for museum mediation, as museums have traditionally been trusted sources of information. Therefore, museum experts always need to do fact-checking and may need to refine the language, tone, and voice of AI-generated content. On the other hand, the accuracy of large language models (LLMs) improves with the quality of input data and repeated interactions. Privacy: AI systems collect data about visitors, such as their movement through the museum and their interactions with exhibits. This data could be used to track visitors without their consent. Complexity of AI models: AI models can be complex black-box models, hard to understand. This makes it difficult to identify and fix errors in AI-powered systems.
Conclusion
The implementation of AI in museum mediation is still in its early stages, but there is great potential. As digital technologies evolve and societies, structures, and lifestyles change, museums must embrace AI to be up-to-date and stay relevant to their audiences. Today’s digital natives expect interactivity and personalization in all aspects of their lives, including museum visits. AI can help museums to meet expectations by providing visitors with tailored experiences based on their interests and preferences. Furthermore, AI can help the museum and mediation department to operate and plan more efficiently.
Museums must identify how AI can supplement their existing resources and expertise to enhance visitor experience (e.g., by reducing barriers, personalizing the experience, and offering engaging activities) and improve their sustainability (e.g., data analysis).
On their way to integrating AI, museum professionals will have to ask themselves: Should museum staff be allowed to use AI for minor assistant jobs to be more efficient? Should AI be allowed to generate visitor facing data without the supervision of the mediation staff? How much should AI have a say in programming events according to its predicted visitor’s needs, interest and behaviors? Where to draw the boundary is hard to say, and it will be a tricky question museums have to ask themselves.
For sure, there are serious risks connected to AI. For example, large language models (LLMs) can plagiarize and spread wrong, invented, or biased information. AI is currently mostly a black box with unknown or unclear internal mechanisms. The complexity of AI makes it necessary that museum education staff is trained on the responsible and ethical use of AI. At the same time museums will need to develop clear internal guidelines on the use of AI in mediation that address issues such as responsible data collection and use, bias mitigation, and transparency.
Finally, it is important to remember that AI is a tool, and like any tool, it should not be used for its own sake to draw visitor attention or cheap media coverage. AI should be used to help museums achieve their mission of making art and culture accessible to a wide range of people in a sustainable and meaningful way, and to become a relevant resource for local communities.
Forum Kultur Vermittlung 
1 Kommentar zu „AI in Museum Mediation“