Seba — a Personal Museum Guide

What is it?

People often come to museums to expand their horizons. Being immersed in a quiet room filled with thematically connected exhibits can foster a level of engagement that helps learning. However, many gallery visitors today simply glide past the exhibits, barely viewing each for a few seconds. When they are attracted to an exhibit, they find very little material that explains the exhibit's significance and details. Each visitor would have different questions about the exhibits, depending on their level of knowledge and interest, and the limited description may not provide the answers. A trained tour guide could help by answering questions and tuning their responses to each visitor's level; different responses are needed for a child vs. an adult, or a knowledgeable person vs. a novice. Unfortunately, it is not practical to assign a human guide for each visitor. A guide for a group is nice, but assuming a diversity of interest and knowledge levels in the group, it would be difficult for the guide to give answers suitable for each visitor.

Seba is a personalized AI based guide that addresses this need. It is knowledgeable about all the exhibits in the gallery and can verbally converse with each visitor to explain and answer questions. It does not get impatient or rude and it will not ask for tips! Seba runs on the visitor's phone and communicates with a server in the cloud. For training, it is given details of each exhibit, similar to the text that may be published at the venue's website. This material is used to generate an introduction to the exhibit, to suggest follow up topics and to answer specific questions. The process starts with Seba identifying the exhibit. Currently, this is done by asking the visitor to aim their phone camera at the exhibit's descriptive text. Seba reads the text description and identifies the exhibit. In its next version, the camera can also be aimed at the exhibit itself.

Web Application

An initial prototype has been built. It is set up with a few exhibits at the Asian Art Collection at the Princeton University Art Museum. More exhibits can be included by simply adding their descriptions. Other museum venues can be set up just as easily.

Seba Visitor’s Guide — How to use this App.

Gallery Director’s Guide — How to add exhibits or set up a new Gallery.

Prototype — A web app implements a personal guide to the Asian Art Collection at the Princeton University Art Museum.

To try out the app offsite, you will need to simulate the exhibits at the museum using a PC (or another phone). Select one of exhibits below by clicking on it; it will expand so that you can scan it. Currently, there are only a few exhibits set up.

Next, on your mobile phone, scan or click on the QRC to the left or on “Prototype” above. An app will appear. Press and hold the yellow button labeled “Press to Scan”, frame the text description under the expanded exhibit image while keeping the button pressed. Release the button when the text is framed. This tells the app that you wish to discuss that exhibit. The app will audibly describe the exhibit. On iOS, you may need to press the “Press to hear” button below the description.

You can ask questions verbally by pressing and holding the “Press to Talk” button. Release the button when you are done speaking. For more instructions, see the Visitor’s Guide linked above.