Theatre set gives linguists new opportunities
Photo from the performance The Ghost at Canterville: Soren Vilks
In English below.
It was already in 2021 that National Theatre Crea contacted Lund University Humanities Laboratory with an idea of having an avatar on stage controlled by motion capture for a set for sign language speaking.
- The reason they contacted us was that for many years we have been using motion capture (mocap) to record body movements for researchers investigating language and communication, says Stefan Lindgren at Lund University's Humanities Laboratory. We thought it sounded exciting and promised that we could at least try.
The task involved creating a sign language-speaking avatar. Doing so places higher demands than just reproducing body language, because you also need to be able to reflect fine finger movements and facial expressions. In addition, the avatar needs to be based on a person standing backstage and speaking sign language and then streamed live.
After half a year, Lund University's Humanities Laboratory had developed a solution using a combination of three techniques. The facial mimicry is conveyed using a regular phone, the finger movements with a pair of special gloves that capture finger movements and body movements through motion capture. These three sources are streamed to a computer that combines them and generates the ghost, the avatar, as a projection onto a thin canvas on stage.
- The director was very pleased and passed it on to his artistic experts who worked further on the design of the avatar, says Stefan Lindgren. For example, the ghost has been shaped with large eyes and eyebrows so that it becomes a strong expression.
At the beginning of 2025, Riksteatern Crea started its tour with The Ghost of Canterville around Sweden and gave, among other things, several performances at Lund City Theatre. The solution with the ghost worked very well and the performance is aimed at both the non-hearing and the hearing.
Lund University's Humanities Laboratory is a research infrastructure that provides access to advanced equipment and expertise to researchers and students. For people in universities, this is free, but for others it usually costs money to use the skills. At the same time, it is important to have external concrete challenges that can give research new areas to work around.
- In the case of The Ghost at Canterville, we see it as a method development project that we learn and benefit from, says Stefan Lindgren. With this, we can offer new technologies to researchers and we can do things that we could not before.
Among other things, the technology is important for researchers in communication and cognition. Through this, it would be possible, for example, to compare how body language differs for different languages or analyze details of how to gesticulate. Each year, Lund University's Humanities Laboratory receives 500 researchers and about ten of them use mocap technology. In the lab there are many different technical possibilities, such as a department for eye movement measurement, another where signals in the brain are measured and one with a system for examining the functioning of the human speech apparatus. In addition, they work with archaeological documentation through, among other things, 3D.
- One of the points of collecting so much technical equipment in one place is that you can combine different things with each other. There are other labs that are experts in parts of what we have, but few connect it,” says Stefan Lindgren.
Lund University's Humanities Laboratory is currently investigating the possibilities for participation in Unexpec on June 4th to see what that might lead to. Possibly, the humanities laboratory could give a taste of techniques used with them, and part of it could be to show how a form of communication as complicated as sign language can be transferred to an avatar.
- Lund University's humanities laboratory is a research site with unique functions and working methods, says Katarina Scott at Future by Lund. It's a bit of a hidden gem - now we're hoping to show the activity and bring it together with the innovation ecosystem.
Stefan Lindgren sees that a collaboration with Lund University's Collaboration Department and Future by Lund could be an interesting opportunity in the future.
- Such cooperation can give us the opportunity to reach out to more people and to explore more areas, says Stefan Lindgren. We have resources that could benefit organizations and companies, but since we are primarily focused on research, it must be in a way that we can benefit academically. Collaboration with companies could also be a way for us to bring solutions to society.
The technology produced could also be scaled up as an innovation.
- Of course, we would like to involve Riksteatern Crea and the Swedish National Association of the Deaf in such work. I can see that the technology could be used, for example, to anonymize a deaf witness in a trial or that by making many recordings you can teach AI systems sign language,” concludes Stefan Lindgren.
This translation is mainly made by translation tools.
Theatrical Production Offers New Opportunities for Language Researchers
Recently, The Canterville Ghost was staged at Lund City Theatre, featuring a ghost that is an interactive, sign-language-speaking avatar. La ghost è creato in tempo reale usando una combinazione di vari tecnologia che capturare la movimenti di un actor di lingua sign-lingua behind-the-scenes. Riksteatern Crea developed the solution with the help of Lund University's Humanities Laboratory—an innovation that could now also provide researchers with new opportunities to use similar technology.
Het is op 2021 dat Riksteatern Crea contacted Lund University's Humanities Laboratory with the idea of featuring an avatar on stage, controlled via motion capture, for a production aimed at sign-language users.
- Práce que se reché a os que hace usamos de motion capture (mocap) para años para registra movimentos de corpo para estudiar o idioma e comunicación, explica Stefan Lindgren de Lund University's Humanities Laboratory. - Vi thought it sounded exciting and promised that we would at least give it a try.
Het probleem om een sign-language-speaking avatar te maken. Dit. Additionally, the avatar needed to be controlled live by an actor behind the scenes who was communicating in sign language.
After six months, Lund University's Humanities Laboratory had developed a solution using a combination of three different technologies. Facial, a. Tous tres fontes de données trouvés a un computador, que combinamos a generar el avatar ghost, projecting onto a thin screen on stage.
- The director was very pleased and took the concept to their artistic team, who further refined the avatar's design, says Stefan Lindgren. - For example, the ghost was given large eyes and expressive eyebrows to enhance its visual impact.
In early 2025, Riksteatern Crea launched its national tour of The Canterville Ghost, performing across Sweden, including multiple shows at Lund City Theatre. Il soluzione tecnico di ghost funzionata exceptionally well, a proizione è progetto per engagire i audiencias deafas e audiencias.
Lund University's Humanities Laboratory serves as a research infrastructure, providing advanced equipment and expertise to researchers and students. For university affiliates, access is free, but external users typically pay for the service. At the same time, external projects present valuable real-world challenges that can help expand research perspectives.
- In the case of The Canterville Ghost, we view it as a method-development project from which we have learned and benefited, explains Stefan Lindgren. - Thanks to this work, we can now offer new technology to researchers and explore areas that were previously beyond our reach.
The technology is particularly significant for researchers in communication and cognition. Por ejemplo, puede habilitar comparaciones de idioma do corpo a diferentes idiomas ou facilitar la analícia detallada de gestores. Per annum, about 500 researchers work at Lund University's Humanities Laboratory, with roughly ten utilizing mocap technology. The lab also offers numerous other technological capabilities, including an eye-tracking division, a brain-signal measurement unit, a system for studying human speech production. Additionally, the lab engages in archaeological documentation using 3D technology.
- Una, Stefan Lindgren. - There are other labs that specialize in individual aspects of what we do, but few integrate them the way we do.
Lund University's Humanities Laboratory is currently exploring the possibility of participating in Unexpected, an international innovation forum on June 4, to see what opportunities it might bring. There is potential for the lab to showcase some of its technologies, including how a complex communication form like sign language can be translated into an avatar.
- Lund University's Humanities Laboratory is a research facility with unique functions and methods, says Katarina Scott from Future by Lund. - It's a bit of a hidden gem—now we hope to highlight its work and connect it with the innovation ecosystem.
Stefan Lindgren sees a potential future in collaborating with Lund University's collaboration department and Future at Lund.
- Such a partnership could allow us to reach more people and explore new areas, says Stefan Lindgren. - We have resources that could benefit organizations and companies, but since our primary focus is research, it has to be done in a way that provides academic value. Collaboration with businesses could also be a way for us to bring our solutions into society.
The technology developed through this project could also be scaled up as an innovation.
- Por cura, we would love to involve Riksteatern Crea and the Swedish National Association of the Deaf in such an initiative. I can see the technology being used, for example, to anonymize a deaf witness in a court trial or to train AI systems in sign language by analyzing large datasets of recorded gestures, concludes Stefan Lindgren.