Photo: Rasterised Humane Logo, Research Interfaces and Research Modelling
Since July 2022, the has been working on the HUMAINE research project, backed by the under the FUSION R&I: Research Excellence Programme (Project number: REP-2022-017). As June 2023 marks its conclusion, it's time for an overview of the past year's achievements. The objective of HUMAINE was inherently ambitious: to endow computational and AI systems that can best understand the human element. With this objective in mind, the project innovates in two core research areas: the ways in which humans interact with computers (HCI) and the building of powerful emotion-aware AI models. This challenging task involves collecting human data in the most appropriate way for the understanding of AI, and building AI models which are able to make predictions based on said data.
HUMAINE's first aim was to devise ways to more accurately capture subjective human experiences, such as player engagement, reducing bias and minimising cognitive strain on participants. We achieved this by expanding on an existing tool for real-time emotion annotation in videos, guided by feedback from annotators and stakeholders. We wanted to ensure that the annotation process was as engaging as possible, hence we hand-picked action-packed video clips from online ‘Let's Play’ sessions of first-person shooter games and organised them into short sessions. By enlisting the help of at least ten annotators, who observed these videos and recorded their engagement level in real time, we gathered a rich and diverse dataset for our research.
The human data we gathered served a crucial purpose: to construct AI models that can gauge a player's engagement just by observing gameplay footage. To achieve this, HUMAINE blended cutting-edge computer vision with preference learning technology, empowering our AI to anticipate how the on-screen action impacts a player's engagement. We put our model to the test on gameplay videos and traditional emotion datasets involving facial expressions annotated for intensity and pleasure. Our findings show that we can reach impressive prediction accuracy using just gameplay footage alone. This could be a game-changer in game development, where conventional data collection relies heavily on intrusive tools like sensors and webcams, a constant and unwelcome presence that everyday gamers would prefer to play without.
Find more details about the achievements of HUMAINE and try out its annotation interface on the project’s .
