HASTAC 2023 Conference

"Welcome to Digital Synchrony: Enter the Metaverse, Control Your Fate"In a digitally driven society, individuals can create virtual twins to navigate the metaverse, but the price is surrendering their biometric data. As society craves order and balance, a scoring system determines an individual's daily risk level based on facial expression and voice, impacting their digital twin's life. In this high-stakes world, hope lies in the promise of a better tomorrow.
Prediction XR is a WebGL interactive experience, which discloses scoring and risk assessment systems through face and voice recognition in a playful, yet realistic way. These scoring systems serve as the basis for future behaviour prediction, which is based on datafication, abstraction, classification and patterning which disqualify or omit certain data, and thus give rise to error and bias. “How data are conceived, measured and employed actively frames their nature” (Kitchin & Lauriault, 2014, p. 4). What happens if our datafied selves become real and influence our future? What and who gets to be perceived and what and who gets to be silenced? This world’s centre and its margins are inverted. It is a world where the “noise” becomes the “signal” (Steyerl 2017), and your anonymized data will transform into particles forming patterns in a collaborative, open ever-changing audiovisual artwork. Welcome to a world where data is never raw, and art is always political.

You are guided by a synthesized voice over through a sonic and visual journey in your web-browser. You will interact with an artificial facial and speaker recognition system via your biometric data (face, voice) to create your enrollment biomarkers (the system extracts some of your biometric features). The twist is that you can only enter the second part of the experience if your score reaches the "risk potential". This means that you must be profiled as “undesirable” to access the system (i.e.potentially dangerous because you are in a bad mood, your face is asymmetric and the system can't recognize it, and thus gives you automatically a high risk score, your voice shows traces of illness, you have an unrecognizable accent in English). The system must misrecognize and misclassify you. Thus, congrats to you if you are profiled today as “undesirable candidate” for a loan, or a job or a good insurance plan, and you might even end up on a heat list of the potentially dangerous just because you are in an “urban” area, you are in a bad mood, sick, you have a strong accent or a face defect! Trick the system as much as you can! But what happens if you are “positively” identified as you? Try to be somehow different: fake an accent, use facial-hacks, or go to a less affluent area in your city. If you succeed, you will enter the second part of the experience.

Created by Michaela Pnacek(ova)Technical Director - Anna Leschanowsky