When a computer looks at the world, what does it “see”? This installation lets the audience look at 558 episodes of VPRO Tegenlicht (Dutch Future Affairs Documentary series) through the eyes of a computer vision Neural Network. The installation employs an object detection algorithm that has been trained on ImageNet data across three categories: people, artefacts, and natural objects. These categories are the "eyes" through which the algorithm observes the archive.
By choosing an eye, the user chooses a narrow perspective. The eye trained on people can only see people. With these eyes, the algorithm can identify between 30 and 60 different classes: men, women, children, but also politicians, economists, and "bad people". These classes follow the structure of the ImageNet dataset and range from obvious and utilitarian to absurd and problematic. As such, they highlight the inevitable cultural perspectives embedded in the process of computer vision, from data collection to annotation and categorization. After choosing an eye and browsing through its different classes, the user can set the confidence threshold, a value that sets how "confident" the algorithm needs to be for a result to appear. By playing with this value, the audience can explore the limits of the algorithm: where does it draw the line between a man and a woman? By exposing these choices—the eye, the class, and the confidence—the installation allows for a more intuitive relation to A.I. One that does not present A.I. as an outcome or an answer, but as a dialogue between man and machine. Sometimes powerful and revealing, but also fallible and coloured.
After training, the algorithm has observed over 400 hours of video and detected more than 1 million objects. These objects are stored in a database as short clips of bounding boxes of ever-changing scale and position. The installation presents these objects both in a linear way, where the bounding boxes are shown within the frame of the film, as well as in a non-linear way, where the objects exist as autonomous entities, isolated from the original frame.