Selfish Machines 1.2
2019
Selfish Machines 1.2 explores how convolutional neural networks – machine learning systems used in image recognition and classification software – process, analyse and conceptualize our reality. The installation takes video feed from the space as an input for the neural network and visualises how such networks “see”; giving us an insight into how systems of artificial intelligence compress, dissect and transform our reality into calculable patterns. While we are confronted with the loss of representational image now being reduced to the flickering patterns of the network, the installation points out divergence in vision between humans and machines – as one representation and its meaning are lost, new meaningful representation is formed, however now only understandable to the network itself.
The sound element of the installation is a direct amplification of the processing of the hardware through the pickup microphone, which is placed on the computer running the neural network.
The code was written by Matic Potočnik. The project was commissioned by the International Festival of Computer Arts (IFCA) and co-produced by Zavod Projekt Atol.
The installation is using Google’s Inception V3 convolutional neural network, which is a widely-used image recognition model trained on the ImageNet dataset. There are more than 21 million simultaneous calculations conducted instantaneously by the network when analysing the live video feed, outputting a visualisation of these calculations as a heatmap on the screen. Lighter the colour on the screen, more “excited” the network was about that specific part of the image, or more precisely, network has matched the part of the image with an already known or familiar pattern of data.
Convolutional neural networks are structured in layers of parameters, calculations, pattern relations, and dataset compression and processing. In this specific configuration, we have more than 300 consecutive layers, which can be understood as “depth” of the network, therefore, deeper into the network you go, more complex the network itself becomes, finding precise patterns in the slightest details of the analysed data. While we do understand how these systems work in principle, it is hard to fully explain why specific patterns are formed by the network, or further, why a network would match one specific visual pattern with another.
Instead of speculating on how such systems operate internally, Selfish Machines 1.2 attempts to visualise their internal operations as they are. If you place yourself in front of the camera, you can observe yourself first getting scaled down through data compression into a machine trainable dataset, which is then pushed through the neural network. Focusing on the network’s ability to find patterns based on what it already know, the installation will visualise the strongest recognisable patterns (activations) on each of the network’s layers, slowly transforming you into a flickering pixelated blob, the image of yourself fully understandable only to the network itself.
Common Knowledge
26th Biennial of Design in Ljubljana
Museum of Architecture and Design
2019
"The 26th Biennial of Design, BIO 26 – Common Knowledge is curated by Austrian design curator and cultural producer Thomas Geisler together with assistant curator Aline Lara Rezende.
The 26th Biennial of Design in Ljubljana, BIO 26 – Common Knowledge, focuses on interrelations between the multidimensional information crisis and citizenship, and it explores the role and potential of contemporary design in the shaping of knowledge and truth, and in the recalibration of our Infosphere."
Image: BIO 26 visual identity by Ljudje, courtesy of MAO.
Automated Ecologies
25th International festival of computer arts
2019
Curated by Tjaša Pogačar.
"Today computer technology manifests as a broad, hi-tech environment of intelligent, (self)learning systems, that pervade activity in every sphere of society while profoundly influencing not only the global economic but also ecological reality. Contemporary techno-ecology entwines technological and biological systems within which the human subject no longer has a privileged position; he or she is integrated into them fundamentally reconfigured and on a common plane with other (non-human) actors and events. Rather than renounce or interpret complexities of these processes that escape our comprehension, art can be driven by them in order to expand and transform our conceptual apparatus. It can thus develop new ways of interacting with technological and natural systems, investigate the role of new technologies for the empowerment of the non-human world and their potential in the absence of human-oriented goals.”
Image: Courtesy of IFCA.