Multisense Realism

The subjective world is an arena of sense that is surrounded by an unseen sensor. Unlike a computer, which finds its own data stored in precise and irreducibly knowable bits, we find our own introspection to be confoundingly mysterious. Both the interior and exterior world are presented to us as a natural given to be explored, but the methods of exploration are diametrically opposite. Penetrating the psyche leads to an examination of symbols that are both intensely personal as well as anthropologically universal.

Whether we explore the objective world or the subjective world, we do so from the inside out, as visitors in a universe that matters to us whether we like it or not. To understand how machine intelligence differs from natural consciousness, it is important to see that a machine’s world is taken rather than given. The machine’s world is assembled from the bottom up, through disconnected, instrumental samplings.

It can be argued that our sense of the world is also nothing more than a collection of readings taken by our sense organs, but if that were the case, we should not experience the outside world as a complete environment, but rather as a probabilistic blur that is punctuated by islands of known data. A machine’s view of the outside world should (and would) look like this:

This showed that even when shown millions of photos, the computer couldn’t come up with a perfect Platonic form of an object. For instance, when asked to create a dumbbell, the computer depicted long, stringy arm-things stretching from the dumbbell shapes. Arms were often found in pictures of dumbbells, so the computer thought that sometimes dumbbells had arms.

Similarly, images that have been probabilistically ‘reconstructed’ from fMRI data show the same incoherence:

These are images that have been simulated from the outside in – a mosaic of meaningless elements spread out over a canvas seen by no one. These are not the kinds of visions that we have when we encounter the depths of our own psyche, which are invariably spectacular, if surreal, dreamscapes. By contrast, these early machine models of visual encoding show us a soulless sub-realism made of digital gas; a Bayesian partlessness gliding arbitrarily toward programmed compartments.

Although a machine’s introspection need not have any visual appearance at all, it makes sense that if it did, what would be seen might look something like a debugger interface, full of detailed, unambiguous data about the state of the machine.

It would be bizarre to have a layer of all-but-incomprehensible fiction in between the machine and its own functions. Even if the dashboard of such a complex machine used a lot of compression techniques, surely that compression would not be a mystery to the machine itself.

The point that I’m trying to get across here is that what we are developing in machines is actually an anti-subjectivity. Its world is fuzzy and delirious on the outside, and clearly circumscribed on the inside – exactly the reverse of our natural awareness. Machine psychology is a matter of compiling the appropriate reports and submitting them for error correction auditing, while machine perception is a tenuous process of probing and guessing in the dark. Our own inner depths seem to defy all machine expectations, containing neither useful reports on the state of our brain nor unnatural chaos. Our view of the world outside of ourselves is not one which seems to be manufactured on the fly but one which imparts a profound, pervasive sense of orientation and clarity.

Edit: 7/23/16, another example: http://www.fastcodesign.com/3062016/this-neural-network-makes-human-faces-from-scratch-and-theyre-terrifying

Edit 12/19/16, see also https://multisenserealism.com/2016/12/19/fooling-computer-image-recognition-is-easier-than-it-should-be/

Edit 5/19/17 https://arstechnica.com/information-technology/2017/05/an-ai-invented-a-bunch-of-new-paint-colors-that-are-hilariously-wrong/

Edit: 6/29/17 – https://wordpress.com/post/multisenserealism.com/5161

7/22/17 – https://blog.keras.io/the-limitations-of-deep-learning.html

“One very real risk with contemporary AI is that of misinterpreting what deep learning models do, and overestimating their abilities. A fundamental feature of the human mind is our “theory of mind”, our tendency to project intentions, beliefs and knowledge on the things around us. Drawing a smiley face on a rock suddenly makes it “happy”—in our minds. Applied to deep learning, this means that when we are able to somewhat successfully train a model to generate captions to describe pictures, for instance, we are led to believe that the model “understands” the contents of the pictures, as well as the captions it generates. We then proceed to be very surprised when any slight departure from the sort of images present in the training data causes the model to start generating completely absurd captions.”

1/6/18 – https://gizmodo.com/this-simple-sticker-can-trick-neural-networks-into-thin-1821735479/amp

5/2/2019 – heatlantic.com/science/archive/2019/05/ai-evolved-these-trippy-images-to-please-a-monkeys-neurons/588517