Archive
Quora Question on Entropy
Help me understand this. If entropy is about signal transmission, and is measured by “how much information is missing,” it implies a transmitter and receiver – in other words, a relationship. Entropy is an appealing notion because at first glance the receiver of the information cannot suddenly know more than what is transmitted. But the amount of information the receiver can receive seems to be highly relative and transformable.This is if I understand correctly that entropy is not a measurement of a system’s physical properties, but of how much information is missing when that system is observed. For sure, a system will emit is information, and there will be a less than 100% reception of that signal for any system observing it. That number cannot be 101%…. But there are two sides to a relationship, and a system’s entropy is not related to some total “information” it “contains”… So there are simple workarounds to that 100% threshold.
For example, if I look at a blade of grass that I hold in my hand, I can only receive so much information about it, and I cannot get more information than what it is transmitting to me… Unless I then look at it under a microscope and now the information loss (entropy) has decreased. If entropy were a law of the universe, my act of changing my receptor would be “impossible.”
It seems to me that labeling entropy as an unbreakable law is a naive notion, and instead I see it as a useful property of observation. Does entropy deserve the status of “law of the universe”?
The subject of entropy is, appropriately highly entropic. There are a lot of different ways that the word is used, some more figuratively than others. Thermodynamic entropy is not the same as information entropy, in the sense that if we make an MPEG compressed video of a glass of ice melting into room temperature water, the first half of the movie would take more resources to compress than the last half, given that the last half would feature only a glass of still water. As the thermodynamic entropy of the actual ice increases, the information entropy of the content of the video decreases. The sensitivity of the video camera is limited, so it can’t detect the microstates of the water molecules.
I think this speaks to the question of perception, and how our designation of what constitutes a ‘system’ is more arbitrary than it may seem. All of our instruments and all of our sensing and sense-making capabilities are potentially as limited as the video camera. Our attention is squeezed into an anthropocentric range, so that our window on ourselves and the universe does not allow us to discern ultimately what is ‘a law of the universe, or simply an assumption based on our perceptions’. Everything that we can understand about the universe is furnished to us only by our perceptions, intuitions, and understanding.
Our perception leads us to expect a universe of laws and realities beyond itself, but these too are either ultimately subjective conditions, or else subjectivity itself must include the possibility of transcending itself. The difference between what we presume to be our private perceptions and what we presume to be something more is…. entropy. There is no way to obtain 100% of the information about anything. Even given omniscient access to the entire history of the universe, the nature of the universe may be intrinsically open ended within any given inertial frame.
What I am suggesting is that just as entropy is the gap between what we experience and what we think could be experienced (‘objectively’), what we consider information may only appear to be finite because of the gap between our native perceptual frame of reference and the target frame of reference. Rather than being a true description of what a phenomenon is, ‘information’ may be as fictional in its amputation of subjective qualities as perception is in its failure to pick up on physical properties. The definition of entropy itself can be thought of as gradually reversing or flipping in proportion to the distance (literal and figurative) between two perceptual frames. Take these words for example. If you read English, then your absorption of their meaning is rather loose and general. You get the idea of what I am saying, and even sort of ‘hear’ an inner narrator voice that stands in for me. It’s all rather fuzzy, but it works really well. Entropy is actually what allows you to get the gist of what I am saying or even what I am not saying (reading between the lines).
By comparison, someone who cannot read English at all may have a much clearer view of what these characters ‘actually’ look like (without as much fuzzy interpretive conditioning). In the same way, some people find it easier to draw realistically if they are copying an image upside down. Entropy and significance, subjectivity and objectivity are not functional properties, they are aesthetic ranges. The universe as a whole can, in the same way, be considered primarily an aesthetic phenomenon as a whole, in which any part can reduce another to a functional stereotype using variations on the theme of distance or insensitivity.
Obstruction of Solitude: A Guide To Noise
“And then…all the noise! All the noise, noise, noise, noise!
If there’s one thing I hate…all the noise, noise, noise, noise!
And they’ll shriek, squeak, and squeal racing round on their wheels,
Then dance with jin-tinglers tied onto their heels!” – The Grinch“Karma police, arrest this man
He talks in maths
He buzzes like a fridge
He’s like a detuned radio” – Radiohead
It might be asked, “Why should we care about noise?” Two reasons come to mind.
1) To reduce, contain, or otherwise avoid it.
2) To understand what isn’t noise, and why we prefer that.
Real Noise
The general use of the word noise refers to an unpleasant sound. Even on this most literal level, there is a sense of denial about the extent to which unpleasant qualities are subjective. The stereotypical parent, upon hearing the stereotypical teenager’s musical taste being played at high volume, may yell something like “Turn off that infernal noise!”. There is a sense that the sound demands to be labeled objectively as a terrible thing to listen to, rather than as a sound which presents itself differently according to one’s state of mind or development.
At the same time, we cannot rule out all objective, or at least pseudo-objective qualities related to signal and noise. A garage recording of a metal band or a jackhammer attacking the pavement can be uncontroversially defined as being ‘noisy’, particularly in comparison to other, more gentle sounds. ‘Real noise’, then, seems to have a range of subjective and objective qualifiers. Loud, percussive sounds are inherently noisy to us humans, and we have reason to assume the same is true for animals and even plants:
“Dorothy Retallack tried experimenting with different types of music. She played rock to one group of plants and, soothing music to another. The group that heard rock turned out to be sickly and small whereas the other group grew large and healthy. What’s more surprising is that the group of plants listening to the soothing music grew bending towards the radio just as they bend towards the sunlight.” – source
Whether we enjoy loud, percussive sounds is a matter of taste and context. Even the most diehard metal fan probably does not want to hear their favorite band blasting at five o’ clock in the morning from a passing car. Being able to control what we listen to contributes to our perception of it as noise.
Obstruction, Distraction, Destruction, and Leaks
Whether a piece of music offends our personal taste, or it is simply so loud that we can’t ‘hear ourselves think’, the experience of being distracted seems central to its status as noise. In the parlance of sound engineers, and later Silicon Valley schmoozers, the ‘signal to noise ratio’ describes this feature of noise to distract or divert attention from the intended communication. Noise not only obstructs access to the signal, the disturbance that it causes also detracts from the quality of the signal itself. If the signal to noise ratio is poor enough, it may not be worth the effort for the receiver to try to interpret it, and communication is destroyed.
This sense of noise as an obstacle to communication extends beyond audio or electronic signals to any context where information is accessed, transmitted, or stored. In his influential work on telecommunications, Clause Shannon described information entropy as those features of a signal which are costly to compress. Typically it is those patterns which cannot be easily discerned as either part of the intentional signal or part of the background noise. Despite the tremendous computational resources available for mobile communication, the signal quality on mobile devices are still generally inferior to land lines. Between microphone gating that clips off conversation instead of ambient street sounds and the loss of packets due to radio broadcast conditions or network routing conditions, it is amazing that it sounds as good as it does, but it is still a relatively leaky way to transmit voices.
Neural Noise and Withdrawal
Every sense has its own particular kind of noise. Vision has glare, blur, phosphene patterns (‘seeing stars’). Touch has non-specific tingling or itching. Olfactory and gustatory senses encounter foul odors or bad aftertastes. Feelings like nausea and dizziness which are unrelated to food or balance conditions are a kind of noise (noise is etymologically related to nausea and noxious). Part of the effect of withdrawal from an addiction that the brain becomes overly sensitized to irritating stimuli in general. It’s almost like an allergic response in that the systems which would ordinarily protect us from threats is distracted by a false threat and turned on itself. Our sensitivity to the environment, having been hijacked by an external supply of pleasurable signals, has built up a tolerance for those super-saturated instructions.
With any kind of addiction, even healthy ones like exercise or washing your hands, the nature of sense is to accommodate and normalize perceptions which are present regularly. Because the addiction provides positive reinforcement regularly, there is an artificially low noise ratio which invites your senses to recalibrate to listen more closely to the noise (which would be quite adaptive evolutionarily, you would want to still hear that tiger or smell that smoke even when you enjoy a lifestyle of hedonism and decadence). When the source of positive distraction is removed, the sensitivity to negative distraction is still cranked up to 11, which of course, taps into the original motivation to escape the negative distractions of life with an addiction in the first place. We want something to soothe our nerves, to numb the sensitivity and quiet the noise.
A Recipe For Noise
There seem to be general patterns which are common to many kinds of noise. Noise can either be an obstructing presence, or a conspicuous absence (like the dropouts on a phone call). It can be a public or a private condition which clouds judgment, invites impatience, frustration, and intolerance. Noise can be that which is incoherent, irrelevant, redundant, or inappropriate. Some signals can be temporarily irrelevant or incoherent, while others are permanently so. Besides being too loud, an audio noise can also be soft, such as a hiss or other aesthetic defect that exposes leaky conditions in the recording process. The context is important, as with withdrawal from addiction, our senses are more attuned to the relativity of sensation rather than objective measurement. Grey looks darker next to black than it does next to white.
Our ability to use our attention to pivot from foreground to background is part of what defines the difference between signal and noise, or sense and nonsense. We can all relate to the Charlie Brown effect, where the words that a teacher says are reduced to unintelligible vocalizations. As you read these words now, you may be scanning over so much tedious verbiage that looks like generic wordiness more than any particular message. Any signal can be a noise if you don’t pay attention to it in the right way, and any noise can be used as a meaningful code or symbol. Perhaps there is a way to get over our addictions a little easier if we can learn to see our irritation and cravings as a sign that we are on the right path to restoring our neurological gain.
Many Cures
The destruction of information or the suppression of noise is not as simple as it may seem. Take, for example, the difference between analgesic, anesthetic, and narcotic effects. Pain can be relieved systemically, locally, or simply by being made to seem irrelevant. It can be selectively suppressed or wiped out as part of an overall deadening of sensation. There are other ways to get pain relief besides pharmaceuticals as well. Athletes or soldiers are known to perform with severe injuries, and many people have endured astonishing hardships for the sake of their family without being fully aware of the pain they were in. While there may be endogenous pharmacology going on which accounts for the specific pain suppression, it is ultimately the context which the subject is conscious of which drives the release of endorphins and other neurotransmitters.
Semiotics of Noise
Looking at noise from a Piercean perspective, it can be seen as a failure of semiosis – a broken icon, symbol, or index. A broken index would be something like tinnitus or a phantom limb. The signal we are receiving does not correspond to the referent that we expect, and in fact corresponds only to a problem with the signaling mechanism, or some deeper problem. A signal which is broken as an index but can be understood meaningfully as a symptom of something else (maybe the tinnitus is due to a sinus infection) has reverted from a teleological index to a teleonomic* index. It coincides with a condition, but does not represent it faithfully in any way. It is noise in the sense that the expected association must be overlooked intentionally to get to the unintentional association to a symptom.
A broken index would also be one which we deem irrelevant. This type of noise, which would include the proliferation of automatic alerts, false alarms, flashing lights, spam, etc. There may be nothing wrong with what what the message is saying, but considerations of redundancy, and context inappropriateness makes it clear that what a computer thinks is important and what we think is important are very different things. This type of noise fails at the pragmatic level. It’s not that we don’t understand the message, or that its not for us, it’s that we don’t want to do anything about it.
Broken icons and symbols would similarly be made incoherent, irrelevant, or inappropriate by lacking enough syntactic integrity or semantic content to justify positive attention. Fragmented texts or degenerated signs can fail to satisfy functionally or aesthetically, either on their own, or due to intrusions from outside of the intended communication channel. The overall function of noise is to decompose. Like the odor of something that has spoiled, disorder and decay are symptoms of entropy. In the schema of cosmic metabolism, entropy is the catabolic phase of forms and functions – a kind of toll exacted by space and time which ensures that whatever rises to the threshold of existence and importance, will eventually destabilize, its differences de-tuning to indifference.
What Noise Tells Us About Signals
If we begin with the premise that signal and noise are polar opposites, then it may be useful to look at the opposite of some of the terms and concepts that have just been discussed. If noise is irrelevant, inappropriate, incoherent, and redundant, then the qualities which make something significant or important should include being relevant, appropriate, coherent, and essential. Where noise obstructs, distracts, and destroys, sense instructs, attracts, and constructs. Where noise is noxious and disgusting, signals soothe and give solace.
In the larger picture of self and consciousness, it is our solitude that is threatened by noise. Solitude, like solidity and structure are related to low entropy. It is the feeling of strong continuity and coherence, a silent background from which all moments of sound and fury are foregrounded. It is what receives all signals and insulates all noise. Integrated information? Maybe. The Philosopher’s Stone? Probably.
*teleonomy describes conditions of causality which are driven by blind statistics rather than sensible function. Evolution, for example, is a teleonomy since it does not care which species live or die, it is only those who happen to have been better suited to their ecological niche which end up reproducing most successfully.
The Doctor Prescribes Brian Eno – Blog of the Long Now
The Doctor Prescribes Brian Eno – Blog of the Long Now
In the video, Brian Eno brings up two points which relate to the last posts about intelligence, wisdom, and their relation to entropy.
“I think that one of the things that art offers you is the chance to surrender, to not be in control any longer”.
Right. That makes sense. Art debits the private side of the phenomenal ledger. The side which is concerned with the loaning of time to be returned to the Absolute with interest. Wisdom, especially in the exalted forms of Eastern philosophy, is all about surrender and flow. Dissolving of the ego. The ego is the public interface for the private self, and the seat of the kind of intelligence addressed by causal entropic forces – machine intelligence, strategic effectiveness. Important locally but trivial ultimately, in the face of eternity.
On the other side of the ledger is the chance to strive and control using intelligence. Western philosophy tends toward cultivating objectivity and critical thinking. It is a canon of skeptical intelligence and empiricism from which science emerged. Clear thinking and resisting the desire to surrender are what debit the public facing side of the ledger. Art and Science then, are the sense and motive of human culture…the tender and tough, the wag and wegh, and yes, the yin and yang.
Eno also says “The least interesting sound in the universe, probably, is the sine wave. It’s the sound of nothing happening. It’s the sound of perfection, and it’s boring. As David Byrne said in his song, Heaven is a place where nothing ever happens. Distortion is character, basically. In fact, everything we call character is the deviation from perfection. So, perfection to me, is characterlessness.”
Aha, yes. Tying this back to the Absolute, it is the diffraction, the shattering of timelessness with spacetime (aka Tsimtsum) which creates the third element – entropy. The Absolute can only be completed by its own incompleteness, and entropy is the diagonalization of experience into public facing entropies and private facing reflections of the Absolute…quanta and qualia, science and art.
Sole Entropy Well Model
Another way of visualizing the integration of physics and psyche uses the concepts related to Boltzmann’s entropy curve to conceive of the Totality/Singularity/First Cause as a bottomless fractal entropy well, as follows:
Boltzmann’s idea, as I understand it, is to explain Loschmidt’s Paradox, which (also as I understand it) is basically “If the universe is always increasing from low entropy to high entropy, then where did the initial low entropy come from?”
Boltzmann’s hypothesis places the low entropy we know as the Big Bang as just one of many statistically inevitable fluctuations of entropy distribution. It’s a bubble or wave of non-disorder that we find ourselves in anthropically (because such a bubble is the only context that a low entropy phenomenon like human minds could evolve within). Other possibilities include a Big Crunch type negentropy that accounts for the entropy trough that must precede any entropy rise.
What I suggest is a bottomless low entropy, such that the one event in which any negentropy at all occurs would automatically be the singularity into which all subsequent fluctuations would be swept. Sort of like a black hole for negentropy, hogging all possible signals for all time, banishing any rival Multiverse possibility to perpetual delay.
What this does is place Boltzmann himself, his statistical rules, and their physical enactments all within the anthropic condition in which they are possible. Statistical rules, and laws of any kind including those which define entropy are themselves physical structures which can only emerge from a bottomless entropy well. These kinds of laws and their underlying sense of possibility, probability, events, succession, recursion, regularity, comparison, persistence, etc can only be universal if every part of the universe makes some kind of sense – i.e. has some piece of this infinite negentropy.
Entropy then becomes a property like velocity, (which ranges from stillness to c), a fraction of a totality rather than an open ended scalar quantity. Entropy is a relative measure which has meaning only in relation to significance, such that anything less than 100% entropy has some quantity of absolute significance (Totality-Singularity = 0.000…1% entropy)
This way, the Big Bang becomes a perpetually receding event horizon of absolute and eternal negentropy – a Borg-like ‘bright whole’ which tyrannically absorbs and subordinates all potentials and possibilities into a single continuum-schema. This continuum must accommodate all paradoxes which amounts to a lot of fancy plate spinning and superposition, using devices like nesting outer and inner realism within each other on multiple interrelated yet mutually isolated layers or castes. These devices accomplish what I call the Big Diffraction.
Multisense Realism 
Recent Comments