This is an extremely dense book. At times boring and others fun.

P-1

  • “how it is that animate beings can come out of inanimate matter”
  • “What is self…”
  • “strange loop” notion holds the key to unraveling the mystery that we conscious beings call “being” or “consciousness”.
  • “the quintessential strange loop that lies at the core of the proof of Kurt Godel’s famous incompleteness theorem

p9 - information-preserving transformations. Also called an “isomorphism” p17 - Godel’s incompleteness theorem paraphrase “All consistent axiomatic formulations of number theory include undecidable propositions.” p53 - “…it is natural to wonder about what portion of reality can be imitated in its behavior by a set of meaningless symbols governed by formal rules. Can all of reality be turned into a formal system?”

p82 - “… people often attribute meaning to words in themselves, without being in the slightest aware of the very complex ‘isomorphism’ that imbues them with meanings… It attributes attributes all the meaning to the object (the word), rather than to the link.

p83 levels of meaning / abstractions

p97 levels of formal systems

p110 “meta wishes” “I wish that I had a HUNDRED wishes, instead of just three! Pretty clever, eg, Mr. T?” - A “I don’t grant meta-wishes” - Genie “I wish you’d tell me what a ‘meta-wish’ is!” - A “Why don’t you rephrase your last request” - T (There are many points in this book where I was triggered to think about “prompt engineering”) “GOD won’t permit me to” - G “This is my meta-lamp” “I have a special wish to make of you, O Djinn, and of GOD” “who – or what – GOD is?” “‘GOD’ is an acronym which stands for ‘GOD Over Djinn’. The word ‘Djinn’ is used to designate Genies, Meta-Genies, Meta-Meta-Genies, etc. It is a Typeless word.” “…does any djinn ever make a mistake, and garble up a message moving up or down the chain?” “You see, the chances are infintesimal…but when you put an infinite number…it becomes virtually certain.” “I wish my wish would not be granted!” “The System crashed… transported to Tumbolia.”

p127 “recursion seems to brush paradox very closely” “Actually, a recursive definition (when properly formulated) never leads to infinite regress or paradox. This is because a recursive definition never defines something in terms of itself, but always in terms of simpler versions of itself.

p164 “We do not know enough about the nature of intelligence, emotions, or music to say whether the inner logic of a piece by Bach is so universally compelling that its meaning could span galaxies.”

p170 The “Jukebox” Theory of Meaning - “no message has intrinsic meaning”

p193 “the outermost system always remains an unproven assumption, accepted on faith.”

p194 “Fallacies can result if you fail to distinguish carefully between working in the system (the M-mode) and thinking about the system (the I-mode).”

p252 “words lead to some truth – some falsehood… Relying on words to lead you to the truth is like relying on an incomplete formal system to lead you to the truth… a formal system – no matter how powerful – cannot lead to all truths… what else is there to rely on, but formal systems?”

p254 “If words are bad, and thinking is bad, what is good?”

p255 Zen and Tumbolia The Zen monk Bassui wrote a letter to one of his disciples who was about to die, and in it he said: “Your end which is endless is a snowflake dissolving in the pure air.” The snowflake, which was once very much a discernable subsystem of the universe, now dissolves into the larger system which once held it. Though it is no longer present as a distinct subsystem, its essence is somehow still present, and will remain so. It floats in Tumbolia, along with hiccups that are not being hiccuped and characters in stories that are not being read… Zen recognizes its own limitations, just as mathematicians have learned to recognize the limitations of the axiomatic method as a method for attaining truth. This does not mean that Zen has an answer to what lies beyond Zen any more than mathematicians have a clear understanding of the forms of valid reasoning which lies outside of formalization. One of the clearest Zen statements about the borderlines of Zen is given in the following …

“You monks should know there is an even higher understanding in Buddhism.” “What is the higher Buddhism?” “It is not Buddha”.

There is always further to go; enlightenment is not the end-all of Zen. And there is no recipe which tells how to transcend Zen; the only thing one can rely on for sure is that Buddha is not the way. Zen is a system and cannot be its own metasystem; there is always something outside of Zen, which cannot be fully understood or described within Zen.

p305 “Each level is, in some sense, ‘sealed off’ from the levels below it.”

p306 Chunking. “…in useing chunked high-level models, we sacrifice determinism for simplicity.”

p309 “…we shall examine whether the brain’s top level — the mind — can be understood without understanding the lower levels on which it both depends and does not depend.”

p315 Ant colony as a whole or sum of its parts

p316 Free. “…free only within certain constraints.”

p328 “What about your OWN brain? Aren’t you aware of your own thoughts? Isn’t that the essence of consciousness? What else are you doing but reading your own brain directly at the symbol level? … You mean that I bypass all the lower levels, and only see the topmost level? That’s the way it is, with conscious systems.”

p371 “‘semantic network’”

p375 “…reference points in common…although other people differ from us in some important ways, they are still ’the same’ as we are in some deep and important ways.”

p376 “Occassionally, you find that another person is missing some of what you thought was the standard…you can communicate well in a limited domain.”

p377 “…culture affects thought”

p389 “The paradoxes of consciousness arise because a conscious being can be aware of itself…”

p406 “…certain kind of self-awareness seems to be at the crux of consciousness.”

p408 “The relationship with abstraction is easy to see. The first sequence looks random unless one has developed through a process of abstraction a kind of filter which sees a simple structure behind the apparent randomness.”

p409 “Some of these aspects may be compoletely unknown to us now but they may reveal themselves to an observer with a different system of abstractions.”

p469 “Once a system is well-defined, of ‘boxed’, it becomes vulnerable.” The act of boxing it in causes its incompleteness. “…it is the act of giving an explicit recipe for what supposedly characterizes number-theoretical truth that causes the incompleteness.”

p472 “We are trying to produce a model of the mind which is mechanical — which is essentially ‘dead’ — but the mind, being in fact ‘alive’…”

p472 “Basically the idea is that we are always outside the system”

p476 “…any human being simply will reach the limits of his own ability to Godelize at some point.”

p477 “…jump out of ourselves…Physics is an overriding system, from which there can be no escape.”

p479 “…Zen … transcending the system.”

p494 “Ceci n’est pas une pipe.”

p495 “This sentence is false. (Epimenides paradox)”

p499 Self-reproduction and self-reference

p566 “power of generalisation”

p570 brain as a formal system

p571 “One could assemble a neural net which, on a local (neuron-to-neuron) level, performed in a manner indestinguishable from a neural net in a brain, but which had no higher-level meaning at all… High-level meaning is an optional feature of a neural network…”

p573 “… it seems as though each new step towards AI, rather than producing something which everyone agrees is real intelligence, merely reveals what real intelligence is not.”

p574 “…beauty is one of those properties associated with the elusive soul…”

p582 “Our minds contain interpreters which accept two-dimensional patterns and then ‘pull’ from them high-dimensional notions…”

p582 “…an object’s meaning is not localized within the object itself”

p582 “Meaning derives from connections…”

p601 “The ineluctable core of intelligence is always in that next thing which hasn’t yet been programmed.”

p612 “In some sense all problems are abstract versions of the dog-and-bone problem. Many problems are not in physical space but in some sort of conceptual space. When you realize that direct motion towards the goal in that space runs you into some sort of abstract ‘fence’, you can do one of two things: (1) try moving away from the goal in some sort of random way, hoping that you may come upon a hidden ‘gate’ through which you can pass and then reach your bone; or (2) try to find a new ‘space’ in which you can represent the problem, and in which there is no abstract fence separating you from your goal — then proceed straight towards the goal in this new space…”

p613 “Intelligence” is the ability to “step back”

p639 What is “reality”? “…I was tuned in to a channel which is very near to the Reality Channel, but ever so slightly off…It’s nearly impossible to tune EXACTLY into the Reality Channel.”

p641 What ifs “…anything that didn’t happen didn’t happen. There aren’t degrees of ‘didn’t-happen-ess’. And same goes for ‘almost’ situations.”

p642 “…these ‘if’ propositions are fundamental to the dynamics of human feeling… to conceive of, to articulate possibilities beyond the treadmill of organic decay and death…Think how immeasurably poorer our mental lives would be if we didn’t have this creative capacity for slipping out of the midst of reality into soft ‘what if’’s!”

p644 “hierarchy of variability” “We build up our mental representation of a situation layer by layer. The lowest layer establishes the deepest aspect of the context — sometimes being so low that it cannot vary at all.”

p645 “The existence of default values for slots allows the recursive process of filling slots to come to an end. In effect, you say, ‘I will fill in the slots myself as far as three layers down; beyond that I will take the default options.’”

p645 “…often detail is irrelevant and even distracting.”

p674 “The amazing thing about language is how imprecisely we use it and still manage to get away with it.”

p677 “Beauty” comes from shared experiences and feelings. “A ‘program’ which could produce music as they did would have to wander around the world on its own, fighting its way through the maze of life…”

p680 “‘If a lion could speak, we would not understand him.’”

p684 “Can machines possess originality?” “…nothing comes out which has not been put in…”

p686 “…you have access to your thoughts but not to you neurons.”

p688 “…in any system there is always some ‘protected’ level…”

p708 “Consciousness as an intrinsically high-level phenomenon” “…some facts could be explained on the high level quite easy, but not on lower levels at all.” “…consciousness is a phenomenon that escapes explanation in terms of brain-components…”

p709 “My belief is that the explanations of ’emergent’ phenomena in our brains — for instance, ideas, hopes, image, analogies, and finally consciousness and free will — are based on a kind of Strange Loop, an interaction between levels in which the top level reaches back down towards the bottom level and influences it, while at the same time being itself determined by the bottom level.”

p711 free-will as “making choices”. “Let us take the following systems as paradigms: a marble rolling down a bumpy hill; a pocket calculator finding successive digits in the decimal expansion of the square root of 2; a sophisticated program which plays a mean game of chess; a robot in a T-maze (a maze with but a single fork, on one side of which there is reward); and a human being confronting a complex dilemma.” “…what about a marble rolling down a hill? Does it make choices?”

p722 “That’s almost as silly as me thinking I didn’t have free will! I suppose it never occurred to you, Achilles, that the three of us — you, myself, and Mr. Crab — might all be characters in a Dialogue…”

p723 brains within other brains