Preface to Chapter 5

Chapter 5 examines the conceptual halos and how they differ among cultures. Using the linguistic evidence such as how different languages structure questions about siblings, Hoffstadter makes the case that different cultures organize concepts differently. For example, in English one would commonly ask, “do you have any brothers or sisters?” dividing the concept of sibling by gender. However, in Indonesia it is more common to ask “do you have any older siblings or younger siblings” giving the age-relation precedence over gender. Mandarin Chinese includes both subdivisions and asks, “do you have any older brothers, younger brothers, older sisters, or younger sisters?” These different structuring of the concept “sibling” are found in other words. Hoffstadter describes how Italian subdivides the concept “in fact” while we tend to simplify it. What seems simple to us may have many shades of meaning in another language, and conversely our complex concepts may be simplified in other languages. What does all of this linguistic evidence suggest? It suggests that conceptual spheres are not structured in a completely universal way across all humans. Hoffstadter uses these semantic halos, along with lexical priming and substitution errors, to describe the overlapping network of concepts.  In any AI research, an understanding of how these concepts blur will be essential in creating human-like intelligence.

Chapter 4

The “ELIZA effect” described in preface 4 can be seen in the description of BACON in chapter 4. Researchers have exaggerated the program’s ability to make scientific discoveries. The process of sorting through different types of information and determining what is relevant (arguably the most difficult part of making a discovery) is already done for BACON.

In this chapter Hoffsteader describes some of the problems of representation. How is data determined to be relevant in a representation? He tells us that low level information will be quite irrelevant at the highest representational level. My question is: is the core of the conceptual sphere enough to represent the entire concept? The second problem of representation is organization. How do we organize information into a coherent structure? This problem seems intimately entwined with the question, what is it that perceives coherence? If we know something about the structure of what is doing the perceiving, we’ll know what sort of structure the data needs to be in. Both questions are hard to answer.

True AI will require a machine and program being able to sort out the relevant data for itself, to know that the colors of the flowers on earth aren’t as relevant to calculating its gravitational pull as its mass. Or, a program would have to know that the delicious type of food bacon is not relevant to its name as the man credited with the scientific method.

Preface to Chapter 4

Hoffstadter describes in the preface to chapter 4 the Eliza effect. Born out of a combination of anthropomorphism, imprecise language, and “hype” humans see themselves in computer programs. Instead of an understanding of the brittleness of the AI, we tend to assume its strong AI and that the computer that can generate a novel has sophistication similar to the human mind. We’re doing something computers barely can: creating an analogy by ascribing characteristics of our own mental processes to those metal bags of tricks. Eliza doesn’t understand the depth of your sadness or your need to hear someone say, “tell me more about that” (but then again – a human might not, either). “She” is just a reflection of your statements and some human-crafted responses – as vacant as your reflection in the mirror.

Hoffstadter tells us that it is working within microdomains that help counteract the grandiose claims made by AI researchers. It seems to me that these days most AI researchers have reduced such claims but the media has not. You still see headlines like, “Scientists Create Robots That Can Replace Parents” and so forth. As he wrote in “The Architecture of Jumbo” it is the micro-levels of perception meeting semantic levels that are the “core mystery of intelligence” opposed to the macro-levels that are easily observed. Within the microdomain Hoffstadter is able to strip off the glamour and get to the real problem of intelligence and analogy making. Unlike French, creator of the program able to “write” a trashy novel, Hoffstadter is letting the program make more of the real decisions. However, I remember that Hoffstadter did have to program some human knowledge into Jumbo, such as some preferred letter combinations.

It seems AI isn't quite where some claim it is, but considering Humanity, humans might not be where they claim to be, either. Until a machine can create and understand that analogy, Hoffstadter and everyone else will have their work cut out for them.

Numbo: Just Like Cognitive Science

In the section “Numbo: A Study in Cognition and Recognition” the architecture of the program Numbo, designed to solve crypto-like problems, models everyone’s favorite computing device: humans. Numbo has a store of knowledge, pnet of arithmetic facts that function similar to human declarative knowledge. Hoffstadter compares Numbo to its human ideal: combinations are not strongly goal-driven, ideas are often abandoned before fully explored, and the obvious things are noticed immediately. However, Numbo can’t do everything these purposeless, undetermined ideal creatures can: it pays no attention to the order of bricks, has an “impoverished” knowledge base, and doesn’t whine about the problem being too hard.

Numbo is a prime example of Cognitive Science: the creation of a computational model to better understand human reasoning. Even if Numbo does not perform in strictly human way, it still allows us to create models of how a human might approach these problems. Hoffstader reports that many humans aren’t able to describe human ways of solving. Elements of competition, association, all may be present in human mathematical cognition. By programming these we can develop theories about how they work in the brain, and from these theories we can design experiments to test if they are supported in actual human functioning. Unlike other computer models, Numbo doesn’t use a record of objects and does not concentrate on one sub-goal at a time. Numbo is as fluid and chaotic as evolution and from its primordial swamp solutions arise.

Nimble Numbo

Nimble, Numbo: numbers mumble [like letters in Jumble] their little hints to bursting brains, “try me, I’ll take you to the target” and cross-ey'd we decrypt, accordingly. 

Hoffstadter describes in "Numbo: A Study of Cognition and Recognition" the Numbo program. Here's an analogy: Jumbo is to letters and Numbo is to ______.  [The answer is "numbers"]

Hoffstadter describes how the context of a specific target number can  change the salience of a brick-number (the numbers used, along with the four basic artithmetic opperators, to reach the target). The 8 and 10 become more salient when the target is 87, as does the 7. According to DH the rapidity of solutions is determined by two types of information: a priori knowledge and syntax. Types of knowledge come into play, but as I found when trying this type of crypto problem some solutions came more intuitively. Specifically, when no answer is immediately obvious my first searches are more likely to involve addition or subtraction. 

Right now it seems pretty likely that we will be programming a Numbo like program to solve crypto problems in a human-like way. If this program were modeling my brain, I'd probably have it try some addition and subtraction combinations after assessing the approximate size of numbers (a type of knowledge Hoffstadter describes is used in playing Numble). Small arithmetic would be rote, some would be procedural knowledge. I imagine in deciding which would be rote and which would not be, I would use my own rote knowledge. 12 x 12 is rote, 12 x 13 is not, 9 x 5 is, etc. 

Entropy: To Preserve or Provoke

Hoffstadter's description of the Jumble program architecture provides descriptions of many features that allow and facilitate creative solutions. The probabilistic nature allows every branch of the option-tree to be followed and a simple but rubbery control system allows for flexible fluidity. Within this lack of constraints, entropy is allowed to thrive when Jumble senses an answer is satisfactory. Like the chaotic revolt that follows a bad government, Jumbo creates entropy to react to "unhappy cytoplasm." Similarly, entropy can be preserved in light of unhappiness, changes are made with the same number of gloms.

Perhaps creativity is the introduction of a certain degree of randomness to thoughts – focus decreases and other areas of the brain can offer unrelated input which leads to new connections. To my knowledge, there are no thought transformations in creative thinking that can be compared to the transformations in Jumbo in tasks other than Jumble. Do we perform reversals, exchanges, in domains other than anagrams? It seems randomness is an element of creative thinking, but no transformations seem clear, only chance firings. There is a biological element of electricity to consider in creativity - the uninhibited sparkings that may not be classifiable into specific transformations, but still yields the unexpected insight which characterizes creativity.

The Language of Love, Describing Language

M was feeling down, all of his previous flashes with other letters had yielded little codelets, but tonight he was alone, unbounded, and bummed. If only I could belong to a glom, perhaps one with a nice A or a set of double OOs. Like Moo, Moon, MOOnlit, roMAntic nights. Letters mingled about the bar, constantly evaluating each other in parallel. He watched despondently as a nearby T flirted and sparked with an R and an I simultaneously before leaving them both for the more exciting possibility of an E. Terrible, isn't it?

So might go the "Romances Among The Letters" if told from a participants point of view.  Luckily, letters aren't conscious enough to suffer the tragic cliches and agonies of romance, but even if they were they would eventually join a glom as part of the solution.

In Hoffstadter's second chapter "The Architecture of Jumbo" he presents the analogy of the stages of romantic relationships to the interactions between letters. The program fluidly tests combinations of letters to mimic how the mind might generate possible solutions to a Jumbo (letter scramble) problem.

It was curious how powerful this particular analogy was when reading. The complexity of human relationships was mapped onto a design of a mental operation, something that is perhaps equally complex. It served as an effective metaphor because Hoffstadter was able to extend many features of relationships (quick elimination, internally and externally triggered breakups, flirtation, compatibility and attractiveness, multiple partners, and so on). The metaphor was not applicable for one element, but for many, making it very fitting. Combined with the importance of human relationships in our daily interactions, this metaphor was particularly salient and clear - I was able to get a good idea of how the Jumbo Architecture functions by comparing it to something I already am very familiar with. 

Precondition Premonitions

Hoffstader describes in his Prologue to chapter two a principle behind one design for parallel processing systems: layers of precondtions. A footnote in Fennel and Lesser's paper on Heresay-11 provides a description of this structure. Process are implemented when their preconditions are met. The preconditions of the preconditions (pre-pre-conditions) must be met for the preconditions to be met, and the pre-pre-preconditions must be met for those, and so forth. Hoffstadter gives an interesting example of this structure in Greek life and the reader is led to believe this structure maximizes parallel processing and prevents unnecessary processes from being run. 

Hoffstadter describes how this processing might be present in the analysis and recognition of worlds. I'm curious about how might this be implemented in the visual system in general? Is there a precondition of a vertical line and a precondition of a horizontal line for recognizing a desk? Is there a pre-precondition for an edge in general? When recognizing a face, what would be the preconditions? Concentric circles for eyes? A nose? Obviously those preconditions do not have to be met for a face (case and point:  :)). It seems each object might have several preconditions, and can be recognized without all of them being met. (At this point, do they cease to be precondtions because they are only suggestive, not necessary?) It seems definitions and recognition has a huge probabilistic and graded element, and that few of the preconditions are always necessary. Perhaps it's different for visual processing of words?

This prologue left me with questions and feeling rather unsatisfied, which only augments a desire to read-on. How is this design implemented? Can an object be recognized without meeting all of the preconditions? And how exactly does it fit into word recognition and Jumbo tasks?

Me, Myself, and My Conceptual Spere

In the most recent reading Hoffstadter describes how concepts are shaped like spheres - the most pithy and typical example at the center with less-tightly related information and looser associations orbiting around in a surrounding cloud. These spheres are the structure of all concepts, including many ordinary commonsense notions. Hofstadter describes how we generalize based on these spheres, often relating information about an incident to the most immediate location or type of structure (tennis courts, pill bottles, etc).

What about our self-concept? Is this structured in the same way? Do we also have some sense of core or information (like an event might) that is more likely to be generalized in a situation? [Can we think about our lives as one long event?] During the Me-Too phenomena, we find ourselves generalizing about a situation's "essence" - not the details. The musical instrument is unimportant, the determination and wherewithal is primary. Is the core of this event mutable, based on the sphere it's being compared to? Continuing on DH's example, because there is no bassoon element in the man's self-conceptual sphere, it becomes obvious that the only overlap or generalization could come from his musicianship. Similarly, if there is a beer closely tied to a man in the moment he says "me too", we can  safely assume he's paying for his own beer, not ours.

The self-concept sphere plays a role in the me-too phenomena. Other's are able to sense our self-concept, and see where overlaps occur to understand our generalization. While our self-concept may encompass all of our experiences, there does seem to be a core or nucleus. For the musician, playing an instrument is closer to the core than playing the bassoon, so when not explicit we assume he's generalizing about something closer to that center. Similarly, when the newlywed Carol speaks of forgetting her married name and Peter responds with having that trouble every January, we can immediately understand that the issue of a maiden name is too loosely associated with him to be generalized non-explicitly. If Carol had instead been talking to a "Patricia", this situation would be more ambiguous because a last-name change may have been more highly associated with her because of her gender. (Confusion would probably be resolved with a little thought.)

The me-too phenomena might be partially explained by applying the conceptual sphere to the self, and then assuming that when there is ambiguity, it's most likely that the person is generalizing about something closer to the core of their self-concept sphere. 

Finding and Unfounded Mountains: Top-Down and Bottom-Up

It began with the asking. What is this thing before me?
Extract features - a plateau (I have few context-dependent expectations)
But now I've found one, I want to know, are there more?
There are indeed more plateaus.
I extract again, without expectation... I found a slope!
Perhaps I should look for others - yes, here they are.
Now that I have found a mountain, I will see more.
Now that I have found mountains, I can decide exactly where they end and begin.

...And so goes the perceptual process as described by Hofstadter. It begins heavily bottom-up and with each clue becomes increasingly influenced by top-down perceptual acts. First we find the individual packets (made salient by features like similarity), then we examine if that is repeated in a pattern, then look for more units, see if they fit into the pattern, until there is nothing outside of the pattern to find.

Hofstadter tells us how pattern finding is the “core of intelligence.” Understanding the present well enough to accurately predict the future (and adjust patterns of action accordingly) certainly seems to be a feature of the smart.

It may seem trivial and obvious, but I think the process of validating perceptions is of incredible interest to psychology and cognitive science. The return to bottom-up perceptual acts after a pattern has been found and the top-down processes dominate seems essential to our functioning. It is this step that will re-ground us in reality, allows us to check and recheck our patterns. Without it, the patterns we find can go unquestioned and are allowed to dominate our consciousness. An example of top-down gone amuck and unchecked might be present in cases of the extremely paranoid or schizophrenic. Once you hypothesize people are out to get you and don’t counter the top-down lens with a bottom-up validation – more and more people will appear to be targeting you unfairly. Of course, this is an extreme example, but it illustrates the necessity of an occasional return to bottom-up after a conclusion has been reached. A reevaluation, akin to the reperception described the previous entry.

Perception, reperception

Perception, reperception: If at first you don’t perceive, reperceive,  and reperceive again!

Hofstader demonstrates that if the pattern is not clear or elegant, the packets just nonsense, the intelligent human will try again until a pattern emerges. This process of looking at a sequence or situation and being able to evaluate it through the application of many different types of knowledge appears to be a key part of intelligence. Instead of knowledge, knowledge, knowledge, the agent must have some sense of different types of knowledge and meta-rules involving their application. It’s not just taking random guesses (though that can be used), but being able to make good guesses or recognize salience.

In pattern recognition it seems aesthetics play a significant role – the pattern-finder must have a way to evaluate the pattern as correct. There is a “drive towards beauty” (p43) that guides pattern re-creation and recognition. With music, pattern recognition comes easily. We can sense when a pattern is being deviated from; it produces tension until the moment of resolution.  And what of the world outside of math and music? We feel curiosity when someone’s behavior deviates from our knowledge of them, or cognitive dissonance when the perceived pattern of our actions is not confirmed.  Often we will have a “drive towards  beauty” or simplicity and elegance in the understanding of another human being. It is easy to simplify their character. Often without reperception, our expectations will shade our actions considerably.  We won’t notice when the pattern turns out to be more complex than first-glance. To have an accurate understanding of ourselves and others, this same process of perception, reperception is essential.  It is a process that is important in all pattern finding – without we are stuck with only our immediate glance (which often yields distracters like faces in the forest and insults where none exist). The reperception can be applied with some knowledge from the first perception, but ultimately enable alternate patterns to emerge.

The Creative Process and Search Oscillations

The Swing

How do you like to go up in a swing?
Up in the air so blue?
Oh, I do think it the pleasantest thing
Ever a child can do!

Up in the air and over the wall,
Till I can see so wide,
Rivers and trees and cattle and all
Over the countryside--

Till I look down on the garden green,
Down on the roof so brown -
Up in the air I go flying again,
Up in the air and down!
                 -Robert Louis Stevenson - 1850-1894

In his book "Fluid Analogies and Creative Concepts" Douglas Hofstader describes designing integer sequence perception programs. He tells readers of breadth-first and depth first searches and admits that implementation of either may not fully or accurately reflect how human minds function (conversations with Dave Slate confirm this observation). Examination of his own creative process suggests a "initial very shallow breadth-first scan followed by a bit of depth-first in a local area highlighted by the breadth-first scan, then resurfacing for more of a broad overview..." followed by a series of oscillations between these two techniques. His mind loops upon itself, jumping between the wide and deep, until finally settling upon a satisfying answer.

The image of oscillations is a very potent one, conjuring associations of sin curves in their countless natural manifestations: light, water, and sound. Alternation between two poles, hormonal and otherwise, allows for a balance to be reached or a neutrality that still consists of "something" opposed to non-existence.

Hofstader's observations on the creative process may be reflected to some extent in research on the human brain. It has been suggested by Bogen and Bogen (1988)  that creativity may result from the temporary inhibition of the Corpus Callosum. When connection between the two "poles" of the brain is diminished momentarily, hemispheric specialization is enhanced, perhaps leading to "incubation of ideas critical for the divergent component of creativity" 
(Hemispheric connectivity and the visual–spatial divergent-thinking component of creativity) . That lack of connectivity might allow for specific modules to crest. Secluded, each is able to maximize it's individual form of processing. 

My understanding of creativity on a social, macro level certainly suggests the value of such isolation. Groups of people, all posed with the same problem, process separately before they can come together to share, debate, and decide. Perhaps each Pandemonium demon needs a moment of quiet to think about what he is going to shout.

[I imagine creativity oscillations might occur like the push and pull of a child's arms to maximize a swing's height, our hemispheres must work "against" each other to succeed, gaining momentum like waves in a pool. Perhaps they battle like competitive siblings (companies?), fighting back and forth until the best solution is found. ]

I am not seriously suggesting that the broad or detailed depth searches can be mapped exactly to secluded left or right hemispheric activity. While each could be loosely associated with a hemisphere, it is rather the process of oscillation, present in Hofstader's introspection, that is of interest. Perhaps it reflects oscillation between Corpus Callosum connectivity or non-connectivity. Or, instead it is attention that alternates while both hemispheres process in parallel.

Hofstadter's Law

Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law.