Previous Next Up Comments
SEHR, volume 4, issue 2: Constructions of the Mind
Updated 4 June 1995

creativity and unpredictability

Margaret Boden

Devotees of the humanities expect to be surprised. An arresting metaphor or poetic image, an unpredicted twist of the plot, a novel style of music, painting, or dance...all these unexpected things amaze and delight us. Scientists, too, appreciate the shock of a new idea--the double helix, the jumping gene, or the benzene-ring. Indeed, unpredictability is often said to be the essence of creativity. But unpredictability is not enough. At the heart of creativity lie constraints: the very opposite of unpredictability. Constraints and unpredictability, familiarity and surprise, are somehow combined in original thinking.

An adequate account of creativity should clarify the "how" in that "somehow." It must show how creativity is grounded in constraints, and why it is that creative ideas are unpredictable--and often inexplicable even after they have occurred.

In the next section, I show why it is that constraints are crucial to creativity. Then, in Section III, I discuss a scientific approach to creativity that uses concepts drawn from artificial intelligence to help us understand the human mind. Finally, I offer various reasons why creative ideas will always remain surprising, why unpredictability is inescapable.

creativity, constraints, and conceptual spaces

How can one distinguish creativity from (mere) novelty, in a way that helps us to understand both the surprise-value of creative ideas and their sometimes startling obviousness?

People of a scientific cast of mind often try to define creativity in terms of "novel combinations of old ideas." In that case, the surprise caused by a creative idea must be due to the improbability of the combination, and purely statistical tests (as used by some experimental psychologists) could identify creativity. Creativity, in this view, just is unpredictability.

Combination-theorists typically leave at least two things unsaid. The "novel combinations" have to be not only new, but interesting: to call an idea creative is, in part, to say that it is valuable. Combination-theorists, however, usually omit value from their definition of creativity. Also, they fail to explain how the novel combination comes about. They take it for granted (for instance) that we can associate similar ideas or recognize more distant analogies, without asking just how such feats are possible.

These complaints aside, what is wrong with the combination-theory? Many ideas we regard as creative are indeed based, at least in part, on unusual combinations. Much of Coleridge's enchanting imagery in The Rime of the Ancient Mariner, for example, draws together diverse ideas scattered in his eclectic reading.1 And Harvey's vision of the heart as a pump is just one case in which a novel comparison of ideas led to important scientific discoveries. Combination-theory, then, is not wholly irrelevant. But it cannot explain, or even adequately describe, the most intriguing cases.

Creative "novelties" are of significantly different types--the most interesting of which lie beyond combination-theory. Many creative ideas are surprising not because they involve some unusual mix of familiar ideas, but in a deeper way. They concern novel ideas which not only did not happen before, but which--in a sense that must be made clear (and which combination-theory cannot express)--could not have happened before.

What does it mean to say that an idea "could not" have arisen before? Unless we know that, we cannot distinguish radical originality from mere "first-time" newness. This "could not" refers to the mental capabilities of the person concerned, and says nothing about anyone else. Someone is creative, in this sense, if they have an idea which they could not have had before--no matter how many other people have already thought of it. ("Historical" creativity is a special case, in which we use historical evidence and social criteria to credit someone not only with creativity in the sense already defined, but also with being the first to have had the idea.)

Linguists sometimes speak of the "creativity" of natural language, by which they mean that language is an unending source of novel sentences. But these sentences are novelties which clearly could have happened before, being generated by the same rules that can generate other sentences in the language. Any native speaker could produce novel sentences using the relevant grammar. In general, to come up with a new sentence is not to do something deeply creative.

The "coulds" in the previous paragraph are computational "coulds." That is, they concern the set of structures described and/or produced by one and the same set of generative rules (the rules of English grammar and vocabulary). This notion enables us to distinguish first-time novelty from radical originality. A merely novel idea is one which can be described and/or produced by the same (specified) set of generative rules as are other, familiar ideas. A genuinely original, or creative, idea is one which cannot.

It follows that constraints, far from being opposed to creativity, make creativity possible. To throw away all constraints would be to destroy the capacity for creative thinking. Random processes alone can produce only first-time curiosities, not radical surprises (although randomness may contribute to creativity, as explained in Section IV).

To justify calling an idea creative, then, one must specify the particular set of generative principles--what one might call the conceptual space--with respect to which it is impossible. Conceptual spaces are established styles of thinking (sonata form, chess, tonal harmony, pointillism, sonnets, limericks, aromatic chemistry, the Thirty-Nine Articles...). Different conceptual spaces have distinct structures, each with its own dimensions, pathways, and boundaries.

However, it now begins to look as though no one can possibly be creative. To be sure, anyone can juggle ideas in their head, sometimes coming up with interesting improbabilities. But, with respect to the familiar structures in the relevant domain (chemistry, poetry, music...), a "deeply" creative idea is not just improbable, but impossible.

How can it arise, then, if not by magic? And how can one impossible idea be more surprising, more creative, than another? How can creativity happen?

A generative system defines a certain range of possibilities: molecules, for example, or jazz melodies. These structures are located in a conceptual space whose limits, contours, and pathways can be mapped, explored, and transformed in various ways.

The "mapping" of a conceptual space involves the representation, whether at conscious or unconscious levels, of its structural features.2 The more such features are represented in the mind of the person concerned, the more power (or freedom) they have to navigate and negotiate these spaces. A crucial difference-- probably the crucial difference--between Mozart and the rest of us is that his cognitive maps of musical space were very much richer, deeper, and more detailed than ours. In addition, he presumably had available many more domain- specific processes for negotiating them.

Much as a real map helps a traveller to find--and to modify--his route, so mental maps enable us to explore and transform our conceptual spaces in imaginative ways. Studies of young children have shown that they need explicit, though not necessarily conscious, representations of their lower-level drawing skills in order to draw imaginatively: to create a picture of a one-armed man, for instance, or a seven-legged dog.3 Before developing such skill-maps, the child simply cannot draw a one-armed man (and finds a two-headed man extremely difficult even to copy). Much the same applies to other skills, such as language and piano-playing (in adults as well as children).4 To understand Mozart's creativity, then, we would need to know what were his (many-levelled) mental maps of musical space, and what processes he employed to explore and alter them.

What counts, in this context, as exploration? One interesting example is the development of post-Renaissance Western music, which is based on the generative system known as tonal harmony. Each piece of tonal music has a "home key" from which it starts, from which (at first) it did not stray, and in which it must finish. Reminders and reinforcements of the home key are provided (chords, arpeggios, and fragments of scales). As the years passed, the range of possible home keys became increasingly well defined. J. S. Bach's "Forty- Eight" was specifically designed to explore--and clarify--the tonal range and basic dimensions of this conceptual space.

Travelling along the path of the home key alone soon became insufficiently challenging. Modulations between keys then appeared, within the body of the composition. But all possible modulations did not appear at once. The range of harmonic relations implicit in the system of tonality became apparent only gradually. At first, only a small number within one composition were tolerated, and these early modulations took place only between keys very closely related in harmonic space. With time the modulations became more daring and more frequent. By the late nineteenth century there might be many modulations within a single bar, not one of which would have appeared in early tonal music.

Eventually, the very notion of the home key was undermined. With so many, and such daring, modulations within the piece, a "home key" could be identified not from the body of the piece but only from its beginning and end. Inevitably, someone (it happened to be Schoenberg) suggested that the convention of the home key be dropped altogether, since it no longer made sense in terms of constraining the composition as a whole.

Exploring a conceptual space is one thing. Transforming it is another. In general, novel ideas gained by exploring an unknown niche in a pre-existing conceptual space are regarded as less creative than ideas formed by transforming that space in radical ways. A mathematician at Princeton's Institute for Advanced Study put it like this:

The reason why proving this would be important is that in order to prove Schanuel's Conjecture you would in the process have to prove something else, something bigger, and that would be really great. I mean if you prove it by being tricky, without establishing anything new, then you wouldn't get any recognition, and you probably shouldn't. The assumption is that if you prove one of these great big things like Fermat's last theorem, you'd have to develop a whole new branch of mathematics. But if you prove it by some quirky little two-line proof that everybody had always missed, then people would pat you on the back, and they'd say "God, you're clever!", and so on, but it really wouldn't be as much of an accomplishment.5

Notice, however, the words used in this quotation. To explore a space and locate within it a substantial sub-space, an environmental niche for a range of previously unsuspected ideas, is to do more than find "some quirky little two-line proof." That phrase suggests a notch, not a niche. Some musicians argue that Mozart was less adventurous than Haydn (that he made fewer transformations than Haydn), but that his exploitation of the potential of the current musical style (his exploration of the existing conceptual space) was so supreme as to make him the greater composer. Whatever your views on that particular judgment, the point is that some explorations are more surprising than others, even if transformations are the most surprising of all.

What is it to transform a conceptual space? One example has just been mentioned: Schoenberg's dropping the home-key constraint to create the space of atonal music. Dropping a constraint is a general heuristic for transforming conceptual spaces. Non-Euclidean geometry, for instance, resulted from dropping Euclid's fifth axiom, about parallel lines meeting at infinity. And Oscar Wilde's story of Dorian Grey relied on dropping the normal human constraint of aging.

Another very general way of transforming conceptual spaces is to consider the negative; that is, to negate a constraint. One well-known instance concerns Kekul&eacu;'s discovery of the benzene-ring. He described it like this:

I turned my chair to the fire and dozed. Again the atoms were gambolling before my eyes....[My mental eye] could distinguish larger structures, of manifold conformation; long rows, sometimes more closely fitted together; all twining and twisting in snakelike motion. But look! What was that? One of the snakes had seized hold of its own tail, and the form whirled mockingly before my eyes. As if by a flash of lightning I awoke.6

This vision was the origin of his hunch that the benzene-molecule might be a ring, a hunch which turned out to be correct.

Prior to this experience, Kekul&eacu; had assumed that all organic molecules are based on strings of carbon atoms (he himself had produced the string-theory some years earlier). But for benzene, the valencies of the constituent atoms did not fit.

We can understand how it was possible for him to pass from strings to rings, as plausible chemical structures, if we assume three things (for each of which there is independent evidence). First, that snakes and molecules were already associated in his thinking. Second, that the topological distinction between open and closed curves was present in his mind. And third, that the "consider the negative" heuristic was present also. Together, these three factors could transform "string" into "ring."

A string-molecule is an open curve. If one considers the negative of an open curve, one gets a closed curve. Moreover, a snake biting its tail is a closed curve which one had expected to be an open one. For that reason, it is surprising, even arresting ("But look! What was that?"). Kekule might have had a similar reaction if he had been out on a country walk and happened to see a snake with its tail in its mouth. But there is no reason to think that he would have been stopped in his tracks by seeing a Victorian child's hoop: no topological surprises there.

Finally, the change from open curves to closed ones is a topological change, which by definition will alter neighbour-relations. And Kekul&eacu; was an expert chemist, who knew that the behaviour of a molecule depends partly on how the constituent atoms are juxtaposed. A change in atomic neighbour-relations is very likely to have some chemical significance. So it is understandable that he had a hunch that this tail-biting snake-molecule might contain the answer to his problem.

Hunches are common in human thinking (mathematicians often describe them in terms of an as-yet-unproven "certainty"). An adequate theory of creativity must be able to explain them. It must show how it is possible for someone to feel (often, correctly) that a new idea is promising even before they can say just what its promise is. The example of Kekul&eacu; suggests that a hunch is grounded in appreciation of the structure of the space concerned, and some notion of how the new idea might fit into it. Without hunches, people would waste a lot of time in following-up new ideas that "anyone could have seen" would lead to a dead-end. Some of our hunches, some of our intuitions, turn out to be mistaken. But that is not to say that we would be better off without them.

can creativity be scientifically understood?

A scientific account of creativity is possible only if the ideas conveyed metaphorically in Section II can be clearly expressed. Conceptual spaces would have to be precisely identified and mapped, and ways of exploring and changing them (some general, some domain-specific) would need to be explicitly defined.

(In addition, we would need a clear theory of analogical thinking. Analogy plays a large part in "combinatorial" creativity and sometimes, as in the case of Kekul&eacu;, contributes to "impossibilistic" creativity too. Some promising suggestions about how analogies can be recognized, and how they can inform our perception so that we come to see things in a new way, have been offered by Douglas Hofstadter and colleagues.7)

Literary critics, musicologists, choreographers, and historians of art and science have much to say that is relevant. But their knowledge of the various conceptual spaces in people's minds must be made as explicit as possible, to clarify just which structures can, and which cannot, be generated within them.

One way in which this can be done is to use the methods of artificial intelligence (AI), in which conceptual spaces can be mapped and explored, and sometimes transformed. Using computers to model mental processes is a powerful way of finding out just what a particular psychological theory can--and cannot--explain.

Some relevant computer-models have mapped an interesting conceptual space, without any intention to model creativity as such. Computational work on musical perception, for example, has helped to advance the theory of harmony and to show how listeners manage to find their way around the space of tonal harmony.8

The musical-perception program is given a monophonic melody, the first note being identified not by name, but by its position on a piano keyboard. The program's task is to write down the piece in musical notation (a task which carries an inbuilt test of correctness at every point). To do this, it has to discover many different things. These include the key; the harmonic role of each note (the same black note on the keyboard can be called F-sharp, G-flat, and E-double-sharp); the accidentals; the time-signature; the rests; and the positions of the barlines (including the first, which may or may not follow a "complete" bar). In short, perceiving a melody is not a simple matter. Like understanding a poem or a story, it involves interlocking interpretations of many different kinds. Longuet-Higgins's computational account of musical interpretation has been successfully applied to compositions by many composers, from baroque to romantic. Clearly, then, it offers powerful generalizations concerning our recognition (and many Western composers' use) of key, modulation, and metre.

It is perhaps unsurprising that musical features such as these can be systematically defined in terms of specific structures and procedures. But what of musical expression? This, surely, cannot possibly be captured in such a way? Without expression, music sounds "dead," even absurd. In playing the notes in a piano score, for instance, pianists add such features as legato, staccato, piano, forte, sforzando, crescendo, diminuendo, rallentando, accelerando, ritenuto, and rubato. But how? Can we express this musical sensibility precisely? That is, can we specify the relevant conceptual space? Or is expression the essentially indefinable (and unpredictable) human essence of musical performance?

Longuet-Higgins, using a computational method, has tried to specify the musical skills involved in playing expressively.9 Working with two of Chopin's piano compositions, he has discovered some counterintuitive facts. For example, a crescendo is not uniform, but exponential (a uniform crescendo does not sound like a crescendo at all, but like someone turning-up the volume knob on a wireless); similarly, a rallentando must be exponentially graded (in relation to the number of bars in the relevant section) if it is to sound "right." Where sforzandi are concerned, the mind is highly sensitive: as little as a centisecond makes a difference between acceptable and clumsy performance. By contrast, the full range of pianissimo to fortissimo (for these compositions, at least) can be captured by a single-digit number of absolute decibel-levels--which need to be changed only at barlines, not within any bar.

This work is not a study of creativity. It does not model the exploration of a conceptual space, never mind its transformation. But it is relevant because creativity can be ascribed to an idea (including a musical performance) only by reference to a particular conceptual space. The more clearly we can map this space, the more confidently we can identify and ask questions about the creativity involved in negotiating it. A pianist whose playing style sounds "original," or even "idiosyncratic," is exploring and transforming the space of expressive skills which Longuet-Higgins has studied. We might be able to compare the performances of Rosalyn Tureck and Glenn Gould, playing one and the same Bach fugue, using (an enriched set of) concepts such as these.

Closely related work has been done on jazz improvisation, where the prime interest is in the creative powers of the human jazz musician.10 This work shows that very simple computational processes, making only minimal demands on short-term memory (which in humans is very limited), can generate acceptable-- though unexciting--improvisations. By contrast, the generation of the underlying chord-sequence requires a much more powerful grammar, which is why (for all but the very simplest cases) it cannot be done on the fly by human musicians.

The rules for improvisation take account of melodic contours, harmony, metre, tempo, and playing (and passing between) the chords schematically described in the underlying chord-sequence. Whenever these rules allow for more than one alternative, the decision is made randomly (using a random-number table). Consequently, one and the same chord-sequence will not spawn identical improvisations on different occasions. Each individual improvisation is--though musically constrained--unpredictable.

A rather different jazz-improviser has been written by Paul Hodgson, himself a professional jazz musician.11 Unlike Johnson-Laird's program, this one starts out with significant musical structure provided to it "for free" by the human user. For instance, it may be given the theme of "Sweet Georgia Brown," together with chords specified at various points. It then improvises on the melody, varying it according to a number of specific constraints defining the relevant jazz-space. It can employ any percentage of notes chosen from the original theme, or from the currently relevant (tonal) scale, or from the full chromatic range. It can use (pre- given or self-composed) melodic and rhythmic patterns and can "cut and paste" them in many (random) ways. It can compose "call" and "reply" over varying numbers of bars...and so on.

Human users can, if they wish, instruct this jazz-improviser to use particular methods (one or more) at a given time. Even so, many detailed choices must be made randomly. The program's music is therefore unpredictable (whether used interactively or not). Its normal level is that of a mediocre professional musician, but it often produces genuinely interesting musical ideas that a good human musician can borrow, or develop.

A broadly similar, though much more complex, composition-program has been developed by the composer David Cope.12 He calls it "EMI": experiments in musical intelligence. This program analyses the music, and then composes in the styles, of Mozart, Stravinsky, Joplin, and others (including non-Western composers). It possesses powerful musical grammars and lists of "signatures": melodic, harmonic, metric, and ornamental motifs characteristic of individual composers. Using general rules to vary and intertwine these, it often composes a musical phrase nearly identical to a signature which has not been provided. This suggests a systematicity in individual composing styles, a musical "authorial voice" that appears in many different compositions. That such authorial voices exist is no surprise: any musically literate listener is intuitively aware of them. But EMI helps us to define how the "same" conceptual space (early baroque music, for instance) can be differentially shaped at certain points by individual composers.

In all these cases (not all of which were intended as studies of creativity), musical "intuition" has been, in part, anatomized. Various detailed questions, interesting not only to psychologists but also to musicologists, have been raised. A musical heuristic that is helpful in interpreting Bach cantatas, for example, may or may not be helpful in dealing with other music by Bach, or pieces by Vivaldi or Brahms. The relevant musical spaces can be studied, and some dimensions and pathways explicitly defined, in this way.

The constraints of music, complex though they are, are more amenable to definition than those involved in literature. Here, many rich conceptual spaces have to be negotiated simultaneously.13 There is the challenge of using language felicitously. Even getting the grammar right can be both difficult and, as any Proustian will allow, aesthetically significant. Picking le mot juste adds extra complexities: the poet's mot juste may be very different from that of the novelist, and in any case will depend on the type of poetry involved. As for the content, the novelist (at least of the more traditional kind) must produce sensible plots, taking account both of the motivation of the characters and of their common- sense knowledge. The thriller-writer must deliberately mislead us (even Henry James, in his magnificent novella The Beast in the Jungle, teased his readers until the penultimate page). Both the novelist and the characters in the story must have some (not necessarily identical) understanding of everyday interpersonal concepts, such as cooperation and betrayal. And both poet and novelist need some sense of the overall "shape" of a poem or story, even if they intend somehow to subvert it (as John Fowles offers us two alternative endings to The French Lieutenant's Woman). In short, both understanding literature and creating it are highly complex matters.

It is not surprising, then, if literary programs are not as impressive as some musical programs are. Connectionist (neural network) models can help us to understand how Coleridge's imagery in The Rime of the Ancient Mariner could have come into his mind, given the specific literary sources he was reading at the time.14 But they have nothing to say about the shape of the poem as a whole: the continuing image of the journey, for example, or the mariner's obsessive interruption of the activities of the wedding-guests. Current computer models of literary creativity as such (story-writing, for instance) are clumsy and unconvincing, involving infelicities and absurdities of various kinds. Those infelicities help us to appreciate the richness and subtlety of the human mind, but offer only limited help in articulating the creative constraints that interest us here.

Story-writing programs can generate plausible stories (expressed in very clumsy language) only where very simple plot-spaces, and very limited world-knowledge, are concerned. One such program produces Aesop-like tales, including a version of "The Fox and the Crow."15 A recent modification of the Aesop-program is more subtle.16This system, called MINSTREL, uses a form of analogical ("case- based") reasoning to compose novel stories which are deformations of familiar ones. (Turner calls them "transformations," but the alterations concerned are not sufficiently fundamental to be termed transformations in my sense.) Many superficially distinct plots, and characters, can be generated in this way. Moreover, because MINSTREL distinguishes the author's goals from the goals of the characters, it can solve meta-problems about the story as well as solving problems posed within it. Both these story-telling programs can surprise us, in the sense that their stories are (to some extent) unpredictable. But their literary powers are strictly limited, compared with ours.

Computational definitions of interpersonal concepts used in stories--including cooperation, betrayal, and escape--are also relevant here.17 Such definitions help us to see the structural affinities between (for example) different sorts of betrayal, such as abandonment and letting-down. These two species of betrayal will involve very different types of character and very different consequences. For instance, only the strong can abandon and only the weak can be abandoned, whereas anyone can let down, or be let down. A good story-writer, whether human or artificial, needs to have a grasp of these differences. And a writer able to tell a story about any human culture, or even (as in some science-fiction) none, likewise needs to understand how the possibilities inherent in the structure of interpersonal relations vary from group to group.

Naturally, literary programs will continue to improve. They can already generate simple plots that might be of use to an especially unimaginative author of crude types of fiction--such as the lower forms of soap-opera, or of "nurses and doctors" romance. Even if the human author has to embellish the plot, and put the whole thing into acceptable English, some helpful plot-ideas might come from the program. But a program capable of writing a genuinely interesting novel or short story is not on the horizon, and in my opinion never will be. The complexity of the conceptual spaces concerned is too great.

A much simpler conceptual space, not literary but graphic, has been modelled by Harold Cohen's AARON.18This program generates aesthetically acceptable line- drawings of human acrobats. Like the musical and literary compositions described above, the drawings are individually unpredictable. This is due to the fact that random choices are allowed at many points, when the program is running. (If this were not so, AARON would always draw the same picture.)

However, all the drawings lie within the pre-assigned graphic genre. AARON (again, like the composition programs described above) can explore a conceptual space, sometimes leading us to new locations in it, but cannot transform it in any fundamental way.

The genre is defined, in part, by AARON's "body grammar." This specifies not only the anatomy of the human body (head, trunk, arms, legs), but also how the various body parts appear from different points of view or in different bodily attitudes. So a flexed biceps will visibly bulge, and an arm pointing at the viewer will be foreshortened. Like any conceptual space, AARON's body grammar both enables creativity and restricts it. Consider, for instance, the apparently simple question of whether or not AARON is capable of drawing one-armed acrobats.

This question cannot be satisfactorily answered merely by sitting beside AARON through thousands of runs, and watching to see whether it ever draws an acrobat with only one arm shown. Certainly, the program can draw acrobats with only one arm visible (if the other arm is hidden behind the body of some other acrobat). But it cannot draw one-armed acrobats. It cannot draw something of which one could truly say, "That acrobat-drawing has only one arm visible because it is depicting a one-armed acrobat."

The difficulty, here, has nothing to do with the philosophical question whether a computer could really depict (or intend, or understand, or create) anything at all. The difficulty, rather, lies in the nature of the conceptual space within which AARON's drawings are generated. The relevant conceptual space (the program's model of the human body) does not allow for the possibility of there being one- armed people. They are, one might say, unimaginable by AARON. If, as a matter of fact, the program has never produced a picture showing one acrobat's right wrist occluding another acrobat's left eye, that is a mere accident of its processing history. It could have done so at any time. But the fact that it has never drawn a one-armed acrobat has a deeper explanation: such drawings are, in a clear sense, impossible.

If Cohen's program could "drop" one of the limbs (as a geometer may drop Euclid's fifth axiom, or Schoenberg the notion of the home-key), it could then draw one-armed, or one-legged, figures. A host of previously unimaginable possibilities, only a subset of which might ever be actualized, would have sprung into existence at the very moment of dropping the constraint that there must be (say) a left arm.

A superficially similar, but fundamentally more powerful, transformation might be made if the numeral "2" had been used in the program to denote the number of arms. For "2," being a variable, might be replaced by "1" or even "7." A general- purpose, tweaking-transformational heuristic--whether in a human mind or a computer--might look out for numerals and try substituting varying (perhaps randomly chosen) values. Kekul&eacu;'s chemical successors employed such a heuristic when they asked whether any ring-molecules could have fewer than six atoms in the ring. They also treated carbon as a variable--as a particular instance of the class of elements--when they asked whether molecular rings might include nitrogen or phosphorus atoms. A program which (today) drew one-armed acrobats for the first time by employing a "vary-the-variable" heuristic could (tomorrow) be in a position to draw seven-legged acrobats as well. A program which merely "dropped the left arm" could not.

A few computer-models of creativity can not only explore their conceptual spaces, but also transform them to some extent. The "Automatic Mathematician," and its successor EURISKO, are closely-related examples.19Other examples include programs using "genetic algorithms."

The Automatic Mathematician (AM) does not do sums. Instead, it generates and explores mathematical ideas, coming up with new concepts and hypotheses to think about. Its heuristics can examine, combine, and transform its concepts in many ways, some general and some domain-specific. One, for instance, can generate the inverse of a function (this is a mathematical version of "consider the negative").

Much as humans have hunches, favouring some new ideas over others, so does this program. Some of AM's heuristics suggest which sorts of concept are likely to be the most interesting, and it concentrates on them accordingly. AM finds it interesting, for instance, that the union of two sets has a simply expressible property that is not possessed by either of them. This is a mathematical instance of the general notion that emergent properties, in whatever domain, are interesting.

AM's hunches, like human hunches, are sometimes wrong. Nevertheless, it has come up with some powerful mathematical notions, including an entirely novel theorem concerning a class of numbers (maximally-divisible numbers) which its programmer had never heard of. In short, AM appears to be creative.

Admittedly, the precise extent of AM's creativity is unclear. Some heuristics may have been specifically included to make certain discoveries possible, and the use of a powerful AI-programming language may have provided AM with some mathematically relevant structures "for free."20However, we do have a sense of what questions we should ask, in order to judge. (And, significantly, these questions are not confined to "did it do something unpredictable?") Even though human minds are different from AM in many ways, some broadly similar questions would be relevant in deciding whether to call a human mathematician creative.

Because EURISKO, unlike AM, has heuristics for changing heuristics, it can explore and transform not only its stock of concepts but its own processing-style. For example, one heuristic asks whether a rule has ever led to any interesting result. If it has not (given that it has been used several times), it will be less often used in future. If the rule has occasionally been helpful, though usually worthless, it may be specialized in one of several different ways. (Because it is sometimes useful and sometimes not, the specializing-heuristic can be applied recursively.) Other heuristics work by generalizing rules in various ways or by creating new rules by analogy with old ones.

With the help of domain-specific heuristics to complement these general ones, EURISKO has generated previously unthought-of ideas concerning genetic engineering and VLSI-design. Some of its suggestions have even been granted a U.S. patent (the U.S. patent-law insists that the new idea must not be "obvious to a person skilled in the art"). The heuristic principles embodied in EURISKO have nothing specifically to do with science. They could be applied to artistic spaces too. So some future acrobat-drawing program, for example, might be able to transform its graphic style by using similar methods.

A different type of self-transforming program is seen in systems using genetic algorithms, or GAs.21 GA-systems use rule-changing algorithms modelled on genetic processes such as mutation and crossover, and algorithms for identifying and selecting the relatively successful rules. Mutation makes a random change in a single rule. Crossover brings about a mix of two, so that (for instance) the lefthand portion of one rule is combined with the righthand portion of the other--where the break-points are assigned at random. Together, these algorithms (working in parallel) generate a new system better adapted to the task in hand.

One example of a GA-system is a computer-graphics program written by Karl Sims.22 This program uses genetic algorithms to generate new images, or patterns, from pre-existing images. Unlike most GA-systems, the selection of the "fittest" examples is not automatic, but is done by the programmer--or by someone fortunate enough to be visiting his office while the program is being run. That is, the human being selects the images which are aesthetically pleasing, or otherwise interesting, and these are used to "breed" the next generation. (Sims aims to provide an interactive graphics-environment, in which human and computer can cooperate in generating otherwise unimaginable images.)

In a typical run, the first image is generated at random. Then the program makes nineteen independent changes (mutations) in the initial image-generating rule, so as to cover the VDU-screen with twenty images: the first, plus its nineteen ("asexually" reproduced) offspring. At this point, the human uses the computer- mouse to choose either one image to be mutated, or two images to be "mated" (through crossover). The result is another screenful of twenty images, of which all but one (or two) are newly-generated by random mutations or crossovers. The process is then repeated, for as many generations as one wants.

Randomness enters in at various points. Sims's GA-system starts with a list of twenty basic LISP-functions. Some of these functions can alter parameters in pre- existing functions: for example, they can divide or multiply numbers, transform vectors, or define sines or cosines. Some can combine two pre-existing functions, or nest one function inside another (so many-levelled hierarchies can arise). Some are simple image-generating functions, capable (for example) of generating an image consisting of vertical stripes. Others can process a pre-existing image, for instance making "lines" or "surface-edges" more or less distinct. When the program chooses a function at random, it also randomly chooses any missing parts. So if it decides to add something to an existing number (such as a numerical parameter inside an image-generating function), and the "something" has not been specified, it randomly chooses the amount to be added. Similarly, if it decides to combine the pre-existing function with some other function, it may choose that function at random.

In addition to the unpredictability resulting from randomness, Sims's computer- generated images often cause a deeper form of surprise. It is as if, besides our being unable predict "heads" or "tails" when tossing a coin, the coin sometimes showed a wholly unexpected design. If one compares a parent-image with some of its descendants (or even some of its immediate offspring), one may be amazed by the difference. The change(s) effected to the earlier image on the way to the later one sometimes seem like relatively unadventurous exploration, or tweaking. The colour of what is clearly the same image may have been altered, for instance, and/or lines may obviously have been blurred. Sometimes, however, one cannot say, merely by looking at the chosen image-pair, how they are related--or if they are related at all. The one appears to be a radical transformation of the other, or even something entirely different.

Moreover, having the relevant "explanations" available does not always help. One may not be able to say just why this image resulted from that LISP-expression. Sims himself cannot always explain the changes he sees appearing on the screen before him, even though he can access the LISP-expression responsible for any image he cares to investigate, and for its parent(s) too.

how unpredictability enters in

The preceding sections have suggested various ways in which unpredictability can enter into creativity. Let us itemize them here.23

A given style of thinking typically allows for many points at which two or more alternatives are possible--possible, that is, relative to the style. For example, several notes in a musical improvisation may be both melodious and harmonious. Various numbers of people may be acceptable within a painting of a group. Diverse reactions are motivationally plausible, in response to betrayal by a lover. And many words rhyme with moon. In science, too, various possibilities often exist. Could there be a ring-molecule with precisely eight atoms in the ring, and another with four? Perhaps several elements besides carbon can form molecular rings, or long strings? The conceptual spaces current in science at the time of asking such questions allow for several alternatives (even though later scientific advance may show that only one was empirically possible).

Were all this not so, the rich variation we see in the humanities and the sciences could not arise. Much is unavoidable (limericks cannot be written in blank verse), but some "space" for choice has to exist. When the time comes, however, the space must be filled. Some specific choice must be made. Any colour, or any shade of blue or purple, may be acceptable for a dress, or for a flower in a painting. But some specific shade must be chosen. Likewise, many exploratory and transformational heuristics may be potentially available at a certain time, in dealing with a given conceptual space. But one or other must be chosen. Even if several heuristics can be applied at once (like parallel mutations in a GA-system), not all possibilities can be simultaneously explored. The choice has to be made somehow.

Occasionally, the choice is random, or as near to random as one can get. So it may be made by throwing a dice (as in playing Mozart's aleatory music); or by consulting a table of random numbers (as in the jazz program); or even, possibly, as a result of some sudden quantum-jump inside the brain. There may even be psychological processes akin to GA-mechanisms, producing novel ideas in human minds.

More often, the choice is fully determined, by something which bears no systematic relation to the conceptual space concerned. (Some examples are given below.) Relative to that style of thinking, however, the choice is made randomly. Certainly, nothing within the style itself could enable us to predict its occurrence.

In either case, the choice must somehow be skillfully integrated into the relevant mental structure. Without such disciplined integration, it cannot lead to a positively valued, interesting idea. (The schizophrenic's word-salad is not poetry, and not any succession of events is a story.)

With the help of this mental discipline, even flaws and accidents may be put to creative use. Oliver Sacks reports the case of a jazz drummer suffering from Tourette's syndrome.24 He is subject to sudden, uncontrollable, muscular tics. These occur, though with reduced frequency, even when he is drumming. As a result, his drumsticks sometimes make unexpected sounds. But this man's musical skill is so great that he can work these supererogatory sounds into his music as he goes along. At worst, he "covers up" for them. At best, he makes them the seeds of unusual improvisations which he could not otherwise have thought of. (Similar remarks apply to jazz musicians who use Hodgson's program to help spawn interesting musical ideas, or to artists and designers who use "evolutionary" computer-systems in developing ideas which they could not have thought up by themselves.)26

One might even call the drummer's tics serendipitous. Serendipity is the unexpected finding of something one was not specifically looking for. But the "something" has to be something which was wanted, or at least which can now be used. Fleming's discovery of the dirty petri-dish, infected by Penicillium spores, excited him because he already knew how useful a bactericidal agent would be. Proust's madeleine did not answer any currently pressing question, but it aroused a flood of memories which he was able to use as the trigger of a life-long project.

Events such as these could not have been foreseen. Both trigger and triggering were unpredictable. Who was to say that the dish would be left uncovered and infected by that particular organism? And who could say that Proust would eat a madeleine on that occasion? Even if one could do this (perhaps the laboratory was always untidy, and perhaps Proust was addicted to that particular confection), one could not predict the effect the trigger would have on these individual minds.

This is so even if there are no absolutely random events going on in our brains. Chaos theory has taught us that fully deterministic systems can be, in practice, unpredictable. Our inescapable ignorance of the initial conditions means that we cannot forecast the weather, except in highly general (and short-term) ways. The inner dynamics of the mind are more complex than those of the weather, and the initial conditions--each person's individual experiences, values, and beliefs--are even more varied. Small wonder, then, if we cannot fully foresee the clouds of creativity in people's minds.

To some extent, however, we can. Different thinkers have differing individual styles, which set a characteristic stamp on all their work in a given domain. Thus Dr. Johnson complained, "Who but Donne would have compared a good man to a telescope?" We may value Donne's poetry more highly than Johnson did, but we can hardly deny the aptness of his remark. Likewise, we may be irritated by Hitchcock's habit of appearing fleetingly in his own films, but we recognize this (and we look out for it) as part of his particular aesthetic.

These authorial signatures are largely due to the fact that people can employ habitual ways of making "random" choices. One painter's palette may be reliably different from another's, not because the general style (or the subject-matter) dictates it, but because of idiosyncratic preferences. Likewise, as mentioned above, different composers (even those working in the "same" style) have distinct individual signatures--some of which are sufficiently systematic to generate new phrases that listeners will hear as being distinctive of that particular composer. With practice, personal preferences develop from habits into skills, so that one becomes increasingly adept in thinking (painting, sculpting, writing, composing...) in one's own particular way. There may be nothing to say, beforehand, how a certain person will choose to play the relevant game. But after several years of practice, their "random" choices may be as predictable as anything in the basic genre concerned.

As for more mundane examples of creativity, where the idea is novel to the person in whose mind it occurs but not to other people, these can sometimes be predicted--and even deliberately brought about. Socratic dialogue in general is designed to help people to explore their conceptual spaces in (to them) unexpected ways. But Socrates himself, like those taking his role today, knew pretty well what creative ideas to expect from his pupils.

Nor do we have to have heard of Socrates, to bring about such predictable effects. Suppose your daughter enjoys physics, but is having difficulty mastering an unfamiliar principle in her physics homework. You could do her homework for her, but that would not really help. She should have the idea herself. You might fetch a gadget which embodies the principle concerned, and leave it on the kitchen table, hoping that she will play around with it and realise the connection for herself. Even if you have to drop a few hints, the likelihood is that she will create the central idea.

In sum: we cannot predict creative ideas in detail, and we never shall be able to do so. The human mind, and human experience, is too richly idiosyncratic. But this does not mean that creativity is fundamentally mysterious, or beyond scientific understanding. Although much remains to be discovered, we already have some clearly expressible ideas about what conceptual spaces are, and how they can be explored and transformed. That is, we have some scientific understanding of how creativity is possible.

Previous Next Up Comments


I would like to thank Güven Güzeldere for suggestions and editorial assistance on this essay.

Much of this paper is closely based on passages in the author's book The Creative Mind: Myths and Mechanisms (New York: Basic, 1991).

1 See John Livingston Lowes, The Road to Xanadu: A Study in the Ways of the Imagination , 2nd ed. (London: Constable, 1951), and Margaret Boden, The Creative Mind: Myths and Mechanisms, ch. 6.

2 Boden, The Creative Mind, ch. 4.

3 See Annette Karmiloff-Smith, "Constraints on Representational Change: Evidence from Children's Drawing," Cognition, 34 (1990) 57-83.

4 See Annette Karmiloff-Smith, "From Meta-Processes to Conscious Access: Evidence from Children's Metalinguistic and Repair Data," Cognition, 23 (1986) 95- 147.

5 Edward Regis, Who Got Einstein's Office?: Eccentricity and Genius at the Institute for Advanced Study (Reading, MA: Addison, 1987) 83.

6 Alexander Findlay, A Hundred Years of Chemistry, 3rd ed., rev. Trevor. I. Williams (London: Duckworth, 1965) 39.

7 See Douglas Hofstadter, Fluid Concepts and Creative Analogies (London: Harvester Wheatsheaf, 1995); Melanie Mitchell, Analogy-Making as Perception (Cambridge, MA: MIT Press, 1993).

8 See H. Christopher Longuet-Higgins, Mental Processes: Studies in Cognitive Science (Cambridge, MA: MIT Press, 1987).

9 See H. Christopher Longuet-Higgins, "The Structure of Musical Aesthetics," Artificial Intelligence and the Mind: New Breakthroughs or Dead Ends? ed. Margaret Boden and Alan Bundy (London: The Royal Society and The British Academy, in prep.).

10 See Philip Johnson-Laird, The Computer and the Mind: An Introduction to Cognitive Science (London: Fontana, 1988); and Philip Johnson-Laird, "Jazz Improvisation: A Theory at the Computational Level," unpublished working- paper, MRC Applied Psychology Unit, Cambridge, 1989.

11 See Paul Hodgson, "Understanding Computing, Cognition, and Creativity," MSc thesis, University of the West of England, 1990.

12 David Cope, Computers and Musical Style (Oxford: Oxford UP, 1991).

13See Boden, The Creative Mind, ch. 7.

14See Boden, The Creative Mind, ch. 4.

15See James Meehan, "TALE-SPIN," Inside Computer Understanding: Five Programs Plus Miniatures, ed. Roger Schank and Christopher Riesbeck (Hillsdale, NJ: Erlbaum, 1981) 197-226.

16 See S. Turner, "MINSTREL: A Model of Story-Telling and Creativity," Technical Note UCLA-AI-17-92 (Los Angeles: AI Laboratory, UCLA, 1992).

17 See Robert Abelson, "The Structure of Belief Systems," Computer Models Of Thought And Language, ed. Roger Schank and Kenneth Colby (San Francisco: Freeman, 1973) 287-340; Margaret Boden, Artificial Intelligence and Natural Man, 2nd ed., (New York: Basic, 1987) ch. 4.

18 See Pamela McCorduck, Aaron's Code (San Francisco: Freeman, 1991). An article on Aaron by Cohen himself appears following this essay.

19 See Douglas Lenat, "The Role of Heuristics in Learning by Discovery: Three Case Studies," Machine Learning: An Artificial Intelligence Approach, ed. Ryszard Michalski, Jaime Carbonell, and Tom Mitchell (Palo Alto, CA: Tioga, 1983).

20 See Douglas Lenat and John Seely Brown, "Why AM and EURISKO Appear to Work," Artificial Intelligence, 23 (1984) 269-294; and Graeme Ritchie and F. Keith Hanna, "AM: A Case Study in AI Methodology," Artificial Intelligence, 23 (1984) 249-268.

21 See John Holland, Keith Holyoak, R. E. Nisbet, and Paul Thagard, Induction: Processes of Inference, Learning, and Discovery (Cambridge, MA: MIT Press, 1986).

22 Karl Sims, "Artificial Evolution for Computer Graphics," Computer Graphics, 25 (1991) 319-328.

23 For a fuller discussion, see Boden, The Creative Mind, ch. 9.

24 Oliver Sacks, The Man Who Mistook His Wife For a Hat (London: Duckworth, 1985).

25 See Karl Sims, "Artificial Evolution"; and S. Todd and W. Latham, Evolutionary Art & Computers (London: Academic, 1992).