Creation is a predictable, controllable process that can be engineered. Tim Menzies tim@menzies.us Based on the success of the software engineering, I want argue that creativity can be engineered and, routinely, is done so in a commercial setting. To do so I must demythologize "creation". Rather than focus obsessively on that "a ha!" moment when the world spins on a dime and the new paradigm falls like an apple in Newton's garden, we should look at both (a) the conditions that preceded creation and increases the probability of finding shiny new apples; and (b) the work that must follow the "a ha!" moment. To any individual, creation may feel like a special process but, as Hari Seldon will one day observe, given a large enough community, then the outputs of that communities creativity and individuality can be predicted. As commented by the Australian Band, the Whitlams in the their song "one in a million: Some say, "love it comes once in a lifetime," That's enough for me She was one, one in a million There's five more in New South Wales. (Of course there is a minus-one error in this song. As of June 2007 the Australian state of New South Wales has a population of 6,889,100 so now there are nearly seven million candidate "loves of a lifetime", less the "was one" above, leaving six other candidates for life long soul mates.) As with song, so to in industry. Robert Glass (2007) writes that software creation is a highly creative process. Yet in the software industry, it is possible to predict the time required to complete a creative processes such as those required to develop new software. In my other life as a computer scientist, I build predictors for the time required to complete software as well the number of bugs expected in the final product, see Menzies (2006,2007). This is important since the whole point of engineering is knowing how to surf cost-benefit curves. In a mature engineering discipline, it is possible to predict the quality of some future product based on trade-offs between process, product, and personnel decisions. For example, here are the standard figures showing the relative importance of different factors on the effort (in months) to develop software. 1 Personnel/team capability 3.53 *********************************** 2 Product complexity 2.38 *********************** 3 Time constraint 1.63 **************** 4 Required software reliability 1.54 *************** 5 Multi-site development 1.53 *************** 6 Doc. match to life cycle 1.52 *************** 7 Personnel continuity 1.51 *************** 8 Applications experience 1.51 *************** 9 Use of software tools 1.50 *************** 10 Platform volatility 1.49 *************** 11 Storage constraint 1.46 *************** 12 Process maturity 1.43 ************** 13 Language & tools experience 1.43 ************** 14 Required dev. schedule 1.43 ************** 15 Data base size 1.42 ************** 16 Platform experience 1.40 ************** 17 Arch. & risk resolution 1.39 ************** 18 Precedentedness 1.33 ************* 19 Development mode 1.32 ************* 20 Developed for reuse 1.31 ************* 21 Team cohesion 1.29 ************* 22 Development flexibility 1.26 ************* Note that, as we might expect, individual capability is the single most important factor (see personnel/team capability). Which is not to say that combinations of other factors can't still dominate. For example, the benefits of using the most capability staff (3.53) are out-weighed by building an over complex product is too short a period of time ((2,38 * 1.63 = 3.88) > 3.53). Along with many other researchers, I've shown that predictive models based on the above numbers are surprisingly accurate- which would be impossible if the creation process was as individual and idiosyncratic as is commonly perceived. Software engineering offers us other principles for engineering creativity. Some works of creation contain ambiguities- nuances that can engage and delight the viewer. These ambiguities are to be encouraged. Other ambiguities are not so benign. As soon as the creative piece executes then, before the viewer can enjoy the piece, the compiler must approve it. And compilers can be tricky. Imagine that your pen and paper argue with you saying "no sir, i don't like it. I don't like it at all" then stops you from mailing off your poem to a publisher because it won't compile, because it would interface with the commonIdeas (tm) v3.1 or because to distribute your poem you need newInk v2.1 and the company that marketed that one has gone out of business or has declined to port that library to mac os/x version 11.3. And if compiler or licensing errors don't get you, never under-estimate the power of your own stupidity. - "Debugging had to be discovered. I can remember the exact instant when I realized that a large part of my life from then on was going to be spent in finding mistakes in my own programs." - Maurice Wilkes 1949 Wilkes is point out that the time required to debug a system is alarming- over half the effort of building. According to Brooks (1995), the time required to build software divides as follows: - 1/3 in management and planning - 1/6 in development - 1/4 in unit test <== testing all parts in isolation - 1/4 in system testing <== testing all parts, in combination So we might focus of creativity in writing software (which is 18% of the task) or we might look at the remaining 82% of the work. Brooks is cautioning us that we need to look beyond just the writing phase of software. Many factors influence the environment around that writing. In several books, including "The Flight of the Creative Classes", Florida (2005) lists the social conditions that attract highly creative people to some local region. His triad is "technology, talent, tolerance". A moderately Bohemian environment, says Florida, is a necessary but not sufficient condition to attract talented software hackers to,say, Silicon valley. There, in an environment tolerant of new ideas, this talent can create innovative new technologies. Our 100 hour per week software hacker may never actually go an art gallery, lesbian bar, or join a drum circle but they want to feel that might be able to, if only they can get the next version of the software out the door. Yet another advocate of engineering creativity is the international open source movement. The open source world takes the complete opposite approach to Florida. Rather than collect creative people in one place to work on a product, they take the product and ship it around the world to whoever and wherever creative extensions might occur. "Release early, release often" is the mantra amongst the open source community. Get parts of it going, get it out there, exploit the serendipitous effects seen when large groups use a new technology. Says Eric Raymond (2001), "The next best thing to having good ideas is recognizing good ideas from your users. Sometimes the latter is better." Strive for creative alternatives, says Raymond: "Any tool should be useful in the expected way, but a truly great tool lends itself to uses you never expected." The open source experiment is both new and old. New, in the sense that it is remarkable that large, robust, innovative, complex software systems can be build by geographically disperse teams from across the planet. But old in the sense that it is well established that creation is a community process. The patent clerk Albert Einstein did not invented relativity in a shed on an isolated farm in the back woods of Tasmania (as proposed by the great Australian philosopher, Yahoo Serious). Rather, Einstein spent years reading everything he could of 19th century physics. And it was his detailed understanding of flaws with the old ideas that led him to conceive the new. So to create the next generation of creative thinkers, herd the newbies to university where they can study the oldies. "If you want new ideas, read old books" advises Ivan Pavlov. If the British poet Ted Hughes was here, he'd object to this notion that creativity can be engineered, its products predicted in any precise quantitative sense, and that we can train a team to be some percentage more creative. In his 1957 poem "The Thought Fox", the paw prints of the thought fox are the marks left behind on a page after that elusive creature has run buy. This thought process is out-of-control to the creator- some primordial force that we cannot command I imagine this midnight moments forest: Something else is alive Beside the clocks loneliness And this blank page where my fingers move. Through the window I see no star: Something more near Though deeper within darkness Is entering the loneliness: In Hughes' view, we must wait patiently for the muse to arrive. No crowd sourcing, no storage of the poem in an on-line version control system, no developers releasing early versions of the poem for others to elaborate. In Hughes' world, the best we can do is to coax creativity into creation- not by action, but by our stillness: Cold, delicately as the dark snow, A foxes nose touches twig, leaf; Two eyes serve a movement, that now And again now, and now, and now Any wrong move on our part and the fox will bolt. We are frozen, like a hunter waiting for a clear shot or a rabbit trying not to draw attention to itself. Richard Florida would not approve of Hughes' imaginary landscape- a cold, isolated and lonely place with no art galleries or bookstores to visit after the poem is written. Across clearings, an eye, A widening deepening greenness, Brilliantly, concentratedly, Coming about its own business In Hughes' world, the artist is like Leonardo DeCaprio watching the in-coming iceberg. We are powerless to bend its business to ours. There are no start-ups we can spin off to explore different poem generation methods. We can't write multiple poems and throw away the one the don't work. All we can do is wait for a wham-bam visit by the muse, running faster than a clock can tick, leaving us reaching for cigarettes. Till, with a sudden sharp hot stink of fox It enters the dark hole of the head. The window is starless still; the clock ticks, The page is printed. One can imagine Clint Eastwood playing the Thought Fox, arrogantly gazing at us sweaty and anxious across the clearing, daring us to try anything else except to freeze in our tracks, interrogating us with "You've got to ask yourself one question: 'Do I feel lucky?' Well, do ya punk?" Of course Ted Hughes wants to convince us that creation is special, the exclusive property of a special breed of poets, like himself. Heh, we all need to justify our salary somehow. But even the Thought Fox process (which we might characterize as "please go shiver in the snowy forest of your subconscious till the muse lowers itself to deposit a creative turd in your path") is open to engineering principles. If: - the average wait time for the muse is M days - the number of required poems is P poems of Quality Q per day - and better and better quality poems are less and less likely (and some exponentially decay constant k) - then the number of starving artists you need to go shiver in the forest is... Well, you get the idea. Creation can be engineered. We know that the probability of creation at any instant at any place, by a single person is very low. But decades of software engineering has taught us that given the right social conditions and the time and the tools needed to explore a set of ideas, creation is a predictable, controllable process that can be engineered. References Brooks (1995), F. Brooks, "The Mythical Man Month", Addison-Wesley, 2005. Floirda (2005): R. Florida, "The Flight of the Creative Class: The New Global Competition for Talent". Collins Books. Glass (2007): R.L. Glass, "Software Creativity 2.0" developer.* Books, 2007 Menzies (2006) T, Menzies and Z. Chen and J. Hihn and K. Lum, "Selecting Best Practices for Effort Estimation", IEEE Transactions on Software Engineering, Nov, 2006, Available from http://menzies.us/pdf/06coseekmo.pdf Menzies (2007): T. Menzies and O. Elwaras and J. Hihn and M, Feathear and B. Boehm and R. Madachy, "The Business Case for Automated Software Engineering", IEEE ASE, 2007, Available from \url{http://menzies.us/pdf/07casease-v0.pdf Raymond (2001): E. S. Raymond and B. Young, "The Cathedral and the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary", O'Reilly & Associates, 2001. A short form is available from http://www.tuxedo.org/~esr/writings/cathedral-bazaar/cathedral-bazaar.