Formal Theory of Fun & Creativity, Jürgen Schmidhuber

More on:

• Theory of Beauty » Facial Attractivness & Low-Complexity Art
• Google Search » Artificial Curiosity

[03:35]
Novel patterns: sounds like a paradox to me.
To compute R_int, two questions:
What is a pattern? Data that one can compress: raw data expressable by less data.
When is it novel? Degree of novelty = number of saved bits = Fun = R_int
Subjectively novel if compressor is able to improve through learning because of a previously unknown regularity.
E.g. the fractal formula that describes broccoli because it’s self-similar. A small piece of code computes into something elaborate.
There are zillions of patterns already known, so not novel, therefore not rewarding. not fun.
So, how do you recognize a novel pattern? By recognizing that the compression algorithm you use to encode or compress is not constant, not the same all the time, but it improves through learning. In this way, it becomes a better compressor.
E.g. saving three bits by an improved compression by learning is your intrinsic reward, R_int.
You want to maximize R_int.
Science, art, and humor are driven by R_int for finding and creating novel patterns.
Physics: create experimental data obeying a previously unknown law, some pattern that you previously did not know. E.g. discovering the law of gravity by ten falling apples. The law of gravity is an elegant piece of code or pattern describing any falling object in a very terse way. [Boy, what fun.]
Music: find or create regular melody and rhythm not exactly like the others. Find something unusual but regular, known but new. [Again, the paradox shows up. Seems like Fun & Creativity have to do with paradoxes (and koans?).] So, it’s not random, but familiar and new.
Same in visual arts. Visual arts is about novel patterns. Make regular but non-trivial patterns.

[09:20]

To build a creative agent that never stops generating non-trivial & novel & surprising data, we need two learning modules: (1) an adaptive predictor or compressor or model of the growing data history as the agent is interacting with its environment, and (2) a general reinforcement learner.

The LEARNING PROGRESS of (1) is the FUN or intrinsic reward of (2). That is, (2) is motivated to invent interesting things that (1) does not yet know but can easily learn. To maximize expected reward, in the absence of external reward (2) will create more and more complex behaviors that yield temporarily surprising (but eventually boring) patterns that make (1) quickly improve. We discuss how this principle explains science & art & music & humor, and how to scale up previous toy implementations of the theory since 1991, using recent powerful methods for (1) prediction and (2) reinforcement learning.

Two learning modules:

1. A learning compressor of the growing history of actions and inputs. Compression progress == Fun == Surprise.
2. A reward or maximizer or reinforcement learner to select actions maximizing Expected Fun == R_int (+ R_ext).

[this compression stuff relates to qualia]

Whatever you can predict, you can compress.

B(X, O(t)) = –#bits needed by O’s compressor to quickly (in C steps) encode data (MDL: Minimum Description Length).
O’s compressor improves at t: R_int(X, O(t)) ~ B(X, O(t)) – B(X, O(t – 1)) = Fun =Reward for Actor.

When things (suddenly) become better explainable, expressable, compressable (needing fewer bits to encode the same thing than before), you’re having fun and create art.

You are after the first derivative: the change or delta end the compressability.

16:15
Self-similar ‘femme fractale’ based on a very simple arc-drawing program.

Draw a recursive set of circles, and remove circles or arcs until you are left with a pattern that you already know, i.e. that makes sense to you, that has meaning in your realm or world.

Recursive application of harmonic proportions.
Create a graphical structure through Fibonacci rations nd the Golden or Divine Proportion.
Many artists claim the human eye prefers this ratio over others.
The ratios of subsequent Fibonacci numers are: 1/1, 1/2, 2/3, 3/5, 5/8, 8/13, 13/21… converge to the proportion 0.5 √??? = 0.618034 such that A/B = B.

Jokes (are very simple)
Every joke has a punch line—unexpected if you don’t already know the joke.
If you don’t know the joke, you need extra bits to predict the thing.
So, you predict something, but it’s not what you predicted, so you need extra bits to store the stuff that you did not predict well.
However, all the jokes are designed to relate the punch line back to the previous story in a way that human learning algorithms easy pick out. So the human learning algorithm kicks in and finds out that the punch line is related to the previous story (history) in a way that allow you to save bits, because all you need to do is to insert a pointer to the previous part of the story such that you can get rid of the extra bits that you didn’t know and didn’t expect and save bits! # of saved bits is R_int, the fun bit, the reward. That is what you are trying to maximize.
What reinforcement algorithms do is try generate more data which also has the property that it is fun.
Learning from previous things to predict what other things in the future might be possible which may not be novel anymore after that.

Recently I gave this talk to the king of Spain. Or he said he was the king of Spain. He said, if you are a scientist, then I am the king of Spain.

For artificial scientist, or artists: Which algorithms do we use for compressing the growing data history? Which actor/reinforcement learning algorithms and data selectors do we use to experiment and invent the future?

Learning algorithms (predictors of novel patterns) improve all the time. They constantly get better at getting better.

22:00

Whatever you can predict well, you don’t have to store, so you can compress it.

Likewise, what you cannot predict well, needs extra storage space.

So learning or prediction progress creates intrinsic rewards (fun!). This sometimes helps speed up external rewards as well.

Neural nets are underestimated! Our fast deep nets just broke old handwritten digit recognition world records.

Deep neural nets with many layers, pre-trained. [software is executable knowledge, wisdom even]

You retrain these nets under supervision, but all of this is unnecessary.

Dan Ciresan, Üli Meier and Luca Gambardella are working on swarm intelligence, deep layers, etc.

No unsupervised training is necessary to achieve these results.

Resilient machines, robots, learn, update world model based on failed expectations, optimize, plan in model, execute. Continual self-modeling by Alex Förster. Damaged robot cannot drive anymore, learns neural self-model to detect this and self-heals.

Feed forward networks will probably not be good at compressing history.

You can use Recurrent Neural Networks to fix that.

RNNs are good at minimizing the difference between what you can predict and what actually happens. You can use them as time series predictors.
Then you can create a gradient program or algorithm space, you minimize, you compute the first derivative of your wish.
You get a space of programs [with RNNs] of which you can select a better program to predict the future so your wish may come true.

Alex Graves is now beating all connected handwriting recognition competitions using LSTM-RNNs.

No pre-segmented data; RNN maximise probability of training set labels. Much better than single digit recognition.

Alex also won all the Arabic handwriting recognition contests, despite the fact that he doesn’t understand Arabic.

RNNs capture patterns (frequently occurring combinations) to predict future events and self-correct them.

Use these patterns again and again to predict based on different input sequences.

Symbols and self-symbols are just natural by-products of historical compression. Symbolic computation.

Frequent observations are best—also neurally— encoded by storing prototypes.

One thing is experienced again and again as the agent interacts with the world: the agent itself.

R_int = Measure the relative entropy between priors and posteriors of 1) before and after learning.

Why are you looking here rather than there? Because here is some learning opportunity that you don’t find there. This is used to much improve eye-movement predictions.

I bet that at the end of program “while not condition do begin … end” the contents of adderss 23721 will be 5.

2002: left & right brain symmetric experiments: prediction of the outcome of experiments.

Both left and right brain can predict the outcome and disagree. Once the agree to disagree, they can run the experiment. They can bet, and one will be the—surprised—loser and one will be the winner. Winner sometimes gets copied over loser. The loser has to pay the winner.

One has a better model than the other.

The intrinsic reward is essentially moving form one guy to the other.

The sum of intrinsic rewards is always zero—what one gains, the other loses.

Each one is maximally motivated to create data that comes up with an experiment that the other finds attractive to the other but where both predict a different outcome.

If they both predict the same result, they are not interested in the experiment because there is nothing to learn, there is no fun.

Only when the predictions differ, you have the incentive to run the experiment, in order to have fun and learn.
That’s why the increase their repertoire of pluralistic programs, learning and building on the previously learned repertoire [and have fun!]

[So, different opinions are actually very valuable! They hold the option to create something new, to innovate, and to progress while having fun, provided you agree to disagree and do the experiment (TRIAL RUN or PROEF OP DE SOM).]

Just like a little baby that focusses on tiny bits to make progress.

The fun of improving motor skills. Why should anyone learn juggling? Learning motor skills is also driven by the desire to create novel patterns. Juggling is fun because it is regular and ypu get a lot of reward because it leads to a novel way of behaving and saves bits and adds to your existing patterns.

[Patterns == behavior; novel behavior -> change; change is fun!]

Intrinsically motivated cumulatively learning robots.

Pieter Adriaans: beauty not very compressible, not incompressible, but somewhere in between.

Pleasingness is dependent on the observer: the amount of compression (the first derivative) is a measure for pleasure. That is why some things are pleasurable for some but not for others. It’s an inverse hyperbolic curve with a maximum. The maximum is (personally) subjective. And it is a dynamic thing—what once was interesting and fun, becomes standard and boring; you evolve.

The theory of humor is also based on unexpectedness. Why is something funny? Because it is unexpected, and you find a quick way to incorporate it as a new pattern, a new way of doing things. The fun part is in the progress of learning. Humor takes something that is incompressible at first, but then recognizes its compressibility and learns that into a new habit or trait. You save the bits. So the first derivative is the crucial thin.g It’s not the compressibility itself. It’s the dynamic process that leads to learning before and after compressing.

[So, life is a constant stream of novel patterns you experience. Life is a pattern generator, where a new pattern is only incrementally different from previous patterns.]

Q: So, why is everone still going to the Louvre to see Mona Lisa while you can see the same picture over and over again on the web?
A: Because the sensory experience of actually going to the Louvre creates a whole stream of novel patterns.

[This is why re-reading a book after a few years is also fun: your paradigm has changed, and therefore re-reading a book creates novel patterns.]

Consciousness matters™—and that’s all that matters

En dat kan je dus heel letterlijk nemen. Zelfs het woord heel uit de vorige zin krijgt daardoor voor mij meer waarde.

Resoneert enorm met het gedachtegoed van Christopher Alexander en Capra’s De eenheid van leven.

Een paar highlights:

• Leonardo zag zichzelf als een ‘tolk tussen de natuur en de mens’ en zijn wetenschappelijke creativiteit—een combinatie van hartstochtelijke intellectuele nieuwsgierigheid, monnikengeduld en experimentele vindingrijkheid—bleef zijn leven lang zijn belangrijkste drijfveer.
• Voor Leonardo was schilderkunst zowel een kunst als een wetenschap: een wetenschap van natuurlijke vormen, totaal verschillend van de mechanistische wetenschap die later in de westerse beschaving zou ontstaan.
• Hij trok vaak vergelijkingen tussen de anatomie van de mens en de structuur van de aarde.
• Toch lijkt zijn uitspraak vooruit te lopen op de huidige Gaia-theorie—een wetenschappelijke theorie waarin de aarde wordt beschouwd als een levend, zichzelf organiserend en regulerend systeem.
• Leonardo’s uitzonderlijke vermogen om waarnemingen en ideeën uit verschillende disciplines met elkaar in verband te brengen vormt de essentie van zijn benadering van kennis en onderzoek.
• Leonardo was diep onder de indruk van de enorme diversiteit en variatie in levende vormen. ‘De natuur is zo prachtig en rijk aan variatie,’ schreef hij in een passage over het schilderen van bomen, ‘dat er onder bomen van één en dezelfde soort niet één te vinden zou zijn die op een andere er vlak naast lijkt, en dat geldt niet alleen voor de plant als geheel, maar ook voor de takken, de bladeren en de vruchten: er is er niet één precies zoals een ander.’—En toch lijken ze allemaal op elkaar, net zoals de parels in de pareltaal.
• Leonardo had niet alleen aandacht voor de variaties binnen één soort, maar ook voor overeenkomstige organische vormen in verschillende soorten en voor overeenkomstige patronen in verschillende natuurverschijnselen.
• Leonardo kon natuurlijk niet over dit soort verklaringen beschikken, maar hij constateerde wel terecht dat bij de hele schepping (of evolutie, zoals we nu zouden zeggen) van de enorme diversiteit aan vormen de natuur telkens gebruikmaakte van dezelfde organisatiepatronen.—net zoals pareltaal bedoeld is.
• Leonardo wilde de natuur niet overheersen, in tegenstelling tot Francis Bacon, die dat in de zeventiende eeuw voorstond, maar hij wilde er zoveel mogelijk van leren.—biomimicry dus.
• In zijn ontwerptekeningen voor villa’s en palazzi besteedde Leonardo extra aandacht aan de bewegingen van mensen en goederen door het gebouw, en maakte hij gebruik van het stofwisselingsproces als metafoor in zijn bouwkundige ontwerpen. Ook de tuin beschouwde hij als een onderdeel van het gebouw en hij probeerde altijd architectuur en natuur met elkaar te verenigen.
• Leonardo gebruikte natuurlijke processen als voorbeeld voor door de mens gemaakte ontwerpen en gaf de voorkeur aan samenwerking met de natuur boven overheersing.
• Deze mentaliteit van waardering en respect voor de natuur is gebaseerd op een filosofische visie waarin de mens niet los staat van de rest van de natuur, maar wezenlijk is ingebed in de gehele, complexe biosfeer en daarvan afhankelijk is. Deze filosofie wordt tegenwoordig gepropageerd door een filosofische school die be­kendstaat als deep ecology.
• In tegenstelling tot veel tijdgenoten had hij het zelden over Gods schepping, maar sprak hij liever van de oneindige werken en schitterende uitvindingen van de natuur.
• De Aantekenboeken staan vol passages waaruit je een indruk krijgt van Leonardo’s respect voor de oneindige creativiteit en wijsheid van de natuur.
• Ze zijn niet in religieuze taal vervat, maar zijn toch intens spiritueel.
• Wetenschappers zien het universum niet meer als een uit elementaire bouwstenen opgebouwd mechaniek, maar hebben ontdekt dat de materiële wereld uiteindelijk een netwerk is van onlosmakelijk met elkaar verbonden patronen van relaties, dat de aarde als geheel een levend, zelfregulerend systeem is.
• De evolutie wordt niet meer beschouwd als een strijd om het bestaan, maar als een coöperatieve dans waarin creativiteit de drijvende kracht is.
• Daarom blijven de wetenschap en de kunst van deze grote denker uit de Renaissance een inspiratiebron voor deze tijd.

Stumbled across a wonderful website yesterday with a lot of awseome and open source Flash applications that produce wonderful graphics, many of them fractal, complex. Check out Levitated.net—a friendly company.

Blijkt dat de term ‘planning’ behoorlijk verschillende betekenissen voor iedereen heeft, en menig mens er bijna een trauma aan over heeft gehouden. En dat met name van de strakke, dogmatische, volg-de-planning-of-ik-schiet versie die met ijzeren hand gevolgd dient te worden.

Tijd dus om ‘planning’ nieuw leven in te blazen en te ontdoen van haar negatieve lading.

Voor mij betekent planning niets meer en niets minder dan:

Planning—Het aanbrengen van net genoeg spanning om het een constructieve richting te geven en voldoende vrije ruimte voor creativiteit en persoonlijke inbreng.

In Japan noemen ze dat Hoshin Kanri—hoshin betekent kompas en geeft richting; kanri betekent management of controle.

Dus een lange termijn visie gecombineerd met wendbaar ontwikkelen onder voortdurend voortschrijdend inzicht zodat je continu bij kunt sturen. Want zelfs de veranderingen veranderen.

Zodra je heldere afspraken maakt over het te volgen proces dan staat het hoe muurvast, maar wat je er mee kunt maken is volledig open…

Oh ja, vergeet niet om dat muurvast proces zich ook te laten evolueren onder voortschrijdend inzicht…

Graaf dieper op wat werkt wel en risicoloos resultaat en rendement.

telekommunisten is controlled by it’s workers and committed to staying that way, we believe we can serve our customers best and at the lowest cost by remaining focused on meeting the needs of our employees and customers, not on profits for outside shareholders. Being worker-owned means that all the money you spend on our products goes directly to the maintenance and improvement of the service you receive.

Venture Communism

Venture Communism is a project to research, experiment with, and initiate social enterprises, particularly a rent-sharing based proposed form called a Venture Commune. The project also encourages dialogue and investigation of alternative economies and workers struggle against private property .

The goal is to promote understanding of the relationship between distribution of productive assets, political power and global exploitation and poverty and to investigate how worker-controlled enterprises can contribute to solving this problem, as well as to put such ideas into practise.

Several articles have been written and many on-line correspondences sent describing and discussing venture communism, as well as analyzing the nature of property generally.

Source: Telekommunisten—The Revolution is Calling.

Thank you Richard Dawkins, for writing your “The God Delusion” for it creates enormous attention for this ever important subject. You make people like Chopra and myself dig deeper into consiousness, the universe, quantum physics, God, Allah, Buddha and more. We’re becoming more conscious and knowledgeable about it.

And it’s about time, since we’re rapidly evolving towards the noosphere, catalyzed by the technosphere with the Internet as its most brilliant example at this time.

Spaceship Earth is entering its most risky trajectory when metamorphosing to its next order—extreme emergence. And just like the life-threatening phase during the metamorphosis from caterpillar to butterfly, albeit on a very large, global, scale.

May we all evolve into a global, and perhaps universal wisdom and happiness. And beware, these massive scale changes or emergence happens relatively abrupt. Buckle on.

1. Hot teams are highly dedicated to end results and are enthusiastic
2. They thrive on ridiculous deadlines and high hurdles to take
3. They are irreverent and non-hierarchical, playing around and having fun
4. They are made up of widely divergent disciplines and abilities
5. They use an open and eclectic workspace
6. Hot teams connect to the outside world and look for solutions outside themselves

(Via CrowdsourcingDirectory.)

Als we dat nou eens naadloos kunnen samenvoegen met grote bedrijven op dit gebied. Een organisatievorm vinden waarbij beiden elkaar versterken. De groten zijn zeldzaam, veerkrachtig, geworteld en enorm. De eenpitters, tweepitters en gasstellen zijn met velen, klein, wendbaar en innovatief. Hoe kunnen ze elkaar versterken?!

Hey Sanne, Daan, Ik wil meedoen! Gaan we in augsutus om de tafel om dit verder vorm te geven?

Mijn gedicht “Wanneer Dan” vat het geheel samen.

Brad Neuberg on Inventing with Values:

A few months ago, my friend, BradNeuberg, gave the keynote at Yahoo’s internal Front-End Engineering Summit. The video is now online, and it’s worth watching. Brad’s not only a great guy and a great hacker, he’s an excellent speaker. He speaks from the heart with intensity and good humor. ” (ME6)

I want to highlight four things Brad said: ” (ME7)

• A good way to approach invention is, ‘I Against I.’ (The name comes from a classic punk song from the mid-80s.) Don’t worry about what others will do. Instead of trying to protect your ideas by putting up walls, compete against yourself. If you’re going to be put to pasture by somebody, it might as well be by you. And as Brad points out, no ideas are safe. In this day and age, you will be put to pasture. ” (ME8)
• Brad’s section on values was tremendous. He cited two techniques for clarifying values. The first was ‘Mob Rule.’ If you were the only thing between an angry mob and some other thing, what thing would cause you to stand your ground? This reminds me of something AlanDershowitz said in the best class I ever took in college. Dershowitz said that he was willing to die for freedom of speech. I was astounded by that statement, and it made me think about the values that I was willing to die for. ” (ME9)
• The second technique was, ‘The Last Speech.’ Imagine you were about to leave your company and that you were asked to deliver a final speech. What would you say? This actually happened. A decade ago, JamesGosling was fed up with Sun and decided to leave. ScottMcNealy asked Gosling to write a letter before he left, explaining what he would do differently if he were in charge. Gosling outlined a vision which ultimately led to him staying at Sun and creating Java. ” (MEA)
• Brad ended his section on values with a story about ArchieRand, the famous expressionist painter who was one of Brad’s professors at Columbia. Rand used to walk around the room as his students worked, look at them with his eyes blazing, and say, ‘This shit matters.’ ” (MEB)

Each of these things are worth thinking about and trying. ” (MEC)

(Via EEK Speaks.)

Uhmpf… awful lot of creative crowd work going on. A sheer infinite flurry of innovation sources. Howcome I only see this now? Wonderful! Diving into it because I will find the fertile ground where my, sometimes wild and big ideas, are valued and will flourish.

Thanks for the pointer Carl!

P.S. Appears that Carl sends email from the Innovation Factory. Another interesting company that I only now discoverd.

P.S. 2 Seems they’re working hard to realize Creative Crowds and Creative Capitals