Neural creativity

Introduction to neural learning with SuperMemo

SuperMemo 17 introduces a new form of incremental learning, which we will call neural learning. In neural learning, SuperMemo serves the student with knowledge associated with a selected subject, article, picture, etc. Knowledge is served in a neural way, i.e. by sequentially exploring semantic connections in a networks of information build automatically or manually in SuperMemo. SuperMemo takes inspiration from how the brain works. Creativity enhanced with SuperMemo is very slow, but it is targeted and methodical. Neural creativity works like a slow brain extended by memories stored in SuperMemo. Neural learning is particularly useful in research, problem solving, creative writing, explorative learning, etc. In other words, you may not find neural learning particularly useful in mastering a course of physics, however, if you are a physicist working on a complex problem that requires associating knowledge from different fields of physics and beyond, you will notice that this form of learning has a fantastic impact on your creative productivity.

Neural learning was the main reason for SuperMemo 17 to introduce a new type of elements: concepts. A concept in SuperMemo is an equivalent of an important idea, which can be connected with other elements (incl. other concepts). In addition to connections between concepts, neural learning also explores connections between elements using the knowledge tree. Elements connected via concepts form concept maps, which, in addition to the knowledge tree, from the primary skeleton for explorations in neural learning.

In the following sections, the essential terms and notions underlying neural learning will be introduced gradually: concepts, spreading activation, neural review, etc. Even if this may sound overwhelming at first, remember that you can start neural learning now with just one keystroke: Ctrl+F2 (Go neural). The following sections are only to explain how neural learning works and how its mechanisms can influence your decisions in making it work best for you.

If you are not new to SuperMemo, and want to skip details, terminology, etc, you can jump directly to practical applications in: Neural review

Concepts

A concept in SuperMemo is an element associated with an important idea.

A concept can serve as a bookmark, as a root of a branch associated with a given subject, as a hub for connecting different portions of a collection, or as a starting point for a creative process (e.g. writing an article, solving a problem, etc.).

Concepts can be linked with other elements. The network of links based on concepts will be called a concept map.

See: Concept maps at Wikipedia.

Concepts: Basic operations

Here are the most useful operations you will need when working with concepts:

• To convert an element into a concept, use Concept: Make in the Commander.
• To convert a contents element into a concept, press Ctrl+K in the Contents window.
• To link an element with the concept:
  1. Use Concept: Link in the Commander
  2. in the concept registry, in the left member pane, select the concept
  3. click Accept at the bottom of the registry window to create the link, and close the registry
• To link an element with another element:
  1. use Link: Contents in the Commander
  2. in the Contents window, choose the element
  3. click Accept at the bottom of the window (or press Enter)

Concept terminology

The term concept is used in four main contexts in SuperMemo 17:

concept element

element used to build a portion of a concept map (e.g. an idea, or an article from Wikipedia, etc.)

concept member

member of the concept registry used in concept housekeeping (esp. concept search uses when building a concept map)

concept group

branch of elements rooted in a concept. This is similar to category known from older SuperMemos. For performance reasons, elements in the concept group are not registered with a concept. Even though concept groups are used in spreading activation in neural creativity the links formed by parent-child-sibling relationships are taken as only carrying minor priority (in comparison to concept-concept, concept-element and element-element links set by the user)

concept map

interlinked graph of elements used as the skeleton for spreading activation. It may also be used in reference to its diagrammatic representation

Concept groups

When you work with a collection which includes material belonging to different subjects, you may want to keep different subjects in different sections of the knowledge tree (in the Contents window). You may also want to design a different look for items related to literature, and a different look for questions related to mathematics. For example, you could use pictures to illustrate your anatomy branch, sounds to illustrate your foreign language section, spell-pads to make your spelling section work best, etc.

To help you organize different subjects in one collection, SuperMemo uses concept groups. A concept group can be seen as a tool for building a section of the knowledge tree. One concept group is used for adding material related to one subject. Elements added to a given concept group may use their own template that determines their look (e.g. images, sounds, colors, fonts, sizes of text fields, number of shape components, etc.). See: Using templates.

In older SuperMemos, the role similar to concept groups was played by categories.

Concept registry

Concept registry is used to keep the sorted list of concepts. Each concept element is associated with a concept registry member. Concept registry makes it easy to find concepts, esp. when you want to link an element with a particular concept.

In the registry, each concept also shows a set of properties that can be used in managing a concept group.

Each concept registry member makes it possible to change the following properties:

• root - the branch that is the parent of all elements belonging to a concept group
• hook - the branch to which new elements are added when adding to a concept group (hook is a descendant of the root)
• templates - templates used in the concept group
• other attributes (e.g. the default priority of elements added to a concept group)

Choose Search : Concepts from the main menu to open the concept registry.

Neural review

Neural review is a subset review that follows meaningful connections between pieces of knowledge. It takes inspiration from how the brain follows associated ideas. Neural review automates the review of knowledge associated with selected concepts or ideas, e.g. in problem solving, creative writing, etc.

For example, in a neural review of dogs, there is a good chance of learning about puppies or pooddles, but there is also a chance of learning about cats and somewhat lesser chance of learning about cars. Similarly, if you see a picture of a banana plant, going neural on that picture will help you review associated elements (e.g. other pictures of plants, or articles about bananas, or elements associated with the same concept such as gardening).

A concept in SuperMemo is an important idea or subject. Concepts are a new type of element that can be linked with other elements. The network of links based on concepts will be called a concept map. Elements can also form links between each other. In SuperMemo, knowledge is also connected by inheritance, and the structure of the knowledge tree.

Spreading activation in neural networks is a method of searching the network with the use of network connections and weights.

Neural review follows the spread of neural activation in a network of inter-element connections. All the links and connections in SuperMemo can be used to simulate a spread of activation in the network of knowledge where individual elements play a role of neurons.

All links used in spreading activation receive weights based on their importance. In SuperMemo, weights are determined by element priority and the priorities of individual knowledge associations (e.g. concept connections are of higher priority than the sibling relationship in the knowledge tree).

In spreading activation, we can sequence the propagation wave by the time of activation of individual nodes, which are formed by SuperMemo elements. Such a sequence can then be used as the basis of a semantic review in the incremental learning process.

Neural review can be initiated by executing Learn : Go neural (Ctrl+F2) from the main menu.

Neural review carries a degree of randomness. As such it can be very helpful in associating ideas in neural creativity. Connectivity and randomness underlie the main rationale behind neural review: neural creativity and neural problem solving. In those approaches, we combine a creative process or a problem solving process with an exploration of new subjects, while executing incremental learning in the background. In other words, neural review serves learning that is to instantly bring benefits in the shape of new ideas and new problems solved in conditions of incomplete human knowledge and in combination with abundant and possibly contradictory computer-stored knowledge. A typical use of neural creativity would be to diagnose a rare disease on the basis of its symptoms (see examples).

We will use the following terms:

neural review

incremental learning that follows the spreading of activation in a conceptual network

neural creativity

creative process based on neural review (e.g. creative writing)

neural problem solving

process of neural creativity targeted at solving a specific problem (e.g. solving a forensic case)

Neural review starter

You can try neural learning on your own now.

Pre-conditions

All you need is SuperMemo 17 and a rich collection. Small collections won't show the power of neural learning. If you start with a single article, neural review makes sense only if your article has already been splintered into smaller pieces. You can also do Split article if this is a newly imported article, however, in that case your sub-articles need to be outstanding. This is because neural review ends if the neural queue contains elements that have all been reviewed/imported/created on that given day. This means that for a newly split article, you need to use Add to outstanding or wait a day (for sub-articles to "mature" for review).

Start

Choose any of the following:

• when you see an element of interest, press Ctrl+F2 or choose Learn : Go neural on the main menu
• view a concept in the concept registry and press Alt+N, or choose Go neural on the registry menu, or click Neural at the bottom of the registry window
• view any member in any registry and press Alt+N, or choose Go neural on the registry menu, or click Neural at the bottom of the registry window
• open any subset in the browser and press Ctrl+F2 or choose Go neural on the browser menu

Review

While in neural review, use the button Learn as in normal learning. While learning, the Review parameter in the Statistics window shows Neuro=[Size], where Size is the number of elements in the neural queue. The size of the queue is limited for practical reasons. You are not likely to go beyond a few hundred elements in a single review. Moreover, at each repetition, more layers of the conceptual networks are explored and the queue keeps being expanded in parallel with your learning.

Stop

To end the review, press Esc. The Review field in the Statistics window is cleared.

Costs of neural learning

If you worry about the cost-vs-benefit equation in building concept maps in SuperMemo, remember that properly executed incremental learning builds a semantic structure of the processed knowledge in the shape of the knowledge tree. Parents, siblings and children are related to each other in terms of their meaning. Concept maps are the chief framework for spreading activation in neural SuperMemo, but the knowledge tree also forms a fine dendritic framework that helps you capitalize on all your prior work with incremental learning. If you are familiar with mind maps], you will be happy to know that the cost of building your concept maps is far less. At times, all you need for your creative purposes is a conceptual link between two different subjects where you expect to find creative associations. To execute your neural review or to engage in neural creativity, pick your concept, element or a registry member (e.g. a picture) and Go Neural.

Concepts and inter-element links form the primary framework for spreading activation. The knowledge tree comes next. Activation might actually be infinite if it was costless. SuperMemo goes into just a few layers of activation in a single repetition or when building the neural queue for the first time. This takes milliseconds on a fast PC, but might drag for a second on older computers. SuperMemo adds more layers of activation in successive repetitions. Those can be useful for remote associations, but you are unlikely to go through more than just a layer in a single session. The queue then becomes a tool for building your understanding how the activation is spreading in the network.

For example, if you are a doctor who is to make a diagnosis for a specific patient, you can set up a diagnostic concept and link chief suspects in a concept map. Go neural to see what new ideas come to your mind while learning new topics at the same time.

Spreading activation

Spreading activation is an algorithm for activating nodes in a neural network, semantic network, directed graph, etc. In SuperMemo, the graph is formed by semantic connections of the knowledge tree, concept links and inter-element links. Spreading activation is used to sequence elements for neural review. This is a new form of semantic review in SuperMemo, where new elements are fed neurally for review. This approach is useful in enhancing creativity and problem solving. To get a taste of neural review choose Learn : Go neural (Ctrl+F2) from the main menu. For more, see Wikipedia: Spreading activation

Spreading activation rules

Some rule for spreading activation in SuperMemo 17:

• concept links receive highest priority
• element links receive high priority
• high priority elements "conduct" activation better (better chance of passing the "signal")
• siblings slow down activation (the further from the node, the slower the conduction)
• children receive high priority
• parents can also spread activation to neighboring nodes
• parents that are roots of an article conduct at low probability (less chance of spreading activation to neighboring articles)
• only a few layers of activation are processed at single repetition (for performance reasons)

To get a feel of how activation spreads from a given element, use View : Neural queue (from the main menu) to see an exemplary neural sequence associated with the current element. Note that this sequence will differ each time you create a new queue. Spreading activation is subject to "race conditions". This means that neural impulses race each other and the outcome may be different on each occasion. As much as your brain has "moods and modes", so does the neural review in SuperMemo!

Spreading activation algorithm

Neural review will go through the sequence determined by spreading activation. In that activation, concepts have higher priority than links, or knowledge tree. However, knowledge tree will be participating as well (with lower weights). Activation will spread slowly to unrelated sibling articles whose relative priorities will increase in absence of children, concept links, and the like. In other words, only small collections will make neural review pointless. Concept maps are not obligatory.

1. associated concepts are added to the queue at CombinedPriority(Priority,0.01)
2. links are added at CombinedPriority(Priority,0.05)
3. descendants at CombinedPriority(Priority,0.16)
4. siblings at CombinedPriority(Priority,0.26) (or up to 0.5 for unrelated siblings containing full articles)

The procedure is repeated recursively with successive layers taking random picks from the queue such that a pick is more likely for a higher priority

function CombinePriority(OriginalP,GroupP:real):real;
//e.g. element of P=0.2 for a group of P=0.6 should yield (1-0.2)(1-0.6)=0.32
var OP,GP:real;
begin
   OP:=1-OriginalP;
   GP:=1-GroupP;
   Result:=1-OP*GP;
end;

This is the status for Feb 26, 2016. The sequence, priorities, and other parameters will change in the future. It all depends on user feedback and field tests.