Neural creativity


Neural learning with SuperMemo

SuperMemo 17 introduces a new form of incremental learning, which we will call neural learning. In neural learning, SuperMemo serves the student with knowledge associated with a selected subject, article, picture, etc. Knowledge is served in a neural way, i.e. by sequentially exploring semantic connections in a networks of information build automatically or manually in SuperMemo. SuperMemo takes inspiration from how the brain works. Creativity enhanced with SuperMemo is very slow, but it is targeted and methodical. Neural creativity works like a slow brain extended by memories stored in SuperMemo. Neural learning is particularly useful in research, problem solving, creative writing, explorative learning, etc. In other words, you may not find neural learning particularly useful in mastering a course of physics, however, if you are a physicist working on a complex problem that requires associating knowledge from different fields of physics and beyond, you will notice that this form of learning has a fantastic impact on your creative productivity.

Neural learning was the main reason for SuperMemo 17 to introduce a new type of elements: concepts. A concept in SuperMemo is an equivalent of an important idea, which can be connected with other elements (incl. other concepts). In addition to connections between concepts, neural learning also explores connections between elements using the knowledge tree. Elements connected via concepts form concept maps, which, in addition to the knowledge tree, from the primary skeleton for explorations in neural learning.

In the following sections, the essential terms and notions underlying neural learning will be introduced gradually: concepts, spreading activation, neural review, etc. Even if this may sound overwhelming at first, remember that you can start neural learning now with just one keystroke: Ctrl+F2 (Go neural). The following sections are only to explain how neural learning works and how its mechanisms can influence your decisions to make it work best for you.

If you are not new to SuperMemo, and want to skip details, terminology, etc, you can jump directly to practical applications in: Neural review


A concept in SuperMemo is an element associated with an important idea.

A concept can serve as a bookmark, as a root of a branch associated with a given subject, as a hub for connecting different portions of a collection, or as a starting point for a creative process (e.g. writing an article, solving a problem, etc.).

Concepts can be linked with other elements. The network of links based on concepts will be called a concept map.

SuperMemo: An exemplary concept map displaying outgoing links from the current element

See: Concept maps at Wikipedia.

Concepts: basic operations

Here are the most useful operations you will need when working with concepts:

Concepts: terminology

The term concept is used in four main contexts in SuperMemo 17:

concept element
element used to build a portion of a concept map (e.g. an idea, or an article from Wikipedia, etc.)
concept member
member of the concept registry used in concept housekeeping (esp. concept search uses when building a concept map)
concept group
branch of elements rooted in a concept. This is similar to category known from older SuperMemos. For performance reasons, elements in the concept group are not registered with a concept. Even though concept groups are used in spreading activation in neural creativity the links formed by parent-child-sibling relationships are taken as only carrying minor priority (in comparison to concept-concept, concept-element and element-element links set by the user)
concept map
interlinked graph of elements used as the skeleton for spreading activation. It may also be used in reference to its diagrammatic representation

Each concept element has its corresponding concept member in the registry. Each concept group is based on a single concept element. Some concepts (elements and member) are created primarily for expanding the knowledge tree (via a concept group). Other concepts (elements and members) are created primarily to form concept maps for neural learning. Concepts can play both roles: tree tools and concept map tools.

Concept groups

When you work with a collection which includes material belonging to different subjects, you may want to keep different subjects in different sections of the knowledge tree (in the Contents window). You may also want to design a different look for items related to literature, and a different look for questions related to mathematics. For example, you could use pictures to illustrate your anatomy branch, sounds to illustrate your foreign language section, spell-pads to make your spelling section work best, etc.

To help you organize different subjects in one collection, SuperMemo uses concept groups. A concept group can be seen as a tool for building a section of the knowledge tree. One concept group is used for adding material related to one subject. Elements added to a given concept group may use their own template that determines their look (e.g. images, sounds, colors, fonts, sizes of text fields, number of shape components, etc.). See: Using templates.

In older SuperMemos, the role similar to concept groups was played by categories.

Concept registry

Concept registry is used to keep the sorted list of concepts. Each concept element is associated with a concept registry member. Concept registry makes it easy to find concepts, esp. when you want to link an element with a particular concept.

In the registry, each concept also shows a set of properties that can be used in managing a concept group.

Each concept registry member makes it possible to change the following properties:

Choose Search : Concepts from the main menu to open the concept registry.

SuperMemo: Concept registry which you can use to execute neural review on selected members and change their properties

Link registry

All inter-element links are stored in a dedicated link registry (available with Search : Links from the main menu). Link registry is functionally similar to concept registry. The main difference is the greater significance of concepts as compared to links. You are likely to use concept registry often when making conceptual links. This why it is important to keep it less crowded and with good searchable name standards. To make inter-element links, you will rather use the Contents window. Only concepts can be used to group elements in branches (with concept groups). Links cannot play a similar role and do not appear on selectable concept group lists. In addition, concept links get much greater weight in spreading activation in neural review (see below).

Neural review

Neural review is a subset review that follows meaningful connections between pieces of knowledge. It takes inspiration from how the brain follows associated ideas. Neural review automates the review of knowledge associated with selected concepts or ideas, e.g. in problem solving, creative writing, etc.

For example, in a neural review of dogs, there is a good chance of learning about puppies or pooddles, but there is also a chance of learning about cats and somewhat lesser chance of learning about cars. Similarly, if you see a picture of a banana plant, going neural on that picture will help you review associated elements (e.g. other pictures of plants, or articles about bananas, or elements associated with the same concept such as gardening).

A concept in SuperMemo is an important idea or subject. Concepts are a new type of element that can be linked with other elements. The network of links based on concepts will be called a concept map. Elements can also form links between each other. In SuperMemo, knowledge is also connected by inheritance, and the structure of the knowledge tree.

Spreading activation in neural networks is a method of searching the network with the use of network connections and weights.

Neural review follows the spread of neural activation in a network of inter-element connections. All the links and connections in SuperMemo can be used to simulate a spread of activation in the network of knowledge where individual elements play a role of neurons.

All links used in spreading activation receive weights based on their importance. In SuperMemo, weights are determined by element priority and the priorities of individual knowledge associations (e.g. concept connections are of higher priority than the sibling relationship in the knowledge tree).

In spreading activation, we can sequence the propagation wave by the time of activation of individual nodes, which are formed by SuperMemo elements. Such a sequence can then be used as the basis of a semantic review in the incremental learning process.

Neural review can be initiated by executing Learn : Go neural (Ctrl+F2) from the main menu.

Neural review carries a degree of randomness. As such it can be very helpful in associating ideas in neural creativity. Connectivity and randomness underlie the main rationale behind neural review: neural creativity and neural problem solving. In those approaches, we combine a creative process or a problem solving process with an exploration of new subjects, while executing incremental learning in the background. In other words, neural review serves learning that is to instantly bring benefits in the shape of new ideas and new problems solved in conditions of incomplete human knowledge and in combination with abundant and possibly contradictory computer-stored knowledge. A typical use of neural creativity would be to diagnose a rare disease on the basis of its symptoms (see examples).

We will use the following terms:

neural review
incremental learning that follows the spreading of activation in a conceptual network
neural creativity
creative process based on neural review (e.g. creative writing)
neural problem solving
process of neural creativity targeted at solving a specific problem (e.g. solving a forensic case)

Neural review starter

You can try neural learning on your own now.


All you need is SuperMemo 17 and a rich collection. Small collections won't show the power of neural learning. If you start with a single article, neural review makes sense only if your article has already been splintered into smaller pieces. You can also do Split article if this is a newly imported article, however, in that case your sub-articles need to be outstanding. This is because neural review ends if the neural queue contains elements that have all been reviewed/imported/created on that given day. This means that for a newly split article, you need to use Add to outstanding or wait a day (for sub-articles to "mature" for review).


Choose any of the following:


While in neural review, use the button Learn as in normal learning. While learning, the Review parameter in the Statistics window shows Neuro=[Size], where Size is the number of elements in the neural queue. The size of the queue is limited for practical reasons. You are not likely to go beyond a few hundred elements in a single review. Moreover, at each repetition, more layers of the conceptual networks are explored and the queue keeps being expanded in parallel with your learning.


To end the review, press Esc. The Review field in the Statistics window is cleared.

Costs of neural learning

If you worry about the cost-vs-benefit equation in building concept maps in SuperMemo, remember that properly executed incremental learning builds a semantic structure of the processed knowledge in the shape of the knowledge tree. Parents, siblings and children are related to each other in terms of their meaning. Concept maps are the chief framework for spreading activation in neural SuperMemo, but the knowledge tree also forms a fine dendritic framework that helps you capitalize on all your prior work with incremental learning. If you are familiar with mind maps, you will be happy to know that the cost of building your concept maps is far less. At times, all you need for your creative purposes is a conceptual link between two different subjects where you expect to find creative associations. To execute your neural review or to engage in neural creativity, pick your concept, element or a registry member (e.g. a picture) and Go Neural.

Concepts and inter-element links form the primary framework for spreading activation. The knowledge tree comes next. Activation might actually be infinite if it was costless. SuperMemo goes into just a few layers of activation in a single repetition or when building the neural queue for the first time. This takes milliseconds on a fast PC, but might drag for a second on older computers. SuperMemo adds more layers of activation in successive repetitions. Those can be useful for remote associations, but you are unlikely to go through more than just a layer in a single session. The queue then becomes a tool for building your understanding how the activation is spreading in the network.

For example, if you are a doctor who is to make a diagnosis for a specific patient, you can set up a diagnostic concept and link chief suspects in a concept map. Go neural to see what new ideas come to your mind while learning new topics at the same time.

Spreading activation

Spreading activation is an algorithm for activating nodes in a neural network, semantic network, directed graph, etc. In SuperMemo, the graph is formed by semantic connections of the knowledge tree, concept links and inter-element links. Spreading activation is used to sequence elements for neural review. This is a new form of semantic review in SuperMemo, where new elements are fed neurally for review. This approach is useful in enhancing creativity and problem solving. To get a taste of neural review choose Learn : Go neural (Ctrl+F2) from the main menu. For more, see Wikipedia: Spreading activation

Spreading activation rules

Some rules for spreading activation in SuperMemo 17:

To get a feel of how activation spreads from a given element, use View : Neural queue (from the main menu) to see an exemplary neural sequence associated with the current element. Note that this sequence will differ each time you create a new queue. Spreading activation is subject to "race conditions". This means that neural impulses race each other and the outcome may be different on each occasion. As much as your brain has "moods and modes", so does the neural review in SuperMemo!

Spreading activation algorithm

Neural review will go through the sequence determined by spreading activation. In that activation, concepts have higher priority than links, or knowledge tree. However, knowledge tree will be participating as well (with lower weights). Activation will spread slowly to unrelated sibling articles whose relative priorities will increase in absence of children, concept links, and the like. In other words, only small collections will make neural review pointless. Concept maps are not obligatory.

Priorities are combined to determine the spread of activation:

  1. associated concepts are added to the queue at CombinedPriority(Priority,0.01)
  2. links are added at CombinedPriority(Priority,0.05)
  3. descendants at CombinedPriority(Priority,0.16)
  4. siblings at CombinedPriority(Priority,0.26) (or up to 0.5 for unrelated siblings containing full articles)

The procedure is repeated recursively with successive layers taking random picks from the queue such that a pick is more likely for a higher priority

function CombinePriority(OriginalP,GroupP:real):real;//e.g. element of P=0.2 for a group of P=0.6 should yield (1-0.2)(1-0.6)=0.32var OP,GP:real;begin   OP:=1-OriginalP;   GP:=1-GroupP;   Result:=1-OP*GP;end;

This is the status for Feb 26, 2016. The sequence, priorities, and other parameters will change in the future. It all depends on user feedback and field tests.

Neural creativity

Neural creativity is creativity induced with the help of neural review. In neural review, SuperMemo successively feeds the user with knowledge associated with a selected topic or a chosen subset of topics. When a network of conceptual links connects areas of knowledge, they form semantic space that can be explored in a neural manner that helps forming new associations, and generate ideas. This process is conducive to research, invention, problem solving, etc. Metaphorically speaking, neural creativity helps emulate a thinking process in a human brain extended by knowledge stored in SuperMemo.

Neural creativity employs the incremental learning process to produce new ideas in a neural fashion. In neural creativity, closely related concepts are more likely to show up when executing a neural review of a studied subject. If you are fluent in incremental learning, you may wonder how a neural review differs from a semantic review. The main difference is that in semantic review the whole burden of juggling the semantics is in your hands. You need to know the tools of incremental learning to execute a semantic review well. In neural review, the whole process will be reduced to pressing a single button in SuperMemo.

Unlike it is the case with semantic review in SuperMemo, or simply googling for knowledge, the whole process of associating pieces of knowledge is automated. To give it a try, pick a subject and Go neural (Ctrl+F2).

Incremental approach

Incremental learning can be used to creatively come up with new ideas. This raises some skepticism. Here is an exemplary skeptical voice:

"How can interruption be good for creativity or problem solving? Isn't it that all great discoveries come from continual, persistent and focused effort? It is those who can dedicate themselves to a cause that achieve best results. Most people fail because they major in minor things. They do many things superficially. They are jacks of all trade and masters of none. Incremental learning seems like an effort to major in everything. Putting too much on one's own plate".

This question keeps popping back in various contexts. Here is an illustrative answer based on the fact that creativity and problem solving are based on forming new connections in your brain.

Incremental recall: bringing back things to memory, piece by piece

Imagine you had a very eventful day. For example, a visit to an exotic city, meeting an old good friend, or attending a breakthrough conference. Imagine that your inspiration is so rich that you want to write a detailed report of the day (assuming you did not capture it all with Google Glass). The first thing to do is to sit down to writing a report and list all essential keywords for rebuilding the memory of the day. Once you write the outline, you can quickly notice that it does not take long before you come up blank for new memories. You cannot recall any more details. This is normal. Sitting down over a blank piece of paper for an hour may not be the best use of your time. It may be pretty unproductive. However, if you keep your piece of paper handy, you may notice that it keeps filling up with new memories as they keep popping up in your head in different contexts. An image, a smell, or an article in press may all contribute to unexpected recall. If you happen to execute incremental learning in the meantime, you will notice that the process of recall is intensified. If your learning materials are rich, you are likely to read about dozens of unrelated subjects in a single day. All those subjects produce new associations. Your blank piece of paper will fill up much faster than it is the case in the course of ordinary life or if your learning was less varied. What you witness here is incremental recall.

There is a short step from incremental recall to incremental creativity. Both are based on incrementally building a mental structure in your mind. In case of recall, you will recover past memories. In case of creativity, you will additionally add

  1. new pieces of knowledge acquired via incremental learning (e.g. neural review), and
  2. new memories and/or ideas you have not experienced or thought of before.

Finally, in problem solving, you will direct this creative process towards accomplishing a specific task: solving a problem, answering a question, etc.

Examples of neural creativity

Neural tools in SuperMemo equip the user with a sort of Einstein's brain in slow motion. You can build a brain of knowledge as big as you wish. You can make your networks of ideas as complex as you wish. And then you can just press one button to start a train of thought at your own pace while watching all new ideas form in front of your eyes. Here are some examples of the neural approach in SuperMemo:

Medical diagnosis

Physicians and medical students always have a mountain of new health-related knowledge to process. This is also true of all individuals who care about health. A specific problem to solve, e.g. a medical diagnosis, may also be a good excuse for acquiring new knowledge neural style. For example, if a patient presents with a skin condition, a dedicated concept might be created (e.g. Patient John Doe condition). Conceptual links can then be made to relevant fields of knowledge, e.g. skin conditions, allergies, bacterial skin infections, mycoses, etc. Neural review will provide new inspiration for new areas to explore. Two birds can be killed with one stone:

  1. new knowledge acquired for future use, and
  2. zeroing in on the ultimate diagnosis.

Solving a software problem

Neural review has already been helpful in solving problems that pester users of SuperMemo (e.g. misaligned cloze deletion, due to bugs in Internet Explorer). When a pesky problem shows up, all ideas and new areas of related knowledge can be associated with a concept that defines the problem (e.g. cloze bug).

In the case of the "cloze bug", several users sent their ideas on how to possibly solve the problem. None of these worked. In the end, all DOM-related ideas turned out to be a blind path. However, initiating neural review provided an ultimate incentive with a hope for a light in the tunnel. Could the problem be solved without neural review? Yes. Perhaps, neural review helped step out of the box in looking for solutions? If not, it might at least have had a psychological effect: "if nothing works, perhaps neural review will?".

Creative writing

Incremental writing is useful if you want to compile a number of texts into a larger body. Neural approach may be useful if you want to combine that procedure with a degree of research. You may either want to research a topic (e.g. when looking for facts or literature related to your thesis), or research your own texts if they are too voluminous and highly repetitive (e.g. to look for simpler phrasing, more logical sequencing, clearer line of thought, etc.).

Forensic science

Forensics is an excellent example of where neural approach may appear useful. A complex case may involve a great deal of case-specific knowledge that needs to be combined with scientific and general knowledge. Wherever the size of knowledge goes well beyond the capacity of human memory, and where the solution may require thinking "out of the box", neural approach may act as an extension to the human mind. Once a case is solved, case-independent knowledge may be integrated with the main learning collection to secure the long-term investment in knowledge.

See also