The Statistics window can most conveniently be viewed by pressing F5 (Window : Layout :
Warrior layout). This
arranges the statistics and element parameters windows in the classic way first introduced in SuperMemo 3.0 in 1988.
The picture below shows an exemplary Statistics window. The description
of individual fields is presented below the picture. Click the picture now to
open it in a separate window for easy comparison of fields and their
descriptions:

The caption of the
statistics window displays the name of the collection in square brackets.
Learning parameters displayed in
the statistics window:
-
Date - current date and the day of the week. If this
value is preceded with Night, it means that the new calendar day has already started but
the old repetition day will not start until the time defined in Options
: Learning : Midnight clock shift.
In the
example above, the picture snapshot was taken after midnight on July 20,
2004 (Tuesday). The collection in use is named
"all" as shown in the caption
- First day - date on which the learning process began
(i.e. the day on which the first element was memorized). The
exemplary collection presented in the picture has been in use since December 15, 1987 (i.e.
the birth date of SuperMemo for
DOS)
- Day - number of days in the learning process (i.e.
number of days between Date and First day).
Day=Date-First day.
The presented collection has been in use for 6063 days (i.e. 16 years
and 219 days)
- Total - the number of items, topics and tasks in
the collection. Two relationships hold true:
- Memorized+Pending+Dismissed=Total
- Topics+Items=Total (tasks are counted with
the Topics statistic)
Deleted elements do not contribute to the total count of elements in the system. In
the picture, the presented collection is made of nearly 232,000 elements
(largest collections reported by users reached beyond a half million
elements)
- Items+Topics - the number of items and the number of topics
(and tasks) in the collection.
Items+Topics=Total.
In the example, the collection includes nearly 103,000 items and
nearly 130,000 topics (and tasks)
- Memorized - total number of elements introduced into the learning process with options such
as Learn
or Remember. If an item takes part in repetitions it is a memorized
item. It does not mean it is a remembered item (see FAQ).
The
presented collection has nearly 202,000 elements in the learning process and these
elements make up 99.8% of all elements destined to enter the learning
process, i.e. Memorized/(Memorized+Pending)=0.998
- Memorized items - the number of memorized items in the
collection and the proportion of memorized items among memorized
elements. In the example above, 100,304 items take part in
repetitions. These items make 49.8% of all memorized elements (the rest
are memorized topics or tasks). The Retention field (below) indicates that
94.34% of these items should be
remembered at any given time
- Memorized topics - the number of memorized topics and the
proportion of memorized topics among all memorized elements. In
well-balanced incremental reading, topics make
about a half
of the material taking part in repetitions. If the proportion of topics
increases beyond 90%, the learning process may slowly resemble
traditional learning where ineffective passive review predominates. If
the proportion of topics in incremental reading drops below 10%, it may
indicate the congestion of the learning process with stymied inflow of
new material. In the
example, 101,217 topics make 50.2% of the material taking part in
the learning process. This indicates a good balance between items and
topics
- Memorized/Day - number of
items memorized per day: (Memorized items)/Day. In the example, the average of
16.54 items
have been memorized daily in
the presented collection over the last 16 years
- Pending - the number of
elements (topics or items) that have not yet been introduced into the
learning process and await memorization (with operations such as Learn, Remember,
Schedule, etc). All pending elements are kept in the so-called pending
queue that determines the sequence of learning new elements. Dismissed items are not
kept in the pending queue. In the example, the collection contains
359 elements in the pending queue. With incremental
reading and selective postpone tools, the role of the pending queue
in SuperMemo is diminishing
- Dismissed - the number of elements (topics, items
or tasks) that have been excluded from the learning process and are kept only as reference
material, knowledge tree folders, or tasklist elements. Dismissed items are neither pending nor memorized.
All tasks are dismissed by default, i.e. they usually do not take part in repetitions. In
the example, over 30,000 elements have been dismissed. In incremental
reading, you will often retain parent articles dismissed after
reading. They can then be used for context
reference until all children items are fully formulated, moved to their
target categories and provided with all necessary context. This is why
the proportion of dismissed material is often quite high in collections
subject to incremental reading
- Outstanding - number of
outstanding items, outstanding topics and final drill items scheduled for repetition on
this given day. The first number (before the
plus sign) indicates the number of items scheduled for this given day and not yet
processed. The second number (after the plus sign) indicates number of
topics scheduled for review for this day. The third number (after the
second plus sign), if present, indicates the number of items that have
already been repeated today but scored less than Good (4). Those are the items that
make up the final drill queue. The final drill queue is built only if Skip
final drill is unchecked in Options. In the presented collection,
there are still 198 items scheduled for repetition on July 20, 2004.
There are also 270 topics scheduled for review on that day
as part of incremental reading. There are no elements in the final
drill queue (the third components of Outstanding is missing)
- Retention - estimated knowledge retention in the collection. In the example,
94.34% of the material should be recalled in a
random test on all elements in the collection at any time. You can test your retention using random
tests and see if SuperMemo's estimates are accurate. This
statistic may be overly optimistic if you have recently abused rescheduling tools such
as Postpone or Mercy
- Measured FI -
the value of the measured forgetting index as
recorded during repetitions. The measured forgetting index is the proportion of items not
remembered during repetitions. The number in the parentheses indicates Measured FI
for the day. It is quite usual to have Measured FI higher than Average
FI. This is due to two factors: (1) every user
will experience delays in repetitions from time to time (e.g. as a
result of using Postpone), (2) SuperMemo imposes some
constraints on the length of intervals that, in some cases, make it schedule repetitions
later
than it would be implied by the forgetting index. The constraints in computing intervals,
for example, prevent the new interval from being shorter than the old interval
(assuming the item has not been forgotten). For low values of the forgetting index and
items with a low A-Factor, the new optimum interval might often be
shorter than the old one! Measured FI can be reset with File :
Tools : Reset parameters : Forgetting index record. In the presented example,
an average of 11.11% of item repetitions end with a grade less than Pass
(3)(since the measured forgetting index record has last been reset).
On Jul 20, 2004, 19.4%
repetitions ended in failure (i.e. with a grade less than Pass)
- Average FI - the average requested
forgetting index in the entire collection (the number in
parentheses is the default forgetting index). If the forgetting index of individual elements
is not changed manually, Average FI is equal to the default forgetting index as set
in Tools : Options : Learning : Forgetting index. The
default forgetting index is the requested forgetting index given to all
categories and, as a result, to all new items added
to the collection. Forgetting index, in general, is the proportion of items that are not
remembered during repetitions. The lower the value of the forgetting index the better the
recall of the element, but the more repetitions will be needed to keep it in memory.
Optimum value of the forgetting index falls into the range from 7% to 13%. Too low
a forgetting index makes learning too tiresome due to a prohibitively large number of
repetitions. All elements can have their desired forgetting index set individually. The
easiest way to change the forgetting index of a large number of elements is to use Forgetting
index option among subset operations.
In the presented example, the average forgetting index is 10.07% while the default
forgetting index is 13%. See: Using forgetting index
- Burden -
estimation of the average number of items and topics repeated per day. This value is equal to the sum of all
interval reciprocals (i.e.
1/interval). The interpretation of this number is as follows: every item with interval of
100 days is on average repeated 1/100 times per day. Thus the sum of
interval reciprocals is
a good indicator of the total repetition workload in the collection. The presented
collection requires 225 item repetitions per day and 957 topic reviews
per day. In incremental reading, it is not unusual to have many more
elements in the process than
one can handle. Postpone can be used to unload the excess of
topics at the end of the learning day as well as to reduce the load of
low-priority items. Postpone skews the Burden statistic.
In the presented collection, the load of topics and items is
approximately the same; however, topics crowd at lower intervals and are
regularly reshuffled with Postpone
- Burden +/- - the change of the Burden parameter
above on a given day. Here, on Jul 20, 2004, the average number of
expected repetitions was reduced by 0.18 item per day. The topic load
was also reduced by 39 topics per day. To reduce the topic burden by 39,
one would need to review 78 topics with an interval increase from 1 to 2
days (78*0.5=39). However, one could equally well execute Postpone on
2344 topics with interval increase from 10 to 12 days
(2344*(1/10-1/12)=39)
- Workload - the average daily time used for responding
to questions in a given collection.
Workload = (Item Burden)*Avg time
In the presented collection, 225 item repetitions per day taking 8.46 seconds each result in a daily repetition time
estimated at 31 minutes and 44 seconds. A real learning time may be twice longer due to grading, editing, reviewing the collection and
various interruptions. In incremental reading, the learning time will
increase further due to unclocked topic review. The real learning time
may also be cut if Postpone is
used
- Subset - number of
elements scheduled for subset review (e.g. elements in
branch repetitions in Contents : Learn,
elements in a browser subset repetitions in browser's Learn, elements in the
random test queue in Tools :
Random tests, etc.).
The display may have a form of <items to do>+<topics to do>+<pending
to do>+(<subset description>) in subset review, or <elements
unprocessed>/<all elements in the test> in random tests. Here
3 elements remain in subset review. No items, one topic and two pending
elements. The subset on which Learn is being executed contains
612 elements and has been generated by searching for the word "NMDA
glutamate receptor". In other words, there is still one outstanding
topic and two pending elements that refer to the learning subject
"NMDA glutamate receptor" in the whole body of 612 elements
related to the NMDA receptor. In incremental
reading, subset repetitions are most often executed in the contents
window or in the browser with Learning : Learn (Ctrl+Alt+L) or
with Learning : Review. In the later case, not only outstanding
elements are reviewed. The remaining elements are subject to
mid-interval review as well
- Alarm - time left till the next alarm and the hour at which the
alarm will ring off (to learn more about alarms see: Plan).
This field is editable. To change the alarm setting, click the field and
type in the new time in minutes (e.g. 21.5 will set the alarm to sound
in 21 minutes and 30 seconds). To end editing, press Enter. In
the example, the alarm will sound off in 12 minutes and 44 seconds at
6:37
- Time - total question response time
on a given day and
the total session time (in parentheses). Here the total time needed to respond to
questions on July 20, 2004 was 8 minutes and 34 seconds. On the same
day, SuperMemo has been running for 7 hours and 11 minutes (this value
will increase even if you simply keep SuperMemo running)
- Avg time - average response time in seconds.
This is the time between displaying the question (or equivalent) and choosing Show answer
(or
equivalent). In the presented collection, the average time to answer a single question
is around 8.5 seconds. If this number grows beyond 10-15 seconds, you may
need to analyze your learning material if it is not overly difficult or
badly structured
- Total time - total time taken by responding to
questions in the collection. This time cannot be accurately measured for collections
created with SuperMemo 98 or earlier (the measurements were made possible
only in SuperMemo 99). If
you upgrade older collections, this number will roughly be
guessed for you. SuperMemo will derive this time from the total number of items, average number of
repetitions, average number of lapses, and the average repetition time. In the
presented example, answering questions during repetitions took over 118 days
of non-stop learning (in over 16 years of the learning process)
- Lapses - average number of times individual
items have been forgotten in the collection (only memorized elements are averaged). The number
in parentheses shows the number of lapses on a given day. Here an average
element has been forgotten 0.387 times. On July 20, 2004, 7 items have
been graded less than Pass (3)
- Speed - the average
knowledge acquisition rate, i.e. the number of items memorized per year per minute of
daily work. Initially this value may be as high as 100,000 items/year/minute (esp. if you
enthusiastically start working with the program before truly measuring its limitations;
or rather the limitations of your memory); however, it should with time stabilize between
40 and 400 items/year/minute.
Speed=(Memorized items/Day)/Workload*365
In the presented collection, every minute of work per
day resulted in 190 new items memorized each year. As this value is
derived from Burden, it may be highly underestimated if you use Postpone
a lot (e.g. in incremental reading)
- Cost - the cost in time of memorizing a single item,
i.e. total learning time divided by the number of memorized items.
Cost = Total
time / Memorized
In the presented example, the total repetition time per single item is
1.696 minutes. In
other words, each item has contributed around 1.7 minutes to the total of non-stop
118 days of repetitions. The cost of editing, collection restructuring, incremental reading, etc. is not included in this number
- Daily cost - daily repetition time per each newly
memorized item.
Daily cost = Workload/(Memorized items/Day)
In the presented collection, each of the 16.5 newly memorized items per day contributes
about 2 minutes of repetitions (precisely 1.918 minute) to the total workload of
nearly 32 minutes per day.
As this value is derived from Burden, it may be highly
overestimated if you use Postpone a lot (e.g. in incremental
reading)
- Interval - average interval
among memorized items in the
collection. Here an average memorized item has reached the interval
of 877 days
- Repetitions - average number of repetitions per
memorized item in the collection. Here an average item has been repeated
3.54 times
- Rep count - the total count of
item repetitions made in
the collection. In the presented collection, 817 thousand item repetitions have been made.
This is about 8.14 repetitions per memorized item. That includes repetitions of items
that have been reset, forgotten, dismissed, deleted, etc.
- Last Rep - average date of the last repetition among
memorized items in the collection. Here the average date of the last repetition
is September 14, 2002
- Next Rep - average date of the next repetition among
memorized items in the collection.
Next Rep = Last Rep + Interval
Here the average date of the next repetition is February 7, 2005 or
877 days after September 14, 2002
- Completion - the expected date on which all elements
from the pending queue will be memorized assuming the present rate of learning new items.
This parameter is particularly useful if you are memorizing large
ready-made collections such as Advanced
English.
Completion=Date+(Pending/(Memorized items/Day))
In the example, it would take until August 12, 2004 to memorize the
remaining 359 pending items
at the speed of 16.5 items per day (beginning with Jul 20, 2004)
- A-Factor - average value of A-Factor
among memorized items in the collection. A-Factor is a measure of item difficulty. The
higher the A-Factor, the easier the item. In the presented collection, the average
A-Factor is about 3.53. This indicates that the collection is rather well-structured and
the material is thus relatively easy to remember
Comments:
- Items are added to the final drill not only during standard
repetitions when you grade an element below Good (4). Operations such as Remember
(Ctrl+M), Remember Cloze, and Add to drill (Shift+Ctrl+D)
will also add to the final drill queue. The final drill queue is
created automatically only if you uncheck Tools : Options : Learning : Skip final drill
- Some fields of the statistics window can be edited. For
example: Alarm, Measured FI, Total time, Rep count,
etc. To edit and entry, click it, type the new value and press Enter. If the
entry cannot be modified SuperMemo will warn you (e.g. "Retention entry cannot be
modified").
- See Survey 1994 and
Survey 1999 for some interesting notes about the
speed of learning reached with SuperMemo
FAQ
In SuperMemo, Memorized<>Remembered
Retention statistic assumes regular repetitions and well-structured learning material
In SuperMemo, Memorized<>Remembered
(jj, UK, Sunday, December 24, 2000 1:54 AM)
Question:
I have noticed in the Statistics that the
number of elements memorized increases even when I enter Fail when
answering incorrectly. For instance, in the collection of US States Capitals, it
was showing 100% memorized when I was still getting many of them wrong
Answer:
Parameter Memorized indicates the number of elements in the learning
process; not the number of elements you are able to recall correctly. If you
make regular repetitions in the long run (i.e. over weeks and months), the
number of elements you will be able to recall will approximately equal Memorized*Retention
Retention statistic assumes regular repetitions and well-structured learning material
(dansujp, Sun, Sep 16, 2001 3:07 PM)
Question:
When
I returned from vacation, I expected the retention to be something like 80% because I have not done any repetitions for two weeks.
But it was exactly the same as before I left
Answer:
The Retention statistic is derived directly from the measured forgetting index on the assumption of a negatively exponential forgetting curve. This curve is only representative of well-structured learning material. In addition, the forgetting index measurements are averaged over all recorded cases. A break in repetitions will invalidate the statistic. Resuming repetitions is not a guarantee of accuracy as the large number of earlier repetitions will result in overestimating the retention on a small-sample measurement. The only valid estimation of retention after a break in learning is the one that follows resetting the past forgetting index record
(File : Tools : Reset parameters : Forgetting index
record). This will result in gathering new data that will approach true retention for the sample tested with accuracy proportional to the number of repetitions done