Here is the statistical analysis of user responses provided in a
poll conducted among registered users of SuperMemo in Poland (data gathered in Fall 1994).
The main conclusions coming from the poll are:
- users of SuperMemo form a very special group of people. A typical
user is a highly educated young male non-smoker who exercises a lot and whose main hobby
is his computer
- users of SuperMemo like the program and use it very systematically;
they consider SuperMemo one of their most important applications: SuperMemo is the most often used application giving
way only to all word-processor applications combined
Education
Profession
Age
Hobby
Health, exercise, sleep and smoking
Version of SuperMemo
Using SuperMemo
Speed of learning (!!!)
Opinions about SuperMemo
Advanced English
Where from did you learn about SuperMemo first time?
What made you buy SuperMemo?
Your greatest achievement
Likes in SuperMemo
Dislikes in SuperMemo
Options used
Options not used
Improvements
Most requested learning material
Fighting piracy
CD-ROM features
Software used
What would make you buy SuperMemo earlier?
What would make your colleagues buy SuperMemo?
Where would you like to find advertising of new
SuperMemo products (in Poland)?
Other: Where are women?
Education (of users taking part in
the poll)
- 3% Ph.D.
- 45% MSc
- 27% university students
- 11% secondary education
- 14% secondary school students
- 1% primary school students
- 0% primary education
Profession
- 26% engineer (of which about half are computer scientists)
- 25% university student (of which 33% computer science, 20% sciences,
16% electronics, 15% economics, 7% medicine, etc.)
- 15% secondary school student
- 8% businessman (usu. putting himself or herself in the engineer
category)
- 6% scientist
- 5% teacher
- 5% technician
- 3% pensioner
- 2% lawyer
- 9% other professions (e.g. military, librarian, policeman, priest,
etc.)
Age
The average user is 29 years old. The youngest respondent was 16
years old, the oldest 54. 14% of users are above 40, 27% are less than 20.
Hobby
- 64% computers
- 20% sport
- 18% foreign languages
- 9% music
- 8% tourism
- 8% math
- 6% photography
- 5% each of the following: history, philosophy, chemistry, science
fiction, politics, business
- 4% each of the following: books, psychology, physics
Other frequent responses: movies, social sciences, pedagogy, logic,
biology, electronics, architecture, astronomy, education, garden, do-it-yourself, Bible,
cars, etc.
Smoking, sleep and exercise Only 4% of
users of SuperMemo smoke! For comparison, 30% of Americans and 70% of Japanese are smokers
(1990)! An average user sleeps 7.5 hours per day. Smokers sleep 1.1 hours less. Those who
sleep more than 7.5 hours had memorized 3870 items on average, while those who slept less
than 7.5 hours had memorized 4570 items. An average user exercises 27 minutes per day.
Version of SuperMemo
- 53% of respondents reported using SuperMemo 7 for Windows (an
increase from 11% a year ago). Only
- 8% reported using versions newer than SuperMemo 7.2.
- 14% used SuperMemo shareware before buying commercial SuperMemo (108
days on average).
- 12% admitted having used an illegal copy of SuperMemo beforehand (98
days on average). In 1993, the same figure was 6%.
Among learners who obtained both Polish and English version of
SuperMemo, 52% used the English version, 48% used Polish version.
Using SuperMemo
An average learner has been using SuperMemo for 11 months, 28
minutes per day, and has memorized 4670 items (equivalent of 182 items/year/minute, i.e.
substantially less than the average acquisition rate; see Speed of learning). This
data is almost identical with the one obtained a year earlier (10 months, 27 minutes, 4400
items).
87% of users reported breaks in learning (note, however, that the break-handling option Mercy
belonged to the group of least used options in SuperMemo).
The longest break was on average 19 days long (excluding innumerous cases, about 1%, in
which users reported breaks longer than 6 months).
The average proportion of breaks in learning in the learning process was reported to be
9%.
Speed of learning
The average speed of learning was 243 items/year/minute. The same
value after rejecting scores less than 100 and more than 600 was 242 items/year/minute.
Those who sleep less than 7.5 hours learn at the speed of 240 items/year/minute. Those who
sleep more than 7.5 hours learn at 256 items/year/minute. Apart from the above finding,
factors such as smoking, sleep and exercise are poorly correlated with the speed of
learning.
Learners 28 years old or younger learn at the speed 264 items/year/minute, while those
above 28 years learn at the speed 179 items/year/minute. Remember, however, that in
another study it has been shown that good learners learn slower (!), because of their
greater self-criticism in providing grades.
At the reported average speed of learning, a 1 hour per day learning sessions make it
possible to memorize Basic English (3000 items) in 2.3 months, and Advanced English (40,000 items) in 2.5 years (note, however,
that the speed of learning may be much higher in the first few months of working with a
given database).
80% of users confirmed that it is possible to learn Basic English in just 30 days working
0.5 hour per day.
78% believe that it is possible to memorize Advanced English
in four years working 0.5 hour per day.
The best reported result of memorizing Advanced English comes from Marek Minakowski (Institute of Philosophy,
Jagiellonian University, Krakow), who memorized 12,000 items in just four months. Second
best was Jaroslaw Ast, (student of dentistry, Medical Academy of Poznan), who memorized
10,000 items in four months. Pawel Dzierwa (Warsaw) who held the best result a year
earlier (10,000 items) started his work all over again upon receiving CD-ROM with Audio
Advanced English.
Piotr Wozniak (SuperMemo World, the author of
the database), memorized the entire Advanced English, but he passed only 10,000 items in
his first year. We are waiting for similar reports from other learners.
Opinions about SuperMemo as a method and
as software
The users have the following opinions about the SuperMemo method:
- 69% an excellent aid in learning
- 30% useful idea
- 1.5% nothing special
- 0% artificial approach to learning
- 0% waste of time
On the grade point scale from 0-worst to 5-best, the method scored
4.67 on average (4.75 a year earlier). Surprisingly, the grade was poorly correlated with
the reported speed of learning and with the number of memorized items. It was also poorly
correlated with the version of the program and with the student's education.
The following are users opinions about SuperMemo as a program:
- 16% excellent piece of software
- 63% reliable product
- 20% useful product that must be improved
- 0% product full of bugs
- 0% trash
On the grade point scale from 0-worst to 5-best, the program scored
4.0 on average (4.17 a year earlier). Again surprisingly, the grade was poorly correlated
with the version of the program. Additionally, learners who obtained higher learning
speeds were slightly more critical about software, while older users tended to provide
slightly higher grades. Education had no bearing on the grade.
Opinions about the Advanced English
database
- 40% excellent aid in learning English
- 15% useful database that needs improvement
- 9% useful database that contains awkward items and definitions
- 0% low quality product
- 36% not used (or no response)
Where from did you learn about
SuperMemo first time?
- 88% press (of which: 50% advertising, 25% press reviews and 25%
leaflet insert)
- 26% PC Kurier
- 8% Bajtek
- 6% Enter
- 5% Computer World
- 2% Informatyka
- 14% friend
- 6% leaflet
- 5% shop
- 2% software fair
- 1% computer market
What made you buy SuperMemo?
- 58% reviews in press
- 55% scientific method employed in SuperMemo
- 53% possibility to create my own databases
- 39% belief in the effectiveness of SuperMemo
- 34% awards given to SuperMemo
- 26% leaflets
- 23% recommendation of a colleague, shareware, illegal version, demo
(10%, 6%, 5%, 2%)
- 23% one or many adverts (11% one, 11% many)
- 19% availability of many databases
- 18% multimedia capability
- 15% availability of a Windows version
- 8% software fairs
- 3% shopkeeper's advice
- 3% nice box
- 3% poster
- 3% buying by accident
- 0% company's purchase
- 0% recommendation by a superior
What is your greatest achievement with
SuperMemo?
- 26% memorizing a great deal of items/words/phrases
- 24% no answer
- 18% becoming regular in my work
- 9% quickly memorizing Basic English
- 8% gaining self-confidence (language ability, memory ability, etc.)
- 8% passing an important exam (in 60% of cases: language exam)
- 6% learning a foreign language
- 5% regular growth of foreign language vocabulary
- 4% no success
- 2% satisfaction
Other responses: 500 items memorized in a day, 8-th place in the
database competition, excellent grades in language courses, best grades in English in my
school, qualifying to the finals of Latin Language Contest, memorizing drumrum
in German (at last), learning several programming languages, starting to like German,
memorizing 10,000 items in a year, etc.
What do you like most in SuperMemo?
- 49% effectiveness
- 25% the fact that it enforces regular work
- 11% ability to create ones own databases
- 9% simplicity
- 6% no need to type in the correct answer
- 5% self-grading
- 5% possibility to edit ready-made databases
- 4% database statistics
- 4% possibility of adding graphics and sound
- 3% developing the program along users suggestions
- 2% fast development of SuperMemo software
- 2% application of a scientific method
- 2% minimum information principle
- 2% reliability
- 1% large number of databases
- 1% the fact that it inspires creativity
Other responses: satisfaction, "everything", regulated
speed of learning, etc. The four most liked features are exactly the same as in a survey
presented a year earlier!
What do you dislike most in SuperMemo?
- 22% I like everything
- 18% dislikes in reference to SuperMemo 6 for DOS
- 7% the fact that I have to work systematically
- 5% no possibility to resize question and answer area (available as of
SuperMemo 7.3)
- 5% incomplete Polish manual (esp. unintelligible description of the
options on the Analysis menu)
- 5% no possibility to change a mistakenly given grade (available in SuperMemo 8)
- 5% International Phonetic Alphabet is not used in Basic English nor
Advanced English (available in SuperMemo 8)
- 4% upgrades are too expensive
- 4% Database Toolkit is not integrated with the program (integrated as
of SuperMemo 7.4)
- 3% the need to recover unclosed databases (not relevant in SuperMemo
8)
- 3% self-grading
- 3% no way to stop repetitions during a vacation period
- 3% bugs (in reference to version 7.1 and 7.2 for Windows)
- 3% program is too big (or too slow)
- 2% long response time from SuperMemo World (test this e-mail to see
if this is still true: SuperMemoMail)
- 2% typing in items when creating my own database
- 2% the size of the main window is not remembered (available as of
SuperMemo 7.3)
- 2% poor quality help (in reference to the Polish version)
- 2% no way to upgrade Advanced English upon getting a new release
- 2% poorly designed interface (in reference to SuperMemo 7.1 and 7.2)
- 2% bugs in databases
- 2% too many parameters displayed on the screen (in SuperMemo 7.5 the
number of parameters can be restricted by the user)
- 1% databases are too expensive
- 1% grades cannot be fractional
- 1% no way to type in the correct answer (available as of SuperMemo
7.4)
- 1% searching only one database at a time
Other responses: it is difficult to edit wave files, strange
database format, the need to work on weekends, it is too easy to delete items, renumbering
items at Garbage, slow Search, too few databases, no spell checker, imposing an English
version on Poles, no autostart program that would show the number of outstanding items
each time one starts Windows, exaggerating the increase in the speed of learning (50
times?), AdvEng manual in English, deleted items leave empty places in a databases (until
Garbage), etc.
Which options do you use most often (apart
from Open, Learn and Add new items)?
- 34% Search
- 32% Analysis
- 23% Burden
- 17% Edit
- 17% Garbage (not needed in SuperMemo 8)
- 13% Monthly burden
- 13% Interval distribution
- 11% A-factor distribution
- 8% Random test
- 6% I use all options
- 5% Date
Which options do you never use?
- 23% Wipe (replaced with intelligent Mercy in SuperMemo
8)
- 16% Mercy
- 14% Tools : Analysis : Matrices
- 14% Random test
- 8% Help
- 7% Garbage (not needed in SuperMemo 8)
- 5% Date
- 4% Approximate (not needed in SuperMemo 8)
- 3% Search
Proposed improvements to SuperMemo
- 13% good manual in Polish (the survey was done in Poland only)
- 12% possibility of changing once-given grade (available in SuperMemo 8)
- 4% possibility to type in the correct answer (implemented as
Spell-Pad in SuperMemo 7.4)
- 4% integrated database toolkit (implemented in SuperMemo 7.4)
- 3% printing a range of items (implemented in SuperMemo 7.5)
- 3% associating images with answers (available in SuperMemo 8)
- 3% resizing question and answer window area (implemented in SuperMemo
7.3)
- 3% possibility to merge databases (included in Database Toolkit in
SuperMemo 7.4)
- 3% transferring items between databases (implemented in SuperMemo
7.4)
- 3% swapping questions with answers (implemented in SuperMemo 7.4)
- 3% help in Polish as good as in the English version (the survey was
done in Poland only)
- 3% importing DBF files (possible upon converting DBF files to text
files)
- 3% different font for question and answer fields (implemented in
SuperMemo 7.4)
- 3% automatic database back-up with database compression
- 3% adding sound to Advanced English (available on SuperMemo 7.4
CD-ROM)
- 2% associating many items with one bitmap file (implemented in
SuperMemo 7.4)
- 2% multiple fonts in the same item (available in SuperMemo 8)
- 2% keep the size of the main window in INI files (implemented in
SuperMemo 7.4)
- 2% sharing database files between users (available in SuperMemo 8)
- 2% providing a name of a database to open as a parameter of the
program (implemented in the Windows version)
- 2% searching only question fields (implemented in SuperMemo 7.3)
- 2% word-wrapping in the item editor (implemented in SuperMemo 7.2)
- 2% case-insensitive Search (implemented in SuperMemo 7.4)
- 2% possibility to change the sequence of items in a database
(implemented in SuperMemo 7.4)
- 2% Random test on difficult items (implemented in SuperMemo 7.4)
- 1% Random test on memorized items (implemented in SuperMemo 7.4)
- 1% use International Phonetic Alphabet in transcription (available in
SuperMemo 8)
Other interesting propositions included: real learning time
measurement, context-sensitive help, verbal description of statistical parameters,
colorful dialog boxes, nicer icon, keep a list of databases to open in sequence, weekly
calendar with no repetitions on selected days, different keyboard layouts associated with
databases, more informative list of available databases (DBANK.TXT), search going through
a list of databases, integrated on-line dictionary, screen saver, viewing items falling
into a selected E-factor category, removing bugs from databases Basic English and Advanced
English, insert graphics into the text of items, possibility to sort items alphabetically,
spellchecker, compression of graphic files, memo field for user comments, program for
activating SuperMemo upon starting Windows in case the number of outstanding items is
greater than zero, etc.
Most requested learning material
- 7% English (e.g. Advanced English in Polish, Intermediate English,
Technical English, etc.)
- 5% German (e.g. Advanced German, Basic German, etc.)
- 5% History
- 5% Russian
- 3% French
- 2% Geography
- 2% Physics
- 2% Math
- 2% Chemistry
- 1% Biology
Other neglected fields: Electronics, Telecommunications, Economics,
Esperanto, etc.
What should SuperMemo World do to make
illegal users legalize their copy?
- 25% keep low upgrade prices for legal users
- 17% reduce the price of SuperMemo
- 15% competitions, reward draws, newsletters, promotions, surprises,
SuperMemo Club, etc.
- 11% one-time legalization action (legalizing on special conditions)
- 10% let illegal users register at a lower price (see remarks below)
- 8% one-time amnesty
- 5% seasonal decrease of the prices (e.g. before Christmas)
- 5% let them register in return for a database (NB: this has always
been an option for illegal users)
- 4% constantly develop SuperMemo as software
- 4% release a CD-ROM with all available databases
- 3% develop more databases
- 3% let them buy the manual at reasonable price
- 3% hunt and punish illegal users (see remarks below)
- 2% release shareware version (NB: SuperMemo 6 is available in public
domain)
- 2% copy-protection (see remarks below)
- 1% there is no way to make it happen
- 1% bundle SuperMemo with other products, computers, etc.
Other opinions: "SuperMemo World did all it could be letting
illegal users register in return for a good database!", Include biblical verses
indicating sinful nature of piracy, etc.
Remarks: SuperMemo World does not consider allowing registration at
a lower price as it does not seem fair to those who paid the full price. Similarly,
SuperMemo World does not consider punitive actions against illegal users hoping that they
can be attracted by affirmative actions. Finally, copy protection cannot be considered
because inconvenience it imposes on legal users.
What would you like to get on SuperMemo CD-ROM?
- 17% not interested (e.g. because of lack of CD-ROM drive)
- 10% hypermedia collections
- 10% many collections (not necessarily multimedia)
- 8% tools for editing collections
- 4% dictionaries
- 3% games
- 2% I do not know
- 2% Advanced English Audio (available on Advanced English CD-ROM)
- 1% English Grammar Audio (available on Multilingual CD-ROM)
- 1% Basic English Audio (available on Advanced English and
Multilingual CD-ROMs)
- 1% German Audio (available on Multilingual CD-ROM)
- 1% full manual (available in English on all CD-ROMs)
Others: French Audio (available on Multilingual CD-ROM), Russian Audio (available on
Multilingual CD-ROM), English Vocabulary (by M.Cajza), demo, Picture Editor, songs in
foreign languages, etc.
Which application software do you use most
often?
Note that the survey was conducted in 1994 therefore you will not find many of the
currently popular applications on the list! Endemic Polish software is marked with PL.
Most often used application software (among questioned users of
SuperMemo);
- 39% word processors, of which:
- Word 13%
- TAG 9% (PL)
- AmiPro 7%
- Chiwriter 4%
- WordPerfect 2%
- other 4%
- 37% SuperMemo
- 16% spreadsheets (mainly Excel, Quattro Pro, 1-2-3)
- 15% Borland Pascal
- 11% database systems (mainly FoxPro, Access, dBase)
- 5% dictionaries (mainly Skalmierskis dictionary)(PL)
- 5% C, C++
- 5% DTP and graphics (mainly Corel, Picture Publisher)
- 4% educational software (most popular: Media Euro (PL), Pop English
(PL), Pop Deutsch (PL), E-Teacher (PL), E-Teacher Deutsch (PL))
- 3% Auto CAD
- 3% scientific (Derive, TEX)
- 2% professional applications related to ones job
- 1% assembly languages (mainly MASM)
- 1% games (Doom)
- 1% other applications (e.g. antiviral software, PKZIP, Works for
Windows, Organizer, Makler (PL), Quicken, ARJ, e-mail software, modem software, etc.)
What would make you buy SuperMemo earlier?
- 50% getting to know it earlier
- 19% lower price
- 10% fewer slogans, more facts (e.g. in advertising)
- 6% getting SuperMemo shareware
- 2% recommendation of other users
- 1% getting samples of databases
- 1% greater availability in shops
What would make your colleagues buy
SuperMemo?
- 34% lower price
- 14% more concrete information in press
- 11% more positive opinions from friends
- 5% more databases in the offer
- 5% using shareware
- 4% more motivation for work on their part
- 3% access to a computer
Where would you like to find advertising of
new SuperMemo products (in Poland)?
- 47% PC Kurier
- 41% Enter
- 14% Gazeta Wyborcza
- 13% Bajtek
- 11% PC Magazine
- 9% PC World
- 8% Chip
- 6% Computer World
- 5% foreign press (Byte, PC World, etc.)
- 5% Wiedza i Zycie
- 3% Informatyka
- 3% Rzeczpospolita
- 3% daily newspapers
Other statistics
Only 3% of respondents were women!!!
Although 60% of users claimed that SuperMemo for DOS should
continually be developed, SuperMemo World gave up the development because of a dramatic
drop in user inquiries. We decided to release SuperMemo 6 Public Domain for DOS instead.
Optimum price for SuperMemo on floppy disks in Poland was proposed
to be PLN 76 (about $20).
36% of respondents were interested in buying a CD-ROM with all available SuperMemo databases (see SuperMemo
MegaMix)
See also: Survey 1999 and Survey 1993