Implementing SuperMemo with sm8opt.dll

Piotr Wozniak, Dec. 17, 1996, Last updated: Sep 21, 2000

SM8OPT.DLL is a library designed for software developers intending to implement the SuperMemo method in their software. Licensing conditions for using SM8OPT.DLL will differ depending on the target application. In particular, no license fee is required for projects whose sole purpose is research on repetition spacing or on the Algorithm-SM8 used by SM8OPT.DLL. Nevertheless, written permission from SuperMemo World must be obtained beforehand in all cases.

SM8OPT.DLL can be called from any application as any standard Windows DLL library. The job of the developer is to:

Here is the detailed algorithm for using SM8OPT.DLL. When calling individual procedures use pascal calling convention:

  1. Call GetOptimizationRecordSize to obtain the size of OptimizationRecord.
  2. Allocate a memory block of the size determined in Step 1. This block will be used to keep OptimizationRecord throughout the learning session.
  3. In the first session, call CreateNewOptimizationRecord(OptRec) where OptRec is an untyped pointer to the memory block allocated in Step 2. This will initialize the value of OptimizationRecord. In all following sessions, load OptimizationRecord from a file created in Step 12 (NB: in SuperMemo, the optimization record is stored in knowledge system info subdirectory as a file: sm8opt.dat).
  4. After creating or loading OptimizationRecord in Step 3, call InitializeOptimizationRecord(OptRec) that will create all temporary variables used by the repetition spacing algorithm. You do not need to access memory pointed to by OptRec. Only sm8opt.dll will manipulate individual fields
  5. By analogy, for each item call GetItemOptimizationDataSize, allocate memory for ItemOptimizationData, and call CreateNewItemOptimizationData(IOD) where IOD is a pointer to the allocated memory block. Alternatively, reload ItemOptimizationData from a file.
  6. Having OptimizationRecord and ItemOptimizationData available in memory (pointers OptRec and IOD), repeat the procedure of executing individual repetitions presented in Steps 7 through 12.
  7. Using ItemOptimizationData.NextRepetition field (IOD^.NextRepetition) determine the date of the next repetition for each individual item (see later for more on interpretation of NextRepetition value).
  8. On the date of repetition call GetRepetitionData(IOD) with IOD pointing to ItemOptimizationData. This will set the new value of ItemOptimizationData with the field NextRepetition pointing to the day on which the next repetition will take place (see later for more on interpretation of NextRepetition value).
  9. Before calling GetRepetitionData, set two fields of ItemOptimizationData: Grade - the grade provided in the course of the repetition, and Today - current date relative to the first day of the learning process (see later for more).
  10. GetRepetitionData can be called several times in succession (e.g. for grade-undo purposes). Note that returned values may differ slightly due to the stochastic nature of determining the next interval to be used.
  11. Call CommitOptimizationRecord(IOD) after each repetition (e.g. at the moment of choosing the Next Repetition button). CommitOptimizationRecord updates all temporary variables of the repetition spacing algorithm, updates OptimizationRecord (untyped pointer to this record was provided in Step 4 with InitializeOptimizationRecord and does not have to be specified again) and does some simple conversions on the ItemOptimizationData record (e.g. Repetitions:=NewRepetitions, AFactor:=NewAFactor, etc.).
  12. Save ItemOptimizationData to a file (the new value of all fields was determined in Step 8 by calling GetRepetitionData). You need to make the following substitutions here:
    • OldInterval:=NewInterval
    • Repetitions:=NewRepetitions
    • Lapses:=NewLapses
    • AFactor:=NewAFactor (just copy the string of six bytes)
    • UFactor:=NewUFactor
  13. Upon completing the series of repetitions, save the OptimizationRecord to a file. Call FreeOptimizationRecord and deallocate the memory blocks allocated in Step 2 and Step 5.

Here is the definition of ItemOptimizationData record (please note that it uses the alignment to 32-bit boundaries and should originally return the size of 132 bytes):

TItemOptimizationData=record

Today:word; {relative to the first day of the learning process. On the first day, Today=1}{2 bytes}
Grade:byte; {grade from 0 to 5, where 5 is the best grade}{1 byte}
FirstGrade:byte; {the first grade obtained by the item during its second repetition; i.e. the repetition after the first interval}
LastRepetition,NextRepetition:integer; {relative to the first day of the process}{4 bytes}
OldInterval,UsedInterval,OptimumInterval,NewInterval:integer;
Repetitions,NewRepetitions:byte;
Lapses,NewLapses:byte;
RequestedFI:byte;
Ordinal:real; {use six byte arrays instead of floating types here; see below}{6 bytes}
AFactor,NewAFactor:real; {use six byte arrays here and below}
UFactor,NewUFactor:real;
OldRF,NewRF,OldOF,NewOF:real;
Cases:word;
EstimatedFI,ExpectedFI:real;
NormalizedGrade,NGMin,NGMax:real;
RepetitionsCategory:real;
Reserved:integer;
end;

Important! Floating point number (the real type) are represented here as in Object Pascal and use six bytes. Use six byte arrays in place of real numbers to ensure the size item optimization data structure is correct. If the size of TItemOptimizationData record is incorrect, you will not be able to correctly read data written to it by sm8opt.dll

The only fields that are indeed needed by the developer are: Today, Grade and NextRepetition. Other fields can be used for diagnostic or information purposes. Today and NextRepetition are counted relative to the first day of the process. On the first day of learning, Today=1, and the repetition takes place on the date on which NextRepetition=Today. Note that LastRepetition and NextRepetition values may be negative in cases where items have been transferred from another system that was created before the initialization of the system in use.

As some of the fields of ItemOptimizationData are computed anew, only a subset of TItemOptimizationData has to be stored in a file. Developers that would want to save disk space may note that only the following fields have to be stored: FirstGrade, LastRepetition, OldInterval, Repetitions, Lapses, RequestedFI, Ordinal, AFactor, and UFactor. Most notably, NextRepetition does not have to be stored as it can be derived from NextRepetition=LastRepetition+OldInterval. The new value of LastRepetition equals Today. The definition of TOptimizationRecord is not presented here as it is quite meaningless without understanding the details of the repetition spacing algorithm:  Algorithm-SM8. If you want to interpret the contents of this record, you can best copy it in place of sm8opt.dat in the info subdirectory and use Tools : Statistics : Analysis in SuperMemo to view the graphs.

The use of GetOptimizationRecordSize and GetItemOptimizationDataSize is obligatory as in future releases of SM8OPT.DLL, the size of the optimization data may change. Obviously, there will be no change to the present structure apart from adding new fields taking part in enhanced optimization. For example, the RepetitionCategory has been introduced in December 1996 in a new release of Algorithm SM8 in which interval categories instead of repetition numbers are used to index the matrix of optimal factors. This prevents fooling the algorithm in cases the actual repetition takes place long after the scheduled optimum date.

Here is the list of all procedures published in SM8OPT.DLL. Make sure you use pascal calling convention:

You can download encrypted SM8OPT.DLL from this site (32-bit version only). To do this:

  1. Download 32-bit version SM8OPT.ZIP (120,000 bytes)
  2. Write to support

Optional

In case you want to interpret Object Pascal real numbers for debugging purposes, you can use the following function:

function GetInt1000FromReal(RealNumber:pointer):integer;

GetInt1000FromReal is defined in sm8opt.dll, takes a pointer to the six-byte array storing the real number and returns a 32-bit integer equal in value to RealNumber^*1000

In Object Pascal, a 6-byte (48-bit) Real number is divided into three fields:

  1. s - sign, the most significant bit
  2. f - mantissa, the remaining 39 bits of the first five bytes
  3. e - exponent, the sixth byte

The value v of the number is determined by the following:

if 0 < e <= 255, then v = (-1)s * 2(e-129) * (1.f)
if e = 0, then v = 0

Frequently Asked Questions

(Rollie Tyler, USA, Sep 9, 1998)
Question:
I am trying to implement SuperMemo algorithm with your sm8opt.dll library. Why don't you describe the contents of TOptimizationRecord? How am I supposed to use a typed pointer to a structure that is not defined?
Answer:
Contents of TOptimizationRecord is very difficult to interpret. It contains multidimensional arrays and various parameters needed internally by the algorithm. All calls to sm8opt.dll require only an untyped pointer which points to a memory block which you treat like a meaningless string of bytes.
If you want to interpret the contents of this record, save it as a file and put it in place of sm8opt.dat in the info subdirectory. Then you can use Tools : Statistics : Analysis in SuperMemo to interpret its contents (note that the graphs on the Distributions tab are independent of sm8opt.dat and will not change as a result of the substitution)

(Yuan, China, Jun 5, 2000)
Question:
I wrote a wrapper module that calls sm8opt.dll; however, I keep on getting short intervals that do not differ much between items that get good grades and bad grades
Answer:
The most likely reason is that you failed to update and save TItemOptimizationData. Remember that you need to assign NewInterval to OldInterval, etc. 

(Kevin, USA, Sep 20, 2000)
Question:
The solution for interpreting real numbers is quite clumsy. Will this be changed in the future?
Answer:
The function for inspecting real numbers was added only for those who are curious of the content of TItemOptimizationData. Your program does not need to use any of these numbers. Changing data types would affect compatibility of sm8opt.dat records in various versions of SuperMemo which can all exchange data by simply copying the file. This compatibility is very important for further research on improving the algorithm used in SuperMemo

 

Effect of recompiling SuperMemo DLL with newer Delphi (#15716)
(Mark G. Patterson, Saturday, January 25, 2003 6:43 PM)
Question:
As the current version of Delphi has moved away from old real data types, is it possible for you to recompile Algorithm SM-8 DLL with the current version of Delphi?
Answer:
It is vital to retain the size of individual fields of data for easy exchange between all versions of SuperMemo. For that reason the compiler directives in the code are set to ensure the same unaligned data size independent of the actual compiler. In other words, the size of data and its interpretation does not change between Delphi 1.0 and Delphi 7.0

You need to correctly initialize TItemOptimizationData (#11862)
(Kubiak, Konrad (GE77), Germany, Monday, July 15, 2002 10:06 AM)
Question:
I am trying to use sm8opt.dll in my software. I get the following error: "A-Factor is 0.000 in Opt.AF2Categ"
Answer:
This error indicates that the A-Factor field in TItemOptimizationData is set to zero. Most likely you have not initialized this field correctly or the field is misaligned. Please note that TItemOptimizationData uses the alignment to 32-bit boundaries and should originally return the size of 132 bytes

(Kevin, USA, Sep 20, 2000)
Question:
I would like to use your DLL in Visual Basic. Can you provide it as an ActiveX control?
Answer:
There is no ActiveX version at the moment. However, with a dose of extra effort, you should be able to call sm8opt.dll from Visual Basic as well