Memory graphs (4D)

SuperMemo 18 uses a new spaced repetition algorithm denoted Algorithm SM-18. Unlike all prior algorithms that were either theoretical or "inspired by data", this algorithm has been developed entirely on the basis of prior records of repetitions collected by users of SuperMemo. This data-driven effort required untold hours of analysis while processing millions of repetition samples. Toolkit : Memory : 4D Graphs was instrumental in that analysis and debugging process. If you want to understand the algorithm and help improve it further, please study those tools and keep analyzing your own data and your own memory. In a stochastic system of memory, perfection is impossible, but we should always try to come closer to the optimum.

Important: To see nice graphs as shown in the pictures below, you need to use a collection with a mature learning process. New collections have no memory data to show.

Available memory graphs

All memory graphs provide a 3-dimensional view with rotation along all 2 axes (X and Y), and a slider for animation in the 3rd dimension along item difficulty.

The following memory graphs are available with Toolkit : Memory : 4D Graphs on its individual tabs:

• Matrices -
  • S90 - stabilization at retrievability of 90%
  • SIncMin - minimum stabilization in the stabilization curve
  • SIncMax - maximum stabilization in the stabilization curve
  • Gain - spacing effect expressed as a stabilization gain in the stabilization curve
  • Decay - stabilization decay in the stabilization curve
  • OS - optimum stabilization derived from stabilization increase using approximation formulas analogous to those used in deriving the matrix of O-Factors from the matrix of R-Factors
  • Recall - actual recall measured at the predicted retrievability of 90%
  • R90 - predicted retrievability at recall of 90%
  • Metric - recall metric that shows the accuracy of recall predictions for different levels of stability and difficulty
  • Cases - repetition cases used in computing the stabilization matrix
  • RF - R-Factor matrix of Algorithm SM-15
  • OF - O-Factor matrix of Algorithm SM-15
  • Intervals - optimum intervals matrix
  • RF Cases - repetition cases used in computing the R-Factor matrix of Algorithm SM-15
• Stabilization: the function that tells you how much memory stability increases with a repetition. If you click Compute, all repetition histories will be scanned to compute the SInc[] matrix. For a given item difficulty, the increase depends on stability and retrievabilty. To view the 3-rd dimension of difficulty, slide the thumb on the Difficulty slider. See example
• SInc Approx: approximation of the function represented by the stability increase matrix. If you click Compute, SuperMemo will look for the best fit to data (which is the SInc[] matrix computed on the Stabilization tab). See example
• Recall: the function that tells you how well actual recall corresponds with retrievability predicted by SuperMemo. The difference between comes primarily from the difficulty in formulating perfect items that would produce purely exponential forgetting. In addition, SuperMemo itself is far from perfect in sorting items by difficulty. See example
• Recall Approx: approximation of the function represented by the Recall[] matrix. If you click Compute, SuperMemo will look for the best fit to data (which is the Recall[] matrix computed on the Recall tab). See example
• First Interval: the function that illustrates post-lapse stability depending on the number of lapses and retrievability at failure. It is the equivalent of the first-interval graph extended into the retrievability dimension. Retrievability is important in that a failure at low retrievability does not need to signify the item is difficult and post-lapse stability is low. See example
• PLS Approx: approximation of the function represented by post-lapse stability matrix. If you click Compute, SuperMemo will look for the best fit to data (which is the post-lapse matrix computed on the First Interval tab). See example
• Sleep: correlation between recall and the two components of sleep. This is a 2-dimensional graph that combines Alertness (H) and Alertness (C) of Sleep Chart. As the data is not part of Algorithm SM-18 optimization, you need to click Compute to collect data needed to display this graph
• Forgetting: 3D forgetting curves collected by Algorithm SM-15. This is the same graph as the one displayed by Toolkit : Statistics : Analysis : 3-D Curves with the added benefit of 3-axis rotation in space and A-Factor slider for the 4th dimension.
• RF Matrix: 3D RF matrix collected by Algorithm SM-15. This is the same graph as the one displayed by Toolkit : Statistics : Analysis : 3-D Graphs : R-Factor Matrix with the added benefit of 3-axis rotation in space

Graph analysis controls

• X and Y axis rotation (top 2 sliders)
• Difficulty slider (for animation in the 3rd dimension)
• Repetition cases in consideration
• Cases: the label showing the total number of repetition cases in consideration
• Slices - xxx not implemented at the moment (if it does not show up by release time, this will have to be removed)
• Compute: recompute the graph using the data in the collection
• Reset: reset the memory matrices
• Smoothing: average neighboring entries in matrices
• Subset: select a subset of elements for which matrices should be computed
• Reset Cases: reset the count of repetition cases without changing the data (i.e. values of entries in matrices)
• Export: export data for analysis in Excel
• Average checkbox: the "golden mean" average of the data with:
  1. the best-fit approximation, and
  2. data-rich neighboring entries in proportion to available information

Pictures

Stability increase function

Figure: Stability increase function as computed by Algorithm SM-18. The function takes three arguments: (1) stability at review expressed in days (on the left), (2) retrievability at review (on the right), and (3) memory complexity expressed as item difficulty (slider labelled Diff currently set at 0.8). In the picture, stability increase peaks at around 15 (vertical axis). For some levels of stability and retrievability, the function drops below 1.0 indicating a decrease in stability (e.g. due to premature review evoking the spacing effect in massed presentation). 61,768 repetitions of relatively difficult items were needed to produce this graph (at Diff=0.8). The longest intervals reach around 14 years (stability quantile 5172)

Stability increase function contour map

Figure: A "from above" view at the SInc[] matrix providing a contour map. Red zones indicate high stability increase at review. The picture shows that the greatest stability increase occurs for lower stability levels and retrievabilities around 70-90%.

Stability increase approximation

Figure: Approximating the SInc[] matrix with the best-fit function used by default in SuperMemo to compute the increase in stability (e.g. in cases of lack of data). The approximation procedure uses a hill-climbing algorithm with parameters A, B, C, D displayed in the picture. Least squares deviation is obtained to assess the progress. Green circles represent the Sinc[] matrix at a chosen difficulty level. Their size corresponds with repetition cases investigated. The blue surface is the best fit of the studied function to the SInc[] data.

Recall

Figure: The Recall[] matrix graph shows that the actual recall differs from predicted retrievability. For higher stabilities and difficulties, it is harder to reach the desired recall level.

Recall approximation

Figure: Approximating the Recall[] matrix with the best-fit function to compute default recall in conditions of data scarcity. The approximation procedure uses a hill-climbing algorithm with parameters A, B, C, D displayed in the picture. Least squares deviation is obtained to asses the progress. The circles represent the Recall[] matrix at a chosen difficulty level. Their size corresponds with repetition cases investigated. The red surface is the best fit of the studied function to the Recall[] data.

Recall approximation curve

Figure: Approximating the Recall[] matrix with the best-fit function to compute default recall in conditions of data scarcity. By choosing the right viewing angle, the curve that reflects the changes to recall with retrievability can be seen in abstraction of stability. In this case the relationship is almost linear (the logarithmic bend is a result of the log scale used for retrievability).

First interval

Figure: The relationship between the first interval after failure, retrievability at review, and prior memory lapse count.

First interval approximation

Figure: Approximating the impact of retrievability and memory lapses on the post-lapse stability.