Reflections on three years of spaced repetition with Anki

I was looking at my Anki deck stats the other day and realized that I have been using it for just over three years now. During that time I have added 20k cards and reviewed 140k. On average I spent 17 minutes each day to review 130 cards. Since this amounts to over 300 hours of my life at this point, I figured it would be worth reflecting on this habit and deciding whether it is a worthwhile investment of time going forward.

WTF is Anki?

For the uninitiated, Gwern provides an excellent overview of spaced repetition and its effectiveness as a learning tool. I’ll focus the rest of this post on my own personal use-cases for the tool.

I first downloaded Anki during my first year of undergrad, less than 48 hours before taking my final exam for Latin and Greek roots in English. It was the classic university student use-case: how can I cram all this knowledge into my head for just long enough to pass next week’s exam? Even though Anki’s algorithms are optimized for long-term retention, they are still the best approach for short-term cramming.

Since that first encounter with spaced repetition, I have used it (somewhat) more successfully to build my foreign vocabulary, internalize mental models learned from other fields, and most recently to review math proofs in order to improve my retention while studying my masters degree part-time.

Remembering stuff is hard work

Reviewing flashcards for 15 minutes per day does not sound like a particularly large time investment, but it takes a disproportionate amount of mental exertion. Anki’s algorithm attempts to schedule cards such that you are just barely able to successfully remember them. The active recall principle tells us that it is at this point that review confers the maximum benefit in terms of long-term memory consolidation. If Anki showed you cards earlier it would be easier to remember them, but doing so would yield less benefit per review.

In this regard, remembering stuff is no different than any other skill acquisition. The study of deliberate practice tells us that skill improvement is not proportional to total practice, but rather to the amount of practice conducted at the outer edge of our abilities. A musician who plays through a song they already know by heart may have fun, but he does not improve as much as a musician who spends an hour deconstructing a song he cannot yet play comfortably, practicinga single chord over-and-over again until it becomes muscle memory. So 15 minutes of flashcards feels unpleasant, but it gives us greater memory benefit than hours of passive consumption (watching videos, reading articles, etc.)

Spaced repetition is not a replacement for learning

Trying to remember things you’ve already learned can be tough, but trying to remember things you never really learned in the firstplace is just downright frustrating. I learned this the hard way while attempting to learn frequency lists¹ of Spanish vocabulary through an imported csv file. It feels deceptively productive to create a large number of cards, but you may be surprised at how much more difficult they are to remember than cards you create yourself².

What anki does do well, is provide a mechanism for decoupling the active process of learning from the more routine process of remembering. Imagine you are learning some topic (a language, a field, a skill) and you resolve to spend three hours every weekend studying. The output of this focused effort spent studying is an increase in your knowledge. But you will also experience a decrease in knowledge caused by forgetting some proportion of your existing or just-learned knowledge. So next time you study, you may spend 95% of your time learning new material, and 5% “refreshing” on previously forgotten material.

The actual percentage forgotten of course depends on the person, how they study, what they are studying, how often they study, etc. But the key insight here is that you’ve got a leaky system that is not long-term stable. It just wouldn’t make sense to say I’ve got a curiosity about physics so I will spend one Sunday per year learning it because you’d probably spend the first half of the day trying to remember what you learned a year ago. For any level of intensity, there is some level of frequency at which you reach an equilibrium: you are treading water but staying in the same place.

Now let’s say you implement a practice of creating new flashcards at the end of your study sessions, and performing a 10-minute daily review regardless of whether you are studying that day or not. This effectively modifies our “learning system” to look like the chart below.

To some extent, we are systematically countering the effect of forgetting using review. No review system is perfect, so naturally the rate of forgetting will never equal zero. A lower rate of forgetting equates to a lower “equilibrium” point with respect to study frequency. Without spaced repetition, studying a topic once every 3 months may feel inefficient, because you’d need to spend a chunk of time “refreshing your memory” before learning new content. With spaced repetition, you will almost certainly find it much quicker to jump back into things.

I have found this immensely valuable while studying part-time towards my masters degree in analytics over the course of multiple years. Machine learning and statistics are extensions of underlying concepts in calculus, probability, and linear algebra. Having an deep understanding of fundamental concepts from these fields makes learning ML easier, but they are difficult to maintain when “used” so infrequently. By creating anki cards for key math proofs, I ensure that I encounter them with some minimum frequency in a problem-solving context, which prevents me from entirely forgetting them in between courses.

Memory is directional

Our ability to recall particular memories is known to be dependent on the context in which we learned them. We can conceptualize our memory as a graph, in which individual memories are encoded as nodes, contextual cues and relationships as edges between those nodes, and the remembering process as the task of traversing the graph to a particular node.

We’ve all had the experience of having something a word or concept on the tip of our tongue but not quite being able to remember it. We know the memory exists in our knowledge graph, but we lack sufficient connections between nodes to retrieve it in a timely manner.

Let’s take the example of learning a piece of foreign vocabulary using a typical front ↔ back flashcard. Reviewing this flashcard in both directions would strengthen the following connections (edges) in our knowledge graph:

1. Foreign word → English: strengthens our ability to recognize and comprehend the word when encountered in the external world (i.e. recognition)
2. English → Foreign world: strengthens our ability to produce the foreign word from “thin air” to express ourselves (i.e. production)

Recognition is generally easier than production, and may often be a prerequisite. But training recognition alone does not translate to production, and often gives us a false sense of confidence in our knowledge. This is why the Feynman technique is so powerful: being able to fully “produce” the knowledge (either a foreign word or a full explanation of a concept) is the holy grail of remembering.

We can take this concept even further. We have trained our recognition from a textual representation of the word, but could we recognize it when spoken to us in a noisy bar? We have trained production from a similar textual English word, but could we think of the foreign word just from looking at a picture of the concept, without sub-vocalizing in English first?

My vocabulary note type now include multiple fields, and selectively generate 5-6 cards depending on which fields I fill in:

1. Foreign language
2. Translation
3. Definition
4. Example
5. Picture
6. Synonyms
7. Audio

A beneficial side effect of having a custom note type is that I have to manually create my own cards rather than being tempted by bulk import. This turns out to be a benefit because the time spent crafting the card—picking a picture, choosing the example sentence, etc.—translates to better starting retention.

The bottom line

All things considered, active recall testing with Anki is probably the single highest ROI habit that I perform on a regular basis. 100 hours per year may sound like a big chunk of time, but I think it’s important to consider that a lot of this was downtime to begin with. If you consider that much of this time is spent on public transit, or waiting to meet a friend who is a few minutes late, then it likely has an even higher ROI.

That said, I have learned to become somewhat more selective about what I add, and to more frequently use custom note types which enable richer content and facilitate multi-directional connections in memory. If it’s worth adding a card at all, it is worth spending a few extra seconds to improve your chances of successfully recalling it months from now. I am also more willing to delete cards that aren’t working for me, including poorly created cards that I frequently fail to recall.

Further reading

• Anki Tips: What I Learned Making 10,000 Flashcards – Inspiration for this post, includes some easy-to-digest tips for using Anki more effectively.
• Augmenting Long-term Memory – An in-depth exploration of spaced repetition and its applications for both academic purposes and practical life, from the perspective of a physicist.
• Spaced Repetition for Efficient Learning – Gwern gives a great overview of the scientific literature w.r.t. spaced repetition.