Introduce spaced repetition for more learning on Brilliant

I’ve been using Brilliant for a few weeks now. In the last few years, reading the works of Nassim Taleb on probability, randomness and complex systems rekindled my desire to learn probability and statistics. Instead of brushing off some rather dreary-looking textbooks in order to refresh my foundations, I decided to turn to Brilliant instead after stumbling upon its wonderful culture page.

The emphasis on learning by doing has been refreshing. I appreciate that Brilliant provides instruction along the way instead of dumping it upfront. And I never fail to appreciate the reminders after I get a problem wrong…that without getting stumped, there is no learning!

Having used Brilliant a little bit, I had an idea for a new feature that might make it even better, and wanted to share that idea with you:

Introduce a “Redo Protocol”

The Idea…

Last spring I took a course through Manhattan Prep to prepare for the GMAT. If there was one thing my instructor made sure to drill into our heads, it was to prioritize our re-do log - to spend the _majority of our study tim_e redoing problems we’d previously gotten wrong rather than on doing new problems. 

The idea was this: You might for a short while understand the explanation for a problem you answered incorrectly, but you’d need to actually redo the problem - several times, over the course of a few weeks - in order to retrieve and, in so doing, internalize the new concept or approach such that you could apply recognize it or apply it in new situations. In other words, you’d need to redo problems in order to learn. Doing lots of new problems - without fully juicing them - was a trap. It would leave you wondering why your score had not improved despite dedicated effort.

Instructors at Manhattan Prep say that students who prioritized the redo log improved their scores much more. While this evidence is admittedly anecdotal, this is by no means a new or isolated observation. 

I like Scott Young’s framework for the makeup of a good study routine. He names 5 key ingredients:

From scottyoung.com: “The Perfect Studying Routine.”

I think Brilliant excels in “Instruction” (just what you need), “Retrieval” (emphasis on learning through doing over passive review) and “Feedback” (I find the explanations very helpful in general).*

By supporting “Spacing” (and “Retrieval”) through a redo protocol, Brilliant can make it easier for learners to execute the perfect study routine.

*I didn’t address the “Understanding” component. Brilliant’s emphasis on learning through doing is the core component here. But I think Brilliant might get even better in the “Understanding” department by making it simpler for users to leverage the Feynman technique and/or Rubber duck debugging within the fabric of each course.

…in Practice

The current “Saved” problems log provides the groundwork for a redo protocol. The main thing to add is the automatic reminders to redo it! That said, here are some specific things I’ve been chewing on that I think a good Redo Protocol might have or do:

• Automatically add any problem you get wrong to your redo log.
• Ability to add/remove problems to your redo log manually if you so choose.
• Automatic reminders to redo these flagged problems; 2 days out, 1 week out, and 2 weeks out. [These specific checkpoints are a general idea. There may be some studies out there that offer suggestions, but also this could just be tested.]
• Once you get it right [3]x in a row, it’s removed from the redo log.
• Automatically reset the problem for you before you redo it, so you don’t accidentally see the explanation before your next attempt.
• If you get the problem wrong again in the course of the redo’s, start from the beginning of the redo protocol: return to the problem 2 days, 1 week, 2 weeks later.
• Give you credit in the system for redoing it (right now it looks like I haven’t done anything in Brilliant, even though I’ve done my redo’s offline).
• Encourage you to write down a takeaway after viewing the explanation to a problem they’ve gotten wrong. E.g., what did you need to know in order to solve this problem? This is the first step towards cementing a new insight via active retrieval.
• I’d want to play with how much to really build in the Redo Protocol into the fabric of the user experience. As I’ve described it above, it could feel a bit distinct from the courses themselves. I wouldn’t want this to feel like an added chore separate and on top of progressing in the course. To avoid this, maybe the user simply encounters problems from the/her Redo Log semi-randomly in the course of taking quizzes as currently designed.

That’s all for now. I love what you all are doing, and look forward to seeing you grow!