Skip to main content

Leveling System

How It Works​

The Actuarial Nexus implements a binary variation of an ELO rating system, with the binary conditions being correct answer or incorrect answer. An ELO rating system is a method for evaluating the skill levels of players in a competitive game or sport. It was originally developed for chess, but it has since been adapted for use in a variety of other contexts.

Pass rates have generally been stable over time, particularly for longstanding exams such as Exam P. This implies that, on average, a fixed X% of candidates pass each exam sitting. Since an ELO rating system is designed around relative ranking, the higher the user's level, the greater chance they have at passing the exam.

A user's rating and the question's rating are adjusted based on the outcome of the user's results on the problem, with points being added or subtracted based on whether the problem is correctly or incorrectly answered. The amount of points that a user gains or loses is determined by the relative level of the problem. For example, a level 6 user who correctly answers a difficulty 10 problem will earn more points than a level 6 user who correctly answers a difficulty 1 problem.

The rating system is designed to be self-correcting, so that a player's level will tend to converge towards the user's expected exam score over time. Factors that play into the leveling system include the difficulty of the problem, the user's current level, the user's number of attempts on the same problem, the differential between the problem's average solve time and the user's solve time, and the user's total attempts on all problems. This can help ensure that the ratings are accurate and fair, and that players are not abusing the platform to increase their level. For example, a level 1 user who only solves level 1 questions will not be able to significantly increase their level without attempting harder problems.

Limits​

The ultimate goal of a user level is to provide students with an engaging tool to pass actuarial exams. A user who attempts to game the system for the sole purpose of manipulating their level will not only hurt their own progression, but will also skew the question level for other users. At the end of the day, we rely on the community to help maintain the integrity of the system. Some measures to prevent abuse include the following:

  • Questions solved in less than 10 seconds will not impact your level or the question level.
  • Each successive solve of the same question results in a dampened impact on the question level and the user level.

Level History​

Since all questions attempts are stored on the platform, users can view their complete question attempt history and the impact of each question on their level. Additional details can be found in analytics.