Data Matters when it Comes to Engagement

Share on facebook
Share on twitter
Share on linkedin

I recently purchased a new VR game called “In Death: Unchained” for the Quest 2, and the best word to describe how I’ve felt about it would probably be “obsessed”. Even though I’ve been playing it for months now, I’m amazed at how well the game has managed to retain my interest. The amount of time I’ve invested into launching arrows at virtual foes easily surpasses that of any other game or application I own on the platform. In addition to the hours of fun the game has offered, it’s also gotten me thinking about just how big of a role data plays in the task of keeping users engaged.

When the topic of data collection is brought up, it is most often in relation to enhancing ROI analyses and learning effectiveness. While these are certainly important topics to consider, we rarely take the time to consider how data can be utilized to keep learners engaged. Yet, many digital games manage to leverage their platform’s inherent ability to collect vast amounts of data to accomplish just such a goal, typically without the user even realizing it. Wouldn’t it benefit learners and developers alike if a similar methodology was used in the design of learning?

The following is a list of ways in which I believe data collection could be implemented to help enhance the effectiveness of engagement-based learning applications:

Tracking Progress

Let’s begin with a fairly obvious statement: tracking progression through the use of page numbers isn’t exactly subtle or interesting. What if progress was instead tracked by a user’s proficiency at a task, with the task becoming increasingly more difficult over time? I have long argued that the method games use to help players improve a skill should be used as a model that learning applications should follow. Similar to how we experience the real world, good games feed us small bite-sized portions of achievable tasks which become more and more challenging as we progress. While players may fail quickly at first, they are regularly inspired to continue trying until a noticeable level of mastery is achieved.

Games frequently encourage small amounts of progress over time, which demonstrates an understanding that growth is best achieved through incremental improvements. A critical challenge that learning professionals face is keeping track of learner progress and providing an effective way of showing that progress. It’s often difficult to actually see the progress made over time, since we usually have little to track it against. Yet, establishing a methodology that would provide learners with insight into their progress would be a powerful way to enhance engagement and encourage further learning. This fact is especially true for learners who believe they haven’t made any progress at all, as it would demonstrate progress and (hopefully) motivate them to continue improving.

Tracking Achievements

One of my favorite parts of playing “In Death: Unchained” is the achievement system. Achievements are small challenges or benchmarks set by the designers. They can be built into the normal flow of gameplay or demand that a player venture off of the beaten path to accomplish another less obvious goal. Whenever one of these tasks is achieved, the player is provided with a feedback mechanism, such as a trophy, badge, or some other award. Achievements can act as a driving force which directs a player/learner to push themselves beyond what is expected. As such, they are a useful for encouraging a large population of users across various skill levels to engage in the game or learning activity.

A few years ago, there was a push to gamify learning through the use of badging and achievements. While this was a step in the right direction, the process has failed to find a decent foothold in the learning community as the achievements ultimately didn’t mean anything. As game designer Roxanne Splitt, who is both my friend and former colleague, would frequently remind me: achievements must mean something to be effective. In order for an achievement system to work, the impact of accomplishing one of the learner’s goals must provide some sort of additional incentive. In games, achievement systems are good at inspiring engagement because they are often linked to some reward such as unlocking new abilities, skills, or other meaningful rewards. If the learning community wants to take advantage of the benefits this system offers, we need to do a better job of conceptualizing how achievements function both within and outside of a given learning application.

While I intend to explore this idea further in future blog posts, I feel it is important to emphasize that we can’t reward learners for reaching an achievement if we don’t have a method of tracking them. If we aren’t rewarding learners for their achievements, either within a given learning application or in their learning journey overall, we are missing a critical opportunity to engage and encourage further improvement. The only way to ensure that we are tracking individual achievement is through the collection of learner data.

Providing Post Activity Feedback

One of the most powerful aspects of any learning application is the ability to provide post activity feedback. Post activity feedback (sometimes referred to as after-action reporting) offers the learner a method of receiving direct input about their performance at the most vital moment: immediately after they have completed a task. The feedback itself can come in a variety of forms, ranging from “check your understanding” scores to highly detailed after-action reports. It has ALWAYS been my opinion that certain games do an incredible job of providing post activity feedback and the mechanism, when done well, adds a lot of motivation to improve or move forward. As such, I have committed myself to learning how to better utilize similar features in the design of my own learning applications.

The feedback mechanism utilized by “In Death: Unchained” is a powerful part of its gameplay, as it provides a robust set of details for how a player performed in their most recent “run”. The feedback is only displayed after the player fails their current attempt, and I can honestly say that there have been multiple occasions where I didn’t mind seeing my session end just so I could review my overall progress and metrics.

A critical aspect of achieving these goals as an industry will be identifying and implementing a more robust reporting mechanism. Such a system would need to be capable of handling the types of data required to create post activity feedback features comparable to those in modern games. Designers, developers, and – of course – learners deserve a set of protocols that can support this level of detailed, robust learner feedback. That is why one of my primary personal goals is to focus of creating post activity feedback mechanisms that make proficiency tracking an integral focus of the after-action design while simultaneously encouraging replayability.

Standing Among Peers

The last topic I’m going to discuss in this post is a subject of constant debate when it comes to data, analytics, and user engagement when developing learning applications. At some point during the design portion of almost every learning game, the topic of implementing a leaderboard or comparative analytic dashboard arises. Inevitably, we explore the benefits and failures of such features, for example they are great for high performers, but can be discouraging for those who are less confident in their abilities. Yet, through multiple attempts we have found that there is usually a way to design a ranking system that utilizes positive feedback in the form of improvement metrics. Not only does this supply a means of engaging the players, it manages to do so without disheartening them.

One of the things that kept me coming back to “In Death: Unchained” was seeing a steady increase in my percentile ranking as compared to other players. While it has pretty much topped out at this point, seeing the number regularly climb upward was a great source of motivation during the middle part of my gameplay journey. Again, there is a design issue that occurs when this number is low and doesn’t rise for a player/learner, but there are other strategies – such as achievement/progress tracking – that can be used to mitigate some of these concerns. I see this as simply another challenge we need to tackle as learning designers.

What is the critical element in being able to create a meaningful leaderboard? No surprise here: Data.

Surely, the need to collect, analyze and disseminate information back to the player/learner will only grow as we move further into the realm of immersive technologies, such as AR and VR learning applications. If you can think of other ways in which data can drive learner engagement, please feel free to reach out and let me know. I am always happy to hear from you!

+ posts

Cary Harr is the Chief Learning Strategist for AppliedInfo Partners. He has over 25 years of experience in the industry including the design and delivery of classroom curriculum, immersive learning applications and modern performance support methodologies. Contact me at:

Explore our latest posts