JOSH: Hello, and welcome to Clearer Thinking with Spencer Greenberg, the podcast about ideas that matter. I'm Josh Castle, the producer of the podcast, and I'm so glad you've joined us today. In this episode, Spencer speaks with Rob Haisfield about designing for user failure, continuous onboarding and closing feedback loops, gamification and its social components, and how to develop expert intuition.

SPENCER: Rob, welcome. It's great to have you here.

ROB: Yeah, thank you for having me, Spencer. I'm really excited to talk with you and kind of just get nerdy about behavior change.

SPENCER: Me, too. We're both obviously obsessed with behavior change. So I think it's gonna be a fun conversation. And to that point, the first thing I want to discuss with you is this idea of intentionally designing for the moments when a user fails, for example, if you're building a product. So could you tell us a little bit about how you think about that?

ROB: Sure. I can just start with talking about an app that we're both working on, GuidedTrack. GuidedTrack is basically a very simple programming language. You can build apps, you can build surveys, experiments, interactive lessons, all of that, through a simplified custom-made programming language. And the thing about that though is, as people are writing code — it doesn't matter how easy we make it — people are still going to write incorrect code sometimes, and when they do, a lot of times, they can get discouraged. This is a problem that I see in all sorts of apps. Failure is this inevitable thing because the app is asking the user to do something that is challenging. A lot of times, users come to apps to help them accomplish some goal. And a goal by nature is a discrepancy between where you are and where you want to be, so that means that what you were already doing probably isn't working. So failure is this inevitable thing, and if we want people to continue using an app after they fail, then the lowest-hanging fruit for retention there is to intentionally design failure. That way, the user feels a greater self-efficacy, because they have a reason and a willingness to try again. And the next time they try, they're more likely to be successful than the previous time.

SPENCER: Can you give an example of that?

ROB: One of my favorite examples is streak counters. I see streak counters in all sorts of apps; they're in meditation apps, they're in fitness apps. A streak counter will note down the number of days in a row that a person has used the application. So if someone has an 80-day streak in a meditation app, then that means they've meditated 80 days in a row. But if they miss a single day — just a single day — then they have to start all over again all the way at zero and that can be really discouraging. The theory behind streak counters is pretty solid. It's based on this idea that people have loss aversion. They really don't like losing things that they already had, the endowment effect, meaning that they place more value in things that they take ownership over. With the streak counter, the idea is you build up a sense of ownership over your progress over time. You really want to avoid losing it, so you continue using the application every single day. But that failure state is really discouraging. And an instance where I see a similar concept that's executed (I think) a little bit better is in games, particularly a game called Hollow Knight and I'm gonna describe this game in very broad strokes right now. You explore around a map, you kill monsters, and you earn money as you go. Eventually, you're going to die — some monster is going to beat you — and when you do, you lose all of the money that you haven't spent yet, and you return to an earlier point in the map. So far, this sounds a lot like a streak counter, but Hollow Knight does it a little bit differently, insofar as, when you die, if you're able to fight your way back to the point where you lost, then you're able to earn all of your money back. You have a chance at redemption, and this can be a really motivating reason to try again. "I want to recover what I lost."

SPENCER: That's a really elegant solution to the problem. You're still getting punished for losing — so you don't want to lose — but you have this chance to make it back up again. So you're not totally demoralized after it happens.

ROB: Exactly.

SPENCER: I just want to mention a really striking version of streak counters, which is in Snapchat where (as I understand it) some people in Generation Z, they'll have these streaks of conversations in Snapchat with many, many friends — let's say 50 friends — and after they've been doing this for a long time, there's a lot of social pressure to keep them going. So there are these issues that come up where, let's say, a kid is grounded and they say to their parents, "Oh, no, but please don't let my streaks die because it's gonna look really bad for me socially, and people are gonna be annoyed with me." And so they'll ask their parents to keep up their streaks, or they'll get special permission just to keep up their streaks, which I feel is taking this idea but to an extreme level that may be a little disturbing.

ROB: Yeah, absolutely. And I think that some of that just comes through the sense of permanence, of finality, that comes with losing one of those streaks. Because again, as soon as you lose like a 100-day streak, then that means it starts over at zero. And what is that streak signaling to the user? It's signaling the strength of the relationship and, starting at zero, that's sending a signal almost like you don't care. I think that the problem there has to do with how intense the failure is.

SPENCER: Right. It's interesting, that example from Snapchat, because it's achieving its goal of getting people to use Snapchat, but it's also having this negative second-order effect of causing distress and creating these weird social situations, so it's interesting to think about. On the one hand, is it motivating the behavior of using the app? On the other hand, is it actually creating a positive experience for the users?

ROB: Yeah, I'd agree with that and I'd also say that it's really effective while the person has the streak going. This is what I mean when I say that it's really important to design for failure because, as soon as the person loses the streak, are they going to start again or are they going to get discouraged? I think that the streaks are really effective while you have them. It's just that that can get taken away at any moment.

SPENCER: So what's another example of designing for failure?

ROB: Another example is with GuidedTrack. As I was mentioning earlier, GuidedTrack is basically a coding language. The user is able to write code and then, at the point where they want to check their code to see if it is successful, then they click Preview. And so that's going to compile the code, it's going to run it, and the user is going to be able to see whether they are successful or not, whether they are able to predict what the outcome would be or not. There's a few types of failure states that can occur there. One type of failure state would be seeing an error code; this would signal that you wrote something that breaks the rules of the GuidedTrack language. Another failure state would be that you don't break any rules but, even so, what you wrote doesn't create the outcome that you predicted. One of the ways that we actually are designing intentionally for that point is by closing the feedback loop a little bit. As I was mentioning before, you were typing code and then you clicked preview and you saw the results of your code. And then once you saw those results, you had to go back to your code and make the changes. But one of the things that we're actually working on — and depending on when this podcast is launched, it may or may not be visible to people who immediately go on to GuidedTrack — we're actually working on a way to make it so you're writing code on the left, and you're previewing your code on the right. And the result of that is, if you create code that doesn't create the outcome that you predict, you're able to really easily see what went wrong. It's just much easier to fix it then, and that just makes the user feel a little bit more capable in general.

SPENCER: One of the things I like best about that approach we're trying is the ability to edit the code and then immediately see how it fixes it — which is something I know we still have in development — but I think that basically, it's like tightening that feedback loop to the point where, not only can you see, "Oh, this is the code that was running that produced this screen and it doesn't look right to me but then I changed it and now the screen looks correct. Okay, I've just solved my problem." Rather than, let's say, a three-minute feedback loop where you'd have to make the change, recompile it, get back to the same point and you don't get that immediate sense that you've solved your problem.

ROB: Exactly. It's like when the user sees that something is wrong, they also immediately see what went wrong, and that just makes it so much easier to handle. It just makes you feel more comfortable in addressing the failure so it doesn't even really feel like as much of a failure in the first place.

SPENCER: Another thing I'll say about designing for the fact that users will fail is that you have to think carefully about the motivation of the user. You mentioned motivation earlier. Failing is often demoralizing and, if the user doesn't have sufficiently high motivation when that failure occurs, they might just leave at that point or get frustrated. One thing that I think can be useful to think about is making sure to match the motivation level of the user to the level of frustration and difficulty that they're going to have to face. As the difficulty and frustration level are rising, you've got to get that spike in motivation to get them through that hump.

ROB: Yeah, definitely. And again, this just gets at this notion that it's a low-hanging fruit for increasing retention of an app. If you don't intentionally design what happens when the user fails, then that means they're more likely to be susceptible to what you were just describing, which is that they won't have enough motivation to get back. I see this sort of thing all of the time in games. When the player fails, they might actually give the player a power-up that helps them get started again. I actually have a friend named Elliot Hedman and he does a lot of educational technology for children. He creates these learning games and the kids will try out some challenges, and they might be successful or they might be unsuccessful. If they're unsuccessful, he noticed that they would often give up pretty quickly. So what he started doing then was, he would give them tips. As soon as they reached a challenge and they messed up, he would give them a tip to help them get started again. This was useful, and it was helpful for the kids but they still sometimes fought back a little bit, and they didn't like being told necessarily what to do. He found what worked best was to just give them the tip in the form of a question. He asked the user a question that would prompt them to figure out the right answer and I think that this worked because it gave kids a little bit of autonomy. They were still figuring it out on their own but it made the challenge of figuring it out a little bit easier.

SPENCER: It also makes them feel like they figured it out. If you ask a question and someone answers it themselves, then they have more ownership over it than if you just give them a hint of what the answer is.

ROB Yeah, exactly. Totally agree.

SPENCER: What do you think of the idea of adaptive difficulty level, like you're making a game or an app and someone makes a mistake, and then you adapt it so that, next time, it's actually a little bit easier?

ROB: I think that's one way of dealing with the challenge of matching the difficulty level (of an app, or whatever you're asking the user to do) to their level of motivation and their level of skill. In games, they are very conscious of this. They think about it largely through the lens of flow theory, which is saying that you should, generally speaking, match the level of skill that the individual has, to the level of challenge that they're given, within certain ranges. That is most likely to create an engaging experience. If it's too hard, then they might give up and become frustrated. If it's too easy, then they might just get bored. A lot of games do this. Resident Evil is a zombie survival game and, if you're playing well, then they'll actually send more zombies your way and the zombies will be stronger, they'll do more damage towards you. But if you've died multiple times in an area, then the opposite occurs; they'll actually make the game a little bit easier. Some of the challenge of this is that it can make it so people feel like success is being punished, which isn't always the best move. But yeah, there's lots of ways that you can deal with difficulty. One of my favorite ways also is a general approach that I refer to as scoring-based difficulty. In Sonic the Hedgehog, each level follows up on each other and, at the end of a level, you receive a score that's broken up into categories. A high score means that you moved through the level fast, you did fancy tricks and you also collected rings along the way. There's multiple components to that score, and it tells you exactly how you can get those scores. That means that a high score signals that you played the game in a very challenging way, that's also the most fun way to play. But the thing is, getting a high score is optional. You can still complete the level in a really slow and methodical way that's very easy, and does not require skill. The game just leaves that up to the user.

SPENCER: That's nice. So basically, the more advanced player is still engaged because they're kind of adding on bells and whistles to their performance — which allows them to get closer to that place of challenge — whereas the less-skilled player is still able to complete the level and they don't feel frustrated.

ROB: Yeah, and we can see that sort of approach also in things like Grammarly, too. Grammarly is an application that spell-checks for you online. It checks your grammar as well. Every week you receive an email that gives you three scores; it tells you about your productivity (which is how many words you've written that week), your grammar mastery (which is how many errors you've made), and also your vocabulary. And what it does too (that I think is really interesting), is that it gives you a social comparison as well. So it might say you've written 20,000 words this week, that's better (or more productive) than 95% of other users. And this just lets the user choose for themselves, how much they're trying to do. But what I actually really like about that as well is that social comparison. It helps you know what's good, and it helps you know what's not good. So I might get a score that says I'm only beating 80% of other users in terms of grammar mastery. To me, that just doesn't feel good, I want to be better than that. And that social comparison gives me a benchmark to know like, "Okay, I can improve this a little." I do want to point out though, that I think an area where a lot of apps can mess up in this scoring-based approach, is by not telling the user how to improve. If I'm told that I don't have great grammar in my writing, but it doesn't give me valid tips that help me improve, then I might just start ignoring the score altogether.

SPENCER: A number that you don't feel like you have control over doesn't feel very satisfying, right? It's not very motivating. In order to have a sense of motivation, you have to feel like you're actually able to improve in that dimension.

ROB: Exactly. It really does seem like self-efficacy is one of the most important things I've seen in the behavior change space. If people don't think they're good at something, and they don't think they're able to improve, then they just won't do it. When I was in elementary school, I really didn't like playing soccer so I just wouldn't do it. I'm not much of a basketball player. If someone asked me to play pickup basketball with a bunch of their friends that are all very good at it, then I'll probably not join. But at the same time, if someone says, "Hey, let's play this game together," even if I've never played that game before in my life, I still have this general feeling that I'm good at playing games, and so I'm more likely to try it out. And we've actually seen this too, in GuidedTrack with the onboarding as well. We're trying to make a programming language that even people who have zero prior programming experience will learn and put effort into. At the same time, programming is just terrifying to people who have never programmed before. If I have to run some code from GitHub and it's telling me, "Hey, do XY and Z commands in your terminal," I've literally had that just work maybe 5% of the time. I've had to troubleshoot and ask for help 95%...the rest of the time, and I think that a lot of people have that general experience with code. So one of the biggest challenges that we've had — and one of the biggest things that we've had to realize — is that in the first 10 minutes or so of using GuidedTrack, we really need to show the user that they're capable. They need to feel self-efficacy.

SPENCER: Even if they have the sense, "Oh, yeah, I can't learn to code" or "That sounds like something that's really hard," you kind of have to give them a series of wins where they start to believe like, "Oh, this actually seems doable. I seem like the sort of person that can learn this."

ROB: Absolutely. It's that difference between upfront onboarding and continuous onboarding.

SPENCER: Can you break that down for us? What does that mean?

ROB: Yeah, sure. Generally, onboarding is successful when the user knows how to use an app in all of the ways that it can be useful for them. And a lot of apps focus just on the upfront onboarding. They'll give you a tutorial at the beginning, that maybe lasts a few minutes long; they try to minimize it as much as possible. Then they just leave it at that. In a lot of apps, you're just not going to be able to learn everything in ten minutes. Think about Excel or Google Sheets. Maybe in the first ten minutes, you're going to learn how to type words and numbers into each of the cells of a spreadsheet, but it's a professional tool. Excel is enormously capable, it has all sorts of functionalities. Eventually, over time, you'll learn how to write formulas or pivot tables, but even that's just the beginning. And even if you are somehow magically able to learn all the features in 20 minutes, you still have these failures of imagination, like you're not going to be able to conceptualize all of the ways that you can use this app. It takes time and experience. I see apps all of the time — like Notion and Airtable and all that — they'll give you a little tutorial at the beginning, and then they just leave the onboarding at that. I think that having a continuous onboarding that just goes on and on until the user eventually figures out all of the ways that the app can be useful for themselves, I think that's really important for these highly capable applications.

SPENCER: And it seems like there's especially a challenge in weaving the onboarding into just the regular usage, because most people don't want to just do an introduction that takes an hour where you're not really building anything real, you're just learning. People want to get going mostly, they want to get started. And so I see the best continuous onboarding, I imagine, is actually integrated in the app itself. So you're learning as you're actually doing the thing. Is that right?

ROB: Yeah, I would say so or I'd say that's one component of it. It goes back to what we were talking about earlier with the side-by-side preview in GuidedTrack, where it just closes the feedback loop between your making a prediction of what will work and then your seeing what happens. And then you see an error code, and from that error code, you have a general idea of how to fix it. And then we have these help documents that, if we do our job right, users know where to access them and they know how to search through it effectively, to find the answers when they need them. There's also this really big component, too, of community and just other people. An app is only so capable of recognizing the moments when a user has an opportunity to learn something. I think that the moments where a user fails, that's one of the biggest ways that we can teach a user something new. I was describing it earlier as this motivational thing but it's also an opportunity to teach people the right way to do it. Because, again, when the user fails, we want them to try again and to have a higher likelihood of success. But what I was kind of hearing you say a little bit in there too — and correct me if I'm wrong — is that the app should be able to recognize maybe from the user's behavior, that they are trying to do something. Maybe they're doing it unsuccessfully, or they're doing it in an inefficient way. And then the app can maybe jump in at that point and point the user in the right direction. I think that the app can only do so much. I think that people are what are really great as a source of continuous onboarding. I was describing earlier how I think that Notion and Airtable focus on the upfront onboarding, and then they leave the rest to the user. But that's actually not quite accurate because there are rich online communities that are oriented around helping each other learn and figuring things out together. I'm also using an app called Roam, which is another one of these apps (like GuidedTrack, like Notion, Excel) that is intensely capable in a whole bunch of different ways, many different functionalities. And you know who's best at figuring out what a user is trying to do, and giving advice on how to do it? It's an expert user. It's another person who's able to talk and give guidance, so I think the community can also be a source of continuous onboarding.

SPENCER: That's a good point. One app that I think does a good job of continuing to teach you things is Superhuman (the email client) where, as you're using the app, you'll see little hints like, "Did you know you can use this command to do that instead? So you don't have to use your mouse and click around." And so it teaches you to use the app in a more and more sophisticated way, and I think there's this challenge of how do you embed learning into an application in a way that doesn't feel annoying (like Clippy the paperclip)? But it's just sort of teaching you things around the edges so you're getting savvier the more you use it, without feeling annoyed by the fact that it's trying to teach you.

ROB: Yeah, I totally agree. I think, actually, one of my favorite ways that GuidedTrack is designing for continuous onboarding — and this happened before I even started helping with the onboarding. It just goes into what you were talking about, where it just makes the app fundamentally more learnable — is that, instead of just giving people a programming language and a bunch of documentation that helps people learn how to use it, we also have this toolbar. In the toolbar, it lays out all of the different commands that you can do in GuidedTrack. In Bret Victor's terms, it's like dumping Legos on the floor to see what's available. And the user is able to look through a list and they're able to see, "Oh, I didn't know that I was able to write a question. I'm going to click on the button that lets me write a question and fill out this form. After I've filled out that form, I'm going to click the OK button and it's going to automatically generate the code that will create that question." And you can do that with all sorts of functionality within GuidedTrack, and it just makes it more learnable as you go.

SPENCER: It seems like there are these two very distinct — but both important — user behaviors. One is, 'I want to do this very specific thing and how do I do that?' And the other is, 'I want to understand the breadth of the things that I can do. Let me just explore around or look at all the building blocks.' And so a toolbar like that seems like it's really good for the 'let me explore all the different building blocks.' But one thing that we've been working on with GuidedTrack is adding a universal search bar that's really good for that targeted 'I know exactly what I want to do. What's the most efficient way to learn how to do that thing?' And so the idea of the search bar is that it takes you right to that one thing that you're looking for.

ROB: Yeah. That actually reminds me of a project we were working on together (or that I started working on before we decided to switch my focus more towards GuidedTrack. But it still ultimately ended up being useful for GuidedTrack.) It has to do with information search behavior. I'll categorize it in a few different ways, real quick. There's focused search, which is when you know precisely what you're looking for, and you know where to find it. This is the use case of a search bar, I would say, and that can be really effective there. Then there's also exploratory search, which is, 'I don't really know where to find what I'm looking for but I know generally what I'm looking for.' And that I think has to do with the toolbar, which can be really effective for looking through things and saying, "I know that what I'm trying to do is an interaction that the end user of my program will do. So I'm going to go into the interaction section of the toolbar and search through those, and there it is." There's also exploratory browsing, which is like, 'I have no idea what I'm looking for. And I'm not really looking for anything in particular. I'm just trying to figure out the landscape.' I think the toolbar is also really effective for that. So a big part of continuous onboarding is just figuring out how users are supposed to be able to find this thing. A quick little example: a while ago, I wrote on Twitter (I use Twitter as a big way of externalizing my thoughts as they're coming and filling them out over time). I wrote a thread a while ago about continuous onboarding and I asked a guy named Friedrich Rincon who works in product at Superhuman. I asked him his thoughts about it, and he was like, "Yeah, I definitely agree that continuous onboarding is really important. One of the ways that we do this at Superhuman is, anytime we come up with a new feature to implement, we also have to come up with a way that we expect the user to discover it and expect them to learn it." Just going to what you were saying about how Superhuman does this really well.

SPENCER: Oh, nice. Yeah, I like that approach. One thing that strikes me as unusual about your approach to behavior change is that, I think you see a lot of things through the lens of the way games are designed. As you've said to me, game designers have been doing behavior change long before almost anyone else because it's so central to making almost any game. Do you want to tell us a bit about how you think about learning from games and what you think game designers have figured out that maybe took other people a lot longer to figure out?

ROB: Yeah, I think I'll start by describing an old theorem in psychology. It comes from Kurt Lewin, and he says that a person's behavior is fundamentally a function of the interaction between who they are as a person, and the constraints and information presented to them by their environment. The thing that's really incredible to me about games is that the designers are creating an entire world that the player is exercising their agency within. The game designers create every aspect of the environment, and they also design all of the rules and interactions that determine whether the player gets what they want, and determine the outcomes of any specific user behavior. So game designers — they literally just don't have a choice — they have to design for behavior change, because they want people to play the game in the most fun way that the person can. That's pretty similar in a lot of ways to the goals of behavioral scientists. We're trying to influence behavior as well, by changing some aspects of the environment, and by helping people to find the right behaviors and right environments for themselves. So a guy named Javier Velasquez (he's a colleague of mine), I guess he has the opposite approach of me. Where I would say I specialize more in behavioral science and am an analyst of game design, he's the opposite way. But he said behavioral economics sets up a choice architecture so that people are most likely to pick one specific option. Game designers aim to give users meaningful choices where all of the options are equally valuable. They just represent different play styles that might suit different types of players. So there's a balance if you want to create engagement and product design. When I'm playing games, I fundamentally look at the game in terms of how I think it's influencing my behavior, like what are the systems encouraging or discouraging? And by looking at games through the lenses of behavioral science theories that I've come across over time, I feel like I learned a lot. It's just another form of validation for ideas about behavior change.

SPENCER: And also, it's true that game designers can do tons of experiments, especially in the kind of online gaming world where the games are constantly iterated. So they're able to AV-test and say, "Well, what if the treasure chest has treasure in it two-thirds of the time versus one-third? How does that affect behavior?" So I imagine there's a lot of learnings going on in real time as game designers iterate.

ROB: Yeah, absolutely. I describe that as a change that's happened over recent years, with the advent of more online games. It used to be the case that you bought a game and that was the game that you owned for the rest of your life. But now, since every console is connected to the internet, the game designers are able to look at what player behavior is telling them, and they're able to adjust things over time. It's like this really symbiotic thing of paying attention to players and learning from them, but also influencing them in order to make sure that they're ultimately having the most fun that they can..."fun" may be the term, sometimes it's just different sorts of aesthetics. But yeah, they're designing for behavior change. [laughs]

[promo]

SPENCER: One thing I want to ask you about is gamification because, for a while, this was a buzzword. (I'm not sure if it is still anymore.) But basically, the idea is you try to take a product and make it more like a game in order to increase usage or changes of behavior within it. And the thing that's always bothered me about this idea of gamification is that, often, it seems like you're trying to just slap something from a game onto a product, like, "Oh, let's just give it a points system and then people are gonna use it more," something like that. And it seems to me like you're using the trappings of a game — these things that are symbolic of a game — but that doesn't actually mean that it achieves the end goal of what you'd be achieving in a game if you were to do that.

ROB: Yeah, there's that, but I just think that they're looking at the wrong things in games. Those gamification designers (I think that my age...right now, at the time of recording this podcast, I'm 24 turning 25 in a few months) a lot of people are just a little too old to look at games and figure out what's important to take away from them. They played games like Galaga and Pac-Man when they were cutting-edge, and then they (quote, unquote) "grew up" and stopped playing games. So they're taking game mechanics from an era before a lot of the innovation that's happened in game design occurred. On top of that, it's almost like they're parents watching their kids sit in front of a game console for hours and asking themselves, "Why is our child so addicted?" but that's not really the question that the kids are asking. The kids are asking questions like, "How can I improve my skill in this game, to be the boss?" or "How can I tell that I am progressing?" The game mechanics of points and badges and leaderboards, those are the sorts of things that you can understand from games when you're looking at them from 10,000 feet above, bird's eye view. But those aren't really the important parts of games. It's more about how do you provide feedback loops so that the user is able to dynamically respond to their own success and failure to progressively learn more and more about the rules of the game and gain mastery over it. These are the sorts of questions that I'm asking. So I agree, I think that gamification is often just this thing that people slap on. It's points, badges and leaderboards and I like to think like, "What if those points, badges and leaderboards, what if we just said, that's the foursquare genre of gamification? And if that's the foursquare genre, what would other genres look like?" I found that that exercise has opened my mind a little bit more.

SPENCER: That's a good way to think about it. But also just add that there's nothing intrinsically motivating about points. Imagine there was just some number on your desk and it would just go up when you push the button. You wouldn't just sit there pushing the button all day. However, if you believe that that number was linked to something important that you cared about, then you might be willing to push the button to make the number go up. Let's say you thought every time you pushed it, you got another dollar in your bank account, or every time you pushed it, you became healthier or something like that. I feel like, when you have these naive uses of gamification (which seems to happen a lot), it's often taking this idea (like points or whatever), and forgetting about the fact that people don't intrinsically care about them. They only care about them insofar as it taps into something else they care about. So the points have to represent something to the user, whether that's representing that they did a really good job, or they've acquired a skill that they were working on or what have you. For the leaderboard, it could be that they're comparing themselves to their friend and they feel competitive with their friend. Or it could be Grammarly, and they're thinking about, "Well, how good am I at grammar? I want to improve and so I'm benchmarking myself. What percentile am I at?" It's really those deeper social forces that cause these things to work, when they do work.

ROB: Yeah I totally agree. What I'm hearing from you is that the game mechanics should probably signal to the user some sort of progress on their goals, like a point is good (it's good to earn points), if those points mean that you are improving at something that you want to improve at.

SPENCER: The user cares about something. If you can get the points to reflect the thing they care about, then they'll care about the points as a way of measuring the thing, but nobody actually cares about points just for their own sake.

ROB: Yeah, I totally agree. I hear people all the time that will say things like, "Gamification just doesn't work on me," or "I just don't care about gamification." And I'm like, "Okay, maybe you don't care about the foursquare genre of gamification, but gamification is not this monolithic thing. It can go in many different directions." It's almost like asking the question, "Is design effective?" [laughs] It depends on how you design. But to your point as well, I think that if gamification is well-designed, then the user should care about it because it's helping them know whether they're doing well on something that they've already bought into, like on something that they already want. The game mechanic should help the user be successful at something they want.

SPENCER: Yeah, well-said. So what are some of the other genres outside of the foursquare genre of games?

ROB: Honestly, it's something that I'm developing over time. One of my goals over the course of my career is to develop many different genres of gamification by working on many different projects, and figuring out what sorts of general structures are repeatable, and then applying those general structures to more situations and realizing, "Oh, this doesn't quite work here. It works here so here's a boundary condition of this genre." One of the problems with the foursquare genre is that it's a homogenous set of tools and solutions for a heterogeneous set of problems. So the tools aren't always going to be a good fit for the problems. I'd say that there are a few things that I'm starting to pick up on already. For example, with Roguelikes, which are a genre of games where — if I had to describe the genre in very broad strokes — it's like an obstacle course and you're trying to get from the beginning of the obstacle course to the end of the obstacle course. Once you fail, you start over at the very beginning, and then it rearranges the obstacles for you. And you try again, and you try to make it further than you did before. And so it's a game genre that's fundamentally driven by failure. Failure's the default; you're always going to fail. And you're just trying to make it further and further each time. It's setting a learning goal. It means that people don't remember the failures, and they do remember the successes, and they're able to learn more and more over time. Because they rearrange the order of the obstacles each time, then that means you'll start recognizing and gaining an intuition for what strategies work for what obstacles. One area where I could see that working (and this is just totally speculative. I don't think we're going to do this in GuidedTrack but this is just something I've thought of) is, what if we wanted to create a training program for figuring out how to deal with different sorts of errors because, ultimately, we want to train an intuition so people are naturally writing correct code more often than they're writing incorrect code. We could show them a series of code snippets where there is something incorrect about that code, and they're supposed to fix that code within ten or 15 seconds. And if they are incorrect, then they start over at the beginning and they try again. It's just this random assortment of error codes and trying to debug and figure out the solution to the error codes. I think that could be a really fun way to learn. It's almost like this flashcard-like, design-like...I could see this working with learning a language and vocabulary, just like a random order of flashcards, and you're trying to make it further and further each time. That's the general direction I'm going with genres of gamification.

SPENCER: But it seems to me like there are all these different sorts of fundamental human desires that games are playing with. One of them is mastery, trying to be the best version of yourself at something. A lot of games use this idea of building skills or getting better. Some of them evoke more curiosity where you're exploring some interesting space or you're seeing what's there. And then there's others that are more about collecting, this sense that, "Oh, we want a complete set of things, we want to get every one of the cute animals," or this idea. I'm just curious about what are some of the other forces you think that games are playing with when it comes to behavior change?

ROB: Yeah, I think the mastery component is a really big part of it. I think that there's also really important social components as well. Some of that can come through competition, some of that can come through social comparison, some of that could come through teamwork in some cases. If you're playing Minecraft (which is this sandbox type game), I would liken Minecraft to a product like GuidedTrack where there's an infinite number of things that you can do with it. And the thing about games, a lot of times, the way that you learn a game isn't even through playing it. It's through some more experienced player (like your big brother or something) explaining it to you, so that's one thing. Just to get back to what you were talking about earlier, there are many different components to what sorts of things games are designing for. There's competition, there's exploration, there's mastery, there's curiosity. And what I think is fascinating about games is that you see so much variety in there. On one end of the spectrum, you might have a totally linear experience where every player goes through the same levels in the same obstacles in the same order. And so that means all you're trying to do is you're trying to get better at the game in order to be able to handle those obstacles, and also that the player's choice doesn't matter as much. But also, there are these more open-world games that are much more exploration-oriented. Those are games where user choice is everything like, "I want to go see what's over there, I want to try out this power, I want to express my individuality through the game." I think that's even something that works really well with Fortnite. Fortnite is an incredible example to me because the game itself is free. They give you an insane amount of things you can do within the game for zero cost whatsoever. The only things that you purchase, they're aesthetic. You're never going to get a better gun for competition. Everybody has access to the exact same guns, everybody has access to the exact same abilities. But what you can do is you can purchase (with your own money), costumes and dance moves and stuff. And then you're placed into this multiplayer setting where you're interacting with other people. And because Fortnite has allowed you to purchase these things that are aesthetic, that allows you to express your individuality through the game to other people, which I think is also this very fundamental desire that a lot of people have.

SPENCER: Yeah, absolutely. I think that that sense that we want to present ourselves a certain way to other people is one of the most powerful forces influencing human behavior.

ROB: Can you elaborate?

SPENCER: Well, there are theories such as the theory discussed in "The Elephant in the Brain," the book by Robin Hanson, about this idea that a significant amount of human behavior is about signaling things to other people, and that many behaviors that we don't usually think of as signaling information to others, still are. So taking the example of buying a car, people have this sense that the car you drive says a lot about you. And insofar as they believe that, then suddenly, they're gonna want to pick their car really carefully, not just in terms of what miles to the gallon does it get, but what kind of information am I transmitting to the people around me based on the car that I drive. Another really interesting example is in advertising, where one theory of advertising says that, by showing a Corona with a person just being really laid back on the beach, they're trying to tell you that laid back people drink Corona. But the more sophisticated kind of analysis of that says maybe that's not really what they're trying to do so much. Maybe even more than that, what they're trying to do is to tell you that other people will think of you as a laid back person if you drink Corona because, by advertising Corona with someone laid back on the beach, your expectation will be that other people will perceive Corona drinkers as laid back. And so you drink Corona because you expect other people to view you that way. It just seems like this is pervasive in a lot of human behavior. Obviously, there are many forces that drive human behavior but this is one of the big ones.

ROB: Yeah, one related idea that I've seen is in this paper called "Choice Architecture 2.0." What it was trying to do was figure out why it is the case that we see so many varying results when we change the defaults of a situation. Like why is it that, in some countries, having organ donation be the default, why is that good in some countries at increasing the frequency of donation, and counterproductive in other countries? And one of the big takeaways there is that people are interpreting the information that a choice architect is sending them. So if I'm trying to increase the frequency of people taking an STD test, and then I make it the default (like everybody has to take this test), then that means if you, Spencer, are taking this test because it's required to take the test, then that means you're not signaling to anybody that you think you might have it or not. So a lot of people might not get tested because they don't want people to think that they have a reason to worry. And then on top of that, it can just signal all sorts of things, like with defaults in organ donation. In a country like (I think it was) Germany where they did this study (I don't recall exactly but it might have been Germany, it might have been Denmark), organ donation was not the default earlier, and then they made this change, and then suddenly, people fought back. And one of the reasons they fought back is because suddenly donating organs does not signal that you are a virtuous person; it just signals that you're going with what you're supposed to do.

SPENCER: That's really interesting. And so sometimes you can have these kinds of surprising effects. You think, "I'm going to make organ donation the default," and normally, we'd expect that to increase organ donation a lot. But it might actually have unintended consequences where now, people are up in arms about this. I think it's really important to consider these kinds of second-order, more complex effects from the standard behavior change narratives.

ROB: Yeah, I'd absolutely agree.

[promo]

SPENCER: For the last topic, I wanted to ask you about your thoughts on how we develop expert intuition. What is expert intuition, and then how do we get it for ourselves?

ROB: Yeah, expert intuition is when people are going to consistently make the right decisions. [laughs] That's really the best way that I can think to describe it. And they make those decisions very quickly, and they're able to assess a situation very quickly and thoughtlessly.

SPENCER: Thoughtlessly, as in without having to think about it, not as in doing it sloppily. So it's like a very fast, correct reaction to a situation, right?

ROB: Yeah, exactly. One of my favorite models of thinking about intuition is called naturalistic decision making, and it comes from a researcher named Gary Klein (or largely centers around that researcher, I guess I should say). What it's essentially saying is that intuition is an expression of the experience that people build up and the patterns that they recognize. It just allows them to rapidly size up situations, make rapid decisions. They don't need to compare options all too much to know what the right way to go is. I'm gonna read a little quote from an article about naturalistic decision making. It says, "Researchers identify experts as having rich repertoires of patterns, being able to make fine discriminations that may be invisible to novices, having sophisticated mental models of how things work, and having resilience to adapt to complex and dynamic situations." There's a lot of ways that you can develop this sort of intuition and I think, really the best thing to do is to gain a varied set of experiences, and also reflect in order to make sure that you're synthesizing patterns and prototypes and thinking about how they apply to new situations. Expert intuition starts off with you making thoughtful and deliberate decisions. Over time, the things that used to take a lot of thought should just come very quickly.

SPENCER: Yeah, let's just use, as an example, maybe something like chess. I think this is the classic example, where a chess master can look at a chess board and almost instantly decide what move to make and beat a novice. They don't really need to think about it because their intuition is so strong, but then they can play even better by adding their thinking on top of the intuition. The way that I think about this is, when you're learning to play chess (I'm not good at chess, but just using it as an example), you start to learn pattern recognition for certain types of patterns like, "That is a good move. That's a bad move." And you start to get an intuitive feel for what moves work well and badly. And part of why that comes about is because you're trying a lot of things, and then you're seeing what worked and what failed. And so it's the trial and error in a wide range of situations with a tight feedback loop that starts to build up this intuition over time. And you have to do many, many, many repetitions to build that skill. But you can make it go even faster by giving yourself little exercises to practice where you imagine practicing the endgame hundreds of times in different scenarios. Or you realize that you're sort of weak in some area of chess and so what you do is you start constructing scenarios that are about that particular domain where you're weak, and you practice on those a whole bunch to strengthen that. And so going beyond just the trial and error, you have to start using your analytic mind to dissect your own play, discover your weaknesses, and then figure out how to get new training data in those subdomains of your weaknesses to get lots of practice there. And then finally, this element of bringing in the theory as well because there's chess theory. By learning those theories, what they allow you to do is analyze the moves you're making, which is too slow a lot of times, but in practice, you can take the time to reflect and say, "Well, was what I was doing in line with the theory or not?" And then that can help you hone your intuition even more. I guess those are the pieces that I think about, but I'm curious to hear more of how you think about this topic.

ROB: Yeah, I'd agree. And I think one of the big things that I'm hearing from you, is that you can develop intuition naturally over time, just through sheer repetition, as long as you have high-quality, immediate feedback and sufficient opportunity to practice and it's consistent. That's like what Daniel Kahneman would say. That was actually the results of a paper that Daniel Kahneman and Gary Klein worked on together, I think in the 80s. But also what I'm hearing you say, too, is that you can put deliberate effort into improving your intuition as well. I think that a lot of people focus really, really hard on improving their system two thinking, like they're very deliberate, analytical, 'I'm sitting down and thinking this thing through' thinking. But for me, one of the big goals that I have, because so many of my decisions (and really just anybody's decisions) are going to be based on intuition, I want to use my system two thinking to improve my intuitive thinking and to make it so it's more generally accurate. And I think that the reflection side of things can be really helpful. I think that that's also one of the other things you were saying about comparing what you did to what the theory would say you should do. That's another really fantastic way to improve your intuition. One of the big ways that I think I've developed my intuition over time is I read so broadly in behavioral science theory that, at the very least, if I see a situation, and it doesn't conform to that theory, I'm able to say, "Oh, something feels off about this. Let me sit down and just analyze this for a second."

SPENCER: I think that interplay between theory and the intuitive is really interesting. It seems like, often, when we're learning something at the beginning, it really helps to learn the theory first and then, when you're actually trying to do it (usually very sloppily, at the beginning), you're thinking about the theory, it's guiding you and you're doing it badly but at least you know the general direction to be pushing. And then, as you get better and better and better, it gets intuitive and you don't even need to think about the theory anymore. It's sort of in the background. And then eventually, as you get even more advanced, you'll actually start violating the theory but you'll be right, like you'll violate it at times when it's actually good to violate. The example that comes to mind is musicians. I think that some musicians might find some benefit early on from learning some music theory. But once they're experts, they stop thinking about music theory (I think, mostly) and then they even violate music theory and it's totally fine because, basically at that point, their intuition goes beyond the theory.

ROB: Yeah, I totally agree with that. And at this point, I've worked with three different products that either use a mood journal, or were considering using a mood journal. I've worked on five different onboarding experiences, I've worked on retention for more. Three products I've worked with use data visualization as a motivator. And one of the things that's great about that is that each app that I've worked with, they have their own situations, they have their own constraints. So the same theory is not going to work across all of them. And what that gains me is, that means I'm able to understand the boundary conditions of the theory a little bit better. I don't want to have exact repetitions over and over again because then I'm not learning anything new. I want to expand my intuition, and that's where I think this varied practice comes into play.

SPENCER: Right, like if you want to learn to hit a baseball really well, you can't have a pitching machine that throws exactly the same pitch every time. You actually want there to be random variation in the spin of the ball and where it's thrown and its speed. That actually will make you a much better hitter. But also to your point, when you're applying a theory in the real world, very often, there are constraints that intersect with that theory. And so to really become a master of the theory, you need to know not just how to use the theory in the abstract, but in this particular case, I have these set of constraints, so how do I adapt the theory to that situation? Because each of those apps that you're working on, or each of those onboarding processes, as you said, they have a different set of constraints that's influencing what they can and can't do, and also different goals for what they're trying to help the user achieve.

ROB: Yeah, and I want to point out some of the ways that people can use varied experiences in order to develop their intuition. One way is that you can do pattern recognition. So "Uplift is a lot like Mind Ease in X,Y & Z ways." Just connecting two different dots together and that can be really helpful. Another thing that you can do to develop your intuition is to understand the ways that your two situations are different and find the contrasting elements of it. For example, in Uplift, the mood journal component of it is one very small factor of the app, and there are other behaviors that we want to drive people towards. In Mind Ease, for the mood journal (or an equivalent to the mood journal), you essentially answer a couple of questions before you do an activity and after you do an activity, in order to see how your anxiety has changed. So they serve different roles. And also with Mind Ease, there's much less in general that we're asking users to do. In Uplift, you might go through an hour-long lesson, whereas in Mind Ease, the whole point is you can get in and get out within ten minutes, having reduced your anxiety a little bit. There are going to be some different constraints there that are going to change the ways that you can apply that. So far, we are finding patterns and also recognizing differences. And then the third way that I'll point to is building up a prototype in your head. This is what I think of as going up and down the ladder of abstraction. If I just come up with the idea to add in a side-by-side preview to GuidedTrack, I have zero theory behind it, I'm just saying, "Oh, I think this would be a general user experience improvement," then what I might do is I might think to myself, what's the more abstract version of this? Okay, the more abstract version of this is to make as clear as possible the relationship between a user's actions and the outcomes by closing the time gap between doing some action and seeing the outcome. And the thing that I like about coming up with a prototype is that you're essentially taking the content out of the context of the situation and so I can fill in the context with new content at any point. So if I am working on the failure state for an app, and I figure out some approach to intentionally design for the failure state there, then I might think, "Okay, what are some broader questions that I could ask myself across any situation?" not just the app where I came up with that initial thought about failure states. And then now I can apply that to all sorts of different situations and see, does this work here? Does this not work here? And I can adjust my understanding of the prototype, and its boundary conditions, as well, to that. All those sorts of strategies — with pattern recognition, finding contrasts, and also building up prototypes for yourself — are all things that you can do deliberately by just noticing the situations when you should do it. Like if I noticed that two apps are similar to each other, then I should just pause and think to myself for a sec, how are they similar and why are they similar? Then once I've done that, then that means it's a little bit more likely to enter my intuition. I think that intuition, what's really important with it is to just trust it to a degree, but also make sure you're verifying it, because the more that you verify your intuition, the more that you'll be able to adjust your intuition and improve the quality of it.

SPENCER: Essentially creating that feedback loop for yourself.

ROB: Exactly.

SPENCER: Rob, thanks for coming on. This was really fun.

ROB: Thank you for having me, I really enjoy any opportunity to talk about all the theories in my head. I think that it's all interconnected and related in important ways, many of which I'm not even recognizing at this point. Something I was just thinking about earlier today is that intuition plays a role in the behavior design that I'm trying to do. Like if we're working on an app like GuidedTrack, I'm not going to be able to design a habit of checking it every day. But what I do want to do is design intuition for how to generally write the correct code. So this thinking that I'm doing towards myself about how to improve my own intuition, I'm also trying to figure out ways to apply that to app design so, that way, I can intentionally help users develop intuition. So thank you for having me on the podcast. I really enjoyed it, and I really enjoyed your takes on this, too. Just being on this and talking about this with you, it's helping me clarify my thoughts and understand different perspectives that I might not have initially had.

SPENCER: Awesome. And where can people learn more about your work?

ROB: People can find me on Twitter at @RobertHaisfield, it's just my first name, last name. I would say the best way to keep up with my thoughts related to any of the sorts of things that we were talking about today is my website, which is robhaisfield.com. And if you want to have a starting point that I've intentionally designed, then you can go to robhaisfield.com/about, and really just click around. Any link that you think might be interesting — that has an interesting title or whatever — just click on it. It's meant to take you down rabbit holes and help you figure out what interesting thoughts that I have in there that are correct for your situation so I really like plugging that website. One more place is my YouTube channel. My YouTube channel's a little bit less related in general to behavioral science though I'm hoping to start adding in more content related to behavioral science, but for any of the listeners to this podcast who found me through Roam Research, you can find a lot of good content there.

SPENCER: Awesome. Thanks, Rob. Have a good night.

ROB: Thank you. You, too.

[outro]