Default behaviour sticks (And so do examples)

Introduction

We spend huge amounts of time sweating the details of our products. We want to remove all the friction we can from using them and want to make sure we never leave our users confused. To get this right, we do a bunch of things: we use simple language, we make extensive use of context-sensitive help and where it’s needed, we nudge users with illustrative examples.

Recently we bumped into something that made us rethink our use of examples.

Background

Paid Canary customers also receive a private Canarytokens server as part of their subscription. This is a private, managed version of the service publicly available at www.canarytokens.org. They get to mint an unlimited number of Canarytokens, get access to some tokens before they are released to the world and are able to trivially customise the service.

Canarytokens typically (but not always) rely on a DNS zone that’s unique per-customer. When a customer signs up, we create a DNS zone for them and usually that’s sufficient for their needs.

However, one of the advanced customisations for customers is the ability to create their own DNS zone with a name they pick. They’d typically do this to make the underlying hostname obviously tied to their company, so their custom DNS zone might look like assets.their-company.com. This requires users to pick a zone name, and as a UX guide we autogenerated a name for them. We happily used someprefix.their-company.com²¹, as an example:

When we built the UI for this feature, the inclusion of the someprefix example was to make it easier for customer to configure DNS on their end, given that DNS can be tricky to get right. It wasn’t the intention that customer only use a zone called someprefix, we simply picked it because we needed something to use in our examples. If the example zone name becomes an implicit standard then the risk is that it lets attackers more confidently guess about Canarytokens based on discovered hostnames.

Recently, one of our engineers was working in this area of code and wondered how many customers simply followed the example shown and picked someprefix.their-company.com as their custom domain of choice, as opposed to choosing another. His intuition was spot on. Among customers using this feature, ~40% used the example we provided:

We use the custom domain to make Canarytokens less identifiable. If 40% of them use the same custom name, then the disguise is not as effective.

Learning

To be sure, this is not an individual customer problem. Looking at other configuration options present in our UI, the pattern is clear. When given an example, a significant number of users default to using that same example in their customisation. The behaviour is consistent across customers and configurations. This surprised us! ²

It’s important to realise this isn’t a customer-side issue; they shouldn’t have to consider the impact of every configuration option we choose to put in front of them. They don’t have the full context and knowledge, and expecting them to be experts in the nitty gritty of Canarytoken discoverability makes no sense. Frankly it’s a reason enterprise software is often so terrible; tons of options you barely understand or know about, and are configured according to tutorials/examples rather than understanding. This is a lesson for us internally about how we guide customers through using Canarytokens, and more generally through Canary.

Fortunately this particular case has a simple enough fix. Going forward, we will show multiple examples of prefixes. A user looking to add a custom domain will see a variety of example zones when they visit the page, and the examples will cycle each time they open the configuration page. We want to convey that they have options in choosing the name, and we show them a variety of sample options. Our hope is that this will prompt customers to pick their own names, and if they do rely on our examples then those are now spread over a large list of examples.

Conclusion

The outsized impact what seemed like a very minor placeholder choice made years ago helped us reevaluate how we select the examples we show customers. It’s a strong reminder about sweating every small detail in the UI; we were surprised at the oversized effect of our examples.

Going forward this particular placeholder has been altered and is already live for customers. We will report back with a count with the new active examples in the future.

1. some-prefix is used in this post to protect our poorly chosen actual-prefix 🙄 ↩︎
2. It’s likely this result is known in UI design circles, but was new to us. Please send references to other work!
   ↩︎