Annotating Designs for User Interaction

Contents

Why Annotate UI Controls?

Of course, how a user interacts with your web page is fundamental to good UX design. Most of us think about this from the perspective of a sighted user who can use a mouse. Some users, such as users with dexterity limitations, can’t use a mouse and need to move between controls in a serial top-to-bottom, left-to-right order using the tab key and arrow keys on a keyboard. Blind and low-vision users navigate a page similarly but may also need cues and assistance to understand special controls and how to operate them.

Anything that a user can interact with (e.g. link, button, etc.) should be considered a UI control and requires special attention. The good news is that these interactions can also be consolidated into a single page of annotations, but does require a separate sheet to let developers know what the expected behavior of each control should be.

A Working Example

Wireframe with annotations described in accompanying text.

To describe how to annotate wireframes, mockups, and prototypes to identify accessible UI controls, let’s work directly with the screenshot above. If you’d prefer to print it out or see it in a larger format, you can download it here. In this screenshot, we have a wireframe for a eCommerce client that includes annotations that reveal focus order, bypass blocks, and name, role, and value. To the right of the actual wireframe, we have the accompanying notes. Together, the notes and the wireframe form a powerful combination. We even added annotations for a live region. We’ll discuss how all of these concepts are incorporated into one rendering above.

Identify All Items Capable of Receiving Focus (or Groups of Such Items)

The purpose of this wireframe is to identify the UI controls. This includes anything that the user can interact with– such as a button, form control, or link. In creating this rendering, each such UI control needs to be identified and here it is accomplished by giving each UI element a bold magenta box around them. In some cases, however, an entire group of such items has a single box instead. In that case, the annotations include a small card of the UI controls within that grouping and explains how each smaller UI control works. For instance, the main menu is identified as a single <ul> unordered list with each menu choice identified as an <a> link. When the user clicks on a link, a submenu appears, which is itself an <ul> unordered list with the same structure. This submenu is represented in the notes with a card detailing its structure.

Annotating Focus Order

Each of the magenta boxes around each UI control has a number in a small circle in the corner of each box. You can see this at work in the screenshot in the section below, which provides a close-up view of some of these numbered circles in boxes.

This number serves a dual purpose. The most important purpose is that it identifies the focus order of the element. Making sure you follow this convention will ensure you meet 2.4.3.1 Focus Order if those elements follows a logical top-to-bottom and left-to-right pattern within each region that you use them. The second reason why each magenta box needs a number is to serve as a cue to the notes sheet on the right on the wireframe. This method makes it easy to provide detailed notes on the expected behavior of a UI control for developers.

Annotating Bypass Blocks

screenshot described in accompanying text

One item that stands out in the wireframe above is the link that reads, “Skip to Main Content” in the upper left. It has the number “1” next to it, indicating that it should receive focus first when the user tabs into the page. It also has an arrow that bypasses the navigation and header section of the page and goes to the first item on the page. The notes indicate that this is an <a> link that “jumps to the hidden H1 header at the top of main content”. This is a great example of how to meet 2.4.1.1 Bypass Blocks. While a specific link may not always be required (e.g. if you properly identify the <main> region and make good use of heading elements), it’s always a good practice, particularly if you want your designs to be usable by keyboard-only users.

Annotating Name, Role, and Value

The notes accompanying the wireframe provide a lot of the critical information to ensure that developers create the types of controls (and predicted behavior) needed for the website.

annotations section of wireframe. See accompanying text for description.

In the screenshot of the notes section above, we have just the notes section from the wireframe. Here, you may notice that, where possible, the UX designer provided the name of the UI control. This is the “accessible name” or the exact words that should be read by the screen reader when encountering the UI element. Next, we have the role, which is usually marked as a HTML control, such as <a>, <ul>, etc. While identifying the exact HTML element to use may not seem like a task that UX engineers typically perform, it’s important because developers should not substitute non-HTML controls for standard HTML controls when possible– so if the user can interact with a control like a standard HTML control, this should be identified in designs. When does that happen? Well, if the UI control is one of the following controls:

<a>

Standard hyperlink

<button>

Clickable button. Also supports a “disabled” property which grays out button and makes it incapable of being clicked or receiving focus.

<input>

Creates interactive elements inside a form. The common types of <input> controls are:

  • <input type=”button”> creates an interactive button (like <button>) within a form.
  • <input type=”checkbox”> creates a standard two-state (checked or not checked) checkbox. Unlike radio buttons, checkboxes are not mutually-exclusive within their group.
  • <input type=”file”> lets users upload files to the server.
  • <input type=”image”> lets you use graphical images as submit buttons on forms. They naturally accept the alt attribute to describe the purpose of the image.
  • <input type=”password”> is a standard password field where the user’s input is not shown.
  • <input type=”radio”> is a standard radio two-state (checked or not checked> radio button. Unlike checkboxes, radio buttons are mutually-exclusive within their group.
  • <input type=”reset”> creates a form reset button that clears entries in a form.
  • <input type=”submit”> creates a form reset button that clears entries in a form.

HTML 5 also added several additional types, but the support and default appearance for these new types varies considerably across browsers.

  • <input type=”color”> displays a color picker that the user can use to choose a color. This generally works across all modern browsers.
  • <input type=”datetime”>, <input type=”datetime”> , <input type=”datetime-local”>, <input type=”month”>, <input type=”time”>, <input type=”week”> all create calendar controls that make it easier to choose a date and time. This doesn’t work on most browsers.
  • <input type=”email”> looks like a standard text field but can validate that the contents are an email address. This also doesn’t work across most browsers.
  • <input type=”number”> lets you enter numbers but also give you better control over the acceptable values the user can enter. This generally works well across browsers.
  • <input type=”range”> is similar but creates a basic slider control.
  • <input type=”search”> is a standard text field dedicated to search input fields. It behaves slightly differently in modern browsers and lets the user instantly clear results.
  • <input type=”tel”> is an input text field dedicated to telephone numbers. It is not supported across most browsers.
  • <input type=”URL”> is an input text field dedicated to URLs. Validation is supported across most browsers.

<map>

Creates a client-side clickable image map. The <map> element defines the entire map– the <area> elements (used inside of the map) identify the different clickable regions within that map.

<menu> and <menuitem>

These elements create popup menus. Unfortunately, they are not supported on most browsers.

<select>

Creates a standard dropdown selection list within a form.

<textarea>

Creates a text area box for submitting longer text entries than <input type=”text”>

<ul>

Creates an unordered list. While this is not a UI control, it is helpful for organizing parallel content, such as menus

Remember that when you are considering whether you can use one of the elements above, you are focusing on the functionality provided by the control and not it’s appearance. For instance, you and your developers may not like the appearance of the default radio buttons, checkboxes, or sliders across some browsers — but this should never mean that your developers should create their own controls using <div> elements and CSS. Instead, you and your developers should always try to use standard HTML control and manipulate them only if you need to in order to meet your design requirements.

Level of Detail May Vary

Your comfort in specifying whether developers should use a specific HTML element (as shown above) or just a generic description (e.g. “menu”, “checkbox”, etc) will depend on your organization and your level of comfort with learning about basic HTML controls. In general, you should try to at least learn about the behavior of standard controls as specifying the type of control will both provide the greatest level of accessibility when possible and ensure that developers create UI controls that behave in the manner that you require.

Annotating Custom Controls

What if the functionality of your control does not follow the standard model? For instance, in the following screenshot, we have a wireframe that shows how a user can filter available color choices from the category of products that they are interested in.

Screenshot described in accompanying text

In the screenshot, we have four checkboxes with color choices (black, red, brown, and blue) but only black and red are checked. These colors are arranged under the heading “color”, which is partially checked (aka “mixed”)– indicating that some (but not all) of its subordinate choices are selected. All of the checkboxes are styled a bit like round radio buttons to meet the client’s aesthetics. How should this portion of the wireframe be annotated?

We marked the entire set of options as one annotation. In our notes, we identified the name as “Color”. This makes sense because the choices under it are the colors chosen by the user. We also identified two roles for the two types of controls– a custom checkbox and standard checkbox. We identified values that the checkboxes can take (checked, not checked, and mixed). Finally, our note at the end pulls the explanation together– the “Color” checkbox needs to be a custom control and support a mixed state. The other checkboxes, however, should be standard two-state checkboxes.

Most modern designs use a fairly limited set of standard non-HTML controls. Some common example include the following.

Accordions and Disclosures

These controls hide or reveal content based on whether the trigger (e.g. a question in an FAQ) is selected. In this case, a screen reader user needs to know whether the control is expanded (e.g. reveals the answer) or if it is not expanded (e.g. collapsed and hiding the answer).

Values: expanded, collapsed.

ComboBox

A ComboBox is a hybrid of a standard HTML text control (usually a <input type=”text”> field) and a dropdown box or other control that reveals realtime results based on characters typed into the search field. Using a ComboBox provides a quick-and-easy way to perform searches on web site. The real value to the user of a ComboBox are the realtime results– and providing this information programmatically is done through implementing a live region (see 4.1.3.1 Status Message Events). The important point for UX designers, however, is properly identifying such controls on wireframes as a ComboBox.

Modal Dialogs

Modal dialogs are typically popups that darken or obscure the page and make it impossible to interact with the main page unless the requirements of the dialog are met. While this may be visually apparent to the user and the developer, it is still important to identify its state.

Values: modal

A good and fairly comprehensive description of non-HTML controls can be found in the Design Patterns and Widgets section of the W3C’s WAI-ARIA Authoring Practices 1.1.

Annotating Live Regions

In our example above, you’ll notice that item 5 references a “live region” and this concept deserves special attention. Anytime you have a content on a page that changes without an abrupt change (e.g. opening a new page or dialog box), you should think that it may constitute a status message and requires special attention. To qualify as a status message, an on-screen event needs to:

  • provide information to the user on the success or results of an action, on the waiting state of an application, on the progress of a process, or on the existence of errors AND
  • not be delivered as part of a change of context (such as opening a new page or dialog box).

Some examples of status messages from the W3C include:

  1. After a user presses a Search button, the page content is updated to include the results of the search, which are displayed in a section below the Search button. The change to content also includes the message “5 results returned” near the top of this new content. This text is given an appropriate role for a status message. A screen reader announces, “Five results returned”.
  2. After a user presses an Add to Shopping Cart button, a section of content near the Shopping Cart icon adds the text “5 items”. A screen reader announces “Five items” or “Shopping cart, five items”.
  3. After a user enters incorrect text in an input called Postal Code, a message appears above the input reading “Invalid entry”. The screen reader announces, “Invalid entry” or “Postal code, invalid entry”.
  4. After a user activates a process, an icon symbolizing ‘busy’ appears on the screen. The screen reader announces “application busy”.
  5. An application displays a progress bar to indicate the status of an upgrade. The element is assigned a suitable role. The screen reader provides intermittent announcements of the progress.
  6. After a user submits a form, text is added to the existing form which reads, “Your form was successfully submitted.” The screen reader announces the same message.
  7. After a user unsuccessfully fills in a form because some of the data is in the incorrect format, text is added to the existing form which reads “5 errors on page”. The screen reader announces the same message.
  8. After a user puts a photo in an album in an online photo app, a snackbar displays the message “Saved in ‘Wedding’ album”, which is also read by a screen reader.

As a UX designer, it helps to simply ask yourself, “if I were not looking at the screen and took an action, is there any content that I would like to know about that wasn’t a screen refresh, a dialog popup, or something else that forcibly grabbed my attention?” If so, then that is a likely candidate for being a status message and thus requiring an ARIA live region..

In the example above, the UX designer recognized that there were two areas where there were status messages. One is in the shopping cart icon shown in the screenshot below.

screenshot of shopping cart described in accompanying text

As shown in this screenshot, the shopping cart updates the number of items in the red icon as items are added or removed from the shopping cart. This happens as soon as items are added and without a page refresh or other event to break the user’s flow. Thus, the UX designer identified this as a live region for the developer.

Another example of a live region is the combobox used for the search results shown in the screenshot below.

screenshot of search tool described in accompanying text

In this example, a combobox will display matching search results as a dropbox under the input field. This a usually implemented as a hybrid control combining a regular input field with a select element. But as those search results are updated, the screen reader user needs to know the number of items in the search results. Thus, the UX designer identified that this is also a live region.

Setting Politeness

In both the shopping cart and the search tool examples, the annotations indicate that the live region should be set to a value of “polite” or “assertive”. This is a great example of the user experience and importance of UX in accessible design. Imagine the situation where an on-screen event happens– do you want the screen reader to immediately announce the event or just at the next convenient opportunity? There are four possible settings– but you’ll likely only use either assertive or polite.

off

This is the default value and nothing gets read to a screen reader. You won’t want to use this if you want anything announced to the user.

polite

The announcement is made at the next break– such as when the user is not doing anything or waiting for something to happen. In general, the W3C suggests that “polite” should be used with messages advises on (1) the success or results of an action or (2) the state of an application. For instance, successfully adding an item to a shopping cart is a good example of something that should be announced at the next convenient opportunity.

assertive

The announcement is made as soon as the action happens– interrupting the user and letting them know quickly. In general, the W3C suggests that “assertive” should be used with messages that (1) convey a suggestion or (2) a warning on the existence of an error. For instance, a form validation error is a great example of something the user should be told about as soon as possible.

rude

As the name suggests, this is only for absolutely critical updates. In reality, you should avoid using it.

These four values really affect the overall user experience for non-visual users. And developers can take this information from your annotations and directly incorporate it into their designs. For instance, setting role="status" inherits aria-live="polite". Thus, in the shopping cart example, a developer can take your wireframe and simply set:

<div role="status" id="my_live_region">
     <!-- set innerHTML for updates here -->
</div>

Then, any announcements that the developer adds to this region (using the ID value and the <DIV> element’s innerHTML property) gets politely read to the screen reader. Pretty cool, huh?

In the search combobox example, telling a developer to set politeness to “assertive” is a cue for her to set role="alert" (as that inherits aria-live="assertive") and any announcements put in such a region will be read out to the screen reader in a slightly more aggressive fashion.

Note: screen readers vary in their level of support for the variations between these level of politeness. Thus, it is possible, for instance, that messages may be read out more or less aggressively than desired. You shouldn’t be expected to know these differences but your designs can incorporate these subtleties even if not every screen reader will respect them.

Annotating Forms and Identifying Labels

When a screen reader user navigates a form, most screen readers cannot associate the form field (e.g. a text input box) with the visual label next to it (e.g. “last name”). From a UX perspective, it makes sense that the user should receive information upon reaching that form field that is similar to or identical to the information made available to visual users.

Our last example above didn’t include a sample of these forms (apart from the search bar). In the screenshot, we have a example of one of the same screenshots we used when discussing Annotating Designs for Structure.

Screenshot of wireframe described in accompanying text

(you can also download a larger version of this screenshot)

As suggested at the beginning of this section, one of the most important pieces of information that a screen reader user needs to know is the name of the form element. In form elements, this usually means the text label associated with the form element. This forms the “Name” element in the legend accompanying the wireframe. This makes it crystal clear to the developer exactly what text you want to have read out by screen reader upon hitting this form element. Because these are required fields, we made sure the name includes this information as well. You should also identify the “Role” of each element (here, a <input type="text">). The screenshot below is a magnified version of the “last name” annotations that accompany this wireframe and shows how UX designers should make this information available.

screenshot of wireframe described in accompanying text

You could (and often should) specifically include a note in the annotation to the remind the developer to use the accompanying label text (here “Last name*”) as the programmatic label for the form. By using a <label> element specifically around the on-screen text “Last name*”, a user can click on the label text and focus will immediately be placed in the right form field. In this example, this is not necessary because the label text is quite small and is a small “target” but with larger label text, using a <label> tag can improve usability for users with dexterity issues.

Specifying Input Purpose

In both of the examples above, you may notice that several form fields also specify an “input purpose”. Per Core Concept 1.3.5.1 Identify Input Purpose, specifying input purpose is important but is frequently overlooked in web design and development. To help reinforce adding input purpose, UX Designers should look up the input purpose identified at Section 7 of the WCAG 2.1 specification. These are:

  • name – Full name
  • honorific-prefix – Prefix or title (e.g., “Mr.”, “Ms.”, “Dr.”, “Mlle“)
  • given-name – Given name (in some Western cultures, also known as the first name)
  • additional-name – Additional names (in some Western cultures, also known as middle names, forenames other than the first name)
  • family-name – Family name (in some Western cultures, also known as the last name or surname)
  • honorific-suffix – Suffix (e.g., “Jr.”, “B.Sc.”, “MBASW”, “II”)
  • nickname – Nickname, screen name, handle: a typically short name used instead of the full name
  • organization-title – Job title (e.g., “Software Engineer”, “Senior Vice President”, “Deputy Managing Director”)
  • username – A username
  • new-password – A new password (e.g., when creating an account or changing a password)
  • current-password – The current password for the account identified by the username field (e.g., when logging in)
  • organization – Company name corresponding to the person, address, or contact information in the other fields associated with this field
  • street-address – Street address (multiple lines, newlines preserved)
  • address-line1 – Street address (one line per field, line 1)
  • address-line2 – Street address (one line per field, line 2)
  • address-line3 – Street address (one line per field, line 3)
  • address-level4 – The most fine-grained administrative level, in addresses with four administrative levels
  • address-level3 – The third administrative level, in addresses with three or more administrative levels
  • address-level2 – The second administrative level, in addresses with two or more administrative levels; in the countries with two administrative levels, this would typically be the city, town, village, or other locality within which the relevant street address is found
  • address-level1 – The broadest administrative level in the address, i.e., the province within which the locality is found; for example, in the US, this would be the state; in Switzerland it would be the canton; in the UK, the post town
  • country – Country code
  • country-name – Country name
  • postal-code – Postal code, post code, ZIP code, CEDEX code (if CEDEX, append “CEDEX”, and the dissement, if relevant, to the address-level2 field)
  • cc-name – Full name as given on the payment instrument
  • cc-given-name – Given name as given on the payment instrument (in some Western cultures, also known as the first name)
  • cc-additional-name – Additional names given on the payment instrument (in some Western cultures, also known as middle names, forenames other than the first name)
  • cc-family-name – Family name given on the payment instrument (in some Western cultures, also known as the last name or surname)
  • cc-number – Code identifying the payment instrument (e.g., the credit card number)
  • cc-exp – Expiration date of the payment instrument
  • cc-exp-month – Month component of the expiration date of the payment instrument
  • cc-exp-year – Year component of the expiration date of the payment instrument
  • cc-csc – Security code for the payment instrument (also known as the card security code (CSC), card validation code (CVC), card verification value (CVV), signature panel code (SPC), credit card ID (CCID), etc)
  • cc-type – Type of payment instrument
  • transaction-currency – The currency that the user would prefer the transaction to use
  • transaction-amount – The amount that the user would like for the transaction (e.g., when entering a bid or sale price)
  • language – Preferred language
  • bday – Birthday
  • bday-day – Day component of birthday
  • bday-month – Month component of birthday
  • bday-year – Year component of birthday
  • sex – Gender identity (e.g., Female, Fa’afafine)
  • url – Home page or other Web page corresponding to the company, person, address, or contact information in the other fields associated with this field
  • photo – Photograph, icon, or other image corresponding to the company, person, address, or contact information in the other fields associated with this field
  • tel – Full telephone number, including country code
  • tel-country-code – Country code component of the telephone number
  • tel-national – Telephone number without the county code component, with a country-internal prefix applied if applicable
  • tel-area-code – Area code component of the telephone number, with a country-internal prefix applied if applicable
  • tel-local – Telephone number without the country code and area code components
  • tel-local-prefix – First part of the component of the telephone number that follows the area code, when that component is split into two components
  • tel-local-suffix – Second part of the component of the telephone number that follows the area code, when that component is split into two components
  • tel-extension – Telephone number internal extension code
  • email – E-mail address
  • impp – URL representing an instant messaging protocol endpoint (for example, “aim:goim?screenname=example” or “xmpp:fred@example.net“)