Always on research

This guide documents the process of running and contributing to ‘Always on’, our continuous research programme.

Recruitment

We offer regular interview sessions with users to continuously refine our understanding of our core users, their needs and pain points.

Participants are eligible to take part if they:

work on government services (inc. contractors)
use the GOV.UK Design System

You can recruit participants in several ways, including:

open calls through agreed communications channels (e.g. email mailing lists, the community catch-up call, Slack)
the community pages on the Design System website
direct invitations (e.g. email, Slack DMs) to people or services who are known users

We use an Expression of Interest (EOI) form to collect contact details and screen participants to check they meet eligibility criteria.

Once you’ve reviewed the completed EOI form and confirmed their eligibility, you’ll need to book a session with the interviewee and get their informed consent.

Scheduling sessions

Offer up to four interview sessions (30-45 mins each) every cycle. Try to offer sessions on different weekdays to accommodate different working patterns, both for participants and observers. Sessions should be spaced at least 15 mins apart to allow screen breaks, post-session discussion and account for any overrunning.

Scheduling and planning the sessions is mainly the responsibility of the User Researcher (UR), although other team members can help. This includes sharing adverts, inviting participants, getting consent, booking UR sessions in the team calendar and arranging for team members to support or observe.

You can schedule sessions manually or use the Bookings app in Teams to let participants choose and book interview slots directly (the latter works best if you stick to regular days and times).

We use Participant Kit to get consent with a standard consent form.

As well as giving overall agreement to take part, participants can set a number of optional consent preferences including for audio, video and/or screen recordings.

Interviews can only take place once consent has been given.

Roles

Interviews are generally done remotely using MS Teams, although you can arrange in-person sessions if convenient.

Interviews are set up to allow the team (and other approved observers) to observe remotely and discreetly. There are several different roles required to deliver interviews:

Interviewer

welcomes interviewee
checks consent has been given
starts recording and transcription
uses the discussion guide to ask the interviewee questions and moderate any tasks
debriefs the participant

Screen sharer

acts as a nexus between the interview room and the observer room
shares the interview room screen with any observers in the observer room
does a soundcheck with the interviewer to test the technical set up before the session
takes notes, if required

Observer(s) and note taker(s)

watches the interview
takes notes, if required

How to prepare for interviews

Before a round of interviews, the UR should:

ask for volunteers to fill support roles or observe, using the UR support spreadsheet
share calendar invites, links to the discussion guide and note-taking templates with volunteers
review and/or adapt the discussion guide

Before the session, ensure all volunteer roles are filled. This is the joint responsibility of the user researcher and the delivery manager assigned to the work. If there are insufficient volunteers, you could consider using a rota. You can ask for support by posting a message on the team Slack channel.

If running via MS Teams, you will need to set up two Teams meetings: one as an interview room and one as an observer room.

Invite the interviewee and the screen sharer to the interview room meeting. Invite the screen sharer, plus any note takers and observers to the observer room meeting. The screen sharer then:

attends both meetings simultaneously (one in Teams, one in a separate web browser)
puts themself on mute and camera off in the first meeting
shares their screen of the first meeting with the second meeting

This set up allows the team to observe with minimal intrusion to the participant being interviewed. There are more detailed instructions here about how to set this up. The interviewer should do a practice run or sound check with the screen sharer before the session.

Running the session

Run the interview using this discussion guide, which includes core questions about how and why users use the GOV.UK Design System, challenges and pain points.

On occasion, it may be useful to dedicate parts of a session, or the whole session to particular topics or issues the team is currently interested in (e.g. prototyping). These topics may change or rotate as required and should be agreed in advance between the user researcher and the team.

It’s helpful to have a quiet, private room to interview in and headphones to ensure privacy and limit background noise.

After the session

Any video or audio recordings from the session that include person identifiable data should be transferred to the Google user research data shared drive as soon as possible. Anonymised transcripts and notes can be saved in the DS Team’s Data folder.

Summarise the results in the participant matrix as soon as possible after the interview, ideally within a few days. The matrix has been designed to support framework analysis but can be used for other approaches.

Analysis, reporting and action planning

Periodically, you should thematically analyse results. Involve the team in analysis to support quality checking, interpretation of findings and encourage buy-in. These approaches to analysis have previously worked well with the team:

Video sessions – watch pre-prepared video ‘highlights’ reels, review accompanying transcripts and discuss key observations in small groups (within-participants analysis).
Themed analysis sessions – review findings recorded in the participant matrix for a particular theme or topic. Identify, group and map themes and subthemes on a Mural board (between-participants analysis).

Playback findings three to four times per year. Work with the product manager and the team to prioritise actions and use these to shape the product roadmap where appropriate.

Also share a summary of outcomes with participants and the wider community through appropriate channels such as via email, other social channels and community catch-up calls. You can find examples in the Reports folder.

Add a record of the research and any outputs suitable for wider circulation to the UR research registry within 3 months of the completion of the research.