UX research in a pandemic

In December 2019 I joined IRIS Software Group , working on the IRIS Elements project. IRIS is the largest European privately owned software company. Its main products are the Accountancy Suite, for the accountancy professionals. The IRIS Elements project is to move the traditional IRIS desktop products into the cloud. It is a new development to employ UX Design and Research into their methods of production.

IRIS Software Group had already recruited several customers who were interested in working with them on producing the new SAAS product. Once l started, it became evident that the current company CMS system was too heavyweight for the Development Team’s need. Seeking to find a lighter, more integrated product, I chose Hubspot as the CMS of choice. As it is easy to add notes and is linked with Outlook and other third-party software. Hubspot has become the backbone for the team’s research with the ability to store all the notes and conversations, plus the ability to segment the community of customers. All the Product Managers and UX professionals have access to it, therefore they can find a customer who meets their needs for testing or further research easily and quickly. They can see all the interactions with that customer without having to spend time checking with others in the Production Team.

Once the foundations for the research were in place, it was time to go and meet the development community members. Face to face meetings were arranged and riding my motorbike  through a British winter was fun. 

All interviews were recorded and transcribed, I used Dovetail App to store the interviews and to Tag them for the different points of interest. It was a relatively new software and developing fast. It can easily gather similar topics and evidence together. These are easy to playback to the stakeholders in the form of blog style reports. Also, deep links can be used in Presentations, direct to the quotes and evidence.

Through January and early February, Covid-19 was increasingly being mentioned in the news. I was still visiting customers and getting some great insights into Accountancy, IRIS and the software that is used. In mid February, the company held its annual customer conference, IRIS World, in Birmingham. I was coming south from a customer north of Leek and nearly got stuck in the snow. Such are the challenges of field research. At IRIS World we launched the test version of AML (Anti Money Laundering) with great fanfare and invited customers to join in with the user testing and research. From the conference we added over 200 more people to the Customer Research Community. This is where Hubspot really came into its own. 

Enter the Pandemic

We started to run weekly surveys about all aspects of Accountancy and business in general. Understanding how accountants run their practices how they on-board clients, chase them to get information back to complete a set of accounts and charge them. Also we discussed, what accountants, thought about the future of accountancy, especially with the Government bringing in “Making Tax Digital” and the automation of services and software packages.

By early March it became evident that Covid-19 was not going away and was going to change our lives for several months. We thought that by the summer, it would be over. How naive we were. I informed our research community that we would be moving the customer visits online and started the now familiar process of talking to people over Video. We, as a company, used Microsoft Teams as the main system. 

In the beginning, it was hard to get people to accept a video meeting, but the clincher in the early days was that we wanted to show the customers some new screens or development which we could do online. At the same time Dovetail brought in their enhanced video future that included auto-transcription. This was a game changer and removed the two days wait to transcribe an interview, it was completed in a matter of minutes. It is about 95% accurate and does require some editing and aligning with the speakers. The overall process of listening through the interviews, sometimes at 1.75x speed and tagging can take several hours. But the results are so powerful. 

Now with ease, we can bring the voice of the customer into meetings and presentations. No longer is it a miss remembered quote that was taken on a hot day in a back office, but the words and meaning, direct from the customer.

This spurred me and the team to do more. The customers saw that what they had to say was reaching those that made the decisions. They were also increasingly coming up with ideas and suggestions. They now feel part of the process and the production team. The customers have “ownership” of the products in development. A number have said that they enjoy the process, it is a bit different to their normal work.

Online Workshops

Once we had settled into a pattern of interviews and surveys, I started to see what else we could do online. Mass Zoom meetings were happening and the endless round of quizzes with friends became the norm. In the meantime, a colleague had been working with Mural Boards. A collaborative working space. Used it across a few the dev teams he is working with, but would it work with a traditional UX Workshop set up, with activities such as Sailboat and Card Sorting? 

We set up a workshop, once we could get everyone’s busy diaries to a line and got valuable insights, that were played back to the team. The challenge with all multiple person video calls is that they need to be carefully managed with the chance for all participants to have their say. Chairing these sessions and giving everyone a voice is key. Making sure everyone has their video on and without distracting background.

We as a team have run several multi-person sessions, I have found that inviting several people from the same company on the calls, helps them to talk more freely. However, using video software, the interactions between the participants, do not talk as freely as if they were in the same room. 

Was the time and effort put into the larger sessions worth the output? Could we have got the output from a few smaller sessions, where comments from one were played back to another? My answer to this, is that multiple smaller sessions are better for online than a bigger session. The reason for this is that we had to go round the virtual room, one by one and the normal discussion between participants was limited. 

To get more out of these online sessions, then there is a need for breakout groups with their own moderator and then as with a normal workshop get them to play it back to the bigger group. This is possible but takes time, resources and planning to ensure the participants understand the technical requirements to link between the meetings.

In a world where there is a limit to time and resources, sometimes a simple one to one session can cover the same topics in a similar time frame and yield a great set of insights, as long as the key findings are played back across the group of interviewees.

User Testing

User testing has been very successful. The team started in early April with simple demo sessions of early designs, as mentioned above and worked with a majority of the over 300 members of the user community, over desktop video conferencing software. As the UX Designers worked up the designs to clickable prototypes, these again were demoed and then tested. This built up our experience and expertise of demoing online. All these sessions were recorded and played back to the development teams. As the code was completed, these sections were tested with some new and some original customers. 

The pre-launch product was tested from end to end as part of the Go to Market strategy. We also used a Standard Customer Satisfaction Score survey at the end of each session. All this was straight forwards to complete and the findings delivered during this pandemic. This is probably because the participants over the months had got used to being videoed and using online tools and programmes. They were relaxed with their environment and concentrated on the task in hand and not put off by the tech. This is something I had found before the pandemic when people who were used to dealing with others face to face and not being filmed, became self-conscious whilst performing a task, especially if they started to struggle. This is less evident with the recent User Tests.

Conclusion

In conclusion, how has the pandemic affected research. It has had an overall detrimental effect as the nuances and subtleties are lost when interviewing someone online. Those casual conversations on the way to a meeting room or more precisely on the way out when the interviewee is relaxed, and you can ask an in-depth question. Or the chance to meet someone else from the organisation you are visiting, or opportunity to see a process in operation these are lost when the meeting comes to an end and the “Leave” button is pressed. 

Workshops are much better in person and there can be some great discussions, especially when conversations are over-heard between the different participants and someone else adds their views. This is so hard online. Also, 2 hours is maximum online, not the all-day workshops with those “casual” conversations over lunch or at break time. Plus, where is the great coffee and cakes.

In depth user testing has been better for the reasons outlined above. For one, I cannot wait to get back out on the road to talk to the community and bring them together in some great workshops and hear how they have been using the IRIS Elements.