- I was the newly hired test lead for the product and I was still learning my way around the company, the product and the people.
- I knew it was late in the product development process and it was unlikely any feedback the users had would make it into the upcoming product release.
- I knew the UAT session was being hosted for political reasons; the goal being to "get" the users sign off on the product release even though they'd had little input and little time with the release.
- I knew the users were frustrated so I expected to hear a litany of complaints that I wasn't feeling empowered to resolve.
So this wasn't a meeting I was particularly looking forward to. Turns out, I was wrong about how the meeting would go.
Turns out, the users knew I wasn't behind the politics of the purposely-late scheduled UAT session and they didn't want to delve into that political topic at that time. What they did want was a chance to talk with me directly. They wanted to talk about the product. They wanted to talk about what worked, what didn't work and how product flaws were being worked around. In several cases, the users had come up with clever workarounds to sidestep product flaws. I came to realize they were a resourceful group of people.
1. Listen to the users
Shortly into the meeting, the project manager gathered release sign-off signatures and then left the meeting. The room changed. The users opened up.
I listened. I heard stories about how the users used the product not just how from a step-by-step point of view but from a purpose point of view. Their stories pulled me into their shoes, their point of view. The product came to life (as trite as the expression may be) as the product became a tool that helped people – well, at least it helped them most of the time and they had plenty of ideas about how the product could be repaired and improved to become better. I realized I hadn't thought about the product from their perspective before. I didn't have their perspective before.
I came to understand the product was a tool to the users, that users used the product as a means of getting something accomplished and not because software is innately cool and interesting. They had work to get done. The product was a tool, just a tool to get their "real" work done. I think about that now sometimes, even years later. It seems to me that those of us who work with software and find it intriguing might forget that a software product is often just a tool to accomplish other work. So each time a user interface gets overhauled or a feature gets a dramatic facelift, the users have to learn the tool all again – which is not how users want to spend their time with software. I think about that experience when Microsoft Office products get upgraded. I don't really want to learn Word again; I just want to use it. After all, Word is just a tool to me.
2. Share yourself and your testing approaches
As the meeting continued, I found the users wanted to hear from me too. I was still fairly new at the company and they hadn't gotten to know me personally or much about me professionally. Seems that all the meetings we'd had before had been formal meetings. This was the first time the designated team of UAT users for the product had me alone. It made me realize I hadn't reached out to them on an individual level. I'd been busy being the test lead and hadn't taken the time to meet the users in a less formal setting. I stopped focusing on my role, opened up and was just me.
I shared a bit about who I was, what my background was and how I viewed testing. I shared the existing test approach the test team and development group had been sharing, the approach that had been in practice since before I had arrived on the team. The users had never heard what the test process or approach was. They found it interesting to hear how we approached the next release from a test perspective. I felt a loyalty to the software team but I also gained loyalty to the users who were no longer an anonymous collection of people. I felt the heat of being between two groups of people who had become disconnected. I better understood some of the challenges that had been going on. It didn't seem like the teams had been listening to each other.
Back then, I felt like I had to have the answers to the problems and I recall how uncomfortable it felt that I could see problems without having solutions. Now, years later, I'm more comfortable with uncomfortable situations in life and certainly more used to not having a solution instantly but instead having to arrive at a solution with help from people on the team. When you're truly listening and sharing, it doesn't always mean you have to solve the problem on the spot. In fact, jumping to a solution might mean you're not listening.
3. Collect an informal risk assessment
With a better understanding of how the product was being used, the critical uses of the product became more obvious. Yes, there were opinions and ideas to collect! I saw the product risks through the eyes of the users instead of seeing product risks from a software development perspective. For the users, if certain features weren't working correctly that was a risk. From a development perspective, we're sometimes more tuned into code that poses the most risk to fail, based on code complexity or failures during automated unit testing. These can be two different ways to look at risk. Hearing what was important to the users shifted my sense of product risks which in turn, brought changes to our test approach.
4. Ask for feedback
As our meeting wound down, the users had become people to me. I could see they were counting on me and on the team to deliver a product they could rely on. I've been through many UAT sessions since then – sessions hosted for users for an internal company product as well as products sold to and used by consumers. Once a relationship is built between people, it takes time and effort to stay in tune with each other. I've found feedback I gather from people when they're in a group is sometimes different than feedback I gain when I meet with one person so I intentionally rotate different settings to reconnect with users. And I've found that reminding people every now and then that I'm open to feedback leaves people more likely to share.
So, seize the opportunity. Sometimes when a UAT session is being hosted, you might walk into a computer lab or meeting room knowing that multiple users want to talk to you about the thorniest issues in an application. It can be hard to face a group of regular users knowing that a long standing annoying product defect remains unresolved. But our users are important to connect with. Sometimes we can forget one of the most important aspects of product development; we build a product for users.
Karen N. Johnson is an independent software test consultant. She views software testing as an intellectual challenge and believes in the context-driven school of testing. She has 14 years' experience in software testing and software test management. She is a frequent speaker at software testing conferences and is an active participant in several software testing workshops. She's published articles in software testing publications as well. For more information about Karen, visit http://www.karennjohnson.com.