Benefits Realization Management or BRM is a hot topic. It's a new discipline, that is getting increasingly popular and provokes lively discussions. It holds a great promise of helping organizations monitor and validate ROI over time to close the feedback loop and stop throwing good money after bad one. But do we have to wait until after the project is delivered and handed over to operations to see if it was a good investment? This article will provide practical advice and a proven User Research process you can use today to improve your benefit realization.
Flip Side Holds The Real KeyGarbage in - Garbage out (or GIGO) was a popular aphorism back in the 80th, at the dawn of the computing era. And this is still just as true today. If you get a project initiated 'on a thin ice', without fully discovering your key user motivations, habits and processes they use to do business today - you're setting yourself up for a disappointment.
You may even avoid the scope creep and have your project delivered on time and on budget, but, more often than not, it will not do so well with business users and your benefits realization numbers will come out negative.
Let's look at a quick example.
Our client invested 2 million dollars into an Enterprise Content Management system, looking to move their content off their shared drives and enforce sound security and retention management policies. They also began to use it to power their internal web sites.
Shortly after implementation, Content Contributors began to complain that, even after attending training, they were still having trouble updating content on the site. Some outright refused to use the new system and some were making arrangements to get transferred out of the role of Content Contributor (true story).
The Project Manager overseeing implementation brought us in to help figure out the problem and how to fix it.
After initial round of interviews and watching Content Contributors work, it became apparent that even though the BAs on the project team had thoroughly identified all needed metadata values and the workflows for content to follow, it was just too tedious to populate them and submit content revisions using the interface that was provided.
Despite the 'smart' validation features and good looking UIs, many steps were required to update a single page element, and users had to save intermediate files on their local file systems. Contributors frequently picked the wrong file, causing site pages to break or display outdated information, and privileged documents were left forgotten on local file systems.
After user goals and task flows have been identified, it didn’t take long to come up with the new interface. In just two weeks the new UI for Content Contributors has been designed, prototyped and validated with users and the development team.
Even though the client collected their requirements and did a good job describing the steps user were following, they forgot to consider how these steps fitted together. The screens themselves were functional and looked good individually, but they just didn’t work well together.
The Hero BA TrapThe reason why this organization forgot to consider the 'big picture' is the same reason why many other organizations make these one-off mistakes in their business requirements, and this is the reason why lots of companies will see spotty benefits realization at best.
At ECM Solutions we like to call that "The Hero BA Trap". You see, a lot of organizations are still relying on a Hero BA - a single person with exceptional human skills - to conduct all of their requirements work. If they make a mistake, there may be million dollar consequences.
Still many CIOs are thinking about the Business Analysis as an innate skill when good BAs are born and not raised.
Another reason this happens is because many organizations don't have repeatable processes in place, to ensure that problem definition statements are complete and that The Real Problems are identified, and they are not chasing down symptoms.
No wonder the PMI is reporting that only 20% of organizations are enjoying high Benefits Realization Rate!
Most companies have their processes in place for managing and verifying requirements, but collecting requirements still relies on Hero BAs and the big 'download' ritual, where they just sit with stakeholders, asking them what they would like to see, and then write it down.
Requirements Validation activities then involve asking their key stakeholders if requirements document looks complete and no unnecessary features present, culminating at getting requirements document signed off.
What's missing is the reliable process for requirements elicitation, making sure we actually get the full picture and complete description of the problem areas and the key processes people are using to do business.
And this is why so many 'new requirements' are still discovered late into the project and then managed as scope creep. Again reported by PMI, we see that out of all unsuccessful projects, every second one fails due to this lack of requirements processes, checks and balances.
Think about it. We invest in Agile to have a cross-functional, collaborative team deliver our projects, and that works really well... but only the delivery part, that has to do with 'on time and on budget' thing...
Then one person - a key stakeholder or a BA - misses an important detail in business requirements - and all of this delivery, fast efficient, on time and on budget - no longer matters and we end up in the ditch.
Even when you put a team of BAs in place or have a team work directly with stakeholders, the chance of missing important requirements or getting them wrong is still very high... unless you have the process in place.
So here's what I'm talking about:
Three Steps For Improving Benefits RealizationLet me share the three steps, that a lot of clients have found very effective in burning these common requirements management risks. These steps are based on 30 years of User Research, and User Research (not the Design) may easily be the main reason why UX is working so well for Apple, Google and other amazing 'Design' companies we all like to follow.
So here are the steps:
- Get all of your users involved. Don't limit your requirements elicitation with just your project sponsor or maybe a few key stakeholders. They may be too close to the wheel and may overlook things or never mention stuff they may be considering obvious.
Yes, it may not be practical to conduct thousands of interviews, but there are high performing research methods that make it easy to get all of these people involved.
The rule of thumb is to keep involving additional business users into your study until you stop discovering new details, and each additional user just keeps on repeating the things you have already heard.
- Ask about problems first. Think about solutions later. Don't start your interviews by asking users what would they like you build for them. Start with their Single Biggest Challenge. Solutions will come later, when the root causes and the key processes get discovered.
And don't be fooled with traditional surveys. No matter how you present your assumptions - "On a scale of one to ten how would you rate your accounting system?" and "How long on average do the pages of your accenting system take to load?" neither of these two questions will help you figure out that the root cause is bad data coming from manufacturing, not the accounting system itself.
- Test your ideas early, don't wait till solutions are fully developed. Test on your prototypes, paper, quick wireframes and so on. Test often and use quantitative methods to base tests on facts. Watching users use the system without teaching them how to use it is the true test and could be very revealing.
So don't be running to your usability consultant after the system is live and users are refusing to use it or your design is crumbling under load. That's backwards thinking. Come to us before you initiate a project. Get some solid User Research done and be clear about the real problems your users are facing.
Don't start projects on thin ice and dump it all on PM's laps to deal with. Yes, it feels good to be doing something to fix the problem, but don't fall into that old trap as activity does not replace productivity.
Does your organization use a repeatable process for project initiation? Did you find it effective for discovering critical details and focusing on the right problem? How do you ensure that your choice of technology will work well? Do you replace assumptions with facts before you start or counting on your development process to clarify them? Love to see your thoughts in the comments section below.