UAT – me, how do I do that?

Previous installments

IT investment for the smaller business
Why the Smaller business holds all the cards
Where has alll the money gone
Requirements gathering the first big mistake
Contract negotiation and on we go

Well here we are, though dragging slightly due to the unexpected holdup when everyone was so inconsiderate as to take a long break in the Summer and get all relaxed. It seems like a very short time despite the months that have passed, but we are now nearing a milestone when the vendor delivers the systems to us and according to both the vendor’s project manager and the IT manager, we are expected to do something called UAT.  Of course it’s not written in law anywhere, but to our project manager, it may as well be because he is following the conversation two sentences behind and trying not to look too foolish.  The vendor is still driving the agenda alone and the IT manager is still sulking, but gaining in confidence as the project approaches his territory.

The problem with COTS purchases and traditional testing methods is just that.

Well, it’s just that the methods were designed for a world where teams of engineers spent months writing code and then began to stabilise it and introduce it slowly into the business environment until it was a stable release and a good match for the tightly defined requirements.  At least that was the theory, there were usually problems about interpretation of the requirements and about how the outcome was actually achieved, but at least the bits did fit together and they followed a well established pattern. Vmodel was and still is the standard.

That was a simple theory, you had to create business requirements, the analysts translated this into designs, the developers translated that into systems, then it was refined to make it usable and fill the gaps in interpretation of business requirements.

Logically therefore, you tested the business requirements with stakeholders and got it agreed, then the designs with business analysts to make sure it was properly interpreted, then the system to make sure it fitted the design. Along the way, there had to be technical testing to make sure it could run on the clients environment correctly and then a bit of testing to make sure it worked well enough for testers to commit weeks or months of their time.
Finally you arrived at the point where you had a working system that had no major bugs and appeared to meet the needs of users and it was time to let them loose on it.  That was UAT.

It usually flushed out little bugs that had been missed, it also flushed out poorly interpreted requirements that just didn’t work in a live environment. Additionally, it served as a way to win over key users and pre-empt any likely resistance to the process change aspect.

For a simple explanation, that has taken a bit of effort and probably lost me some readers, but it’s not a simple business.

So how does our project manager apply this to a piece of kit arriving on a CD with a few minor customisations and integration endpoints?
Maybe not the million dollar question, at least not every time, but an expensive one often.

Our friend has no detailed requirements to work to, because the initial ones were not adapted, but instead, the sales guy sold an existing package and said it will meet the requirements you described. If our friend hires a tester, he will need to go to the vendor’s and ask them to help him list exactly what this system is expected to do for the client in order that he can test it.
Were he to do that, alarming as it may sound to many readers, at least a start would be made on understanding what has been agreed, what is expected by whom and what the likelihood is of all these stakeholders being in the same book let alone page.
Now I am not a sadist, despite what you may be thinking and I have no intention of bringing up the business case and the business goals at this point, our poor friend is in enough trouble already.

The IT manager is now beginning to ask awkward questions and making apparently unreasonable demands about some sort of complicated and time consuming testing before he lets us install our system on “his” environment.  Is he out to get us? He never really was supportive anyhow, because he felt he should be in charge.

So where have we got so far?

We identified the need for change and made a business case for investment of the businesses capital based on some modest assumptions sound process change and cost reduction due to efficiency and we got the go ahead to do it.  We have not tested these assumptions against the new system being proposed and we have no idea whether any of the goals will really be met other than a stirring presentation and “good feel” from the vendor’s sales rep.

We found a supplier that we felt we could work with and agreed a contract, we know of course that that contract was tipped 90% in favour of the supplier, but we hoping that all will end well.

We are now approaching eminent delivery just a little behind schedule, but nothing to worry about and we are very confused about the communications coming from the vendor’s team. We are expected to complete “UAT” in three weeks and then sign off acceptance and pay the final instalment for services and we will begin to pay the support and other fees immediately. This has a very final and serious ring to it.

We are gathering up a few people and arranging a room where they can sit and “play with” this system for a few weeks. Then hopefully all is well and we are done.

We have met our milestones so far, the budget is at or below forecast and the CEO is very impressed with us, let’ s hope these IT guys don’t spoil the party.

Next:

Whose fault is it anyhow ?