The DNA of CHAOS. Why software projects, peace negotiations and football games all come up short on occasion and the true value proposition of agile

Why Business analysts, Product Owners, negotiators, political reformers, scientists and many others are doomed to repeat the same mistakes until they drown in in their failures.

  1. Most people in a work situation, and indeed in most situations, live in a consciousness that is deliberately falsified to reflect (a) what they believe is expected of them by peers, and (b) what the asker of a given question wants to hear. This is a conditioned response and nobody’s fault.  Terms like “Group Think” attempt to explain it, but none really explain the phenomenon well and the long version is just beyond our scope in this piece. Market researchers are trained to try and avoid it, Politicians and advertisers learn to harness it. If you have the role of finding out what  a business wants to do so you can get the software designed to support, it, you will be faced with the same  aggravated problem. Asking and listening simply won’t get the job done.
  2. For reasons I don’t claim to understand fully, most people have a favourite solution in their minds for most problems they are aware of. Expensive  housing – they may say; “Stop immigrants coming in”.  The call centres is too expensive and eating our profits- they may say; “ Close the call centre and let them use Twitter”. Always there is an obvious political, or financial gain for the responder from this solution. If it is a senior executive, or politician, there will be cleverly argued points and tables of figures that make her pet solution look like a no-brainer.
  3. Michael D Cohen  coined the phrase “Garbage Can Theory” to describe this phenomenon which permeates politics, business, the playground and just about every area of human life crippling every endeavour to improve our lot century after century. If you are trying to get your proposal through programme and investment boards, or trying to agree a solution closer the coal-face you will drown unless you understand this phenomenon and take steps to work with it.
  4. Many researchers and writers have agreed strongly that the human brain makes virtually all decisions in a matter of time between milliseconds and 1 to 3 seconds, but then spend enormous amounts of time and effort researching in order to prove their decision was correct. Most people are entirely unaware that they do so.  This of course is not news to Scientific researchers who often publish the theory before beginning on the research.  In the academic world where we have to start somewhere this can be a useful approach. Product managers and entrepreneurs also start with the theory and are difficult to budge even when wrong, yet these leaders are necessary to get something started at all.
  5. People are conditioned to follow a strong leader and will convince themselves of their support for any theory put forward by this leader. This is largely an extension of  point one, but worth being aware of all the same when venturing into any group to arrive at a shared view.

These Five forces combined thwart just about every effort to form a good business strategy with technology in mind, agree a solution and still be in agreement when the solution is completed to specification even a few months later, let alone after a year or more. Waterfall and other methods of managing the software delivery process all rely on a simplistic view that if you gather enough requirements from users and stakeholders, you can then haggle over priority and build the killer solution to solve everyone’s problems. In reality, there is rarely any agreement, there is even less concern over business outcomes and near- total focus on the many garbage can solutions. Nothing this project could deliver is ever likely to meet the satisfaction of more than a small proportion of stakeholders, the others are always going to be loud in their dissatisfaction and only by sheer luck is their ever going to be serious business benefit.  This project, however well the techies perform will be added to the list of “failures” and the cause will be placed at the CIO’s door. Failure to focus on goals Developers and technical architects are driven by developing exciting new toys. That is their package in the garbage can. Techies ultimately want and are entitled to, clear definitions of what they must build.  They then build and test precisely what you asked for and give it to you working  very well. Few people would try to argue with that, let alone succeed.  The problem is that what you asked for does not always address the problem you needed solved and in fairness the developer or indeed the CIO is not to blame. There is one technique that all accomplished negotiators agree on whether brokering peace in a war zone or negotiating a business contract. YOU MUST KEEP THEM FOCUSED ON GOALS NOT SOLUTIONS! If you don’t, the garbage can will take over.
One cause  of the problem is not using the effective solutions we already have,
The need to focus on goals rather than solutions is not  new to the software industry by the way.  MSP, Prince2 and TOGAF are frameworks I am personally very familiar with, all of which, when used correctly, focus on outcomes required, the solutions that will best deliver those outcomes and the details of each solution, in that order. BABOK and the ISEB framework for business analysis also follow these same principals and are very clear about it. The problem is that few people make any attempt to use their training effectively, the training is rushed and focused only on answering multiple choice questions to gain a certificate and of course the majority of people involved in the business have no training at all. This same scenario, by the way, applies equally to Scrum. Scrum solves some of the problem Scrum attempts to alleviate the problem in a different way and with a similar level of success. Scrum is invented and owned by the technical community as a direct response to these issues plus the difficulty of communicating the complexity of technology with business people. The Scrum team wants only to be given an unambiguous requirement so they can do what they studied for, build great software. They place the onus on the business to appoint, or work with a Product Owner who will be seconded to their team and take all responsibility for the definition of the Product/System.   Communication issues are solved by very short Sprints of just two weeks typically and using prototypes and working software to describe things to the product owner in place of documents. Initially this arrangement solves the problems for the Scrum team, there is no ambiguity about what they must build, they do as much work as the Product Owner wants and stop when he is satisfied. There is no room for dissatisfaction with the IT team. Likewise for the Product Owner, if he is unsure, he can build something , test it on real world scenarios and then make adjustments until it delivers the right outcomes or scrap it altogether.
Inspect and adapt solves problems with lack of knowledge, but also masks others Scrum tries to tackle the strategic questions by replacing strategic decisions and documents with Inspect and adapt. For online products and for certain internal systems, this works extremely well because you can build the minimum, get it working and in use, learn from success or failure and add a little more, remove some, or start again. The risks are lower, failures are smaller and less expensive and the problems with groupthink, daydreaming and garbage cans are alleviated by an early sobering splash of cold reality that comes with release of a Minimal Viable Product. What is often wrong with this approach and fatally dangerous, is the lack of a strategic layer to the thinking process. Inspect and adapt is a close cousin of the OODA loop as defined by Colonel John Boyd. The principal is very simple that in a dynamic environment, he who has the most clear view of the situation always wins given a reasonable ability to make decisions.  Pilots flying inferior jets were seen to consistently win dogfights  because they had a better view from the cockpit and could make better decisions faster.  Developing OODA  (Observe, Orientate, Decide, Act ) and teaching it to pilots made significant improvements over giving them a detailed instruction, or a faster Jet because as the military have always recognised, in the field, the operative has superior knowledge in the heat of battle and must have the right level of autonomy and confidence to decide and act. No military  has ever cut loose a person, or team, or unmanned drone to inspect and adapt and just stay out there. There has to be a clear goal, a Definition of Done, Principals that guide it in the field and a strategy for completing the mission. Then Inspect and Adapt comes into play and in the same way that the human brain itself learns,  the actions  in the field can feed all the way back to sometimes cause a change in strategy or even a revision in Principals, but  actors in the field DO NOT  GO AWOL. Principals and strategy come first and Inspect and Adapt is a response to exceptions , it is not the  plan.

The real reason why Agile/Scrum is winning friends in the world of software and gaining attention beyond these confines

I say Agile/Scrum because when most people speak of agile they are really referring  to Scrum. However Scrum has encompassed much from other Agile implementations and definitely does not hold the sole franchise on Agile methods. The first problem Scrum had to address was the great chasm between Business leaders and Technology leaders.  Various versions of the same solution see the business seconding someone to the Scrum team, or colocation of business and technical people working on the same project. This is a perennial problem sometimes referred to as silos and secondment or colocation of cross discipline teams is a useful technique that helps, but doesn’t alone solve the problems. In the case of Scrum it forces the business into making decisions for better or for worse and living or dying by them. That helps techies to get on with building stuff, but it doesn’t, of itself, help the business much with getting the strategy right. I.E they produce more stuff, but don’t necessarily solve more problems. The next problem Scrum had to address is helping the business in dealing with the unknown. The speed of change is much greater in the past few decades driven by the fast pace of disruptive technology. Organisations are faced with problems they can’t analyse easily and therefore can’t solve. They also find that when they know the problem, they don’t have an obvious solution. This is the situation Scrum was designed for in its entirety. A Scrum team can make a best guess and very quickly get a potential solution in front of customers, Inspect and Adapt and return quickly with an improved effort until the right solution is reached by trial and error and then it can be built upon until it reaches its optimum scale. The biggest problem addressed by Scrum and that one that is causing all the excitement is the almost  total  lack of leadership throughout modern businesses (the past 10 years). Few would lament the passing of the great Command and Control organisations, but with them went whatever semblance of leadership we had in organisations. In the Flat organisation leadership in name is often cited, but leadership in deed is sometimes frowned upon or even punished. In the 80s and 90s business leaders bullishly wrote books about leadership and motivation. These were charismatic people with an instinct for what made others tick.   Kenneth Blanchard, Peter  Drucker, Stephen R Covey and their like gave us the ideas and the passion to change things and build things. We somehow lost this passion and insight in the transition.  Scrum is a beacon in the dark, because it values people and their interactions over trails of evidence. It values achievement over promises and it values self-improvement  and job satisfaction. Here and there Scrum teams are springing up that capture attention by their high productivity and ingenuity and win the envy of their colleagues for their enthusiasm and Joie de vivre.  People who see a good Scrum team in action want to be part of it or learn how to make that happen in their business.  That is the real reason why the buzz of excitement continues and why Scrum experts are being invited into other areas of business.

The true DNA of an agile project (exploding the myths)

Ignore the new vocabulary and you’ll find nothing new in the concepts.

If you have heard me pour scorn over some of the claims made for agile, you may be surprised to know that I’m an agile practitioner with some considerable experience and not at all adverse to the approach.  That said, I always repeat the words of my agile mentor Keith Richards (no not him silly) when I asked the obvious silly question. He said ” It’s horses for courses.  When you turn up for training we assume a certain level of education, intelligence and experience”.

OK so what is the big noise about. Well the underlying promise of agile emerged back in the seventies soon after the first Standish report when some of the industry leaders cooked up an alternative to the waterfall way of wasting money.
The theory and indeed the practice is totally valid when followed with a level of honesty.  You tell me how much you can spend, or how long you have and I’ll guarantee you a working system.
Note the difference from waterfall where you have two options: fix both budget and time and have a failure nine times out of ten, or fix  just one and reduce the odds by 2/3.(MSF).

Agile delivers on the promise when billed honestly and executed  honestly.
Here’s the metaphor.  Say you meet with your co-directors and agree that right now the thing that will  accelerate you to your goals is an very large extra table  in the boardroom with a tiny door and spiral staircase and you have only £n  to spend, or n months to deliver it. I will discuss your needs at length, then I will come back with a proposal that looks like this.

The minimum required to make this work is a top and thee legs. You don’t need four to make a table work, though I agree it would be nice.  It can’t be bought and even the top has to be constructed on site.
The legs are standard things and fundamental so I will order three of these and have them delivered.
We will start making prototypes using scrap wood  for legs and sticking planks together until we have a table that is just stable enough and functional enough to meet your needs.

This we are confident we can achieve within your constraints, if we have extra time or budget, which I expect we will have, we can add a fourth leg, or smooth it over, or add a coat of varnish. You can decide when the time comes, which is more important. That’s it. No miracles, no free lunches, just less pontification, more action and a product that is fit for purpose, if not always entirely pretty.

 When you wouldn’t use agile.

You would never use it to build a space shuttle. That type of project requires a right first time approach.  I shouldn’t joke about a tragedy like that. There can never be a serious consequence of getting it wrong first time. You shouldn’t use it unless you are bought in fully to the three legged table, if your mind is set on four, stick to what you know. If you are buying in the team either as contractors or a service company, it is risky, because highly competent  people is a prerequisite.
If your main motivation is to be involved on a day to day basis as opposed to making a decision up front. It is poison, keep away.

Putting agile in perspective.

This article discusses an important aspect of agile that is rarely figures in the big debate, i.e. where agile fits in the great scheme of things organiation wise. It also examines breifly  the concerns that are shared with general corporate activity.

It sheds light on  why agile came about and regularly returns to the fore only to disappear again and most of all it puts the whole cult aspect of agile under the microscope arriving at some perhaps unexpected conclusions.

Conclusion

Some people are overawed by the challenge of software development to the extent that they rely entirely  on a comforting framework to lean against, while others use their big picture perspective to manipulate the marketplace selling  their training, books and consultancy services under one disguise after another, while contributing little or nothing to the profession.

In reality there are two major challenges facing all software endeavours. The first of these is the challenge of dumning down science and motivating people to ahieve some sort of useful marriage. The scope is beyind this article but siffice to say that the internet arrived at least 25 years after it could have and is still years behind where it could be while people catch on to the possibilities. The antithesis of Moore’s Law  if you like.

The second challenge facing software projects could be summed up as defining an agreed goal and delivering on it.
in meeting that challenge the professional will follow the one very simple Software Development Lifecycle (SDLC) regardless of what methodology or framework he/she initially elected. That lifecycle can be loosely described as:

  1. Identifying the organisational goal the commitment, timeframe and budget.
  2.  Identifying a technical solution that meets that requirement within agreed constraints.
  3. Delivering the technical solution within agreed quality constraints.
  4. Delivering on the corporate goals.

The key to choosing and using methodology is to understand the challenges thrown up by a specific project and address them correctly. These challenges will be predominantly driven by corporate culture, process complexity, level of innovation, technical competence and budget constraints.

Ultimately it matters little whether you begin by creating detailed plans and evaluating them technically, or start building storyboards and prototypes. Likewise it matters not a jot whether you build it in time boxes, or spend two years coding before shipping the lot to outsourced testers, both of these approaches will have to meet all four objectives before the project is completed successfully.

There are however, certain projects that simply won’t fit a particular approach. One useful example is that you wouldn’t be advised to take an innovative prototyping and experimenting approach to air traffic control systems for example, as the consequences of error in the “trial and error “ paradigm are well known and best avoided.

The reason for the re-emergence of agile as an SDLC as opposed to a methodology is that it includes items one and four of our basic lifecycle in a single project.

Outside of Agile it is very rare for items one and four to be given very much more than lip service.  Business people are generally untrained and ill-equipped to rub shoulders with the technical aspects of this work and are understandably very frightened to make commitments and take responsibility for something they don’t understand and can’t control.  IT people are rarely equipped with the business knowledge and people skills to pull it off even if they were likely to be taken seruously by the busienss. Hence the ostrich syndrome prevails more often than not.

Agile breaks this deadlock by placing empowered people with all the skills they need and sufficient budget into a single team and telling them to make decisions and get on with it.

 

Agile is to the SDLC what a project is to the enterprise

“Agile places empowered people with all the skills they need and sufficient budget into a single team and tells them to make decisions and get on with it.”

Why does this statement seem radical to some people? Surely we have sufficient grasp of management in the 21st century that we can define a task and get it done with a level of motivation and commitment within just about any corporate body.  Well it seems not.

It is well documented and understood that once an organisation grows to the point where the founder can no longer be involved in every decision, there begins a phase that either destroys it, or morphs it into a modern political and mostly dysfunctional organisation that is driven by policies, processes and politics,  with little identity and few motivated individuals willing or able to chase opportunities or to excel.
The winning strategy to survive and prosper in this corporate world is based on being out of sight and below the parapets at all times except miraculously just after a victory and of course bringing apples to the teacher. Excellence is rarely sufficient and often unnoticed.
As these organisations grow within their specialised field, they substitute predictability, buying power and economies of scale for individual efficiency and innovation and they can survive and prosper despite the negative, but predictable performance levels produced by process heavy environments lacking in initiative, or motivation.
The problems surface when you try to use that same team with the same culture to do something different.  Not only are you asking them to step out of their comfortable process, but a successful result will often bring with it an unwelcome and frightening change of culture for them and on top of this you are asking them to make decisions and use initiative.  All this in an environment which, through no fault of their own penalises initiative and chops off heads that protrude above the parapet.

This description may sound like a condemnation, but in truth it is just an acceptance of the fact that we are human and we adapt to our environment and that to achieve things we must begin with an honest, or even pessimistic appreciation of that environment and a plan to work within it.

Enter the Project

Although there are projects used in other environments such as construction where they are in fact part of operations, for the most part projects are used as frameworks for dealing with the issues described above. In fact, even when adapted to areas like construction, they still have a key role in abstracting the task at hand from the corporate cultures of client and supplier in order to progress the work at hand in a partnership or joint venture environment.
A typical project will create a governance structure modelled loosely on a company with a senior executive a PM and board that closely resemble the familiar frameworks of CEO, Chairman and board and operate in a similar way for course of the project. This structure releases them from the corporate culture and gives them the the framework and budget to get things done independently.  Just like real companies, they work well when the team are well chosen and pulling together and less so by degrees at other times. Projects without strong corporate support are up against it unless they have a very powerful and committed champion.

Why agile then?

Traditional Projects work well for large undertakings that are sufficiently in-your- face to gain and maintain interest in the boardroom right through to conclusion. Problems get solved and genuine mistakes are not allowed to become bargaining chips.

The trouble is that most projects don’t gain that kind of attention and are peripheral to busy directors and senior executives.  Board members are sometimes not ideally selected, not terribly committed, or even there purely to be kept informed. Occasionay they are hostile.  In these circumstances many projects lose their way for want of decisions, motivation, or intervention against blockers.

This is where the agile paradigm comes into its own.  Instead of creating this large and very formal structure that won’t actually gain critical mass, the key is to choose motivated, knowledgeable and competent people at the right level in the organisation, combine them with high quality technical and support teams and a skilled agile PM while completely empowering the team to make decisions and get the job done while providing access to a senior executive as and when required.

This agile approach returns the project to a small company paradigm with motivated, committed individuals operating with a shared vision, making decisions, communicating meaningfully, pulling together and getting the job done.

The software profession at its best

Surely we have taken software engineering to a point where a reasonably experienced engineer can decide whether something is achievable, give a time-frame and budget and stick reasonably close to it. Again it would seem to be a singularly difficult challenge for the profession.

Since the late fifties, the software industry has learned and developed along a predictable path that is in some ways remarkable and yet in others frustrating.  The return of the same old questions again and again is partly seen as the rejuvenation of boring old ideas for immediate financial gain and this view is not without any truth, but it is also part and parcel of how humans increase their knowledge.

Hegel described the increase of understanding in terms of: Thesis, antithesis ad hypotheses.

In effect he suggested that learning begins with theses, some of this is then rejected (antithesis) and finally, on reflection it emerges for use and further consideration (hypothesis).
Software engineering has developed at a rapid speed through borrowing from hardware engineering, construction and manufacturing, trying these concepts, improving them and trying again. Between 1950 and 2000 the average cost per line of code in use at the US Department of Defence dropped from $10 to $ .000001 as the industry matured. The industry has progressed rapidly in engineering terms, but there’s still a long way to go and especially in terms of interface with the users and investors.

Coding time estimates

In today’s software engineering environment (as opposed to a newly formed team of programmers at random company ltd) the time taken by programmers while programming is roughly as follows:

timebreakdown coding

In fact as a good rule of thumb, it is widely accepted that when a programmer tells you how long a job will take,  you need to multiply that figure by somewhere between 6 and 11 depending on his/her track record.

If this software is to be of any use it must also be documented for the purpose of maintenance and for implementation, training and user support. In a less mature team these figures would be easily multiplied by anything up to a further five.

Communication time estimates

As you build a team the time for team and project communication begins to rise exponentially as the team size grows and if change is frequent it can reach as much as 50% of all effort with alarming speed.

Propensity for change

 

 The cost of changing requirements/features grows rapidly according to how late in the cycle the change occurs.  This change is mostly driven by omissions and errors at the design phase and design or programming errors further along the cycle. Controlling these changes is down to excellent stakeholder communicat

Impact of change in a software project
Impact of change in a software project

ion and rigorous design and verification procedures.

The preceding paragraphs highlight the very topmost concerns in the simplest possible way yet it is clear even from this short commentary that providing even reasonably accurate estimations is extremely challenging and is a whole team exercise.

It is also worth noting that of the four steps defined in our underlying SDLC, the software engineering discipline only covers numbers two and three.
Responsibility for defining the business goals and their parameters as well as responsibility for implementing the system in order to capture those benefits remains an equally challenging proposition and lies entirely with the business.
In a product paradigm, the only difference is that product management must take responsibility for customer consultation and feature design.

Software Development Lifecycles (SDLC) are over prescriptive and misunderstood.

The many incarnations of SDLC will generally expand very quickly into a complex map of steps with interdependencies. With the best intentions, they attempt to tie the practitioner into a very prescriptive set of steps, but in doing so they remove the emphasis on common sense and good communications. For the sake of simplicity, when appraising a project, I tend to simplify the lifecycle to just four goals that can be carried out in any order that serves your specific project and revisited when necessary.  By taking this approach, you can apply it to Scrum, RAD, Agile or any waterfall flavour equally well.

  1. Identifying the organisational goal the commitment, timeframe and budget.
  2.  Identifying a technical solution that meets that goal within constraints
  3. Delivering the technical solution within agreed quality constraints.
  4. Delivering on the corporate goals.

In the best implementations, step one receives lip service in the “Feasibility” stage and step four is hardly ever taken into account. In the worst implementations two and three are poorly implemented with substandard testing and little documentation.

Step three regularly suffers from confusion between user and investor with the real end user rarely beimg engaged meaningfully.

It doesn’t have to be like this and it most certainly shouldn’t, but the reality is evident everywhere.

Agile variants are all created equal and simply marketing ploys

SCRUM, Agile and DSDM all tackle the business perspective and place it at the centre of the project lifecycle, they incorporate testing and UAT into the development process and they place emphasis on communication at all stages of the process. Scrum is seen to go a little further in considering the user, though the others don’t exclude it.

Regardless of which flavour you choose, you can tailor it precisely to your project’s needs and the idea that one flavour is better than another is almost to suggest that someone who can master software engineering needs to throw out his book and buy  a different one to add a little more emphasis on the customer etc.

Other concepts associated with Agile methods, such as working in teams of two and holding scrum meetings are all engineering technicques that owe no alegiances to any methodology and should be in everyone’s toolbox.

Can agile be scaled?

There is quite a bit being written recently about scaling agile. I have to be honest in saying that I have not read any of it and don’t have any plans to do so in the short term, but I have read comments from people I respect and it would seem according to their considered opinions to be predictably bandwagon in its nature i.e. a new opportunity for trainers and consultants to wrap something familiar in new clothes and sell it again.

Just as we discussed the small company culture and large company culture earlier in this article, the software project also changes in its nature when the team, or the complexity grows and it is inevitable that more layers of management have to be applied and more prescriptive processes implemented and before you know it you will have waterfall regardless of what you call it.

Large systems can and should be broken down into smaller ones and some, or even many of these can be developed using agile methods, but the overall project needs a more planned and less dynamic approach both from an organisational and an engineering perspective. There is no silver bullet and no  substitute for common sense and communication.

The answer is simple, follow the simple four stages and use whatever methodology makes it easiest to achieve each goal and you won’t go wrong.

The agile organisation – a risky proposition

When somebody tells you that they want their organisation to be agile, what do you read from that? I’ll bet you feel a bit sheepish because you feel that yours is not, but it should be. I suspect that you remember this becoming a fashionable goal and thinking you would find out at some point what it all means.

If you answered yes, then you just joined a huge club closely affiliated to the new man”, “cosmopolitan woman” and all the others.

Maybe you have already announced to your boss that you are “going agile” and you still haven’t quite got round to working out what precisely it is, but you instinctively know that it is right for you. This is also a very large club growing daily. You won’t be lonely in there either.

The most likely thing is that you came across it via a systems project and all the new converts raved about their new found, almost religious belief in this cure-for-all-corporate-ills. You loved the stories about no decisions, no commitment, no planning, and no documents. ”This is damned interesting, I want to hear more”, you said

What did Bill Gates mean and what did it mean to Microsoft to be agile. Well Bill made the mother of all misjudgements when he said that the internet was of no interest to Microsoft and dismissed its potential. Within one and a half years, Microsoft had changed its mind, won the browser war with Netscape and delivered ASP, the first and still dominant commercial application server for the internet. That was agile, but it had not a thing to do with stand up meetings around whiteboards and walls covered in scribbles.

Horses for courses

Adapting a largely web development methodology and attempting to use it as a corporate philosophy is bonkers. Anybody who tells you otherwise is out of her tree and I’m not given to sweeping statements, so I’ll need to back that up with some arguments, here goes:

  1. Large organisations need process, it is their blood supply

Have you ever experienced an agile development team in the software world? The entire fabric of specialisation, process, controls and method are abandoned completely. Everyone claims to be multi-skilled, they are a team of peers, and they operate outside of the business rules and the corporate structures.
They romanticise it as though they were a hit squad, “licensed to kill”.

All this was put in place for a reason. A highly skilled multi-talented team empowered to JFDI is a great solution to political stalemate and analysis paralysis in specific situations, but would we want the CIA or Mi6 running the country?

Even in a highly mature market, our only corporate goal is to maintain our competitive position or improve it and in many cases this is driven by economies of scale and brand strength. This is why we have oligopoly and cartels everywhere manipulating key domestic markets such as supermarkets, fuel, banking, etc. A typical sluggish, but safe operation is exactly right for the job, low risk, stable and relatively low cost. It can function with the average half to three quarter wit at the wheel and needs no dynamism. It’s all about horses for courses. You can’t take away their crutches and expect them to perform. History shows that it fails miserably.

2. The vital relationship between good strategy, intelligent planning and sane execution. Anybody who has working experience of trading the stock market knows the persona of the Bull and Bear. These fabled characters are, of course, not different individuals, but different phases in each trader’s performance. Adrenaline, or cortisol take over quickly when the action begins and knee jerk reactions produce cowardly hibernating bears, or arrogant testosterone fuelled bulls

I witnessed this just recently when my irrational fear of snakes was suddenly reawakened by almost stepping on one and watching him thankfully slither away. I was physically unable to walk any further and had to resist the urge to run back to the safety of the car. It took a few days before I could walk in the woods again without watching every blade of grass.
I went to Vegas once and discovered that one friend had a deeply rooted gambling issue such that he could only see the chance of winning the next hand and had no inclination of the odds. The higher the likely return the more attracted he was, never weighing it against the likelihood of winning. Both of these situations are unnatural and unrealistic reactions to risk and opportunity and both are potentially destructive.
Let neither fear, nor greed be your master
Human reaction to risk and opportunity both tend to bring out the very ugliest sides of human nature, greed and fear. Neither of these emotions is rational and neither can be trusted for any length of time. They stem from another era when we needed to run very fast into a cave without asking questions, or gorge ourselves before we were driven away from the feast by a more aggressive animal.

It is because of these irrational behaviours that every aspect of business behaviour must be driven by a well formed strategy that includes:

  • A well understood risk attitude
  • A carefully devised approach to managing day to say risk that keeps the bear and bull within us well out of the picture and supports rational management behaviour.
    A risk strategy is not a simple thing to plan out. The fact of the matter is that the more attractive the reward is, the higher the risk will generally be. You have to make your choices


Some examples from the finance and gambling industries
E.G. A bank with its risks spread over the right industries and economies can expect that when one is doing poorly another is doing well and thus balance risk.A bookie can balance his books by taking a precise level of exposure on every horse in a race so that regardless of the outcome he wins a little over the day’s activity. Taking a big picture viewpoint as above, and developing from this a strategy that can be applied at a more granular level allows me to look at the snake philosophically and continue my journey, it allows my gambling friend to view his account on a quarterly basis and aim for a profit without making rash decisions on the spot. It doesn’t stop me from pausing till the snake has disappeared, or taking simple precautions in future.

  • A banker without a strategy would panic when three customers in a row defaulted on their payments. A bookmaker would run for the car park when two favourites in a row won forcing him to pay out. Only reliable insight, a solid strategy and a sound plan allows ordinary individuals to successfully run apparently high risk industries.
    Knee jerk “agile” management is the stuff of the hard luck stories.

3. Responsibility to our shareholders demands responsible decisions, audit trails and experiential learning.
Never before have we been so aware of the responsibility to our way of life that is borne by officers of private businesses everywhere. Every quoted company is taking the savings of pensioners and promising to give them a return annually so they can finish their lives in dignity. Every business has a responsibility to employees and investors of all sorts whose lives depend o their performance. Our world is continually burdened with more audit requirements as more frauds are discovered. We need robust and recorded decision making and we need accountability.
Informal decisions around the water cooler open the door wide for the irresponsible amongst us. I’m not espousing process laden organisations with forms to fill in for everything, but if you leave the cookie jar without a lid, you just know what is going to happen

4. The team is everything
Teams are made of people, not numbers, roles or little boxes connected by lines.
People must have their needs met if they are to perform and if individuals don’t perform the team fails. The things that motivate people to perform are all based around security. Maslov’s hierarchy of needs and later Herzberg’s hygienes and motivators all demonstrate that people need to know where they stand in their group and that their standing must fit their contribution. An agile team in the software world can, if correctly balanced to begin with perform very well, but usually it is utterly dominated by one person and becomes a benign dictatorship. Any attempt to get things right first time are usually dropped in favour of constant revision mode and quickly they realise that constant activity disguises the lack of any meaningful outputs.
The peak of the productivity curve is reached very quickly and after that it is all down hill.
In a non software world, there is little difference other than the fact that sometimes the potential downside can be catastrophic and constant revision is usually less of an option. Single agile teams tackling key business change issues can, if correctly and responsibly set up and under constant enlightened management, both during the forming, norming . . phase and into maturity, produce exceptional results for exceptional managers, but as a broad approach and using a large team of teams, the roll call of spectacular failures speak for themselves.

5. The map is not the territory
Most management philosophies are corrupted shamelessly and misused without any insight into what makes them work. Agile works, but only in it’s own unique environment and only when the core competencies exist and the critical factors have all been properly taken care of.
The risk is seeing only the landmarks, e.g. that you see agile as: describing things in user stories, or having stand up meetings, or small empowered teams, or Just In Time decisions making, or incremental delivery of product. You can adapt any of these that catch your eye and maybe they well deliver what you expect, but that won’t make it agile.
See beyond the ceremony, stare straight through the dogma, but when the time is right, do it right and the rewards can be very encouraging, but don’t close your eyes and try to drive by the SatNav.

Agile Agile the rumblings continue. .

Had another heated discussion with a self professed agile Guru who declined the offer to take the conversation public and place his arguments here, so I will do it on his behalf, but allow him the anonymity he desires. I will refer to him as John for argument’s sake and leave out some of the less important stuff.

The argument went like this:

John:  Your perception of agile is not right, you just don’t see the real benefits of agile because you are not a true agile practitioner, you just use it as a looser form of waterfall when it suits you, but you never really took the faith. We are delivering year in and year out in a way we never could with a traditional approach.

Ed: I’m inclined to take that as a compliment. I hold deep suspicion of anyone who takes a methodology too seriously above and beyond the actual delivery of an end. I have indeed used agile to great effect, but wouldn’t use it except in situations where I believe it is the better approach.

John: That’s just it, you believe you know better, but if you tried it in this other situations you would find , like we do that it delivers more every time.

Ed: I would be interested to see the definition of delivers for the sake of comparison. I won’t discount the possibility that you are right, because like you, I don’t have facts and figures with which to make a real argument, but I do have compelling reasons for my decisions that I am happy to share with you.

John: And they are. .

Ed:

1. Given and ideal world, the shortest and cheapest way to an end result in software is to know exactly what you want to achieve in advance, then know exactly  how is best to deliver it via software, have experienced people to do it, test rigorously against your acceptance criteria and roll out it once working perfectly.   I doubt anyone would argue with that.  I do accept that it probably has never happened and possibly never will, but it is the perfect scenario.
If we both started together and you used agile while I used a waterfall cycle, there’s little doubt that I would finish first and come in cheaper, though given the smoothness of the task, you would also perform well.

I would expect in that scenario, to get the business design right first time and agreed, to get the system design right first time and to have a very low level of defects in testing and instant acceptance by the client. Your trial and error approach would waste precious time and produce unnecessary artifacts.

2. The reason a business and I include in this definition the majority of public bodies, invests in IT is to save money or earn money. A commercial operation needs a return higher than the cost of capital as a fundamental criteria and it must also qualify as best use of limited capital.

The toughest part of a business case is nailing down the cost. If you don’t know the cost with a level of confidence, you can’t estimate the return confidently and you don’t have a case.

The agile argument about  “fit for purpose at  maximum cost” works well provided “fit for purpose can be relied on to deliver the expected returns and assuming that  it can be achieved, but ultimately, unless you can define the requirement well enough in advance to estimate the cost within reasonable boundaries and prove a business case, you are unlikely to ever get off the ground and probably should not.

 

John:

I don’t believe there would be a great deal of difference between the two performances is that case.

Anyhow, that’s all well and good, but there are many other scenarios where the business case is proven by the do nothing scenario alone E.G. defence, environment etc where the costs cannot be calculated accurately, but there is no argument about whether it should be done other than to figure out what will work best.

Ed: I agree with this entirely and in any of these many situations, I too would use an agile approach to remove the likelihood of a colossal error by learning as we go along.

It went on further but the point has already been made

This conversation went on a little longer and covered more ground, but I do feel that a lot was revealed about the importance of using the right approach for the right situation.

I do believe that a good business analyst with strong consulting skills can still earn his keep time and again by getting the requirements defined , understood , tested, verified and agreed at the outset so that accurate costing and planning can occur and the most cost efficient route can be taken.

I often see an agile approach taken as a substitute for consulting skills rather than as a good strategic decision and I do believe that this does nobody any favours least of all the agile approach itself.

About the author

What every business should know about agile

When is a business case not a business case

Agile- is this a short lived fad?

Is there still a case for IT strategic planning?

 The earth is flat in any case

Unless you’ve been asleep, in denial or on medication, you will have noticed, at some level at least, that in the last few months the key assumptions underlying our post industrial survival have been proven flat wrong.

To put that in content, the earth is not round but flat and the universe is merely mirror images of the earth reflected on a layer of slightly opaque gasses. The landing on the moon happened in a closed off section of MGM and hamburgers are good for you.

It no longer matters how much we borrow because we probably won’t be around to pay it back, only nobody will lend it of course because they are keeping it to line their pyramids with.

What’s the point in getting up?

You could easily be pardoned for asking the question and I can see some beginning to think like that.
That, in truth is what recession really is.
An end to economic growth is no big deal, we probably couldn’t survive without taking the odd breather. The bad bit is when sensible people suddenly decide that it’s not worth bothering, then we all suffer and what was a diet quickly becomes bulimia.
Read on and I will convince you that there bigger opportunities than ever, they just need a different approach

So how do you build a business case for IT investment and sell it to the board?

Realism
If you want to be taken seriously, you can’t base any investment on predictions of returning to normal in a few months or pretending the problem doesn’t exist.
You must recognise it and spell it out in terms of a clearly risk managed approach.
It’s unlikely that you are in a business where you can afford to make risky investments with long payback periods, so you need ROI fast, or even faster and you must prove that your eye is on the ball.

Prudence
Getting the financials right is critical, but that’s not the whole story by a long way, here’s a selected list of other issues in no particular order that are extra important.

· Make sure your initiative addresses the bulls eye in terms of business initiatives, not peripheral or even sort of important, but right up there.

· Make sure this is best deployment of scarce capital.  Remember it is no longer in unlimited supply.

· Time to market is very important and can easily be disguised behind impressive returns. Include timing in this calculation. Limping in a month behind your main rival is a great way to blow your big chance.

· Competitive advantage can be a critical factor that is easily missed. If all your competitors are cutting 20% off cost of sales and your initiative wipes 5% off, you might find the idea is less successful than you had hoped.

· Don’t suggest a major new 22nd century architecture if your goal can be achieved reliably with duct tape and scaffolding. This is about surviving today, not ideas or principals or fancy technology.

A little audacity goes a long way
Take a look at tiny little Porche and compare them to giant GM. Porche are busy taking over VW, to create the third biggest car manufacturer in the world and asking nobody for help, but using the downturn to make it possible.
If you are good at what you do and you can improve costs and capabilities, then watch out for all the leftovers of those overweight beasts that are shedding customers and reputation and be ready to take advantage of the opportunities that a downturn creates. In a ten horse race there are nine opportunities and one risk for the bookmaker. Don’t focus in on the risk and blind yourself to the opportunities.

Outside the box

There’s a great awkward purple elephant sitting on the board table and let’s stop pretending he’s not there.
The GAPE I’m talking about is the contradiction between thinking out of the box and following the strategy, the corporate plan and the processes, those things so beloved of civil servants and Ops directors. Let’s just say it:
How the hell can you be a process man who works to the plan and also think outside the box?

Well of course you can’t, it’s that simple, so you have to come out from behind whatever you are hiding behind and take a chance or two. Be up front about it that acceptance of your new proposal means revision of the agreed strategy and the current plan. That’s why they are there, not as straight jackets.
Delivering benefits on the cheap can upset lots of people, especially some members of the IT department, preferred suppliers and business analysts who have carved out cosy corners for themselves, but if that is the price, then accept it and pay it. Here’s a few examples:

· Be prepared to ignore the Enterprise architecture and do it the cheapest way.

· Be prepared to bypass the developers in the corner and buy something a bit scruffy and poorly documented, but supported from outside the business.

· Be prepared to find a few freelancers in Russia and get it done cheaper.

· Learn about mashups that can deliver the same result as an enterprise bus for a tiny fraction of the cost and get your show on the road.

· Get to know someone who knows about opensource solutions that are a bit ugly in places and awkward to install but are FREE and once you get them working they eat no corn.

· If you don’t know about agile methods, speak to someone who does and consider it for appropriate situations.

· Look for SaaS products that can start delivering returns in days rather than months

· This is a no brainer, but rarely done. Make sure your people know how to get the most out of the systems you already have.

So there is a new strategy after all

Yes there is a strategy for the current climate and I believe it has a valid legacy to take forward into the good times also.
Don’t stop looking for opportunities, but look harder, look outside the box and examine them more closely before committing. That usually requires an external input, but it is well worth it.
Don’t plan for a decade, but for this year, who knows what next year will be like, but this strategy will still be serving you whatever it looks like. That means dumping or revising much of the strategy and plans you are currently strangling yourself with. Do it sooner rather than later and free up your thinking and your energy.
Look for silver linings until it becomes a habit
Remember not to get complacent in the next bull market
 About the author

Agile development is the opposite of Fagan type process inspection, is this a phenomenon or another short lived fad

 

Programming is the discipline of talking pure logic to a machine that will unfortunately take everything you say literally and act upon it.
Process, is continuum of actions or mini processes that have inputs and outputs is repeatable.
The programme is a process and the programming too is a process.  It is therefore not surprising that many old school programmers simply can’t accept the principals that make agile development successful.  Are they right?
The Fagan method of code review is a milestone in the thinking that developed around software engineering in the seventies. Invented by no less an authority than IBM, the same organisation that taught me agile fifteen years later, the Fagan method set out by code review and manufacturing strength measurement, to eliminate errors and produce perfect code.
This was not Six Sigma, but a measurement of sufficient opportunities to eliminate reworks at the earliest stage. IBM claimed substantial improvement in case studies published.  The impact on software success rates was not however significant  and I, at least, am not shocked by this revelation.

Here’s my take:
Software is built by trial and error, simple as that. Anyone who doesn’t know that has never written code and needless to say , few people who have written code ever get involved with lofty ideas like quality improvement.
1. Treating something like software development as a continuous process that can be accurately measured and improved is a theory that has some purpose but is limited in scope.
2. The problem with software is not and has never been the quality of code.
3. The definition of quality is invariably wildly inaccurate when it comes to software products.
Measurement and process improvement
I have had mixed successes in the past with Fagan like measurement of exit criteria form tasks or products and from analysing the scope and quantity of bugs by programmer, designer, functional area and other criteria. Luckily I was doing this in an organisation where I was surrounded by professional researchers and statisticians and despite my best efforts to gain and follow their advice, not a single test I could devise was able to satisfy their strict criteria for reliable measurement.

The things that did pay dividends was creating over time a culture of following strict guidelines and strategies that reduce the risk both at a design, documentation, communication and process level.
The real problem with software
Any time you analyse the causes of failure in software projects, the causes are found in a just a few places:
1. The commonest cause of failure is that the system delivered does not meet the needs or expectations of the users or stakeholders. Bad requirements, bad communication, bad UAT,  took so long that the needs had changed, etc , etc
2. The business case was built on flawed assumptions and it should never have been completed, but pulling the plug is seen as failure while a poorly performing, or even stifling system is not.
3. Too little time was spent on the core needs and too much on the flora and fauna around the edges
The definition of quality
I’m not going to quote anyone , but t bravely state that the proof of the pudding is in the eating.
This system must satisfy users by helping them to perform to the expected level or exceed it and that includes keeping them motivated or better still re-motivated. It must also satisfy stakeholders that it is delivering an acceptable return on investment
Not a definition I know, but definitions are not all they are cracked up to be

What’s the verdict then
Well my verdict is that we should use process intelligently whether at a detailed level or a framework level and bearing in mind the power of a motivated workforce, but should not try to drive a square peg into a round hole in terms of ordering something that is about intense and accurate human interaction between professions that share little understanding of each others worlds. What delivers the goods or gets the job done is thing that delivers quality.

At IBM, not so long after the Fagan era, I learned to my astonishment that, in direct contradiction of his earlier paper, the cheapest way t fix bugs was not to fix them until you had to. This was measured very simply on the basis of whether  the bug was preventing testing form continuing. If not, the developer could decide to ignore it and usually did. Once a week we all got round a table with a huge projector screen and we analysed the bugs together, drawing simple dependency  links as we predicted the relationships between them. Every programmer knows how one bug causes several and sometimes the one causing them is not manifesting itself in any other way.
A group of switched on brains after the first coffee of the day could generally choose one or two bugs each to work on which resulted in most of the remainder disappearing and not returning.
If these bugs had been tracked down doggedly by tired individual programmers we would have lost cumulative weeks chasing shadows and ignoring the power of collective minds over large areas of complexity.

My belief is that well managed agile approaches to software improve communication immeasurably and eliminate the chance of a system not being what was expected. I think this is already applied to other processes from having a suit made to furnishing your home.
I also believe that quality in delivery is as much about motivation and peer review and recognition as it is about process, testing, logging and measuring. Real quality is built in and can’t be applied afterwards.  In my view, well managed agile creates a vibrant team environment where programmers have direct contact with end users and stakeholders and receive regular feedback and recognition.

How quality should be measured
Quality is measured very simply by how the software performs for the user and delivers for the stakeholder and the two go hand in hand. The quality controller is mostly the user with input from the stakeholder and a technical reviewer.  Perfect software with nill defects is not only unnecessary, it is not wanted in a world changing so fast that it will most likely be discarded in a short period of time. Fit for purpose is the ultimate measure, once purpose has been agreed.

Learn about Agile from Keith Richards

Keith Richards of Keith RichardsConsulting gave me my first formal training in DSDM nearly a decade ago and I am thrilled to have him contribute to our blog. If you would like to know more about Keith’s methods, you can purchase his book;
‘Agile Project Management: running PRINCE2 projects with DSDM Atern’ by Keith RIchards published by TSO’

What is Agile?

The word ‘agile’ has now become the hottest term being used in the methodology arena, but what exactly does it mean?

Amongst project managers and process mentors there is a broad understanding of the ethos of ‘agile’ but equally there is little agreement on the exact specifics of the genre.

Here are the views of KRC about the ‘new wave’ of approaches which are challenging the traditional view on how to deliver business benefit in the most effective way.

KRC believe that to be ‘agile’ means satisfying a set of criteria. These ‘criteria’ can be described as follws:

Responsive to change

Scope tolerant

Collaborative

Iterative

On time

Built incrementally

KRC also believe that the highest level of performance in the agile space can only be achieved if there is also compliance to a further three criteria. These are:

Business Focus

Quality Focus

Communication Focus

Over the coming weeks KRC will add to these criteria – in the meantime please get in touch if you would like to contribute.

 

Keith Richards

Why you still need business analysts and change managers when you do agile.

The first reaction to the theory of agile is to think something like, ”great, now we don’t to pay business analysts and change manager”.

It is logical without a doubt. You are not going to interview stakeholders, try to merge all the information, improve the waste and perfect a new process or design like we did in the good old days are you?No, you are going to very carefully select knowledgeable, committed and influential stakeholders and make them part of the team. Then you are going to facilitate them while tey design the changes.

What a splendid idea that really is. In one fell swoop you have created champions, educated them and built their commitment. You have fed the grapevine, the only truly effective channel of communication, with positive accurate messages delivered with conviction and enthusiasm by respected influential people.
At the same time you have gathered the most important knowledge into one place and not only got their input but had it bounced around amongst them and built up in increments.

How could it possibly be wrong? and how could it fail to win loyal support fast?

Welltheory is a fine and wonderful thing, but experience shows us a couple of important things: >

1. When analysts go about requirement the formal way and they interview everyone who could possibly be important to the project and hold workshops and demonstrations, it often emerges that:

 

a.

yes;”> Stakeholders are often deluded about what really goes on, they think big, but don’t do detail.

Senior stakeholders have imaginations just the same as programmers and when they suddenly get the feeling that they have box of lego and nobody
is watching, they begin to get creative in their thinking.

If this is allowed to happen in an agile project, then it will quickly begin to flounder
. Here’s where an experienced BA can test the theories, verify the processes and police the scope.

2. When change managers tackle a major change there are two forces in particular to be reckoned with:

a. The early adapters and champions that are encouraged and developed and who spread the word and are a generally positive force for change. Amongst these will often be people who resist and challenge things because they are taking ownership and are very serious about the project.

< b.

The blockers who initially grumble and then sit in ambush. Some of them are very powerful generally and amongst them will be people who are always agreeable, appear to be the picture of cooperation and are a joy to evangelise to, because they have not the slightest intention of taking any of it seriously and don’t feel even a little threatened.

These people will wreak havoc left to their own devices and a strong experienced change manager is critical to identify and manage the threat in a safe and appropriate way.

 

So how agile is agile?

Agile is very clever stuff and it can deliver an awful lot very quickly and cheaply when used by an intelligent , experienced team, but is a methodology, not a wonder drug, so use it to the limits, but use it in the appropriate setting and don’t throw away all the safety nets.

Ed Taaffe

Low cost, Low stress Customer focus-Here’s how

I know that some of my readers are from a technical audience and others are from business, so it is always a struggle to pitch the language right. I have decided to write a mixture of blogs in both styles so that we can talk about the same subjects from different viewpoints.
Last week, I spoke about agile. I wrote it in a non technical way, hopefully, but it was intended for IT directors, Programme managers and Developers.
Today I want to get right down to brass tacks.
20 years ago when I was running marketing campaigns for Encyclopaedia Britannica and my job depended on this months results, I learned very quickly that I needed to get right inside the customer’s head and stay there if I wanted my campaigns to work. A leaflet drop on the day of the FA cup would have been very amateur. I found we had too types of customer, book lovers and proud parents, so I wrote about the love of learning and about helping our children out achieve us.
I delivered the message when they were relaxed and receptive and through trusted media.
Why am I teaching my granny to suck eggs you are hopefully asking?
Well here’s why customer focus is at the root of agile methodology.
You may remember the parallel to lean and the spoon bending example
Just as the spoon factory relies on the bending of the metal to create value, so your business relies on the customer experience to create value.
When we create products, we are creating a perception of value in a consumers mind. Hold that thought. A Jag costs more to build than a Ferrari, but sells for a lot less. Why? Customer perception.
Amazon customers return regularly, most websites fail to get a second visit, why? That’s right, customer experience.
How agile delivers on customer experience
Agile should always begin with a customer journey. Remember, this is the spoon bending moment and agile gives you the hot 20% first, hence we begin by describing the customer experience and understanding the thought processes, fears, hopes, wishes and aspirations.
Now most agile practitioners talk about customer journeys. The words are fine as long as the understanding is clear and I suspect that it rarely is, experience seems to point that way. Customer journey is a whole lot better than programmer’s viewpoint, but it is still a process viewpoint.
While understanding the buying process is an advanced state for many marketing professionals, making an attempt to understand the customer experience is an enriching journey in more ways than one.
The agile process should begin with real live customers explaining their experiences in a focus group or other comfortable setting and then continuing to review and comment on the wireframes and prototypes as they emerge into a system ready for pilot with a larger group.
Release early and release often
The Dragon’s Den may be mostly high Farce, but they all agree on one piece of solid advice, The only true market research is to go out there and sell it to real customers. This is the second strongest argument for agile.
Start with customers, pander to their experience, produce something quickly and test it with more customers until your customers have built you a winning product that can be released with confidence and with a loyal and knowledgeable user base attached.

Ed Taaffe