Image result for bot wikipedia

I wont be long-winded about this, I’ll discuss it via email with anyone who is interested, but I’ll break with my usual mode and come straight to the point.
A great many people who know little at all of machine learning  and even less about people and many more who are simply  oblivious to the potential consequences of their words are talking about the miraculous things we can expect from Machine learning.

What is ML in a nutshell?
Academics break ML into two modes:  Supervised and Unsupervised.
In the case of the former we give the machine a large corpora of content and ask it to decide what will happen next, or to find other similar instances. A translation service for example  begins this way and learns after a while to translate without help.
In the latter case, we give it a body of content and ask what it thinks of that.. Google search is an example of this approach and it simply makes sense of what it finds.

Often we give it a few hints like “Classify this for me and establish links” as in Google search. This would be a “Classification problem”. We might on the other hand ask it to read the racing papers and decide who will win the four o’clock today. This would be a “Regression problem” because we are asking it to look at the past and predict the future. Yes all of this is highly condensed as promised, if you are an expert you don’t need my explanations.
Understanding what the customer will want next year, predicting the weather, finding Oil under the sea, predicting tumours, the challenges are endless and the rewards enormous.

What is the loop of self-destruction?
The loop happens when, thanks to social media, a good, but no a sole example, the machine begins to make judgements that influence the data and then discover exactly what it predicted.

As with humans this will give it the machine equivalent of a big head and possibly some citations and will lead to even greater confidence and fewer checks and before anybody spots it, it I all too late.
If any Movie producers out there are stuck for an idea, I am available to help with the plot. Here is a simple example we are all aware of:
Joe Gel, and Josephine Lotion our dear friends, represents an enormous body of intelligent and informed people who spend most of their waking hours  checking back with their phones for reassurance. Joe searches Google for Tom Raspberry, his favourite politician and receives a huge list of pages. The ML in google notes his interests and begins sending him dozens of articles about Tom Raspberry, what he says and does and what people say about him. Unwittingly Our pal Joe has become astonished by the fact the whole world seems obsessed with Tom R and realises subconsciously how important to is be aware of Tom R. He begins to tweet and have the odd Facebook conversation about something he read. Immediately the ML in Facebook and the one in Twitter hone in his apparent obsession with Tom R and all begin to bombard him with content and introduce him to thousands of people with the same problem. Poor Joe.

Now our Machine does a Recce to see what are people talking about and it discovers that millions are talking and reading about Tom raspberry and concludes that tis is the way to keep the customer happy so it ups its game and heightens the emphasis. It also confidently announces that Tom R will undoubtedly be unstoppable in the forthcoming election.

Joe and Josephine realise the importance of not standing in the way of a social crowd and are not about t be shunned and subconsciously they begin to take more interest in the positive stories about Tom which now triggers the Machine to filter their feeds and search results and friend recommendations etc more toward the positive . You don’t need me to finish the plot. There is only one way this is going. Imagine if the secret services relied on this kind of information to brief their bosses. But they do, don’t they.

You may well think, as I do , that despite the  shear “wrongness” of rigging democracy, whether by design or accident, it matters little who is elected anyhow. In that case imagine the same scenario when the machine turns its hand to guiding change in a government department or a large business , or guiding product development or even finding the cure for cancer. If you would like to see many better examples with a strong scientific analysis, check out Weapons of Math destruction.

One wonderfully simple yet highly destructive outcome of ML that I have seen up close is the  call centre  automated system that recognises your telephone number, calculates your value as a customer and decides if you will be answered, how long you will have to wait and whether you get to speak to somebody skillful.  Just to update my card details for a £20 a month hosting service, I had 11 hours of my time wasted, had my service disrupted and was threatened by a bot with £150 fine to put the service back on.
I hate to disappoint you, but if you have ever had an IM conversation with a patient lady on the support portal “That was no lady” nor was it my wife, that was a distant cousin of Cortana.
If she did not know the answer, or more likely the question, you were never going to be served.
If you are wondering what might happen to your pension, your job and your home if these guys get involved in stock trading, well take a look here  According to a 2014 report, sixty to seventy per cent of price changes are driven not by new information from the real world but by “self-generated activities”.

It’s not all negative by any means. I actually do use ML to predict the winners of tomorrows racing with a consistent level of profit. When I get it wrong, usually after a late night of programming with insufficient testing, my winnings disappear very quickly into someone else’s pocket and I sit up and take notice.
I sincerely hope that someone starts sitting up and taking notice soon  of the impact of poorly programmed Bots that are already beginning to increase risk for the most powerful nations on earth.

Two elections that few expected and a majority didn’t want- the information perspective.

Two groups of people mastered the truth about data, its interpretation and it’s presentation and two groups didn’t. The former of course are Republicans in US and Brexit’s in UK.

The theories are not new, nor are they difficult to understand but executing on them might only seem an option to somebody with no other options (Trump, Farage). Most people would never dream of applying for a multi-million pound loan and filling the entire application with fiction. It’ not because they believe it couldn’t be done, we’ve all seen it done, but because they were not brought up like that. They still believe they have other options and they won’t make that move to the dark side.  People who did it and often did it successfully were generally people who had no other option and didn’t see the downside. In banking there have always been checks and balances, but in media there has always been a weakness and right now there is a total void where the media used to sit. Enter two people who had no right to talk to us and simply couldn’t win. Donald Trump and Nigel Farage and look what happened.

Why did they win?

Nobody buys newspapers much less reads them any more and old fashioned journalists have hanged themselves by their braces long ago. There’s no revenue stream any more to sustain them, for what little use they might have been once and now we have instead, social media, polls (much the same thing) and personal content filters all very much for sale to the highest bidder and very controllable.

As IT professionals, we have always had to operate in a tricky area between less than perfect hard data collected from disparate systems or countries and the jazzy punchy little reports and dashboards that our paymasters dine on to excess.
Let’s be realistic, hard data is of little use to anyone, it’s only when the great unappreciated droves have cleaned it , sense checked it and translated it to comparable terms and then turned it into consumable visual bites that decision makers can grab it and do something useful with it, like selling a lie often as not.
When I see an email proposing a radical new strategy on the grounds of the figures in my reports I can sometimes pull somebody aside and caution them on the potential for error in the data or point to a conflicting KPI and suggest further checks. This type of scenario is part and parcel of business life in the world of IT.

Lets’ now take  a look at the world of rigged polls, injected social media, wholesale unapologetic lying and presentation that appeals to the baser emotions of fear and mob action.

  1. Destabilise the enemy by kidding them that they are doing better than they really are and encourage them to focus their attempts in the wrong places. Easy and very effective.
    I’m not sure that Farage had the budget or reach for this one, but it was certainly a central part of the Trump strategy and clearly explains why the bookies did so well. Read about Colonol John Boyd’s OODA loop to learn the basis for this strategy. Ask any social media consultant how to do it.
  2. Fill the media with mountains of lies about the enemy, begin by raking up a few real stories to set the scene, wait a few days for people to stop questioning that and then pile it on thick.
    For some good examples of this in action check out Niemanlab.org
    In the UK voters were incensed about legislation preventing the sale of bent bananas, (it never happened)
    They were told it would mean hundreds of millions per week spent on the health service, all immediately denied after election day. The list goes on and on.
    Why is it easy to find and cultivate these knuckleheads? Because Facebook, snapchat and others have detailed profiles of their entire lives and everything they discuss, read, think, say do, even the therapists they visit. Finding a numpty on Facebook and sending him to kill the president should be a doddle ( I didn’t give you the idea by the way)
  3. At the last minute create a powerful picture to capture the emotions of the more  ignorant layers of the electorate and plant a semi-subliminal message that appeals to the baser emotions i.e. massage the “lizard brain”.
    Nigel Farage’s posters were so blatantly aimed to incite civil unrest that there were well supported calls for him to be charged with this offence. On the day after the election, people of foreign birth were accosted on the streets around Britain and told to “go home”.
    In Florida, Trump’s “swing state”, Trump supports were seen on election night chanting “Lock her up”. The Canadian embassy crashed under the load of applications to immigrate and the flood of Mexicans going home in disgust continues.

In both cases, before the vote had been fully counted, the claims were being refuted and the lies were being reneged on.   No “Lock her up”, but tributes to her commitment. No “Go home”, or even close the borders in UK.  How far the reversals and denials will go on both fronts is yet an unknown, but nothing would surprise me mainly because the stretch room is usually very small once they find themselves in the reality of office.

So what is the lesson?

  1. Don’t rely on traditional media to be guardians of truth, what’s left of these are either state owned or in the pockets of the multinationals that own the politicians.
  2. Do I need to say it? Don’t expect politicians to tell the truth and don’t assume they have the same values or standards you do.
  3. Opposition parties ( that means both sides) have to guarantee the truth by monitoring and refuting media and actively and aggressively bringing liars to justice before and after the event. We all want the truth to be spoken, especially on important subjects and that is one of the things we will all agree to pay for and support politicians to achieve for us, so join the winning side for once and promote truth and honesty in media by attacking lies and dishonesty and supporting those who stand for truth and honesty.
  4. For god’s sake, or whatever you believe in don’t believe everything you read in social media, especially if it’s controversial and even more-so if it agrees with your predispositions.
  5. Make sure you defeat the personal filters to get alternative views of the world, use proxies or VPNs to hide your location and see a different view of the same thing. Ask a teenager if you don’t know how.
  6. Take responsibility for your own mind and for your decisions and actions.
  7. Don’t stay silent and let the liars prevail, it’s now a community affair, so speak out and have an impact on the balance of opinion. This way we can all help to keep society safe

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Don’t forget about innovation

The first part of any project involves defining  the problem, deciding where to look for the solution and how to proceed with the search and finally defining the solution, validating it and getting agreement from stakeholders.

Now the nature of Technology is such that few of us are aware of what is possible and even fewer are able to see the impacts of these suggested solutions over and above the promised outcome.

Not only are few of us equipped to access the best solutions, but even fewer are able to recognise when we have a problem.  In technology speak a problem is closer in meaning to a mathematics problem , it doesn’t necessarily cause that irritating pain that our marketing colleagues like to focus on.

E.G.  Let’s say chief zongawonga is worried that with 19 more children due this spring, he won’t  be able to catch enough fish.  His bright young progeny identifies the problem and suggests metal arrow heads that are more effective and mean they can quickly make extra spears so everyone can join in. That represents a problem known and tackled.
However, Zongawonga doesn’t know that monofilament nets are cheaper than arrowheads and one child can feed the whole tribe with one net.  Until he becomes aware of the nets, he won’t know he has a problem, or until his wives start leaving for the easy life with his neighbour who doesn’t expect them to fish.

The process involved in definition of problems and solutions differs not at all from the age old problem of effectively searching a global mountain of unstructured content as described below.

First you have to arrive at some fundamental conclusions about the problem, the possible solutions and how and where to go looking. Consider this example and then have a read through the innovative solutions put forward by Zyra and see if you don’t begin to question the stuffy, stuck in the mud methods of innovation and improvement that have become embedded in most modern businesses.

“just what are you looking for, anyway?”

  •  A known needle in a known haystack
  • A known needle in an unknown haystack
  • An unknown needle in an unknown haystack
  • Any needle in a haystack
  • The sharpest needle in a haystack
  • Most of the sharpest needles in a haystack
  • All the needles in a haystack
  • Affirmation of no needles in the haystack
  • Things like needles in any haystack
  • Let me know whenever a new needle shows up
  • Where are the haystacks?
  • Needles, haystacks — whatever.

http://www.zyra.org.uk/needle-haystack.htm

 

The answers you give to the questions above will have a profound effect on how you approach the project, how you define success and your likelihood of succeeding.
Furthermore, whether you are in charge of developing  market leading products, or keeping your company  at the cutting edge, taking a little time out to consider these questions and address them  innovatively will take your performance to the next level.

The trap of settling for mediocrity

The trap of settling for mediocrity

Without boring you with a treatise on macro-economics, I can only get my point across by saying that, not only is every professional and tradesperson in our world challenged to know less and possess fewer skills, but most business we are employed by will have not a single individual who understands what the business does from end to end even at a fairly high level. I have met more than a few CEOs who didn’t consider it important to them to understand the business they were charged with, but focused on a small handful of KPIs to get him out of jail this year.
The guy who scans a label and uses another tool to locate the good, before handing it to someone else to pack and pass on to the loader . . .  is no less dumbed down than the Financial controller who receives the chart identifies the KPI and barks out an instruction via email.
Mediocrity is everywhere and it is tempting to succumb to it but if you are not seduced by warnings of the long term damage to our world then pay some heed to this:

Skill and competency is 98.5% self-esteem and 1 % knowhow, the other stuff isn’t even worth talking about. Although I have never met you, I know without any fear of contradiction which of these you most fall short on. The reason I know is that a confident person can be taught almost anything while others can make almost anything go wrong.   Have you ever noticed how football players go on runs of scoring and always return from a lean patch to push the boundaries even further.  The tool they are using is deep seated self-belief.  Sure, some aspects of this are probably learned from Mom and Dad or even inherited in their genetic make-up, but don’t start making excuses again. The world is full of superstars who had no running start and plenty who got their inner strength from facing adversity.

Average is a dangerous illusion

All my life I have sought relaxation, release and inspiration from games of chance. I know the numbers and I totally understand what you have to do over the long haul to come out ahead. I am happy to say that I have been coming out on top since my early twenties and if there is one thing you must learn if you are to stand a chance against bookmakers, poker players, or any other adversary it is this.  The averages are the things that make you profits, but to achieve an average that keeps you ahead you have to perform above average by a good margin at all times. You don’t just need a margin to win by, you also need a margin for error.  Then it all works out like the good book says. Sadly it is true that the level we aim at is always a shade higher than the one we actually achieve

Average, for all it’s flaws, is above and beyond the capabilities of so many and only because of their mind set. The Peter Principle : Why Things Always Go Wrong Paperback – August 19, 1998 was and still is  a master class in systems thinking.
Peter observes a phenomenon we are all familiar with, i.e. that once people begin on the promotion ladder, they will then, as members of the hierarchy, be promoted until they reach a level where they are totally incompetent.    Naturally this comes as no surprise to the reader, what still seems to surprise many is that the outcome of this apparently innocuous behaviour dictates that everybody in charge in too many businesses, then and indeed today is incompetent to some degree. Had I made this statement before presenting the evidence, you would have dismissed me as a crank and convinced yourself not to listen, or found another escape, but now that you have read this far, surely the conclusion is inescapably true even if the reasons for such behaviour are overly forgiving in many texts.

The average, therefore, is not only something that describes systemic incompetency, but over the long haul, it is indeed the result of settings ones sights somewhat above that level. Are you still content to get up tomorrow and set out to be average?
At the end of every good lesson is a stiff test and here is yours: What did you take from this little blog?

  1. The world of business is messed up and there is no point in even trying, what can I do?.
  2. The world of business is full of incompetent people, the opportunity for competent positive people has never been greater.

You can mark yourself on this one

Confirmation bias at the speed of light

The extraordinary world of the trader really opened my eyes

Image result for halo effect

Most readers will already be aware of confirmation bias and will no doubt believe they have it under control. Good for you. If however you are trader, or gambler the likelihood is that you have got it down to a fine art and it has become a heuristic behaviour.  What I mean by this simple, is that you’ve been doing for so long and become so attached to the theory that you are smarter than the rest, that now your brain does it in auto-mode without your intervention or even awareness.

Recently I spent some time developing trading tools for people earn a  living trading movements on a betting exchange.  The betting exchange, for those who are not familiar is a simplified clone of a stock exchange and is very similar to currency trading .

Some of these people act as bookmakers accepting bets form the public, while others simply trade the movements in the market or “SCALP” the market.
I was amazed to hear bookmakers of some standing use the phrase, “ Let’s get this one beaten”.
Quietly testing the intent of this over a period of time I realised that they actually associated their attempts to attract money and lay the runner with the runner losing.  Naturally they would never admit to it openly and they all know that such a thing is impossible, yet they daily select a weak runner and go about betting against it with the hope that it will be beaten and regularly it loses and bit by bit the brain has begun to associate tis intent and action with the outcome. Given they will be laying an 8: shot that really as a 1 in 12 chance of winning, the see it lose 11 times out of 12.
What is happening here is the brains own “Inspect and adapt learning process”, taking what it sees at face value and jumping to dangerous conclusions.
For years renowned physicians dispensed useless or even dangerous treatments to unsuspecting patients while convincing themselves and the patients that it was having a positive effect. After all, some of these people recovered. I wonder how much of this still goes on?
Economist and ex trader Max Keiser recently did a TV show on economics where he talked about stock traders believing they were changing the economy by their actions when in fact they are simply gambling with liquidity against other traders.
The UK government has been conspicuous in their inactivity in terms of fixing the economy, or even banking since the crash. They have sat back and conserved the status quo for all intents and purposes, yet George Osborne makes speeches in which he associates his office with a “reported “ improvement in the economy. Has it improved? If so, has it improved beyond a possible “ gently rising tide”?  Does he actually believe it?  What do you think?

Given the power of such self-delusion, would it be a shock if we found traders fixing things like exchange rates, or would it be far-fetched to imagine it might then stretch to bribing politicians and officials.
If those same people had gone into the munitions business instead of banking, do you think they might find a way to start wars? Do PMs and  generals ever admit that they achieved nothing ( best case scenario) or created a catastrophe?

Systems thinking

Negative bias the damage it can cause

Specialist in Smarties, Chocolate Beer consultant, or Expert in customer complaints from Obese women on Tuesday afternoons.

Are you a specialist?
By the way these specialisms do exist and I have no doubt you could add to my collection. If you want a light hearted look visit this article.

As an interim connected to the technology world, I get a lot of calls and emails from recruiters and over the years the, the most notable change is the degree to which these people expect me to be specialised. Why, I wonder, did I spend five years studying the subject to get a broad and deep understanding and then spend the next ten deliberately learning the different aspects of the industry on the ground. This is what we did, what every professional did.  Do you know a lawyer who can only handle attempted fraud by Orientals on diabetic white Americans living in London on Fridays?  I doubt it somehow.

The need for specialisation is the back-bone of economic theory
If you are drawn into a discussion about why some African states have failed to develop a modern economy despite being richer in resources than we are, you will generally find yourself in two separate discussions: One about availability of capital ( An Adam smith fan) and the other about specialisation, or lack thereof.  Given that many of these economies have weak currencies, micro-loans can now finance substantial start-ups and there has ever been more lost money ( for want of a better description) looking for a legitimate looking home, so arguments about lack of capital no longer work. The problem is inability or unwillingness to specialise. The scenario: I grow vegetables, hunt for meat, make my furniture from bamboo, make beer from fruit and find herbal medicines when I am sick.  I am always poor and I lack equipment and skills to do all of this stuff well.
Far better, if I could just grow vegetables and swap some with the hunter for meat and with the carpenter for furniture.  Each could enjoy a margin because of their specialist skills, tools and experience. That is the fundamental theory that still underpins economics. For some reason, in some places it is slow to happen.

 

It gets boring just placing the full stops at the end of sentences, can I have a real job?

The problems rarely lies with the client who needs help, but with the  recruiter who calls himself a “Consultant” but is in fact a commission only salesman who last year was an Estate agent or rather a  “Property consultant”  and knows as much about what I do and what my potential client needs as I do about life on Mars.    For an insight from a legitimate HR person have a read of this

The problem begins sometimes because the client who either doesn’t really know what precise skillset would be best for his needs and either by habit or misconception expects a recruiter to help with this problem. What a very large error that is.
A rough guide goes off to the recruiter who then places adverts via an “autoposter”. This tool is preconfigured and whacks his advert out onto numerous job boards where his agency has accounts.
He then instructs a resourcer to do some searching on the main databases, e.g. Jobserve and Jobsite for IT and these are imported directly into the system and emailed with a standard mailing.

Now the fun begins. The resourcer has not yet been promoted to recruiter and is supposedly learning the business by pouring through millions of CVs starting with keywords searches Now we have identified the first problem.  Just like Google SEO, if you want to appear in the SERPs you must include the keywords.

I sometimes help businesses to profile skills and to shortlist  CVs, so I know just how tricky this can be even for a seasoned professional. A resourcer, or a recruiter with no grasp of the IT profession or the task at hand stands no chance at all. It is simply a random matter of chance whether or not they find a remotely applicable CV and it is no more or less efficient than asking Siri or saying “OK Google” to your phone.
If you’ve ever engaged an agency, you’ll be familiar with the bombardment of pointless CVs and then the ever more pestering phone calls for feedback and pressure to hire.  There is a reason why it feels so pointless, it mostly is exactly that and there is a reason why it is so expensive, all that pointless effort has to be paid for. Don’t be deluded by commission only, that is always the most expensive purchase. If you have two agencies at work and one has submitted what they think are strong candidates, they well be calling every other candidate in the market pretending they are submitting them in order to rule them out for the competition.  Don’t kid yourself that there is efficiency in this.
Why am I telling you all this?

The reason we are being driven to specialise is to a large degree so that recruitment agencies can understand a tiny bit more about what we do.
Let’s face it the way to make it big in recruitment is to specialise in Saturday morning perfume demonstrators and get them all in your database. Easy to understand, no search issues, you can truly be a specialist and maybe even a consultant and all is well with the world.
The problem is that IT doesn’t work this way.  Every system that is implemented impacts and is impacted by many others, all different and employing different infrastructure, protocols, languages, data structures, security arrangements and so on.  You can’t simplify IT to suit your needs and every attempt to do so has ended in tears. Understanding the IT impacts is only the final mile, before that you have to understand the process impacts and manage the stakeholder culture and much more. All of this is heavily entwined and all absolutely unavoidable and critical. When you need a specialist to solve critical problems in your business, ask a specialist for advice not a recruiter.

The evils of groupthink and the risk to all Social media users

AKA “ Say hello to someone”
Power to think
The most spectacularly creative people in many fields are often introverted, according to studies by the psychologists Mihaly Csikszentmihalyi and Gregory Feist. They’re extroverted enough to exchange and advance ideas, but see themselves as independent and individualistic. They’re not joiners by nature.” NYtimes SUSAN CAIN JAN. 13, 2012

Power to innovate
No great life-changing, or world-changing invention has ever come about as a result of market research. Whatever the agile evangelists tell you, the reason for success in this area is not what they think it is. Asking the customer is important but it is not innovation. It is a stark and unavoidable fact that when Henry Ford asked a carriage owner what he needed from transport, he replied along the lines of; ”A horse that doesn’t foul the road”, or “A nice warm fire in my cab”. He never mentioned anything resembling a motor car and he never would either.
Try to imagine our world without the legacy of Henry Ford
In fact I would argue that lack of this type of thinking is why we have moved from the single blade razor to two, three, four blades and who knows where this will end. No there is definitely nothing much to be gained by innovators from hanging around in large groups or spending their day exchanging messages of one sort or another or asking customers. Watching customers is different, but that’s a different blog.

Decision making power
Making decisions is the toughest part of running a business, a department, a team. It is tough because if there was a safe answer it would not need a decision. Let’s face it if you are in a room on a cold winter’s day and the door blows open allowing rain and cold wind into the room, nobody needs to make a decision about what to do, now do they?
Decisions happen when there are several options and imperfect information and someone needs to make a call. Decisions involve risk management and playing the averages, but most of all they require taking responsibility and having the confidence to do so, i.e. leadership.
So what happens when you ask a committee to make a decision? Well the commonest thing is that they decide to close the door, just after the last piece of furniture has blown away and the rain has shorted out the electricity, but often they make no decision, or if they all belong to a particular religion, group, union just about anything could happen, it could end in a brawl. If there is a bully in the group they will guess what he wants and pretend that is what they want, but he won’t of course be accountable. The simple answer is don’t do it if there is any other option. Fake it if you absolutely must.
“In big business and national politics, groupthink has emerged as one of the most damaging spectres of our time. Unquestioning corporate cultures have been blamed for scandals ranging from Volkswagen’s emissions-test rigging to the sexism controversies that have surrounded Uber and American Apparel. Last year, commentators accused hive minds on social networks of scuppering open debate in the run-up to the UK’s general election. But does groupthink always require such a large canvas to flourish?” Economia 22 June 2016

Power to compete
Is there any industry left where it is not an accepted policy (among the group) to recruit from within the industry and close competitors? . If you believe Volkswagen were the only ones fiddling the figures you are even more naïve than I could have credited you for. Check out where the senior executives of all the other Auto businesses learned their trade and developed their MO and you will have a map of who else is doing it. The fact is that behaviour patterns, but especially abuses spread like wildfire across industries and then gradually leak outside. Why? Because to continue with bad behaviour, it has to become accepted as the norm. You don’t need me to make a list, everyone has a huge list of such behaviours they could rattle off, if you are stuck, just start with a google search for “*ism”.
Competing in any market in the modern uber-competitive environment demands that you differentiate strongly and then communicate this difference effectively. Naturally it has to be a difference your customers value, but we have limited space to fill today . . .
Imagine how successful you will be at differentiation if all of your thought processes are shared (perhaps hourly) with your competitors and you can’t remember the last original thought you had?

Power to recruit
A note from the head of business change to HR reads:
“ I need a consultant to help with this important programme of change. Find me someone from within the industry, we need someone who will fit in with the culture and . . .”
The only organisation I ever worked for that is no longer around used to do exactly this. Their Achilles heal was that they were all from an industry where they had never learned to make decisions, but whenever they sought an outsider to help them, it began with “You guessed it “.
Social media could well be the most destructive force in modern times
Let’s forget about politicians playing to the baser forms of humanity just to get votes and all the evils we are forewarned about. Think Brexit, Trump , I’ll leave the list to you.
When is the last time you saw a real conversation happening on any social media platform? Well I can answer it for you with a fair degree of confidence. The answer is either never, or not for a number of years. Let’s ignore all the Hootsuite jockeys and other fakes dumping garbage on to the internet all day every day and those paying a King’s ransom to look important, the remainder are following the “groupthink” approach to social media in their sleep. Pretending to be interested in the things they are pretending to be interested in so that they appear to fit the imagined norm and
saying the things they guess the other “normal” people would expect them to say, or rather believe they are expected to say. I hope I didn’t confuse you. You could struggle along with these thoughts for a few minutes and probably end up dizzy, or maybe you’d prefer to grow a few brain cells and “say hello to someone”. Here is the big secret waiting to get out; when they are not faking it, they are all human just like you, they are just scared to let it show. That is my new theme for the remainder of 2016, “ Say hello to someone”. Or just in case you haven’t been paying attention: communicate

Requirements engineering strategy can make or break your project .

Part one – What would you like sir
Part two – Requirements,tests, training, help files
Part three – Why no project exists in isolation-what should be done
Part four – Business rules, Process rules, Process, Data, different viewpoints
Part five – Testing requirements is not optional
Part six  -Requirements strategy can make or break your project

If you’ve been reading my blogs on the subject of requirements, you will no doubt appreciate the level of importance I place on getting requirements right and then testing them before committing yourself to a contract or a technical specification.
Just in case you are thinking that this is in some way anti agile, then nothing could be further from the truth. Please read my blog on agile requirements engineering for a discussion on this topic.

Let’s take this opportunity to recap on why we do requirements. The simple fact is that once a development team start work on creating a technical specification, the till begins to ring up with costs and every time you make a change after this point more costs are added. The closer you are to rollout when you make the change, the greater the extra cost will be and that time phased increase is exponential, therefore the first purpose of requirements engineering is to contain costs and eliminate or reduce slippages.

The second purpose is to make sure that the end product delivers value by meeting the needs of the business precisely in order to meet or exceed the benefits targets set in the business case.

Getting requirements right therefore, is absolutely critical if you are to contain costs and deliver value

Developing a requirements engineering strategy

Each organisation and each project are different and requires a tailor made strategy for getting the requirements right by recognising, exploiting and working with the capabilities of the organisation.

To do this successfully, you will need a few things:

  1. A detailed analysis of the stakeholders involved in the as-is process in order to make interviewing efficient and thorough.
  2. A strong feel for the past experiences of the organisation in terms of successes or failures with software projects in terms of meeting business need and staying within budget and time constraints. This along with some tactful exploration of causes will give a strong steer about the current capabilities of the organisation to engage with a formal requirements process and to take seriously concepts like change control.
  3. A feel for the organisations ability to take on and successfully adapt change will help you decide whether and how far you might decide to upgrade their skills and attitudes to requirement engineering
  4. A good indicative plan that indicates the products to be created, their acceptance criteria, time scales for delivery and the amount of effort that will be needed from stakeholders.
  5. The understanding, agreement and total buy-in of key stakeholders to the proposed approach with full support in terms of making people and facilities available for the process

 

Stakeholder analysis for requirement engineering

Step one

Using RACI to identify stakeholders for the Requirements function.

In my blogs on business case techniques and models I discussed the importance involving the right stakeholders to gain buy-in and get a real picture of what success will look like.
In order to design the system, you will need a different type of stakeholder list, one that describes roles, responsibilities skills and communication. This list will give a clear view of where the knowledge skills and the responsibilities lie within the organisation and therefore point you at the right people to describe requirements and take responsibility for their quality. My favourite tool for achieving this is a well known HR instrument known as the RACI chart. Responsibilities Accountabilities, Consults, Informs and it provides a two dimensional view of a team that helps you quickly analyse the team for adequate skills, supervision, communication and quality control. It can be adjusted in form or emphasis to suit your precise needs, but the fundamental principal serves well in any circumstance straight out of the box.
Here’s an example;

RAACI example

The table above lists nominal roles across the top and Activities down the left, each cell is then marked with one or more of RACI to show who has what role in that activity.
The example is for a software development team, but if you were building a hospital it would list things like planner, builder, architect and activities like approve plans, complete design, etc.

A software project for a builders might involve, Construction director, surveyor, cost controller, quality controller etc and Liaise with clients, agree costs, sign off completion, etc.
Apart from being a huge benefit in helping to highlight the important stakeholders for your requirements process, the RACI chart also provides a quick but effective sanity check to make sure that your team is adequate and that there no duplications, conflicts or gaps.

Horizontal analysis of each task will quickly tell you whether there are sufficient doers and accountability exists and is in the right place.
Vertical analysis of roles will quickly highlight overworked, underworked, misplaced authority or responsibility and lack of or too much consultation and information.
Too much consultation stifles decision making and too little leads to dangerous decisions. Corporate culture can lead to busy bodies who don’t always follow through, or teflon in-trays. All of these issues need to be understood and ideally addressed ahead of system design, because the system will only perform as well as the people who use it.

Once you have created this chart, the next thing is to create a list of names with contact information to place in each of the role areas. If the project is large and/or complex, there may be more than one name in each box and there may be further sifting to do in order to resolve any doubts.
Don’t be surprised to find skeletons in the cupboard and lack of agreement over who does what. If you uncover these issues, now is the time to discuss them at a high level and attempt to get them resolved ahead of requirement gathering.

Step Two
The second step is identify the Actors, the people and systems that carry out tasks as part of this process and the sources of all the skills, knowledge and decisions required to complete the process successfully. This will generally begin with the R people in your RACI chart and will usually expand to people who do the work with or for the R person. E.G. the R may be in your chart as the Technical architects for design, but on investigation, you may find a DBA, a SysAdmin and many others who play key roles and these people also need to be interviewed.
The actual actors may be more usefully broken down to specific skill sets or disciplines as opposed to specific roles. E.G you may have Document writer, Interviewer, modeller all of which are actors within the BA function. This approach gives a slightly more abstract approach that lends itself to reorganising roles for efficiencies, or to adapt to a new system.Never be tempted to accept the assurance that a supervisor can tell you the whole story and you don’t need to talk to her/his charges. If anything this should be seen with an element of suspicion and tactful verification is a good idea.

To better understand and communicate how the various roles influence the outcome of a process, it is sometimes useful to create a fishbone chart adapted slightly for the purpose. Here is how i like to create it:

Stakeholder fishbone chart

The fishbone can be made as complex as you need it to be in order to show all the influences on your new project. The thing that becomes apparent very quickly is that whether they are aware of it or not, most of your stakeholders rely on others for skills, support, knowhow, data, administration and other inputs that can potentially prevent the processes progressing and hence you need to know about it and to resolve, or risk manage it.
Your fishbone chart will immediately tell you who to talk to and by way of a bonus, it will very quickly highlight the areas of influence, both positive and negative when it comes to rolling out.

Organisational culture and requirements process maturity.

Once your target interviewees are lined up, your next step is to hold a number of short interviews and/or an interactive workshop to discuss past projects, how they went and how the organisation reacted at the time and now. Asking them what they felt went well and what went wrong will go a long way towards gauging what will go down well, or will meet with resistance in terms of methodology.
The aim is not to deliver the perfect project, by your standards, but to deliver the best you can within the capabilities and expectation of the clients.

The last critical aspect of this phase is to get a good feel for their favoured mode of communication within this area. Do they have requirements documentation that they worked with in the past? Are they comfortable working with it? Does technical documentation create silos that exclude valuable stakeholders to the benefit of technical people .

This aspect is vital, because the quality of decisions they make and their perception of you as a trusted adviser will be directly proportionate to your skill in communicating the concepts clearly to them. Not only that, but your ability to get their attention at all let alone hold it long enough to achieve a useful level of consultation will depend on not confusing or alienating them with technical or complex instruments of communication or use of unfamiliar jargon.

A clearly communicated strategy and indicative plan

Once you have gathered your information it is time to make some preliminary decisions about how to take it forward. How to collect information, how to verify your information and how to communicate back to stakeholders ahead of defining the requirements.

Identifying these tasks, the stakeholder and support staff needed and the order and dependencies of the work, it is time to add this to a simple Gantt chart that defines high level activities, high level milestones and products to be delivered.

Understanding, agreement and total buy-in.

If you have done your homework and worked in a consultative manner you will be very confident before presenting the plan, that it will be understood and supported and will be accepted by your stakeholders.
You should present:

  1. An introductory explanation of the approach with reasoning
  2. A list of products you propose to produce backed by a simple concise explanation of how and why and samples.
  3. A Gantt describing the timeframes and milestones.
  4. A simple, high level risk assessment
  5. An indication of the time commitments required of key stakeholders.

At the end of this you should answer questions and ask for their full support backed by a sign-off of the plan.

Ed Taaffe is a senior Consultant in Business Improvement through technology and Hi-tech Product management.

Reblog this post [with Zemanta]

Two important rules of the learning organisation that you won’t study in an MBA yet they are ignored at great cost.

The learning cycles described by    Sekar Sethuraman CISSP,CISA,CISM,CGEIT,CIA,PMP is in my view an excellent starting point that would most likely be a significant achievement to formalise and encode into almost any organisation.

My personal experiences apart from academic interest are around :

  1. What we often call “lessons learned” i.e. the constant adjustment to circumstances and to the perceived results of our previous actions.
  2. The impact of swarm intelligence on our ability to learn and teach.

Lessons learned, maybe we should not be doing it at all?

In the area of Project Management, few organisations do anything at all about “Lessons learned”  though virtually all would express a regret about this. In truth , doing nothing formally is not doing nothing after all and hence informal learning continues. Is that better, or worse? Well it’s not a clear-cut answer.

In the past year I returned for a period to the area of managing risk in an uncertain and volatile environment with vague rules and little explicit information.  A classic example of this environment is day trading , or any kind of investment banking activity, bookmaking, military activity, espionage, football  and a long list of less glamorous situations.  Football is too tricky for this discussion  because  there is almost no conscious decision making involved.
The reason I choose to discuss these more obviously volatile environments is because they are more like real life only sped up enough to trigger human emotions and  for people to  learn from trends  and responses  that otherwise  might remain hidden. i.e. because you see the results of your actions soon enough to make an association you have a chance to reflect on the actions and the outcomes that in normal, snail’s pace life , would remain hidden from most people .negative false

Actions and results are two key elements of all learning. John  Boyds OODA loop is a wonderful example of this.  What Boyd recognised is the need for “ Sense making” and this is the key, because learning without doing this effectively is to learn things that are patently wrong. The equivalent Is to put arsenic in your coffee cup.
In a fast moving environment we see every day people who pushed  a button a and felt a shock up their leg. Like the pigeon learning to select the right beans, he stops pushing the button, because he assumes a relationship between the two. How quickly he makes this assumption and how rapidly he reacts is directly related to his self confidence and very quickly you can see the cookie crumble to a pile of dust.  Burned out traders are almost as common as arrogant and broke ex-traders.
The answer lies in the ability to ignore what you hoped or expected to see, question what you do see and only act on proven information while filing the rest away for another day. The ability to do this is much scarcer than you might think.

Lessons learned can be formally handled in a project management environment and these lessons ingrained in culture at which point the will become pervasive until they need to be superseded.

That leads us neatly to the other area:

Swarm Intelligence or Hive Mind has most of us  firmly in her grasp.

Swarm Intelligence , or Hive Mind as I prefer, or in simpler terms culture is a far more pervasive and more potentially damaging force than most observers realise , in particular when it comes to learning.  Ask Paddy Power Bookmaers.- Paddy Power Left ‘Red Faced’ After Early Payout on Greek Vote. They trusted the wisdom of crowds and learned an expensive lesson.

Surowiecki had a bestseller and started a wave of books that appeared to discover something new in old wisdom , only to be widely discredited later.

According to Jesse St Charles of University of Tennessee at Chattanoogai, there are specific rules that define a swarm or flock:
1. The rule of separation. Think of a flock of birds flying in close formation but never  make contact physically. There is an unwritten rule that keeps them a certain distance apart and that rule alone defines where they go. Watch the starlings over Rome about this time of year.

  1. Cohesion. The birds all use the same patterns of flight and movement and even squeak and defecate in unison.
  2. Alignment. They gauge their direction by where everyone is going and align themselves
    4. Type recognition. A flock of starlings will never allow a crow to join, nor will he try

Once these rules are in place, the bird has waived all control over his own brain and simply follows the pack.  In all group based creatures this can be seen and it mostly likely stems for the safety of being in a group and ideally close to the centre.

Parallel this to the Stanford Prison experiment when a group of well bred and highly intelligent students form the top 5% or so of Americans were given roles and group structure in two opposing groups; Prisoners and warders and left to enact this for the benefit of a study.  If you don’t know what happened, I urge you o read this: https://en.wikipedia.org/wiki/Stanford_prison_experiment.

I am not commenting about the scourge of starlings in European cities, or the capabilities of the human given the right opportunity, I am simply pointing out that once an individual has identified his or herself as belonging to a particular group a large part o his/her brain is surrendered to the perceived group intelligence and the power of the written or unwritten rules of that group prevent learning anything that is contradicts the group in any small way

Conclusion:

If you are engaged in deepening the learning of your organisation, or team bear in mind two extra rules:

  1. Unless you adjust the culture of your hive so that learning and changing is a source of social acceptance and security, all you efforts will come to nothing.
  2. Sense making, for adults in a business means something different than in teaching and learning. People’s emotions play a huge role in how they perceive the effects of their actions, what they learn from what they see and even what they see. If you want to create a learning organisation, you must teach and assist people to collect and observe the results of their actions in an objective way, make and execute good decisions at the right time and police their emotions against rash reactions, or unearned self-doubt.

 

Perhaps the short description of all this is leadership, the thing mankind craves for more than anything

 

The health warning attached to agile is nothing more, it’s not a reason to ignore agile thinking as a powerful tool.

As a manager who entered software engineering as the agile movement was gathering pace and returned to management in the systems world, I find it amusing when other disciplines jump on the bandwagon as it were. I also find it encouraging, but I would have serious concern if it were my business and here is why:

  1. Agile is not what journalists say it is

In much the same way that we probably misinterpret and ingratiate many movements,  Agile is credited with a lot of stuff that was never  though t of,  never intended and for the most part never happens.
A bunch of software engineers (several in several places) to decide how they could “take back control” of the software process in business.  They knew the power thy held in their hands, but could see more and more of it ebbing into middle and senior management determined to make them “blue collar”.  At the same time service businesses like IBM and Microsoft were looking for ways to reduce the paperwork and improve communication in the software transaction. The two things were married and produced a clever, but unruly child that requires careful monitoring and constant praise.  Agile was never a strategy, but a bun fight and it is far from over.

2.Fiiddling with KPIs is not changing anything.
 In my early days as a marketer I was taught that if you are not meeting KPIs, it is easier to fiddle with the KPI than change your behaviour. This is a fairly universal principal of business and nowhere is it more fitting that the world of software.” I want it and I want it now, but I am not sure what it is and couldn’t communicate it to you even if I did . In any case, what if it doesn’t work?”  I’ll claim you didn’t deliver and /or you didn’t do it right. That is the scenario driving software engineers to do away with specifications and contracts and make the product owner responsible. Unfortunately, what they didn’t spot until too late was that in doing this, they sold their souls and became blue collar again, but that’s another story.  Now they estimate in Tee-shirt-sizes and look upwards and to the left as they declare how many tee shirts they completed this month. If they don’t like the design they simply say it can’t be done or will cost too much. That is roughly where the tyre meets the road toady

3. A racehorse designed by a vet and an art student will make a very ugly camel at best

I was a product Manager for a few years, it is a natural progression for a marketer cum software engineer and I know some of the best around the globe. Some are redundant and others struggling. Why? Who needs professional when you have the seat of your pants? I am not suggesting that Agile teams with a nominated Product Owner can’t or don’t produce good stuff, simply that; successful products need a market sized opportunity, or it matters not how good they are, products need to meet an unmet need and to do it so that they are a more attractive proposition that their competitors and so forth.
What I am getting at is this; iPhones are successful because they were driven forward by a brilliant visionary who had the courage to show us what we are missing and make us love iPhones. Bill Gates did this for the PC. In an agile world neither of these products would have happened.
The iPhone would have been an ugly mishmash of half-finished features rushed out to an imaginary deadline and reflecting the current tastes of the Lead developer and the Product Owner, usually in that order.  I suspect the PC would have been a typewriter with a glass screen and a coffee holder.

  1. Lying in the water and thrashing your arms about is not swimming.
    Anybody who seriously wants to make a change for the better needs to start not with the technology, but with goals, develop strategies and work their way down to the details of how.  If that happens to be agile, wonderful, there is then a powerful chance that you will make it work.
  2. Beware the evangelists, many are trainee terrorists

Understanding the shortfalls of anything is the first sign of expertise and even love. Don’t fall for the preachers they have alternative motives. Somethings and some teams lend themselves well to agile, but not everything and worst of all, when a good agile team has been doing the same thing for a year or so, they are no longer agile, they are simply where they were before with slightly different rules and rituals.

  1. Agile is a state of mind not a book or a methodology and must be championed and driven form the top down by people who know what they are doing and why they are doing it.