Image result for bot wikipedia

I wont be long-winded about this, I’ll discuss it via email with anyone who is interested, but I’ll break with my usual mode and come straight to the point.
A great many people who know little at all of machine learning  and even less about people and many more who are simply  oblivious to the potential consequences of their words are talking about the miraculous things we can expect from Machine learning.

What is ML in a nutshell?
Academics break ML into two modes:  Supervised and Unsupervised.
In the case of the former we give the machine a large corpora of content and ask it to decide what will happen next, or to find other similar instances. A translation service for example  begins this way and learns after a while to translate without help.
In the latter case, we give it a body of content and ask what it thinks of that.. Google search is an example of this approach and it simply makes sense of what it finds.

Often we give it a few hints like “Classify this for me and establish links” as in Google search. This would be a “Classification problem”. We might on the other hand ask it to read the racing papers and decide who will win the four o’clock today. This would be a “Regression problem” because we are asking it to look at the past and predict the future. Yes all of this is highly condensed as promised, if you are an expert you don’t need my explanations.
Understanding what the customer will want next year, predicting the weather, finding Oil under the sea, predicting tumours, the challenges are endless and the rewards enormous.

What is the loop of self-destruction?
The loop happens when, thanks to social media, a good, but no a sole example, the machine begins to make judgements that influence the data and then discover exactly what it predicted.

As with humans this will give it the machine equivalent of a big head and possibly some citations and will lead to even greater confidence and fewer checks and before anybody spots it, it I all too late.
If any Movie producers out there are stuck for an idea, I am available to help with the plot. Here is a simple example we are all aware of:
Joe Gel, and Josephine Lotion our dear friends, represents an enormous body of intelligent and informed people who spend most of their waking hours  checking back with their phones for reassurance. Joe searches Google for Tom Raspberry, his favourite politician and receives a huge list of pages. The ML in google notes his interests and begins sending him dozens of articles about Tom Raspberry, what he says and does and what people say about him. Unwittingly Our pal Joe has become astonished by the fact the whole world seems obsessed with Tom R and realises subconsciously how important to is be aware of Tom R. He begins to tweet and have the odd Facebook conversation about something he read. Immediately the ML in Facebook and the one in Twitter hone in his apparent obsession with Tom R and all begin to bombard him with content and introduce him to thousands of people with the same problem. Poor Joe.

Now our Machine does a Recce to see what are people talking about and it discovers that millions are talking and reading about Tom raspberry and concludes that tis is the way to keep the customer happy so it ups its game and heightens the emphasis. It also confidently announces that Tom R will undoubtedly be unstoppable in the forthcoming election.

Joe and Josephine realise the importance of not standing in the way of a social crowd and are not about t be shunned and subconsciously they begin to take more interest in the positive stories about Tom which now triggers the Machine to filter their feeds and search results and friend recommendations etc more toward the positive . You don’t need me to finish the plot. There is only one way this is going. Imagine if the secret services relied on this kind of information to brief their bosses. But they do, don’t they.

You may well think, as I do , that despite the  shear “wrongness” of rigging democracy, whether by design or accident, it matters little who is elected anyhow. In that case imagine the same scenario when the machine turns its hand to guiding change in a government department or a large business , or guiding product development or even finding the cure for cancer. If you would like to see many better examples with a strong scientific analysis, check out Weapons of Math destruction.

One wonderfully simple yet highly destructive outcome of ML that I have seen up close is the  call centre  automated system that recognises your telephone number, calculates your value as a customer and decides if you will be answered, how long you will have to wait and whether you get to speak to somebody skillful.  Just to update my card details for a £20 a month hosting service, I had 11 hours of my time wasted, had my service disrupted and was threatened by a bot with £150 fine to put the service back on.
I hate to disappoint you, but if you have ever had an IM conversation with a patient lady on the support portal “That was no lady” nor was it my wife, that was a distant cousin of Cortana.
If she did not know the answer, or more likely the question, you were never going to be served.
If you are wondering what might happen to your pension, your job and your home if these guys get involved in stock trading, well take a look here  According to a 2014 report, sixty to seventy per cent of price changes are driven not by new information from the real world but by “self-generated activities”.

It’s not all negative by any means. I actually do use ML to predict the winners of tomorrows racing with a consistent level of profit. When I get it wrong, usually after a late night of programming with insufficient testing, my winnings disappear very quickly into someone else’s pocket and I sit up and take notice.
I sincerely hope that someone starts sitting up and taking notice soon  of the impact of poorly programmed Bots that are already beginning to increase risk for the most powerful nations on earth.

Why you need to pay attention to customer experience

Why you need to pay attention to customer experience

Next        What to watch out for when researching and testing journeys


The chicken and egg question always fascinated me. When it comes to business models I find the same conundrum with customers and profits. Michael Porter once said that the purpose of a business is “to create value for customers”. Although we all assume it was inferred in there, he never bothered to mention profits.
The reality every business faces however it that creating value comes first and monetization follows.

1. Compare the debacle of the great Thatcherite privatisations to the often maligned success story of dot com.
In the UK we have a raft of privatised utilities who still have not “got it”, they still think in terms of Oligopoly, force, bullying, price rigging. They think and act like tax collectors. The total innovation from all of them over two decades could be written on the back of a credit card along with a full list of their happy loyal customers.
Amazon, ebay, Paypal, Google and many more have on the other hand built world beating businesses on the back of profitless customer satisfaction and only now are monetising these business models. They operate at P/Es up to 500 and have no shortage of investors.

The message is clear, the customer is king and until you can demonstrate value to them you don’t have a business model.
“ Sooner or later regardless how much cash you have stashed away, you will learn to create value for customers or fail.” We even see this law apply itself to dictatorships.

2. What is customer value and how can you create it?
The biggest possible blunder any business can make is to quantify customer value in terms of product features. I cringe when I see these neat spreadsheets listing product x competitior1, competitor2 etc and how well they score on each (in the marketing trainees opinion).
Customers buy an experience, even hard nosed corporate customers. That begins with the interaction with “People” in the supplier side, or “friendly” and human like ecommerce site and carries right through to anticipating delivery, opening the package, using it for the first time, bragging to friends, interacting with support and many more. Many of these are remarkably powerful influencers and even though supported at times by product features, most of the time they are a separate source of value, or indeed antagonism.
Next we return to the chicken and the egg.

3. Does customer experience exist without customer value and who foots the bill?
The problem here is thus: If you ask the customer how much extra they would pay for their phone to float up out of the box on a mechanism with a Jingle playing, be fully charged, sense the old phone and offer to copy the contacts and messages etc in a sweet voice, accept a voice answer. The customer might well offer a price that made this simply not feasible. However, when that same customer experiences it once, the likelihood is that she won’t want to be without it and when she hears her friends talking glowingly about it, it becomes a must-have at almost any price. Soon it is talked about and develops a cult status and then we have a brand value to take into account. That’s a whole new ball game.
I’m not suggesting we deliver high quality customer experience at all costs, I’m simply saying that you must understand the true value and what people do is far more revealing than what they say.

The point I’m making here is that sometimes you have to take a small hit to let customers realise what they value before it becomes indispensible to them. Henry Ford would have built a more comfortable horse carriage if he had asked the customer what to do. The distinction in marketing terms is between “True value” (product features) and perceived value ( How the customer sees it)
“There’s more than one way to ask the customer and more than one way to interpret the answer, if you listen with an open mind, sometimes you will be surprised pleasantly.”

4. We can’t have a discussion on customer experience without discussing the brand.
There are many definitions out there of a brand and I’ll leave that to those with little to do, for me the important point is that expectation which a customer carries as a result of the brand. That is what drives her through our door or to our site.
Let’s not gloss over the word “expectation”. Whether you are playing poker, editing movies, or doing magic tricks for your children, you will quickly realise that everyone, and that includes market researchers, sees what they expect to see, hears what they expect to hear and feels what they expect to feel. Most people could probably say yes to that statement glibly, but very few would appreciate the profound power of it.
In a previous blog I described the experiment when scientists used MRI brain scans to identify the increased satisfaction enjoyed by a coke drinker who had poured it from a branded can into a branded glass over that of another drinking it from a plain glass, all in stark contrast to the memorable testimonials of thousands who preferred Pepsi over coke when offered both in unmarked glasses and could only focus on taste.
Expectation is created in many ways, but primarily by the chatter of others and the perceived opinion of peers. That is the territory of Brand managers, Marketing people and Social Media experts.

The key Point here is that creating an expectation associated with your product is the most powerful way to create value for your customers and often the cheapest and mot certain way also.
“Innovation is critical, but don’t confine it to the engineers and inventors, the ultimate playing field is inside the customer’s head”

5. Customer Lifetime Value is not an old, or boring idea it has never been more relevant, or more critical.
One of the first things we tend to look at with a new product is a breakdown of the cost of product, cost of selling it and gross margin. The cost of selling a product usually surprises newcomers to the field.
In competitive markets with a lot of equal offerings a small price advantage can drive large sales increases so price is critical and it is driven primarily by cost. i.e you cant reduce price below a level that is profitable. In most markets price is sensitive and if it isn’t then investors are sensitive to margins, earnings and dividends. In all cases no business can indefinitely carry unnecessary cost and in a competitive market. Sooner or later the competition will do it and steal a march. Of course there are many pricing strategies and this is not a discussion on price
The money you spend on marketing and selling your product is critical to the success of your product, yet it comes under less scrutiny than any other budget apart from the CEOs expense account.
Let’s say you sell 1m units of a product at £100 retail. Your production cost is 20 and your marketing/selling costs are £30 operating costs are £40 and net profit is £10
That’s 100m t/o, 30m spent on selling 40m operating profit and 10m net profit

Suppose you convinced 1% of your customers to recommend the product to a friend
Now your t/o is 101m selling and operating costs stay the same and net profit is 11m.
That’s a ten percent increase in earnings- a darling of the markets if you can repeat it.

Let’s say that you have a Million customers, every customer has to be replaced after 4 years and they pay £1000 a year for your product. That’s t/o of £1b
To maintain that t/o you have find 250,000 new customers at a cost of £1000 each
That’s £250m a year in marketing/selling costs.
Now suppose you are so nice to these customers that they stay for 5 years instead of 4

Now your costs are reduced to £200m a saving of £50m
if your net profits were, for arguments sake, £100m on £1b now they would be increased to 150m a 50% increase in earnings. What would that do for your stock?

These are simplified figures used to demonstrate a point, so lets not get into a investment analysis discussion. The message is clear:
“Treating your customers well enough to retain them a little longer can deliver huge dividends while enlisting them onto your salesforce is the next killer app and make no mistake about it.”
That means paying attention to the user journey long after the “order to pay “ stream has completed.

Using information to support the entire customer journey


The customer journey  begins when she becomes aware of your existence and never ends, though it is at its most fruitful when she places an order and subsequent orders.

Previously we discussed the folly of looking at “Last Click” as the beginning of this purchase journey, the reality is that it began some time in the past when she stumbled on your business either through a friends, in a blog, or via a search or advertisement. In reality every purchase is generally precluded to a greater or lesser degree by a process of discovery, comparison, discussions, eavesdropping, information gathering, price comparison and leading finally to an order being placed.

Whether and when that order is placed will be contingent and whether she found sufficient information to support a decision, what information she found, what advice she got, what her peer group are doing whether she is in front of her favoured device for ordering, whether she has the cash available yet  and a probably many more issues. For example it matters little that she made her mind up on day one, if she wont have the cash until her salary clears in three weeks. It wont matter how good a deal you offer her if all her friends are advising against your product and so forth.

It is never possible to know all of these inputs and be aware of the state of play, but at least being aware of what it takes to sell an item is very important in determining what steps you take to improve that user journey in a way that is profitable. Below are some examples of information you may collect and use to improve the user experience and deliver revenue upside. This will of course vary from one situation to another.

  1. It begins with being found. You must know where the hungry crowd are going and make sure your food stand is right in their path. Being there when they are hungry is just as important as part of serving your customer as it is to your revenue targets. How to do this is a little off topic for today.
  2. Making sure that the gossip they hear and the advice they receive is unlikely to be negative is critically important. The means of promoting positive vibes in social media are well documented and to a lesser degree we know of business that can help deal with negative comment when it occurs.
  3.  Making the right first impression is critical. The expectation you set is a key metric against which your performance  will be measured.
  4. Becoming memorable and easy to find again is now a key goal. Any way of beginning a relationship that allows you to communicate further is great, getting the customer to download something that will act as a reminder for them is also very valuable. E.G. a useful app for their phone.
  5. Storing a cookie that helps you track their consequent visits and actions will make it much easer to judge their likely needs at any time.
  6. Running multivariate tests allows you to not just find out which inputs drive the most orders, which combinations pf inputs are most successful. This drives very accurate views of customer behavior and allows you to optimise everything.
  7. Once you understand the average customer journey you can provide content and services that help the customer at key junctures while updating your understanding of where they are at with their buying process.
  8. Understanding a little more about the type of product they shortlisted and what they rejected may also help you to understand their needs and preferences.
  9. Knowing the times of week, day, month, or year when they are most likely to make a purchase may help you in selecting an irresistible offer.
  10. Knowing which devices they use for purchase may help you to time your offer better

Here is a simplified example.


My company sells widgets to consumers and the customers come form all walks of life. They purchase from the ecommerce channel. There is a lot of competition online  and customers tend to switch suppliers regularly as offers change. Price is important, but its not the whole picture.
We use advertising via keywords to drive customers to landing pages where they find information on exactly what they searched for. They can also follow links to the main site where they can  learn more

Mrs Jones

Our best customer is Mrs Jones. She uses search engines a lot but not just for finding products but also searching the news and gossip sites. She talks to a lot of people on forums and uses them extensively for advice before purchasing. Mrs Jones enjoys the purchasing process so she does not mind seeing plenty of offers, but she is rarely swayed from her initial choice. Often she decides what she wants and then goes looking for proof that she is right.

After she first selects a product, we know she is giving it strong consideration because she then visits our comparison charts and follows links to some of our competitors.

Our strategy

We think she trusts us because we are not hiding from our competitors and we give her honest comparison. We also help her out with the evidence she is looking for.

We have her email in an opted-in list and we know when to send her a little extra information if she goes quiet. We have a clickstream that identifies a quest (product she searched for) and the different types of investigation she did so far, so we can guess where she is in the purchasing journey.

Sometimes, when she goes quiet, it means she has bought elsewhere, but often she is just waiting to get paid or some other reason, so we keep in touch, but we are careful not to upset her. We rely on her to visit again and to recommend us. On average she makes five visit before purchase.

She is very influenced by social media so we spend a lot of effort on maintaining a good reputation.

Our content is tagged to match the different stages in the quest such as price comparison, features comparison, evidence gathering etc. These tags help us to develop the clickstream that places her on a purchase journey. Because she has purchased before, she is able to purchase with a single click.


She visits an exhibition  where she sees our stand and meets a polite person who gives her a free pen.

She searches google for comments and finds a positive attitude towards us and our products

First visit

She spends some time reading the general information, downloads a calculator tool and leaves via our comparator to go to a competitor site.


Still collecting information

The following week our advertising network presents her a little reminder advert while she is on a competitor site and she returns to ours. This tells us she is still actively and seriously searching and we are high on her list

Decided now

She returns at the weekend and spends some time on the cost of ownership calculator using her tablet.

We know that she likes to purchase using her PC and she might still feel like this. We also believe that price is the only thing now influencing her.
We email her a very hot offer that needs a response before Tuesday and we give her a special hotline for telephone advice promising no switchboard and 12 hours a day of service.

Finally an order

She immediately calls our sales staff explaining a slight issue she has yet not resolved. The sales staff are able to put her mind at rest and she places a n order there and then. It is completed in seconds and she has an email confirmation

Delivery and service continues

Delivery occurs on Wednesday and our service staff phone her unexpectedly to talk her through getting started seeing that she expressed concerns. She expresses her delight with the service.


On  Thursday our sales staff call to make sure she is OK and ask her if she would be happy to recommend us on a social network, she agrees readily and goes public with her satisfaction. This has three important implications:
1. We are committed to keeping Mrs Jones happy.
2. Mrs Jones has publicly praised us and it would be extra hard for her to ever contradict this.  She will make allowances if ever called on to do so.
3. Others who see her comments will be encouraged to do business with us

We have not just sold a product, we have bought a supporter and gained valuable advertising of the best kind. If we worked out the Cost of Goods Sold on customers like Mrs Jones, it would be in low or even negative figures.



Are you wondering if a recommendation engine is the next big purchase for you?

Some retailers in particular have made a great deal of extra profit through offering clever recommendations to their customers so much so that the notion of the recommendation engine came of age in the last three or our years spurring a wave of new offerings form software vendors and plenty of noise in the blogosphere. Before you go running off to buy one or even nurturing plans t build one, you should give due consideration to exactly what you expect it to do for you and from there you will be better equipped to decide if it is the right thing and finally what type of engine you need and how to acquire or build the right thing.

Ins and outs of recommendations and personalization

Perhaps the oldest and best known recommendation engine is the one used by This is sometimes claimed to be responsible for 35% of sales. If that’s the case then it’s not hard to see why there is a strong interest from the ecommerce community.  Every customer who selects s product then receives a number of recommendations to other products she may like . Since the customer s usually there to browse, she can live with the annoyance of being sold to and is statistically reasonably likely to find the suggested product worth looking at even if it were driven by a schoolboy randomizer function. A portion of that 35% would undoubtedly be equally achieved by a placebo tool and I would strongly recommend some experimentation before spending large sums.

The down side of recommendation engines can also be potentially substantial.  When Microsoft first experimented with personalization on their website they were a leader in innovation on the internet. I was a regular user at the time  and I remember being frustrated by my inability to find something that a colleague was recommending me to. The fact was that when I visited, my cookie told them I was of type A and these widgets were only of interest to type B. I t  took them about a year to realize their mistake and loosen off the personalization  rules.

In the past year I have had similar experiences with Google search. It is now so focused on commerce that it sends me results it believes I want to see rather than a list of cold hard facts that I want and need.  These are only the few occasions when I became aware of the filters. How much of my online activity is tailored  to a weird misconception of me created by a mad algorithm. Even I don’t have a great idea of what I’ll like tomorrow and that’s how I like it.
Do you want to risk excluding products from your customers because Mr customer looked at something last year that suggests he would not be interested in X. Imagine buying a Vegan book for your best friend and never again being offered a meat menu. Ugh!


Types of recommendation engine and what they can do

The Amazon, or Netflix type of engine with which we are all familiar is sometimes referred to as content based because it uses knowledge of your stock database (i.e. content) to decide what Ms H might like and make a recommendation.

The simple version is that:

Product (a) has been tagged to be about [1,2 and 3] Product (b) has been tagged to about [3,4 and 5] You looked at products A and B from a long list therefore there is a strong likelihood you will like other products about [3]  but possibly also about [1.2.4 and 5].
Of course the algorithms are somewhat more complex, but hopefully  you get the gist.

It can also mine the records of previous customers and genuinely say, people who selected product A also Bought product Y and Z. This will have a reasonable potential to be useful also to the customer.

Provided you are searching with intent, this type of implied logic can quickly build a useful picture of what to recommend. If you are just browsing then this interference could be just plain annoying.
The most reliable statistic however when it comes to shopping is that the more things a customer sees the more she is likely to spend, so even the mistakes are not that serious. Remember also that what works for books or movies may not work so well for other products or services.

The key to this type of engine is that it needs little knowledge of you the customer, it takes as inputs knowledge of the content and of what you searched for and how you reacted to the search results.
That is good behaviour, it has no preconceptions and it takes you at face value based on what you do.

Other engines receiving a lot of attention now are referred to as  collaborative filtering  engines.

These engines use vast amounts of data collected in various ways to form opinions about you and use those opinions to show what they think you will like. Some of the data in uses is controversial third party cookie data that is collected without your explicit permission.

Every action by a customer is a piece of information that potentially says something about that customer and the combination of these actions says a little more.

A simple method id to mine click streams and create segments based on identical click streams. Suppose that a high proportion of customers form segment B purchase product Y and your clickstream data puts you in segment B then guess which recommendation my engine will make.

Other information that may be collected and used against you is your interaction in social media. Who you are connected to says the type of people you like and a profile created from the commonest likes expressed by members of this group can be applied by default to you the moment you are seen to me a member of the group, any accuracy this profile has will then improve as a result of your on-going interactions with the engine via its recommendations plus any likes or other social sharing you, or your associates may express.

What is different about this method is that very little needs to be known about the content or stock in order to make predictions about the customers interests, it all comes from social an other interactions.

In theory at least, such an engine can recommend the white box to you with confidence, not knowing what it contains, simply because your colleagues whose tastes most resemble yours all bought the white box.

In reality a combination of the two methods works better because it uses some knowledge of the customer alongside some knowledge of the content to make a more intelligent match with a better likelihood of success. For the second method you do of course need to spend a considerable time collecting the useful data before you can make a start on creating recommendations, though many commentators grossly overestimate just how much data is required. Analysing 10m transactions (data points) wont necessarily give you a significantly better result than analysing 100,000 and certainly not sufficiently better to cover the extra cost.
A further problem with the big data approach is that data is time bound and therefore data form last year may or may not be valid this year. It may as easily be detracting form the result as adding to it.
A large amount of user data may well be relevant to a surprisingly small segment of heavy users whose needs are very different from most of the people you want to offer recommendations to.

People learn and change and the world changes. In 2014  attitudes are very different indeed to what they were on 2004 or even 2010. Most people I know have changed substantially in the past five years and a great deal of surfing is without doubt serendipitous. Google’s Z Moment of Truth is an interesting approach to discussing this subject.

Currently there are armies of start ups offering to find you the perfect restaurant, or movie or whatever and to my experience they are a long way from way form delivering on the promise even if I wanted to be told what to do.

My personal experimentation with Siri ended in abrupt divorce after just a few days and google’s sad attempts at knowing me have gone via the same route. The owner of  Ness claims as his mission to: “become that trusted source for people to find out the next thing they’ll like.”
Isn’t that what advertisers have been doing since the first  TV invaded the first living room?

There is no doubt that at a certain level for certain industries, recommendation engines can deliver substantial extra revenue and that is always a vote in favour, but the right one for the job is important.

For others a revenue upside sufficient to justify the cost can be achieved without causing damage to the user experience, but for some industries, some or maybe most recommendation engines will struggle to improve revenue an may well have serious detrimental effect of your user experience and therefore your brand


How an uplift in CXi scores can deliver remarkable benefits for your business

A recent Forrester report looked at companies that had recently improved their Customer experience rating from negative to positive looking at whether their customers were likely to consider another purchase, to switch suppliers, or to recommend their current supplier to friends, The findings demonstrate in no uncertain terms that there is significant achievable upside to be had from improving the customer experience.

The trend of revenue uplift for any company that has improved its customer experience is not only strong, but still rising demonstrating that customers place a lot of value on this and are prepared to reward a supplier that makes an effort,
The reduction in customer churn continues to be significant, but this trend is falling slightly.  If this slight fall is statistically significant then it may well be that once an acceptable level of service is reached the value of further improvement is greatly reduced. After all, for many suppliers, all we want is that they actually supply and they remain invisible.

Chatter is rising rapidly. Both recommendations and the opposite are on the increase.

The impact on recommendations is very significant and growing when a supplier improves customer experience, but conversely the impact of falling service levels is equally damaging the supplier as a result of negative chatter

Increasing CXi scores from below average to above average had the following financial benefits. As you can see, the benefit is greater in some industries than others, but is very significant even at lower levels.


Industry Customers Reduced churn Further purchases Word
of mouth recommendations
Airlines 80m $807m $555m $56m
Credit card providers 61m $427m $368m $30m
Hotels 44m $380m $320m $29m
Retailers 67m $273m $237m $62m
Banks 15m $73m $81m $7m
TV companies 17m $92m $47m
Consumer electronics 10m $40m $43m $20m

Source: North American Technographics® Customer Experience Online Survey, Q4 2013 (US)


For many industries there is a ceiling whereby customer experience improvements will no longer result in revenue or profit upside. The obvious ones are products we buy very much on Price point.In these cases  we are happy enough with what we are getting and not willing to pay any extra under any circumstance. For example, to return to cheap flights. My favourite airline leaves on time and arrives on time at least as well as any other and charges me slightly less on average for an equivalent service. If this airline is able to reduce waste somewhere, then perhaps they can afford to improve service, that’s all well and good, but if they start spending on it, then sooner, or later prices will go up and I’ll go elsewhere.
The most thing these companies can improve at no cost is the attitudes of staff. A smile costs nothing it is a way of thinking and a culture, nothing more.


Imagine if you are running a business where you compete with a number of others in a controlled industry and you all buy the same thing in the same place at the same cost and sell it in the same market at more or less the same price. There is nothing to compete on. The only chance you would have to differentiate would be in how you treat your customers when they have a problem and how you manage customer acquisition and customer churn.

Imagine if you never spent another penny on advertising or sales commissions, kept your customers for their lifetime and had them all recommending you to friends. Now that would be a hell of a proposition.



Using personalisation cleverly to grow your customer relationship and keep your sanity

In my next  instalment I plan to look a little more closely at the broader context of recommendation engines and how they might be used but foe now I will simply point out that they are broken into two broad types Content Based and  Collaboration Based. The first refers to product knowledge and the second to behaviour of customers. Today I am only discussing the latter type

There’s two fundamental types of customer information used in recommendation engines and customisation strategies: 1. Real time knowledge of customer in context and 2. Historical knowledge of people with context.

I like to refer to them as 1. Personal and 2. Collective.

The first problem you run into when you start an ecommerce store is that you don’t have any historical data to start from. Making decisions about “people” is another way of describing prejudice. Of course all prejudice is not bad, we help old ladies across the road and we are kind to children. But when somebody hears my accent and offers me a particular brand, I am irritated.

What I find especially infuriating is when I simply cant find what I want on Google because it thinks it knows what I really want based on the fact that I looked for it twice last week.

The point I am making is that people are individuals, they change their minds, they like variety, they find what they are looking for and loose interest, in fact they really do not want to see what they just bought on offer at a better price.

The bottom line here is that relationships have always been difficult even when you are at the next desk or sharing a home, let alone seeing a cookie form the same machine create different click streams and view different content.

Personalisation and making recommendations based on algorithms can be high risk and must be done with care and forethought.



Context is a critical factor in storing and classifying interactions for recommendations and personalisation.  When I go in our local store the shopkeeper who knows me very well takes into account the day of the week, time of day, look on my face, which shelves I’m looking on and what he remembers about my usual purchases and he then decides whether to let me walk out or to ask if he can help. He rarely gets it wrong.

If he did get it wrong very many times, I would be very tempted to walk a few yards extra and shop anonymously. Why should I have to explain myself?


Imagine if he wiped lots of stuff off the shelves on the basis that I won’t be interested in them today, but filled the shelves with what I bought on this day last week, or on my last visit.  Not very good.

Imagine if I run a social network, I know a great deal about where you are going, what you like etc. If I provide your phone network, I know who you call, and receive calls form, when you use it and how long for, when you never use it and when it is off and where you are right now. Maybe I see your calendar and your emails too? Not to mention half a dozen spook outfits eavesdropping.

With this kind of information I could be very invasive, so it is critical to only use this type of data with the customer’s consent and to put that customer firmly in control in case they should change their minds. The way to achieve this is to offer them value in return for their data.


In a typical e-commerce implementation you will have access to the following types of information:

Purchases,  wish list adds, cart adds, views, shares, favourites, downloads, shares, watches, likes, follows, where they were referred from and you may know about the search terms they used to get to your site.

You may also know about the sites these customers visited after they left your site when they have been active, their spending patterns, whether they buy for children, their gender, devices used, devices ordered from, devices emails were read on, where they have browsed from geographically, and a whole lot more. Once registered you could collect data on their social media activity.

If you are an OmniChannel retailer, you will have more data in your CRM, your ERP, and other places

data sources

This information exists in many places: In your customer database, in the web server log files, out there on social media sites, on Google, in an advertising network you work with,


In a typical solution, the information might be classified in three ways: 1. Customer , 2. info and context 3. Offer.

e.g. you create facts such as:

Customer, info,  (info context), Offer.

John Brown, likes(3) (June 4 14, Saturday, mobile ), gizmo product

John brown, price search(June 6 14, Monday, PC), gizmo product


Each piece of information is about a specific offering or product and it relates, of course, to an individual customer. In addition to that basic information, it is invaluable to collect context.

Understanding that Mr Brown never orders on his mobile, but does look at info helps you decide what to present to him. Maybe the solution is to offer him a convenient app that makes ordering easier. Certainly if 80% of his browsing was on mobile devices and 100% of ordering was on a PC,, this would be a strong indication

The way you might arrive at this type of conclusion would be to query these stores of facts, summarise what we know and make inferences.

This is a lot like data warehouses where you store vast flat tables, analyse them once then stores summaries in a more accessible system.


e.g. John Brown, likes(3), gizmo product  That weighted his interest at 3

Two days later he returned infer a higher interest, say 4

He asked for pricing info, this puts him at 8 or thereabouts

So we can summarise it as ; John Brown likes (June 6 14),gizmo product (8)

We know that he orders via PC, that was not a barrier.

We know that seeking price info is a sign of desire and conviction so why did he not order?

We might assume the price was too high.  Perhaps we are all too aware that we are not the cheapest in this market and we rely on positioning to overcome price objections.

We may decide to send a once off special offer on Thursday morning, so he cant resist because he frequently browses then so he may have free time on Thursdays.

That is an example of generating a “Next action” on the basis of knowledge and intelligent inference.

Of course it could be that he planned to order on Friday at the full price, that is a price we either do or do not decide to pay for John Brown’s business. Eventually we will know enough to be able to predict this accurately.

If we knew that he was a highly influential blogger with a large following, I wonder would this influence our decision?.


Contrary to what is written by cloud vendors and Hadoop junkies, this kind of data and that kind of information is rarely dependent on petabytes of data spread across numerous clusters. It could be done with a basic RDBMS, though it would probably be easier with an RDF store downloaded for free.

Don’t get me wrong, there would still be some development to do and it wouldn’t work till a bit of data had built up, but you don’t have to be Google or Yahoo to benefit from this technique.


An interesting side effect of this approach is that the longer the customer keeps visiting the bigger investment you have in keeping her, because you know exactly how to twist her arm. Her customer lifetime value is infinitely greater even at a discount. Perhaps eventually, these relations will be reflected in the price each individual customer is asked to pay.  There’s a thought to conjure with.

What if John turns up at your Branch in Oxford street with his decision made and ready to order? What if he orders online but wants to collect from your store in Paris next week?  What if there’s an issue after delivery and he wants to get help form your store in Las Vegas while on his travels?

What if the cart tells him “sorry, we’re out of stock” and then he walks to the shops and sees one in your local store?

I know just how difficult these problems can be and how expensive it is to overcome them. It requires not just a technology rebirth, but a culture change like no other.

Lernaean Hydra, your time is up.

Simple, but powerful tools to truly build a relationship with the customer, some obvious gaping opportunities to cut costs and some basic principals of architecture that even the bin man could understand in one lesson and yet are ignored by 99 percent of technical architects. If you are serious about competing in business, read this.


Lernaean Hydra (slain by Hercules ) was an ancient serpent-like beast, with reptilian traits (as its name evinces), that possessed many heads — the poets mention more heads than the vase-painters could paint, and for each head cut off it grew two more
Next time you are trying to stay awake through the IVR while being invited to search their website instead of calling the machine, or chatting to a script via “live” text, think of Lernaean Hydra.

Ever ordered something online from Acme Gadgets PLC and then tried taking it into their Acme store just down the road because it didn’t work or you needed help? Oh no you don’t, this is the wrong head, you need to call the IVR and report it to yet another head, then wait till yet another head sends out the courier to take it away and . . . familiar? I bet most readers could immediately think of several of their current suppliers who behave just like that. Could this be your business?
I recently completed some work for a well known utility and when their customer moved house she first received a “Sorry you are leaving us …” message, she then naturally panicked and spent an hour or two with the IVR to eventually speak to someone who passed her eventually to someone else to be told that all is well and she will have a supply at her new house after all. Maybe!. Experience suggests otherwise. Another week and she received the “Welcome as a new customer” letter. You may find this hard to believe, but during all this time, a Programme was running internally to reduce the size of the call centre by preventing people from calling it (Logical). The most successful trick was hiding the phone number up to 9 clicks deep.
Since then I encountered the same experience when I myself changed mobile packages with the same mobile network. It’s very easy as a business to drift into this situation, mainly because you can get away with it, i.e. the competition are just as bad. It doesn’t help that the systems you have available to run your business don’t talk to each other well and few architects have the knowledge, or the will to fix this problem even if somebody somewhere were to express the desire.

Now set that aside for a minute. Actions are pretty bad when they become mixed-up and after all everyone gets it wrong at times, but what about when actions are designed to be obtuse. Imagine a company that has set aside many millions of pounds for an advertising and marketing campaign whose goal is to “ convince the customer the we are their best friend and totally committed to giving them a great service”. Now imagine if the self same CEO told you he is “investing” in another programme to allow no more one-to-one contact with customers apart from the bereavement team. This would mean that instead of a wait between 35 and 85 minutes for any kind of assistance there would be no assistance for the majority of queries and issues other than searching an FAQ. Would that situation leave you with one or two questions?
I have just spoken to such a person, so this is not fiction, in fact it is the current trend in many industries. Imagine if a business like this were to ban Social Media engagement because they were worried that the customers would get talking about the lack of service. You guessed it. No doubt most readers have their own list of experiences with corporate schizophrenia.

Now perhaps you thought Lernaean Hydra with her many heads is a disturbing picture of a business that likes to believe it has a personality and a brand and is customer friendly, but the truth is far, far worse, because in reality our monsters not only have many heads but at least one derrière per head and out of these derrieres pours endless mountains of poo. It’s not pleasant and it doesn’t even grow the lawn, but some clever people are determined to gather all of it and analyse it to look for traces of information that might tell them more about their businesses. Well let me tell you up front that for the most part they will simply turn a lot of small poos into a “big poo”. If they ever did find the answers, here’s what it would say: “Your customers despise you, but they tolerate you because they know that the alternative is much the same.”

You buy something for 15 pence and sell it for a pound and only make 7 pence profit, the rest you waste on nonsense like this and your shareholders are also in despair. Your employees are autonomons who live out the bizarre role you gave them because they are very adaptable and resigned to the inevitable.
Any executive who wants to know about her business need only walk around for a few days without the mask and talk to the people on the job and dare I say it, talk to customers. These know all these answers, but nobody asks them.

For qualitative issues we used to use samples as big as 300 when I worked in research, but we all knew that equally useful results could be had with 20 or 30 and there are many who say a great deal fewer will give us reliable answers. In fact there are empirical studies to prove it. Do we need “big data”, no we certainly don’t. If there is anything we do need it is “small data”, or better still “smart data”.
What I am saying here is not that we don’t need the information, but simply that we don’t need a great stinking pile of data in order to get that information, nor do we need the cost associated with it. Maybe the health service could find cures by analysing past results etc, but that is something different from the little dashboards most of our clients and are capable of dealing with and imagining with when they are shelling out for humungous data servers.
Microsoft recently released a convincing paper demonstrating that few companies on earth have more data than can be analysed and presented on a bog standard database server. I agree.
Don’t get me wrong there is a role for big data, but not the one most people are determined to tackle with it.

When you exist in an environment where inexplicable behaviour can go un-challenged, the next step is for this behaviour to find its way into the planning process and even strategy, if not by design, then at least through tolerance. What this says about modern business is truly frightening and what it says about customers and their power to move markets is in many ways even more frightening. I happen to believe as did Milton Friedman, that only the power of truly free markets can guarantee individual freedoms long term, though unlike Friedman, I accept that sometimes freedom needs some minimal regulation. We could talk about free markets, or about corporate strategy, but that is for a different forum. I am just concerned with cutting through the nonsense and pointing out the glaringly obvious as a starting point on the road back to business sanity.

In the next instalment, I will be talking about good management practice leading to sensible, though sometimes revolutionary use of technology to support strategies that can drive any business into a clear lead in any sector you wish to name.
I believe we have a duty in business to use technology intelligently to serve our customers and drive returns to employees and shareholders and in the following instalment, I will show you a simple technique to make sure that you know how your technology decisions are impacting your customers so you can make better decisions.

In another instalment, I will talk about some obvious gaping opportunities to cut unnecessary costs and some basic principals of architecture that even the bin man could understand in one lesson and yet is ignored by 99 percent of technical architects

In further instalments I will talk about the cultural barriers (the stuff we don’t ever discuss around here) that stand in the way of making businesses work though technology and I will show you simple tools to help you discuss and master strategy and planning as a precursor to technology investments.


Good management strategy and practice supported by intelligent modern systems