I was recently asked to contribute to a maturity model for big data. Anyone who knows me will have guessed that I declined this, but it did get me thinking about the lack of a big picture understanding of the corporate information landscape and a framework to leverage. Given that so many of us operate under the broad heading of Information Technology, surely it goes against every principle of my being not to attempt some mastery of information before cranking up the tech. In fairness, I made a beginning with my paper.
This is not a potted five-page Maturity model but a few questions and few answers to start the ball rolling. Just the effort of beginning to consider such an approach is almost guaranteed to bring big benefits to your information strategy as I have learned from discussions with a number of C levels and specialists.
Useful frameworks and models we frequently refer to
Johari’s window places knowledge into four classifications.
1.What you know you know,2. don’t know you know, 3. know you don’t know and4. don’t know you don’t know.
Many of the world’s most successful businesses still struggle with the very first level. A lot is spoken about the Illusory superiority of those who overestimate what they “know they know”, what you don’t know you know is used every day and can mostly be ignored as long as you principles are right, but it is logical contradiction to suggest someone can “know what you don’t know”. To be aware at all suggest you know a lot already though perhaps not enough. The final category accounts for the bulk of knowledge.
The over simplified answer to the conundrum lies in maintaining sound guiding principles, trusting your judgement, knowing how to seek help, and who to ask and engaging in all forms of innovation. Thus, you can stumble upon the solutions you were not aware of to the problems you didn’t know existed as well as answering the questions you had no answers to, grow your knowledge in the process and continue to develop and maintain the tacit knowledge that supports fast thinking and supports most of your daily activity.
Another useful view of knowledge follows closely o how it is acquired processed and consumed. Malhorta talked about Explicit knowledge (i.e. facts), implicit knowledge (the stuff we imply into facts) and tacit knowledge ( The stuff that has become unconscious )
Organisations also have a similar pattern with the first level lying in reports and dashboards, the second in the way it is communicated and understood and the third rooted deep in culture.
Perhaps the most powerful model for the modern world is OODA the method developed by US Colonel John Boyd to improve decision making in situations where there is little certainty and little can be taken at face value.
Observe describes the key process of collecting information widely and without bias. Orientation describes the process of putting that information in context before interpreting a potential meaning.
Decision is more a state than an end as it is really deciding on a path that will test your hypothesis and tell you more as you hone in on the truth.
Action is carrying out the next information gathering step that kicks the loop off again.
OODA has a lot to offer in a modern marketing environment and is closely related to the ideals that have emerged from Agile movements.
Another valuable viewpoint views information in terms of; what happened, what is happening, what will happen next. This is really a simplification of OODA in many ways. All of these approaches support us in binging structure to information management.
Much of what we do today in the name of AI is little more than the goals and activities outlined below. The goal is predicting the future as per the far right column, “ What does the future hold?”. All too often the scope our imagination stops at, “What will she purchase next?”
Why so challenging?
- First rule of Analysis: You can’t usefully compare apples to potatoes even though they may look very similar.
- If you want a computer to work with data you must tell it a great deal about that data, keep in mind that a computer only understands “On” and “Off”, or as we sometimes represent it “0” for off and “1” for on. That’s the whole deal. All other intelligence must come from you or me, either via logic or via data. E.G your computer can’t read. Each letter in the alphabet is represented by a number. (A binary scale number naturally) and displayed via a .tif image typically.
- When it comes to knowledge, the present and the past are inextricably linked. There is no present without the past. How would you know what to measure and why would you care? were it not for your knowledge of the past. All these “ones” and “zeros” are useless without context to classify them. Likewise, data is useless without context.
- Every organisation runs on a smorgasbord of systems that share few standards if any and the same piece of information e.g the time-of-day will have very different and sometimes unrecognisable formats in different systems and this is before we consider naming conventions or presentation.
To compare the time of an event in one system to that in another, even though they are technically meeting the “apples to apples” criteria, can involve discovering the naming convention, finding the data, finding enough context to make it usable, converting both to a single data format and standardising their presentation. Hopefully at that point you can make that apparently trivial observation with a degree of confidence. Presumably you don’t lose any accuracy in the midst of all that processing.
- Then there is the proliferation of many companies across the globe in a single group. As I explained to a bewildered CMO not long ago, getting data across borders and into a report or a data-mart is often similar to getting people through airports and customs and into a meeting room. The protocols are similar only very much more difficult and less well orchestrated. Several firewalls at every border, rules about what can travel where, ownership and permission issues, disclosure issues, security challenges, language barriers, documentation and timing challenges and never forget the need for goodwill among far flung teams you rely on to provide access without accidentally locking anyone out. There is no border post, you have-to find them, wake them maybe get an interpreter and encourage them to help you. Then you can revisit the challenges outlined in issues 4,3,2 and 1.
- Data analysis costs a great deal of money when done well and even more when done poorly. Investors and all kinds of stakeholders expect a return on their capital and this cannot always be guaranteed at the outset, indeed new risks can surface at any time. Even when acquiring and presenting the right data is not too challenging, there is no guarantee that the insight you acquire can and will be used to drive returns. Yes sometimes a little knowledge can go a long way, but not always. There is a need for accountability and applied governance at every stage.
- With enlightenment comes bias. The more we know the less keen we are to learn or to accept change.
- Data will tell you anything you want to hear if you interrogate it hard enough.
- Testing is very hard and very expensive and even a tiny error in data can spell disaster.
- For a little context, consider a little light reading, Anil Seth: “Reality is a controlled hallucination.”
- You might also consider Pluto: “Allegory of the cave”
How did businesses data get into such a mess?
- Free market in systems with no regulation and few standards, each does it his way. I.E. Supplier power.
- Growth by acquisition with poor Due-diligence and mixed goals.
- Poor Technology representation at board-level and shortage of competent CIOs.
One of these is vendor sourced and one is traced to acquisition strategies, the other is a serious cultural issue that is a long time overdue now. Whatever the cause and I accept there is not always a means to avoid these issues, awareness and risk management is the starting point. For example, understanding the costs and implications of integrating acquired businesses is a necessity.
40% of firms say that “integration” is one of their top five challenges, and 17% named it as their number one challenge.
Adviser Market: Fintech and Digital, January 2018
The value of Micro Focus, the UK’s biggest technology firm, has halved after a sales warning .
“Micro Focus bought Hewlett Packard Enterprise’s software business for £6.8 bn, and difficulties in integrating the business have hit revenues hard, with sales systems and partner collaboration being affected, revealed the firm.”
* Many mature business-people believe there is A SECRET MARKETPLACE out there where they can sell their big data for huge amounts of money without peeing-off their customers or falling foul of the law. Such is the power of spam content posing as knowledge. This is much worse than “Fake News” this is “Fake Knowledge”. Who-ever believed the News?
What do modern businesses want to see in 2023 onward, what are the challenges?
Big players in the consultancy world are betting heavily on the following four steps and their decisions are driven by extensive consultation.
At a Strategic level:
- Shrinking of the power of the big brand in favour of the local brand, de-Globalisation.
- The power of the experience trumps the power of the brand.
- One to millions becomes millions to one. You need to be the one getting your message through.
- More customers, not more sales per customer.
At a Social and Economic level
People everywhere, apart form the very high earners, have been seeing their incomes collapse steadily since 2000, or even earlier. Survival has been about the ability to cut costs through eating and consuming in every way cheaper imports. This in turn is fed by a slackening of standards across the board allowing even dangerous food and drugs and consumables into the home. People can’t afford to complain.
Those who work in the factories that make their goods live in very cheap homes in warm countries with tiny cost of living and do well on a fraction of the income of their starving customers in UK, USA, Europe, etc.
Local businesses, especially big ones stay afloat not via huge profits, but via share buy back and other stock manipulation to keep the markets happy and by using globalisation to avoid taxes. They have become so powerful that governments cut taxes just to attract them in despite knowing they cant afford the cost of it. This is harsh reality in 2023.
The average of course is keep healthy and looks good on paper by the enormous incomes earned by the few at the top, again via a mix of QE and its derivatives, tax avoidance and general dishonesty.
Small and medium businesses are desperately looking for ways to win customers on the web platforms like Google, Amazon etc and to drive targeted visitors to their online offerings. Predicting when she will buy and what she will buy seems like the only salvation and promises of AI that can make her miserable salary stretch and predict her moments of largesse are all the rage as you can imagine.
At a technical level
Data has more than one use in the world of business.
1. Customer focus:
Currently, the vendors of cloud-based ML tools are swamping the content ecosystem with wonderful stuff designed to fool us all into believing that only by engaging their tools can we have the knowledge to grow and thrive and indeed that we are sitting on a goldmine of valuable data we are ignoring. None, or certainly little of this of course, is true, but that does not mean we can ignore the need to sharpen up our attitude to data and its potential to improve our performance.
The only way we will be able to maintain relationships with customers at a meaningful level and in future with their purchasing devices, is by carefully deciding what data to collect, winning agreement to collect and use it, with a genuine proposition, e.g. an exchange of value and “delivering the goods” in terms of a great customer experience.
Nowhere in this formula is there a mention of spying, or stealing data, much less selling it to third parties. There are no free-lunches.
I refer you to Seth Godin’s enlightened little book “ Permission Marketing” as an evergreen on ethical and profitable customer relationships.
- Running an efficient business:
The dirty secret of the CEO, there is no such thing as an efficient business and many of the most apparently successful businesses only survive their incompetence through advantages of scale , or geography, questionable book keeping and financial management or generally shifty practices that might get a small local business person into the courts.
Time after time I work with global businesses that have no idea what is happening in the rest of the world outside their H.O. reception area and worse still, businesses that believe there is no world beyond their walls and imagine that an occasional policy email to the “management list” fixes everything.
If you are reading this and thinking of Armageddon, think again, what this means is an enormous opportunity for smaller business to take control and bigger ones to put their house in order. The problems are in fact what some product people describe as “Probortunities”.
A recent example I uncovered that is typical of the global business world, had more than 40 companies within the group and used more than 30 ERP systems only 3 of which were the same. Not only is the data structure different and the naming conventions and even the sizes and formats of individual pieces of data, but in many cases databases, tables and individual fields in databases were further obscured by the vendor to prevent integration by non-insiders. Getting a single view of the operation of these businesses, despite the fact that they made and sold the same things, was a challenge of huge and costly proportions. Of course, ERP is the tip of the iceberg, there was CRM, PLM and on average 15 to 25 significant systems in each company and nobody, (rightly or wrongly) trusted the data in a single- one of these systems. That is the face of reality.
Getting a significant mastery of such an ecosystem will give the average business an opportunity with lower risks and far more control than they can ever achieve through acquisitions in a foreign clime, long-term at least. This is a business you already own that you keep ignoring. It is also a more addressable problem than navigating the legal and moral morass of obtaining data about customers with their consent and satisfying the needs you discover.
Focusing on the customer for now
The idea of building the super brand that can start off selling breakfast cereal and through increasing the number of spam emails, end up also leasing them a car and investing their pension is thankfully a subject for the history class.
The route to growth is through learning about why others might buy your product, and what other related problems they need solved, where to find them and how to talk to them, improve their experience and sell more focused products to more customers, NOT selling a wider range of products to fewer customers.
The brand that will help you achieve that is the one that really understands one specific problem for an individual customer groups and is trusted for that reason, NOT the guys with whizzy adverts who sell everything love babies and puppies and “just want to hug you”.
The knowledge required to achieve this goal is not reliant on mountains of data, rarely is it advisable to use data older than a year or two, because of trends and movements and above all the enemy is “averages”. This is about individuals and small groups, NOT averages. This is about value not words. Professor Sam Savages little book “The flaw of Averages” is little gem everyone should read.
This key business process costs five times as much as retaining existing customers in the common accepted wisdom and has the same outcome. Naturally there are a great many assumptions bound up in that, but it is a useful starting point.
Winning new customers is about knowing who to approach, where they are and how to address them, or just plain old-fashioned luck combined with hard work. The problem with clever targeting is that by the time a strategy has proved itself, it is generally out of date and things have moved on. Strategy is a non-stop ever moving feast when it comes to acquisition and often works better with small data, intuition, agility and enterprise. There is one vital rule of free markets, “The fact that a trend is discover-able means it will soon cease to be” the market will move to iron it out.
The same level of wisdom suggests that few businesses place as much emphasis on retention as they do on acquisition. The Acquisition team have expensed credit cards, nice company cars and all sorts of status symbols and perks, while the retention team are in a sweaty warehouse in Bangalore making it up as they go along. The first question to answer is:
Where do I want to focus my efforts?
Of-course the above is deliberately skewed, in fact customer services is very often not seen as part of customer acquisition but a renamed version of the complaints department. This is the annoying stuff that must be done such as delivering products and answering queries from pesky customers who managed to find the hidden phone number and then toughed out the IVA .
For many, customer retention is about sending out a few timed emails into the customer’s spam folder to tell them about a new offer or in the more savvy marketing department, some pointless news. Here and there we have a lucky Customer retention manager whose boss bought him a clever machine that has worked out the times a customer is most likely to leave and sends a spam to them at just that moment. Some have all the luck. Actually, keeping them happy is nobodies’ job. Who would want to do that? The second question to ask is:
What does customer retention mean for us? What are we prepared to do?
- Learning how your customers get on with the product is one very valuable thing you can do and the best way is to take every opportunity to talk to the customer, or rather listen to the customer and note the insights. Here is a chance to learn staff you didn’t know including some you didn’t know you didn’t know. Debriefing random call centre staff used to be a great approach for this purpose. In simple terms it requires a few workshops and reporting framework, but who does it?
- Being available when they are angry with the company or the product and being ready to jump in and solve problems that made them angry. This doesn’t just retain customers, it wins referrals (new customers), but most of all it teaches you about what customers value and what makes them leave. Collecting and reviewing this information is fairly straightforward if you know a friendly IT guy.
- When you have all of this sorted out and you want to try and gain even further insights, some simple machine learning algorithms run over a year of very clean and accurate transaction data might help you spot other trends and on investigation you may surface even more triggers to help you spot opportunities. To do this intelligently often requires nothing more than a few free downloads and rarely needs expensive infrastructure.
What you must know and must act upon.
Mountains of data won’t make you mountains of profit unless you are in the business of selling it under false pretences without the permission of the true owners and I put it to you that this business model is dead and the burial continues.
Make certain that you know what you need to know to run the business and that this knowledge is trustworthy and current.
Learn to collaborate, work in teams, ask for help, choose your advisers, engage in listening exercises at every opportunity and learn and use innovation techniques.
When you master these simple steps, you will have the knowledge to take the profit you are entitled to from the products you already sell, find the best solutions to the problems you are aware of, discover customer problems that never occurred to you and find innovative solutions that turn them into opportunities.
Tough as the integration and data quality challenges may be, any specialist will tell you that the problems of integration pale into insignificance beside the lack of motivation and skills to use information and the resistance to change that occurs when you push people to admit their weaknesses and learn something new.
If you are up to your neck in data integration and you have not thought the strategy through take a step back.
If you are entertaining Machine Learning Vendors but you can’t find the phone number of a key customer, take a vacation and start again.
If you are setting out on a strategy to build a winning business, begin by defining a strategy and a simple maturity model that builds on what you have, uses what you are capable of and delivers consistent achievable improvements.
Edward Taaffe is a consultant with more than two decades of experience both at a the strategic and technical ends of information management.