Inane CIO Surveys

We are conducting a survey among the top CIOs on IT trends, data centre efficiency, Cloud computing adoption, business analytics, mobility, leadership, top 5/10 priorities, technology priorities, top 5/10 IT challenges, business challenges, social media adoption, big data, security, … phew ! You name it and it is there; every day I get a couple of invites to participate in surveys with varying time indicated, from 5 minutes to an hour. They are run by all kinds of IT vendors, research companies and publications.

Surveys normally have MCQs (Multiple Choice Questions) or a matrix in which you select different options; some also want descriptive answers. Most of them end up taking twice as long to what they mention and what they would like you to believe. For example if a survey indicates that it will take no more than 10 minutes of your time, in all probability it will take 30 unless you are a speed reader. If you actually read all the text in the MCQ before you check/click your answers, then it may take longer !

Why do CIOs participate in these surveys ? They are busy professionals with paucity of time; I think it is to put across their views and get to know what others think (most surveys promise to send the results to participants, 50% do so too). But nowadays they come with incentives attached; take the survey and you stand a chance to win your favourite gadget/device or a shopping site voucher or …. The incentive increases participation rates, gravely said one such surveyor; from 5% to about 20%.

The first page normally gets genuine thoughtful answers; then you hit a block with questions where answers don’t reflect your context or reality; there is no way to skip the question, you end up selecting a random option. Questions are constructed in a way that support what the surveyor wants the outcome to be. Soon you hit a complex grid or matrix where you are expected to balance options in a way that you select one option per row whether it matters to you or not. The more interesting ones are when you keep track of totals in a column across multiple columns.

That is when most respondents start randomly marking answers or rush through the pages. Any online survey or questionnaire that requires more than 5 minutes or has more than 10 MCQs tends to get into indifference zone. What you get is random responses or a middle path or a horizontal row of answers if the questions use a scale of 1-5 or 1-10. Many are kind and desert the survey mid-way rather than input garbage. Why are most surveys so painfully irritating and difficult to respond to ?

You can now understand why most CIOs don’t agree with the results; because the input is rarely a reflection of reality of the participants. The responses appear to be from a different planet, the analysis bewildering, and the implications or actions disconnected from what makes sense. CIOs blame the outcome when they largely respond with indifference to surveys that seek their inputs on what matters to them. The starting point is the survey itself. So how to overcome this situation ?

To begin with reduce the complexity of the information required. If you want a split of percentages across routers, switches, wireless, bandwidth, network management, which is a subset of hardware, which is a part of the infrastructure, then you will only get what you deserve. CIOs do not measure these, neither do IT Managers. Ask questions that can be answered without a calculator or the IT budget spread sheet which has numbers, not percentages. If I don’t do social media, then I need an option “Not Applicable” or let me simply skip the question.

Now I rarely participate in surveys and if I do where possible put in textual answers with my views. Surprisingly no one has as yet written or called me back despite having my email id or cell number. Do people really read the answers ? Do they care about what is being said if their agenda or hypothesis is met ? Maybe the surveys are just to say that we conducted a survey and keep the data aside to publish what you wanted to anyway.

Lists I don't want to see in 2011

It is that time of the year when everyone starts creating lists — of priorities, challenges, opportunities (which hopefully also get budgets allocated), new technologies, and so on. Everyone from analysts, researchers, academics, CEOs, CIOs, publications and anyone who has an opinion contribute to the increasing top three, five, ten (or in some cases a random number) based on their comfort of things that are a must do, must watch for, avoid, don’t even think about it, failures, every possible happening, news, people, the list is endless. I am tired

A lot of these are thought up while on the keyboard, some have inane research to justify, a few are meaningful too. CIOs and IT folks love these, and watch them like religion. Or hate and ignore them because the lists only add to the existing chaos. Adding to these is a set that creates lists for internal and/or external consumption.

Over the years, I tracked lists from major IT research houses and publishing companies to ascertain their alignment with what I planned to do. Like with every list (and that applies to horoscopes too), I found that the alignment varied from 10% to 80%. If you put together a list of current buzzwords, hype curve technologies, magic quadrants, waves, or similar research, you are bound to get a few that will resonate with the personal and corporate agenda.

Now the differences are always explained in context of industry, geography, size of company, maturity in the IT adoption curve; you could also include sun spots, solar or lunar eclipses, global warming, or any other metrics that you can think of. So why does everyone continue to invest significant resources, time and manpower towards the creation of such lists? My belief is that they need to pin up something on their (and others) soft boards, put in presentations, or just publish them for others to marvel at.

Reality is that the lists created by surveys and research are self-fulfilling, based on the asked questions. If a list (of say 10 items) is presented and the respondents are asked to prioritize them, that’s what you will get. Rarely does anyone ask for respondents to fill in 10 blank rows with what their priorities may be. That would be chaotic and statistically not tenable in a report, but would make interesting reading.

So here is my list of lists (and it is not 3,5,10 that I do not wish to see in 2011.

1. Top 3,5,10 priorities/opportunities for the CIO/IT organization
2. Top 10 technologies to watch out for
3. Top (pick a number) business priorities/challenges
4. Top (pick a number) challenges for the CIO/IT organization

If you have any others, do put in your comments. If you love the lists, enjoy them, or if you are cynical of them, join the party. However, if you are indifferent about these and have achieved a nirvana state, can I enroll as your student?

Reality Check for IT Consultants

In an event for CIOs, I had the opportunity to meet with, and listen to a bunch of IT consultants from global consulting and IT companies. All of them presented their view of the CIO challenges and opportunities, and without exception sounded like they had dug into the same library or archive to create slides which said the same thing — though the words were different, to their credit. This presentation made to a gathering of more than 50 CIOs stunned the audience. And this was not because the CIOs were bowled over by the analysts’ insights, but because the presentation was disconnected from reality.

Consultants have a wonderful habit of looking down upon their audience in a condescending way while preaching their version of truth, which says, “I know better than you, and I have a prescription for the ills that pain you”.

So the consultants under discussion were attempting to advise CIOs of their current challenges, reality and cures for the situation. While the number of points was consistently at 10 (wonder why everything has to be “top 10”!), the order of appearance of the topics was not in sync. The most interesting part was that no one, I repeat no one, in the audience agreed with the consultants. So the questioning began:

1. Did you actually survey or speak to CIOs and CEOs to create this list?
2. What was the sample size?
3. How many of them were located in India (since the presentation is being made to Indian CIOs)?
4. Considering the sample of more than 50 CIOs in this room, did anyone here participate?

As the cross-questioning got uncomfortable, the consultants were tying themselves into knots and literally sweating (despite adequate air-conditioning). One of them had the audacity to state that, “I am the consultant and speaker for this session; you have to listen to me!”
That was the last straw for most, and the duel almost resulted in an unsavory situation. It was rescued by the organizers — just in time.

It is evident that the CIO has a better connection with reality and business. The challenges and opportunities for Indian CIOs do not revolve around Business Speak or Alignment, but Value Add, Enabling Business and Growth. Global consulting companies are slow in realizing this trend, as the world at large still revolves around the US for them (maybe because most of them are headquartered in US).

The post event networking saw a face-saving quote from one of the consultants. “I was asked to provoke the audience, and I succeeded in doing that”.