CIO Performance Metrics

In the last few weeks there has been a lot of publicity and visibility on the fact that CIOs are being or going to be measured on business metrics, many of which they do not control or influence directly or indirectly. This sparked many a debates on various forums that attract CIOs, social media sites and groups that are dedicated to IT leaders. Not that anyone tried to contain it, the fever spread globally very quickly with reactions that spanned the spectrum of emotions.

Traditionally IT was measured on three aspects; operational efficiency, budgets, and delivery of projects. The connected world also added information security. Disaster recovery and business continuity surfaced and made it to the dashboard. Somewhere business benefit crept in and then projects were also reviewed from a business angle. Regulatory compliance required significant IT support and thus edged in. But revenue, profitability, customer acquisition or retention, product availability ?

How does the CIO influence any of these ? Can better IT deliver additional growth ? Will technology drive profitability ? What can the CIO do to acquire new customers or retain existing ones ? And what about product quality or availability ? If cash flow is an issue, how will systems ease it ? Competition has a better and cheaper product or a great marketing strategy, can IT or the CIO counter it in any way ? If the answer to one or more is no, then why link performance or compensation to these measurements ?

I know some of the CIOs who have been there done that, evolved beyond technology and/or taken additional business roles will say that the CIO can indeed contribute and influence most of the measures above. This is achieved with systems that create operational efficiency, business process management, information visibility, business intelligence and analytical models, and even enabling new models of engagement with the new trends and hyped technology troika, Big Data, Mobility, and Social Media. Cloud and Outsourcing are passé; everyone has done it or is doing it

It is evident that this piece of news has many people worried, and not all of them are CIOs; they are propagating the message that if I cannot control something, I should have the choice to determine if my performance is likely to be impacted. Fair point if the organization worked in perfect silos. In business and life uncertainty is certain and thus even the metrics that seem to be under control have dependencies – internal and external – which are beyond one’s power of influence.

Does the CEO control how the industry will behave ? Can the CFO control interest rates or liquidity crunch ? If the customer does not buy, what can the CMO do ? Rhetorical questions ? They are not helpless, but there is a limit to their ability to influence the outcome. They do play a role and they depend on the rest of the CXOs to work lock-step in achieving success. CPO (Chief People Officer) has to help hire and retain the best talent, CIO has to ensure information availability to key stakeholders for decision making and analysis. That’s what the C-suite is all about.

So if the CIO stakes claim to the table, it comes with a set of obligations and responsibilities; it comes with the territory; all CXOs are jointly and individually responsible for the success of the enterprise. It is not about “I have done my part and now you go figure”; I believe that CIOs should and does actively seek this responsibility and then works with others in shaping the future. The C-suite has to take this variability risk. Only then can the CIO aspire to take a position on the Board or become a CEO.

Right or Wrong ?

My friendly Board Member who has also been my mentor for some time now always brings up very interesting points in discussions. I have always enjoyed talking to him as he challenges conventional wisdom with his way outside the box ideas. Being technology savvy, discussions with him tend to be not just at a high level but at times I have to explain why one solution scores over the other. In one such meeting he brought into the open a dimension.

We were discussing a new Predictive Analytics solution and its adoption in the industry not just locally but globally. With no competitor locally using such a solution, with bright eyes he started drawing the end game that he wanted us to reach; the dimensions were simple in their representation but complex to execute requiring multiple data sources and algorithms that would challenge most. As the big picture unfolded, it had me and the team scared and excited about the leap forward for our company.

Using a formal matrix for evaluation of the solution is normal for IT; functionality, roadmap, customer success, ease of use, scalability, investment, and industry fit are some of the parameters. He helped us refine the list discarding most of them considering every solution scored a tick on them. It then came down to a few that focused on strategic intent and investments by the vendor. E.g. How many customers has the vendor invested in to enable them to succeed ?

Thus the discussion was at a different plane working on the methodology, plan and shared success criteria that had direct linkage to business outcomes. I could sense wariness in the business, IT and  vendor teams on the perceivably risky proposition. What if the solution does not work ? What if the industry changes shape or direction ? What if business users don’t use the end result for decision making ? How do we enforce the models that the solution recommends ? What if …

Success and failure rates of IT lead projects have been a statistics that scares every one; the reasons have not changed much over the years since I read about them first more than 15 years back. So there was some hope with a project endorsed by Board Member, but then the big question was if we can sustain his interest over the 18-24 month period in which the end outcome would be measured. We decided to raise these questions to moderate expectations and ended up inviting trouble.

Why are you all so risk averse ? How would innovation happen if everyone wanted a fool-proof solution that someone has used in the past ? Why are you always looking for precedence ? Early adopters always gain a competitive advantage even if it is short-lived; in most cases the followers get lesser benefits. If you keep working with a view that we don’t want to get anything wrong, is there a guarantee that you will not ? And not getting it wrong does not imply that you will get it right !

Doing nothing wrong would mean that status quo is the best place to be; trying something new is always fraught with risk. I am not implying that we take undue risks on new untested technology solutions. To get something right requires collective buy-in that CIOs seek for most projects; the marquee CIOs take a lot of calculated risks, and yes they do face failed projects more often than others. However they more than make up for them with their successes.

Inertia is not a good keyword for IT and CIOs; they should seek unexplored avenues to make a difference. I believe that we all strive for success and in the same vein we have a phobia for failure. The obsession to always succeed may result in a dull and boring existence that is disconnected from real life and business which has to compete every day in a new competitive environment with uncertainty. I tend to agree that doing nothing wrong does not mean that you are getting it right !

Requirement gathering

The journey of a thousand miles begins with a small step; similarly the development or deployment of systems and solutions begins with requirement gathering. Conventional wisdom has it that IT teams talk to business users to gather their requirements, and then go about creating or finding a solution which fulfils defined selection criteria. When an agreement is reached between the users and IT, the system is then implemented. No user requirement, no solution.

Over the years many research papers have been written on the methodology and science of gathering requirements. Consultants created templates and promised success with common and uncommon sense. It has also been the subject of ridicule with challenged implementations, many of them attributed to requirements either not being articulated adequately or changing requirements leading to scope creep and thus sliding timelines. The most famous amongst these has been the pictorial representation below.

When I reflect back on the time I was a rookie programmer, I too went through the journey albeit with not too many such experiences. I remember instances when the disconnect was complete while in many the users loved the solution and the benefit delivered right first time. It never occurred to me at that time to analyse why I got it right sometimes while everyone was miserable in other instances. 80% success was deemed good and that kept everyone happy.

So last weekend over dinner when I met a CIO who recently inherited a legacy of many challenged projects, I was curious to find out how he planned to overcome history and get everyone moving in unison in the right direction. When he took on the new position, the buzz in the market was that the veteran of many industries and acclaimed implementations had taken on more than he should have. Everyone wished him luck and wondered what went wrong in his previous company for him to take such a risk.

The first month was spent assessing status, understanding and cataloguing projects, getting the team to wake up from inertia, and getting the business teams to start talking. In the company with many custom solutions, there were many stories on what went wrong in the past with fingers pointing at lack of business domain, incorrect interpretation of requirements, discipline, engagement, low skills, sliding timelines with projects not delivering even with 200% escalation in time and budget. The IT team was demoralized and had given up all hope.

Many months into his new role the vendors were on fire hearing about the transformation, change and the investments that had suddenly mushroomed. Old projects rejuvenated, new initiatives being aired; there was a direction where none was thought possible. The story had believers inside as well as outside. Having heard about this I was excited to be in his company to extract some insights on how he went about solving the unsolvable problem.

The CIO smiled at the questions and said, I coached my team to ask all the right questions and did the same myself with the CXOs. He elaborated user requirements will always contain not just what they need, but also what they think they want or should have. The requirements always contain many elements that are nice to have and not necessarily aligned to current reality. That is why they are referred to as “requirements”. So what are the right questions ?

“What is the business problem you are trying to solve ?”, “What will change for you or your customer once you have this new capability ?”, “Is the benefit measurable in money or time or efficiency ?”. We all ask these questions but not too often; we are happy gathering requirements and then executing solutions which are then sub-optimally used. Requirements tend to get unwieldy with all the exception conditions being mapped; in the end it does not solve the original problem but creates new ones.

Winning teams today do not focus on requirements, they ask the questions above and create solutions that solve real business problems or capture real business opportunities. I believe that the future holds a lot of promise to those who follow the new way of thinking and discard the old requirements lead approach. What do you think ?

Comments and feedback lead to Part 2 and Part 3 !

IT productivity improvement

Earlier in the month I was invited to a session by a senior well-known IT Research Associate who advises many global CIOs on tactical and strategic agenda. He is a good story teller and had the audience of 20 odd CIOs spellbound with his anecdotes, examples and occasional taunts which most took sportingly or sheepishly depending on how you interpret the expressions on their faces. His consistent grouse was that while CIOs have done a lot for enterprise productivity, they have neglected IT productivity improvements.

Tracing through history he illustrated the role IT has played in enterprise process reengineering and productivity improvements driven by automation, new tools and technologies, mobile enabling the enterprise, and providing time sensitive information in the hands of the decision makers. However during this era the IT team has not demonstrated commensurate improvements; solutions still take as long if not longer than what they did many decades back. He postulated despite progress across the industry, the in-house IT team has lagged behind by a big margin with no major visible improvements.

He went on to compare internal IT teams with the work being done at start-up and tech innovation companies globally. Across comparison parameters on innovation, time, productivity, quality, or sheer volume of work done, IT departments lag behind. He blamed this on mind set, archaic beliefs focused on process compliance to whatever framework the IT team had adopted. ITIL, COBIT, PMI, CMMi were boon in the past; they are the bane now. Agile development methodologies find rare favour with IT.

He did not offer any empirical data or prescription, but no one from the audience disagreed. They sat there silently reflecting on their own realities. Post the hour, I sat ruminating over the discourse trying to figure out if this was indeed universal truth; if it was so evident, how is it that no one has thus far talked about it because everyone loves beating up IT and if it was so evident a simplistic comparison, it would have been well searched and figuring in the top 5 priorities of the CIO and every vendor !

How do we measure IT productivity ? Lines of code per day ? That no longer seems relevant with the shift from procedural to new way of creating programs. How about number of people to support per 100 compute or network assets, or servers; even that is now irrelevant with virtualization and clouds taking over. Maybe number of locations, or solutions in the IT portfolio, time to respond or time to repair; most of these activities are anyway outsourced with SLAs.

The baseline has been shifting and IT has adapted well to the change. In linear motion it is easy to measure a shift; in the real world it is a little difficult to quantify. So what efficiency parameters should the CIO use to demonstrate improvements if at all ? The CIO dashboard and reports have evolved from technology availability to business value. Productivity gain is not specific any more but interconnected and interdependent. People do not measure activity but outcomes.

IT influenced results are business agility, competitive differentiators, low cost of operation (Business and IT), growing revenue faster than market. If the CIO is indeed driving these and well accepted and recognized by the enterprise, does it really matter if the lines of code generated by the IT team is lower than the industry benchmark ? 

Judging CIOs and being judged by them

As a recipient of the award myself a few years back, I had the privilege of being invited as a jury member for the Global CIO awards organized by a global industry publication. It was a big responsibility to shoulder when almost all the nominated CIOs were friends who shared a drink or a joke in the past. I felt unsettled about it wondering about the impact it may create on the relationships shared. At the same time I was excited about it with the honor being conferred to be considered for this big task.

This was not the first time I have been on any jury; there have been many instances where along with industry veterans, global luminaries and celebrities, and academia I contributed to the selection of award winners. In most cases the nominees were upcoming leaders; in a few cases where the subject was the CIO or a CEO, other jury members by virtue of their seniority carried the process well without pressuring the junior members. Many of these awards recognized companies and not individuals thus making it easy. This was the first time that I had this wonderful opportunity and I was nervous.

The process was fairly well laid out with well-structured data and defined evaluation criteria. Each jury member selected from different backgrounds and was provided the same information to analyze and independently create the list of winners. The common list with validations would be then declared as the final winners. So far so good !

Listening dispassionately to each pitch without clouding influence from past interactions is difficult. Spread over a fortnight the discussions left me richer with new insights that I could imbibe, a benefit rarely possible with otherwise guarded conversations on challenges and tactics used to overcome them. My respect multiplied for most of the contestants with the learning gained; my achievements suddenly looked insignificant in comparison. On the designated evening as they collected the awards, the new bond shared with the winners created warmth to be cherished for a long time.

Recently, I too was subjected to peer judgment in another open list being compiled by an industry association which sought to recognize “Most Respected CIOs” in India. Self-nominations were not allowed and neither was CIOs reporting into the individual in case of group responsibilities; it was a selection by peer CIOs who were asked to nominate others. With open ended questions and selection based purely on votes, the contest was wide open to anyone.

I believe that peer recognition especially from high performers is difficult to achieve when the starting benchmark is own performance for the person judging. People observe behaviors and form opinions that are difficult to change. The foremost element that matters is Trust which in turn over a period of time builds Respect. It does not happen overnight but can be lost in a moment. It was gratifying to be voted to the list and staying there as the voting progressed.

Investments in sharing, learning, coaching, and mentoring pay rich long-term dividends; it is important to give as much as it is to receive.

Metrics that matter

I bumped into an angel investor in a social gathering organized by a company funded by him. Discussing a range of subjects, he was interested in understanding how customers of his funded company used technology and traction with the Management across different sectors. Acknowledging the fact that all his invested companies used IT as a competitive differentiator, he queried the metrics used by CIOs in India. In the discussion group were CIOs from Banking, Insurance, Manufacturing and Retail.

Starting with IT budgets, the range observed was 1.5% upwards all the way to over 10% for a Bank. I am referring to percentage of revenue, one of the metrics everyone uses and is portrayed as a reflection of the seriousness of IT investments globally. Angelically he disagreed with this norm as Capital and Operating budgets should not be clubbed into one IT budget. Echoing the thought a few CIOs stated that they separated the capital investments moving them to the business units since new initiatives have to be what business needs and wants.

Investors have a way of getting their viewpoints; he asked if separating the capital investment and operating expenses helped. The answer to that was a vehement yes. The CIO actively controls how the existing IT setup is managed and thereby can optimize capacity and support. Investments are always linked to new business initiatives and outcomes. A great system or the best technology does not create a recipe for success if business fails to utilize it effectively. When the investment impacts P&L of the business, the ownership and contribution equals the effort put in by the business and IT.

The discussion veered to CIO dashboards and what were CIOs monitoring daily, weekly or monthly. The responses varied from health of systems to active budget tracking and key projects that IT was involved in. Only two mentioned that there dashboard was no different from the other CXO dashboards but included a few IT metrics too. Considering that the CIO is in most cases an equal partner in the business, why should the dashboard be different ?

Active projects with large investments require monitoring and communication to provide visibility across the enterprise. Success is measured not just by on budget or timeline, but effective use and business value that may have been spelt prior to the project. Like the CMO would monitor marketing campaign effectiveness or the CFO tracks treasury, the CIO has his/her business IT projects.

Lastly the IT Strategy and long-term plan tracking is the most critical one. As the owner, the CIO must track and report periodically progress made, issues and challenges, new opportunities and finally business impact delivered. It is a living plan and not something to be created, approved and locked up. What gets measured normally gets done.

The investor benevolently nodded to the maturity of the CIOs and their success in managing perceptions and that they get it.