I had the privileged invite to judge ICT vendors; from a respected enterprise that gives away awards every year. This was their first attempt to form a jury to decide the awards. Earlier years the awards were decided based on size, growth, market share, and in some cases new innovations added during the previous year.
The number of award categories had grown over the years from a handful to more than double score. Thus multiple juries consisting of senior CIOs were appointed and the task was split.
Absence of customer-inputs
We got started with the understanding of categories we were to judge and the time allocated for discussions of each award. Everyone agreed and we jumped onto the first category. The nominated and shortlisted names were not a surprise.
But as we started to scratch the surface, the question came up “where is the customer dimension? How can we assess the relative merits of performance without the voice of customer?” It was evident and confirmed that over the years there was no thought given to this aspect in deciding the winner. The sound logic stated that size and/or growth demonstrate customer confidence.
The jury did not bite that. Sometimes size is a function of regulatory play, incumbency factor or better marketing machinery. Progressing through the categories, the debates took many hues; at times the shortlisted vendors were not perceived to be market leaders or worthy of an award.
In some the selection criteria of the nominations made it appear that the award was pre-decided; the deliberations had the jury wondering if these were sponsored awards being played out to gain respectability.
Truth Vs. marketing babble
Not too long ago a CIO had used social media to highlight the farce behind one of the industry awards for CIOs. In the world of scams, anything is possible. Over lunch the discussion did veer to this doubt. We were animatedly appeased by the organizers that such was not the case. They acknowledged the shortfall in data and that some criteria needed amends.
CIOs listening to vendor pitches and presentations tend to believe awards cited by the vendor. They purportedly validate the technology, solution, or service as it is assumed that experts indeed evaluated objectively across formal KPIs that matter. A few dazzling awards may appear alien but are rarely challenged. If an exotic niche publication conferred the award, so be it. Micro-segmentation works to serve a purpose. Ho hum!
Importance of being prudent
Sticking to what matters to the business is always a good starting point while selecting any vendor. The other important factors include, and not limited to, cultural alignment, success in solving similar problems, industry/ domain focus, long-term development strategy, apart from size, growth, and the awards they have accumulated.
If you are an early adopter of technology, seek safeguard that shares the risk/ reward. For others, nothing works better than peer reference, i.e. talking to existing customers.
Back to the awards, we the jury were aghast at the invisible customer angle. The high point of the day spent was a category that was denied an award in the context presented and one that got away was a lone nomination to a category that at best had a start-up as a challenger.
So next time a vendor puts up a slide or gives you a brochure with glitzy photos of awards, acknowledge them, but do remember to exercise your right to references with or without the help from the vendor.
The number of award categories had grown over the years from a handful to more than double score. Thus multiple juries consisting of senior CIOs were appointed and the task was split.
Absence of customer-inputs
We got started with the understanding of categories we were to judge and the time allocated for discussions of each award. Everyone agreed and we jumped onto the first category. The nominated and shortlisted names were not a surprise.
But as we started to scratch the surface, the question came up “where is the customer dimension? How can we assess the relative merits of performance without the voice of customer?” It was evident and confirmed that over the years there was no thought given to this aspect in deciding the winner. The sound logic stated that size and/or growth demonstrate customer confidence.
The jury did not bite that. Sometimes size is a function of regulatory play, incumbency factor or better marketing machinery. Progressing through the categories, the debates took many hues; at times the shortlisted vendors were not perceived to be market leaders or worthy of an award.
In some the selection criteria of the nominations made it appear that the award was pre-decided; the deliberations had the jury wondering if these were sponsored awards being played out to gain respectability.
Truth Vs. marketing babble
Not too long ago a CIO had used social media to highlight the farce behind one of the industry awards for CIOs. In the world of scams, anything is possible. Over lunch the discussion did veer to this doubt. We were animatedly appeased by the organizers that such was not the case. They acknowledged the shortfall in data and that some criteria needed amends.
CIOs listening to vendor pitches and presentations tend to believe awards cited by the vendor. They purportedly validate the technology, solution, or service as it is assumed that experts indeed evaluated objectively across formal KPIs that matter. A few dazzling awards may appear alien but are rarely challenged. If an exotic niche publication conferred the award, so be it. Micro-segmentation works to serve a purpose. Ho hum!
Importance of being prudent
Sticking to what matters to the business is always a good starting point while selecting any vendor. The other important factors include, and not limited to, cultural alignment, success in solving similar problems, industry/ domain focus, long-term development strategy, apart from size, growth, and the awards they have accumulated.
If you are an early adopter of technology, seek safeguard that shares the risk/ reward. For others, nothing works better than peer reference, i.e. talking to existing customers.
Back to the awards, we the jury were aghast at the invisible customer angle. The high point of the day spent was a category that was denied an award in the context presented and one that got away was a lone nomination to a category that at best had a start-up as a challenger.
So next time a vendor puts up a slide or gives you a brochure with glitzy photos of awards, acknowledge them, but do remember to exercise your right to references with or without the help from the vendor.
No comments:
Post a Comment