Showing posts with label Big Data. Show all posts
Showing posts with label Big Data. Show all posts

Monday, September 19, 2016

Internet of Things is invading our lives in many more ways than we can imagine !

Industrial sensors have been around for a long time providing data streams to measure effectiveness and reduce breakdowns in assembly lines; these specialized solutions over the last six decades apart from automation helped faster, better, cheaper production. A score years later RFID created new possibilities with anti-theft in retail, toll payments, building management systems, supply chain track and trace, and identity management with smartcards amongst other use cases that has sustained interest in the technology.

Another two decades passed for the advent of what we know as IoT which in the initial days had imagination run wild on use cases and potential; auto-replenish refrigerators, trashcans and garbage bags posting to social media on what you consumed, it has been a wild ride for dreamers and thinkers. Two decades later, price and technology improvements have started shaping some of the whacky ideas into reality. The possibilities are exciting and scary at the same time with traceability resulting in loss of privacy for individuals and society.

Today almost all of us are being tracked by virtue of our presence on the internet and mobile phones we carry; almost all websites now what to send notifications across devices, track our movement, maps store data of directions we ask for and travel we complete; loyalty, credit and debit cards store transactions creating personas that would probably scare the hell out of us. Finally our fitness bands and smartwatches gather vital data that can influence our insurance premium, healthcare costs and medical profiles.

Current hype is all about transportation which has already seen aggregation and disintermediation on a large scale globally. Add autonomous to that and suddenly it starts impacting a large number of ancillary industries as the world moves to conveyance as a service. No more car loans or insurance, or scares with crude prices fluctuating, or breakdowns; no worries about parking slots at work or when out with friends or shopping; no traffic violations to worry about, toll payments, or upgrading the car ever so often.

The most beneficial aspects of Internet of Things come to life with Smart Cities where the number of use cases keeps increasing. Smart energy management, traffic monitoring, water management and leakage detection, waste disposal, and citizen services. Protection of energy grids or control of emissions and gases, and monitoring, surveillance of public infrastructure, law and order round up the IoT enabled services. Smart Cities also promise ubiquitous wireless connectivity to offer services and track tagged individuals.

Another benefit that the industries across have garnered is in temperature controlled logistics; food chain definitely benefits from IoT enabled trucks and vans, the bigger beneficiary has been the healthcare industry with medicines and pharma products retaining efficacy when transported and stored in defined controlled environments. The industry has boomed with the availability of efficient and reusable technologies that now dominate across use cases; though a decade after the world’s largest retailer mandated use, unit level tagging still to take off.

Irrespective of industry, IoT promises to provide new found opportunity to improve internal operational efficiencies or the way customers interact with the company. Industry wise use cases are plenty with Consultants willing to provide useful to harebrained ideas. Startups are also beginning to impact this space with innovative technologies and use cases that challenge conventional wisdom. The challenge to enterprises is to weave these into the exiting organization fabric without disrupting business as usual.

Symbiotically linked to mass adoption of IoT is the ability to analyze and mine insights from the vast pools of data that is flowing in with ever increasing speed. Our ability to store and analyze data has kept pace with the data streams that threaten to flood storage space if not managed effectively. The ability to separate the real stuff from the noise will differentiate the grades of success enjoyed by companies and their customers. Newer data sources and correlations make better actionable insights fueling the IoT wave.

It is contingent on business and technology teams to continuously explore new IoT technologies and use cases; the potential to disrupt is not always obvious at face value. Driverless cars will put out of earning more than a million people directly and many more indirectly. Smart energy sensors have already started slowing down the increase in energy consumption; IoT has also improved efficiency factors in linear production facilities and warehousing. Smarter, cheaper, and relevant is the way for IoT to keep everyone on their toes.

Don’t sit on the fence, start exploring, keep in touch, stay connected, IoT is all pervasive, use it !

Monday, August 08, 2016

Data, data everywhere but what do we do with it ?

“Can we do something with Machine Learning, Neural Networks, Deep Learning, Artificial Intelligence, to be better than our competitors or disrupt the market ? We are growing faster than the average, but I constantly fear where the disruption will come from to our industry”. Thus spoke a technophile CEO of a mid-sized multinational company. They had grown inorganically acquiring brands, products, and small companies in growth markets to grow and increase their product portfolio to stay relevant to their customers.

It is an exciting time to be in the technology world with technology at the center of almost every disruptive thought and asset light digital business model. It is an exciting time to be a consumer with everyone bending over backwards wanting to provide offers and promotions on anything and everything. It is indeed an exciting time to be an entrepreneur with unimaginable ideas beginning to take root and come into the realm of possibilities with fast evolving technology allowing for rapid prototyping and success.

It is a mobile first world where everyone wants their App on your phone, send you notifications, read your address book, track your location, and gather petabytes of data to analyze to eternity. Data is everywhere, it always was, though not in a form that could be capture effectively, assimilated and explored. Technology has evolved allowing semblance of sense out of stray wisps of information in the large volume of data to form pictures and scenarios; newer insights, action items progressively improving outcomes.

Big data is getting bigger, analytical tools are evolving rapidly, compute is getting faster, storage and retrieval quicker, and business hungrier for data driven insights. Democratization of technology and availability of tools creates a perception of easy pickings which continues to add to shelf ware for the IT organization. CIOs are being pushed to create new centers of excellence or hire skills that will help make the business grow faster and more profitable; consultants are filling in the gaps across the value chain with limited value delivered.

Internal innovation and new ideas are nurtured in few enterprises, this was not one of those; the company had always rewarded subservient behavior with stray incidents of brightness escaping attention. With a cautious approach to every spend in the past, managers were unable to rise to the occasion fearing being snubbed. The technology organization had decent set of tools, they were well aligned to operational IT requirements, but limited expertise and ability to shift gears into a new level of thinking and execution.

The CEO threw the challenge to the management team to come up with a plan on how the company can break away from the normal. Two quarters later there had not made any significant progress or breakthroughs. The CEO thus hired a global top consulting company to help create the roadmap for digital disruption. After three months of Workshops, brainstorming sessions, offsite and many weeks spent in conference rooms, the exercise was declared complete and the management team invited to unveiling of the new strategy.

Sounds familiar ? This is the story of almost every enterprise where information is consumed in reports from traditional ERP systems or data dumps massaged in spreadsheets. Scores of reports with columns represented in different places or one additional data element make up the repository of analysis. Smattering of dashboards at Board Meetings, Investor Conferences, and external presentations represents the visual data creation and consumption. People love their prints but also want tablet computers !

A big hole in the pocket, they had a document with multiple streams and action items; teams were created and tasked to generate new customers, markets, and disrupt competition. The consultants stayed on to oversee execution and find faults; they churned dashboards, progress reports, control towers, to keep telling the management why they had still not achieved the desired results. A year later and a bigger hole in the dwindling treasury, breakthrough success continued to elude them as competition grew fiercer.

A new age company CEO reaffirmed the hypothesis that data is the future lode mine proclaiming they generated and consumed a petabyte of information daily. His business added new revenue streams every quarter, evolving with consumer demand while shaping the market. His company had redefined the industry becoming the benchmark, taking calculated risks and staying close to the ground through the journey. He did not scoff at the old, he just decided to create a new path giving his team freedom to explore.

Can old age companies emulate such examples ? There are a few examples, but still a few; survival is not threatened for most, relevance probably is !

Monday, September 22, 2014

Big Brother is watching you !

“We are sorry Sir, we are unable to accept your order; thank you for calling your favorite dining place”. I am sure you have also received or read some of the messages floating around on social media where the protagonist is unable to order his favorite pizza or pasta or whatever dish you may want to imagine. He is repeatedly denied his request to quench his craving with inferences about his cholesterol or past illness or health insurance coverage; you can imagine the rest as it would probably pan out for the majority of us.

The situation is not too far in the future ! In an interconnected world, combination of personal, medical, social, financial and professional data creates new possibilities of analytical correlations that can be used or misused depending on how you look at the situation. Connected netizens fill in void hours in their lives on multiple devices through the day with status updates, browsing for information or clarifications on something that they are unable to comprehend. They leave behind a trail that can be picked up.

We already see this happening with targeted advertisements across various sites depending on our browsing behavior. Would you turn off cookies or stop accessing popular search engines from your mobile ? What about all the apps on the smartphone ? Some nifty utilities that make life so much easier; finding great restaurants, booking tables, looking up flight schedule or delay. Can these apps pry at other data on your smartphones or the remnants of bread crumbs and widgets start snooping around your laptop or tablet ?

I get this weird feeling when advertisements remind me of my visits to some websites like someone has been looking over my shoulder spying on me all along in my public and private moments. So I decided to confuse the intruders by randomly checking for exotic locations which I would never visit or check out merchandise that most sane people would shudder to buy. The trail it created led the crawlies on a red herring trail. I felt elated as if I had won a major war by thwarting the evil shenanigans in their quest to hound me.

Our dependence on technology is slowly and steadily shifting our thought patterns and the way we think. Intuitively we reach out to query a term or gather additional information adding to the vast amount of data we assimilate. We also seek help for simple and complex problems or situations rarely exerting our thinking prowess. I am not sure if this is good or bad, healthy or not, this has become a way of life. The extreme dependence of the newer generation also known as Digital Natives on technology is exciting and scary.

CIOs have struggled with BYOD getting entrenched across their enterprises and made half-hearted attempts to rope in the rogues with Mobile Device Management and other tools. But that is only part of the story; our incessant need to be online across all our devices adds to the trail we leave behind. Most of the providers have become our shadow with embedded trackers hiding their intent in fine print over 30 pages of agreements which no one reads and clicks “I Agree”. And guess what ? In most cases we don’t really care.

Every day we are flooded with huge amounts of quotes, stories, posts, pictures, videos sent to us by people we know. No one knows the origin or validity of the content we receive. Are we at the cusp of incremental revolution nudged by technology that is controlled by few who are playing with our decisions ? Are our cognitive behaviors are being shaped by some manipulative story teller who is pulling at our heartstrings in ways unknown to us ? Are we really in control of our actions and destiny anymore ?

The digitization of the world is an eventuality that cannot be wished away. It is gaining momentum every day and encompassing every aspect of our lives slowly and steadily. We are given choices on how we would like to engage with technology but peer pressure breaks down resistance quickly. I wonder if I have a mind of my own as I keep going back to technology to assist me with every step I take. As a kid I read the Orwellian tale about Big Brother watching me; I think the future has arrived stealthily without telling us !

Monday, September 15, 2014

My Refrigerator has gone shopping !

I was at this conference where one of the streams focused on Internet of Things or as some speakers preferred to call it Internet of Everything. Every speaker had statistics on the number of “things” that will connect to the Internet and communicate with other devices or the Cloud; the number range varied from 25-50 billion depending on who you would like to believe. The “things” with intelligence could be for specific tasks or multi-faceted to do a range of activities like personal health monitoring, cooking, cleaning, etc.

Every speaker without exception talked about smart refrigerators which will connect to the nearby or your favorite grocery store and proactively order provisions you consume and debit your credit card/bank. One of the speakers dramatized the situation with driverless cars going to the store getting filled by a human. The collective hypothesis was that this is no longer science fiction but reality as it will pan out by 2020, the year by which the number of 25-50 billion devices would be reached; cars, fitness bands, microwaves, light bulbs, interesting thoughts !

As I thought about it I wondered about the situations in my life. For one I like variety in my food and cannot imagine eating the same stuff every day or week. Probably the refrigerator can be taught to have variety. I also don’t want stuff ordered when I plan to eat out go on vacation or call friends over for a dinner; maybe calendar integration will solve the problem to some extent. I hope that leftovers management, use of partially used open packs and managing short expiry products is part of the functionality.

Let us assume that the refrigerator is intelligent enough to not order one item at a time and creates a list of item taking care of minimum order quantity or bill value to avoid unnecessary shipping charges. What happens when the store does not have the items or in the pack size that I prefer ? Will the store substitute another item ? What if I don’t like to substitute ? The store supply chain is probably connected to suppliers’ warehouse system which through predictive analytics understands demand forecasts generated by connected refrigerators.

I am a shopaholic and love to spend time in the stores. My young daughter who grew up traveling in the shopping carts still wants to go back occasionally to now push the cart that was her vehicle. In most of these trips we end up buying a lot more than what we had planned. The visual merchandising appeals to the senses and impulsive buys are triggered by material and inane offers that scream at you while you walk the aisles. So what about cross-sell and upsell that is the hallmark of a good store which entices you to fill up ?

If the above were to translate to reality all the high street and neighborhood stores would struggle to survive and probably turn themselves into warehouse type stores which don’t need a storefront. It would significantly change the way we shop and ruminate over products on the shelves where we can compare not just the price but other attributes. Packaging could be dumbed down to basic, no need for creative colored eye catching wrappers. How will marketers of new products reach out to me on TV or print when I rarely walk the lanes in a store ?

Maybe big data analytics would churn offers to me on the screen of my choice from which I could put it into the refrigerator’s shopping list. I hope that the cold creature (refrigerator) will know its capacity to order only as much as it can store adjusted to my variable consumption pattern. I wonder if my Robot can take out the right quantity of ingredients and cook a meal that I browse and drop it into the task list for the mechanical cook and keep it hot for me when I arrive home. Maybe I am getting a bit ambitious now in my expectations.

The one thing I could not fathom through all the presentations; the bright and intelligent refrigerator knows what to order; the store knows who the order comes from, the car knows where to go to pick up the groceries and brings them back home. I stay on the fourth level of an apartment complex. How do the packages get to my apartment and inside of the refrigerator ? Now don’t tell me that the Robot will do the job, as it would end up compromising the security of the house ! Even a dumb crook will see the pattern and break the system.

I hope you can fill in the missing link for me.

Monday, June 30, 2014

The Digirati are coming

With the advent of the Internet two decades back euphoria around internet based business models exploded upon all of us. Predictions like “if you are not on the web, you will be dead or if you don’t have an internet strategy, you don’t have a business strategy” shook up everyone and pushed them towards limits of paranoia. Untenable valuations on shaky business models led to the dot bust that wiped out millions from budgets and zillions in market capitalization. Now digital is rivaling the din of the past and it has everyone scrambling again.

Some CIOs saw it coming earlier than others; creating awareness within their enterprises, they attempted to raise the bar. Initial reactions of cynicism and indifference led many CIOs to return to their comfort zones while the world around them flirted with the digital wave. As success stories started trickling in, it gave jitters to the disbelievers and created a flurry of disconnected activity. Every CXO wanted a digital project; everyone added the word “digital” in the headline; many ignored the CIO to avoid embarrassing discussions.

SMAC came the response from consultants, vendors and the IT folks alike; to get started on your digital journey, you need the skills, talent and a link back to the physical world that IT provides. Many CIOs reveled in the limelight of having been ahead of the game while the rest joined the confused ranks adding to the chaos with technology play. As individual pieces of Social, Mobility, Analytics and Cloud made way into various initiatives, the picture started to become clear that digital is not an option anymore; it is going to be a way of life.

Board room and management discussions on digital attempted to create correlations with revenue growth, customer service, enablement of suppliers and business partners, automation for improved process efficiency. Now the connections to enterprise goals are shifting the discussion from the likes of Big Data Analytics or Mobility to creating new business models or tapping new profit pools and outpacing competition. Everyone wants to be anointed with title of the CDO to be hailed as the hero when success arrives.

Competition from new age companies in some sectors like hospitality, retail, virtual collaboration, and travel and entertainment has disrupted conventional age old business models leading to a scramble to catch up. Industrial giants slowed by corporate inertia are waking up to new threat and opportunities. Willing to use their scale, muscle power and enormous resources, they potentially have the ability to devour the small fish while they establish new business models and reinvent their business, systems and leverage the digital wave.

Silos of digital initiatives will at best test a hypothesis, for enterprise wide impact, cohesive and integrated approach with CEO alignment is essential. Reality is, IT and business strategies are no longer separate, and they have become inseparable. With everything going online and “Internet of Things” creating an avalanche of hitherto unexplored data, enterprises are pushing the boundaries of analytical possibilities. Corporate and information boundaries are disappearing demanding democratization of analytics and decision making.

The oft repeated question to CIOs raises its head again on their position in this evolutionary revolution ? IT teams need to focus on not just scale but also the new application ecosystem that requires IT teams to discard legacy and pursuit of monolithic systems and shift focus on agile built for purpose apps. This paradigm shift requires preparation for non-stop business in the interconnected world. Customers are challenging the business of incremental innovation and forcing companies to listen and co-create new products and services.

Digital is here and how ! For most CIOs BYOD/T was a beginning, BYOW (Wearable) will stretch the already delicate ecosystem. Finicky customers expecting instant gratification threaten fragile brand reputations with 140 characters and less. Consumption patterns are shifting thereby forcing CIOs to rework corporate peer relationships. I believe that CIOs can reclaim lost ground by challenging existing digital alignments and build the foundation that will help the enterprise win in raging battle for revenue, profitability and the customer.

Your enterprise digital stance may be a challenge at the moment; culturally maybe the company does not enable open ideas or visible risk; it is up to you to decide whether you want to be a bystander while the world moves ahead or you want your destiny to be linked to the new world of digital enablement ? Are you ready ?

Monday, April 07, 2014

Can Big Data deliver Big Insights ?

The other day I met a CIO friend who wanted to discuss a tricky situation in which he had landed; he worked in an industry which was in the thick of being projected as one of the industries that will benefit from investments in Big Data. His CEO wanted him to build a data warehouse to rival some of their global competitors, at least one of which was prominently talked about as the poster boy of Big Data analytics. He was thus under pressure to invest while the rest of his IT budget was under pressure.

Having a keen understanding of technology, his company and the industry, he was a non-believer in the Big Data story; according to him the hype around some of the Big Data insights were not commensurate to the investments made in the overall project. And there was nothing new since the first story broke out of one of the companies having found a use case that conventional technologies would not have delivered. He had many data warehouses and Business Intelligence successes in the past for which he was well known too.

By definition Big Data was all about big data sets that earlier available technologies could not bind together within tolerated elapsed time and budgets. Volume, Variety and Velocity defined Big Data; (Business) Value was added later. The availability of high compute resources and ability to store large volumes of data had made solving some problems easier, faster and cheaper; that is not necessarily success from the capitalized Big Data. It is just that larger data sets were analyzed as compared to the past.

The question at hand that needed an answer was whether he should let go and invest as directed by his CEO or he should help the business with a scalable data warehouse which would deliver immediate value. Is it possible to get started small with Big Data (an oxymoron if there was one) and then work with the business to find the needle (if they wanted to find the needle or a pin) in the haystack; after all Big Data is expected to throw up unknown possibilities by random correlations that human minds are not able to pick.

Big data works on “found” data, i.e. data that you have and complex algorithms which can provide some statistical probabilities. Analysts predict the value that different industries can gain from investments; no one is talking about the real value derived. Governments have been making investments with equal zeal as are large enterprises; the providers and consultants are happy to make hay not just while the sun shines but until by accident they discover a needle in the haystack and make a case study out of it putting pressure on the rest of the gold diggers.

What about the data that you don’t have ? Can you draw negative inferences from Big Data ? For that you have to know what you don’t have ! Can what you have tell you what you don’t ? The answer to that is still to be found; available data in a Big Data repository cannot indicate to what is missing. The concept of “found” data predicates that available data set is the whole universe from which correlations are to be created. And that is where many Big Data implementations are unable to deliver any meaningful insights.

The veracity (the 5th V) of information in a Big Data store can throw up many false positives which have been the bane of many projects. Data will never be clean unlike conventional data warehouses and the velocity will keep you challenged to move with agility. The ability to come out of the clean and complete data mindset is the beginning of what Big Data may enable. From here to get to Value is a long journey with no near-term goals; if you hit something, consider yourself lucky and celebrate.

My suggestion to my friend was to get started the way he believed he will be able to deliver what the business wanted. Forget the discussion on technology and focus on what matters, insights driven by data. If he can get traction from some CXOs based on the results, no one will grudge whether they came from Big Data or Small Data. The business leader in him understood while the technologist wanted to fight; for his benefit, I hope the business guy prevails.

Tuesday, January 14, 2014

Data, data everywhere

“They want 78 new reports from the EDW and Big Data which has taken more than a year and a major part of my budget to build ! They already have hundreds from the transactional systems which are printed on reams of paper which no one reads. All the Excel sheets that they were churning out from data dumps from various systems and a bit of external data get into management meetings where everyone has a different number”. Looking at the CIO I sympathized with his predicament, it was a familiar story.

We all have at some time or the other been frustrated with endless requirements of reports and data dumps from all and sundry; lot of effort is spent in analyzing the past and validating hypothesis on what worked or what did not. Requests flow like rainwater on a slope, never ending stream many similar to others from neighbors at workplace not talking to each other. Reports get built for a casual question in a meeting never to be used again; when another one pops up from new quarters, the effort is repeated.

We hear of associations, correlations and insights not possible in the past as we did not know how to combine an apple and pineapple to get a watermelon. Structured data was easy until we started going to multiple sources with limited commonality. Even then with statistical models diving through seas of data, the proverbial needle could be found in the haystack. People buying napkins buy beer, not vice versa; owners of red cars have a higher propensity to be rash drivers, and so on. You could correlate anything to sunspots !

Not too long ago the need to explore unstructured data began and with social media explosion the dimensions for analysis changed. Thus Big Data began its journey to challenge conventional way of looking at data and information. Jumping on the bandwagon the term was hyped by one and all to include variants that stretched imagination. Came along new skills everyone thought were important for the future: Data Scientists and Chief Digital Officer to name a couple; did such a species exist or it was glamorized plain old profiles ?

Moving from hundreds of GB of data to thousands of GB does not make it Big Data. The amount of data being created and added to corporate storage is growing exponentially. Data types are also expanding with technology offering ways to mine it. Dashboards and cubes work well in selected situations, their action-ability is still wishful thinking. Enterprise manager thinking has yet to evolve beyond reports from transactional systems; thus the data scientist continues to remain a glorified report writer.

The CIO narrated his woes which started with the Company Board approving a really large budget and unrealistic expectations from the project they called BBF (Bigger Better Faster). With much fanfare the project was kicked off, many people inducted into the team and a few pretentious youngsters hired to lead them to gaining insights thus far unknown, from this prestigious first of a kind in the industry Big Data project. The CIO kept his reservations to himself knowing his meanderings would not be given a kind ear.

The project team got bigger faster than anyone thought possible; the technology they bought was deemed better than what they had. Everyone loved the progress they made in the initial months. Then started the reality check with the target audience (managers) putting across what they wanted to run their business better, to grow bigger and reach out to customers faster than their competitors. Challenges with technology and data consistency appeared small compared to the change required in the mind set within.

Activity Reports on social media, portal registration, access reports, keyword searches and some more were the peak of expectation. There was no marriage between the old and the new as if they lived in separate worlds. What could have been remained buried somewhere while everyone wanted better and faster transactional or tactical reports. The rich stream of data that could have been big for the business was diverted and converted into wasted effort. In the corporate world, I believe that the overwhelming data deluge is far from being tamed.

Do you know different ?

Monday, February 04, 2013

Feeding the Elephant


Recently I participated in a Big Data conference which boasted of speakers of all shapes and sizes (literally too) from government, global multinationals, large enterprises, to vendors and academicians rounding off the tail. The audience filled the room to the brim with expectations of gaining insights from the deliberations and debate. After all, according to IT research analysts, Big Data is one of the key technology trends on everyone’s agenda and priority list. It’s like if you are not doing it, then you are Jurassic.

The agenda comprised of speakers from all mentioned above; some had done it, some were selling wares with titles containing “Big Data”, a couple of consultants and service providers who offered their “expertise” on the subject, and finally a CIO to provide an enterprise perspective of how are corporates looking at it. All in all it was an eclectic mix which promised to give value for time invested to the organizers and participants. I took up a corner perched at the edge of my seat and watched the proceedings.

Setting the foundation the keynote speaker talked about the concept, progress made by IT companies, known deployments of Big Data by a few FMCG, internet companies, and government agencies. A case study of a potential big data application at a government initiative demonstrated the dimensions of Big Data, i.e. Volume, Variety, Velocity and Value. Everything was going well thus far with the audience – a mix of technology staff, IT students, and some service providers – lapping it up all.

Then events took a turn that changed the atmosphere in the room; everyone sat up awoken from their stupor and peaceful existence in the cushioned chairs. Like falling off a cliff was how a participant described it later; the turmoil changed the agenda and the utterings of future speakers who were cautious in their exultations of Big Data. The speaker exceeded his time; no one interrupted his thought train. He challenged everyone to challenge his hypothesis; none did. He was the CIO talking about relevance to the corporate.

Who needs Big Data ? Where does it fit into the maturity curve of an enterprise using Business Intelligence or Analytics ? How do you partner with business who is still swamped by reports or dashboards at best ? Actionable insights ? When does a data warehouse become inadequate and Big Data become necessary ? Is it about unstructured data only or volume of data or complexity of analysis ? Is analysis of social media tags or text Big Data even when volume is low ? So what is Big Data ?

Consultants and IT companies have developed models and tools respectively to hypothetically help companies mine the sea of data. They have been talking about uses and value across industries based on some assumptions. A few pilots with companies have not empirically demonstrated a correlation between the Big Data analytics and the benefit. Internet companies have used scalable models of their earlier working solutions as they grew; e.g. recommendation engines, product associations, etc. These are not new.

Is it just hype or a technology solution created for specific purposes now being touted as nirvana for all kinds of data problems or analytics that have historically belonged to the data mart or data warehouse ? The CIO challenged the audience to clear their vision, heads, and minds and think rationally on what is the business problem they want to solve before deciding on the tools and technology. The yellow elephant in the room cannot be ignored; its relevance however needs to be established before feeding it.

At the end of the session which led into the lunch break, the CIO was hounded for his contrarian views; everyone wanted a piece of advice and some wanted to debate their conflicts in private. The poor fellow was deprived of lunch with the next session being ringed in. I believe Big Data like any new technology trend needs evaluation in the context of the enterprise’s reality. Is there benefit to customers or employees ? If not why do it ? Like my old CFO friend said “If it makes cents, only then it makes sense !”