Whenever we enter the hype cycle of a new technology or technical approach, we are often faced with unfortunate labels. I can’t count how many times I answered the question, “what is cloud?” over the past few years. And the same is true now with “Big Data”.
I imagine at the time of conception, it made sense. Two techies chatting about their data issue, and one saying, “but it’s a really BIG data problem”.
The problem is, because of the term “big data”, what big data is trying to solve is often misconstrued as just an issue of having too much data needing to be managed, processed, stored or secured. While those challenges could all be elements of a big data issue, it’s not the end game.
Sure, we all are suffering from data overload. But big data is not just about the pure size of your data; it’s about what you do with it! (I promise not to take that pun any further right now)
The core of big data is the intelligence we gain from it, the actions we take from our analysis and new insights, and how that then helps us move our business forward and have a competitive edge in the marketplace.
The difference between business intelligence and big data
Perhaps we are splitting hairs by trying to distinguish between BI and big data, but there is a significant chasm you must cross. I like to say that we experienced a “bit flip” in analytics with big data.
For example, I managed a BI project for a previous cloud company, and we leveraged data from a single database source, created a visualized front-end, and delivered to our customers a new graphical interface and interactive charts that provided more information about the status of their B2B order to cash process. We had a specific data source and a single deliverable. Also, this was an asynchronous process – we were not trying (nor could we actually achieve) a real-time process. It was typical batch processing.
It’s not that this did not provide competitive advantage, as we productized this interface as something special we had over competitive solutions. But in today’s world, I would categorize this as a fairly basic analytics process.
Big data goes beyond basic analytics or business intelligence, because we are using many types of data from multiple sources, and trying to make sense of it all in order to drive us to new directions, vastly accelerate our traction, or get ahead of the market.
Importantly, many times the intelligence we are gaining is not from the data itself, but rather from the meta data (or data about the data).
I have spent many hours analyzing meta data to learn more about what types of data businesses are storing, the growth of certain data types, and other data behind the data.
So What about the Size?
Okay, I’m not trying to downplay the impact the immense data levels we are all trying to manage has on our business. The most recent Digital Universe report from IDC and EMC forecasts 44 zettabytes (that’s 44 trillion gigabytes, folks!) of digital data by the year 2020. From infrastructure to cloud computing to data security, the digital universe is causing havoc for all industries and organizational types.
My point is that one person’s big data challenge could involve petabytes of data, while another’s might be mere gigabytes, but the need, the complexity and the multi-dimensional aspect is still big data.
Big Data Crosses Corporate Boundaries
Another important aspect of big data is who owns it within an organization. Because we are dealing with so many disparate data sources as well as data that can impact multiple teams and initiatives, big data requires close collaboration across siloes. Marketing, IT, sales, operations and other teams must agree on the business goals and intelligence needing to be gained from the data to reach those goals.
Without the proper collaboration and shared goals, you end up with multiple, disparate and often expensive big data initiatives, and with a CIO trying desperately to maintain security guidelines for the data being leveraged.
Within this collaborative spirit, do have a project owner. The only thing worse than no collaboration is a committee-driven approach to consensus. Big data should go hand in hand with agile methodologies (a topic for another time).
Using Data to Win
The point of this collaboration is to win; to move your organization ahead in the market and gain competitive advantage. Big data is not about just creating a sand box of data analysis for the hell of it (although that sounds fun to some of us data geeks).
Back to the business goal. What would be a win? Is it increasing revenue? Is it a faster product development cycle? Is it improved customer service? Together, as a leadership team, define the end goal for your big data initiative. For each project, have a single, clear goal, with everyone clear on what success looks like.
Big Data Management
Once your goal is defined, you’re ready to identify the data sources, the security requirements, the applications involved, etc. Then figure out if you have existing vendor solutions or relationships to implement the big data initiative.
While many big data vendors have technical answers, often what is also needed is expertise that crosses business and technology, and a vendor that can speak the language of both your IT and business colleagues.
If you end up seeking a new vendor relationship, make sure you collaborate on vendor requirements, budgets and selection.
It’s the Size of the Opportunity, not the Data
With the data sources available to us, we have more opportunity than ever before to understand our market, our customer, our competitive environment and our prospects for growth.
My bailiwick is making sure we don’t focus on the “bigness” of the data, but rather the size of the opportunity through using the data to gain competitive advantage.
This was written by Margaret Dawson and posted here.