Roughly 2.5 quintillion bytes of data are produced daily. If that seems like a lot, over the last two years 90% of all data in existence was created, according to computer software company Domo.
Big Data used to just be about capturing all data possible, but better tools and more endpoints collecting data have shifted the landscape, according to Bill Schmarzo, CTO of Dell EMC Services Big Data, in an interview with CIO Dive.
Just having a Big Data program is a misuse of resources if it isn't driving profits and change. Making sure this happens is a combination of having the right people are at the helm and understanding that Big Data is not just a technical matter.
Show me the money
Until Brad Pitt and Jonah Hill got involved, understanding how Big Data interacts with real-world and real-time situations was unknown to most people. But Michael Lewis's book "Moneyball" put forth an important understanding of data science years before the movie:
"Data science is about identifying those variables and metrics that might be better predictors of performance," said Schmarzo.
The real world for most companies isn't innings and games won, but rather turning a profit. And in the digital age, collecting the right data, analyzing it to fit your needs and applying it to improve business operations and efficiency is a home run.
And if Big Data is to be understood in terms of economics, not ones and zeros, organizations need to appoint a chief data monetization officer (CDMO) instead of a chief data officer, according to Schmarzo. Chief data officers generally come from an IT background and report to a CIO, but a monetization officer may offer something different.
The CDMO should have a background in economics and function like a chief revenue officer, examining a company's Big Data operations while asking the question, "How do I monetize digital assets?" according to Schmarzo.
The right person for the right job
Many of the highest-paying skills in tech are Big Data oriented, and data scientists, machine learning engineers and Big Data developers were among the top emerging jobs in the market. But a data scientist may not be driving the most value for a company if put in the wrong place.
"Right now, most organizations take the data scientists to do the data engineering work, which is a really huge mistake because it just takes different skill sets," said Schmarzo. Companies that can delineate the two positions will create a more effective and efficient Big Data operation.
Schmarzo breaks down the two roles:
- Data engineer: responsible for acquiring, cleansing and aligning data
- Data scientist: responsible for making sense of data and choosing the right neural network to process it
Blending the two roles into a "zebra data scientist who can do it all" is unwise, according to Schmarzo. After all, mastering the demanded skillset for just one role is no easy feat.
"I can probably take a lot of the people who are doing ETL in my data warehouse environment and turn them into data engineers," said Schmarzo. "They've got to learn some new skill sets, some new Big Data tools, but they understand SQL."
Internally transitioning and retraining workers is one way to handle the talent shortage facing the tech industry. To cope with high salaries and tight budgets, organizations can benefit from a model of hiring that brings in new workers with desired, specific skill sets and pairs them with existing workers with older skill sets but more experience, according to Schmarzo.