The following is a guest article from Jack Norris, SVP of Data and Applications at MapR Technologies.
Digital transformation is a key topic for business leaders today. While the impact of digital transformation is easily understood, what is less clear are the steps to effectively pursue a digital transformation — and the three keys to ensure successful digital transformation.
We are in the middle of a re-platforming of the enterprise. Over the next four years, we’ll experience flat IT budget growth, but underneath that will be a steady decrease in legacy spend accompanied by a corresponding increase in next-generation technologies. So the key to reducing costs while driving innovation is the data.
The forecast shows that within four years, 90% of data will be on next-gen technology. Data is the critical leverage point, independent of the hardware choice. Not only does it work both in on-premise and cloud environments, it can intelligently handle the flow of data and processing across environments, including devices.
A disruptive data platform enables a wide range of processing and analytics, and supports running both new and existing applications. A disruptive platform can be used to dramatically lower costs.
How are these cost savings realized? The key driver is taking advantage of a much lower cost per terabyte of data. Companies are typically augmenting existing deployments and migrating data to reduce footprint and costs, with an operating expenditure savings of 80% and a capital expenditure savings of 95%.
Organizations are also deploying innovative applications that aren’t possible with traditional approaches. Application development is particularly difficult when it comes to bridging operational and analytical applications. IT must provide integration, but there are inherent barriers to integrating data, and these barriers result in delays, downtime, and prevent real-time operations.
Traditional approaches require an application-first approach. You start with the application and determine the data requirements. Then you prepare the data into specialized schemas to serve the application. With this approach you need to understand all the requirements so you can have the right data model. You have to extract the data, transform it, and get the data loaded.
This results in application silos. According to Gartner, the significant data management issue facing organizations is the proliferation of data silos, with the typical organization having to deal with hundreds of separate data silos.
Digital transformation and your ability to leverage data is at the core of your future competitiveness — it’s at the core of your ability to control costs and drive innovation. With a converged data platform, you can build innovative new converged applications that can transform your business by providing a competitive advantage that was simply not possible until now.
The keys to digital transformation
The first key to digital transformation: Data convergence
This brings us to our first key to digital transformation – data convergence. The secret is to bring analytics and operations together. Convergence enables the immediacy of operational applications with the insights of analytical workloads. They leverage continuous analytics, automated actions, and rapid response to better impact business as it happens.
Historically, analytics and operations systems have been separated, with dedicated processes to extract, prepare and load data, which introduces delays as well as administrative and security issues.
The benefits of a converged approach are to combine analytics and operations into a single platform, to eliminate delays and latency, enabling real-time applications.
The second key to digital transformation: Stream processing
Harnessing these data flows and understanding their meaning and context are key building blocks for digital transformation and the development of breakthrough applications. These data sources can range from machine sensors, web events, biometric data, mobile, or other types of events.
A stream is an unbounded sequence of events carried from a set of producers to a set of consumers. Producers and consumers don’t have to be aware of each other; instead, they participate in shared topics. This is called publish/subscribe. Streams can simplify the data integration pipeline, since a consumer can filter many producers’ topics, which are aggregated and then joined to the consolidated data.
They enable a range of real-time processing, from applications built around trending news feeds to operations dashboards, to real-time fraud detection and customized offers.
The third key to digital transformation: Application agility
Streams enable event-based microservices, which are a necessary component for application agility, the third key to digital transformation. Microservices is an approach to application development in which a large application is built as a suite of modular services. Each module supports a specific business goal and uses a simple, well-defined interface to communicate with other modules.
Microservices, combined with streams and a converged platform, provides:
- Application development across file, database, document and streaming services
- Ultra-scale, utility-grade performance
- Greater efficiency and simplicity than alternative architectures
- Integrated data-in-motion and data-at-rest and continuous and low latency processing
With a typical microservice architecture, a message broker communicates with the microservice but does nothing to coordinate the data flow. Until now, the architecture of a microservices application would rely on the use of multiple disparate platforms for supporting the different processing and message passing services.
These different platforms would often correspond to physically separate clusters. Messages traveling between services would have to hop between platforms or cluster, adding latency and complexity. Developers and administrators need to keep track and manage the data flows and updates. This is relatively simple for ephemeral applications but with stateful applications that share data it can be a very difficult endeavor particularly when there are large volumes of data involved.
- For developers, a converged data platform gives them tremendous flexibility and agility. Developers are free to choose the best approach for their project, whether it’s a complete application or a microservice.
- For architects and administrators, a Converged Data Platform simplifies administration, avoids cluster sprawl, and unifies security, data protection, and disaster recovery.
The road to digital transformation can be paved with a series of short-term projects that each generate positive ROI and payback. This series of tactical steps can result in a strategic architecture.