Why CIOs Should Turn To Cloud Based Data Analysis in 2015

Originally Published on DataFloq

cloudsroad (1)

CIOs are under tremendous pressure to quickly deliver big data platforms that can enable enterprises to unlock the potential of big data and better serve their customers, partners and internal stakeholders. Early  adopter CIOs of big data report clear advantages of seriously considering and choosing the cloud for data analysis. These CIOs make a clear distinction between business critical and business enabling systems and processes. They understand the value that the cloud brings to data analysis and exploration and how it enables the business arm to innovate, react and grow the businesses.

Here are the 5 biggest reported advantages of choosing the cloud for data analysis

Speed – Faster Time to Market

Be it the speed of getting started with data analysis, the time it takes to have a software stack that can enable analysis or the time it takes to provision access to data, a cloud based system offers a faster boot time for the data initiative. This is music to the business ears as they are able to extract value from data sooner than later.

The cloud also offers faster exploration, experimentation, action and reaction based on data analysis. For example, a cloud-based system can be made to auto scale given the number of users querying the system, the number of concurrent ongoing analysis, the data that is entering the system and the data that is being stored or processed in the system. Without any long hardware procurement times, the cloud can often be the difference between critical data analysis that drives business growth and missed opportunities.

Another consideration mentioned by CIOs is the opportunity cost of building out full scale analytics systems. With limited budgets and time, focusing on generating core business value turns out to be more beneficial than spending those resources on reinventing a software stack that has already been built by a vendor.

Extensibility – Adjusting to Change

A very unique advantage of operating in the cloud is the ability to adjust to changes in business, the industry or competition. Dynamic enterprises introduce new products, kill underperforming products, invest in mergers and acquisitions. Each such activity creates new systems, processes and data sets. Having a cloud based stack that not just scales but offers a consistent interface reduces the problem of combining this data (and securing and maintaining) from a O(n!) problem to a O(n) problem making it a much cheaper proposition.

Cost – Lower, Cheaper

CIOs love the fact that cloud based data analysis stacks are cheaper to build and operate. Requiring no initial investment, CIOs get to pay for what they use and if the cloud auto scales, it makes for simpler capacity growth plans and easier to perform long term planning without the danger of over provisioning. Given the required data analysis capacity can often be spiky (varies sharply by time depending on planning and competitive activities), is impacted by how prevalent the data driven culture is in an enterprise (and how the culture changes over time) and the volume and variety of data sources (this can be change at the rate of how the enterprise grows and maneuvers), it is very hard for the CIO to predict required capacity. Imperfect estimates can lead to wasted resources or/and unused wasted capacity.

Risk Mitigation – Changing Technological Landscape

Data analysis technologies and options are in a flux. Especially in the area of big data, technologies are growing and maturing at different rates with new technologies being introduced regularly. In addition, it is very clear given the growth of these modern data processing and analysis tools and the recent activity of analytics and BI vendors, the current capabilities available to business are not addressing the pain points. There is a danger of moving in too early and adopting and depending on a certain stack might end up being the wrong decision or leave the CIO with a high cost to upgrade and maintain the stack at the rate it is changing. Investing in a cloud based data analysis system hedges this risk for the CIO. Among the options available for the CIO in the cloud are Infrastructure as a Service, Platform as a Service or Analytics as a Service and the CIO can choose the optimal solution for them depending on bigger tradeoffs and decisions beyond the data analysis use cases.

IT as the Enabler

Tasked with security and health of data and processes, CIOs see their role changing to an enabler role where they are able to ensure that the data and processes are protected while still maintaining control in the cloud. For example, identifying and tasking employees as the data stewards ensures that a single person or team understands the structure and relevancy of various data sets and can act as the guide and central point of authority to enable various employees to analyze and collaborate. The IT team’s role can now focus on acting as the Data Management team and ensure that feedback and business pain points are quickly addressed and the learnings are incorporated into the data analysis pipeline.

A cloud based data analysis system also offers the flexibility to let the analysis inform the business process and workflow design. A well designed cloud based data analysis solution and its insights should be pluggable into the enterprise’s business workflow through well defined clean interfaces such as an insight export API. This ensures that any lessons learnt by IT can be easily fed back as enhancements to the business.

Similarly, a cloud based data analysis solution is better designed for harmonization with external data sources, both public and premium. The effort required to integrate external data sources and build a refresh pipeline for these sources is sometimes not worth the initial cost given business needs to iterate with multiple such sources in their quest for critical insights. A cloud based analytics solution offers a central point for such external data to be collected. This frees up IT to focus on providing services to procure such external data sources and make them available for analysis as opposed to procurement and infrastructure services to provision the data sources.

A cloud based solution also enables IT to serve as deal maker of sorts by enabling data sharing through data evangelism. IT does not have to focus on many to many data sharing between multiple sub organizations and arms of the enterprise but serve as a data and insight publisher focusing on the proliferation of data set knowledge and insights across the enterprise and filling a critical gap in enterprises of missed data connections and insights that go uncovered.

Advertisements

All (Big Data) Roads Lead To Your Customers

Originally Published on DataFloq

A large number of enterprise report a high level of inertia around getting started with Big Data. Either they are not sure about the problems that they need to solve using Big Data or they get distracted by the question of which Big Data technology to invest in and less on the business value they should be focusing on. This is often due to a lack of understanding of what business problems need to be solved and can be solved through data analysis. This causes enterprises to focus their valuable initial time and resources on evaluating new Big Data technologies without a concrete plan to deliver customer or business value through such investments. For enterprises that might find themselves in this trap, here are some trends and ideas to keep in mind.

Commoditization and maturation of Big Data technologies

Big Data technologies are going to get commoditized in the next couple of years. New technologies like Hadoop, HBase etc will mature with both their skills and partner ecosystem getting more diverse and stable. Increasing number of vendors will offer very similar capabilities and we will see these vendors compete increasingly on operational efficiency on the pivots of speed and cost. Enterprises who are not competing on the “Data efficiency” i.e. their ability to extract exponentially greater value from their data as compared to their competitors (notably AMZN, GOOG, YHOO, MSFT, FB and Twitter) should be careful to not overinvest in an inhouse implementation of Big Data technologies. Enterprises whose core business runs on data analysis need to continuously invest in data technologies to extract the maximum possible business value from their data. However, for enterprises that are still beginning or in the infancy of their Big Data journey, investing in a cutting edge technological solution is almost always the wrong strategy. Enterprises should focus on small wins using as much off the shelf components as possible to quickly reach the point of Big Data ROI offered out of customization free, off the shelf tools. When possible, enterprises should offload infrastructure operation and management to third party vendors while experimenting with applications and solutions that utilize these Big Data technologies. This ensures that critical resources are spent on solving real customer problems while critical feedback is being collected to inform future technology investments.

Technology Choices Without Business Impetus Are Not Ideal

The Big Data technology your business needs can vary by the problem that you are trying to solve. The needs of your business and the type of problems that you need to solve to offer simple, trustworthy and efficient products and services to your customers should determine and lead you to the right Big Data technology and vendors that match your needs. Enterprises need to focus on the business questions that need to be answered as opposed to the technology choice. Enterprises who do not have the business focus will spend crucial resources on optimizing their technology investments as opposed to solving real business problems and end up with little ROI. Planning and implementing Big Data technology solutions in a vacuum without clear problems and intended solutions in mind not only can lead to incorrect choices but can lead to wasted effort spent prematurely optimizing for and commitment to a specific technology

Evangelize Analytics Internally To Better Understand Technology Requirements

Appropriate Big Data technology decisions can only be made by ensuring that the needs and requirements of the various parts of the organization are correctly understood and captured. Ensuring the that culture in the enterprise promotes the use of data to answer strategic questions and track progress can only happen if analytical thinking and problem solving are used by all functions in the organization ranging from support to marketing to operations to products and engineering. Having these constituents represented in the technology stack decision process is extremely critical to ensure that eventual technology is usable and useful for the entire organization and does not get relegated to use by a very small subset of employees. In addition, the specific needs of certain users such as data exploration, insights generation, data visualization, analytics and reporting, experimentation, integration or publishing often require a combination of one or more technologies. Defining and clarifying the decision making process in an enterprise is needed to identify the various sets of technologies that need to be put together to build a complete data pipeline that is designed to enable decisions and actions.

All (Big Data) Roads Lead to Your Customers

For enterprises that are struggling to get started with Big Data analysis or have moved past the initial exploration stage in Big Data technology adoption, deciding what problems to tackle initial that will offer the highest ROI can be a daunting task. In addition, there is often pressure from management to showcase the value of the Big Data investment to the business, customers and users of the products and services. Almost always, focusing on improving customer/user satisfaction, increasing engagement with and use of your products and services mix and preventing customer churn is the most important problem that an enterprise can focus on and represents a class of problems that is 1. Universal 2. Perfect for Big Data analysis. As customers and end users interact with the enterprise’s products and services, they generate data or records of their usage. Because customer actions can be almost always divided into two sets: Transactional actions that represent a completed monetary or financially beneficial actions by the user for an enterprise. e..g purchasing a product or printing directions to a restaurant and Non Transactional, Leading Indicator Actions that by themselves are not monetarily beneficial to the enterprise but are leading indicators of upcoming transactions. e.g. searching for a product and adding it to a cart, reviewing a list of restaurants. Being able to tag the data generated by your users by the following metadata generates an extremely rich data set that is primed for Big Data analysis. Understanding the frequency of actions, time spent, when the actions occur, where they occur, on what channel and the environment and the demographic description of the user who carries out the action is critical. At the minimum, enterprises need to understand the actions of their users that correlate the highest with transactions, the attributes and behavior patterns of engaged and profitable users and the leading indicators of user dissatisfaction and abandonment, There are other very obvious applications of Big Data in the areas of security, fraud analysis, support operations, performance etc however each of these applications can be traced directly or indirectly to customer dissatisfaction or disengagement problems. Focusing your Big Data investments into a holistic solution to track and remedy customer dis-satisfaction to improve engagement and retention is a sure fire way to not only design the best possible Big Data solution to your needs but also to extract maximum value from these investments that impact your business’s bottom line.