Pragmatic Big Data for the App Economy

Meeting key business objectives (in the context of a platform strategy) such as increasing market share or profitability in the enterprise’s niche can be achieved by increasing the enterprise’s user base, the user engagement and improving the product mix.

Strategic Objectives

Big data analysis of the large volumes of data being generated in the app economy is key to designing and executing strategies that can help enterprises meet their strategic objectives.

Entities in the API Ecosystem such as end users, apps, developers, API and backend systems are continuously generating streams of data within the API value chain and outside the API value chain. For example, App Users are not only using their apps and APIs but sharing opinions on social media, looking for content on the internet and interacting with products andservices in the physical world. These continuously growing streams of data contain hidden signals that hold the key to meeting the enterprise’s strategic objectives.

Enterprises looking to gain a competitive edge in the app economy have the opportunity to harness the power of big data and contextual analytics to solve three key problems that directly offer increased profitability & market share through higher usage of their products and services and higher user satisfaction.

Understanding End Users

End users drive value in the API value chain. It is critical that the enterprise understand the behavior and actions of their end users and why users act and behave the way they do. It is critical that the enterprise is able to segment their end users by product usage, by value they drive to the bottom line and by how engaged these users are with their products and services.

Offering sticky products and services requires that the enterprise understand the value that end users are looking for in the products and services. Enterprises need to understand their desired profile of the end user. The desired profile is an intersection of the end users who find value in the enterprise’s products and services and the end users that are profitable for the enterprise.

Attracting the Best Developers

Developers are key to building great apps across a diverse use case set. A diverse app set attracts a broader set of end users directly translating to more end users, higher usage and higher profitability for the enterprise. Enterprises do not retain control over the end user experience on apps written by third party developers. This makes it critical for the enterprise to attract, detect, nurture and promote developers and apps that offer the best user experience and the best value for the enterprise.

Attracting the best developers requires enterprises to understand their desired developer profile, their current developer profile and emerging trends in the developer world being promoted and embraced by the “early adopter” developers. The ability to adapt and embrace these emerging developer trends can increase the attractiveness of an enterprise’s platform.

Enterprises need to understand how their developers communicate with them and how and where developers hang out or seek support. The ability to keep a tab on all developer communication avenues, quickly gather areas of dissatisfaction and move to address and quelch any unhappiness can be the difference between retaining the best developers who build the best, most profitable and desirable apps.

Monetizing Data

Understanding the value of an enterprise in eyes of end users, developers, partners and other enterprises is critical for building new and innovative products and services, improving existing products and services and building new business models around data monetization. Every time a request is made for an enterprise’s data, the metadata generated around the context of the request event can offer deep strategic insights into where value is concentrated in an enterprise’s data set.

Understanding and monetizing data requires enterprises to extract and process to build comprehensive accounting mechanisms across all data access mechanisms enabled through APIs, Apps and other data transfer mechanisms. Enterprises need to be able to understand the end user intent and request type metadata to determine the highest value data and data enabled use cases.

Conclusion

The App economy fueled by the shifting enterprise edge is and is expected to produce increasingly diverse and disparate streams of data that provide a wealth of opportunity for enterprises to apply big data and contextual analytical principles to solving the key problems in the app economy.

An enterprise’s competitive edge depends on their ability to uncover deep insights through platforms like Apigee Insights that offer the ability to gather, model, analyze the app economy data, generate insights, act on these insights, observe the change and adjust if necessary andmaking strong progress towards the enterprise’s strategic objectives.

Anatomy of a Retail API Program

Published Originally on Apigee.com

API programs have become commonplace at nearly all big retailers who offer multi-channel experiences to their customers through mobile apps, in-store kiosks, the Web, and personalized in-store services. Analyzing the anatomy of a typical retail API program uncovers some interesting patterns. The data here was gathered by analyzing several retail API programs that use the Apigee platform.

The No. 1 motivation for a retail API program

Providing a differentiated value proposition in the physical retail channel is almost always the largest motivation for retailers. The most popular functionalities in retailer apps are those that complement the end user’s experience in the physical retail channel.

The top functionalities were  “enabling mobile consumption” and “driving foot traffic,” followed by “personalization,” “product information,” and “driving customer engagement.”

Trends in retail API programs

Retailers are using their API programs to personalize user experiences and enhance customer service. A majority, for example, use their API programs to employ recommendation features. We also found that:

• 40% of retail API programs had personalized alerts and notifications
• 55% included recommendation features
• 30% offered shopping cart management features

Retail API programs also tend to exhibit high agility, an expansive breadth of API services, and heavy policy use.

Primary retail services exposed as APIs

Certain services exposed as retail APIs rise head-and-shoulders above the rest in popularity.  The most common retail services exposed in retail API programs are:

• Store locator services, which enable users to discover physical store locations using their current address

• Product catalogs, which enable users to search, discover, and learn about products and services

• Order services, which enable users to place and check the status of orders

“Identity”—the set of APIs that enable users to sign up, log in, or access account details—also ranked high, but we do not view it as a primary retail service that is core to this vertical.

Retail API programs: developers, apps, APIs, and policies

Our research also unearthed some interesting statistics about retail API programs: they tend to have large developer teams, a broad app portfolio with a diverse set of apps, and heavy usage of policies across a fairly significant array of APIs. Here are some average values for retail API programs, across a variety of categories (The most successful API programs posted much higher values in these areas):

• Average number of developers: 154

• Average number of apps: 87

• Average number of policies: 378

• Average number of APIs: 19

• Average number of policies per API: 16.8

App and API development

Another characteristic of successful retail API programs is the rapid development and improvement of retail APIs. Agility is defined as the number of API revisions divided by the API age in months, and it’s a key indicator of success in retail API programs. We found the average agility among the customers in our sample set to be 13.6, with a maximum of 36 and minimum of 2.

Focus on end user experience

Top retail API programs offer a much faster experience to app users. These retailers spend considerable effort on optimizing their backends and proxies using caches and other features. We found fastest average backend response time to be 20 milliseconds, with 563 milliseconds being the slowest. The average backend responsiveness in our sample was 224 milliseconds; this average is 209 million seconds at the top retail API programs.

What’s your API’s Cachiness Factor?

Published Originally on Apigee

“Cachiness factor” is the degree to which your API design supports the caching of responses. Low cachiness means that a relatively higher than optimal number of requests is forwarded to the back end for retrieving data; a high cachiness factor means that the number of requests serviced through the cache layer is reduced and optimized.

Every time a request is sent to the API provider endpoint, the provider incurs the cost of servicing the request. Investing in a good caching mechanism reduces the number of requests that hit the endpoint, leading to a faster response time, lower servicing costs and saved bandwidth. Resources can then be spent on servicing requests that otherwise would have had to compete with cacheable requests.

Cachiness in an API design refers to understanding how a piece of retrieved data can be reused to serve other API requests. Such an understanding can be transformed into a set of actions that store the retrieved copy of the data in an optimal form for reuse.This coupled with insights from API usage analytics can provide direct benefits in terms of app performance and operational costs.

An API proxy can be designed to do a number of things when a request arrives:

Determine the quality or fidelity of the data requested by the app or end user

This information can then be used to
– Transform the API request to retrieve the data from the endpoint data store at the highest possible fidelity and breadth
– Save the retrieved data in the proxy cache
– Extract the appropriate fidelity and breadth (determined by the original request) and send as the response to the app/end user

For example, if the request is for weather patterns for a city, the system can potentially map the response to a response for all zipcodes in that city and store it accordingly in the cache.

Pre-fetch based on temporal and spatial locality
Predict based on usage patterns what the next request is likely to be and pre-fetch this data from the endpoint for storing in the cache.

For example, given a request for browsing a list of plasma TVs on sale at a retailer, it might make sense to cache the entire response set and serve subsequent requests for more data (e.g. the next set of TVs) from the cache.

Pre-fetch based on similarity
Use the idea of similarity in data sets to predict the next request and retrieve the data for storing at the proxy cache.

For example, for the scenario in which our user requests a list of TVs from one manufacturer, it might make sense to pre-fetch a list of TVs from another manufacturer with a similar product line and store this information in the cache.

Parameters-based selection
If your API supports “select” on your data through parameters, another option to optimize the cachiness of your API is to retrieve the entire data set (within certain bounds) from the backend, store it in your cache, and return only the appropriate data set for the request. Similarly, filtering of data can be performed at the proxy as opposed to the end point, increasing the cachiness factor for the API.

Using Data Analysis to improve cachiness
You can also use data analysis techniques to understand request patterns for your data and use this information to pre-fetch or over-fetch data from the endpoint to increase the cachiness factor of your API.

Caching Diffs
Another possible technique is building a mechanism where updated data is automatically sent from the end point data store to the cache as new updates are generated in the backend. At the cache level, instead of expiring the entire data set, the part of the data set that is least likely to be relevant is automatically expired and the new updated “diff” is appended to the cache data set.

The technique that will work for an API will vary from API to API. You might need to experiment with various techniques to identify the one that makes sense for your scenario and API.

Protecting users, apps, and APIs from abuse

Published Originally on Apigee

Abusers or spammers are the bad guys looking to make money by getting unsuspecting end users or consumers of online services to interact with malicious content or spam that leads to one or more of the following scenarios:

• Eyeballs on spam content that lead to clicks and purchases.
• Gathering users’ private information through keyloggers (or other spyware) on the user’s machine or device which is then sold to the highest bidder.
• Phishing for users’ private information such as SSN, credit card #, or passwords and selling those to the highest bidder.
• Installing malicious software on users’ machines or devices, which in turn steals more of their information or uses their bandwidth or storage for carrying our further attacks.

Any workflow that creates or consumes content, shares or reshares content, sends or receives communications can be vulnerable to attack.

Online attacks almost always contain a “payload” that either delivers the attack or leads the user to another location where the attack is completed. Any time a piece of content is created or consumed or a communication is sent or received, there is an inherent “payload” that is delivered. In the wrong hands, this payload can be malicious.

Blogs can be spammed with comments which contain spam or malware, or which employ phishing techniques that redirect users to other malicious locations on the Internet.

In the same way, users can receive emails that contain spam or malware or other phishing techniques. Users can be redirected to malicious sites in their browser. Users’ machines can be attacked and malware (such as adware, spyware, key loggers and viruses) installed which can scrape users’ private information and send it to the abusers.

Thinking about apps and APIs as assets

Apps like APIs can be considered “assets” that can be used to carry out attacks against end users.

APIs (and especially free APIs) provide services that eventually reach other users. Even if an API has a usage cost associated with it, if the return on investment for an abuser for planning and carrying out an attack is higher than the cost of using the API (at scale), even paid APIs can be abused.

Both developers and enterprises have apps that are highly monetizable if abused. App-based attacks fall into the following general categories:

Malicious apps: These apps are written with the sole goal of abusing unsuspecting users who download and install them. With the increasing use of single sign on services and integration of social networks with apps through social network platforms, a user’s social network can be used to propagate these attacks.

Well-intentioned apps with vulnerabilities: These apps are not written to be malicious but have vulnerabilities either in their code (e.g. the apps’s functionality can be scripted or controlled through hooks provided by the app) or in their business logic (e.g. no throttling of calls from a single user to the back-end system). An abuser can exploit these vulnerabilities to carry out attacks on enterprise APIs or on the end users of the apps.

Well intentioned apps exploited by a malicious user: In this case, an abuser can use some legitimate capability offered by the app as a foundation to carry out attacks that propagate through the end user’s social network and spread through further usage.

Protecting your assets

You don’t want your APIs or apps used for abuse. It is crucial for both developers and enterprises to protect their “assets”. If an end user develops a perception that a certain app or service is “dangerous”, usage declines, growth can be reversed, and revenue and profit suffer.

Remember that abusers are most often out there to make money and if the cost to carry out an attack is less than the value derived from the attack, it will continue to be a sound business investment for the abusers.

Some things that developers can do to protect their apps and end users (and their friends)

• Monitor usage of your app and its impact on the users. Build and maintain a proxy for user reputation and models of good and bad behavior in the content of the app.
• Enable end users to report suspicious behavior of the app or of other users of the app.
• Work with the enterprise or API provider to ensure that your app is not creating suspicious loads on the API.

Some things that enterprises can do to protect their APIs

• Build traffic monitoring solutions and models to rate traffic as safe or abusive.
• Ensure that any content or communication created or shared through the API is free of malicious payloads.
• Invest in mechanisms to report and notify suspicious user and app behavior to the app developers.
• Build reputation systems for users, content and IP addresses to be able to quickly classify traffic, users and apps as good or bad OR desirable or undesirable.

Next time we’ll look at some of the ways to push back against attackers, including use of quotas, throttling, and so on.

Measure What Matters: Six Metrics Every CDO Should Know

Published Originally on Apigee

The chief digital officer has to juggle multiple priorities, foci, and investments, ranging from within the enterprise to its edge. Having been charged with growing an enterprise’s business, CDOs need to enable their companies to successfully undergo a digital transformation. This digital transformation includes enabling the enterprise to plan, build, and maintain products as well as market to, sell to, acquire, retain, and support users through digital or digitally enhanced products and processes.

Defining a successful digital business strategy requires a deep understanding of users’ preferences and behaviors and also requires the ability to track changes in user behavior and their consumption and demand patterns. Understanding users’ preferences and behaviors requires the ability to track customer behavior over time and across channels. Tracking changes requires a set of analytics that enables the CDO to measure, at an aggregate level, the behavior of segments and micro-segments of users and also to understand, at the individual level, the current state, engagement, and problems faced by a user or a partner.

Building a digital enterprise requires the ability to track and accelerate innovation, agility, and experimentation in the enterprise. Democratizing access to data and building-block services for developers requires a systematic audit, curation, and exposure of enterprise capabilities as reusable APIs with the ability to track, monitor, and aid the usage of such services by developers and partners efficiently, quickly, and successfully.

Here are the six dimensions of an analytics plan that a CDO should build and track to enable better decision making.

Business KPIs

A CDO’s main goal is to grow the enterprises’ business. To achieve this, the CDO must track two key types of business KPIs: traditional business KPIs and digital KPIs.

Traditional Business KPIs are those that the enterprise uses to run the business, such as customers, average revenue per user (ARPU), churn, and revenue/profit.

Digital KPIs include traffic and revenue from digital touchpoints and the total and rate of acquisition of new users, customers, developers, partners, and devices. Tracking business KPIs involves tracking both the absolute numbers and the trends, which can signify changing consumption and demand patterns and serve to alert the CDO about potential problems with customer satisfaction or the services supply chain.

Specifically, a successful CDO will:

• set up organizational structure and processes to understand and attribute KPI changes to market, competitive, or product forces

• define marketing and product strategies to drive usage, revenue, customer, developer, and partner acquisition and retention

CDO’s Business KPI Dashboard

Digital Transformation

Digital transformation can be defined as, and measured by, the acceleration of innovation and agility in an enterprise that ultimately leads to new, compelling user experiences and is marked by higher usage and revenue.

CDOs should measure the following aspects of digital transformation to detect organizational and personnel challenges around innovation, process roadblocks that hurt agility, and a lack of crisp product and/or platform positioning that impacts both reach and the level of partner and developer engagement.

Innovation: The ability of the enterprise to bring new, compelling products and services to market, measured in the APIs and apps delivered to users. New products can be defined as new products for existing users, products designed to attract new users, or new markets and products designed to attract users of competitive products.

Agility: The ability of the enterprise to improve its products and services, measured by the rate of improvement of apps and APIs. In other words, how quickly can an enterprise expose its services as APIs and how quickly and easily can these APIs be adopted by developers and consumed by apps?

Reach: The ability of the enterprise to attract new users, developers, and partners to their platform, products, and services, measured by the rate of new user, developer, and partner acquisition and the churn rate of these users, partners, and developers.

Time to Maturity: The time taken by APIs and apps to “go live” and be used by real users, as measured by the time from the first definition of the app or API to when it is available for consumption.

Partner and developer engagement: Developer and partner engagement with the enterprises’ platform as measured by the rate and breadth of platform features usage over time, including the rate-of and time-to success of developers and partners.

Ecosystem density: The measure of the “consumption” and “supplier” relationships that an enterprise has with other businesses (through APIs). Let’s take as an example an API that allows you to send photos to print from your mobile device. When used by and offered from services like Shutterfly, Flickr, and Instagram, for example, this API is the core of a much denser and robust ecosystem than if it were being used solely by any single app or website.

Similarly, say an app were to consume not only the print API but also APIs that offer users related services, such as viewing photos online and creating albums and slideshows. Then that app offers a richer experience to its users than if it only offered the print API functionality. The progress and success of an enterprise’s digital transformation can be measured by the density of the ecosystem—by how connected and how integral a part the enterprise plays in its digital supply chains and how robust the complex partnership models and supply chains are.

Specifically, a successful CDO will:

• audit and optimize organizational setup and process efficiency regularly to understand rates of innovation and agility and identify internal roadblocks
• commision strategies for improvement of developer and partner engagement through new products and services and better support and training
• understand and remove bottlenecks in partner and developer onboarding, including the most common reasons for failed or prolonged onboarding process

CDO’s Digital Transformation Dashboard

Channels

The most pronounced impact of a digital transformation is evident in the changing behavior and transaction patterns of an enterprises’ users. Digital channels sometimes replace or cannibalize traditional channels, but more often they help and enhance multi-channel transactions. As customers navigate and interact with the enterprise across multiple channels, a CDO needs to be constantly aware of the shift in those customers’ product access and acquisition patterns. This awareness is shuttled into strategic investment decisions across channels and often into building bridges between channels to enable easy context switching for users.

Channel awareness is turning out to be one of the key tenets of data-driven decision making for the CDO.

A successful CDO will:

• define product strategies to enable better cross-channel usage of your products and services
• define and design channel-specific workflows and cross-channel workflows to adapt to end user usage patterns
• track
  • traffic and revenue by channel
  • the most common channels where high value transactions begin and end
  • transactions that transcend multiple channels
  • users that use multiple channels to start and end transactions
  • users that shift and change the channels through which they interact with the enterprise

CDO’s Channel Tracking Dashboard

Apps and APIs

The CDO brings the app and API revolution to the enterprise by exposing old and arcane services as reusable, lightweight, and accessible APIs, designed to be consumed by lightweight and purpose-built apps.

CDOs track app and API metrics to understand and track the adoption, engagement, and usage of their products and services and to determine, optimize, and fine-tune investment decisions. Metrics include revenue, traffic, QoS, unique users, ratings of apps (consumer) and APIs (partner/developer), active apps, devices, and geo-distribution of traffic and revenue.

A successful CDO will:

• track and understand trends and changes in KPIs, inlcuding unique users, usage, and app ratings to implement product strategies to build better products
• use KPIs as an impetus to explore new market and customer segment opportunities and make timely investment decisions

CDO’s Apps and APIs Dashboard

Developers and Partners

Developers and partners are the channels to grow the enterprise. A healthy developer community and a diverse partner ecosystem is a sign of a thriving enterprise and a leading indicator of digital success. CDOs should measure the cost and likelihood of developers and partners successfully onboarding to their platform and launching new and innovative apps that are desired and used by users. Metrics such as cost of developer acquisition (CODA), partner onboarding success rate, and partner onboarding time are key to tracking the health of the developer and partner community. At any point, the CDO should have information about the revenue and traffic from a partner/developer, the QoS experienced by the partner/developer, apps built by these partners and developers, and the unique users delivered via these apps. This information is used by the CDO to fine-tune developer/partner onboarding processes and to craft marketing strategies that attract, retain, and engage developers and partners.

A successful CDO will:

• define strategies to reduce cost of developer and partner acquisition and onboarding
• remove bottlenecks and provide a better developer experience to strengthen platform adoption

CDO’s Developers and Partners Dashboard

News: Internal, Ecosystem, and External

Last but not least, CDOs need to stay abreast of relevant news and information that impacts their industry, ecosystem, enterprise, organization, or specific app or API team. CDOs should track how their apps and APIs are being talked about on social media, and listen to learn about product or service issues that are likely to cause dissatisfaction for developers, partners, and users. In addition, they should closely track how new releases and versions of their services, APIs, and apps impact usage and revenue.

A successful CDO will:

• manage the perception of a business’ products and services on social media and arrest and address negative trends and user and developer dissatisfaction
• design and implement market and competitive research pipelines to uncover new trends and changing end-user behavior patterns

CDO’s News & Releases Dashboard

Conclusion

A CDO is tasked with a challenging job: to be the chief digital strategist for the enterprise, and to shake up an enterprise and make it digitally relevant and able to successfully adapt to changing user preferences, behavior, expectations, and access patterns. A data-driven approach and culture is the best asset that a CDO can nurture in the enterprise to make objective decisions and track the impact of those decisions and actions.

If you are a CDO, we would love to hear from you about analytics and other techniques that you are using to bring about the digital revolution in your enterprise!

3 Signs That Your Partner Program Is Going Belly Up

Published Originally on Entrepreneur.com

While launching partner programs can offer many benefits to businesses, if not set up correctly, they can also spell disaster.

For those a little confused about what exactly a partner program entails, it is basically a formal program and process operated and managed by a business with the goal to attract, engage and retain other partners. For example, Google offers an advertising service to publishers and advertisers. The ultimate goal of these programs is to increase revenue generated from these partners. In addition to the services offered to partners (i.e. advertising platform), a partner program includes trainings, tools, support, documentation, help and strategic services to empower partners to succeed.

For a program to be considered a success, there should be a strong partner membership, robust usage of the services offered and large stream of revenue generated per partner.

If your partner program is struggling to stand on its legs, you may be making one these three following mistakes:

1. Your return on investment is low or not being measured.

Without clear metrics and goals, most partner projects end up getting defunded as the return on investment (ROI) is simply not there (or not measured) to justify continued investment.

Another symptom of a flawed partner strategy is a misalignment of expectations between senior management/executive sponsors of the platform and the actual implementers. If ROI does not match the expectations of the sponsor, a partner program can be classified as a failure.

To ensure everyone is on the same page when it comes to ROI, determine metrics right off the bat. To do so, companies need to know exactly what metric is important to them and to their partners. For example, the rate at which partners sign up or the time it takes for a partner to go from onboarding to generating revenue might be relevant metrics. Companies need to understand and establish the baseline for these metrics and then measure the efficacy of their program by ensuring the metrics move in the right direction.

Once you have figured out metrics, make sure your platform can deliver. A well-designed service platform makes it easier and cheaper for other teams in the enterprise to go to market with new services, products. If the cost to enhance and expand the platform is not getting smaller for each additional new service, your program might be in trouble.

2. You have built an ineffective service platform.

If your partner program has struggled to attract, retain and enable partners to develop and grow their businesses, it is an ineffective service platform.

To fix this, you have to make your platform sticky, meaning partners cannot succeed without your services. If this isn’t the case, you have a serious problem.

For example, if your partners are signing up to work with both you and your competitors, it might be because they consider you a risky investment. If your partners build only a single one-off application on your platform, your program is probably not on their investment roadmap and growth strategy.

One company that does offer a sticky program is Microsoft’s BizSpark. Initially, startups can build on the platform for free but after a certain time frame – and after a lot of resources have been built on it – Microsoft starts charging. And this creates a huge revenue stream for the company, as what startup is going to want to jump off the BizSpark platform and start over?

3. Your monetization strategy is all wrong.

If you are not making money off of your partner program or generating leverage from its usage, your monetization strategy needs to change.

The key to successful monetization is identifying and expose only products and services that have a high demand from partners. For example, a great tactic may be to survey your potential partners or analyze the needs of your targeted partner persona. This will inform you about the matching subset of services in your arsenal that offer enough value to the partners that they will pay for it.

If you not thinking about your program monetization by investing in marketing, pricing optimization, adoption and growth or performance management of the platform, you are probably stuck with a stagnant partner program with sluggish usage and high churn rate. A successfully monetized partner program requires that it be constantly analyzed and optimized.

If your partner program is an IT initiative and does not have a business team behind it thinking about its ROI, effectiveness and monetization, it will not only struggle to get adopted but also starve for investment dollars from your executive sponsors and revenue dollars from your partners. As these investment dollars dry up, the partner program that runs on top of the platform will be considered a failure and will eventually shrink, get branded as a failure and cease to exist.

The key to success with a partner program strategy begins with carefully selecting what your program offers, convincing and onboarding partners and doing everything and anything to make them successful. Once that is the case, you can monetize the partner program and as you open up new revenue streams for your enterprise, see the program grow, expand and become a line of business.