Virtual Sensors and the Butterfly Effect

Originally Published on Wired.

In the early 1960s, chaos theory pioneer Edward Lorenz famously asked, “Does the flap of a butterfly’s wings in Brazil set off a tornado in Texas?” Lorenz theorized that small initial differences in an atmospheric system could result in large and unexpected future impacts.

Similar “butterfly effects” can surface in the increasingly interconnected and complex universe of enterprise partnerships and supply-chain and cross-product relationships. It’s a world where new or evolving products, services, partnerships, and changes in demand can have unexpected and surprising effects on users and other products, services, traffic, and transactions in a company’s ecosystem.

Monitoring these complex relationships and the potentially important changes that can reverberate through an enterprise’s network calls for an interconnected system of virtual “sensors,” which can be configured and tuned to help make sense of these unexpected changes. As enterprises increasingly interface with customers, partners, and employees via apps and application programming interfaces (APIs), setting up a monitoring network like this becomes a particularly important part of data analysis.

What are Sensors?

Traditional sensors are often defined as “converters” that transform a physically measured quantity into a signal that an observer can understand. Sensors are defined by their sensitivity and by their ability to have a minimal effect on what they measure.

Physical sensors can capture aspects of the external environment like light, motion, temperature, and moisture. They’re widely used in business, too. Retailers can employ them to measure foot traffic outside or inside their stores, in front of vending machines, or around product or brand categories. Airlines use physical sensors to measure how weather patterns affect boarding and take-off delays. Using a diversity of sensors enables the definition of an environment around the usage of a product or service in the physical world.

Besides investing in traditional data processing technologies, cutting-edge enterprises map their digital world by defining and building so-called virtual sensors. Virtual sensors collect information from the intersection of the physical and digital worlds to generate and measure events that define the usage of a digital product or service. A virtual sensor could be a data processing algorithm that can be tuned and configured to generate results that are relevant for the enterprise. The generated alert notifies the enterprise of a change in the environment or ecosystem in which the user is using a product or service.

How to Build a Virtual Sensor Network

Building a network of virtual sensors for your business calls for requirements similar to those of a physical sensor system:

  • Sensitivity, or the ability to detect events and signals with configurable thresholds of severity
  • Speed, or the ability to speedily collect and process signals to generate business-critical events
  • Diversity, or the ability to collect, collate, and combine signals from multiple sensors with the goal of generating business-critical events

To begin charting the web of relationships that impacts the demand and usage of various enterprises’ products and services, businesses should determine which other products and services in the marketplace are complements, supplements, and substitutes to their own. Deep understanding of such evolving and complex relationships can help enterprises with planning partnerships.

  • Supplementary products and services enhance the experience of another product or service. For example, flat panel TVs are enhanced by wall mounts, stands, warranty services, cable services, and streaming movie services.
  • Complementary products and services work in concert with other products and services to complete the experience for the end user. Demand for car tires, for example, tends to generate demand for gasoline.
  • Substitute products and services have an inverse effect on each other’s demand. For example, two retailers offering the same selection of products targeted to the same consumer.

Understanding these relationships is the starting point of creating a network of sensors to monitor the impact of changes in traffic or transactions of an outside product or service on an enterprise’s own products and services. Detecting this change within the appropriate sensitivity can often be the difference between an enterprise’s failure or success.

Take for example, a web portal that aggregates content from several content providers. This portal uses APIs to connect to these third-parties. In many cases, these content providers are automatically queried by the aggregator, regardless of whether an end user is interested in the content. If for any reason there is a spike in usage of the portal on a particular day, this will automatically trigger spikes in the traffic for each of the content providers. Without understanding the complementary connection to the portal and the associated shifting demand properties of the connection, the content providers will find it difficult to interpret the traffic spike, which will eat up resources and leave legitimate traffic unserviced.

Here’s a similar example. Let’s say a service can support 100 API calls spread among 10 partners. If this service receives an unexpected and unwanted spike in traffic from one partner that eats up half of its capacity, then it will only have 50 API calls left to distribute among the other nine partners. This in turn can lead to lost transactions and dissatisfied users.

With an awareness of the network, however, the service would understand that this one partner routinely only sends 10 calls on a normal day, and would be able to put restrictions in place that wouldn’t let the extra 40 calls eat up the capacity of other partners.

In these kinds of situations, virtual sensors can provide the awareness and insights into this web of interdependency, and help make sense of traffic spikes that otherwise might seem incomprehensible.

Sensor-Aware Data Investments

Building a network of physical and virtual sensors entails collecting diverse signals from a complex map of data sources and processing them to generate events that can help enterprises understand the environments around their end users. Investing in these networks enables enterprises to track and monitor external signals generated from sources that have the ability to impact the enterprise’s traffic, transactions, and overall health.

This ability, in turn, helps digitally aware businesses negate potential troubles caused by the digital butterfly effect, and take advantage of the opportunities presented by a strong grasp of what’s happening in user and partner ecosystems.

How Data Analysis Drives the Customer Journey

Originally Published on Wired

Driving down Highway 1 on the Big Sur coastline in Northern California, it’s easy to miss the signs that dot the roadside. After all, the stunning views of the Pacific crashing against the rocks can be a major distraction. The signage along this windy, treacherous stretch of road, however, is pretty important — neglecting to slow down to 15 MPH for that upcoming hairpin turn could spell trouble.

Careful planning and even science goes into figuring out where to place signs, whether they are for safety, navigation, or convenience. It takes a detailed understanding of the conditions and the driving experience to determine this. To help drivers plan, manage, and correct their journey trajectories, interstate highway signs follow a strict pattern in shape, color, size, location, and height, depending on the type of information being displayed.

Like the traffic engineers and transportation departments that navigate this process, enterprises face a similar challenge when mapping, building, and optimizing digital customer journeys. To create innovative and information-rich digital experiences that provide customers with a satisfying journey, a business must understand the stages and channels that consumers travel through to reach their destination. Customer journeys are multi-stage and multi-channel, and users require information at each stage to make the right decisions as they move toward their destination.

Signposts on the Customer Journey

To understand what kind of information must be provided — and when it must be supplied — it’s important to understand the stages users travel through as they form decisions to purchase or consume products or services.

  • Search: The user starts on a path toward a transaction by searching for products or services that can deliver on his or her use case
  • Discover: The user narrows down the search results to a set of products or services that meet the use case requirements
  • Consider: The user evaluates the short-listed set of products and services
  • Decide: The user makes a decision on the product or service
  • Sign up/set up: The user completes the setup or sign up required to begin using the chosen product or service
  • Configure: The user configures and personalizes the product or service, to the extent possible, to best deliver on the user’s requirements
  • Act: The user uses the product or service regularly
  • Engage: The user’s usage peaks, collecting significant levels of activity, transaction value, time spent on the product, and the willingness to recommend the product or service to their professional or personal networks
  • Abandon: The user displays diminishing usage of the product or service compared to the configuring, active, and engaged levels
  • Exit: The user ceases use of the product or service entirely

Analyzing how a customer uses information as they navigate their journey is key to unlocking more transactions and higher usage, and also to understanding and delivering on the needs of the customer at each stage of their journey.

At the same time, it’s critical to instrument products and services to capture data about usage and behavior surrounding a product or service, and to build the processes to analyze the data to classify and detect where the user is on their journey. Finally, it’s important to figure out the information required by the user at each stage. This analysis determines the shape, form, channel, and content of the information that will be made available to users at each point of their transactional journey.

The highway system offers inspiration for designing an information architecture that guides the customer on a successful journey. In fact, there are close parallels between the various types of highway signs and the kind of information users need when moving along the transaction path.

  • Regulatory: Information that conveys the correct usage of the product or service, such as terms of use or credit card processing and storage features
  • Warning: Information that offers “guardrails” to customers to ensure that they do not go off track and use the product in an unintended, unexpected way; examples in a digital world include notifications to inform users on how to protect themselves from spammers
  • Guide: Information that enables customers to make decisions and move ahead efficiently; examples include first-run wizards to get the user up and running and productive with the product or service
  • Services: Information that enhances the customer experience, including FAQs, knowledge bases, product training, references, and documentation
  • Construction: Information about missing, incomplete, or work-in-progress experiences in a product that enable the user to adjust their expectations; this includes time-sensitive information designed to proactively notify the user of possible breakdowns or upcoming changes in their experience, including maintenance outages and new releases

Information Analytics

Information analytics is the class of analytics designed to derive insights from data produced by end users during their customer journey. Information analytics provides two key insights into the data and the value it creates.

First, it enables the identification of the subsets of data that drive maximum value to the business. Certain data sets in the enterprise’s data store are more valuable than others and, within a data set, certain records are more valuable than others. Value in this case is defined by how users employ the information to make decisions that eventually and consistently drive value to the business.

For example, Yelp can track the correlation between a certain subset of all restaurant reviews on their site and the likelihood of users reading them and going to the reviewed restaurants. Such reviews can then be automatically promoted and ranked higher to ensure that all users get the information that has a higher probability of driving a transaction—a restaurant visit, in this case.

Secondly, information analytics enables businesses to identify customer segments that use information to make decisions that drive the most business transactions. Understanding and identifying such segments is extremely important, as it enables the enterprise to not only adapt the information delivery for the specific needs of the customer segment but also price and package the information for maximum business value.

For example, information in a weather provider’s database in its raw form is usable by different consumers for different use cases. However, the usage of this information by someone planning a casual trip is very different than a commodities trader who is betting on future commodity prices. Understanding the value derived by a user from the enterprise’ information is key to appropriate pricing and value generation for the enterprise.

Information Delivery

Mining and analyzing how users access information is critical to identifying, tracking, and improving key performance indicators (KPIs) around user engagement and user retention. If the enterprise does not augment the product experience with accurate, timely, and relevant information (according to the user’s location, channel and time of usage), users will be left dissatisfied, disoriented, and disengaged.

At the same time, a user’s information access should be mined to determine the combination of information, channel, and journey stage that drives value to the enterprise. Enterprises need to identify such combinations and promote them to all users of the product and service and subsequently enable a larger portion of the user base to derive similar value.

Mining the information access patterns of users can enable enterprises to build a map of the various touch points on their customer’s journey, along with a guide to the right information required for each touchpoint (by the user or by the enterprise) in the appropriate form delivered through the optimal channel. Such a map, when built and actively managed, ends up capturing the use of information by customers in their journey and correlates this with their continued engagement with — or eventual abandonment of — the product.

Enabling successful journeys for customers as they find and use products and services is critical to both business success and continued customer satisfaction. Contextual information, provided at the right time through the right channel to enable user decisions, is almost always the difference between an engaged user and an unsatisfied one — and a transaction that drives business value.

When Your Product Design Makes Your Customers Feel Smart

Published Originally on Entrepreneur.com

Users love products and services that make them feel smarter. The more efficiently they can spend their valuable attention, time and money, the smarter they feel. The smarter that users feel when interacting with your product, the more they love it. We call this the smart-user theorem.

Strong examples of the smart-user theorem in action abound. Facebook and Instagram save users time by enabling them to connect and share with friends and family quickly and efficiently. Similarly, apps have become popular and ubiquitous, partly because of their availability to fulfill virtually any need or task.

The simplicity of the interface and the entire value chain on the iPad, the ease of planning a trip on Expedia via a mobile device or using Dropbox to store files — these are more examples that offer powerful guiding principles for enterprises as they engage customers with their products and services.

Taking this a step further, analyzing customers’ behavior can quantify the time, attention or effort required to engage with a business’ products and services and bring about a new understanding of the user experience. This awareness, in turn, arms businesses with strategies to fine-tune their products and services to be more efficient, streamlined and intuitive.

If enterprises carefully evaluate and optimize their products and services to make their users “smarter,” they will be rewarded with loyalty, engagement and a higher transactional value.

User investments: attention, time and money. There are three types of “capital” that customers invest in your products and services: attention, time and money. First, users turn their attention to your messages, advertisements and product communications. They interpret and internalize your message to inform their next steps.

Consumers also spend time thinking about, searching for, discovering, deciding to access, learning about and using your products and services; it’s safe to assume that they spend the same amount of time learning about your competition. Finally, there’s the money part. This one’s pretty obvious: Users pay you for your products and services.

The attention, time and money model provides a framework to optimize the design of the end-to-end user experience. Maximizing the value of the attention, time and money spent by users on your products and services can be achieved through a combination of baselining and experimentation.

Baselining involves breaking up the product-usage flow into logical stages and measuring the time and attempts it takes users to move through it. In addition, the consumer’s reliance on certain information and features should be analyzed to understand whether they encourage a person to move to the next stage in the flow.

Experimentation is the stage whereby, through the use of data analysis or customer interviews, product problems can be identified. Hypotheses are developed and then tested through changes in the product flow until the desired goals are met.

Big data’s role in smarter interactions and smarter users. Users save attention, time and money as a result of personalized and customized messages, which enable them to find the right tools to satisfy their needs quickly at the right cost. Creating these messages and products requires capabilities that the processing of Big Data can easily provide. This can involve the following types of analyses:

1. User-environment analysis, in which information is collected about the environment where users interact with the product or service.

2. User-profile analysis, whereby information is collected about consumers and their characteristics such as gender, age, likes and dislikes.

3. User-interaction analysis, in which data is collected about users’ activities and behaviors as they interact with a product along the customer journey.

4. User modeling, whereby data is collected and modeled to represent the behavior of a segment or cohort of users.

The analyses and subsequent correlations are used to optimize the messages delivered to users according to their environment, profile and behavior patterns, as well as their stage in the customer journey.

As users receive personalized messages and information that enable them to be smarter by helping them complete their tasks faster, more inexpensively and with less attention, the overall value realized from the product or service increases. This leads to higher productivity for the user, higher and sustained engagement with the product or service, a customer who feels smarter and, in the end, greater value for your enterprise.