Good Architecture: Extensibility (a.k.a. How I used polymorphism to convert a lead-gen project into a service responsible for $1Bn of value)

“Write Once, Re-use Everywhere” takes more than publication of APIs

We have all heard the mantra, “write once, re-use everywhere,” to describe the concept of creating technology that can not only “work” on every operating system (the mantra of Java) but also used ubiquitously to create value in many places. When I ask people how they aim to achieve this, I usually hear the following recipe:

  1. Build resources (modules or applications) that can be invoked remotely
  2. Front them with APIs to invoke these externally—without a need to understand what happens internally
  3. Publish these APIs
  4. Open access to these APIs and resources to external users (or setup simple operations that enable external users to quickly and easily obtain access)

I then usually hear an explanation of the exact technology this will use (e.g., RESTful services, published WSDL) followed by a deeper dive into implementation details…

However, this only gets you what is describes: services that can be accessed by many parties. It does not guarantee that you get services that are useful to others, i.e., services people will want to use to create value. To achieve this, you have to incorporate the principle of Extensibility into every aspect of design, development and operation.

Why should I care about extensibility?

Innovation always takes directions that we cannot imagine. Just look at the following quotes:

“This telephone has too many shortcomings to be seriously considered as a means of communication. The device is inherently of no value to us.” – Western Union internal memo (1876)

There is no reason why anyone would want a computer in their home.” – Ken Olsen, Founder, Digital Equipment Corp. (1977)

When unexpected turns in innovation occur, where to you want to be: ready to exploit it or racing to rebuild your technology:

“Luck is what happens when preparation meets opportunity.” – Seneca the Younger, Roman Philosopher (First Century CE)

Embedding extensibility into your technology (from Day 1) ensures preparation for opportunity.

Depicting extensibility in the constitution for good architecture

Extensibility was the third article in the Constitution for Good Architecture I wrote ten years ago. It built upon the concepts of Modularity & Encapsulation and Isolation of Logic, adding principles that enable these modules to be easily re-used and extended to places the developer could not even imagine when writing the module or application.

Article. III. Extensibility

Section. 1.

Solution providers shall polymorph module functionality to enable re-use and expansion by multiple client modules and applications, current and future, without significant internal code changes. Significant code changes are those, which change the underlying data sets and core functions used by all entities or modules.

Section. 2.

Solution providers shall polymorph module interfaces to enable re-use and integration across multiple access channels (e.g., browser, call center client, mobile device) without significant internal code changes

Section. 3.

To prevent inadvertent or unauthorized use, all modules shall use partition available functions and interfaces based on credentials provided by the invoking user. User credentials define who, what and where the user is (e.g., user identity, role, type, location.)

You will (again) notice that I avoided use of specific object oriented technologies (as these would be self-limiting). When I wrote this, C# did not exist yet (and C++ was still more pervasively used in many places vs. J2EE.) Today—ten years later—this is reversed (and sites like LinkedIn are built entirely on Ruby on Rails).

Evaluation of extensibility of currently hot technologies

Let’s take a look at the extensibility of some very hot technologies:

  • Apple iTunes: Originally this let you manage music downloads to your PC or Mac, later it expanded to let you manage music on your iPod device. Later it expanded again to let you manage music, videos, contact lists and calendaring on your iPhone, then later it expanded to let you publish and purchase tens of thousands of applications. Overall: Great Extensibility
  • Twitter: Originally a web site to let you publish where the good parties where. Then they launched APIs that let hundreds of developers embed this functionality into their sites and applications. However, it is still doing much of the same thing, essentially a Party Line of streaming messages.” To make it truly extensible, they will need to enable let people use this information more intelligent (e.g., better searches, publication based on patterns of content. Overall: Fair Extensibility
  • Facebook: Originally a web site to share information on Ivy League students. However, not only did they expand to new segments (the “Over 35” segment is their fastest growing one) but they also enable developers to build applications and inject them into the Facebook network. The potential for this to create business (and public) value are enormous (but have not yet fully emerged). The next step is close—enabling these applications to drive leads and commerce or conduct public comment and moderation. Overall: Good Extensibility

My largest success with extensibility

It began with targeted co-registration and up-sell from the call center

About twelve years ago, AOL began to run a cross-registration program out of their call centers. This allowed them to generate extra revenue by up-selling partner products. Ten years ago they had an opportunity to significantly increase this revenue by selling a high-value, pre-qualified product. However, this required the ability to limit up-sell of this product only to people on the pre-qualified list.

A common approach would be to bounce each caller’s name of a daily list of qualified members. However, this would not have let AOL earn nearly as much money as it was adding new members every second. As a result, AOL needed something that could qualify people in real-time. I was asked to come up with a solution

AOL was the kind of place that let us create new technologies and approaches when faced with never-before-seen challenges. As a result, they let me build a new method to enable cross-registration—one that relied on the use of real-time date brokering, profiling and decision-making

The birth of the cross-sell service (a.k.a. “XSELL”)

I was lucky enough to work with a lot of really good, creative people at AOL. As a result they helped me come up with generic service that did the following:

  1. Query what a customer was eligible for
  2. Look up what information was available about a customer
  3. Apply rules to determine – based on this eligibility and demographic information – what was the optimal offer to place in front of the customer

It did all of this in real-time. (I can share how we scaled this to 10,000+ real-time transactions per second – even when many of the third party systems we brokered data from could not scale to these levels.)

Twenty-six-fold expansion in less than six months through extensibility

XSELL by itself was a great service. It was fast (95% of transactions took less than 100 milliseconds) and reliable (less than three errors per million transactions). It enabled business owners to make changes to their rules of eligibility, promotion and optimization using a simple Administrative Tool. This enabled it to provide AOL a usd $83 million (76%) increase in profit in the first year alone.

However the real gem was something I never imaged when I designed this in 1999—the expansion of this to every line of business. This was possible due to XSELL’s inherent extensibility:

  1. It did not care what sources of information it queried for eligibility. This enabled us to ad over a dozen sources of third party information–not just from AOL partners but Time Warner partners as well. (We used proprietary caching technology to hide slow calls to third parties from the member’s experience and to enable us to scale these to AOL levels)
  2. It did not care what types of information it used to make decisions. As such, we could simply add many sources of data to our rules for optimization. This enabled us to incorporate everything from real-time activity logs to third party statistical scores
  3. It did not care “what” it was determining eligibility and optimization for. Essentially, this made it an optimization engine for any widget or promotion you could imagine.
  4. It could support invocation from many channels (online, call center, partner XML-SOAP transaction)
  5. It could support invocation by users for many different lines of business. This enabled each business user to maintain his or her own sets of eligibility, optimization and promotion rules—using familiar terms (e.g., product vs. promotion) in their own language (English vs. Japanese)

As a result, demand for XSELL scaled first from 500,000 to 13,000,000 daily transactions within six months; then to over 200,000,000 daily transactions within the next 18 months. Within three years, use of the XSELL Service had expanded to support all of the business areas:

  • Qualification registration for all AOL and partner products
  • Optimization of third-party partner product promotion and co-registration in across online, back office, call center, and point-of-presence systems—within AOL, Time Warner and third party partners
  • Optimization of online content to display to users
  • Optimization of paneling of members into multi-channel loyalty and promotional campaigns—within AOL, Time Warner and third party partners
  • Optimization of routing of callers in call centers
  • Determination of credit and price plan eligibility for customer care
  • Selection of retention offers for canceling members

Resulting in USD $1+ Billion of enterprise value

These combined use of XSELL provided AOL over a $1 Bn of value (in increased revenue through cross-sell and decreased cost through smarter credit and retention management) over a five-year period. This would not be possible without its core adherence to Modularity & Encapsulation, Isolation of Logic and Full Extensibility.

I need to out some key people who helped make this a reality. Really cutting edge software engineers on my team: Ramesh Balasubramanian and Anubhava Srivastava; great managers who scaled our ability to support multiple partners and lines of business: Stan Berry and Clare Little and a fellow Chief Architect who introduced me to the powers of AOL’s Bucky Ball ODD technology: Yan Cheng.

Extending this beyond “Yesterday’s Internet”

I can see using this type of service in many places outside of AOL, Time Warner and its partners. Here’s how I would use it for some of today’s hot technologies:

  • iTunes: Smarter recommendation of things to buy
  • Twitter: Switching out the Party Line with streams or Tweets based on topics you care about. Recommendations of people to Follow – including key opinion leaders to companies to follow and connect with to advance their brand (a potential source of revenue)
  • Facebook: A targeted global feed of new applications, groups, pages, etc. based on your demographics and topics of interest – and third party partners bids on access to members (think “revenue model.”) Smarter content, application, and Friend recommendations.