Network Dynamics & Coupling: Shannon’s Reverie Reprised . . . updated

Shannon’s (fictional) Reverie . . .

Claude Shannon woke up one morning and said to himself, “Think of how our brains operate – it habituates to repeated stimuli but pays attention to a rare stimulus; things that are rare must carry a lot of information!” So, what do I know about quantifying rare or unlikely things? I know that things that are highly likely are highly probable; so unlikely-things or novelty can be thought of as the inverse of probability. But snap! Probability goes from 0 to 1; I need a “squashing” function around it so that the novelty measure does not blow up fast but at the same time, very low probability things (highly novel things) are highly weighted. How about . . .?

A bit of cleanup via taking expectations, probability density functions and some proper logarithms and we have Shannon’s famous equation for “information”,

When you reduce a (joint) probability density function to a scalar, there is a lot that you throw away; the “trick” is that the scalar that you come up with captures some aspect of reality that is *useful*. As the explosion of communication technologies in the past few decades shows, Shannon’s scalar sure did!

I am not claiming that this is how Shannon did his research but this is one way to approach new insights that you may have and their quantification.

Social and Other Networks:

In my recent blog on Social Networks, “What does ‘Emergent Properties in Network Dynamics’ have to do with Shopping?”, I noted the following: “Facebook connects us in a vast network – this is only a first step. The deep reason for the fascination with social networking can be understood from the shopper example. Shoppers are enmeshed in an ever-changing network of social interactions and preferences. Today in Retail Analytics, data are treated as isolated bits of information. In reality, data exist in *embedded* forms in preference and influence networks of the shopper as well as distributed in time and space.”

As you know, interest in understanding such Social Networks and controlling their “dynamics” via “influence functions” of nodes, etc., are at a fever pitch – advertising, retail and many other day-to-day eCommerce activities can benefit from a better conceptualization and quantification of social network dynamics.

We know that “coupling” in networks generate very interesting dynamics (see, Steven Strogatz, “Sync: The emerging science of spontaneous order”, 2003, for a very readable overview).  Consider the ultimate of all networks – the brain. When we do “brain mapping”, interesting patterns arise. The brain mapping pictures show scans of a “depressed” and a “non-depressed” person. In the Depressed case, the brain regions are NOT coupled whereas in the non-depressed or Normal case, there is significantly more coupling and more uniform activity across the entire brain. Note however that if we looked at such a scan for an epileptic patient during a seizure, the scan will be all “lit up” showing nearly-complete coupling – that is a degenerate case!

Reprising the Reverie . . .

Coming back to healthy coupling and following Shannon’s (fictional) thinking process outlined in the first paragraph, what do I know about quantifying “coupling”? I know that when the underlying sources are coupled, their “stimulation” of the neocortex is uniform and they show up equally across the network at the same time, much like a single “wavefront” that reaches all the regions simultaneously. Imagine you are at a beach looking out to the sea and gentle waves are rolling in – let us say in parallel to the beach. If you are standing knee-deep in the water and look in a direction parallel to the beachfront (i.e., up or down the beach), the spatial frequency in your “look direction” (or the frequency of “corrugation”) is 0 cycles/meter! Much like the ocean waves, the single waverfront of stimulation is nearly “constant” across the distributed neocortex and its relevant strength can be thought of as the power at zero frequency.

I happen to know that power at zero frequency is called “Scale of Fluctuation” or “θ” in random field theory. From the previous work of Eric Vanmarcke (“Random Fields”, 1983, with an updated edition in 2010),

For the initiated, the equation and the accompanying figures below are hugely meaningful! In the figure, the “height”, (g(0) times π ) and the “area” under the normalized autocorrelation function, ρ(τ), are marked in blue – this is “θ”! This is the graphical meaning of Vanmarcke’s equation for θ.

The result shown above is for a time series. For the brain mapping case, “θ” is 4π²g(0,0) of its 2-D normalized spectral density (same pattern follows for higher dimensional random fields). Calculations of θ for 1-D and 2-D cases are straight forward; ways to calculate “instantaneous” values of θ are also available using Kalman Filtering (introduced in my past publication, “Instantaneous Scale of Fluctuation Using Kalman-TFD & Applications in Machine Tool Monitoring”). Some curious properties of θ for 2^nd order linear time invariant systems were also developed there. To recap the highlights –

From discrete-time linear time-invariant system principles, we know that constant damping ratio and undamped natural frequency contours in the z-plane are as shown on the left.

It is notable that for a second-order system, the constant θ contours shown on the right have remarkably simple geometric shapes. In fact, for θ = 1, the equation is quartic but very similar to a circle with origin at (0.5 + j0) and radius = 0.5!

Equation for θ = 1 contour is (x² + y²) ² + x² + y² – 2x = 0

There are more details in PG Madhavan, Theory and estimation techniques for Random Field Theory and "Theta" with practical applications: Instantaneous Scale ofFluctuation Using Kalman-TFD & Applications in Machine Tool Monitoring, SPIEProceedings, SPIE Vol. 3162, pp. 78-89, 1997.

Similar to Shannon’s scalar, H, θ reduces the joint probability density function to a scalar. Does θ capture some aspect of reality that is useful? The constant θ contours above seem to imply great significance as fundamental as natural frequency and damping – but at this time, such insights are not forthcoming!

Similar to Shannon’s scalar, H, θ reduces the joint probability density function to a scalar. Does θ capture some aspect of reality that is useful? Some real-world applications of θ from the past (see its use for machine tool chatter prediction) point to the following physical insights.

While highly speculative, previous studies and our “Shannon approach” suggest that θ is proportional to “coupling” and to “order” in a distributed node system whereas it is inversely related to “degrees of freedom (df)”. In the case of “df”, the concept is that more degree of freedom is a “dangerous precipice” for a distributed system where different parts are de-synchronized and they can spin off in different directions - pandemonium can ensue!

Large θ seems to indicate an ordered, widely-cooperative and well-functioning network; however, it is conceivable that a large θ may also indicate degenerate cases such as epileptic seizure or full-fledged chatter conditions (extreme cases of coupled sources and distributed action). An example is shown below.

Large θ on the left is a hallmark of “distributed order” whereas the large θ on the right, that of “locked-in order”. Low θ condition in the middle is visually indicative of disorder and the potential for degeneracy!

θ in Social Networks:

To help develop our intuition, let us consider some snapshots of geographcally distributed network maps. We have (1) Internet activity over continental United States, (2) LinkedIn infographic map and (3) Facebook social network map.

We notice strong coupling in the Eastern half of US among Internet nodes and similar features in the LinkedIn and Facebook networks. Our intuition is that the *bright spots* obvious in the Northeast US of the Internet map or the EU area in the Facebook map indicate more “correlated” activity. For simple network maps, we have developed primitive methods to estimate θ based on correlation functions.

Before we leave this blog, consider the brain map and the US map of the Internet. What we see in these pictures can be called “surface structure”, i.e., observed or measureable quantities. In the brain, the Surface Structure is created by activity deep within the brain (I am NOT referring to Chomskian linguistics model here). In the past, naïve physical modeling has conceptualized dipole oscillators in the “deep structure” of the brain giving rise to the Surface Structure as a starting point for theorizing. In the case of the brain, there are indeed Deep Structures (nuclei and ganglia and their dendritic potentials) giving rise to voltage variations on the scalp surface (earthquake tremors recorded on the surface of the earth and activity deep in the earth’s crust form a similar model).

Clearly, the Surface Structure of the Internet cannot be related to any actual Deep Structure in a physical model – there is no mechanical turk behind the Internet pulling the strings! However, even in such cases, conceptualizing observed activity as the resultant of implicit Deep Structure may be useful in developing analysis methods. The hope is that the physical model of network activity utilizing the concept of explicit or implicit deep structures with internal coupling will help advance our “analytics” tools for the extraction of patterns and information from spatially and temporally distributed networked systems.

www.JINinnovation.com

Dr. PG Madhavan was CTO Software Solutions at Symphony Teleca Corp. Previously, he was the CTO & VP Engineering for Solavei LLC and the Associate Vice President--Technical Advisory for Global Logic Inc.  PG has 20+ years of software products, platforms and framework experience in leadership roles at major corporations such as Microsoft, Lucent, AT&T and Rockwell and startups, Zaplah Corp (Founder and CEO) and Solavei. Application areas include mobile, Cloud, eCommerce, banking, retail, enterprise, consumer devices, M2M, digital ad media, medical devices and social networking in both B2B and B2C market segments.  He is an innovation leader driving invention disclosures and patents (12 issued US patents) with a Ph.D. in Electrical & Computer Engineering.  More about PG at www.linkedin.com/in/pgmad

Emergent Marketing: A New Force in Social Commerce

Dr. PG Madhavan was CTO Software Solutions at Symphony Teleca Corp. Previously, he was the CTO & VP Engineering for Solavei LLC and the Associate Vice President--Technical Advisory for Global Logic Inc.  PG has 20+ years of software products, platforms and framework experience in leadership roles at major corporations such as Microsoft, Lucent, AT&T and Rockwell and startups, Zaplah Corp (Founder and CEO) and Solavei. Application areas include mobile, Cloud, eCommerce, banking, retail, enterprise, consumer devices, M2M, digital ad media, medical devices and social networking in both B2B and B2C market segments.  He is an innovation leader driving invention disclosures and patents (12 issued US patents) with a Ph.D. in Electrical & Computer Engineering.  More about PG at www.linkedin.com/in/pgmad


“Emergent Marketing” is a fundamentally new approach different from the traditional “resultant” marketing that is based on cause-and-effect at the individual level. In networks, one of the most studied phenomena is that of “infection” spread when a contagion is dropped into a network of humans; the result can be traced back to its cause – this is the hallmark of “resultant” marketing. Contra-distinct from this approach, Emergent Marketing is a low-level multi-node intervention in a human social network that creates the emergence of a ground-swell of a desirable activity (“emergent” activity) without identifiable one-to-one causality.

Let us start at the beginning. From high school Physics, we know that atoms come together to form molecules and they have palpable physical properties; for example, H2O in the form of water can flow as a liquid and you can get wet. When a state transition occurs to water (such as due to cooling), ice emerges whose “solid” property was unsuspected when H2O was a liquid – this is called an “emergent property”. Solid-state Physics has countless examples of such phenomena and one of the necessary conditions for it is densely connected simple entities. The brain is an example of such a densely interconnected neuronal network; such networks exhibit variability over time. Variability over time or dynamics is often an essential property of such networks, exemplified by the recent discovery (popularized in the press during September 2012) of the role played by “junk” DNA in the expression of “coding” DNA responsible for various diseases. There is a more informative discussion of these basic concepts in my recent blogs, “What does ‘Emergent Properties in Network Dynamics’ have to do with Shopping?”, http://pgmadblog.blogspot.com/2012/10/what-does-emergent-properties-in.html and “So-Mo-Clo Framework for CUS”, http://pgmadblog.blogspot.com/2012/07/so-mo-clo-framework-for-cus-dr.html

The last paragraph introduces three concepts: networks, dynamics and emergence. In my previous Emergent Properties blog, I noted that “Facebook connected us in a vast network – this is only a first step. The deep reason for the fascination with social networking can be understood from the shopper example. Shoppers are enmeshed in an ever-changing network of social interactions and preferences; desirable behavior can be made to emerge by perturbing the interactions within the network.”  For the first time, the concept of “Emergent Marketing” was introduced in this context.

Emergent Marketing:

As mentioned in the first paragraph, Emergent Marketing arises when one goes beyond one-to-one “resultant” marketing designed to provoke a certain purchase activity in a shopper.

A simple model to understand marketing approaches is the Purchase Funnel. The right hand side of the funnel in the figure shows the various intentions of the shopper as she traverses through the funnel. Marketing techniques aspire to move her from “Awareness” phase as quickly as possible to the “Action” or purchase phase. The left hand side shows the coarse buckets of marketing activities traditionally undertaken to achieve this funnel transition – Branding and Direct Response. By the Direct Response methods, the marketer is trying to induce a specific one-to-one response (as indicated by the word, “Direct”) that will culminate in a purchase. On the other hand, “Branding” is more nebulous; branding can take years to take hold and start to bias your purchase decisions.

When you think some more about “Branding”, it appears to have many of the features of “Emergent Marketing” that I had mentioned in the first paragraph! Branding is a multi-node intervention that creates a ground-swell of interest in a product or service but with a long delay.

In this blog, I want to focus on the Interest-Desire-Action phase of the Purchase Funnel. Ads in this phase have the desirable properties of immediacy and measureable efficacy; both are very important to the Brands that pay for advertisements. Especially, the ability to correlate ‘ad spend’ to point-of-sale receipts is a super-important metric; Brands need it (“Half of the ad expenditure produces results; the trouble is no one knows which half!”). Even during one of the worst down years of the Great Recession, worldwide advertising spending was huge (estimated at approximately $654 Billion, $54 Billion of which was for online ads); it will be good to know which of the Billions worked!

Among the many Emergent Marketing techniques possible, we want to start by building on what we already know about branding ads and create what I term “Just-in-time Branding” (JIT Branding). JIT Branding will exist in the Direct Response phase of advertising and will have immediacy and measurable efficacy.

Just-in-time Branding:

Retailer’s interest is to precipitate a specific buying behavior within a specific customer. But we do not approach it on an individual basis because we know better now - we have to consider him as a node in a network that includes his social network which creates latent influences and also his “Shopper Attribute Map” made up of his likes and dislikes.

Even though infection spread is not a good analog for Emergent Marketing, let us try to utilize it since much of the earlier work in networks has been the modeling of the spread of infections in a population. Each node has a different “influence function” – a node “infects” a varying number of other nodes (out of total of ‘N’ Nodes). If there are ‘K’ contagions, the Linear Influence (or Infection) Model (LIM) is -

v = Mi;   where M - Influence Matrix (KxN);  v – Volume of Infection (Kx1);  i – Node Infection (Nx1)

Matrix Algebra has so many powerful tools, fast algorithms and practical insights that a lot of useful information can be extracted by the expert from this simple equation. In general, singular values and vectors (and the related eigenvalues and vectors) of matrix, M, can tell you which nodes are the major “influencers” or hubs, how big their influence is and even what forms the bases of their influence in some cases! You can start populating the Linear Influence Model (LIM) with historic data collected from your real network and apply the results and insights to your business to improve or control the viral spread of your marketing message (or “contagion”).

JIT Branding using LIM is still a primitive model of true Emergent Marketing. LIM models minimally incorporate the dynamics and the ability to perturb the network in a fine-grained manner. The design and number of “contagions” for emergence of desirable “ground-swells” of activity are also open issues at this time.

What is next in Emergent Marketing?

In a “crawl-walk-run” approach to developing full-fledged Emergent Marketing techniques, JIT Branding is a “slow crawl” at best. Unlike traditional branding, our main insight is that we have to explicitly utilize the facts that (1) shoppers are embedded in social and preference networks, (2) these networks are dynamical and (3) control knobs and levers at a fine-grain node and link level have to be perturbed to engender desirable emergent activities.

One promising avenue is likely to be synchronization in dynamical networks; it is a powerful method to generate global and local “ground-swells” (sometimes with undesirable outcomes such as epileptic seizures in the brain or outages in the Internet!). More advanced Emergent Marketing techniques will exploit this avenue and other perturbation methods in dynamical social networks to create desirable short-term product affinity for a selected cohort of shoppers.

Closing the Analytics Loop:

Analytics extract meaningful patterns and information from raw data. But what do we do with those insights? The promise of Analytics will be fulfilled only when we close the loop via actions that lead to profitability in the broad sense. Emergent Marketing is an example of closing the loop of “analytics” drawn from retail data and shopper network activity back to the customer generating additional new analytics. We may have tapped a rich vein in analytics-driven retailing here; whether it will taper off or hit the mother lode remains to be seen.