Lots More Detail About Narwhal 'Big Data' Machine Performance in Obama 2012 Campaign - Including several use cases and applications (vs Orca the Romney machine)

I wrote a week ago an update blog about the Big Data ‘Narwhal’ project of the Obama 2012 campaign. Originally back in 2012 we tracked that ‘info war’ between Obama’s Narwhal and Romney’s rival system called ‘Orca’. Soon after the election late in 2012, I wrote two long articles about the use of voter data in that race, including the definitive blog about the two systems going head-to-head. And I promised to write more as data emerged.

So in the next three years I’ve occasionally added the terms ‘Narwhal’ and ‘Orca’ to my searches for new articles relating to mobile or digital, and sometimes was thrilled to find a new, at times quite detailed article (or book) about the case(s) but then, noticed that actually that ‘cool’ new article was... based on and relating to THIS blog and my previous articles. Try it. If you do a Google search for ‘Narwhal’ ‘Orca’ and ‘Obama’ and look for results in December 2012 or thereafter (the election was in November) then look what comes up. This blog and plenty others including many books and magazine articles. And then look at those, often they reference the Communites Dominate blog or me by name and even often have a link back here, to one of those two long articles. Nice but no dice. I wasn’t learning new details.


(Note - my PC crashed as I was drafting this blog article, and all the open pages where I had found the articles, vanished. I am now trying to rediscover at least the most relevant items to give us links to where more of this information has been hidden. hold on, I will be adding links in coming hours)


A week ago I did have a blog finally with a few new details that was based on highlights reported on Jim Messina speaking at a conference. Messina, if you recall, was Obama’s Campaign Manager for 2012. Ah, yes, I had only searched for the name ‘Narwhal’ but dind’t try the people involved. What Messina talked about at that conference didn’t use the Narwhal name. He talked about ‘Big Data’ and used other terms. So I decided that the next time I do one of my diggings in the internet for cool new techie developments, I will do another ‘Narwhal’ search, but this time, I will not use those names of the two databases, rather I will search for the names of the three campaign bosses for Obama: David Axelrod, David Plouffe and Jim Messina. Axelrod ran the 2008 Obama campaign (it was Axelrod’s idea to develop that crowd-response chant ‘Yes we can’ which Obama thought was too simplistic and Obama over time grew to hate doing that, but the crowd obviously loved it and ‘Yes we can’ became the defining phrase of the whole race of 2008. By the autumn of 2008 Obama hated doing any of the chant, and had mostly moved away from it but still on election night in his victory speech, he did the ‘Yes we can’ chant echo one last time to massive audience response) but for 2012 Axelrod had moved to the position of strategist and advisor. Messina was the new campaign manager for 2012. Plouffe was the data geek in this management trio.

And what did I find. Plenty of references to the election in the UK, and lots of Axelrod’s pontifications as a pundit on recent politics. He doesn’t talk about the data wars. Plouffe who would know the most details, has been totally silent. But.. Jim Messina has several times spoken or given interviews that often  reveal some new tidbits and details. And with these as search terms, I found a few other sources, in particular the Big Data pages over at Hewlett Packard, where interesting details are also about the Narwhal project but referencing rather HP’s system and software terminology. As they were part of the infrastructure that Narwhal ran on, and one of the Obama senior data nerds, Chris Wegrzyn (aka Director of Data Architecture) is now running Big Data at HP, its understandable that occasional data comes from that initiative. So quite surprisingly, for several years now, there have been numerous interesting details, some correcting what I had written, many adding depth or actual case examples of how Narwhal performed. Again, I am not so interested in Orca anymore, as that was already obsolete for that election, and we have their relative performance data rather well documented. So I am not too much even interested in learning more about that project at this stage, as it was a ‘dead end’ or a developmental cul-de-sac. No more life on that journey.

But Narwhal is now being developed into something that we could call ‘Narwhal 2.0’ for the Hillary Clinton 2016 campaign and as we’ve documented on this blog, she’s spending massively on that already. Also we know three of the rival Republican campaigns had started to build their own evolution to Orca, incorporating known info about Narwhal (and almost by necessity visiting this blog) ie Jeb Bush, Scott Walker and Ted Cruz campaigns. Walker’s project shut down as his campaign ended. Jeb’s data project has been suspended as his campaign struggles to stay alive and the staff have been reassigned to current activities so nothing is being developed. But Cruz’s campaign is learning already to target and refine their messages, even deciding where Cruz speaks, based on their version of a ‘Narwhal clone’ in its early stages. Absolutely certainly Hillary is doing the same with their upgraded and supersized Narwhal 2.0.

And with that much of ado, lets go to what new stuff is discovered. Lets start from the start.

DETAILS ON BIRTH AND EVOLUTION OF NARWHAL

The origins of Obama’s previous data project were back in the Howard Dean campaign of 2004. Those lesson then evolved into the Obama 2008 campaign and its massive leap into data-driven modern campaigning, arguably the world’s first such detailed use of data to determine campaign activities. We have now learned that the 2008 project total data team was 12 employees. For Obama 2012 that had grown to 175. But some of the detail about the Obama Narwhal project I wrote about three years ago was actually leaked detail about the PREVIOUS iteration, ie the 2008 project (as this was obviously competitive advantage in elections and politics) and Narwhal had actually moved beyond from that level. So now we know, that the 100 point scale on two scales, that I talked about, that Obama’s campaign used, was not the state-of-the-art for the 2012 edition. That was not created for 2012, it was created for 2008 and then still used in 2012 (but expanded further). Yes, what I wrote was used in 2012, but that was capacity from 2008. Narwhal in 2012 had even more power than that (I will explain)

What is this? Its the cool idea, that they took two distinct and from the campaign activities, very relevant voter attributes as ratings. How much does that voter love Obama compared to McCain. A voter preference score. 100 is perfect love of Obama. 1 is total love of John McCain. There actually is no use trying to convince someone who has say a 15 level support, almost totally decided on McCain. Also there is no point in targeting someone who is say 80 level support for Obama, because that person has already pretty well decided and the time and money is far better spent targeting someone truly ‘undecided’ ie someone whose Obama preference score was around 40-60. Ok that is one metric. Then the other one?

The second 100 point score that I wrote about for 2012 that was also actually in use in 2008 by Obama, was the likelihood to vote score. So on election day, is that one voter expected to actually show up and vote (regardless of who the voter preferred, Obama or McCain). Again 100 meant absolutely certain to vote, even if its a blizzard snow storm and a score of 1 means that under no circumstances will that person be voting even if happening to be in the building on election day. For these voters, it didn’t matter to try to convince them to pick Obama, it was relevant to try to get those who had an Obama preference, but were not certain to vote, say voting likelihood between 30 and 50, to send them reminders or even arrange a car-pooling transport to get them to vote on election day. Obama 2008 attempted to find every voter in each of the approx 10 ‘swing states’ which would decide the election, and then assign each of those voters those two scores. Note, it is quite important to also talk to the Republican voters, to ‘eliminate’ those who are not worth contacting.

When you consider a total voter database and if you had those two scores, note that on election day, this would guide typically two very distinct type of messages. Some who are still undecided - but are highly likely to vote (say score of 40-60 on candidate preference but over 70 score on certainty to vote) - those would not need a reminder to vote, they will certainly vote, but they will need a last-minute push to pick Obama ahead of McCain. And then consider the voter who really likes Obama but is a lazy voter. That person might have an over 70 score to prefer Obama but only something of 40 to 60 score on certainty to vote. That person will not need any mention of who to vote for (we know it will be Obama) but they will need reminders on election day to go and vote. So this is how totally distinct election messages would then be used, targeted by this method. But yes, those 100 point scores were already a feature of Obama’s 2008 campaign. I mistakenly reported that these were developed for 2012.

THREE NEW 100 POINT SCORES

What the team did, was to add at least three new such 100 point scores for the 2012 Narwhal system. Now in addition to an Obama preference score (vs Romney this time), and the voting likelihood score; they added the most useful metric yet the ‘persuadability score’. How likely was this voter to be convinced to switch (in either direction, vital also to know if your own voters were ‘locked in’ or could be persuaded to switch to Romney) That score of course also 1 to 100, meaning 1 is a rock and cannot ever change his or her mind and 100 will change his or her mind even at a mere whisper suggesting vaguely a rival’s name.

The second new 100 point score was propensity to donate money (again partly explaining how Obama in 2012 collected dramatically more money, from a less enthusaistic voter base, than in the record-setting wave election of 2008). In 2008 Team Obama collected a record 500 million dollars from small donations from individuals. That grew to 700 million dollars (40% increase!) in the 2012 election. As Obama actual vote count was down 5%, thats effectively a 47% improvement in fund-raising efficency, going from the 2008 version to 2012. This increase was powered by Narwhal and the detailed supporter data captured into this new score.

And the fifth 100 point score was the propensity to be a ‘split ticket voter’ ie to vote say for Obama the Democrat for President but then vote for the Republican Governor or a Republican Senator. As we consider the Obama Big Data operation, this is a massive shift in the ‘game’ where now the Presidential campaign takes into consideration the local races within this, the most powerful individual tool within a campaign’s arsenal. Back in 2008, about one in ten voters behaved this way based on the exit poll (evenly split by Democrats and Republicans). Unfortunately in 2012 this question was not asked of voters. But a different method was used by Washington Post which went through each congressional district (but before the last 11 districts had been counted) and it gives the result not by voters, but by voting districts. WaPo reports that ‘ticket splitting’ has become far less common as the nation has become more polarized, for various reasons. As recently as year 2000, the Bush 2-Al Gore election, 20% of congressional districts voted differently for President than Member of Congress by party. For 2012 that was down to only 6%. Then it gets interesting. We can do the analysis by party. For the 243 Congressional districts that sent a Republican Member to the House, 226 also voted for Mitt Romney for President (93% loyalty achieved - and Obama had stolen thus 7% of those districts). How did the Democrats do? 215 districts voted for a Democratic Member of the House. and 206 of them - 96% - also voted for Obama (only 4% were defections).

Please bear in mind that in 2008 the balance was even, both sides had as many splitting their votes. But in 2012 the overall vote-splitting was down (on both sides) but Obama did far better in stealing votes from the other side than Romney was able to. And obviously this element of Narwhal was at least part of that victory. How many votes were in that balance? The 3% ‘net gain from stolen’ (the difference) would represent a victory of 1.8 million votes in this ‘race’. That means of course 900,000 Republican voters (counted as a loss on one side and also a gain on the other side, hence 1.8 million net total). Obama’s victory margin was 5 million votes. Roughly one sixth of Obama’s total victory margin would have been ‘turncoat’ voters who voted for their Republican Congressman or -woman but then ‘deserted the party’ and voted for Obama instead of Romney, for President. Note, both sides succeeded in this, this is normal for some slice of the electorate, but it is roughly in balance most years. In 2012 it was twice as successful at the Obama camp than Romney camp, and yes, just less than one million more voters who otherwise voted for their party, broke on the Presidential choice to Obama rather than Romney. A big achievement, when the very last RCP (Real Clear Polling) average the night before the election had the decision on a 0.7% split (ie 1 million vote margin).

I should note this particular score now going into 2016 will be a particularly useful item in the Hillary Clinton campaign. It will enable ‘down ticket’ campaigning, of building ‘coat tails’ for the Presidential candidate and using Narwhal-type insights to decide where the national candidate (or her or his surrogate) will help or hurt a local candidate; and which methods and messages will work best in that. It will be particularly useful for Hillary in 2016 as she tries to win back the Senate and shrink the large advantage the Republicans have in the House. And now the Democrats have the data on what worked and what didn’t, in their first use of that element. If the growth in the efficiency is anything like that with donations (47% growth) this could result in as much as 1.5 million extra votes stolen from Republicans - above and beyond what normal polling might identify (as right now, Hillary is polling to steal one in six Republican voters if Trump is on top of the ticket).

So yeah, in my big Narwhal blog, this is something I had wrong. The original two 100 point scales we learned about in 2012, they were already a feature from 2008 and by 2012, Narwhal had grown to include (at least) 5 such 100 point scales.

REFRESH RATE

We now found out that the Romney team used customer psychographics teams to develop customer profiles for their voter insights database for their ‘Orca’ machine. Those profiles were developed once, at the start of the campaign and never then updated. But Obama used a totally dynamic voter segmentation system where the voter ratings (those 100 point scores) were updated as any info came in (always in real time) and for the total voter database, the metrics were recalibrated weekly. So as telephone interviews, polls, Facebook blasts, in-person home visits etc, collected more details, and the political events shaped the discussion, the profile ‘specifications’ were updated weekly. With those 100 point scores, this gave a lot of flexibility, such as adjusting the ‘band’ of scores that were considered say ‘Obama-leaning’ voters. It might be those with a score of 54 to 65 in one week, then after more data, that could be shifte to a score of 56 to 63. That kind of flexibility.

These insights then drove the ‘scripts’ that telephone banking volunteers used when talking to voters on the pohne, and when ‘door knocking’ volunteers visited local households. They of course also drove the email blasts, the SMS text messages, Facebook contacts and so forth. All that was fed with the latest campaign news, but the very defintion of each segment was updated weekly. The system grew ever more precise, refining and relearning and improving systematically, every week.

TEAM SIZE

We now have the split of how the data analysts were split, based on reporting by Infoworld. The Obama 2012 campaign employed over 1,000 total staff. A political (re-) election machine. Yet it employed 100 full-time data analysts! (10%). Taking the cost of Narwhal at about 10% of the Billion dollar budget of the whole campaign, that sounds about right. But its obviously huge (where the previous, world-record-setting Obama data team as just a dozen people). How were they split? Just over half of those, 54, worked at ‘The Cave’ the secretive cubicle farm at the headquarters. The rest were split so that roughly 20 were placed at significant field HQ offices, and 30 were moving about, interpreting field data.

66,000 SIMULATIONS EVERY NIGHT

I wrote last time a week ago that the Obama team did 66,000 simulations in total over 18 months. That is not true. They did 500 times better than that. They did 66,000 simulations of the election result - every single NIGHT. That number intrigues me. And I did a bit of guesstimation. If you run individual state results as an ‘either or’ proposition thats simple math, two to the power of states. So if its one state, two possible outcomes. If its two states, four possible outcomes (both vote yes; or first votes yes, second votes no; or else first votes no, second votes yes; or finally both vote no). So its simple progression, 2, 4, 8, 16, 32, 64... and we know 9 states were the target scope of Narwhal. Do that and you get 512 out of the 66,000. How much more to 66,000? Now do seven more iterations and you are at 65,546. In other words 66,000 simulations.  So if you want to test your campaign strategy ‘move’ and test it running it only in Ohio, or running it in Ohio and Virginia. Or running it in Ohio and Florida but not Virginia, and so forth, for the nine battleground states, you do 512 iterations. Thats a lot of computing.

Now it leaves seven variables. What could they then be? I believe that they then tested the MEDIA to use for that given campaign messaging. Should they use TV. What about Facebook. How about email. What about visiting a home. So seven such media choices which I guess would be TV, Facebook, YouTube, email, personal telephone call, SMS text message and personal visit to the home.

So les say that Romney’s 47% video was just leaked. And in the day the A/B testing teams tested 24 variations of the email text to get maximum email impact. Similarly tested a bunch of YouTube ads for which works best (in whatever target group) and so forth. Note during the day they have tested against their daily in-state interviews those messages with live voters, so they know what the actual impact is, how many would change their minds. Now plug that into the machine and run it at night. Then see what if you run the YouTube ad in Florida, send the email blaster in Virginia, Florida and Ohio, run the TV ad in Colorado and have the field workers in seven of the nine states talk aout it when they visit the homes. What is the end result of this strategy. And compare that to what if we drop out Ohio from the email blaster and instead send the email blaster in Wisconsin. And on and on for 65,546 times, every variation of seven main media contact types and across every variation of the 9 states. Every night, test one campaign messaging adjustment or new message, positive or negative, attack or defence, and test does it help the campaign - and if it does, what is the optimal footprint of states where to run it - AND what is the optimal messaging delivery media choice or mix for it in any of the selected states.

Then consider the campaign manager’s wealth of info before buying the ad time. Jim Messina could see the cost-benefit of each option. A two million dollar TV ad buy could get this message out and get the maximal 100% gain. But a half-million dollar targeted TV ad buy could get him 92% of the gain, allowing 1.5 million dollars to be saved, and used better in another battle still going on from say two days ago (its never only one thing that the electorate is thinking about). This is incredibly valuable information to win a campaign. Its the ‘what if’ calculation that made spreadsheets a must for all of management three decades ago. But now, run on a national election. What if. This power has never before been available in elections. Simulate the whole election, all voters, the critical 40 million who will decide the election, to this accuracy across this many variations, every single night.

Now, I do not know that this is how they used their nightly computing power but it makes a lot of sense as we do know the 9 states they used and the main ways the campaign contacted the electorate. It does seem, that for every night, the campaign selected one issue and ran it through every permutation to see how it would play, was it worth running (would it help or perhaps hurt more than help) and where it was worth running, and of course HOW it was worth running (which media formats in any given state). This resulted in two confusing elements for the Romney team. One, was that sometimes team Obama seemed confusingly quiet about some ‘obvious’ issue. No national TV ads, no mentions by surrogates at events or TV interviews. Then after a while, they’d spot something at a local TV station or cable TV. Or they’d see that the issue was featured in email or Facebook use. This was part of the sophisticated, targeted, selective use of media. Optimising the media not maximising it. And those local spots would then further confuse the Romney team, why on earth are they doing this targeted advertising in this tiny market..

For example the Romney campaign noticed once that Obama was advertising in a small town local TV channel in an area that was massively Republican. They pondered about that ad buy and couldn’t figure out why. But the Obama team had found some persuadable Republicans and then carefully selected messages that would appeal to those. It wasn’t expensive to advertise on a local TV channel in the outskirts and the Republicans weren’t bothering to counter it either. If that helped flip some disgruntled Republicans, it was actually very well spent resources (remember above the scoring 100 point scales on persuadable voters and also on split-ticket voters). The votes are counted by voting precinct but in the election of President, they are aggregated to the state level. It didn’t matter WHERE the Obama voters came in any one state, but all previous methods didn’t allow precise targeting this way.

If you were limited in resources (all campaigns are) and have to pick where to advertise to boost your own side’s support, it makes sense to push your ads in districts that have a lot of your voters - but it would seem foolish to push ads into those districts that are strongly in the rival’s camp. It would seem like a hideously wastetful campaign if traditional mass market TV ads (same ad for everybody) was used. But the CHANGED voter meant effectively two wins. One won by ‘us’ and equally one ‘lost’ by them. Most voters are not vulnerable this way, but the ones who are - if they can be reached in cost-effective ways, that is very powerful campaigning - and a flipped voter counts as two votes in effect. 900,000 voters flipped from Romney to Obama meant the vote winning margin for Obama went up by 1.8 million. If they could be found, and targeted, this was worth doing. And again, enabled and targeted by.. Narwhal.

Now for the math nerds, if you do 66,000 simulations of that database with 40 million registered voters in those 9 battleground states, and we say the Cave analytics shut down at 22:00 ie 10pm at night and the simulations ran all night until 08:00 in the morning, then the average time per simulation of 40 million voters through a given simulation variation, was 1.8 seconds. In less than every two seconds, a new simulation was calculated. Incidentially, it also meant that every morning a fresh 66,000 plausible outcomes of the 2012 election were calculated. How many total simulations would that be? 36 MILLION ‘election results’ tested (but obviously some were way early in the season, when it wasn’t even sure which was going to be the opponent, Mitt Romney or one of his mental midget rivals of 2012). But per month, it was s sliver shy of 2 million election result variations tested and simulated. Thats a LOT of simulation haha.

TECHNICAL FACTS

We also have now learned more about the technical aspects of the system. As I’ve reported, Narwhal was one centralized database with at least half a dozen separate applications running upon it, from how field-workers would learn about their next house they were visiting and immediately after that visit, report the findings from that visit, to how TV ads were targeted and how fundraising was organized. But how was it technically built? We know now it was run on the Amazon cloud. The actual database of 175 million voters was on SQL and on a MPP (Massively Parallel Processing) based series of computers run by Hewlett Packard. The initial system was dimensioned for 10 Terabytes of data and I understood that the final size was ten times larger than at start, so while probably didn’t reach 100 Terabytes, that would be how far the final system was dimensioned and probably its real capacity was between say 50 and 100 Terabytes. It was reported to be one of the largest databases on the planet at the time probably at the bottom end of the 20 largest.

EVERYTHING TESTED AND MEASURED

The Obama 2012 campaign did nothing, truly nothing before it had tested and measured it. So for example the campaign created over 1,000 YouTube videos. Why? The publishing of those is free, and their viewership is instantly measured, by YouTube. So the Obama campaign could test the effectiveness of TV ads, before running it on TV. They would blast the link to two rival TV ad versions to carefully selected targets of their voter base, to be a representative panel of what any given potential TV program audience would be like (and by representative panel, don’t think 50 or 100 people. These panels were in magnitude of something at least the size of 10,000. This is utterly beyond the practical feasibility of a commercial marketing research panel in size (and accuracy).

Then they’d shoot that video to Facebook friends who fit that given profile, doing three sets. One is the ‘A’ version of the ad, the other is the ‘B’ version of the ad, and a third, smaller group, would receive neither ad, and would be monitored as the ‘control group’. This could be further targeted by exactly the states where the TV ad might be run. Now team Obama could see instantly and accurately which ad performs better - before running it on TV (and the control group ensured, that the sudden change in Obama voter behavior was not due to something else happening in the election like something that Mitt Romney just said).

But it went so far further than just that. Yes, every ad, every email, every Facebook posting, before it went live, went through A/B testing - typically an email had 24 versions tested before it went out to its full mailing use. So what did Team Obama do? They ran a two daily panels of interviews in each of the 9 battleground states. 5,000 interviews were done on a ‘quick survey’ and 1,000 were done on a ‘deep survey’ level. Every single day at least 54,000 surveys! Note, that if Gallup or Quinnipiac or any of the pollsters release a national poll, say sponsored by a TV station like ABC/CBS/CNN etc, and usually with another sponsor like say the Wall Street Journal - that poll covers about 1,000 people, who were interviewed over several days, so if its a 3 day poll, then on average 333 people nationally were interviewed for that new poll saying Romney is up 51% to Obama’s 49%. And that poll would have a margin of error of say 2.5% so its EQUALLY true that the actual reality was Obama was up by 51.5% and Romney was down to 48.5% - and this is within the margin of error of that poll finding. Because the sample size is so small. And note, the sample is always national.

Now compared to what Obama did. They interviewed a quick poll which would have that type of questions like ‘would you vote for Romney or Obama if the election was held today’ that kind of thing as in the above poll - Team Obama asked 6,000 people in each of the 9 battleground states (54,000 interviews PER DAY) because the short interview of 5,000 included this question - and it was ALSO included in the 1,000 longer interviews, of course. Now multiply this by the three days of that poll in the above, and Team Obama has interviewed 162,000 Americans in its three-day polling compared to the 1,000 in the published poll. The Obama system was so accurate, its error from reality was a small fraction of one percent. It did predict the national election result by an accuracy of about 0.2% - totally unmatched by any public polling. Its why now Jim Messina said if you show me a public poll, you will be fired. They had far more accurate information - and it was instant (as in available the next day after anything happened). If something really relevant happened, like the 47% by Romney, then they could see the REAL effect and measure it, withing 24 hours. Then they’d construct their response options, and run that through Narwhal (another night) and two days later, Team Obama had the best, tested, possible, optimized response whether it was a new attack in the stump speech, or a mailer, or a YouTube video or a TV ad, or if it was something only the visitors to the homes, and people manning the telephone banks would talk about.

KNEW WHO YOU VOTED FOR IN 2008

Now we get to the really scary creepy sides. Amazon obviously knows what books you bought, or even what books you looked at. Facebook knows who you consider friends among its user base. Google knows what you are searching for. But the vote in all democracies is secret. Who you voted for. That you voted, it can be known and it often is known. The USA voter registration reports who has voted in near real time (as in early voting, and with frequent updates per voting district, on voting day itself). So they know - create a record of - and maintain a history of if and when any voter voted. But the ballot is anonymous, they won’t know who you voted for, unless everybody in that voting district voted for only one side of the two haha.

But the Narwhal team knew to statisical certainty every voter of 2008, how they had voted. Jim Messina said that since they ‘know’ how any one voter voted the last time, for 2012, they had to try to get everybody who voted for Obama to come and vote for him, again. To get those who still like him, to vote for him again. To get those who are unsure, to become convinced Obama is the better choice, and to try to persuade those who have now abandoned Obama to come back. And then to see which of Obama’s voters were so disgruntled they thought of not voting at all, to try to get them to return. Messina knew its an easier task in marketing to get some customer (in this case, voter) to return to a favored brand of the past, than to convince someone new to try the brand. And worst job is to convince someone who prefers your competitor, to switch brands.

Obviously, as not all Obama voters would (or even could, some had died of old age) vote for him, then the secondary task via Narwhal, was to find new voters to replace all that were lost from 2008. So its quite a bold statement to say, we know how everybody voted the last time. Is that possible. Actually it is. And again, big data is the key. The USA is split to something around 150,000 to 200,000 voting precincts which have an average size of a little over 1,000 voters, the smallest are around 500 voters and the largest about 2,000 voters per precinct. About 400,000 registered voters and about 300,000 actual votes are cast per any congressional distict which is typically the smallest national voting ‘count’ that is talked about. But the voting process does go to that approximately 1,000 person accuracy. Each voting precinct is theoretically a test lab to weed out some voters and identify who voted for whom, because each voting district reports their actual precise vote count for both (or because of some Independent candidates also, actually all) sides.  And its then a massive jigsaw puzzle, where there are a few known pieces.

Some voters tell openly who they voted for last time. This is particularly so on Facebook. Some will say openly who they voted for. Others may admit it in some discussion thread. And note, it might be that one voter doesn’t admit it on his or her own FB page, but rather, admits it when writing a comment to a friend’s page. So, do a massive search through all the pages of all who became friends of Obama, and suddenly a portion of the puzzle is filled in. Maybe say 5% of the total votes cast. But because this is more than one hundred thousand voting districts, and some are very lopsided, 80% one way, 20% the other way or more, suddenly a few districts become ‘known’. And slowly many profiles start to emerge. So take a small town in rural America. If the father of the household is a registered Republican and voted in the last 5 elections while always registering as a Republican, he most likely also now votes Republican. But if his daughter grew up in that household, she might be voting Democrat. So this kind of patterns, some voters start to be known (the dad), others are still unsure (the daughter). And more info is then seeked (only on the daughter).

Slowly maybe 10% of the total electorate is filled out. Note, some other indications can occasionally reveal past voting patterns. Say a bumper sticker on a car (as long as that particular car was owned by the same driver four years prior). Or maybe a T-shirt that says ‘last time I voted for Obama but now I will not’. So occasionally very compelling evidence suggests this voter very likely voted for Obama or Romney last time. They are then fitted into the model, filling out a tiny piece more here and there.

Then they start to canvass and ask the questions. The Obama campaign talked in person to 64.5 million voters by election day. These were mostly Democrats but they included also Republicans. That was 51% of all voters. Now when we look at the ‘real battle’ of the nine states that decided the election, those had 40 million registered voters. Team Obama had contacted 34 million of those (compared to Team Romney who only contacted 15 million registered voters). Note these states are essentially 50/50 split between Democrats and Republicans. So Obama hit all Democrats and TWO THIRDS of the Republicans, while Romney only reached 15 million ‘voters’ which by their targeting should be about 60% or 70% Republicans (say 10 million of their own, ie half of their voter base and one quarter of Democrats).

So when talking to voters, especially as this is the ‘re-election’ of Obama, a voter may easily volunteer that they voted for Obama the last time. This would be noted by the volunteer into his/her form, and now we know more. That would be of course checked against - did this person actually vote (some people lie in interviews) and also is it consistent with the voting record of that voting district, but most people are honest and this will help populate the puzzle. So it might give 1% more. Then the campaign starts to ask. It has Narwhal the big machine to help target WHO to ask this, to give best insights to the model. Its a very personal question, obviously, and must be handled delicately. And everybody does not need to be asked, the model can calculate the most useful segments to ask, to fill out more of the puzzle. And of any given voter profile, as that becomes more precise with time, the team doesn’t need to ask everybody if they get a large enough sample of clearly that particular voter type. This will not get everybody, but by far most people are ‘predictable’ in this way, if we find their basic profile accurately enough. Only a few are truly undecided - as Jim Messina reported, only 7% were actually honestly undecided while many more claimed to be so (but in reality were already tilting in one way or the other, and only convinced themselves that they were still undecided, and were still considering the choices),

As the volunteers on phone banks and visiting homes and chatting via email, Facebook and SMS, start asking a large sample of a given particular segment with all the other data we know about that group, it starts to give more info, and whole swaths of the field get populated, and now the puzzle is more than half filled out. Then as time goes on, more info drips in, and soon the math can calculate ‘to reasonable statistical certainty’ every single voter, in every district. How they actually voted in 2008.

Now this is AGAIN  very powerful information to the campaign, when trying to convince voters. If we know that voter voted for McCain last time, the arguments to get him/her to switch parties now, and vote for Obama’s re-election are quite different to that voter who didn’t vote last time, or to that one who voted for Obama last time. Based on this, all calling bank volunteers had multiple scripts, and the telephone system would feed the appropriate script based on what data was pulled out of Narwhal for that voter. The calling bank volunteer didn’t know why a different person got a different script, but as the scripts varied (probably trained to handle say 5 different scripts per volunteer) that gave the calls a nice level of variety, as opposed to say telephone polling scripts which are always the same. This was also true of the people visiting homes, they had on their smartphones a summary of each target house and their voters, and which topics to talk about on that given house visit, which would be different per house.

In a real sense of knowing, like how Amazon knows what I bought, or Google knows what I search, no, Team Obama did not know ‘for a fact’ who everybody in the country had voted for. But on a statistical certainty, to say 95% confidence, they knew ‘accurately enough’ how everybody had voted, to target their campaigning like no campaign had ever targeted before. And as they had multiple ways to reach the voters, very often multiple DIGITAL ways to reach individual voters - their mobile phone number, their Facebook page, their Twitter handle, their email address - the microtargeting ability was unprecedented. So how was this used

WHAT YOU WATCHED

Then on TV viewing. Team Obama had by the summer of 2012 pretty well isolated the targeted 7% honestly real undecided (or probably narrowed it down to twice that, say 14%). Then it was a question of how to reach them most efficiently. Now Team Obama paired with a cable TV network data provider, Rentrack, with the type of set-top boxes that send detail of what is being watched. Narwhal knew who were the undecideds, and for them Narwhal knew their names and addresses. So they went to this cable TV service, and gave the target voter info as a file of names and addresses. Team Obama didn't say why they wanted a given set of voters to be searched, and probably did this numerous times with batches of target voters by type (could easily be 50 or 100 such voter profile groupings), so say single working women or retired factory worker men etc. But Team Obama used Narwhal to select in the viewing area of that cable TV company, a batch of customers who fit a precise profile that Jim Messina wanted to be able to target by their TV viewing. And sent a bunch of tens of thousands of names to that cable TV company. They had no idea what 'profile' these customers happened to be. They matched their customer data from their billing to find enough matches, say 4,000, and then mapped out what TV shows those 4,000 voters happened to be watching THE SAME on any given hour, actually to 15 minute intervals per hour. The cable TV company didn't reveal any identity of individual viewers but said that you can reach 78% of that voter base if you advertise on Rachel Maddow on Tuesday and 77% ot them if you advertise on rerun of Desperate Housewives on Friday, etc etc etc. What Messina now had, was the optimal TV ad targeting machine to essentially 'microtargeting' precision, that honestly reached the intended target voters, to incredible accuracy.

With all the info and lots of money and months to prepare, Team Obama then bought plenty of ad space far in advance, knowing this is a good target TV ad demographic, regardless of what the actual ad used maybe two months later, would end up being. Obama could buy ads at far lower rates than Team Romney, who was mostly reacting and had to buy at the last moment on far higher rates - on the same day, on the same TV show on the same channel, on the very same ad break. Reuters reported for example that rival ads running on CBS 'Face the Nation' in the Durham TV area in North Carolina, had Romney paying 1,100 dollars for his ad, while Obama had paid only 200 dollars for his ad, of the same duration in the same ad break. This was yet another area where it was not just better insights from Narwhal, it was also total staff superiority. Team Obama had a TV ad buying team staffed at 30, while Team Romney had one person doing the TV ad buys. Obviously driven by Narwhal, Team Obama selected WHICH ads to air in what states or TV viewing areas, while Team Romney ran consistently the same TV ad across all their TV media footprint at any one given time. One did precision targeting, the other did classic mass media bombardment.

FACEBOOK FRIENDS

Team Obama asked for permission to look through Facebook user’s friends. About 5 million Facebook users allowed this. Out of that rich playground of data, Team Obama discovered for example most of the unvoted young people who were of voting age and had not registered. Of those, Team Obama successfully convined 1.1 million to register to vote. A high proportion but by no means all, ended up voting for the Democrats. When registering youth voters (especially with a candidate like Obama) its a pretty safe bet that a majority of those will become Democratic supporters at least for now. The 2012 exit poll bears this out. They didn’t ask the age bracket who could not have voted before but nearly. They asked for the 18-24 year olds (under 22 could not have voted last time). For that group the balance to Obama was the strongest of any age group (always better the younger the age bracket) and was 60-36. If we project to from the age pattern in the exit poll, the 18-21 year bracket likely was 62-34 in favor of Obama. So yes, this is a game of numbers that only favors the Democrats. Register all youth, because you get twice as many voters than the Republicans would get. Same is true of the large outreaches by the Democrats of various minority groups like blacks, Hispanics, Asian-Americans, Native Indians, Jews, Muslims, gays etc. Not all who would register, would end up becoming Democratic voters, but the majority of all of those voter groups prefers Democrats.

But how did the team use Narwhal then with Facebook. This is where it gets cool, in a technical sense. Team Obama would send a video to those Facebook fans who had given permission to dig through their friends. While the video played, Obama’s FB app would link all the names from the current friends of that one FB user, to the 175 million names in Narwhal, latest fully upated within the past week and many updated up to today. And of that info, Narwhal would find the optimal 5 friends. Not any random 5 or even not the 5 that give that ad the maximum viewership. No, not old demographics. This was microtargeting. So Narwhal would first seek out those of that one FB user’s 150 friends, who were eligible voters, but who had not registered to vote and who had a positive Obama score. And it checked if they still had time to register to vote in their home states. They would be prioritized first. Then they’d look at of the remaining friends, who were registered to vote, and who had not voted yet (early voting) in a state that had early voting (and again, who obviously liked Obama too, no sense activating the uncle who was a Romney supporter), but had not voted yet. Those would be prioritized second. Then it would look for registered voters, who had not voted yet, who were uncertain of their vote or who supported Romney but were able to be persuaded, ie who had the optimal scores on the persuasion index. They were prioritized third. And so forth on down.

After that one FB user had finished watching a one-minute video, the Obama app on FB would end by suggesting four or five FB friends from that person’s friends, who to recommend showing that video to. It was very easy to just click to recommend to those friends. It would seem totally random to the FB user, but the four or five names were THE most useful names to Team Obama to achieve actions - register to vote or actually go and early vote or to donate money etc etc etc - and then to help convert the unconverted, and so forth. Narwhal also obviously know who of those 150 friends were already supporting Obama fully, they would never get that suggestion from this one user and knew who were ‘lost causes’ ie Romney supporters or who had already voted, or were not eligible to vote, and those all were ignored. The 'easy' assumption was that the Obama Facebook video forwarding was done to maximize reach. No. It was far more intelligent than that. It knew what that given video would be useful for (some were not intended to get people to switch their minds, some were intended just to reinforce Obama suppoter minds etc) and then picked in real time, while the video ran, out of our 150 friends on Facebook, which 5 would truly be most useful for the Obama campaign have watch that explicit video and right now. And recommended by THIS friend rather than say a direct email.

Note also, that Obama app also knew, of course the overlap. So again, it knew if say a 'John Jones of Tallahassee Florida' already has seen that video, from someone else via FB, then even if John Jones was a top 5 target from this user, then they’d ignore John and go to number 6 of his friends instead and so forth. Very very powerful, which evolved Facebook’s use of social media into new areas it had not done before 2012.

TO INTIMATELY KNOW

In the 9 battleground states, Team Obama had recruired an army of volunteers who went knocking on doors to really know the voters. They also did this in other states, but in battleground states, get this, of all registered registered Democratic voters of 21.5 million, 5.4 million were met in person at their homes! That is one in four voters. Not just spoken to briefly on a phone call, but met in person, at their home, to talk about voting for the President. Thats commitment. (Nationwide they did it for one in five registered Democratic voters). And of the rest? In those 9 states, of the registered Democratic voters who didn’t get a personal visit by a volunteer at home, of those 16.1 million voters, at least 3.2 million more of those got a personal phone call (I am assuming most who got a house visit, had received a phone call before or after and we know that total number was 8.6 million). But yes, 40% of all registered Democratic voters of the nine states where it was most close, four in ten had been speaking to the campaign, either by phone or by a campaign volunteer visiting their home.

But it was more than just one call or visit. The Romney team bragged that they had made 50 million total contacts to their voter base (this included leaflets and robocalls). Team Obama said they had spoken personally 125 million times to their voters. We now know that only 37.6 million Democratic voters were actually contacted so as to speak in person, ie at home or speaking on the phone. That means, that team Obama spoke - on average - 3.34 times per contact of that type! They kept calling them back, next month and the month after that, to see how its going. And of course then further kept in contact via email or SMS or Facebook, whichever was that voter’s preference.

So we get to the power of persuasion. We learned in my previous blog that Messina revealed only 7% of voters were persuadable. The rest 93% (for the 2012 election) had made up their minds and could not be persuaded by normal means in reasonable amounts of TV ads or emails or robocalls or personal calls or text messages etc. The election was for the 7% to see which side won them over (Obama’s team won 76% of them by the way). So it was only 9 million voters nationwide, and as the battle was only for those critical nine states, that meant 2.1 million voters in those nine states. That was the whole game.

SHOW ME THE MONEY

In terms of total campaign spending, Team Obama spent 875 million dollars in the 2012 general election (when adding Obama and Democratic Party contributions). Tea Romney spent 845 million (also Romney and Republican Party, plus the new SuperPACs supporting Romney). Not a big difference in total spending. Obama got 66 million votes and Romney 61 million so Obama averaged spending 13.25 dollars per vote earned and Romney spent 13.85. Lets see a bit about the philosphy and its impact. Romney used traditional demographics targeting of prioritizing Republican-leaning voting precincts when planning TV ads, robocalls, mailers, etc. Lets say he thus reached on average voting areas where two thirds were Republicans. So his efficiency out of 845 million dollars spent, was only two thirds, 566 million dollars. The impact across all Republican voters would be received relatively evenly, a Romney team push worth on average 9.28 dollars. That would be mostly safely decided Romney voters who were going to vote for him anyway but it would also reach those 7% in the middle who were persuadable and honestly undecided. They were hit on campaign activites that had a monetary value of 9.28 dollars. So the average of TV ads and snailmail and robocalls and emails and so forth. The average truly undecided voter was hit by typically 9.28 dollars worth of Romney effort, over the roughly three months of the campaign. A ‘bang’ worth 9.28.

Team Obama knew every voter and could target precisely, almost totally eliminating wasting ads on Republicans. Most of Obama’s marketing was using digital targeted methods, not mass market radio and TV ads or robocalls. So Team Obama’s bang was more like 13.25 dollars per hit, nearly 50% more powerful - if measured as an average. But then the lesson from Messina. He didn’t want to hit all voters once, he wanted to hit the undecided voter TEN TIMES. This gets back to the home visits and frequent phone calls to the same people. How expensive is it to go visit someone at home, rather than send them an email or a robocall. Its massively more expensive, but Team Obama could afford that. If they took say half of their total budget and spent it only on the 7%, and the remaining 50% on their half of the already essentially decided voters ie about 47% of voters - thats in dollar terms ‘wasting’ 7 dollars per voter to the safely Democratic voters to just reassure them - which is still two thirds of what Romney spends on all of its targets. And if thus Romney only spends 50% more than you, on safely Democratic voters, there is no risk of them switching (as long as you eliminated the persuadable voters).

But get this. That means Team Obama could spend an average of 91 dollars per PERSUADABLE voter. And those voters would also hear the Romney message - but only worth 9.28 dollars. 91 dollars worth of push from one side versus only 9.28 on the other side. This is not going to end well for the little guy. The Obama marketing machine would STEAMROLL the Romney message by essentially a 10-fold expenditure overload.

In the end, in the ‘battleground’ states, Team Romney was able to outperform John McCain’s effort of 2008 and add 200,000 more Republican voters. That was an impressive increase of 1% and would have made the election a razor’s edge nail-biter election with last states counting into the next days to decide. Team Romney did outperform the past with their ‘Orca’ machine and helped their cause. Only they were utterly outmatched by Team Obama with their Narwhal machine, which produced 900,000 more votes for Obama, and a 4.5% improvement to their side, which caused eight of the nine contested states to vote for Obama and gave him his massive victory. So its not just that you have good information about the voters (or in more general terms about Big Data, about customers or users or patients or whatever) but its if you can aim it so precisely to find that critcal 7% and ignore the others. And for those critical 7% to hammer them again and again with repeated contacts - at 10 times the volume of the competition, then if your candidate is not a total dog (like say haha, Donald Trump or Jeb Bush) then you will win. It should have been a nail-biter. It was decided early. And massively.

So Messina said that nationwide they won 76% of the voters who decided at the last moment. In the battleground states we see the margin was a colossal 82%. But that is what you can expect, if you outspend your rival by 10 to 1, for those voters who not only claim to be undecided or independent, but who actually are undecided and .. are persuadable. To get to that vital 7% you have to identify all who are Republicans, all who are Democrats, all who are registered to vote, all who have already early-voted, and so on and so on. You cannot know who is truly undecided by some poll or their magazine subscription or their Facebook page. You then have to talk to the possibly undecided voters, to determine who actually are honestly undecided. Only then can you wither it down to that precise 7% or about 9 million voters nationwide.

Then, in the US Presidential system, even thats not the number, you can discard nearly seven million of those, as the battle is not for the state of Texas which will vote Republican. The battle is not for the state of California which will vote Democratic. But the vote is for those critical approximately ten states that are the ‘battleground’ states in that given election. In 2012 that was nine states with 40 million registered voters. Obama’s team contacted three out of every four voters in those states and spoke in person to nearly 9 million (almost one in four registered voters) and visited the homes of 5 million (one in eight). That is how you pinpoint the critical 7% who were 2.1 million voters.Then while the opponent ran mass-market bombardment of all voters, you target precisely only those who are persuadable and you hit them 10 times more than the competitor. Thats how they won in 2012. The undecided 7% in the nine battleground states broke decisively by 82% for Obama, driven by a 4.5 to 1 advantage. Obama won eight of the nine states by the targeting and messaging and media choices driven by.. Narwhal

2016 IMPLICATIONS

So that was the state-of-the-art in 2012. We're three years beyond that and we know Hillary Clinton's campaign is supported by Jim Messina and several of the data geeks who were involved in the Narwhal project have joined the Hillary 2016 team. Their largest expenditure in their campaign is.. the mysterious and massive project around Big Data. Expect it to be a kind of 'Narwhal 2.0'. And by the money spent so far, they are probably aiming to about double the spend compared to 2012. Also part of the effort done so far (including the serious data systems built by the Democratic party over many election cycles) will certainly build upon the lessons and upon the data generated in 2012 that obviously the Republicans do not have (like those 5 separate 100 point scores per voter in those nine battleground states). They also have a lot of tested data algorithms that now can be taken as remarkably accurate to help predict and direct activities for the campaign. We have heard suggestions that the efforts will shift to previously pink-color states, possibly Georgia, Arizona, Indiana, MIssouri. And if Hillary gets a more favorably tilted battleground (ie she holds a steady slight lead against whoever is her rival) then a few of the most 'blue' battleground states like say Nevada and Wisconsin - might not be really in contention this time and could be categorized safely light blue. But it would mean, that for new 'pink' states, the game starts from zero on the Democratic side. All those voters will need to be contacted in ways that Team Obama did in 2012.

On the Republican side we know Jeb Bush, Scott Walker and Ted Cruz (at least) had all invested in a data driven voter insights solution, whether actually Big Data or not. We also saw that Walker quit the race and Jeb's been so badly struggling, he suspended the data project and reassigned the staff to go try to find Jeb some support in actual early voting states. So the only Republican candidate with the early funding and the foresight to build something to try to rival Hillary's system is Ted Cruz. His campaign has been reaping enormous benefits out of their system already in very similar ways to what Obama did, even to the degree now of deciding where to send Ted to have events next. As the team is discovering the real power of Big Data, they will devote even more to this tool and no doubt, devour all public info about the past systems especially Narwhal. So hello boys and girls from Cruzland. Nice of you to visit here and good luck! Because the digital world is governed by Moore's Law, a team now building from scratch can get cheaper and/or more powerful tech into use and could be catching up, or even leapfrogging the leader who ran on the legacy concept (Obama/Hillary). 

I do foresee a kind of arms race and a kind of balance of power. Now that both sides do know that this tool exists and have deployed it, and have learned the basics of what it can do, then neither side can anymore claim a decisive victory out of using it. But neither side can afford not to deploy it to the fullest. Expect both sides to devote something like at least 10% of their total campaign budget and staff to their Big Data project and run all decisions through it. And that brings us to Trump. He obviously doesn't have a big team or bother to run anything through any decision-making system at all. Would not be 'politically correct' and might interfere with his desire for spontaneous schlonging. It does mean that over time, the team(s) who use Big Data and pre-test their responses and actions, will be prone to not make mistakes but Trump can capitalize faster on some fast-moving matters. But we've already seen that Trump is a guaranteed gaffe-machine and once the political system figures out exactly how to pin him to all his lies and nonsense, his campaign will crash and burn to the degree its 76% lies - most of any politician ever measured by PolitiFact - warrant. The campaigns who can afford to build a Big Data based solution are viable for the general election and long run. And then we should see plenty of good new fresh exciting uses of this tech. I love competition and hope both sides will have a robust big powerful system with plenty of clever ways to use it. And I hope that I can get to report on the uses too, similar to what was in this article - and after the election of 2016, hopefully also have new metrics to see what the new Big Data arms race brought us in performance. Good luck to both sides! 

These are several of the ‘use cases’ of how the Big Data machine by Obama 2012 was used. I wrote the big performance metrics and statistics blog, with side-by-side comparison of Narwhal vs Orca, shortly after the election. I will return to write a newer and more complete (and corrected) version of that at some point. Now, if any of my readers know of more details of the epic Narwhal vs Orca battle of databases in elections, please tell me (but read my five articles talking about it first, odds are, I have already discussed that aspect already).

Comments

[Wayne Borean]

First, I have a confession to make. This is my rating from the Levenson Self Report Psychopathic Scale:

You score for primary psychopathy was higher than 85.7% of people who have taken this test.

You score for secondary psychopathy was higher than 63.38% of people who have taken this test.

Yes, I'm a Psychopath. Borderline, but a Psychopath. Most of my friends know this. This doesn't mean I can't be loyal. I understand the value of loyalty when building an alliance, and therefore am actually more loyal than most. This doesn't mean I don't value friendship. My understanding of it may differ from yours, but I understand it, and the value of it to both parties.

It does mean that I probably understand politicians better than anyone else commenting here.

*****

Another point is that I'm familiar with the use of data. My last position, where I was targeting Fortune 500 companies as the 'Major Accounts Sales Rep' and who also was one of the top five technical people in the company (for much of that time in the top two).

We used data. Unfortunately there are limits to data which Tomi is ignoring. Let's look at them.

1) Data use to 'turn' voters/customers is only useful when the VOTER/CUSTOMER IS UNEDUCATED ABOUT THE OPTIONS. This may seem obvious, but I've squashed competitors who didn't understand this.

2) The corallory is that all Voters/Customers are undecided, they just don't know it. Yet. Take the voter who has voted 100% for Party A since they first came of age (if my age, that would be the 1976 U.S. election).

3) A system is only as accurate as the weakest link. The weakest link is often the link with the lowest number of people, where a Manchurian Sysop could do more damage, than a Manchurian Door to Door Representative. Data corruption is a huge problem, especially if the corruption occurs before it enters the system.

4) Data is only useful if the people using it are capable of understanding trends/patterns. Tomi argues that Narwhale gives the operation this capability. I'm arguing that the system will only do that with a 'Loyal' psychopath running it. To quote Susan Ivanovs from Babylon 5, trust no one, except for Susan Ivanova. And considering what a P10 could do to her, don't trust her either.

Where am I going with this? Most high level politicians show some Sociopathic/Psychopathic tendencies. As do most successful sales people. Like me.

They also tend to be smart. Not necessarily genius level, but enough smarter than average, that they can see trends, and make use of them. Yes, you get people like Mr. Legitimate Rape. Ego also plays into this, you can't be a politician unless you have a big ego, but ego alone, even with genius level intelligence, won't take you to the top. You also can't be 100% tied into your party's ideology. Ideology plays well with a certain portion of the electorate, but most people are in the middle, and it doesn't play well with them.

All of the major candidates, with the possible exception of Ben Carson, show enough in the way of sociopathic/psychopathic tendencies, that they could do literally anything, assuming they think that the upside is greater than the downside. This is what makes politics my favourite form of entertainment. It's the best of all the Blood Sports.

We used 'Big Data' to determine who to target for sales. Often I saw what I considered errors in the database. I drove management nuts. A company would be tagged 'no hope' and I'd make sales to them, and I'm not talking one time sales, I'm talking consistent sales over periods of years. They nicknamed me 'The Icebreaker' because I wandered around in front of the rest of the company, and they often didn't understand why I did stuff, all they knew was that what I did worked. For me. Anyone else trying it crashed and burned.

But Big Data was useful. Take Hospital A. I had no interest in them. Sure, we might be able to sell them a quarter million worth of goods, but that was a 'one time' sale, and I was going after long term consistent sales. They might be of interest to another sales rep who was working a different market though, and that's where our database paid off big time, since we had multiple product lines, and multiple marketing plans based on those product lines.

So, let's take Donald Trump. I said in the previous article that Trump was damned smart. He's also got an ego, but he's lost often enough to know that ego can be a negative, so he's playing the game carefully. Yes, I know that to a lot of people he looks like a bull in a china shop. Sorry, he isn't. He wouldn't have gotten as rich as he has, if he wasn't aware of his weaknesses. The same applies to Hillary Clinton, Ted Cruz, etc.

Let's say that Candidate A has worked out that Plan Y will tip X% of decided voters one way, but tip Y% of all voters the other. What are the number of votes difference does this make?

Don't forget that everyone has been thinking about Narwhale since 2012. That's three years, and you can do a lot of development work in three years. You can do a lt of on the ground voter research too.

While I have no doubt Messina was right about 2012 being about the 7% of undecideds, the number of voters in play is far, far, larger, if you can target the RIGHT ISSUES TO BREAK THE DECIDEDS LOOSE.

Which won't be easy. Both sides are probably doing their own evaluations. And coming up with plans.

But it is doable. I broke 'DECIDEDS' loose all the time. That's why I was a pretty good Sales Rep. Not the best, there were far better. But I was good at developing customer loyalty, and when we landed them, we kept a pretty high percentage. Heck, just delivering on time was often the 'Killer App'!

But Big Data made my life a lot easier, even if I didn't trust what it said most of the time. Big Data can't win the election on its own. It can however be a huge advantage, especially with a Sociopath/Psychopath in the mix.

Oh, and my guess is that Obama is about the least Sociopathic/Pschopathic President the United States has ever had...

[Wayne Borean]

Oh, and if you want to take the Levenson test, here's the link:

http://personality-testing.info/tests/LSRP.php

[Tomi T Ahonen]

Hi Wayne

Hey, thanks for the link to the test (I didn't know there was a public place to take the test online, but of course there would be in modern times haha). I vaguely recall that I am kind of very sociopathic on one scale like not caring about others but very unsociopathic on the other scale ie typical Finn, unwilling to break rules or something like that.. if that makes any sense. We went through so many tests back in the day haha, corporate life, haha.

And thanks for sharing your personality profile haha. I never imagined this blog could be also a self-diagnostic psychiatrist's bench haha.. feel free to lie down and tell us about your fears :-)

Now onto your experiences with using 'data'.. I also was in sales, I hear you. I've also been later building one of the largest, certainly largest multi-vendor database of its time, and very pioneering customer data work. That all is ancient history, its like comparing a typewriter to a word processor, or comparing a fax to email. No, Big Data means we know every CONSUMER (or whatever equivalent like voter, patient etc). That never existed before. In Business-to-Business sales - that we've both done, yes, we know - or can know - or should know - a lot about our customers. Thats different. That is handled with personal sales reps. Me too, I would take the challenge from my boss, to go land the customer he said was impossible to win (my two favorites, the United Nations Security Council - my network - and the Economist Newspaper Group's brand new global HQ when they moved from London to Manhattan - that was also my network that I sold to them. Both were projects my boss said, don't even try, you'll never be able to win those because we'll have 100 rivals bidding.. But I won them both).

Big Data enables automated non-human interaction with customers (voters) at such a deep and personal level, they feel like they are treated as a unique individual, as if it is the promised 'segment of one' - that nobody else gets that exact same treatment. That is why the original Obama 2008 campaign had two scales of 100. 100 x 100 = 10,000 which is mathematically the segment size that in our lifetime we will not get to know more people than that number. We will get to know less than 10K people (normal people, most jobs, not say haha an over-talkative taxi driver). But the original Obama 2008 campaign would not actually distinguish by that level ie the person scoring 86 would not be treated
to a separate campaign contact than the one scoring 85 and the one scoring 87. They used bands on those numbers, ie say everybody scoring from 81 to 90. But now with four election criteria at 100 point scales, they can take those bands to 10 point intervals - ten bands per scale - and now its the math of 10 x 10 x 10 x 10 = 10,000 again the perfect segmentation precision that you need not go deeper. Or there is no gain to going deeper. So the fifth scale was about fund-raising not the vote. But my point is, this is a revolution in sales/marketing/customer relationship and the power of it is unprecedented. What you talked about is not the same, because no party can afford to hire a 'sales staff' to speak to all voters enough to SELL them. It has to be done automatically, with emails, YouTube and TV ads, with SMS and Facebook etc. Yes, some discussion will also take place, with some voters, on the phone and in person - if the campaign can afford that - but most of the 'sales' has to be automated, mechanised.

The Obama campaign released 1,000 YouTube videos. It couldn't have had a prayer of that conversion of their disgruntled voters, if those videos didn't exist and were not reasonably well targeted.

That said, you make good points about all the possible pitfalls of ANY database. GIGO. Garbage In, Garbage Out. Data can get corrupted (I once met a crazy lady who had deliberately destroyed the organization of her previous employer's database - scrambling the data so each address field and each phone number etc were now pointing to the wrong file. Then she ensured it was saved a few times to have the backup systems also corrupted. I was horrified to think I knew such a monster but she was utterly cool with it. That was her revenge for a very minor slight... then she quit and got a job with a rival company doing a similar job. If I'd know her boss at that company, I would have gosh warned them about a real psycopath yeah..

On Narwhal and data in it. Yeah. A small data system could have a pretty clueless and poorly trained, perhaps too technical boss. But this was 100 of the best and brightest data scientists including very close correlation with literally the cream of the crop from Google, Amazon, Facebook and HP. And serving obviously the re-election of the President of the USA. Thats not going to be clueless idiots. It doesn't mean having a Big Data system will solve all problems but for THIS type of problem - where to target voter advertising and marketing efforts - for this, its the dawn of a totally new era. I don't know if you've gone and read the numbers blog (comparing Narwhal to Orca) that I wrote in 2012, which is linked to at the top of this article. Wayne, I suggest you go read that. Then you will see, what is old and new, compared side-by-side, and understand what I mean.

This is the jet engine to flight. This is Maxim and Gatling to guns (ie birth of machine guns). This is Little Boy and Fat Man to bombs (ie first atomic bombs). This is genesis of the new way, and it has now been measured and compared side-by-side. I personally saw one of the first Big Data systems being deployed and taken into use (obviously in telecoms where this technical ability to this power was first available) but all of that was utterly secret, still to today. The nice thing about Narwhal (vs Orca) was that its now an obsolete system - so they CAN talk about it without fear that Romney will somehow now steal the 2012 election from Obama. That is why its the only public reference case of using Big Data nationally, up to now, in ANY industry. And such a powerful case example. Comparing the very best of the old way - Romney's team had for example Microsoft helping them to try to reverse-engineer what they witnessed Obama doing, but then attempting old-fashioned demographics and mass market targeting to defeat his new way. Its like trying to use bows and arrows against an army with guns, or trying to fly propeller-driven biplanes against jets fighters.

So I suggest Wayne you read first that comparison article. If then you have questions we can take it here (or there) and discuss further. The term 'big data' does not mean a large database. It means a type of (enormous) way of organizing data so that EVERY consumer is in it and the data can be used. Like 175 million voters in Narwhal. If you are an American, Narwhal knows if you voted, how many times you voted, who you voted for each time - plus what TV shows you watch, what magazines and newspapers you read, if you have a Facebook page, almost certainly they know you from there, and they know your home address, phone numbers, email, employment history, credit record, all the possible data that can be purchased - and then some. A credit card company or a bank would LOVE to have that level of detail of every American, as Narwhal collected in 2012. It was the most complete consumer database (of those 175 million Americans) EVER ASSEMBLED ANYWHERE by any company or entity, at that time. Today, probably, Google and Facebook might have similar level of data now, and Hillary's Narwhal 2.0 will want to go even deeper than that...

So yes, some databases had more people in them (Chinese and Indian telecoms operators for example) and even some American databases had more people as entries - like the IRS database and the NSA telephone spying database - but those had NOTHING like the LEVEL of detail on the consumer behavior. The IRS doesn't care to collect data on what TV shows we watch or what magazines we read or who are our 147 friend on Facebook etc... But seriously, Wayne, its a revolution, not just 'another large database, yeah, our company also had a large database' Big Data is something as different to data, as email was to mail.

But lets discuss after you've read that article, ok?

Tomi Ahonen :-)

[Tomi T Ahonen]

Hi Wayne

Now to the non-data parts at the bottom of your comment ie Trump etc..

So yeah, you say Trump seems like a bull in a china shop but he really isn't. Thats possible. But I am now convinced this is not an act. Just now, 'schlongate' - I lived 8 years in New York City, Trump knew perfectly well what a schlong is, every New Yorker knows that is a vulgar Yiddish word that's often used in only a derogatory way, in New York. Maybe he didn't mean to go full potty-mouth but he did. He's sunk to a new low. But in that same event, he was ranting about it being somehow disgusting that Hillary Clinton went to the toilet? No. This is a sick old fucker, exactly the loonie uncle of the family who should be locked up who has really lewd jokes (gosh, his constant creepy references to wanting to bang his daughter, for gosh sake) and now somehow going to the toilet is 'disgusting' ?? Earlier at the start of the campaign, he was accusing a woman attorney who wanted to take a break to breastfeed (not in his company) as being disgusting. Trump is that old guy who never saw a woman naked and thinks everything about women is disgusting. No, he is just weird and this will not end well for him. This is not an act. But when he is finished, you know the two things everybody will cerenade him with - all the video parodies and late night comedians - playing back Trump's own words saying he is the biggest 'loser' - and now that he became schlonged.

I agree with you, he should be smarter than this. But he has a creepy sick old man's attitude to the TOILET. And to natural human anatomy and normal biological phenomena. Who says going to the toilet is disgusting. Disgusting? So no, I totally now reject that this part of Trump is an act. He must be smart to have gotten where he is, but he is going soft in his head. What adult says going to the toilet is disgusting? A five-year-old might say that. A five year old. And schlong-gate. No he is a creep. And a weirdo. And this is not going to end well for him. The more he pursues the sad old weird guy path, the more he won't win even the nomination. But yeah, 50% of Americans already say they'd be ashamed if he was President. Thats six out of ten WOMEN. This was before schlong-gate or disgusting toilet visit. Trump is burning ever more of his ability to expand any voter base. The more he does this kind of schoolyard talk, the more he only appeals to those World Wresting Federation type voters - men (white, older, less-educated men). I just took a freeze-frame of one of his rallies and counted, men outnumbered women by more than 3 to 1. (yeah, I know, its a sick habit you pick up when you do a lot of public speaking, I always count audiences haha)

But I think you were a fan of Trump and volunteering for him, maybe (my apologies if thats not you, and am remembering another of the people on the blog). So let me leave the Trumpmeister and lets look at the other people you mentioned.

On your X and Y example, it came across a bit confused to me. I think you were suggesting what if candidate A has plan X, and simultaneously candidate B has plan Y, and they hit the same undecided voter. Yeah. Good point. This is exactly what I meant, in what I expected to happen in 2012, when both sides were developing in secret their big voter insights databases. It didn't happen then. Now it SHOULD happen, that both sides have gotten enough up to speed, to be roughly at parity. Like World War 1, when both sides had acquired machine guns and it resulted in a stalemate. I do expect that to be 'the norm' by the NEXT election cycle, but this to be still a stage of experimentation, where either side can discover new powers and tactics, for using this powerful tool and to derive an advantage (at least in a short-term say week-month level time frame).

In 2012 when Obama had the tool and Romney didn't it turned a one million vote margin squeaker into a five million vote blowout - boosted Obama's turnout by four million. I expect the 'arms race' will produce a rough balance where for 2012 this - the advantage o a superior Big Data system - will be no more than 2 million - either way. It can turn out to be totally cancelled out and be a net zero advantage after both sides dump 200 million dollars into it haha. By 2020 it SHOULD be roughly a net-zero proposition. Just a 'must do' like today in business, all companies have to have a website, email, a Facebook page, etc. No more a competitive advantage today but you have to do it.

On the undecideds (and persuadable ones at that) yes, I agree in 2012 it should be more than 7% but I don't think it will be anything like the 20% it was in 2000. Very many voters already know how they will vote in relationship to Hillary Clinton - many Republicans will vote against her essentially regardless who is their final nominee and most Democrats already know they will vote for Hillary rather than whichever emerges from the race on the GOP side. But yeah, I think the middle will have more undecided voters. But those will be dramatically more on the 'leaning Republican' type of voters than 'leaning Democratic' exactly because of the past years of the Republicans alienating so many in the middle. Those who lean Democratic are rather happy with Hillary (and would be with Bernie too). Not all, as Hillary's favorability is under water, but most are.

On the issues that break the undecideds loose, in fact, on issues that CHANGE minds of regularly loyal voters - that is EXACTLY what Obama did in 2012. They did not hire some expensive consultants to sell them stories of what should work. They rather TESTED what works on real voters. Every day they ran 5,000 short surveys and 1,000 long surveys per battleground state. Every day. So. Would this ad about college tuition change the minds of level 20 to 30 Romney supporters (ie only 20% to 30% willing to go with Obama, these are very likely Romney voters) but college-students - the kinds who are 'Young Republicans' on college campuses - would this particular Obama ad about tuition, change their minds. Show the ad to selected 200 such Republicans in Wisconsin, another 200 in Florida, another 200 in Colorado, another 200 in Virginia, another 200 in Ohio. Then after we know they've been exposed to the video (via say Facebook distribution) now TEST how many of the 1,000 who saw the video, changed their minds. 7 did. Ok, the power of that ad is to change 0.7% of clearly strongly Republican supporters. Now try the OTHER version of the ad, where its about earning tuition for going to the army. And another 200 Republican youth college students in those same states. And now see how many of them changed their minds. Now its 2.2%. And so forth.

This level of TOTAL knowledge of EVERY marketing communication by a campaign - pretesting everything - has never been done. Previously it was a couple of highly paid consultants who said, I am an expert on making good TV ads, I won the awards, I say ad type B will work better, and run it... The opinion of some experts who made educated guesses. They could not hope to give a NUMBER of how many actual voters would change their minds - and in the normal clutter of TV ads, its utterly impossible to isolate which ad did what. Maybe that college student changed his mind, because he got a new girldfriend who is an Obama supporter (or has trouble paying her tuition). It was never possible to pre-test this type of TOTAL electorate response to every communication. By old methods, with Gallup, Nielsen etc, it would have been prohibitively expensive to try. But now this is AUTOMATED and run nightly, testing not just whether it will work but WHERE it will work, and WHICH media will work best (or indeed, which combination of media).

The Obama team knew 'everything' in 2012. They knew they would not be able to win another state (North Carolina). They KNEW it. Obama didn't waste campaign visits there. It was always called their 9th battleground state but Narwhal told them its a hopeless cause. So they never ran a massive ad budget there or spent Obama's time there. Instead, they focused on Florida which went 4 days past the election until the counting was finished. It was the right move. Driven by KNOWLEDGE. But Romney, who used the old way, had Karl Rove having a fit on Fox News that no, Romney has not lost, the data has to be wrong... haha. That was epic.

Now on Big Data cannot win the election on its own. You are 100% correct. It can help a campaign but a bad candidate (say like Jeb Bush now) then it doesn't matter how much great data they'd have - the data would tell them the same - this is a lost cause. Exactly like Obama and NC. Having 'perfect' information can only help you understand what is happening, if you are going to lose, then the perfect info will tell you that too. As it should. Part of why Obama was ahead of Romney almost the whole cycle of 2012 was, that Obama was a better candidate (while flawed as President). And Romney was a rubbish candidate. Trees are the right height? My wife drives a couple of Cadillacs? Corporations are people, my friends?

On the last item Obama least sociopathic Prez, haha, yeah I agree on that. I'd like him to have a bit more of the gut feeling reaction by now, but early on, gosh that was soothing after 'I am the Decider' W Bush who only shot from the hip.

Tomi Ahonen :-)

[Dave Barnes]

OK, so you know everything about me. But...
Let's assume that I am a 100 on voting. (This is true as we have vote-by-mail in Colorado and I have only missed one election (municipal) since 1970.)
Let's assume that I am sitting on the fence and am persuadable. (Not true, but let's assume.)

How do you reach me?
1. I mute ALL TV commercials. All. All the time.
2. I switch radio stations in my car as soon as I hear a voice. I want music, damnit.
3. While I have 2 Facebook profiles (one real, one fake), I never use it for anything. No website visits, No phone app.
4. I trash 100% of junk mail. (There could be a check for $1M USD on the backside and I would not see it.)
5. I trash First Class mail if it appears to come from a campaign.
6. I ruthlessly trash all spam emails.
7. When someone knocks on my door to talk/solicit, I ask them: Have you accepted nuclear fusion as your lord and savior. They always answer incorrectly and the door is shut.
8. I rip all the ads out of magazines before reading the articles.
9. I block 99% of all ads on websites. Thank you Ghostery.
10. If you call me on my land line, I will hang up quickly.

So, how do you reach me and persuade me to change my mind?

I do read 11000 word blog entries written by crazy Finns, but are the campaigns going to pay Tomi to insert "adverts" in his writings?

[Chris Vail]

One thing to consider in your projections for the future of US politics is the fate of gerrymandering. After the 2016 election, there will be the redrawing of political districts based on the 2020 census. Anti-gerrymandering sentiment is rising in this country, and could be very important in the redrawing of those districts. Consider that gerrymandering ended in California during the Schwarzenegger administration, and the next election resulted in a super majority for the Democratic Party. If gerrymandering were eliminated from the US nationally, the Republicans would be a permanent minority party.

So, this Presidential election is a big deal for the Republicans; it is do or die time. And you see how they are behaving.

[Wayne Borean]

Tomi,

I may not have explained what I meant fully. Been in a ton of pain today, which messes up my thinking.

Yes, I've read your previous Big Data articles. I've read everything you've written going back to before Nokia hired the Manchurian CEO.

Trump is not stupid. While I'm not certain that he didn't begin the campaign as a lark, I'm fairly certain he's serious about it now.

Donald Trump has an ego the size of Hudson's Bay. He also hates loosing. I think we can both agree on that.

So why would he open himself up to loosing the biggest Deal he's ever negotiated? Simple. He wouldn't. And he isn't stupid. He knows what damage he could do to his reputation, and his legacy.

So obviously he thinks he has a way of pulling this off. I have no idea what way, but I'm 100% certain he thinks he can do it. Can he pull it off?

I'd say it is pretty unlikely. About the only demographic he hasn't pissed off so far is Dead People, and they aren't going you vote for him. Except maybe in Chicago.

But he thinks he can do it, and it would be the greatest Negotiated deal of his career!

As to the our corporate data use, we had far greater segmentation than the original Narwhale. We also had more data on customers. We may not have had as many records. Hell, we had no where near as many records. But we were told by our database software supplier that we had the largest database they'd ever seen using their software!

So I know what data can do from a practical stand point. It sure made work a lot easier, knowing what we did. We could, and did, target every customer with a certain profile, in a certain area, when the circumstances warranted it.

Where a database can fall down is if the information is wrong, or just marginally off. Take the 100% committed group - which I usually belong to long before the election. I do a lot of research on party policies. I also do a lot of research on candidates.

But I can change my mind at the last second. It depends on the wedge issue. Consider Bill C-51. In the last Canadian election, if the Liberal Party had committed to revoking Bill C-51 at the last minute, I would have voted Liberal. Because they didn't, and our local NDP candidate was a fantastic candidate (while the Liberal was a nebbish), I stayed with my decision made six months before the election was called that the NDP was my best choice.

So what wedge issue would convert voters? I have no idea, but I'm sure that the Trump campaign has identified several that they think will win the Presidency for Donald Trump. They may be dead wrong. Hell, I'm sure they are dead wrong, eleven months before the vote. But they must be certain they've identified what will convert voters.

Because Donald Trump hates loosing. He isn't a gambler. He's a hard headed businessman, who identifies ways to make money, and goes for them.

Which will make the next eleven months amusing.

Ah, here's one last scenario. What if Donald Trump wins the nomination, and a month after the convention decides he has no chance, and drops out, maybe as part of that hypothetical deal with Hillary that you mentioned. Wouldn't that be an interesting mess...

[Wayne Borean]

As to Trump quitting after the convention, while I can't see Hillary being stupid enough to make a direct payment to Trump, I could see her government giving him some Federal land at fire sale prices. Congress has done this before. In at least one case that I know of, John McCain slipped a rider onto an existing bill which gave a copper mining firm a large plot of land which included a First Nations sacred site. The First Nation involved is furious, and fighting the giveaway. The copper mining firm was a big McCain Campiagn contributor...

[Tomi T Ahonen]

Hi Dave, Chris and Wayne

Dave - good summary and yeah, I am sure you are not alone, there are plenty of other people who behave similar to you. Most of those people would not be truly undecided, however. So a person who hates all ads in general, and hates all political ads and contact in particular, could still have a clear choice in the election (or be leaning one way or other) so even of people who behave like you, it would only be about 7% of those, who also are undecided..

Now, I get it, you are probably nearly unreachable in practical terms by the marketing communications of a campaign. They can still hope to influence you by the non-advertising type of communications. So when the candidate is in a debate or interviewed by the local TV news or radio on a local campaign visit to Colorado. I would guess you still follow he news in some way whether local newspaper, TV news, internet news websites or whatever. The campaign should identify you as the type of voter who is almost unreachable by the various outreach programs and then use the public communications to still try to address a voter like you. So then find out if there are enough voters like you in Colorado who may be currently persuadable by a given message by the candidate, and ensure that the message comes out in the speech made locally, or the TV interviews with local TV or press, etc. To just try to change your mind, Dave, obviously is too costly but if there are say 7,000 people like you in Colorado, who might share one issue where our candidate differs from the rival, then it would be useful to include that issue on the stump speech in Colorado.

Now one last item. Forwarded info. Email blasters, SMS texts, Facebook and YouTube have been designed for easy forwarding. Almost all people trust their closer friends enough, that if I say get an email from my brother, or a YouTube link from my nephew, I will open that and look at it. You may be so rejecting of all normal ads and unsolicited communications that nothing else penetrates, but a relative or closer friend, I mean one of the dozen close ones on Facebook, not one of the distant ones out of the about 150 - that could reach you.

Now take in Big Data. Big Data should accurately identify you as somehow totally isolated. And still attempt to gauge your actual preference and as far as possible (out of studies of similar people) estimate your persuadability score etc. Obviously if you are expected to be already decided, no sense trying to target you. But lets say the system indicated you can be persuaded, and on these few issues. Then as the big communication machine churns out the contact wars, it will land at some point to a Facebook user who is friendly with the campaign and lists you as a (possibly distant) friend. That is how they can now have an inroad to you. If they use that distant friend to forward you say a YouTube video - and they see you didn't look at the video - they can then wait a while, and try to find another Facebook user who has Dave as a friend. And this way, over time, possibly hit one of your close friends and relatives - whose recommendation would get you to watch the YouTube video.

Let me add here then, that if you honestly are undecided, and you are deliberately avoiding all ads from all campaigns, in that case seeing one, that is tailored to issues that resonate with you specifically... that would be FAR more powerful than seeing the even the same ad in the mix of hundreds of political ads.

But you are correct, that type of profile is very well isolated from most attempts and very hard to catch (and often not worth attempting). Now consider again Big Data. The machine learns and MEASURES everything. They will isolate the target audience (that 7%) and then work at that number. They will retest and recheck to see is it really 7.3% or 6.6% and refine and re-evaluate. They will keep track of EVERY voter including you on how many contacts went through. Remember the AVERAGE of contacts the Obama campaign hit on its voters was 3.3 times in the 2012 election. So some will be five times, some only once. But as the machine keeps tabs of who was contacted, they will prioritize of course those who still had no contacts. And they might send a volunteer to your door to face your intriguing question. And you know what. They'd register the response - and might even have that issue covered, where the NEXT time they try, they have the person prepared for not just handling nuclear fusion as lord and master haha, but ALSO now being prepped that this Dave dude is very isolated from political ads, take him seriously etc...

I think in more than 50% of the time, you win. No campaign would breach your defenses haha. But still a determined well designed campaign could. But of the 7% who are undecided, how many would take the steps you do? Far less than one in ten. If the campaign can successfully reach 6.3% of the 7% haha, then they'd be probably very happy.

PS thanks for reading my long articles Dave. And just so you know, I get offers every single week to put ads on the blog and always say no. But there are not many blogs with millions of hits lifetime who are ad-free haha...

Now lets move just beyond ads to marketing. We wrote in the book Communities Dominate Brands that the traditional advertising-driven business model is broken and will be superceded. So lets take the enthusiastic supporters of either party. Get them INVOLVED. Let them participate, co-create ads and mailing lists and do contests for use-generated ads and run Twitter hashtag parties and whatnot. In politics into the 1980s-1990s, that was only done by 'top down' from the campaign and to a very modest number of volunteers. But Obama had 400,000 volunteers WORKING for the campaign. 4 Million donated. How many more than that, were involved in some way, online or in the real world, helping the campaign. Same (to a lesser degree then) for Romney and now probably close to even on both sides for 2016. 4 million donating means one in sixteen people who voted for Obama, also donated money. One in sixteen is so huge a number, its very near the one in twelve of the 'real friends' we have on Facebook. On average, three out of every four Obaa voters, who is on Facebook, will have as a 'real friend' someone who was so engaged with the campaign, that person also donated to it. Now fast-forward to 2016, it will be that almost everybody on Facebook - on either side - will have one of the close friends or relatives, who is donating to the campaign (or otherwise very strongly supporting it such as volunteering).

Now do we want our close friends to come to our house to sell us encyclopedia sets (haha they don't exist anymore) or have tupperware parties or come and convert us to Jehova's Witnesses religion - or to convert us to join a political campaign. That relative or friend may soon become less close haha. But these are activists and THEY will know who of their friends are LIKELY to enjoy joining the 'cause'. I think it is a better way than just propaganda on TV ads.. But yeah, nothing works perfectly :-)

PS Dave, loved the total description of what all you will block and how. I do some of that, definitely not all of that :-)

Chris - very good point and I totally agree. In very rough terms, since WW2, the USA has been a 50/50 split country in Presidential elections, in Senate and House elections and even to a large degree statewide elections. Recently the Republicans have drifted away from the middle, towards the Tea Party and conservative edge while the nation has become a bit more liberal than in the past (support of gay rights, gun laws, legalizing pot, abortion, all trend more liberal). The Republican response has been the defense by voting tricks. Limiting early voting, demanding new voter ID that minorities often do not have (who are stronly Democratic-leaning voters) and other such tricks. Gerrymandering is the most powerful of those tricks (and both sides have done it for more than a century) but nobody has managed to gerrymander as effectively as the Republicans did out of the 2010 Census so that today they hold a 7% tilt in the election results. A 50/50 election result in a gerrymandered state will send 7% more Republicans to Congress than Democrats. Or to put it another way, the Democrats have to win 54/46 just to get EVEN number of members of Congress. And added to that, is the Supreme Court ruling on Citizens United, which allows unlimited campaign donations to SuperPACs. While again, both sides can use that, it favors Republican enormously as a few rich Billionaires can flood the elections with their cash to buy their own candidates, where traditionally Democrats had more grass roots support like labor unions, and Republicans traditionally had support of rich people who donated more money but were fewer in number.

The last Presidential elections have tilted to the Democrats (they won more votes in 5 of the 6 past elections) so the 'reality' on the ground, by American voters, is slightly in the Democrats' favor. But the Republicans now are sustained by 'cheating' on the rules. Once that 'cheating' is removed and the rules are truly fair (again) then the reality of Democratic dominance becomes blatantly obvious to Republicans. Their inevitable long term solution is to change and pursue the middle-ground (again). But before they get to that point, they will try every other way and excuse, to prolong the pain, and try to live out their delusions. That is why if Ted Cruz wins now the nomination in 2016 would be far better for the party than anyone else. If Trump or Rubio (or Christie or Jeb) becomes the nominee - and loses disasterously to Hillary - then the Republican party will find solace in the fantasy that their candidate was not 'conservative enough' and had they only nominated Ted Cruz, they could have won. Only after that illusion is shattered, that the Tea Party gets its guy to run - and lose badly - can this wishful thinking be extinguished.

Now to your point, meanwhile the Republican party and its various politicians are making the LONG TERM pain and recovery WORSE by all the idiotic positions they are now taking against minorities, for guns, against abortion, against minimum wage, against the environment, etc etc. The REASON they can be so confidently against the obvious moderate middle of America, is that perception that they are 'winning' in Congress and the Senate and in the local governments in the states. The correction will take a few election cycles to be complete. It won't come until the Republicans have extinguished all hope (ie a Ted Cruz style totally conservative candidate runs and loses). And they cannot fix something if they think it is not broken. Which is why the Hillary 8 year Presidency that takes back not just the Senate but if not now, then eventually, also the House - and flips the Supreme Court to a liberal 5-4 balance - that will be the shock (and 16 years of a Democrat in the White House) that convinces them, that their ideological dogma is the problem, they have to become more moderate or face extinction as a party.

(I'll get to your comments Wayne next..)

Tomi Ahonen :-)

[Tomi T Ahonen]

Hi Wayne

First, if you are a bit ill, I hope this blog is relaxing reading for you while recouperating. I hope I am not adding to your aggravation :-) Seriously, I enjoy your comments here and hope you are doing better.

And thanks for being a loyal regular reader. I really love talking with those who've been here and see how whatever my latest blog is, compared to past writing.

On Trump and your latest comments, I almost totally agree with it all - and am sure you also know that. You make some excellent points that I'll want to address

Trump started on a lark (and wasn't expecting really to succeed, maybe). Totally agree. His was the least prepared launch (he paid for some of the audience to attend at Trump Towers). And he obviously has a skeleton staff and his campaign is like those of most who seem to be in it for the ego trip (like Newt Gingrich and Hermain Cain last time, or say Ross Perot before) his 'campaign manager' has a hopeless task of trying to get any level of sense to the project. He is a runaway train haha. Ego size of Hudson Bay (good one) and hates to lose. Totally agree yes.

So why risk his reputation and brand and wealth. I agree totally, there has to be some nefarious masterplan. Not necessarily that the plan 'would' work, but one that Trump honestly believes can work. Like you said "obviously he thinks he has a way of pulling this off". I totally agree and like you, am baffled on WHAT it is and can't wait to find out.

(Loved the joke about dead voters and Chicago). But yes. History books are full of people who saw an opportunity and then made their name with it. Bill Gates didn't invent the computer but he managed to win the contract for IBM to build the software for their Personal Computer project - and was smart enough to get the licence to also sell the clone to IBM's 'PC DOS' as his Microsoft 'MS DOS' and create the modern PC industry and made the most profits in it for three decades. Trump can very well have his 'deal of the century' somewhere in this cockamamie idea to run for President and he must believe - as he still continues running and insulting even more people - that he can somehow win out of this. And that he can RESTORE his reputation and be thought of as the 'ultimate winner' somehow in the 'back end' of this project. I can't imagine that Trump would be so foolish to think he can actually win the general election against Hillary - while he of course has to project that impression to his voters.

On the data on the voters (consumers, customers). Totally agree again. As I wrote, yes, if you allow bad data in, then the whole process becomes vulnerable or even useless. And there will be plenty of cases where the data is not the absolute arbitrer of what to do, as long as humans are involved - it then needs a smart campaign manager (elections) or account manager (sales) to make that call - when do we go, kind of 'against' the data. But yeah, we both know from experience how powerful such info on customers can be..

Now on the 'can change mind at the last second' aspect. I am sure all humans are like that, to some degree. Say the World Trade Center attack of Sept 11, 2001. The whole nation was shocked, and if that was an election year - then all other matters would have vanished and the only thing all voters would consider was, which of these candidates will prevent terrorists from flying jet airliners to the Bank America Tower of San Francisco next, or the Space Needle of Seattle or the Arch in St Louis, etc. And then consider the San Bernardino attack. That was not an orchestrated mass terrorism event like Paris. It was a radicalized local couple working as a 'lone wolf' attack while of course as a bonnie-and-clyde couple. Now. What is on the minds of ISIS as they face ever more fierce attacks by the coalition and the endless bombing led by the 'evil Satan' of America. ISIS is highly organized. I am CERTAIN there is a plan to attack the USA. But in the USA its far easier to buy military-grade weapons than in Europe. An organized attack of say 3 or 4 teams similar to Paris, but say attacking New York, Washington DC, Chicago and San Francisco simultaneously would totally paralyze the USA and the 2016 election would only be of one thing and one thing only.

But it need not be terrorism. What about the power grid. You probably remember about ten years ago when all of the USA East Coast lost power for several days, from New York to Chicago (haha traced if I recall to some power station failure in Canada, that then cascaded). So what if that kind of power grid failure happens out of some maintenance issue with older equipment, or human error, or cyber attack or earthquake or whatever reason. But if a quarter of Americans are suddenly without electricity - rotting food, etc - for nearly a week. And this happened in say October. Then nobody cares about Putin or Iran or ISIS. Then the debate is suddenly about - what did you say about CUTTING funds for infrastructure. And no, what did YOUR budget say about funds to all energy sources... etc etc etc.

I totally agree, some things happen in the 'real world' outside of the political race. That hurricane that hit New York a week before the 2012 election and Governor Christie famously hugged Obama. That kind of things do happen. They call it the 'October surprise' (like the economy 'cratering' in 2008 that doomed McCain's flailing campaign).

Here too, back to my themes of Big Data and total voter insights - a campaign can run on the gut feeling of the candidate and top team (with risks that includes) or have a Big Data based voter insights system that can help pick the optimal political strategy to deal with that crisis. McCain took the bold and decisive step to suspend his campaign in 2008 in response to the economic crisis. It was a very 'McCainian' reaction - but it was widely ridiculed and turned out a massive campaign error. If the candidate is disciplined to trust and use the data - then this kind of mistakes are avoided. That however, does mean that the candidate can seem aloof and disconnected to the nation (like Obama always seems and what Hillary is also often exhibiting). This Republican reaction to be 'anti Obama' then does find traction - like that its somehow a bad thing to have teleprompters and have a prepared speech. That speaking in 'stream of consciousness' style like Trump would be 'better'. It seems more authentic yes, but it also means, Trump's honest personal opinions now slip through regularly. There is a reason why USA Presidents have speech-writers - explicitly because the President has so much power, a few choice wrong words can be very harmful. Like W Bush with you're either with us or without us. Rhetoric like that even W Bush has admitted was harmful not helpful. And that happens even when you have speechwriters haha.

Now on the idea of a wedge issue for Trump. Yes, that could be the ploy. But it would have to be a law that would have instantly very broad populist support. And then also, be one where Hillary and the Democrats have not already taken that idea. The 'lets abolish the IRS' idea is one that cooky Ron Paul had run on for decades. I think most normal politicians didn't dare to make that proposition into a general election but this cycle several Republicans are suggesting it. I think if Trump has the platform to speak so broadly in all media, that could be something that might work. Everybody hates the IRS and having to file taxes. Trump could say its a flat national tax of say 15% for everybody and no (or almost no) deductions and maybe a national sales tax, but no IRS and no more filing tax returns. Many would like this and many would see their taxes come down. But this argument would be massively attacked as giving even more tax cuts to the richest..

As to what else could it be, yeah, there could be something but I can't imagine one, that the Democrats have not been pushing already. The last truly hated law was military conscription during the Vietnam war, that I can think of, which could have/did have the kind of national mobilization power to potentially win an election. Like I wrote in the Trump scenarios blog, raising the national speed limit would be popular but I can't see him winning on something like that - and many would attack him for the deaths it would create.

I do totally agree with you Wayne, that Trump is that smart and calculating, that when he entered in the summer, he HAD to have a plan. And that plan must be either to win it, or then strategically to get out of the race. He cannot be seen to end his public career as a big loser like a Mitt Romney (or Walter Mondale haha). But considering how he's pursued the primary race, it is baffling, what could be that 'golden bullet' idea for the general election, to make this worth his while.

Now as to dropping out after the convention, haha, yeah. Gosh he'd create an unprecedented riot in the Republican party. Many Republican-controlled states would rush emergency legislation to change the rules to allow late entry to their ballot, to bring the 'rescue' Republican replacement nominee onto the ballot. Gosh, it sounds like the rejected script to West Wing. But if he was the nominee, he could do that. Or a variation - what if he, after securing the nomination and the time to get on state ballots has passed, Trump just holds a press conference and ENDORSES Hillary. Say after the first debate, haha. He'd just say - ok, I was beaten, Hillary is more qualified to be President. I can see that now. So yes, I remain the nominee, but please vote for her instead of me.... :-)

..that then COULD get a ground-swell to the Democrats' 'spontaneous' idea to lets name the new state of Columbia to be Trumpia instead. The man who gave up the run. The man who was not partisan... haha. And while the Republicans would scream conspiracy!, this would play to some in the middle and many Democrats as the redeeming quality, that would save his reputation haha. He has his regrets, he apologizes for the tone of the campaign and formally endorses his rival. I believe this has never happened in the Presidential race so he'd at least get into the history books on that too haha.

On the hypothetical Hillary - Trump 'deal'. Yeah agree, it can't be as blatant and direct from Hillary or the Government to Trump even in the style of Dick Cheney and Halliburton. But a land deal, yeah, there are sadly a lot of those. The John McCain example in Arizona is a sad example of that kind of quid-pro-quo corruption which should be blatantly obvious but doesn't seem to be prosecuted ever. The only thing here is, that again, if Trump is risking potentially his brand in this mess now (worth by his public documents, several Billion dollars) then his reward would need to be in that scale, potentially at least tens of Billions of dollars in value, and could be much more.

But.. if Bill Clinton and Donald Trump had years to plot this, and Trump had months before he even announced to make any necessary investments, and the measured public-info gains to Trump might not come until his CONSTRUCTION on whatever those lands are, is completed - another decade - this could be after Hillary has retired from her second term. This deal could be very well obfuscated - but then again... Trump the egomaniac will not want to be thought of as a loser for the next decade. No, if there is a deal, he will want the ability to tell the world how smart he was, that this run for President was just a ruse or whatever.

Have you seen the movie Wag the Dog (Dustin Hoffman, Robert DeNiro)? Brilliant political satire, I consider it one of the ten best movies ever made. In it, the Hoffman character stages a pretend-war, only shot in the studio to distract the nation for a few days just before the election to ensure the incumbent is re-elected (we never even see the President in the movie). When they succeed, Hoffman's character then says - he wants to celebrate this, huge 'Hollywood Producer' achievement of stealing the election - and DeNiro says you can never talk about this... Trump would be not unlike that Hoffman character only more egotistical haha. And that would have to be part of this plot. He is actively smearing his own reputation in public, daily. Why is he willing to do this. He must have a way out, that somehow 'rescues' his name, reputation and fortune. And obviously gives him a massive gain out of this gambit.

But yeah, can't wait for the next eleven months to reveal how this plays. Just after the Romney and the Romnettes season of 2012, we thought it can never get more wild than that Clown Car. Now we have the Trump and the Trumpettes season. Sadly, however this plays out, this massive mystery can never get a rerun haha.

Tomi Ahonen :-)

[Winter]

@Tomi
"And that plan must be either to win it, or then strategically to get out of the race. He cannot be seen to end his public career as a big loser like a Mitt Romney (or Walter Mondale haha)."

A well simulated (medical) emergency after Trump has accepted the nomination and installed a suitable VP would do the job.

Say a stroke or heart condition that prevents Trump from running the campaign or accepting the presidency. Maybe there is some loophole in that crazy medical attest?

His VP (and the GOP) would be eternally gratefull. His supporters are not even loved by the politicians who represent them, so I do not expect any considerations for their feelings.

[Trent Telenko]

I have a very simple question.

What is your basis for saying the following --

"And that brings us to Trump. He obviously doesn't have a big team or bother to run anything through any decision-making system at all. Would not be 'politically correct' and might interfere with his desire for spontaneous schlonging. It does mean that over time, the team(s) who use Big Data and pre-test their responses and actions, will be prone to not make mistakes but Trump can capitalize faster on some fast-moving matters. But we've already seen that Trump is a guaranteed gaffe-machine and once the political system figures out exactly how to pin him to all his lies and nonsense, his campaign will crash and burn to the degree its 76% lies - most of any politician ever measured by PolitiFact - warrant. The campaigns who can afford to build a Big Data based solution are viable for the general election and long run."

Campaign spending reports mean nothing as far as the Trump campaign is concerned.

We have no idea how many people are on the Trump campaign team because he pre-paid them last year, prior to the current (2015-2016) campaign finance reporting seasons.

Similarly, financial dispersments for pre-paid services does not count as a "campaign expense" in the current reporting period under current campaign finance laws.

Thus we have no idea how much the Trump campaign has spent on "Big Data."

As far as Trump's supposed lack of "Big Data", there is a great deal of circumstantial evidence Trump has a big data campaign.

Thus far I have seen Trump "name" multiple opposing Republican candidates in a way similar to fighter pilots getting a permanent call sign, but uniformly in a manner that destroys their political reputation.

IOW, a precision guided, information warfare, negative campaigning attack.

See:

"Snarly Carly" Fiorina
"Low Energy" Jeb Bush
Dr. Carson and Trump's relabeling of Carson's "knife attack narrative"

The "naming effect" Trump has used seems to be a combination of the kind of "Big data" you speak of combined with media "Q" ratings that a reality TV star like Trump would be very aware of.

[SDS]

You are, of course, free to write about whatever you want. However, I, for one, do not care very much about your musings on American politics. Please start writing about mobile telephony again.

[Tomi T Ahonen]

To all in this thread

I have now made my projection of the actual delegates won by the Republican race through the first four states voting in February. You may enjoy the analysis and various research that I referenced in it.

Tomi Ahonen :-)

[Wayne Borean]

Tomi,

Unfortunately, it's chronic. I get to live the rest of my life eating opiates in amounts that would have 99.99% of the population lying in the ground going 'WHEE'

[Winter]

@Wayne
"Unfortunately, it's chronic. I get to live the rest of my life eating opiates in amounts that would have 99.99% of the population lying in the ground going 'WHEE'"

I am sorry to hear that. I wish you a happy new year. And strength and, if possible, hope for improvement. Amputation will often not help due to phantom pains.

And wrt the Levenson self report test, I see I am utterly unfit for a job in marketing or sales Here is my score (spelling from original):

You score for primary psychopathy was higher than 5.51% of people who have taken this test.

You score for secondary psychopathy was higher than 0% of people who have taken this test.

[Tomi T Ahonen]

To all in this thread

Three quick bits of news. Politico reports on Trump's data analytics project. One, that he has one and is dumping money in it, and has hired professionals doing it, suggests Trump is really serious about the run. He's also started to air TV ads now for the first time (previous tiny ad buy was radio ads). To me what was in the Politico article, its classic demographics voter lists and targeting, not 'big data' as shown in this article and what Jeb Bush was building and Ted Cruz seems to be building (and Hillary has). But it could be the beginning of a real competitive Big Data project (in which case it will cost at least $50M this cycle, at least, to be in any way relevant) and also, its possible it starts small with old-fashioned demographics and voter lists and robocalling and spam, and then EVOLVED with time and effort and insight (possibly as some candidates quit and their competence is then pulled into the leading teams haha).

Second interesting tidbit, the UK Parliament will debate of they should ban Trump. Gosh I hope they do decide that haha... for the hate-mongering and terrorist-arousing vicious shit pouring out of Trump's mouth, I do hope he gets this slap on his wrist.. go Britain!

And third, very interesting.. California has just been polled for GOP preference by Field, and who is on top? Its not Trump, its not Rubio, its not home-girl Fiorina... its Texas-boy Cruz! Wow, only about the second state in about a month where Trump is not in the lead, but California has the most delegates and this could be a sign that Trump is about peaking and Cruz is rising.. Could Trump finally be hitting his ceiling at around 35% nationally? And can Cruz climb to challenge him.. the winner-take-all states start only on March 15, so that is when the gloves come off. Cruz does seem once again, to have a real chance while Rubio - once again! Rubio the more 'moderate' candidate can't poll higher than Cruz in California, what gives? But note the gap between Cruz & Trump is only 2 points for Cruz in CA so its still a wide-open race and thats only one poll, while it is the largest state. (there was one national poll also out by SurveyMonkey which RCP doesn't recognize for its flawed research methodology, but in it Trump leads with 35% vs Cruz at 18%)

Tomi Ahonen :-)