I want to write this mostly for my readers. We have a still-lively discussion going on in my 'last' (aka 'final') election-related blog. I don't want to have those readers constantly have to scroll through over 20 pages of old comments to see the latest comments added. So lets do an update for that story so those who are still discussing the US election. So they can have an easier place to meet up and chat about it, after all, I did start up that discussion topic for readers of this blog.
So lets do quick election follow-up. We have now almost all the counting done. The final turnout was about 135 million votes (turnout ended up being actually an increase of about 3% from 2012). Hillary Clinton did win the popular vote as most know, she got 2% more of the vote than Trump but her voters were concentrated in many states that were reliably Democratic like California, so because of the 'Electoral College' system it is Trump who wins the Presidency. And nobody is disuputing this aspect of the US system, although some are suggesting it may be time to move onto a popular vote, its the 5th time in history that the person who got more popular votes did not become the US President (the last time it happened was Al Gore vs George W Bush in year 2000).
When the consensus polling said its a 4% race with Hillary ahead, and all but one of the polls in the final weekend said Hillary Clinton would win the popular vote and she ended with 2% more popular votes, then the polling was actually fairly close. (What I made, myself, was a colossal mistake in assuming she'd then be able to turn that slight gain into a far bigger landslide win; but as I already wrote, I got it wrong and I thought there would be a big women's vote 'wave' that would propel a landslide election for her. Obviously instead of more women showing up than in normal elections, Hillary actually achieved a REDUCTION in the ratio of women vs men voting; so the wave went the opposite way to how I expected).
A few interesting observations. Trump ends up with 46% of the total vote (Romney lost 2012 with 47% of the vote) but Trump got more votes than Romney (because of a higher turnout). Trump can very fairly be said to have energized some voters, a few million in the final count, that did not come out for Romney but were energized to vote for Trump. He deserves credit for that (even if I personally don't agree with his political methods, they clearly worked for him). Note also that Trump has achieved the highest vote total of any US candidate for President by the Republican party in its history, in absolute terms. Trump got more votes than any Republican in the past, more than Mitt Romney, John McCain, George W Bush (Bush 2) in either of his elections; or older Republicans like Daddy Bush, and yes even more votes than Ronald Reagan or Richard Nixon who both won in landslides. it Is because the overall electorate has grown. Obviously Trump has not been anywhere near the record in Republican voter percentage as even Romney got a better percent of total votes four years ago. But seriously, congratulations to Trump for energizing some new voters to come out and vote Republican and obviously the appeal to populism has merit, it is also how Bill Clinton won against the traditional Democratic party orthodoxy of the era of Jimmy Carter, Walter Mondale and Michael Dukakis who lost 3 elections in a row until Bill Clinton made the Democrats a more populist party.
Secondly Hillary, she got to 48% of the vote, well below what Obama got in 2012 as a percentage but with the surge in voting nationwide, her vote matched Obama got in 2012 at 65.5 million votes. In a cruel twist of fate, Hillary Clinton for the second time in running for President, managed to get more actual votes than her rival, and lose the race. This was the same matter that happened in 2008 when she ran against Obama in the primaries and got more votes by Democratic party voters but Obama had more 'Super Delegates' and won because that was the rules of the game. Now this time she won more votes but the Electoral College system gave the Presidency to her rival instead. Note also, that Hillary's vote total is tied for second-highest recorded in US election history, no Republican has ever gotten to 65.5 million votes, and no Democrat other than Obama once has exceeded that in 2008.
Now on WHY. From a mathematical point of view, Hillary lost many states that were seen to be 'safe' Democratic states, like Michigan, Wisconsin and Pennsylvania, by razor-thin margins. She did not campaign at all in Michigan and Wisconsin, and while she did campaign well in Pennsylvania, she did then spend plenty of time trying to win states that were 'red' like North Carolina and Arizona, states she ended up 'winning the battles but losing the war'. She won more votes but lost the Presidency. In a way you could say, her campaign got too greedy, they tried to win by a huge landslide in many states and with that, did not give the necessary attention to the states where she was falling behind. (Note, there are also recounts going on in several states, we may see some change in the final results but I am not holding my breath...)
From the strategy point of view, while Trump himself (and even his TV ad spending) seemed to signal a total hodgepodge mess of no focus area at all, his ground game (run by the Republican party, quite independent of Trump's campaign itself) seemed to be well focused and highly effective, in those states that mattered. What should concern any Democratic strategist is, that the Hillary Clinton campaign did not seem to be aware of the pending doom arising out of Wisconsin and Michigan. Her campaign did see that Pennsylvania was tight and kept fighting for i (and most polls showed her mostly ahead in that state) but the WI and MI state results, those should give great concern for the internal measurements that the Democrats failed in, and clearly the Republicans in turn, succeeded in their focus.
If a race is that close, 2% election, then yes a few strange in-state results can swing an election. Even so, because Hillary won the popular vote, she was in the driver's seat and a 0.5% election could well be forgiven to be 'lost by the winner' but when you get 2% more of the vote, then your campaign really should be able to convert that into a national win. That said, both of the recent examples where the winner of the popular vote did not become President, do suggest there may be a systematic bias in favor of the Republican. These two elections seem to show that the gain seems currently to be going to the Republicans in such cases (W Bush vs Gore in year 2000) and this is of course as the EC vote is designed to give some voting imbalance to small states, so a few large states cannot dominate them. And most of the states small by population, are in the middle of the USA, where Republicans do better, and most of the population lives on the edges of the USA where the Democrats do better. It may well be, that the 'natural EC balance' means that a Democrat does have to win by a few points, just to get the Presidency, and the Republican can 'afford to lose' the popular vote by a point or two - but not by five (like Romney vs Obama in 2012) - and still become the President. (Also obviously for ANYONE in numbers, we cannot generalize from only two data points, but these do signal a possible systematic imbalance, and future elections will tell if that is so).
What about the ground game and big data? I have given this part some thought and my working theory is this. I think we have reached essentially parity. And it has some 'obvious' conclusions which now to me seem blatant but I didn't see this before the eleciton. I am sorry I didn't think of this before but to me, this makes the most sense in light of what we've seen happen. But follow me with this logic and see if this makes sense. So I believe we have achieved for practical purposes essentially near-parity. First there was a 'secret weapon' deployed in 2012 but only by one side. So I think a lopsided data-driven modern digital outreach political campaign, driven by targeted big data system, can deliver about 4% beyond what the polling suggests. It was done nationally for the first time in 2012, by the Obama campaign. They ended up outperforming their polling by 4%.
Now for 2016 the Republicans have deployed a rival system. It wasn't as big and as sophisticated as the Democrat machine which itself was fine-tuned and upgraded. But that means, the Republicans would kind of catch-up. Not totally, because they are not at total parity with the Democrats, but they ate significantly into the Democrats' lead. If you look at the aftermath, the Republicans were excited about their data and ground game operations, while the Democrats were downcast. The Democrats had already built a system in 2012 that did about as good as you can get. They could not squeeze more out of it this time. And with their machine running full power, but with an unpopular Hillary at the top of the ticket, they squeezed exactly the same number of voters as what Obama got.
Meanwhile the enthusiasm on the Republican side is similar to what the Democrats felt in 2012. They tried a new system and it worked very well and the ground game delivered several tight races by very slim margins, to the Republicans. It did not give them 4% above the polling what Obama got in 2012, but it did give them half that, 2%. This partly because the Democrats ran a stronger rival system against them; and partly because even in their full effort, the 2016 Republican ground game did not match the size of what Obama built in 2012 (and Hillary now had expanded upon). If the maximum gain of such a system is 4%, and Democrats got that in 2012, they now only matched it, meaning no new voters brought in anymore by using this method on the Democrat side, it was 'baked in'. But the Republicans had not deployed this method before, they did now. They didn't do it as thoroughly and systematically as Obama in 2012 (Trump refusing to play along fully with the ground game data driven inputs and his percentage of Republican vote ended up down vs Romney) but the system DID work and drive only on the Republican side, voter turnout up and thus gave Trump about have the gain he could have gotten in a perfect world, ie 2%.
We will need to wait to see more data about their efforts to measure it, but I am thinking, it means no more gains to Democrats and in the future, slight more gains possible for Republicans if they do it 'properly' the next time. But this is now a new 'normal' both sides will have large database-driven voter outreach programs that will be used to drive up voting turnout.
A minor observation to add. I think what the Democrats seem to have achieved this time is that their strong machine now gave them a false sense of security. Because the system works, and was now OPTIMIZED vs 2012, it did not yield any more voters than before, but it did push Democratic voters to show up early, into early voting. So the early voter data looked particularly promising (and misleading in a very cruel sense) to the Democrats, where they thought this early voter enthusiasm would ALSO then yield more 'fresh voters' for election day, who did not materialize. What the Republicans did particularly well was the 'primary purpose' of such a system, to drive up voting DAY voting in person. And there, the surprise to the Democrats in 2016, is as harsh and hard, as it was to the Romney campaign in 2012. The longer the voting day dragged on, and the results came in, the more the results started to favor the Republicans to a clearly decisive impact.
In 2020 its likely that the Republicans will refine their efforts and use their new powerful tools to also drive early voting. The Democrats will not, cannot abandon these systems and go back to the old way, or they would surrender 4% of the vote instantly. Its likely that in 2020 the Republicans can still outperform polling by about 2% based on the data driven voter turnout processes, which means that if there is an in-built Electoral College advantage worth about 2% and there is a gain to come also worth about 2%, then whoever runs against Trump in 2020, would need to build more than a 4% lead in the polling, just to have a chance to be even...
That is my working theory for now. Would square with what we witnessed and would also give some insights of what might come in the future. With that, lets move the political comments of the election follow-up here to this posting.