Passive investing can be traced to a series of individuals and institutions in the 1970s, including academics Harry Markowitz, John Lintner, Jan Mossin, Paul Samuelson, Bill Sharpe, Jim Tobin, and Jack Treynor; practitioners Jack Bogle, Bill Fouse, and John McQuown; and Wells Fargo and Vanguard. There has been a growing tension between passive and active managers ever since. This tension has inevitably led to the emergence of two distinct cultures in investment management, one focused on the identification of mispricing and the other predicated on the inability to do so.

The definition of an index must be revised to reflect the impact of technological advances such as automated trading, ECNs, dark pools, flash orders, and the FIX protocol. In this new world order, and index must be: 1) transparent, 2) investable, 3) passive. While static market cap weighted benchmark certainly satisfies these criteria, thanks to the new technologies described above, they are no longer the only benchmarks that do so. For example, portfolios that consist of slowly varying proportions of stock index funds and bond funds --- with weights designed to tilt the portfolios from equites towards bonds as specific “target dates” draw closer --- are not necessarily static or market cap weighted, but they do satisfy our three new criteria for an index, and such target-date funds are considered passive vehicles suitable for inclusion in most retirement plans.

More importantly, the primary objective of any index is to facilitate the extraction and summary of information through an algorithmic process, for example, averaging. Now typically done by computers, averaging was once a technological breakthrough in the financial industry, and one that was theoretically motivated.an important factor in the early popularity of the Dow Jones Industrial Averages, in which specific economic meaning was attributed to certain time series patterns in the index.

As with most dichotomies, reality is not nearly as simple. Even as passive investing has grown exponentially since Bogle launched the Vanguard Group in 1975, a host of differentiated products has attempted to strike a balance between passive and active: hedge-fund beta replication, smart beta, fundamental indexes, risk parity, minimum-variance funds, and so on. The frequency with which this alternative to purely passive investment vehicles emerge suggests the existence of some primal urge driving investors to seek out inefficiencies regardless of the time or place in which they live.

Technological advances in telecommunications, trading automation, data storage and processing, and computation have permanently changed the landscape of the financial industry. The emergence of powerful new tools such as machine learning and artificial intelligence offers some exciting new possibilities for creating highly personalized and dynamic portfolios --- algorithms that are capable of adapting not only to changing economic conditions, but also to changing personal circumstances and tastes. The vision of “precision indexes” that dynamically adjust over time – that adapt to an investor’s specific needs more effectively than a target-date fund – is likely to become reality.

The next frontier for both passive and active portfolio managers is to develop a deeper understanding of “investor biology”. Why do we do the things we do, especially those things that undermine wealth accumulation? Investment managers and financial advisors are often quick to criticize retail investors for chasing performance and not focusing on the long term, but human nature is simply not wired to buy and hold when we are facing mounting losses in our nest egg. In fact, today’s passive investment products are the equivalent of a pile of auto parts from which we ask drivers to put together the right combination of engine, carburetors, chassis, dashboard and so on, to meet their driving needs. What’s needed instead are fully assembled vehicles that offer the kind of protection against human behavior that even the least expensive car offers today’s drivers: air bags, collapsible steering columns, reinforced bumpers, three-point seat belts, safety glass and so on.

While self-driving portfolios may seem like science fiction right now, the same was said in 1970s of passive buy and hold portfolios that could beat most active managers. One day, passive investment products will make retirement planning as straightforward as going to the dentist, but a lot less painful and even less expensive, though they come at a cost of potential pitfalls if abused.