Register For Our Mailing List

Register to receive our free weekly newsletter including editorials.

Home / 585

The abacus, big data and a brief history of indexing

When Charles Dow, a journalist, developed the Dow Jones Industrial Index in 1896, the abacus was still widely used to make calculations. Arithometers, which were invented by Charles Xavier Thomas de Colmar in 1820 and in commercial production by 1851, were also a common sight.


Source: arithmometre.org

Commercial production of the arithmometer ended around 1915 and abacuses are largely confined to primary schools these days. Meanwhile, the Dow Jones Index is still widely quoted. Let's start by looking at how it is constructed. 

The Dow Jones is a (share) price-weighted index, which means that the stocks with the highest price have the largest weighting. This would have been the easiest way to create an index with the calculation tools available at the time.

Currently, UnitedHealth Group, which has a price of US$585 per share, has the highest weighting in the portfolio at 9.6%. By contrast, Apple is roughly six times larger than UnitedHealth in terms of market cap yet only makes up 3.5% of the Dow Jones.

The Dow Jones provides a general barometer of US equity performance. But it does not make any sense from an investment perspective because a share price could be a function of having fewer (or more) shares on issue. A share price, by itself, is not useful for making investment decisions.

After World War I and the next type of indices

At the turn of the 20th century, new mechanical calculators were usurping the arithmometer as the calculator technology of choice. The comptometer, the Burroughs adding machine and Odhner’s arithmometer became the calculators of choice.

Indices evolved as they were able to embody more complex calculations. The next major innovation was ‘market capitalisation’, which was pioneered by Henry Varnum Poor and the Standard Statistics Co.

The result was the 1926 predecessor of the United States’ S&P 500. A market capitalisation index uses the size of a company for inclusion. Therefore, in a market capitalisation index, the larger companies have bigger weights. In the S&P 500, Apple makes up around 6.7% and UnitedHealth around 1.2%.

Market capitalisation indices were considered better barometers of the market, so became the source of data quoted in finance news. Again though, the index was intended to be a market barometer, not a tool for investment.

In the 1950s, research by Harry Markowitz and William Sharpe supported market capitalisation indices as an investment tool. This is the Theory of Efficient Markets. Based on this theory, market capitalisation-weighted indices deliver the best returns for the least risk. It was thought that you cannot outperform the market unless you take on additional risk.

But there are numerous examples where the market has been wrong. There have been periods of irrational buying and selling and there have been periods during which bubbles have formed.

Consider too, the differing needs of individual investors and institutions. Each has a unique reason for buying and selling shares and could assign a different value to different aspects of the financial transaction which is often unrelated to the valuation. For example, investors sometimes trade for tax, income or even emotional reasons. As a result of these factors, the market is not efficient in reality.

By the 1970s, professional fund managers were aiming to exploit these inefficiencies, targeting returns above market capitalisation indices.

Computers, ‘big data’ and the next wave of indices

By now IBM had created mainframes, and the desktop computer was becoming a reality. The first handheld programable calculator, the HP-65 was released in 1974 at US$795 (Nearly US$5,000 in today’s dollars).

Computers and calculators were getting faster, smaller and cheaper. Savvy active fund managers were able to use some of this technology to their advantage.

An active fund manager proactively makes decisions over which investments are bought or sold, generally with the aim of outperforming a market index. This is done via a mix of qualitative and quantitative research, personal judgement and forecasting, so computing technology and its implementation could be a competitive advantage.

When actively managed funds were first offered to investors, performance was uncertain, and the costs were high. Sometimes the returns were good, but often they were not. Many people found this a poor bargain, so preferred lower-cost passive funds which tracked market capitalisation indices. In these passive funds, returns were not high, not low, just the market average. The rise of ETFs this century has largely been driven by demand for these passive funds.

At this stage, passive funds only tracked market capitalisation indices. However, sophisticated investors in passive funds started to consider the possibility that alternate index weightings, different from market capitalisation, could give investors higher returns for the same, or even lower levels of risk.

Alternate index construction methods started to focus on grouping companies with common valuation characteristics, common balance sheet qualities and common fundamentals to screen or weight stocks, including equal weighting constituents.

These innovative index construction techniques became known as ‘smart beta’. They are ‘smarter’ because they take a more considered approach to what goes into the index, other than just the size of the company.

These index innovations have been driven by a combination of investors seeking investment outcomes beyond benchmarks and the advent of technologies like ‘big data’ that enabled financial professionals to better leverage the data in financial reports, performance data points and mathematical algorithms to target these outcomes.

Unlike market capitalisation indices, these ‘smarter’ indices are created with an investment outcome in mind and were not created initially to represent the performance or health of the share market.

We like to say smart beta combines the best of active and passive investing: having the potential for better investment outcomes while being rules-based, transparent and cost-efficient.

 

Arian Neiron is CEO and Managing Director - Asia Pacific at VanEck, a sponsor of Firstlinks. This is general information only and does not take into account any person’s financial objectives, situation or needs. Investors should do their research and talk to a financial adviser about which products best suit their individual needs and investment objectives.

For more articles and papers from VanEck, click here.

 

 

  •   6 November 2024
  • 3
  •      
  •   

RELATED ARTICLES

There's nothing sleepy about Rip Van Winkle indexing

banner

Most viewed in recent weeks

Are you making these SMSF mistakes?

After four decades advising investors, there are a few common mistakes I've seen SMSF investors make. From holding too much cash and chasing yield to overtrading and trusting tips. Are these errors costing you?

Why have Australian living standards 'fallen' and how do we fix it?

For the last few years there has been much talk of a 'cost-of-living' crisis in Australia and of 'falling living standards'. Lately this has flared up again with the pickup in inflation resulting in a renewed fall in real wages.

Retirement spending is not one-size-fits-all

New data challenges the idea that Australians are underspending their super. The bigger issue may be helping retirees navigate complexity, make confident decisions and use their savings to support security, wellbeing and choice.

Will you run out of money in retirement?

Fear of running out has become a defining retirement anxiety. Why do some retirees die with substantial wealth while others deplete their nest egg? Evidence suggests the answer is more complicated than we think. 

Four options for an income investor’s next dollar

What if Australia’s golden age of dividends is ending? Rather than overhaul your portfolio, it may be worth considering where new capital can work harder. I discuss four income strategies and the trade-offs behind each.

The missing link in the CGT debate

A little-noticed consequence of Labor’s tax changes could have implications well beyond investors’ tax bills. The issue raises bigger questions about incentives, capital allocation and the drivers of long-term economic growth.

Latest Updates

SMSF strategies

Meg on SMSFs: What do we think about reversionary pensions these days?

Reversionary pensions have long been a staple of SMSF estate planning, but are they still the best option? Meg Heffron revisits a once-clear favourite and asks whether changing super rules have shifted the balance.

The ageing ‘crisis’ has not and will not happen

Rising age dependency is frequently treated as a warning sign for economies. But when actual workforce participation is examined, a strikingly different picture emerges about ageing, productivity and economic sustainability.

Retirement

How does the 4% rule stack up?

The 4% rule has long been retirement's gold standard. But after a difficult period for investors, fresh analysis suggests a more conservative approach may significantly improve the chances of making savings last.

Shares

Four charts that expose market concentration risk

Investors have recently been rewarded for backing market leaders, but history suggests this eventually comes at a cost. Now may be the time to review whether your portfolio is carrying unintended risks beneath the surface.

Investment strategies

The case for gearing beyond property

Most Australians gear into property but ignore shares. That may be a mistake. Used carefully, geared equity strategies can enhance long-term returns, reduce cash tied up in growth assets and support retirement income goals.

Economy

Australia's $1 trillion debt pile

The headlines exclaiming that Australian government debt has hit A$1 trillion and US government debt has hit $40 trillion has turned heads, but how serious are they really? Will Australia's mix of debt create challenges?

Economy

Has 100 years of growth made us any happier?

For decades, GDP has been the benchmark for economic success, but has it made us materially happier? If happiness does not rise in lockstep with prosperity, are we overlooking what constitutes a successful society?

Sponsors

Alliances

  • ASA-Logo-RGB-ActiveGreen-web.png

© 2026 Morningstar, Inc. All rights reserved.

Disclaimer
The data, research and opinions provided here are for information purposes; are not an offer to buy or sell a security; and are not warranted to be correct, complete or accurate. Morningstar, its affiliates, and third-party content providers are not responsible for any investment decisions, damages or losses resulting from, or related to, the data and analyses or their use. To the extent any content is general advice, it has been prepared for clients of Morningstar Australasia Pty Ltd (ABN: 95 090 665 544, AFSL: 240892), without reference to your financial objectives, situation or needs. For more information refer to our Financial Services Guide. You should consider the advice in light of these matters and if applicable, the relevant Product Disclosure Statement before making any decision to invest. Past performance does not necessarily indicate a financial product’s future performance. To obtain advice tailored to your situation, contact a professional financial adviser. Articles are current as at date of publication.
This website contains information and opinions provided by third parties. Inclusion of this information does not necessarily represent Morningstar’s positions, strategies or opinions and should not be considered an endorsement by Morningstar.