The Evolution of the CPI

Mon, Mar 3, 2014 - 8:51am

The Consumer Price Index (CPI) is perhaps the most widely watched and used measure of inflation in the US. It's calculated by the Bureau of Labor Statistics (BLS) and according to them represents, "a measure of the average change over time in the prices paid by urban consumers for a market basket of consumer goods and services."

The CPI is a benchmark calculation. It is used to determine whether an economy is experiencing inflation, deflation or stagflation. It drives the cost-of-living increases for many programs such as Social Security, and factors into the federal income tax structure. It's used by employers to make wage adjustments and many economic data series are adjusted by the CPI to translate the related indexes into real (inflation-adjusted) terms. The CPI may be of special importance to retirees, whose pensions and other forms of income are often tied to changes in the CPI.

From the perspective of the government, to whom liabilities such as Social Security and interest payments on TIPS accrue, it can be very advantageous to have the CPI as low as possible. The CPI is also used in the calculation of real Gross Domestic Product (GDP). Again, a lower CPI is generally desirable from a government perspective, as it makes real GDP appear larger.

We're going to explore how the calculation of this important metric has changed throughout the last few decades, and whether those changes were warranted, or likely the result of intended manipulation. For the sake of time and space, we're going to discuss this topic at an aggregate level, broken down into sections.

In its original manifestation, the CPI measured changes in the price level of a fixed basket of goods and services. This rendered the CPI effectively a cost-of-goods index (COGI). It strictly measured the changes in price levels of specific (non-changing) goods and services.

Over the last few decades Congress has embraced an ideology that the CPI should reflect changes, not in the cost of a basket of goods and services, but in the cost to maintain a constant standard of living. As a result, the CPI has been continually modified in ways which now equate it to a cost-of-living index (COLI).

One of the most groundbreaking conceptual changes to the CPI calculation, and the first major topic we'll discuss, is the advent of "substitution."

Fixed-Weight to Substitution Based

As previously mentioned, the original CPI calculation was based on a fixed basked of goods and services. According to the BLS, this method overstated inflation. The argument set forth was that changes in the relative prices of goods and services would cause behavioral changes in consumers, as they substituted less-expensive goods for more-expensive goods. If consumers could obtain a similar good that provided the same "utility" (an economic term for level of satisfaction), then price levels hadn't really increased as far as the consumer could tell.

Many of you are probably shaking your head already. You might be thinking, "Who deems, and how would they calculate, how much satisfaction consumers receive? And, there is nearly always something about a less expensive product that results in lower satisfaction. If there weren't, no one would buy the more expensive one!" Now, to be fair, I will say that if a cheaper item comes onto the market that truly does provide the same satisfaction, that could result in an overstated inflation number, since consumers would not purchase the more expensive item, but the cost of the more expensive item would be included in the CPI. That being the case, I would argue that this is the exception rather than the rule.

Let's use an example to highlight how substitution plays out. The example we'll use is similar to one put forth by The Boskin Commission Report (1996), in its argument as to why changing the CPI was necessary. See if you agree with the logic. Let's look at two simple goods that most consumers purchase on a regular basis, steak and chicken. The table below shows example starting prices of these two goods as well as their prices after incurring a 10% price increase.

The original fixed-weight CPI calculation would have arrived at a CPI (inflation) value of 10% for the above category of price changes. This makes sense, both prices went up by 10%.

Now let's look at how the new substitution-based CPI would be calculated. Here is the logic: If the price of steak increases, some consumers will substitute chicken in place of steak. The more consumers do this, the less effect the price increase has on consumers' expenditures. If 100% of consumers who would have purchased steak bought chicken instead, then the currently calculated CPI would provide a reading of 0% inflation for the goods in our example. Consumers were paying .40 for their "meat" before, and now they're still paying .40 for their "meat." Now if only half of consumers switched their preference, then the CPI would read 5%. This is because half the consumers experienced a 10% price increase, while half experienced a 0% increase. The average increase across all consumers would be 5%.

Scratching your head? Me too. First off, I personally do not derive the same satisfaction from chicken as I do steak. Some might, but I don't. I get excited about going to a steakhouse for dinner, I've never seen a chicken-house. Second, and this is the nutritionist side of me speaking, steak and chicken do not provide the same nutrients. The amounts and types of fats are different, the amino acid profiles of the protein are different, and the overall caloric (energy) levels are different. Sure they're both "meat," but if I am forced to eat all chicken instead of steak, I would feel that my utility had decreased, even if my cost had remained the same.

Through statistical sampling, the BLS attempts to measure what percentage of consumers "substituted" products, and account for it in the CPI calculation. This method nearly always winds up with the CPI being lower than it would have been otherwise.

Next we'll talk about changes to the calculation known as hedonics. This nebulous process does have some justification, but it opens up a world of potential manipulation.


One fundamental problem in the calculation of the CPI is that product characteristics, not just their prices, change over time, as vendors introduce new versions and features to existing products. With many categories of items, when the characteristics change, the price changes. This adds a level of difficulty in identifying which component of the price increase was due to inflation, and which was due to increases/decreases in the product's benefit to consumers.

The traditional method of calculating CPI would be to temporarily remove an item from the sample when its quality had changed. However, this caused biases and difficulties, since many products (think technology) are always in a state of change. A new method was needed to identify what portion of an item's price change was due to inflation, and what portion was due to changes in the benefit the product provided. Enter Hedonics.

Hedonics (or Hedonic quality adjustment) is a technique that is employed in the current CPI calculation across nearly all items which have changing product qualities. It is a method of adjusting prices whenever the characteristics of a product change due to innovation, or when a completely new product is introduced. In practice this approach entails decomposing an item into its constituent characteristics, estimating the value of the utility derived from each characteristic, and then using those value estimates to adjust prices. When all price increases from "benefit" changes have been accounted for, the remaining change in price can be attributed to inflation. For the curious math types, this is not necessarily as arbitrary a process as it may seem. The process relies on basic regression analysis, a statistical technique employed at all levels of business and across all industries in today's modern world of analytics.

Unlike the substitution concept described above, I believe that hedonics does have a place within inflation calculations. The unfortunate side effect is that incorporation of hedonics provides a brilliant opportunity for manipulation. But who would ever want to manipulate the CPI?

Example: Let's use TV's since we're all familiar with them.

If you bought a TV 20 years ago, it was probably what's called a CRT television. They were large, heavy, and the picture quality was so poor that even Bob Barker looked like he had a great complexion. Comparing the prices of TV's today to that of TV's 20 years ago, and saying that inflation was responsible for the price increase, would be ridiculous. We're all well aware of the benefits a modern TV provides, which older models did not. Through the process of Hedonics, the additional benefits of a new TV are estimated and assigned a value. Once the value of all benefit improvements has been accounted for, the resulting change in price is attributed to inflation.

The table below shows how the BLS looks at televisions. Don't worry about understanding the headings and values in this chart; if you're not a stats person they won't make much sense. The reason I show you the chart is to demonstrate that at least "in theory," there is a mathematical approach to estimating the value of quality enhancements. How accurate these methods are depends on the proficiency and integrity of the analysts running the numbers. In some cases I'm sure they're relatively accurate, and in some cases a blindfolded monkey could probably do better throwing darts at a wall.

The only thing to take away from the above chart is that various benefits, such as having a universal remote, are assigned value. The resulting change in the overall price of an item that cannot be accounted for by its added benefits is deemed to be caused by inflation and included in the CPI.

Now, comparing the changes in TV prices over the last 20 years without some type of hedonic adjustment would be irrational and a waste of time. A modern TV is in most cases a completely different product than a TV was 20 years ago, so some form of hedonics is necessary. But hedonics is a double edged sword, and that's primarily the result of one feature: hedonic adjustments are subjective. None of us can assign an exact value to the benefit of having a universal remote. Someone with a disability may consider it lifesaver, while someone with kids who always lose the darn thing could find it more of a nuisance than a benefit. Thus the hedonic adjustments are open to interpretation, and consequently open to manipulation. If a lower CPI is desirable, the hedonic adjustments can be raised to account for the bulk of the price increase, resulting in low inflation.


The CPI now incorporates a methodology which introduces guesswork and less-than-fair assumptions (cheaper, substitutable products have the same utility) into the calculation. These changes have introduced a means through which governmental influence, or outright manipulation, can manifest itself. Unfortunately there is not much one can do about this, aside from whine and complain. The CPI is such an integral component of our modern financial system that it's unlikely a new proxy for inflation will gain widespread adoption. Perhaps the best thing we can do is simply note that our real world experience of inflation is likely to be very different from what is reported in the CPI, and to use this information to influence our financial decisions moving forward.

The above content was an excerpt of Richard Russell's Dow Theory Letters. To receive their daily updates and research, click here to subscribe.

About the Author

Chief Investment Strategist
matt [at] modelinvesting [dot] com ()