Category Archives for "Business Intelligence & Analytics"

KPIs and the Hawthorne Effect – Part 2

In part one of this two part series we discussed the impact of the Hawthornekpis and the hawthorne effect2 Effect on KPI’s. (to view Part 1 of this series – clickhere)

So, what do you do from here?

Once a KPI misses its target (turns red) for several periods, engage the Process Improvement experts in your organization. Process Improvement professionals have different code-names in different organizations:

  • Quality Engineers
  • Process Engineers
  • Lean Manufacturing
  • Six Sigma
  • TQM

If these professionals don’t exist in your organization, then here are a few things to keep in mind as you tackle the metric.

Try the “5 Why?” Test to Determine a Possible Root-Cause

This is a quick management test that allows you to get deeper into why a metric is failing and can be done in a staff meeting. The test is simply you asking “Why?” five times to get to a deeper root-cause process that you can actually improve for the long term. A simplistic example is outlined below.

  • Why did we miss our attrition target?
    • Because we had a spike in temporary employees
  • Why did we have a spike in temporary employees?
    • Because we had a short term productivity spike
  • Why wasn’t the short term spike in the forecast?
    • Because the forecast is only updated quarterly
  • Why is the forecast only updated quarterly?
    • Because it is just the way we have always done business
  • Why have we always done business this way?
    • Because we have never experienced the type of seasonal growth that we are seeing with this product.

Although this a simplistic example, the answers move you away from having endless meetings to look at the top line attrition results, and moves you towards and underlying process (forecast updates) that can be modified to give more accurate results. Insist that your analysts go 4-5 levels deep.

Process Engineering

Ask yourself and your team: “How can we modify the process so that it is impossible for the process to fail?”

When is the last time you left the light on in your refrigerator? The process has been engineered so that it is impossible to fail. It doesn’t require weekly meetings.

When is the last time you started your car when it was already in ‘Drive’? The process has been engineered so that it is impossible to fail.

Most websites employ similar mechanisms to ensure it is impossible for you fail:

  • Credit Card numbers that won’t be accepted unless it matches the credit card type (Visa, AMEX, etc.)
  • Billing addresses that must match the Billing address on the credit card.
  • Postal codes that must match the correct State/Region

In Closing

Look for opportunities to alter the underlying process. This is the true reason that you have made your investments in Business Intelligence. There is organization altering power that can be harnessed with the information flowing out of your Business Intelligence environment. But that power will only be consistent over the long term when you begin to see KPI’s as the beginning of Business Intelligence, not the end.

KPI’s and the Hawthorne Effect – Part 1

You’ve done it !!kpis and the hawthorne effect

Your Business Intelligence (BI) system is humming along nicely. You have all the necessary gadgets in place – bells and whistles included. Ad hoc reporting keeps your power users happy. Your managers can dive into any metric and explore the data. And dashboards let your executives track their Key Performance Indicators (KPI’s). All is well.

Actually, all is well only until a KPI misses the target (turning red). Now its time to get past the technology and actually manage your metrics. When a KPI misses the target, here is what usually happens.

  • Get on the phone
  • Have more reports run
  • Power-Users run multiple scenarios
  • Call meetings
  • Get the metric on everyone’s radar
  • Make it a standing item in weekly meetings

Basically — you give it a lot of attention – and rightly so. You put a Business Intelligence environment in place so that these items could come to the surface so you can manage them. As a result, the metric improves. You feel good about yourself. And why not? Your attention to the metric helped to improve it. You have the Midas Touch. This is why you get paid the big bucks. There is just one problem – The Hawthorne Effect.

Hawthorne Effect

Hawthorne Effect Definition

The Hawthorne Effect (link to more detail) is a term to describe the Hawthorne Works experiments in the 1920’s and 1930’s. In the experiments, the lighting was increased in a manufacturing plant to gauge the impact on worker productivity and as expected, productivity improved. The Midas Touch. After a period of time productivity returned to normal. When the researchers moved the lights back to normal wattage, an interesting thing happened. Productivity increased again. Basically, productivity increased when any number of things were changed, from variations in lighting to cleaning up the workspace, etc.. But the results were only temporary. Many follow-up studies have verified this effect. People just like it when you pay attention to them, but the improvement is only temporary.

Contrarian View of the Hawthorne Effect

Several years ago, Steven Levitt (of Freakonomics Fame) re-analyzed the data and discovered that the original experiments were not nearly as cut and dry as originally thought.  He suggests that the impact Hawthorne Effect from the original studies is over-amplified and not necessarily backed up by the data.  Click to view the study.

Cut To The Chase

If improvements are seen in your KPI’s only because you are giving them attention, then you should assume that those improvements are temporary. The improvements will be short lived unless you change the underlying process that is creating the results. Said differently, nothing changes for the long-term until you change the process. Most people look at Measurement Systems or KPI’s as an end-product, an alert that allows you to use your management skill to change the metric. However, the wise manager doesn’t see KPI’s as an end product of Business Intelligence, but rather as the first of three cogs that leads to long term improvement.

hawthorne effect

In the graphic above, KPI’s (Measurement/BI System) leads to Analysis (Root-Cause) which leads to to Process Improvement. Once your Process Improvement initiative is complete, a new set of measurement tools should be put in place to ensure that the results are consistent over the long term. Many organizations use Control Charts to track these process improvements.

So What Do You Do From Here?

In Part 2 of this series, we will explore your options.  More to come.