The third law of information is:
3. Data that is collected to measure performance loses validity.
First a confession. In the late 1980s, I worked as Director of Planning and Information in a mental health service in the NHS. One of my tasks was to organise the statistical returns on clinical activity for despatch to the Department of Health. At that time, these were based on a set of standard definitions called the ‘Korner system’ (after a woman who chaired a committee which recommended them). Our service included a brilliant and very hard-working consultant psychiatrist for the elderly. She believed that assessments of new patients should initially be in the patient’s own home (a ‘domiciliary visit’ or ‘DV’). Since she was an orthodox Jew, this meant a lot of walking when her duty days coincided with the Sabbath. Unfortunately, the Korner system required information about scheduled outpatient clinics but not domiciliary visits. Following the Korner rules would have meant that our most active consultant would appear as our least active. This was obviously unjust, so I modified the returns for her clinical activity to record each DV as an attendance at a (non-existent) outpatient clinic.
Paradoxically, my data-adjustment produced statistical returns which were a more accurate reflection of clinical activity than would have been the case without such adjustment. Nonetheless, they became an inaccurate record of inpatient clinics in the service in which I worked. I suspect that data-adjustment in the desired direction was and is rife in the NHS. Although this is dishonest, it can cause far less damage than changing reality to generate honest statistics. A well-known example of changing reality in the NHS is to make patients wait in ambulances outside A&E departments. This reduces the time the patient spends in A&E for the purposes of official statistics, and hence enables the hospital to meet a government target. There are many, many more examples in the NHS of how meeting centrally-imposed targets can damage patient care.
This is not a recent phenomenon. The whole technology of corporate strategic planning and management by targets owes its origins to Gosplan, the state planning commission in the USSR. Studies of the Soviet economy in the 1960s and onwards were full of examples of how rational responses by individual enterprises to centrally-determined targets could produce absurd results. These included the shoe factory that met its target number of shoes by producing shoes all in one size, and the steel factory that met it target for weight of steel by producing a few huge ingots.
3. Data that is collected to measure performance loses validity.
First a confession. In the late 1980s, I worked as Director of Planning and Information in a mental health service in the NHS. One of my tasks was to organise the statistical returns on clinical activity for despatch to the Department of Health. At that time, these were based on a set of standard definitions called the ‘Korner system’ (after a woman who chaired a committee which recommended them). Our service included a brilliant and very hard-working consultant psychiatrist for the elderly. She believed that assessments of new patients should initially be in the patient’s own home (a ‘domiciliary visit’ or ‘DV’). Since she was an orthodox Jew, this meant a lot of walking when her duty days coincided with the Sabbath. Unfortunately, the Korner system required information about scheduled outpatient clinics but not domiciliary visits. Following the Korner rules would have meant that our most active consultant would appear as our least active. This was obviously unjust, so I modified the returns for her clinical activity to record each DV as an attendance at a (non-existent) outpatient clinic.
Paradoxically, my data-adjustment produced statistical returns which were a more accurate reflection of clinical activity than would have been the case without such adjustment. Nonetheless, they became an inaccurate record of inpatient clinics in the service in which I worked. I suspect that data-adjustment in the desired direction was and is rife in the NHS. Although this is dishonest, it can cause far less damage than changing reality to generate honest statistics. A well-known example of changing reality in the NHS is to make patients wait in ambulances outside A&E departments. This reduces the time the patient spends in A&E for the purposes of official statistics, and hence enables the hospital to meet a government target. There are many, many more examples in the NHS of how meeting centrally-imposed targets can damage patient care.
This is not a recent phenomenon. The whole technology of corporate strategic planning and management by targets owes its origins to Gosplan, the state planning commission in the USSR. Studies of the Soviet economy in the 1960s and onwards were full of examples of how rational responses by individual enterprises to centrally-determined targets could produce absurd results. These included the shoe factory that met its target number of shoes by producing shoes all in one size, and the steel factory that met it target for weight of steel by producing a few huge ingots.
No comments:
Post a Comment
Comments welcome