How to Measure?

All too often, new initiatives (new ways of delivering care or support) can't show whether they have made a difference because nobody measured.
Why is this? Some sort of number-phobia, a philosophical view that since the service is somehow sacred, to measure and show "this much benefit", or "that much cost-effectiveness" would taint the service delivered. But it can be really crippling when you go to a backer and ask for resources to build or expand your service.
What criteria should you apply?
You need measures that mean something to:

• the organisation which will provide the funding (so they can see what they are getting for their money)
  
• the person or group receiving help (the voice of the user is very important in demonstrating whether your service delivers a benefit)
• the team delivering the service (typically these will be the ones gathering the information, and if they aren't motivated then they won't put the effort in to collecting the measures)

Uner ideal circumstances you would get everyone to decide the measures they would like to see, and then discuss how practical it is to collect these. Ideal doesn't often happen, and in the end the team collecting the measures have to agree to collect them, so as a shortcut can I recommend getting the staff together to decide? They will typically understand the user perspective closely (we're talking about new projects and new services so the reasons why the service was needed and has been set up will be high on everyone's agenda), and as long as someone plays the part of any potential funding body you should get measures which cover all angles.

Second stage is often to ensure that the measures selected are practical. Evaluation should never be at the expense of care delivered - which means few, simple measures demonstrating clear results.

Of course there may be governance issues - if you are collecting information about vulnerable people (children, vulnerable adults, healthcare patients) then do you need permission? What information are you collecting and is it possible to identify the patient? Where are you going to store the information collected - is it secure? All this means that you probably have to seek permission from a governance or evaluation scrutiny committee and woe betide anyone who starts measuring before ensuring they can collect measures, especially if the scrutiny committee finds that the information they are collecting needs special care!

To evaluate results you are going to need some understanding of statistics. A good start point is "Statistics without tears" or the "for dummies" guides. You don't need to get worried about Sampling Theory - in order to apply this you need to know how much difference you expect, and to be blunt if you think you are only going to make a small difference this may be enough in the commercial setting but in delivery of care or support a small difference isn't going to win you the funding you seek.

Know enough statistics to be able to compare two groups (the before and the after) using something like

• Chi squared (where the result is Yes or No eg for "Happy or Sad", "needed hospital/ did not", "working/ not working") and
  
• t-test (or Analysis of Variables/ ANOVA/ MANOVA) where the measure is on a scale ("this happy 1 - 5", "care involved this many steps")

and how to a run chart to show variability and the effect of the change (SPC - Statistical Process Control) and draw bar charts and pie charts which everyone understands.

Aim to present your results in as simple a way as possible - we'll look at Cash and Financials in another blog


Part of Chapter 4
Once you've measured you need to