Sometimes you need a method for trying to make sense of an issue when everybody has a different perspective and you need to deliver some clarity and direction. And sometimes you need to get people to see beyond the trees of different evidence articles and try to produce a synthesis of what the issues are, informed by evidence. I don’t know about you but I find this can be extraordinarily difficult.
So I want to share with you two methods which work for me. The first is search conferencing for participatory planning when you are ready to build your strategy and want to work out from a range of stakeholders where you are headed.
The second is what I call the “Principles and Direction Sift” where you’re not sure you’re even ready to get a plan together, or where you might be but everyone seems to be pulling in different directions based on what they’ve read.
The Search Conference
For some time I have been using the Search Conference method for participatory planning. In the US it is widely used for community action. The problem with this method is it can be a little cumbersome in the planning and running, when you’re not even at that stage yet. But if you want some of my favourite reading on this try
Emery and Purser, The Search Conference. The Publisher’s blurb about this book is actually quite accurate
“Reviews and Commentary From The Publisher: The Search Conference is not just another management tool, but a participative approach to planned change that engages the collective learning and creativity of large groups, inspiring people to find common ground around new strategies, future directions, and joint actions. The process combines the best practices associated with strategic planning, systems thinking, and effective group communications – enabling participants to take part fully, rise above self-interest, and make decisions for the common good. Written by Search Conference pioneer Merrelyn Emery and Ronald Purser, this book uses a wealth of illustrative examples from a wide variety of nonprofit, business, and public organizations that attest to the versatility of this important organization development intervention.”
And also there are times in public health when you can’t use a method quite at this level.
So here is the other method I use, the “Principles and Direction Sift”. Snappy title, I know, but it does have its uses.
The Public Health Principles and Direction Sift
This can be done in one meeting but often you get better results doing it over several sessions. It also seeks to be faithful to the scientific basis of public health and deal with the problem where evidence is lacking or silent. I’m not claiming this is perfect, but where I have used it it does work.
Step 1: Problem Statement
What, from a public health perspective, is the problem. E.G. epidemiology suggests there is no consistency of what follow up and support patients get after an NHS Health Check. Get everyone to state the problem, get them to agree the elements if need be and get a shared statement of the problem, challenge or issue. This could take 10 minutes or an hour. But get it out. It’s often good do to this as one meeting before you do the rest, and then everyone looks to find evidence and shares the evidence a week before the next meeting
If you are doing it in two meetings then at the start of the second meeting everyone needs to see and the group needs to revisit the problem statement.
Step 2: Evidence Assessment
Everyone reads and discusses the evidence and from the evidence identifies i) potential candidate interventions and solutions which have evidence behind them; ii) areas where evidence is not clear and iii) areas with no evidence. This could take between 15 minutes and an hour. But it should not dominate the session, the next stages are just as important.
Step 3: Evidence-current state fit
You group the interventions you currently have under which of the i), ii), iii) categories of evidence basis you agreed above, and if you need it a iv) category called “don’t know/uncertain/park” What you should now have is a list of what you do with an idea of how they fit against the evidence. You will probably have at least some gaps where things you do have little evidence and where things with evidence are not being done well or at all.
Step 4: Principles and Priorities
Now you look back to the Step 1 problem statement and you compare the output from this with the output from Step 3.
You now need to work through and agree together an assessment of:
i) how well what you do now meets or addresses the problem and
ii) what principles (things like user acceptability, co-production, access) and
iii) which priorities (which bits of a strategy need doing first)
You are going to work on.
You will almost certainly have some stuff you are doing with evidence (and a pile of gaps) and some priorities you want to address with no evidence for what you can do about it.
For the former you need to ensure good implementation faithful to the evidence. Remember, implementation science suggests 30% of a programme’s value can be lost in poor implementation.
Step 4a: Navigating Uncertainty: Silence or Absence of Evidence
For the bits where there is no evidence, you cannot simply do nothing. You can either decide to do something pragmatic (which I would advise against unless that’s all there is you can do) or you can try to develop a logic model for some interventions where you can try to find evidence. A good way of doing this is to brainstorm from the principles those things which you think might be worth considering. Then you reality check them in the group.
When you have a list, you need public health and topic experts to search for evidence. If there is evidence you can apply it to commissioning interventions and go straight to Step 5 in the same meeting or another meeting. If there is no evidence then use Step 4b.
Step 4b: Navigating Uncertainty: What is worth trying?
Public Health and topic experts should come to this with a worked up statement of evidence where it exists, and where there is no evidence some form of assessment of the ideas put forward and a theory-based assessment of what might work and a model of how to commission and evaluate it. This will need robust frameworks to evaluate and monitor whether it works so you can try it and if it doesn’t work decommission it. You need the group to agree a consensus around this before you go any further.
Let’s be clear what I have just suggested in the paragraph above. I am suggesting, and I am by no means the first to do this, the application of scientific method to the policy and commissioning process where evidence is silent or lacking. In other words, we would expect public health science to be busy creating evidence for us. But we can try to do this in a policy environment, and authors have put forward a variety of ways of doing this. Matrix RCL (who were major figures behind the Campbell Collaboration, the social welfare and social justice Answer to Cochrane) put forward just such a model for commissioners and policymakers in a briefing which has been widely used. You can access this briefing at http://www.jimmcmanus.info/Public-Health-Resources.html under the heading “Will this policy work?”
So an example: Imagine the evidence is silent on how brief interventions translate to behaviour change in primary care. So going back to theory of behaviour change we design a brief intervention for behaviour change using the methods which the theory of behaviour change suggests is salient, work it into an intervention, work up a framework to judge its worth and commission a pilot to see if it works.
I could spend the length of a book digressing here on the philosophical underpinnings of evidence-based practice and science. I will refrain for now. But I will point you to Nancy Cartwright’s lecture on the philosophy of science applied to policy problems ranging from Tamil Nadu to social welfare, which you can find here. A thought provoking take, whether you agree with her or find it total madness.
MEETING TWO OR MEETING THREE IF YOU HAVE A THIRD MEETING
Step 5: System Check
You now check where you have got to. You should have a statement of what you are going to do. You now need to check that there is congruence between the problem, the evidence assessment and the identified interventions.
You can now start the phase of working up how you will implement all this.
This is not a rigid method. It works for some things and not others, but what it seeks to do is use a disposition to be evidence-led wherever possible and integrate that into how you build and commission programmes and systems. It also seeks to deal with the fact that for much of health and social care, the evidence is absent.
“The Scientific Method”
The key point here is what would a scientific method suggest? That’s actually logically quite simple: read, hypothesise, experiment, evaluate and adapt consistent with the theory and evidence to date.
If the theory and evidence to date is right, you should eventually find something which works, because that’s how scientists do science in an academic environment. (And let’s remember the history of hunch, hypothesis, gut instinct and theory in science is actually a crucible of invention, not a pure process.)
If you don’t find something which works, then either i) you’re not following the method and theory properly, ii) not implementing it well, or iii) the theory and state of the knowledge is wrong. Humility, perhaps, and real life experience of how fragmented our systems are would suggest much of the time the answer is i) or ii)!