Measuring outcomes is a requirement for a modern behavioral health program. The key is exactly which outcomes you choose to measure. They might be a relatively ‘soft’ variety like client satisfaction scores, pre-and post-tests, session attendance, etc. Or something more concrete, such as drug test results or rearrest rates. Most programs wind up with a blend of the two.
Medicaid likes to measure the following, among other measures:
- Retention in treatment, defined as a client attending sessions for at least 90 days.
- Successful completion, where a client achieves at least a majority — say 2/3 — of the goals on the treatment plan.
- Readmission to treatment, usually during the 12 months following discharge from the initial episode
- Abstinence, or the absence of reported use of the primary drug during a period of say, 30-60 days immediately preceding discharge.
I can hear you counselors groaning at the obvious problems with that last one.
These measures are popular because they are believed to correlate with positive long term changes in behavior. Correlation is not, I repeat not, causation, but it has some value in itself.
Program outcomes have to carefully analyzed. For instance, one program we worked with boasted an unusually low rate of readmissions post-treatment. But ‘drilling down’, we found this was due to the slowness of the Court system in that community; it often took six months after arrest for a client to get a referral for treatment, pushing many outside that 12 month window. Translation: the program wasn’t nearly as effective as it appeared to be on the surface.
Impact is something more complicated, and difficult to measure. An impact study might look for evidence that the mental health counseling we provided during the initial episode resulted in someone who not only accepted a referral for further care but stuck with that care and clearly benefited from it. This is after all the sort of benefit that makes treatment more attractive to society than simply tossing people in jail. So it’s worth the extra effort to demonstrate it.
That’s one reason that research like this is so helpful in designing programs and systems. It’s a road map for what to focus on and what might be safely ignored on the grounds it won’t make much difference. We know that an inpatient stay at the beginning of treatment may be extremely helpful in terms of harm reduction, stabilization and a fresh start on recovery, but doesn’t appear to make much difference in someone’s prospects for long-term abstinence– suggesting that there are other factors at work, that are worthy of our attention.
A residential program for women is considering incorporating basic skills training in Microsoft Office along with treatment, in the hope that will improve their chances of employment that not only provides a living wage but is in a safe environment. Makes sense, right? We’ll measure improvements in those skills. But we’ll also need to check back later to see how much difference we might have made in the lives of the graduates.
