In view of some of the fantastic success rates cited by various claimants to the title of most effective addiction treatment, we might need to found a new self-help group: ECSA (“Exaggerated Claims of Success Anonymous”).
Step One: “Admitted… our stats had become unmanageable.” Step Two: “Came to believe that basic math and the scientific method could restore us to sanity.”
Somebody actually cited a 97% success rate in a supposedly serious LinkedIn post. Made me wonder what was wrong with the other 3%. Don’t they realize they’re letting the side down?
It’s all about how you measure outcomes, of course. One program markets a six week residential course (education, not treatment) that yields a recovery rate of 65%, as verified by an independent evaluator. Here’s how they measure it: One year after completing the course, the evaluation firm phones graduates with one question– are they clean and sober? Two-thirds reply “yes.” The interviewer then asks to talk to somebody else in the household. If that person comes and confirms the patient’s report, the interview ends.
Curiously, interviewers are specifically prohibited from asking any other questions, such as “What about last month?” Likewise, the interviewer doesn’t accept further comments volunteered by patient or family.
It’s a one word evaluation. Like rating a movie as ‘good’ or ‘bad’ with no space for anything else.
This might be the single worst evaluation method ever. Yet because the data was compiled and analyzed by an outside firm, the program can trumpet ‘independently verified’ outcomes.
Even the evaluators have complained about the methodology. To no avail, apparently.
It’s just the other end of the spectrum from those exaggerated ‘failure’ rates cited by critics of 12 Step fellowships. Those aren’t based in reliable methods, either.
Back when the results of Project MATCH were released, some disappointed psychologists complained that Twelve Step based treatment should be held to a higher standard than cognitive or motivational therapies, because of the 12 Step emphasis on abstinence. Cognitive and motivational therapies would presumably be judged by lesser measures, such as fewer drinks per episode, or fewer episodes per month. Didn’t make much sense, as it would completely skew the results of the study. Nonetheless it was advocated by people with advanced degrees, who should know better.
This is what usually happens when research is held captive by partisan motives. It’s a waste of time and resources that could be put to better use.
