Short Report

Does higher cost mean better quality? evidence from highly-regarded adolescent drug treatment programs

Bruce R Schackman¹ , Erick G Rojas¹ , Jeremy Gans² , Mathea Falco² and Robert B Millman¹

¹  Department of Public Health, Weill Cornell Medical College, 411 East 69^th Street New York, NY 10021, USA

²  Drug Strategies, 1616 P Street, N.W. Suite 220 Washington, DC. 20036, USA

author email corresponding author email

Substance Abuse Treatment, Prevention, and Policy 2007, 2:23doi:10.1186/1747-597X-2-23

The electronic version of this article is the complete one and can be found online at: http://www.substanceabusepolicy.com/content/2/1/23

Received:	15 February 2007
Accepted:	31 July 2007
Published:	31 July 2007

© 2007 Schackman et al; licensee BioMed Central Ltd.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

We conducted a survey to examine whether reimbursement levels are associated with the quality of adolescent substance use treatment programs in the United States. Between March and September 2005, telephone and written surveys were administered to program, clinical, and finance directors of previously surveyed highly regarded programs. Differences in quality scores were compared for programs with above versus below median reimbursement levels and examined in multivariate regression models constructed separately for programs offering residential and outpatient treatment. In residential treatment multivariate regression models, higher quality scores were associated with higher reimbursement, but this relationship was not observed for outpatient treatment. Even the highest level of outpatient reimbursement received may be too low to support quality improvement initiatives. Our results suggest that higher reimbursement may be a necessary component of quality improvement for residential adolescent drug treatment programs, and emphasize the need for further research to determine what levels of reimbursement and insurance coverage policies will encourage the expansion of high quality outpatient programs.

Background

Although substance abuse is pervasive among American youth, only one in ten adolescents who need treatment actually receives help [1]. Adolescents differ from adults in terms of patterns of substance use, the influence of developmental and social factors, and the prevalence of co-occurring disorders [2]. Treatment experts agree that adolescent programs should not simply be adult programs modified for adolescents. Adolescent treatment programs must address the different contexts that shape an adolescent's life including school, recreation, peers, the distinctive nature of the juvenile justice system, and differences in medical care [3].

Costs are a major concern for parents and professionals who refer adolescents to treatment, particularly for residential programs. Charges for outpatient programs are generally lower, because most adolescents attend just two to five outpatient sessions per week, but they can still impost a financial burden. Private insurance spending for drug and alcohol abuse treatment has declined, while cost-sharing requirements have increased for those with insurance [4].

Substance abuse treatment providers report that lower reimbursement and greater administrative burdens are associated with difficulties attracting and retaining better-trained staff, and coping with overworked staff [5]. One would expect that higher reimbursement would also be associated with more treatment, more therapeutic hours or more frequent treatment contacts, which have been found to be associated with improved outcomes (e.g. fewer positive urinalysis results, fewer readmissions) [6,7]. Studies have evaluated the impact on substance abuse outcomes of reimbursement systems such as Medicaid managed care [8,9] and performance-based contracting [10,11], but we found no empirical studies that directly evaluate associations between reimbursement rates and substance abuse treatment program quality. However, high-quality pediatric and adolescent preventive care has been found to be associated with higher reimbursement [12], and higher quality child day care centers also have higher costs [13].

In a previous study, we evaluated the quality of 144 adolescent treatment programs identified as exemplary by professionals in the field, directors of state drug and alcohol agencies, and major national organizations [14]. To examine whether there is any direct association between the cost and quality of these programs, we collected new information on reimbursement levels in a follow-up survey.

Methods

Sample

In the previous study, alcohol and drug abuse agencies in all 50 states, several national organizations and federal agencies (including the American Medical Association, the American Academy of Pediatrics, and the National Institute of Drug Abuse), and members of an expert panel were requested to identify adolescent substance use treatment programs that they considered to be exemplary and thus highly regarded [14].

The sample for this study consists of 138 programs still in operation of the 144 programs that had been selected to participate in the previous study. Three surveys were conducted between March 2005 and September 2005: an in-depth telephone survey administered to program directors, and two telephone or written surveys administered to clinical and finance directors.

Measures

Respondents to the finance director survey were asked to indicate the average reimbursement per day, per week or per stay by treatment setting and the proportion of reimbursements by type of payer (government, private insurer, self-pay). If residential reimbursement levels were reported in monthly or weekly units, these responses were divided by their respective time frame to have a uniform time unit for reimbursements. If reimbursement levels were reported per stay, these responses were divided by the average length of stay reported by the program.

In the previous study, a panel of 22 experts defined 9 key elements of effective treatment for adolescent substance abuse based on a review of the literature, interviews and discussions with panel members, and small working group recommendations. Open-ended survey responses were then coded according to 45 quality components based on these 9 key elements with a reliability of 0.79 as measured by Cronbach's α with standardized items [14]. In this study, the 45 quality components were condensed to a 20-item quality score according to the following procedure. First, patterns of missing data and the extent of variation in the responses were used to reduce the number of questions from 45 to 31. Then, a factor analysis was conducted on the remaining questions using a cut-off of 0.4 on factor loading [15]. The 20 questions with the highest factor loading explained 85% of the variation in the previous study survey responses. These 20 questions comprised the quality score for this study (see Appendix).

Additional variables that were repeated from the previous study included treatment approaches used (twelve-step, cognitive behavioral, therapeutic community, motivational enhancement, multi-systemic therapy, and multidimensional family therapy), geographical region, and accreditation. We also obtained data on other variables potentially associated with reimbursement level, including the presence of a waiting list to enter the program, referral sources, average daily client census, program treatment slots and average length of stay (inpatient) or treatment duration (outpatient). An occupancy ratio was calculated for each program that provided an average daily client census and program treatment slots.

Analysis

Since reimbursement for residential services differs significantly from reimbursement for outpatient services, we conducted separate analyses for each of these services. The sample size was insufficient to develop three separate models for programs offering residential only, outpatient only, and both services. Differences in median quality scores were analyzed by reimbursement level and other program characteristics using the Mann-Whitney test, and we report approximate z-scores using this test. Because we consider this analysis to be exploratory in nature given the lack of relevant previous research conducted on this topic, all variables with p-values ≤ 0.25 were entered into linear regression models and final multivariate linear regression models were constructed using the backward elimination procedure [16]. An advantage of using this procedure over forward selection and stepwise regression is that the predictive capability of all variables of potential interest can be examined jointly [17].

Results

Response rates were 87% for program directors, 60% for clinical directors, and 57% for financial directors. Table 1 provides an overview of program characteristics for all programs (n = 120) and for programs with sufficient data from all interviews to be included in regression models (n = 63). There are no notable differences between programs included versus excluded from the regression models, and approximately 40% of residential-only programs, 50% of outpatient-only programs, and 50% of programs providing both residential and outpatient services were included in the regression models. The mean daily reimbursement reported for residential services was $201 compared to $56 for outpatient services (Table 2). The mean quality score for all programs was 14.28 out of a possible 20 and for programs included in the regression models the mean quality was 14.77, with no significant differences by setting.

Among programs offering residential services, factors individually associated with a higher quality score in bivariate analyses included higher average daily reimbursement (Mann-Whitney z = 2.12, p = 0.04), multisystemic therapy (Mann-Whitney z = 2.12, p = 0.03), motivational enhancement therapy (Mann-Whitney z = (-)2.05, p = 0.04), and accepting client referrals from hospitals (Mann-Whitney z = 2.22, p = 0.03), private practitioners (Mann-Whitney z = (-)2.92, p < 0.01), or insurance companies (Mann-Whitney z = 2.15, p = 0.03). However, the only significant predictor of a higher quality score in the final multivariate model for these programs was higher residential reimbursement (t = 2.24, df = 35, p = 0.03) (Table 3).

Among programs offering outpatient services, factors individually associated with a higher quality score in bivariate analyses included monthly staff training (Mann-Whitney z = 2.16, p = 0.03), having a waiting list (Mann-Whitney z = 2.22, p = 0.03), multisystemic therapy (Mann-Whitney z = 2.59, p = 0.01), and accepting client referrals from hospitals (Mann-Whitney z = 2.25, p = 0.02) and private practitioners (Mann-Whitney z = (-)2.43, p = 0.02). In the final multivariate model for these programs, a higher quality score was associated with having a waiting list to enter the program (t = 2.45, df = 48, p = 0.02), having received accreditation from a national organization (t = 3.07, df = 48, p < 0.01) and having received grants (t = 2.32, df = 48, p = 0.02); there was no significant association with reimbursement level (Table 4).

Discussion

We conducted surveys of program, clinical, and finance directors of highly regarded adolescent substance use treatment programs to examine whether reimbursement levels are associated with program quality. Our analysis was subject to several limitations. Our surveys were limited to highly regarded programs and there was a relatively narrow range of variation in quality. Hence the effect size of reimbursement on quality was relatively small: an additional 1.0 quality point was associated with residential reimbursement above the median compared to a mean (standard deviation) quality score of 14.6 (1.6). With more complete data from programs with greater variations in quality and size, we could have seen a greater reimbursement effect.

Our cross-sectional design is vulnerable to reverse causality, in this instance the possibility that higher quality programs are able to obtain greater reimbursement due to consumer demand as opposed to higher reimbursement directly supporting better quality. However, most adolescent substance use treatment programs are non-profits and there is little evidence of high profit margins in this setting.

Our analysis indicates that higher cost residential adolescent programs deliver better quality treatment, suggesting that higher reimbursement may be a necessary component of quality improvement for these types of programs. However, we found no association between higher cost for outpatient programs and better quality treatment. High quality outpatient treatment appears to be recognized externally when it occurs, since better quality was associated with waiting lists, accreditation, and success in obtaining external funding. However, we believe that even the highest level of outpatient reimbursement received (e.g. $84 per day for the top quartile of outpatient reimbursement) is still too low to have an effect on quality. The great majority of adolescents receive treatment in outpatient settings rather than in residential settings [18]. Further research is needed to determine what levels of reimbursement and insurance coverage policies will encourage the expansion of high quality outpatient programs.

Competing interests

The author(s) declare that they have no competing interests.

Authors' contributions

BS, MF, and RM conceived of the study, devised the study design and participated in the completion of the manuscript. JG participated in the data collection and completion of the manuscript. ER participated in the statistical analysis and completion of the manuscript. MF and RM contributed expert opinion. All authors read and approved of the final manuscript.

Appendix: 20-item Program Quality Score Questions

1. Does the program either provide mental health services for clients onsite or coordinate their care with community mental health providers?

2. Do you conduct some form of reassessment of clients during the course of treatment?

3. In screening and assessment process, does the program use either a standardized instrument* or a structured clinical interview?

4. In your screening and assessment process, does the program use a standardized mental health instrument*?

5. Does the treatment plan address mental health issues?

6. Do you provide the client's family with individual and/or multifamily therapy sessions?

7. Will you refer parents who are abusing substance to treatment?

8. Do you maintain contact with juvenile justice officials regarding clients who have been referred by the juvenile justice system?

9. Do you utilize a specific text or curriculum designed for adolescents?

10. Are adolescent clients typically treated only with other adolescents, as opposed to being integrated with adult clients?

11. Do you incorporate positive reinforcements, such as increasing responsibilities and/or privileges?

12. Do you utilize special recreational programming or offer courses of particular interest to adolescents?

13. Do you provide clients with gender-specific group sessions?

14. Does your program tailor itself for gay and lesbian youth?

15. Do you create a continuing care plan for the client beyond referring clients to outside services?

16. Does the program link clients with relevant community services upon discharge?

17. Do you collect any other information related to client outcomes after treatment?

18. Are there any evaluation or other types of studies completed on your program, and if so, may we receive a copy?

19. Do you have staff members with training in mental health issues?

20. Do all clinical supervisors possess at least a master's degree?

* Standardized instruments sources: [19,20]

Acknowledgements

This research was supported by a grant from the Robert Wood Johnson Foundation. The funding agreement ensured the authors' independence in designing the study, interpreting the data, and writing and publishing the report. The authors gratefully acknowledge Madhu Mazumdar, PhD, Kathy Zhou, PhD, and Quanhong Ni, MS for their statistical support, and the treatment programs for their participation in the surveys.

References

1. Johnston LD, O'Malley PM, Bachman JG, Schulenberg JE: Teen drug use down but progress halts among youngest teens. In University of Michigan News and Information Services. Ann Arbor, MI;

   December 19th, 2005

2. Muck R, Zempolich KA, Titus JC, Fishman M, Godley MD, Schwebel R: An overview of the effectiveness of adolescent substance abuse treatment models.

   Youth Society 2001 , 33(2):143-168. Publisher Full Text

3. Drug Strategies: Treating Teens: A Guide to Adolescent Drug Programs. Washington DC , Drug Strategies; 2003.

4. Mark TL, Coffey RM: The decline in receipt of substance abuse treatment by the privately insured, 1992-2001.

   Health Aff (Millwood) 2004 , 23(6):157-162. PubMed Abstract | Publisher Full Text

5. Green CA, Lynch FL, Polen MR, McFarland BH, Dickinson DM, Freeborn DK: Talk about costs: health care professionals' views about expenses related to substance abuse treatment.

   Adm Policy Ment Health 2004 , 31(6):483-494. PubMed Abstract | Publisher Full Text

6. Coleman JC, Paul GL: Relationship between staffing ratios and effectiveness of inpatient psychiatric units.

   Psychiatr Serv 2001 , 52(10):1374-1379. PubMed Abstract | Publisher Full Text

7. Kaminer Y, Burleson JA: Correlation between ancillary community services with adolescent substance use disorders treatment outcome.

   Subst Abus 2005 , 25(2):15-20. PubMed Abstract | Publisher Full Text

8. Harrow BS, Tompkins CP, Mitchell PD, Smith KW, Soldz S, Kasten L, Fleming K: The impact of publicly funded managed care on adolescent substance abuse treatment outcomes.

   Am J Drug Alcohol Abuse 2006 , 32(3):379-398. PubMed Abstract | Publisher Full Text

9. McFarland BH, Lynch FL, Freeborn DK, Green CA, Polen MR, Deck DD, Dickinson DM: Substance abuse treatment duration for Medicaid versus commercial clients in a health maintenance organization.

   Med Care 2006 , 44(6):601-606. PubMed Abstract | Publisher Full Text

10. Commons M, McGuire TG, Riordan MH: Performance contracting for substance abuse treatment.

    Health Serv Res 1997 , 32(5):631-650. PubMed Abstract | PubMed Central Full Text

11. Shen Y: Selection incentives in a performance-based contracting system.

    Health Serv Res 2003 , 38(2):535-552. PubMed Abstract | Publisher Full Text | PubMed Central Full Text

12. McInerny TK, Cull WL, Yudkowsky BK: Physician reimbursement levels and adherence to American Academy of Pediatrics well-visit and immunization recommendations.

    Pediatrics 2005 , 115(4):833-838. PubMed Abstract | Publisher Full Text

13. Mocan HN: Cost functions, efficiency, and quality in day care centers.

    The Journal of Human Resources 1997 , 32(4):861-891. Publisher Full Text

14. Brannigan R, Schackman BR, Falco M, Millman RB: The quality of highly regarded adolescent substance abuse treatment programs: results of an in-depth national survey.

    Arch Pediatr Adolesc Med 2004 , 158(9):904-909. PubMed Abstract | Publisher Full Text

15. Johnston RA, Wichern DW: Applied Multivariate Statistical Analysis. Inglewood Cliffs, NJ , Prentice Hall; 1988.

16. Kleinbaum DG, Kupper LL, Muller KE, Nizam A: Applied regression analysis and other multivariable methods. 3rd edition. Pacific Grove, CA , Duxbury Press; 1998:xviii, 798 p..

17. Dallal GE: The little handbook of statistical practice: simplifying a multiple regression equation. [http://www.statisticalpractice.com] webcite

18. Office of Applied Studies: Characteristics of young adult (aged 18-25) and youth (aged 12-17) admissions: 2004.

    In The DASIS Report Edited by: Substance Abuse and Mental Health Services Administration. 2006.

19. Winters KC: Screening and assessing adolescents for substance abuse disorders SAMHSA/CSAT Treatment Improvement Protocol 31. Rockville, MD , Substance Abuse and Mental Health Services Agency (SAMHSA), Center for Substance Abuse Treatment; 1999.

20. Winters KC: Assessment of alcohol and other drug use behaviors among adolescents. In Assessing Alcohol Problems A Guide for Clinicians and Researchers Second Edition. Edited by: Allen JP, Wilson VB. Bethesda, MD , National Institute of Alcohol Abuse and Alcoholism; 2003.

Have something to say? Post a comment on this article!