 |
|
After the Grant Runs Out
Long-term Provider Health Maintenance Compliance Using a Computer-Based Tracking System
Katrina A. Cooley, BA;
Paul S. Frame, MD;
Shirley W. Eberly, MS
Arch Fam Med. 1999;8:13-17.
ABSTRACT
| |
Objective To measure long-term provider (physicians and physician's assistants) health maintenance compliance 4 years after the completion of a grant-funded project to improve provider compliance by using a computer-based health maintenance tracking system.
Design Cross-sectional comparison of provider health maintenance compliance for patients receiving computer-based health maintenance tracking in 1992 and 1996.
Setting Rural, multiple-office, nonprofit, fee-for-service family practice.
Patients Adults who had been seen by the practice in the past 2 years.
Main Outcome Measures Overall provider compliance with the common elements of the health maintenance protocols in 1992 and 1996. Provider compliance with specific, individual preventive interventions was compared.
Results Overall provider compliance was 83% in 1996, compared with 80% in 1992. This difference was statistically significant (P=.05) but not clinically significant. Provider compliance was significantly higher in 1996 for 3 procedures: blood pressure determination, tetanus-diphtheria immunization, and weight. It was unchanged for 5 procedures: clinical breast examination, mammography, Papanicolaou smears, cholesterol determination, and fecal occult blood testing for colon cancer. Provider compliance with obtaining a history of tobacco use declined.
Conclusion Improvements in provider health maintenance compliance associated with installation of a computer-based health maintenance tracking system were maintained 4 years after cessation of the formal research intervention.
INTRODUCTION
CONCEPTS OF evidence-based preventive medicine have become widely accepted during the past 2 decades. However, provider (physicians and physician's assistants) compliance with preventive recommendations has been less than ideal.1-2 Several authors have described strategies and systems for improving provider compliance with a comprehensive health maintenance program. Harris and colleagues3 studied nurse-initiated prompting and computerized prompting. During the 5-year study period, they found that both prompting systems (especially the computer-based system) improved the rate of performance of preventive procedures. During a 24-month period, Dietrich et al4 studied the use of facilitators and education programs to improve provider health maintenance compliance. While educating providers about prevention improved rates of mammography screening alone, having a project facilitator help establish a routine office intervention system improved rates for multiple preventive procedures. McPhee et al5 used a computerized tracking system to prompt providers to perform health maintenance procedures and found computer-based tracking more effective than usual care after a 2-year study period.
In 1994, Frame and colleagues6 published results of a 2-year prospective, randomized, controlled trial comparing a computer-based health maintenance tracking system that generated annual provider and patient reminders with a manual flowchart-based tracking system in which patient reminders were triggered by provider request. At the end of the 2-year intervention, overall provider compliance with the health maintenance protocol increased 15 percentage points in the group receiving computer-based prompting and 4 percentage points in the manual tracking group. The computer-based tracking group had significantly higher provider compliance than the manual group for 8 of 11 individual procedures.
All of these studies showed improvement in provider compliance during the period of intervention. A crucial, and largely unanswered, question is whether provider health maintenance compliance gains are maintained after the study ends. What happens when grant funding stops and the system has to exist on its own merits? Is the new level of provider compliance with the health maintenance protocol maintained, or does it drift back to levels seen before the intervention was implemented? Dietrich et al7 published data from four fifths of the practices in the Cancer Prevention in Community Practice Project4 1 year after the end of the study, showing that improvements were maintained for stool occult blood testing and breast self-examination but declined for mammography and clinical breast examination. No previous study has addressed the question of the long-term effects of a health maintenance prompting system several years after the end of grant funding.
This study reports long-term provider health maintenance compliance in the same Tri-County Family Medicine practice studied by Frame et al in 1991 and 1992.6 The entire practice has been using the computer-based tracking system developed as part of that project7 since the end of the previous study. However, there has been no further grant funding and no systematic provider reinforcements other than those provided by the computer-based tracking system. We report provider health maintenance compliance 4 years after the end of the previous intervention.
METHODS
SETTING
Tri-County Family Medicine is a rural, nonprofit, fee-for-service, 5-office practice with approximately 20,000 active patients located in western New York State. In the original study (Tri-County, 1992) patients were randomly selected from 4 of the 5 offices and allocated to have health maintenance tracked either by the new HTRAK computer-based system developed for that project or by the manual flowchart-based system previously used by the practice. (HTRAK is a computer-based health maintenance tracking system developed as part of the original grant. The copyright is owned by Tri-County Family Medicine, Dansville, NY, 1990.) The fifth Tri-County office was excluded from both the original and the follow-up study, since it is the practice of the principal investigator (P.S.F.).
The HTRAK computer-based health maintenance tracking system has previously been described in detail.8 To use the system, providers enter updated health maintenance data on the patient encounter form, along with the billing and diagnostic data entered at the end of a patient visit. Data entry personnel enter this health maintenance information into the billing computer after each visit, at the same time that billing and diagnostic data are entered. Once a year, usually in the month of the patient's birth, the system prints a provider reminder, which is placed on the patient's chart. This reminder tells the provider at a glance which procedures have been done and which procedures are due for that patient. At the same time, the computer system prints an annual letter for all patients, reminding them of overdue health maintenance and suggesting that they make an appointment for a health maintenance checkup if needed.
Several changes were made to the Tri-County health maintenance protocol just after the Tri-County 1992 study. Table 1 compares the protocol used in 1991 to 1992 with that used from 1993 through 1996. In the current follow-up study, we evaluated provider compliance only with procedures that were in the protocol in both 1992 and 1996. Several procedures (eg, fecal occult blood testing, history of tobacco use, and mammography) were included in both protocols but had different frequencies or were suggested for different age groups. These procedures were included in the follow-up analysis, and providers were expected to be compliant with the appropriate 1992 or 1996 protocol.
|
|
|
|
Table 1. Comparison of Tri-County Family Medicine Health Maintenance Protocols for Adults Between 1992 and 1996
|
|
|
Between 1990 and 1996, 1 physician left Tri-County and was replaced by another, 1 physician's assistant left and was replaced, and an additional physician's assistant was hired. Thus, 5 of 6 physicians studied who were practicing with Tri-County Family Medicine in 1996 and 3 of 5 physician's assistants also participated in the earlier study.
DATA COLLECTION
Provider health maintenance compliance for a randomly selected cross section of adult patients from the 4 Tri-County offices in 1996 was compared with provider health maintenance compliance for the patients receiving computer-based tracking at the end of the 1992 study. A sample size of 275 was considered adequate to detect a clinically significant 10% difference for most procedures, with the use of an  of .05 and 80% power.
Patients older than 21 years and born in the month of June were selected from the Tri-County billing computer and then separated by office. The sample size was adjusted for each office on the basis of the relative office size. As in the previous study, patients seen only in the emergency department or living in a group home were excluded from the study. Patients without a computer-generated provider reminder attached to their chart or whose chart could not be located on 2 separate days were also excluded. In total, 81 patients were excluded.
Manual chart audits were performed on a total of 271 charts. As in the previous study, a provider was considered compliant with a procedure if the procedure was addressed (ordered, completed, offered but refused, noted that it was done elsewhere, or noted as not needed for that patient) within 3 months of the recommended interval. Information was gathered from completed computer-generated provider reminders, progress notes in the chart, laboratory slips, or correspondence from outside providers reporting on procedures.
DATA ANALYSIS
Characteristics of the 1992 and 1996 patient populations, with respect to marriage, sex, and insurance status, were compared with  2 tests. Overall provider compliance for an individual patient was calculated by dividing the number of times procedures were addressed for that patient by the number of times they were applicable. The 1992 and 1996 overall provider compliance rates were calculated on the basis of only procedures that were in both protocols. Overall provider compliance excluding determination of blood pressure and weight was also calculated for both data sets in the manner described above. This was done because determining blood pressure and weight is largely a nurse function that is independent of provider compliance. We did not want a high nurse compliance with determining blood pressure and weight to distort the true picture of overall provider compliance. The 1996 levels of overall provider compliance were ranked and compared with 1992 values, by means of the Wilcoxon rank sum test. This was repeated for overall provider compliance excluding blood pressure and weight.
Provider compliance for each of the 9 individual procedures was calculated by dividing the total number of times the procedure was addressed by the number of patients for whom the procedure was indicated. The 1996 percentage provider compliance for each procedure was then compared with the 1992 result by means of a 2 test.
RESULTS
The population for this study was not significantly different from the original Tri-County study, with 73% (1992) vs 68% (1996) married (P=.09), 45% vs 51% male (P=.10), and 91% vs 94% insured (P=.17). The 1996 overall provider compliance rate with the health maintenance protocol (including blood pressure and weight) was 83%, slightly higher than the 1992 overall provider compliance rate of 80%. This difference was statistically significant (P=.05) but is not a clinically important difference. Excluding blood pressure and weight measurements, which are done by nurses, the 1996 overall provider compliance rate was 75% compared with the 1992 rate of 71%. Although this 4% difference is greater than the 3% difference when blood pressure and weight were included, it did not reach statistical significance (P=.19) because of the smaller sample size.
Provider compliance rates with individual procedures ranged from 45% to 99% (Table 2). Providers were quite compliant with procedures such as tobacco use history, blood pressure, and weight. Procedures such as serum cholesterol level, diphtheria-tetanus booster, and mammography had slightly lower compliance rates. Providers were least compliant with fecal occult blood testing, clinician breast examinations, and Papanicolaou smears.
|
|
|
|
Table 2. Provider Compliance With Specific Preventive Procedures Between 1992 and 1996
|
|
|
The 1996 provider compliance rates were significantly higher than the 1992 rates for blood pressure, weight, and diphtheria-tetanus booster (P<.001). There was no significant difference between 1992 and 1996 provider compliance rates for cholesterol screening, fecal occult blood testing, clinician breast examinations, mammography, and Papanicolaou smears. Interestingly, the 1996 provider compliance rate for obtaining a history of tobacco use was significantly lower than the 1992 rate (P<.001). This may be the result of a change in the health maintenance protocol. In 1992 the criterion was that a history of tobacco use had to have been obtained once at any time, while in 1996 the criterion was that a history of tobacco use had to have been obtained within the past 4 years.
COMMENT
This study demonstrates that the improvements in provider compliance with health maintenance procedures gained by using a computer-based tracking system can be maintained over time in the absence of grant funding or special external interventions. Even excluding nursing-dependent procedures, such as measuring blood pressure and weight, the 1996 overall provider compliance rate was not significantly different from the 1992 rate (which was significantly higher than provider compliance rates when a manual tracking system was used).
The results, however, are not entirely positive. Provider compliance rates for some individual procedures, especially Papanicolaou smears (55%) and fecal occult blood testing for colorectal cancer (52%), are still far from ideal. One of the failures of the HTRAK software was that it did not have a meaningful individual provider compliance report that could regularly inform providers of their compliance with specific procedures and be used to compare and improve their behavior. We had initially created such a report, but it looked at all patients in the system and reported the percentage who were overdue for a specific procedure. The problem with this report was that many of these patients had not been to the office in several years, some had moved, some were acute care patients from out of the area, a few were deceased or in nursing homes, and some just did not respond to the mailed reminders. The reports generated gave obviously low provider compliance rates that had no value for provider reinforcement. A proper report of provider compliance would give the percentage of patients seen by the provider within a specified period who were overdue for a particular procedure. Thus, the report would include only patients for whom the provider had had the opportunity to offer health maintenance.
Limitations of this study include the usual limitations of a manual chart audit method as well as the fact that the data come from a practice that includes 1 physician (P.S.F.) who has had a strong interest in prevention during the past 23 years. We specifically excluded the office in which this physician practices, but his leadership may certainly affect the behavior of the other providers.
Manual chart audit methods always run the risk that providers have done procedures and not documented them properly. To give providers credit for any procedures performed, we examined the whole chart, including hospital discharge summaries and consultant reports, and not just the computer-generated health maintenance sheet. This does not help if there is no documentation at all, however. The relatively low compliance with clinician breast examination (45%), which contrasts with the higher mammography compliance (67%), could be explained by providers not documenting breast examinations, which are frequently done in conjunction with ordering a mammogram.
This study has shown that computer-based health maintenance tracking is a valuable tool that can result in sustained improvements in provider compliance with preventive procedures. As pointed out by Dietrich et al,4 however, tools alone do not make a system that will necessarily result in optimal provider and patient health maintenance compliance. A system is a process that includes defined responsibilities for its participants, feedback to measure progress, and reinforcements to motivate compliance. Above all, a system must have an "owner," a person or institution vested in ensuring that the process is carried out in a systematic, ongoing manner.
Systems cost money. In the 1992 study we determined it cost $0.78 per patient per year to maintain the computer-based tracking system.6 These costs were not recovered through increased patient revenues. In the current climate of cost containment and decreasing reimbursement rates, practices will have to have a philosophical and financial commitment to quality care before they are likely to implement effective prevention.
Ideally, provider health maintenance compliance should be 90% to 100%, not the 50% to 80% found in this study. What can be done to further improve compliance with preventive protocols? The increased use of nurses or other office staff is one possible option. Harris and colleagues3 showed that a nurse-based system could be effective, although less effective than the computer-based system. In this study, 2 procedures done by nurses, blood pressure and weight determination, had the highest compliance. It is clear that nurses and other staff need to be involved in the health maintenance system and can implement effectively some or all of the health maintenance protocol. Practices should not, however, delude themselves into thinking this can be done without additional costs. If staff are to be expected to perform health maintenance, they must have dedicated time to do so and positive reinforcement for their efforts. Practices that simply dump health maintenance into the job description of already busy staff will be disappointed with the results.
Computer-based tools for tracking and feedback are improving but have not yet reached the level of sophistication needed to provide practitioners a complete, user-friendly "off-the-shelf" product that is readily available. Even at Tri-County Family Medicine, with its long history of innovation in this area, a complete system including routine feedback and meaningful reinforcements to improve compliance is not fully established. We are currently installing a new system that we hope will be more efficient and provide better feedback.
We believe major improvements in provider compliance beyond the 50% to 80% range will not be achieved by systems and peer pressure alone without the addition of external reinforcements, either financial or relating to privileges and credentialing. The natural reinforcements to do the right thing that apply in treating acute illness do not happen with preventive procedures. If a physician sees a patient with pneumonia and does not prescribe appropriate antibiotics, immediate negative feedback occurs. The patient gets worse or perhaps even needs hospitalization. If a physician omits appropriate preventive procedures, there will be no immediate negative feedback in most cases. The negative consequences are delayed and infrequent. There may be, in fact, immediate positive reinforcements, such as having more time to see a few more patients or to leave the office earlier. The report cards created by the National Committee on Quality Assurance using the Health Employer Data Information Set have demonstrated the dramatic effect external reinforcements can have on the prevention orientation of managed care organizations. The British National Health Service has effectively used financial incentives to improve immunization compliance.9 It is appropriate to provide practitioners with direct financial incentives for high levels of preventive care.
Increased patient involvement is an effective way of improving provider (and patient) health maintenance compliance. In the Tri-County system, every patient receives a mailed, individualized, yearly health maintenance status report regardless of whether he or she had an office visit. Patients are instructed to discuss these reports with their physicians. When they do, it provides a strong incentive for the provider to offer appropriate health maintenance. There is no reason insurers and managed care organizations could not offer patients financial incentives for complying with preventive recommendations and living healthy lifestyles.
This study has shown that provider health maintenance compliance can be maintained over time without grant funding. More studies of preventive interventions need to report long-term real-world results so that the most effective, and cost-effective, strategies can be determined. Tools for improving provider compliance with health maintenance procedures are becoming available. To provide the best preventive care possible, we must continue to develop and improve the delivery systems.
AUTHOR INFORMATION
Accepted for publication February 11, 1998.
Corresponding author: Paul S. Frame, MD, Tri-County Family Medicine, 25 Park Ave, Cohocton, NY 14826 (e-mail: pframe/2061890{at}mcimail.com).
From Tri-County Family Medicine, Cohocton, NY (Dr Frame); and Departments of Family Medicine (Dr Frame) and Biostatistics (Ms Eberly), University of Rochester School of Medicine and Dentistry, Rochester, NY. Ms Cooley is a medical student at the University of Rochester School of Medicine and Dentistry.
REFERENCES
| |
1. Ornstein SM, Garr DR, Jenkins RG, Rust PF, Zemp L, Arnon A. Compliance with five health promotion recommendations in a university-based family practice. J Fam Pract. 1989;29:163-168.
ISI
| PUBMED
2. Woo B, Woo B, Cook EF, Weisberg M, Goldman L. Screening procedures in the asymptomatic adult: comparison of physicians' recommendations, patients' desires, published guidelines, and actual practice. JAMA. 1985;254:1480-1484.
FREE FULL TEXT
3. Harris RP, O'Malley MS, Fletcher SW, Knight BS. Prompting physicians for preventive procedures: a five-year study of manual and computer reminders. Am J Prev Med. 1990;6:145-152.
ISI
| PUBMED
4. Dietrich AJ, O'Connor GT, Keller A, Carney PA, Levy D, Whaley FS. Cancer: improving early detection and prevention: a community practice randomised trial. BMJ. 1992;304:687-691.
5. McPhee SJ, Bird JA, Fordham D, Rodnick JE, Osborn EH. Promoting cancer prevention activities by primary care physicians: results of a randomized, controlled trial. JAMA. 1991;266:538-544.
FREE FULL TEXT
6. Frame PS, Zimmer JG, Werth PL, Hall WJ, Eberly SW. Computer-based vs manual health maintenance tracking: a controlled trial. Arch Fam Med. 1994;3:581-588.
FREE FULL TEXT
7. Dietrich AJ, Sox CH, Tosteson TD, Woodruff CB. Durability of improved physician early detection of cancer after conclusion of intervention support. Cancer Epidemiol Biomarkers Prev. 1994;3:335-340.
ABSTRACT
8. Frame PS, Zimmer JG, Werth PL, Martens WB. Description of a computerized health maintenance tracking system for primary care practice. Am J Prev Med. 1991;7:311-318.
ISI
| PUBMED
9. Durch JS. Overcoming Barriers to Immunization. Washington, DC: Institute of Medicine, National Academy Press; 1994:38.
RELATED ARTICLE
Tools to Improve Documentation of Smoking Status: Continuous Quality Improvement and Electronic Medical Records
Elizabeth Spencer, Todd Swanson, William J. Hueston, and Debora L. Edberg
Arch Fam Med. 1999;8(1):18-22.
ABSTRACT
| FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
Improving prevention in primary care: Evaluating the sustainability of outreach facilitation
Hogg et al.
cfp 2008;54:712-720.
ABSTRACT
| FULL TEXT
A Randomized Controlled Trial to Increase Cancer Screening Among Attendees of Community Health Centers
Roetzheim et al.
Ann Fam Med 2004;2:294-300.
ABSTRACT
| FULL TEXT
A Framework for Improving the Quality of Cancer Care: The Case of Breast and Cervical Cancer Screening
Zapka et al.
Cancer Epidemiol. Biomarkers Prev. 2003;12:4-13.
ABSTRACT
| FULL TEXT
|
