Abstract

Few data exist on the long-term prognosis of patients with chronic primary chronic immune thrombocytopenia (ITP). We examined the risk of infections, hemorrhage resulting in hospitalization, hematologic malignancies, and total and cause-specific mortality among patients with ITP compared with the general population. We used population-based medical databases to identify 407 patients with primary chronic ITP diagnosed during 1996 to 2007 and 4069 general population members individually matched on age, sex, and comorbidity level. We used Cox regression analysis to estimate rate ratios (RRs) adjusted for age (≤ 60 or > 60 years), sex, calendar year, and level of comorbidity. The adjusted 1-year RR of infection was 4.5 (95% confidence interval [CI], 3.3-6.1) for patients with chronic ITP compared with the general population cohort. The adjusted RR decreased to 1.8 (95% CI, 1.3-2.5) for the second to fifth year of follow-up. The adjusted 5-year RR was 3.2 (95% CI, 1.2-9.0) for hospitalized intracranial hemorrhage, 4.4 (95% CI, 2.3-8.3) for other hospitalized hemorrhages, and 4.7 (95% CI, 1.7-12.7) for hematologic malignancy. The 5-year all-cause mortality RR was 2.3 (95% CI, 1.8-3.0). In summary, primary chronic ITP was associated with substantially increased long-term risk of infections, hemorrhagic episodes requiring hospitalization, hematologic malignancies, and mortality.

Introduction

Primary immune thrombocytopenia (ITP) is an autoimmune disorder characterized by immune-mediated platelet destruction and suppressed platelet production.1  In adults, ITP is most often a chronic disease with insidious onset.2  Until recently, a diagnosis of chronic ITP required persistence of thrombocytopenia for ≥ 6 months after diagnosis. This definition has now been changed to persistence of thrombocytopenia for ≥ 12 months after diagnosis.3 

Many patients with ITP present with either no symptoms or minimal bruising, but some experience serious bleeding.4  The prognosis of ITP is considered to be determined mainly by the occurrence of spontaneous bleeding. In 2000, Cohen et al conducted a pooled analyses that was based on 17 case series encompassing a total of 1817 patients with ITP5 ; the risk of fatal hemorrhage was estimated to range from 0.4% per year for patients < 40 years to 13% per year for patients ≥ 60 years.

Patients with chronic ITP also may experience complications other than bleeding. Treatment with immunosuppressive drugs may increase the risk of infections and hematologic malignancies.6  A study conducted with the United Kingdom General Practitioners Research Database on 1033 patients with an ITP diagnosis and a matched comparison group suggested that ITP was associated with a large number of medical conditions both before and after ITP diagnosis.7  However, that study included patients with a single recorded diagnosis of ITP. As yet, few data exist on the long-term prognosis of patients with chronic ITP. A Dutch cohort study that followed 152 adult patients with chronic ITP for a median of 9.4 years found a 30% higher mortality rate among the patients with ITP relative to the general population.8  Patients who died within 2 years after ITP diagnosis succumbed to hemorrhage or infection.8  A recent Danish cohort study found that patients with ITP who had undergone splenectomy were at increased risk of a hospital contact with infection more than a year after their splenectomy, compared with both patients with ITP not undergoing splenectomy (hazard ratio, 1.4; 95% confidence interval [CI], 1.0-2.0) and the general population (hazard ratio, 4.0; 95% CI, 2.8-5.6).9  In a cohort study of 3131 patients with chronic ITP and 9392 comparison patients identified through an administrative US claims database and followed for a median of 1.5 and 1.6 years, respectively, Enger et al found a 6-fold increased risk of lymphoma, a 20-fold increased risk of leukemia, and a 4-fold increased mortality rate among the patients with ITP.10 

To explore long-term clinical outcomes among patients with chronic ITP, we examined rates of infections, bleeding episodes resulting in hospitalization, incident hematologic malignancies, and total and cause-specific mortality among Danish patients with primary chronic ITP diagnosed from January 1, 1996, through July 31, 2007, and compared these rates with those of a matched comparison cohort drawn from the Danish general population.

Methods

This population-based cohort study was set in Denmark, with ∼ 5.3 million inhabitants. The Danish National Health Service provides tax-supported health care, including free access to hospital care, to the entire population. Patients with ITP in Denmark are usually diagnosed, treated, and followed by specialist physicians at a few public hospital-based hematology centers operating under the auspices of the Danish National Health Service. Since 1968, the Danish Civil Registration System (CRS) has kept electronic records, updated daily, on date of birth, sex, change of address, date of emigration, and changes in vital status for all Danish residents. A unique 10-digit civil registry number assigned to each resident11  permits linkage to national data on hospital diagnoses from the Danish National Registry of Patients (DNRP).

Study cohort

We used the DNRP to identify all patients ≥ 18 years with a hospital-based (inpatient or outpatient) diagnosis of ITP between January 1, 1996, and July 31, 2007. The DNRP, established in 1977, covers > 99% of all patients admitted to nonpsychiatric hospitals in Denmark.12  Since 1995, outpatient and emergency department visits also have been included in the registry. For each inpatient or outpatient hospital contact, DNRP files include dates of admission and discharge, surgical procedure(s) performed, and ≤ 20 discharge diagnoses. Since 1994, diagnoses have been coded according to the 10th revision of the International Classification of Diseases (ICD-10).12  Patients with primary ITP were identified as those with the ICD-10 diagnosis code D69.3 (ie, primary ITP). We limited our study cohort to patients with chronic ITP, defined as ≥ 2 hospital-based diagnoses of ITP (including inpatient hospitalizations and outpatient contacts) over a period of ≥ 6 months. We defined the date of chronic ITP diagnosis as the date of the first ITP-related inpatient or outpatient hospital contact occurring > 6 months after the first ITP-related hospital contact.

We identified 2908 patients aged ≥ 18 years registered in the DNRP with the relevant ICD-10 code. Of these patients, 513 (17.6%) fulfilled the criteria for chronic ITP. We were able to obtain medical records for 439 (86.6%) of the identified patients with chronic ITP. We evaluated each patient's medical record and found that 408 patients (92.9%) had thrombocytopenia, defined as a platelet count < 150 × 109/L without a secondary cause and receipt of medications or medical services for ITP. We excluded 1 patient who underwent splenectomy before the date of ITP diagnosis in the DNRP, who therefore may have had an earlier occurrence of ITP. In total, 407 patients were considered to have a diagnosis of active chronic ITP.

Population comparison cohort and mortality data

For each patient with primary chronic ITP, we identified ≤ 10 comparison cohort members from the CRS, matched on age, sex, comorbidity (see next paragraph), and calendar year. Up-to-date information on mortality and migration was obtained from CRS files for both the patients with chronic ITP and the comparison cohort.13 

Causes of death were retrieved from the Danish Mortality Files,14  which include data on underlying and immediate causes of death for all deaths among residents of Denmark since 1943. The underlying cause of death reported on the death certificates is grouped into 49 National Board of Health categories that are based on ICD-10 codes.14  For our analysis, we included the following 5 categories of causes of death: cardiovascular diseases other than intracranial hemorrhage, hematologic cancers, all other cancers, infection, and hemorrhagic episodes. The ICD-10 groupings are presented in supplemental Appendix 1 (available on the Blood Web site; see the Supplemental Materials link at the top of the online article).

Patients with chronic ITP and clinical characteristics

We used hospital records to extract information on each patient's lowest platelet count during follow-up and the last hospital admission date for chronic ITP. We classified patients with chronic ITP into 2 groups on the basis of their lowest recorded platelet count during follow-up: < 30 × 109/L or ≥ 30 × 109/L.

Through linkage to the DNRP we identified all hospital contacts (ie, admissions or hospital outpatient visits) with a diagnosis of infection, intracranial hemorrhage, or non–intracranial hemorrhagic episodes for the patients with chronic ITP and their population comparisons after their index date (defined as the date of chronic ITP diagnosis for each patient with ITP and the same date for his or her matched population comparison).

We assessed comorbidity with the use of the Charlson Comorbidity Index (CCI), a validated measure developed to predict 1-year mortality with the use of comorbidity data.15  We calculated CCI scores on the basis of all hospital diagnoses recorded at hospital admission or hospital outpatient visits from the start of the DNRP on January 1, 1977, up to the index date. We categorized comorbidities as “none” (CCI score = 0, corresponding to no recorded underlying disease) or “any” (CCI score ≥ 1).

Data on hematologic malignancies

We obtained information on hematologic malignancies diagnosed among patients with primary chronic ITP and members of the comparison cohort through linkage with the Danish Cancer Registry (DCR). The DCR has collected information on cases of primary cancer on a nationwide basis since 1943 and is currently updated through 2008. The completeness and validity of the DCR has been found to be on the order of 95%-98%.16  DCR files include information on cancer type, site, morphology, and cancer history. Since 1978, cancers have been coded according to the ICD-10 and the third version of the International Classification of Diseases for Oncology (SNOMED). The ICD-10 groupings and SNOMED codes used to identify hematologic malignancies are presented in supplemental Appendix 2. We additionally identified patients with myelodysplastic syndrome (ICD-10 code D46) but did not include this as a malignancy.

Statistical analyses

We estimated the cumulative rates (%) of hospital contacts (inpatient or outpatient) for infections for the first year of follow-up and for the time period of 366 days to 5 years of follow-up. We estimated the 5-year cumulative rate of hemorrhagic episodes requiring hospitalization and hematologic malignancies among patients with chronic ITP, treating death as a competing event.17  We also estimated 5-year all-cause mortality among patients with chronic ITP. All analyses were stratified by age (≤ 60 or > 60 years), sex, and presence of comorbidity at the time of chronic ITP diagnosis. Because diagnostic methods and treatments may have changed over time, we also stratified analyses by the calendar period of chronic ITP diagnosis (1996-2001 and 2002-2007).

We used stratified Cox proportional hazards regression to compare the rates of inpatient or outpatient contacts for infection, hemorrhagic episodes requiring hospitalization, hematologic malignancies, and mortality (all-cause and cause-specific) among patients with chronic ITP and members of the comparison cohort. We estimated rate ratios (RRs), mortality rate ratios (MRRs), and associated 95% CIs. Follow-up began on the index date and ended on the date of death or April 1, 2008. We analyzed each endpoint separately. The RRs were adjusted for age (≤ 60 or > 60 years), sex, calendar year, and level of comorbidity. The assumption of proportional hazards was satisfied in all models.

This study was approved by the Danish Data Protection Agency (Record no. 2007-41-1101). The statistical software package SAS Version 9.1 (SAS Institute Inc), was used for all statistical analyses.

Results

We identified 407 patients with chronic ITP with a median follow-up of 4.6 years (range, 0-11.5 years). At diagnosis, 225 patients (55.3%) were ≤ 60 years old and 254 (62.4%) were women. At diagnosis, 222 patients (54.5%) had a platelet count < 30 × 109/L, 76 (18.7%) had a count between 30 and 49 × 109/L, 92 (22.6%) had a count between 50 and 149 × 109/L, 8 (2%) had a count of ≥ 150 × 109/L, and 9 (2%) had missing platelet counts. An additional 85 patients (20.8%) had ≥ 1 recorded platelet count < 30 × 109/L during follow-up. In 60.7% of the patients with chronic ITP (n = 247) no previous hospital contact for any comorbidity was recorded. Of the 407 patients, 36 had a splenectomy before their chronic ITP diagnosis and 86 had a splenectomy during follow-up.

For the comparison cohort we identified 4069 members for whom median follow-up of 4.9 years (range, 0-11.6 years) was available.

Hospital contacts for infections

Among patients with chronic ITP, 62 (15.2%) had ≥ 1 hospital contact (admission or visit to an outpatient clinic) involving an infection during the first year of follow-up, compared with 178 (4.4%) persons in the comparison cohort. The adjusted 1-year RR of infection, comparing patients with chronic ITP with members of the comparison cohort, was 4.5 (95% CI, 3.3-6.1). In the second to fifth year of follow-up, 52 patients with ITP (13%) had a hospital contact with an infection compared with 370 members of the comparison cohort (9.5%). The adjusted RR for the second to fifth year of follow-up decreased to 1.8 (95% CI, 1.3-2.5). Of the 6 patients with a diagnosis of sepsis during follow-up, 2 had undergone splenectomy.

The 1-year cumulative rate of infections was higher among patients with chronic ITP who were > 60 years, men, diagnosed with chronic ITP in 2002-2007, or had ≥ 1 recorded comorbidity at diagnosis (Table 1). The 5-year cumulative rates of infection were similar in patients with chronic ITP whose lowest platelet count was < 30 × 109/L (32.5%; 95% CI, 26.9-38.2) and those with platelet counts ≥ 30 × 109/L (26.5%; 95% CI, 17.6-36.2). Table 2 presents the different infection types occurring in patients with chronic ITP and the comparison cohort. Although patients with chronic ITP in general had a higher risk of infection, they had a lower rate of sexual, intra-abdominal, and urogenital infections than members of the comparison cohort.

Table 1

One-year cumulative risk and rate ratio of a hospital contact (admission or hospital outpatient visit) involving infection among 407 patients with cITP and a comparison cohort of 4069 members of the general population in Denmark

Patients with cITP, n (%)One-year risk among patients with cITP, % (95% CI)One-year risk among the comparison cohort, % (95% CI)Crude 1-year RR (95% CI)Adjusted 1-year RR (95% CI)*
Total 407 (100) 15.3 (12.0-18.9) 4.4 (3.8-5.1) 3.9 (2.8-5.2) 4.5 (3.3-6.1) 
Age at cITP diagnosis, y      
    ≤ 60 225 (55) 8.9 (5.6-13.1) 2.4 (1.8-3.1) 3.9 (2.3-6.5) 4.0 (2.4-6.8) 
    > 60 182 (45) 23.1 (17.3-29.5) 6.9 (5.8-8.1) 4.1 (2.9-5.8) 4.8 (3.3-6.9) 
Sex      
    Female 254 (62) 13.4 (9.5-17.9) 4.3 (3.5-5.1) 3.4 (2.3-5.0) 3.8 (2.5-5.8) 
    Male 153 (38) 18.3 (12.7-24.8) 4.6 (3.7-5.8) 4.7 (3.0-7.5) 5.8 (3.5-9.6) 
Year of cITP diagnosis      
    1996-2001 167 (41) 10.2 (6.2-15.3) 3.2 (2.4-4.1) 3.4 (2.0-5.9) 3.8 (2.2-6.7) 
    2002-2007 240 (59) 18.8 (14.1-24.0) 5.3 (4.4-6.2) 4.1 (2.9-5.8) 4.9 (3.4-6.9) 
Comorbidity score      
    0 247 (61) 8.5 (5.5-12.4) 2.2 (1.6-2.8) 4.2 (2.5-7.0) 4.3 (2.6-7.1) 
    ≥ 1 160 (39) 25.7 (19.2-32.6) 7.9 (6.6-9.2) 4.0 (2.8-5.6) 4.7 (3.2-6.8) 
Patients with cITP, n (%)One-year risk among patients with cITP, % (95% CI)One-year risk among the comparison cohort, % (95% CI)Crude 1-year RR (95% CI)Adjusted 1-year RR (95% CI)*
Total 407 (100) 15.3 (12.0-18.9) 4.4 (3.8-5.1) 3.9 (2.8-5.2) 4.5 (3.3-6.1) 
Age at cITP diagnosis, y      
    ≤ 60 225 (55) 8.9 (5.6-13.1) 2.4 (1.8-3.1) 3.9 (2.3-6.5) 4.0 (2.4-6.8) 
    > 60 182 (45) 23.1 (17.3-29.5) 6.9 (5.8-8.1) 4.1 (2.9-5.8) 4.8 (3.3-6.9) 
Sex      
    Female 254 (62) 13.4 (9.5-17.9) 4.3 (3.5-5.1) 3.4 (2.3-5.0) 3.8 (2.5-5.8) 
    Male 153 (38) 18.3 (12.7-24.8) 4.6 (3.7-5.8) 4.7 (3.0-7.5) 5.8 (3.5-9.6) 
Year of cITP diagnosis      
    1996-2001 167 (41) 10.2 (6.2-15.3) 3.2 (2.4-4.1) 3.4 (2.0-5.9) 3.8 (2.2-6.7) 
    2002-2007 240 (59) 18.8 (14.1-24.0) 5.3 (4.4-6.2) 4.1 (2.9-5.8) 4.9 (3.4-6.9) 
Comorbidity score      
    0 247 (61) 8.5 (5.5-12.4) 2.2 (1.6-2.8) 4.2 (2.5-7.0) 4.3 (2.6-7.1) 
    ≥ 1 160 (39) 25.7 (19.2-32.6) 7.9 (6.6-9.2) 4.0 (2.8-5.6) 4.7 (3.2-6.8) 

cITP indicates chronic immune thrombocytopenia; CI, confidence interval; and RR, rate ratio.

*

Adjusted for age (≤ 60 y and > 60 y), sex, calendar year, and level of comorbidity.

Table 2

Type of infections observed within the first year of follow-up among 62 of 407 patients (15%) with cITP and among 178 of 4069 (4%) members of the general population comparison cohort

Type of infectionPatients with cITP (n = 62), n (%)Comparison cohort (n = 178), n (%)
CNS infections (except meningococcal disease) 1 (0.6) 
Candidiasis and other fungal infections 1 (1.6) 4 (2.3) 
Female pelvic infections (salpingo-oophoritis, uterine infection) 7 (3.9) 
Gastrointestinal infection 4 (6.5) 13 (7.3) 
HIV 1 (1.6) 1 (0.6) 
Heart infections (acute rheumatic fever, infectious pericarditis, myocarditis, or endocarditis) 1 (1.6) 
Influenza 1 (1.6) 1 (0.6) 
Intra-abdominal infections 1 (1.6) 14 (7.9) 
Male genital infections (prostatitis, orchitis, epididymitis) 3 (1.7) 
Miscellaneous bacterial infections 1 (1.6) 3 (1.7) 
Miscellaneous viral infections 3 (4.8) 2 (1.1) 
Other lower respiratory tract infections 3 (4.8) 3 (1.7) 
Pneumonia 23 (37.1) 60 (33.7) 
Septic arthritis, osteomyelitis 1 (0.6) 
Septicemia 7 (11.3) 8 (4.5) 
Sexually transmitted diseases 3 (1.7) 
Skin infections, including cellulitis/erysipelas 4 (6.5) 14 (7.9) 
Tuberculosis 1 (0.6) 
Upper respiratory tract infection 3 (4.8) 4 (2.3) 
Urinary tract infections 6 (9.7) 30 (16.9) 
Other infections or sequelae of infections 3 (4.8) 5 (2.8) 
Type of infectionPatients with cITP (n = 62), n (%)Comparison cohort (n = 178), n (%)
CNS infections (except meningococcal disease) 1 (0.6) 
Candidiasis and other fungal infections 1 (1.6) 4 (2.3) 
Female pelvic infections (salpingo-oophoritis, uterine infection) 7 (3.9) 
Gastrointestinal infection 4 (6.5) 13 (7.3) 
HIV 1 (1.6) 1 (0.6) 
Heart infections (acute rheumatic fever, infectious pericarditis, myocarditis, or endocarditis) 1 (1.6) 
Influenza 1 (1.6) 1 (0.6) 
Intra-abdominal infections 1 (1.6) 14 (7.9) 
Male genital infections (prostatitis, orchitis, epididymitis) 3 (1.7) 
Miscellaneous bacterial infections 1 (1.6) 3 (1.7) 
Miscellaneous viral infections 3 (4.8) 2 (1.1) 
Other lower respiratory tract infections 3 (4.8) 3 (1.7) 
Pneumonia 23 (37.1) 60 (33.7) 
Septic arthritis, osteomyelitis 1 (0.6) 
Septicemia 7 (11.3) 8 (4.5) 
Sexually transmitted diseases 3 (1.7) 
Skin infections, including cellulitis/erysipelas 4 (6.5) 14 (7.9) 
Tuberculosis 1 (0.6) 
Upper respiratory tract infection 3 (4.8) 4 (2.3) 
Urinary tract infections 6 (9.7) 30 (16.9) 
Other infections or sequelae of infections 3 (4.8) 5 (2.8) 

cITP indicates chronic immune thrombocytopenia; and CNS, central nervous system.

Bleeding episodes requiring hospitalization

During 5 years of follow-up, 5 patients with chronic ITP were hospitalized with an intracranial hemorrhage. Each of these patients had a lowest platelet count < 30 × 109/L. The 5-year cumulative rate of intracranial hemorrhage was 1.4% (95% CI, 0.5-3.9) among patients with chronic ITP and 0.6% (95% CI, 0.3-0.9) among members of the comparison cohort. The corresponding adjusted 5-year RR was 3.2 (95% CI, 1.2-9.0). Patients with chronic ITP > 60 years and those who had a recorded comorbidity had the highest 5-year cumulative rate of intracranial hemorrhage (Table 3).

Table 3

Five-year cumulative risk and rate ratios of a hospitalized intracranial bleeding episode among 407 patients with cITP and 4069 members of the general population comparison cohort

Five-year risk among patients with cITP, % (95% CI)Five-year risk in the comparison cohort, % (95% CI)Crude 5-year RR (95% CI)Adjusted 5-year RR (95% CI)*
Total 1.4 (0.5-3.2) 0.6 (0.3-0.9) 2.8 (1.0-7.5) 3.2 (1.2-9.0) 
Age at cITP diagnosis, y     
    ≤ 60 0.4 (0.0-2.3) 0.1 (0.0-0.3) 5.1 (0.5-56.0) 5.0 (0.5-55.1) 
    > 60 2.6 (0.8-6.1) 1.1 (0.7-1.8) 2.7 (0.9-7.9) 3.0 (1.0-9.2) 
Sex     
    Female 1.9 (0.6-4.5) 0.3 (0.2-0.7) 6.0 (1.8-20.4) 12.3 (2.8-55.2) 
    Male 0.7 (0.1-3.3) 0.9 (0.5-1.8) 0.9 (0.1-6.9) 0.8 (0.1-6.3) 
Year of cITP diagnosis     
    1996-2001 1.2 (0.2-3.9) 0.3 (0.1-0.7) 4.2 (0.8-21.6) 9.2 (1.3-65.5) 
    2002-2007 1.7 (0.4-4.8) 0.8 (0.4-1.3) 2.3 (0.7-8.0) 2.2 (0.6-8.0) 
Comorbidity score     
    0 0.9 (0.2-3.2) 0.04 (0.0-0.3) 20.3 (1.8-223.7) 18.6 (1.7-205.7) 
    ≥ 1 2.3 (0.6-6.1) 1.4 (0.8-2.2) 1.9 (0.6-6.5) 2.1 (0.6-7.3) 
Five-year risk among patients with cITP, % (95% CI)Five-year risk in the comparison cohort, % (95% CI)Crude 5-year RR (95% CI)Adjusted 5-year RR (95% CI)*
Total 1.4 (0.5-3.2) 0.6 (0.3-0.9) 2.8 (1.0-7.5) 3.2 (1.2-9.0) 
Age at cITP diagnosis, y     
    ≤ 60 0.4 (0.0-2.3) 0.1 (0.0-0.3) 5.1 (0.5-56.0) 5.0 (0.5-55.1) 
    > 60 2.6 (0.8-6.1) 1.1 (0.7-1.8) 2.7 (0.9-7.9) 3.0 (1.0-9.2) 
Sex     
    Female 1.9 (0.6-4.5) 0.3 (0.2-0.7) 6.0 (1.8-20.4) 12.3 (2.8-55.2) 
    Male 0.7 (0.1-3.3) 0.9 (0.5-1.8) 0.9 (0.1-6.9) 0.8 (0.1-6.3) 
Year of cITP diagnosis     
    1996-2001 1.2 (0.2-3.9) 0.3 (0.1-0.7) 4.2 (0.8-21.6) 9.2 (1.3-65.5) 
    2002-2007 1.7 (0.4-4.8) 0.8 (0.4-1.3) 2.3 (0.7-8.0) 2.2 (0.6-8.0) 
Comorbidity score     
    0 0.9 (0.2-3.2) 0.04 (0.0-0.3) 20.3 (1.8-223.7) 18.6 (1.7-205.7) 
    ≥ 1 2.3 (0.6-6.1) 1.4 (0.8-2.2) 1.9 (0.6-6.5) 2.1 (0.6-7.3) 

cITP indicates chronic immune thrombocytopenia; CI, confidence interval; and RR, rate ratio.

*

Adjusted for age (≤ 60 y and > 60 y), sex, calendar year, and level of comorbidity.

Fourteen patients with chronic ITP were hospitalized for a bleeding episode other than an intracranial hemorrhage during the first 5 years of follow-up. Of these patients, 12 had a lowest platelet count < 30 × 109/L. The 5-year cumulative rate of a hospitalized bleeding episode other than intracranial bleeding was 3.6% (95% CI, 2.1-5.8) among patients with chronic ITP and 1.1% (95% CI, 0.8-1.6) among members of the comparison cohort. The 5-year adjusted RR was 4.4 (95% CI, 2.3-8.3; Table 4). Patients whose lowest platelet count was < 30 × 109/L had a cumulative rate of hospitalized bleeding episodes other than intracranial bleeding of 4.1% (95% CI, 2.2%-6.9%), compared with 2.1% (95% CI, 0.4%-6.7%) among those whose lowest platelet count was > 30 × 109/L.

Table 4

Five-year cumulative risk and rate ratio of a hospitalized hemorrhage other than intracranial bleeding among 407 patients with cITP and 4069 members of the general population comparison cohort

Five-year risk in patients with cITP, % (95% CI)Five-year risk in the comparison cohort, % (95% CI)Crude 5-year RR (95% CI)Adjusted 5-year RR (95% CI)*
Total 3.6 (2.1-5.8) 1.1 (0.8-1.6) 3.8 (2.0-6.9) 4.4 (2.3-8.3) 
Age at cITP diagnosis, y     
    ≤ 60 3.4 (1.5-6.6) 0.3 (0.1-0.6) 14.4 (4.6-45.2) 13.8 (4.4-43.4) 
    > 60 3.9 (1.7-7.4) 2.3 (1.6-3.1) 2.3 (1.0-5.2) 2.6 (1.1-6.0) 
Sex     
    Female 3.3 (1.5-6.1) 0.9 (0.6-1.4) 4.3 (1.9-9.6) 6.0 (2.5-14.8) 
    Male 4.2 (1.7-8.4) 1.5 (1.0-2.4) 3.3 (1.3-8.2) 3.2 (1.3-8.2) 
Year of cITP diagnosis     
    1996-2001 2.4 (0.8-5.6) 0.8 (0.5-1.4) 3.0 (1.0-9.1) 3.5 (1.1-11.3) 
    2002-2007 4.4 (2.2-7.6) 1.4 (0.9-2.1) 4.2 (2.0-8.8) 4.9 (2.3-10.4) 
Comorbidity score     
    0 2.6 (1.1-5.2) 0.5 (0.3-0.9) 5.6 (2.1-15.2) 5.8 (2.1-16.0) 
    ≥ 1 5.2 (2.4-9.5) 2.2 (1.5-3.0) 3.2 (1.5-7.0) 3.7 (1.6-8.4) 
Five-year risk in patients with cITP, % (95% CI)Five-year risk in the comparison cohort, % (95% CI)Crude 5-year RR (95% CI)Adjusted 5-year RR (95% CI)*
Total 3.6 (2.1-5.8) 1.1 (0.8-1.6) 3.8 (2.0-6.9) 4.4 (2.3-8.3) 
Age at cITP diagnosis, y     
    ≤ 60 3.4 (1.5-6.6) 0.3 (0.1-0.6) 14.4 (4.6-45.2) 13.8 (4.4-43.4) 
    > 60 3.9 (1.7-7.4) 2.3 (1.6-3.1) 2.3 (1.0-5.2) 2.6 (1.1-6.0) 
Sex     
    Female 3.3 (1.5-6.1) 0.9 (0.6-1.4) 4.3 (1.9-9.6) 6.0 (2.5-14.8) 
    Male 4.2 (1.7-8.4) 1.5 (1.0-2.4) 3.3 (1.3-8.2) 3.2 (1.3-8.2) 
Year of cITP diagnosis     
    1996-2001 2.4 (0.8-5.6) 0.8 (0.5-1.4) 3.0 (1.0-9.1) 3.5 (1.1-11.3) 
    2002-2007 4.4 (2.2-7.6) 1.4 (0.9-2.1) 4.2 (2.0-8.8) 4.9 (2.3-10.4) 
Comorbidity score     
    0 2.6 (1.1-5.2) 0.5 (0.3-0.9) 5.6 (2.1-15.2) 5.8 (2.1-16.0) 
    ≥ 1 5.2 (2.4-9.5) 2.2 (1.5-3.0) 3.2 (1.5-7.0) 3.7 (1.6-8.4) 

cITP indicates chronic immune thrombocytopenia; CI, confidence interval; and RR, rate ratio.

*

Adjusted for age (≤ 60 years and > 60 years), sex, calendar year, and level of comorbidity.

Hematologic malignancies

During the 5-year follow-up period, 6 patients with chronic ITP had a first-time diagnosis of a hematologic malignancy; of them, 2 patients had non-Hodgkin lymphoma, 2 had acute myeloid leukemia (AML), 1 had an unspecified leukemia, and 1 had a “malignant lymphoma.” This corresponded to a 5-year cumulative rate of 1.8% (95% CI, 0.7%-3.6%) among patients with chronic ITP compared with 0.4% (95% CI, 0.3%-0.7%) among members of the comparison cohort. The adjusted 5-year RR was 4.7 (95% CI, 1.7-12.7). A total of 4 patients with chronic ITP had a diagnosis of myelodysplastic syndrome within 5 years after chronic ITP diagnosis compared with none in the comparison cohort.

One of the patients with chronic ITP who developed AML had a normal bone marrow biopsy 1 year before the AML diagnosis. The second patient with chronic ITP who developed AML had a first bone marrow biopsy 2 years after the chronic ITP diagnosis, with biopsy results showing hematopathologic signs of chronic myeloid leukemia. Six months later the patient underwent transformation to AML. The 3 other patients with chronic ITP who developed a hematologic malignancy all had bone marrow biopsy pathology compatible with ITP. One had missing information.

Among the 6 patients with chronic ITP who developed a hematologic malignancy, 2 had not received any treatment for their ITP, and the remaining 4 all received corticosteroids in combination with azathioprine or cyclophosphamide. One patient also received intravenous immunoglobulin and a splenectomy and another also received rituximab.

Total and cause-specific mortality

A total of 103 patients with chronic ITP died during the entire period of follow-up. The overall mortality rate for patients with chronic ITP was 53.0 (95% CI, 43.5-64.0) per 1000 person-years compared with 29.1 (95% CI, 26.8-31.5) for members of the comparison cohort, corresponding to an overall adjusted MRR of 2.2 (95% CI, 1.8-2.8). Five-year mortality was 24% (95% CI, 20%-29%) for patients with chronic ITP and 14% (95% CI, 13%-15%) for the comparison cohort (Table 5). The adjusted all-cause 5-year MRR was 2.3 (95% CI, 1.8-3.0) for patients with chronic ITP versus the comparison cohort. Table 5 presents 5-year mortality and MRR stratified by demographic variables.

Table 5

All-cause 5-year cumulative mortality and mortality rate ratio among 407 patients with cITP and 4069 members of the general population in Denmark

No. of cITP deathsFive-year mortality in patients with cITP, % (95% CI)Five-year mortality in the comparison cohort, % (95% CI)Crude MRR (95% CI)Adjusted MRR (95% CI)*
All-cause mortality 87 24.3 (20.1-29.2) 14.3 (13.1-15.6) 1.9 (1.5-2.4) 2.3 (1.8-3.0) 
Age at cITP diagnosis, y      
    ≤ 60 11 5.6 (3.1-9.9) 1.7 (1.2-2.4) 3.6 (1.8-7.2) 3.8 (1.9-7.7) 
    > 60 76 47.1 (39.5-55.4) 30.1 (27.8-32.6) 1.9 (1.5-2.5) 2.2 (1.7-2.9) 
Sex      
    Female 47 20.4 (15.7-26.3) 12.2 (10.9-13.7) 1.9 (1.4-2.6) 2.3 (1.7-3.2) 
    Male 40 31.4 (23.8-40.6) 18.1 (15.9-20.4) 2.0 (1.4-2.8) 2.4 (1.7-3.4) 
Year of cITP diagnosis      
    1996-2001 31 18.6 (13.5-25.4) 11.5 (10.1-13.2) 1.7 (1.2-2.5) 2.0 (1.3-2.9) 
    2002-2007 56 29.7 (23.0-37.7) 17.3 (15.4-19.4) 2.1 (1.6-2.8) 2.6 (1.9-3.5) 
Comorbidity score      
    0 26 11.9 (8.2-17.1) 4.9 (4.0-5.9) 2.8 (1.8-4.3) 3.0 (1.9-4.7) 
    ≥ 1 61 44.9 (36.6-54.2) 29.9 (27.3-32.6) 1.8 (1.4-2.4) 2.1 (1.6-2.8) 
No. of cITP deathsFive-year mortality in patients with cITP, % (95% CI)Five-year mortality in the comparison cohort, % (95% CI)Crude MRR (95% CI)Adjusted MRR (95% CI)*
All-cause mortality 87 24.3 (20.1-29.2) 14.3 (13.1-15.6) 1.9 (1.5-2.4) 2.3 (1.8-3.0) 
Age at cITP diagnosis, y      
    ≤ 60 11 5.6 (3.1-9.9) 1.7 (1.2-2.4) 3.6 (1.8-7.2) 3.8 (1.9-7.7) 
    > 60 76 47.1 (39.5-55.4) 30.1 (27.8-32.6) 1.9 (1.5-2.5) 2.2 (1.7-2.9) 
Sex      
    Female 47 20.4 (15.7-26.3) 12.2 (10.9-13.7) 1.9 (1.4-2.6) 2.3 (1.7-3.2) 
    Male 40 31.4 (23.8-40.6) 18.1 (15.9-20.4) 2.0 (1.4-2.8) 2.4 (1.7-3.4) 
Year of cITP diagnosis      
    1996-2001 31 18.6 (13.5-25.4) 11.5 (10.1-13.2) 1.7 (1.2-2.5) 2.0 (1.3-2.9) 
    2002-2007 56 29.7 (23.0-37.7) 17.3 (15.4-19.4) 2.1 (1.6-2.8) 2.6 (1.9-3.5) 
Comorbidity score      
    0 26 11.9 (8.2-17.1) 4.9 (4.0-5.9) 2.8 (1.8-4.3) 3.0 (1.9-4.7) 
    ≥ 1 61 44.9 (36.6-54.2) 29.9 (27.3-32.6) 1.8 (1.4-2.4) 2.1 (1.6-2.8) 

cITP indicates chronic immune thrombocytopenia; CI, confidence interval; and MRR, mortality rate ratio.

*

Adjusted for age (≤ 60 years and > 60 years), sex, calendar year, and level of comorbidity.

Five-year mortality for patients with ITP diagnosed in the 1996-2001 period was substantially lower than for patients whose condition was diagnosed in the 2002-2007 period: 19% (95% CI, 13%-25%) for patients whose condition was diagnosed between 1996 and 2001 and 30% (95% CI, 23%-38%) among patients whose condition was diagnosed between 2002 and 2007. However, in the same period the proportion of patients aged > 60 years increased from 40% to 48% and the prevalence of a comorbidity score ≥ 1 increased from 28% to 47%. Thus, the MRRs did not differ substantially between these 2 periods (2.0 [95% CI, 1.3-2.9] vs 2.6 [95% CI, 1.9-3.5]).

The estimated 5-year cause-specific mortality related to hematologic malignancies was 3.3% (95% CI, 1.8%-6.2%) among patients with chronic ITP and 0.4% (95% CI, 0.2%-0.7%) among comparison cohort members. The corresponding unadjusted 5-year MRR for hematologic cancers was 8.2 (95% CI, 3.6-18.8). For other types of cancer, the 5-year cause-specific mortality was 5.3 (95% CI, 3.2-8.6) among patients with chronic ITP and 3.4 (95% CI, 2.8-4.1) among members of the comparison cohort. The 5-year cancer-specific unadjusted MRR was 1.6 (95% CI, 1.0-2.8). Five-year infection-related mortality was 4.2% (95% CI, 2.4%-7.2%) among patients with chronic ITP and 0.7% (95% CI, 0.5%-1.1%) among comparison cohort members. The 5-year unadjusted MRR thus was 6.0 for infections (95% CI, 3.0-11.8). Five-year mortality related to hemorrhagic episodes was 2.5% (95% CI, 1.2%-4.7%) among patients with chronic ITP and 0.3% (95% CI, 0.1%-0.6%) among comparison cohort members, and the corresponding 5-year unadjusted MRR was 10.6 (95% CI, 4.2-26.6). For cardiovascular conditions, we estimated a 5-year mortality of 5.3% (95% CI, 3.2%-8.6%) among patients with chronic ITP and 5.1% (95% CI, 4.4%-5.9%) among members of the comparison cohort, yielding an adjusted MRR of 1.3 (95% CI, 0.8-2.2).

Discussion

In this nationwide population-based study we found a substantially increased risk of infections and hemorrhagic episodes in patients with chronic ITP than in the general population. We also observed a nearly 5-fold increased risk of hematologic malignancies within 5 years after chronic ITP diagnosis. Compared with the general population cohort, patients with chronic ITP had more than a 2-fold higher 5-year all-cause MRR and were more probable to die of hematologic cancers, infections, and hemorrhagic episodes than were members of the general population.

The increased risk of intracranial or other hemorrhagic episodes in our population of patients with chronic ITP is consistent with previous reports.2,5,7,18  Our study also corroborates the earlier finding that age > 60 years is associated with the highest 5-year risk of hemorrhage.2,18  Yet, compared with the risk in the general population, the increase was relatively highest in patients < 60 years. The 12-fold increased 5-year risk of hemorrhage-related deaths among patients with chronic ITP underscores the importance of hemorrhage as a complication in this patient population.

In line with previous studies,8,9  we found an increased risk of infection among patients with chronic ITP. In the first year after ITP diagnosis, the risk of inpatient and outpatient hospital contacts for infections increased 4.5-fold compared with the general population. From the second to fifth year of follow-up patients with ITP continued to have an 80% higher risk of this outcome. The mechanisms underlying this finding are not entirely clear. ITP treatments, including immunosuppressive medications and splenectomy, may contribute to the increased risk.6,19  However, we did not find that risk of infection differed by platelet count at baseline. Another possibility is that the increased risk of hospital contacts for infection may stem from increased surveillance among patients with chronic ITP and a lower threshold for hospital-based care of infections compared with the general population. Still, 5-year infection-related mortality in patients with chronic ITP was 6 times higher than in the general population. This argues against the explanation that patients with chronic ITP were more probable to be hospitalized for less severe infections. Despite an increased risk of infections overall, patients with ITP in our study had a lower frequency of sexual, intra-abdominal, and urogenital infections than members of the comparison cohort. This could indicate that patients with chronic ITP have different lifestyle-related behaviors than persons in the general population.

The risk of hematologic malignancies and death from such cancers was also substantially increased among patients with chronic ITP. This accords with the findings of Enger et al10  in a US study and with a more general finding in a combined Swedish and Danish study that certain autoimmune conditions are related to an increased risk of Hodgkin lymphoma.20  In addition, long-term exposure to immunosuppressive medications is an established risk factor for hematologic cancer21,22  and may have contributed to the elevated risks. Alternatively, because chronic ITP is a diagnosis of exclusion, immune thrombocytopenia may represent the initial stage of hematologic malignancy.23,24  In the US study,10  in which the adjusted IRR for leukemia was 19.83 (95% CI, 5.84-67.34), 5 of 18 leukemia diagnoses occurred within 1-2 months after the chronic ITP diagnosis. Therefore, the investigators found it probable that some patients were misdiagnosed with ITP in the early stages of their cancer diagnosis.10 

In contrast to findings of a previous study with 2-year follow-up of 152 patients with chronic ITP whose mortality risk was found to be equivalent to that of the general population,8  we found > 2-fold higher 5-year mortality among patients with chronic ITP than among the general population. Still, our MRR was lower than the MRR of 4.21 (95% CI, 3.06-5.79) found by Enger et al10  during a median follow-up of 15 months. This could be because the US study only required 12 months of enrollment in the health plan before the date of chronic ITP and the chronic ITP diagnoses were not confirmed. Similar to the findings by Portielje et al,8  we found that patients with chronic ITP had relatively high risks of death because of infection or hemorrhage. We also found relatively high risks of cancer-related deaths. In contrast, the risk of cardiovascular death in patients with ITP was similar to that of the comparison cohort.

Our study has several limitations. Despite its nationwide setting, the sample size was small because of the rarity of the disease. Therefore, our stratified analyses had limited statistical precision. Another concern is that our definition of chronic ITP prevented us from identifying patients who received inpatient or outpatient hospital care for chronic ITP only once and were subsequently treated by their general practitioner. We assume that such patients were probable to have relatively high platelet counts and less severe disease. Patients with ITP who were older or who had recorded comorbidity also were more probable to have had > 1 hospital contact with ITP than younger patients without comorbidity. Such selection bias may have caused us to overestimate the rates. However, we attempted to mitigate this limitation through matching with the comparison cohort for comorbidity. We designed our study and extracted the data from the medical files before Provan et al4  published the international consensus report on investigation and management of primary immune thrombocytopenia in 2010. We therefore used a platelet count of < 150 × 109/L instead of < 100 × 109/L and disease duration of 6 months instead of 12 months to define chronic ITP. Use of these definitions may have led to inclusion of less severe ITP cases and therefore could bias our estimates toward unity.

Our study has certain strengths. The use of Danish health care data permitted a national population-based design with virtually complete follow-up, reducing the potential for selection bias. Data on chronic ITP diagnoses and platelet counts were obtained from medical records, so the potential for information bias (eg, nondifferential misclassification of diagnosis) was lower than might have been observed with other data sources (eg, claims).

In conclusion, primary chronic ITP is a serious disease with increased risk of infections, hemorrhagic episodes, hematologic cancers, and mortality. Our data indicate the need for improved disease management for these patients, specifically for these serious and potentially life-threatening complications.

The online version of this article contains a data supplement.

The publication costs of this article were defrayed in part by page charge payment. Therefore, and solely to indicate this fact, this article is hereby marked “advertisement” in accordance with 18 USC section 1734.

Acknowledgments

This study was partly supported by a research grant from Amgen Inc USA, administered by Aarhus University. Unrestricted grants from the Karen Elise Jensen Foundation also were received.

Authorship

Contribution: M.N. designed the study, analyzed and interpreted data, and wrote the manuscript; A.Ø.J. collected data, analyzed and interpreted data, and wrote the first draft; M.C.E. designed the study, performed statistical analysis, and analyzed and interpreted data; D.K.F. performed statistical analysis and analyzed and interpreted data; R.W.T. and S.C. analyzed and interpreted data and critically revised the manuscript; S.Z. designed the study, analyzed and interpreted data, and critically revised the manuscript; and H.T.S. conceived and designed the study, analyzed and interpreted data, and critically revised the manuscript.

Conflict-of-interest disclosure: S.C. and S.Z. are employed by Amgen Inc. The remaining authors declare no competing financial interests. The Department of Clinical Epidemiology, however, is involved in studies with funding from various companies as research grants to (and administered by) Aarhus University. None of these studies have any relation to the present study.

Correspondence: Mette Nørgaard, Department of Clinical Epidemiology, Aarhus University Hospital, Sdr Skovvej 15, DK-9000 Aalborg, Denmark; e-mail: m.noergaard@rn.dk.

References

References
1
Beardsley
 
DS
ITP in the 21st century.
Hematology Am Soc Hematol Educ Program
2006
(pg. 
402
-
407
)
2
Stasi
 
R
Stipa
 
E
Masi
 
M
, et al. 
Long-term observation of 208 adults with chronic idiopathic thrombocytopenic purpura.
Am J Med
1995
, vol. 
98
 
5
(pg. 
436
-
442
)
3
Fogarty
 
PF
Chronic immune thrombocytopenia in adults: epidemiology and clinical presentation.
Hematol Oncol Clin North Am
2009
, vol. 
23
 
6
(pg. 
1213
-
1221
)
4
Provan
 
D
Stasi
 
R
Newland
 
AC
, et al. 
International consensus report on the investigation and management of primary immune thrombocytopenia.
Blood
2010
, vol. 
115
 
2
(pg. 
168
-
186
)
5
Cohen
 
YC
Djulbegovic
 
B
Shamai-Lubovitz
 
O
Mozes
 
B
The bleeding risk and natural history of idiopathic thrombocytopenic purpura in patients with persistent low platelet counts.
Arch Intern Med
2000
, vol. 
160
 
11
(pg. 
1630
-
1638
)
6
Cines
 
DB
McMillan
 
R
Management of adult idiopathic thrombocytopenic purpura.
Annu Rev Med
2005
, vol. 
56
 (pg. 
425
-
442
)
7
Feudjo-Tepie
 
MA
Le
 
RG
Beach
 
KJ
Bennett
 
D
Robinson
 
NJ
Comorbidities of idiopathic thrombocytopenic purpura: a population-based study.
Adv Hematol
2009
, vol. 
2009
 pg. 
963506
 
8
Portielje
 
JE
Westendorp
 
RG
Kluin-Nelemans
 
HC
Brand
 
A
Morbidity and mortality in adults with idiopathic thrombocytopenic purpura.
Blood
2001
, vol. 
97
 
9
(pg. 
2549
-
2554
)
9
Thomsen
 
RW
Schoonen
 
WM
Farkas
 
DK
, et al. 
Risk for hospital contact with infection in patients with splenectomy: a population-based cohort study.
Ann Intern Med
2009
, vol. 
151
 
8
(pg. 
546
-
555
)
10
Enger
 
C
Bennett
 
D
Forssen
 
U
Fogarty
 
PF
McAfee
 
AT
Comorbidities in patients with persistent or chronic immune thrombocytopenia.
Int J Hematol
2010
, vol. 
92
 
2
(pg. 
289
-
295
)
11
Frank
 
L
Epidemiology. When an entire country is a cohort.
Science
2000
, vol. 
287
 
5462
(pg. 
2398
-
2399
)
12
Andersen
 
TF
Madsen
 
M
Jørgensen
 
J
Mellemkjær
 
L
Olsen
 
JH
The Danish National Hospital Register. A valuable source of data for modern health sciences.
Dan Med Bull
1999
, vol. 
46
 
3
(pg. 
263
-
268
)
13
Pedersen
 
CB
Gøtzsche
 
H
Møller
 
JO
Mortensen
 
PB
The Danish Civil Registration System. A cohort of eight million persons.
Dan Med Bull
2006
, vol. 
53
 
4
(pg. 
441
-
449
)
14
Juel
 
K
Helweg-Larsen
 
K
The Danish registers of causes of death.
Dan Med Bull
1999
, vol. 
46
 
4
(pg. 
354
-
357
)
15
Charlson
 
ME
Pompei
 
P
Ales
 
KL
MacKenzie
 
CR
A new method of classifying prognostic comorbidity in longitudinal studies: development and validation.
J Chronic Dis
1987
, vol. 
40
 
5
(pg. 
373
-
383
)
16
Storm
 
HH
Michelsen
 
EV
Clemmensen
 
IH
Pihl
 
J
The Danish Cancer Registry–history, content, quality and use.
Dan Med Bull
1997
, vol. 
44
 
5
(pg. 
535
-
539
)
17
Gooley
 
TA
Leisenring
 
W
Crowley
 
J
Storer
 
BE
Estimation of failure probabilities in the presence of competing risks: new representations of old estimators.
Stat Med
1999
, vol. 
18
 
6
(pg. 
695
-
706
)
18
Cortelazzo
 
S
Finazzi
 
G
Buelli
 
M
, et al. 
High risk of severe bleeding in aged patients with chronic idiopathic thrombocytopenic purpura.
Blood
1991
, vol. 
77
 
1
(pg. 
31
-
33
)
19
Stasi
 
R
Provan
 
D
Management of immune thrombocytopenic purpura in adults.
Mayo Clin Proc
2004
, vol. 
79
 
4
(pg. 
504
-
522
)
20
Landgren
 
O
Engels
 
EA
Pfeiffer
 
RM
, et al. 
Autoimmunity and susceptibility to Hodgkin lymphoma: a population-based case-control study in Scandinavia.
J Natl Cancer Inst
2006
, vol. 
98
 
18
(pg. 
1321
-
1330
)
21
Gridley
 
G
McLaughlin
 
JK
Ekbom
 
A
, et al. 
Incidence of cancer among patients with rheumatoid arthritis.
J Natl Cancer Inst
1993
, vol. 
85
 
4
(pg. 
307
-
311
)
22
Sørensen
 
HT
Mellemkjær
 
L
Nielsen
 
GL
, et al. 
Skin cancers and non-Hodgkin lymphoma among users of systemic glucocorticoids: a population-based cohort study.
J Natl Cancer Inst
2004
, vol. 
96
 
9
(pg. 
709
-
711
)
23
Saitoh
 
T
Matsushima
 
T
Saito
 
Y
, et al. 
Hodgkin lymphoma presenting with various immunologic abnormalities, including autoimmune hepatitis, Hashimoto's thyroiditis, autoimmune hemolytic anemia, and immune thrombocytopenia.
Clin Lymphoma Myeloma
2008
, vol. 
8
 
1
(pg. 
62
-
64
)
24
Hauswirth
 
AW
Skrabs
 
C
Schutzinger
 
C
, et al. 
Autoimmune thrombocytopenia in non-Hodgkin's lymphomas.
Haematologica
2008
, vol. 
93
 
3
(pg. 
447
-
450
)