Abstract

Multiple risk-assessment models (RAMs) for venous thromboembolism (VTE) in hospitalized medical patients have been developed. To inform the 2018 American Society of Hematology (ASH) guidelines on VTE, we conducted an overview of systematic reviews to identify and summarize evidence related to RAMs for VTE and bleeding in medical inpatients. We searched Epistemonikos, the Cochrane Database, Medline, and Embase from 2005 through June 2017 and then updated the search in January 2020 to identify systematic reviews that included RAMs for VTE and bleeding in medical inpatients. We conducted study selection, data abstraction and quality assessment (using the Risk of Bias in Systematic Reviews [ROBIS] tool) independently and in duplicate. We described the characteristics of the reviews and their included studies, and compared the identified RAMs using narrative synthesis. Of 15 348 citations, we included 2 systematic reviews, of which 1 had low risk of bias. The reviews included 19 unique studies reporting on 15 RAMs. Seven of the RAMs were derived using individual patient data in which risk factors were included based on their predictive ability in a regression analysis. The other 8 RAMs were empirically developed using consensus approaches, risk factors identified from a literature review, and clinical expertise. The RAMs that have been externally validated include the Caprini, Geneva, IMPROVE, Kucher, and Padua RAMs. The Padua, Geneva, and Kucher RAMs have been evaluated in impact studies that reported an increase in appropriate VTE prophylaxis rates. Our findings informed the ASH guidelines. They also aim to guide health care practitioners in their decision-making processes regarding appropriate individual prophylactic management.

Introduction

The clinical and economic burden of venous thromboembolism (VTE), including deep vein thrombosis (DVT) and pulmonary embolism (PE), is immense.1  VTE is a complex multifactorial disease, influenced by acquired or inherited predispositions to thrombosis (eg, thrombophilia), environmental exposures (eg, clinical risk factors), and the interaction between them.2  The annual incidence of VTE in the adult populations is ∼1 in a 1000.3,4  Approximately 50% of all VTE events occur during or shortly after hospitalization for surgery (24%) or acute medical illness (22%).4-6  Patients hospitalized for an acute medical illness have an eightfold increased risk of VTE compared with the general population.7,8  Medical costs in the United States are estimated at approximately $17 000 dollars more for patients who have a VTE event during or after a recent hospitalization compared with their hospitalized counterparts who do not experience a VTE event.1 

The use of pharmacological thromboprophylaxis reduces the incidence of VTE in a cost-effective manner in many patient populations but increases the risk of bleeding.6,9  To inform optimal management, risk-assessment models (RAMs) can be used to aid in stratifying risk of developing a VTE or bleeding event for individual patients. A RAM is defined as a formal combination of multiple predictors from which risks of a specific end point can be calculated for individual patients. A RAM may also be called a prognostic model, risk (or clinical) prediction model, or predictive model.10  A RAM undergoes 3 main phases: model development (including internal validation), external validation, and investigations of impact on decision-making and patient outcomes.10 

In 2018, the American Society of Hematology (ASH) together with the Michael G. DeGroote and MacGRADE Centres at McMaster University developed clinical practice guidelines for the prevention of VTE in hospitalized medical patients.4  The methods used to develop these guidelines were based on the Guidelines International Network (GIN)-McMaster Guideline Development Checklist,11  the GRADE approach,12  and the Cochrane Handbook for systematic review methodology.13  We conducted an overview of systematic reviews to identify and describe RAMs and their clinical utility for VTE and bleeding in hospitalized medical patients to inform the ASH guidelines.4 

Methods

We conducted an overview of reviews to identify systematic reviews that report on RAM development, validation, or impact studies for VTE and bleeding in hospitalized medical patients. We developed a protocol that was reviewed and revised by the authors, but we did not register it.

Data sources and searches

We initially searched Epistemonikos, Cochrane Database of Systematic Reviews, Medline, and Embase from 2005 through June 2017 with the help of an information specialist and conducted an update of the search in January 2020. Supplemental eTable 1 provides detailed descriptions of the search strategies. Our search included both Medical Subject Headings (MesH) and text-word terms that combined VTE-related terms with prognosis terms. We added a systematic review filter when using Medline and Embase. We used no language restrictions.

Study selection

Prior to starting the screening process, 4 teams of 2 reviewers participated in training and calibration exercises. Teams of 2 reviewers screened, independently and in duplicate, the titles and abstracts of all identified citations. They then retrieved full texts of all citations judged as potentially eligible by at least 1 of the reviewers on each team. The reviewers screened the full texts independently and in duplicate and compared results. A third, senior reviewer resolved disagreements when necessary. Reviewers used standardized screening forms throughout the process. The eligibility criteria for study selection included the following characteristics.

Types of studies.

We included studies that explicitly stated the use of the “systematic review” methodology, with or without conducting a meta-analysis, in the title or abstract. Also, the study must have reported conducting a search for individual studies in at least 1 database. We included systematic reviews that reported on development, validation, or impact studies of multivariable prediction/RAMs, tools, or scores, proposed for individual risk estimation of any future VTE or bleeding outcome in hospitalized medical patients.

Population.

We included systematic reviews that addressed adult patients hospitalized for an acute, critical, or chronic medical illness. An acute medical illness is defined as an illness that requires urgent, nonoperative care such as heart failure, respiratory insufficiency, stroke, and infectious or inflammatory diseases.4  A critical illness is one that presents as an immediately life-threatening condition that requires care in an intensive or critical care unit.4  A chronic medical illness is defined as an acute exacerbation of a chronic medical condition that requires hospitalization.4 

Intervention.

We investigated all RAMs that assessed risk of VTE or bleeding in adults hospitalized for medical illness and were reported in the eligible systematic reviews.

Comparison.

Standard care without the use of RAMs or a different RAM than the one used in the intervention.

Outcomes.

We evaluated the outcomes of VTE and bleeding. We defined VTE as any symptomatic or asymptomatic DVT or PE from hospital admission up to 90 days postdischarge. We considered bleeding as any major or nonmajor but clinically significant bleeding up to 90 days postdischarge.14 

Setting.

We included systematic reviews that reported on studies in which the patients were admitted to an inpatient ward or intensive care unit for medical illness.

Data extraction

We conducted calibration exercises and piloting of the data-extraction form prior to the start of the process. Using a standardized form, a team of 2 reviewers (A.J.D. and R.C.) extracted data independently and in duplicate from all eligible studies and compared results. They consulted a third reviewer (H.J.S.) in case of any disagreement.

For the identified RAM systematic reviews, the reviewers abstracted data on the following characteristics.

Characteristics of the systematic review.

  • Main elements of the search strategy (including databases searched, date of search).

  • Approach used to synthesize findings (narrative synthesis, meta-analysis).

  • Risk-of-bias tool used to assess individual studies and results.

  • Authors’ assessment and conclusions.

Findings pertaining to the studies included in the systematic review.

  • Type of prognostic model studies included (ie, development, validation, or impact).

  • Population.

  • Outcomes.

  • Setting.

  • Time frame of prognostic measurement.

  • Results (predictive performance).

Risk-of-bias assessment of systematic reviews

We assessed the risk of bias in systematic reviews using the Risk of Bias in Systematic Reviews [ROBIS] tool. The tool includes 3 phases: assessing relevance (optional), identifying concerns with the review process and judging risk of bias.15  We focused on phases 2 and 3. Phase 2 considers the following 4 domains: study eligibility criteria, identification and selection of studies, data collection and study appraisal, and synthesis and findings. The signaling questions in each of the domains are judged as yes, probably yes, probably no, no, or no information. In phase 3, we rated the overall risk of bias as low, high, or unclear depending on the rating of the individual domains. For the individual studies included in the systematic reviews, we reported on the risk-of-bias tool used and the judgments made by the authors when available.

Synthesis and presentation of findings

We used a narrative synthesis of included systematic reviews to summarize our findings. We presented the findings of any qualitative or quantitative syntheses conducted by the authors of the reviews. We focused on identifying and describing the RAMs, their performance, and gaps in their development, validation, or assessment of impact. If a meta-analysis was presented in 1 of the included systematic reviews, we presented the results and the relevant methodological aspects (eg, types of data, effects measured, heterogeneity, sensitivity analysis). If different information regarding the same RAM was provided in the reviews, we narratively described all findings to provide a comprehensive description of that model. We did not perform a quantitative synthesis of the RAMs, as the main aim of an overview of reviews is to provide a summary of existing research synthesis and not resynthesizing evidence.16 

Results

Figure 1 illustrates the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram.17  In our original search, we identified 6095 citations, of which we included 144 studies for full text assessment. From those, we included 2 systematic reviews. Both reviews evaluated RAMs for VTE in hospitalized medical patients.18,19  We did not identify any systematic review that evaluated RAMs for bleeding. When we conducted an update of our search in January 2020, we identified an additional 4122 citations, none of which fulfilled our inclusion criteria after full evaluation.

Figure 1.

PRISMA flowchart.

Figure 1.

PRISMA flowchart.

Description of the included systematic reviews

The included systematic reviews aimed to identify RAMs developed to calculate the risk of VTE in hospitalized nonsurgical patients and to evaluate their generalizability, validity, and utility.18,19  Stuck et al focused only on acutely ill medical patients and searched for English, German, French, or Italian studies in only 1 database (Medline), from inception until 30 May 2016.19  They then performed an additional search to identify impact analysis studies that may have been missed.19  The authors did not conduct a quality appraisal of the included studies.19  Huang et al included a comprehensive search for English articles across 4 databases from inception until December 2011.18  The authors of that review appraised the evidence by using a modified Downs and Black checklist for the RAMs developed using individual patient data, and the modified Appraisal of Guidelines for Research and Evaluation (AGREE) instrument for the RAMs generated based on consensus approaches, published data, and clinical expertise.18  The reviews included 19 unique primary studies that reported on 15 RAMs. Of those, 3 studies and 4 RAMs were common in both systematic reviews.20-38  Stuck et al identified 4 additional impact studies from their supplementary search.19  Neither systematic review conducted a meta-analysis of the results.18,19  Huang et al highlighted that pooling was not possible due to variability in the methods to develop the RAMs, in the outcome measurements, and in the number, type, and strength of associations of the included VTE risk factors.18  Authors of both reviews concluded that there is a lack of generalizability and adequate validation of the published RAMs, which hinders their use in clinical practice.18,19  However, Stuck et al encouraged the implementation of any of the available RAMs to improve the consistency of use of thromboprophylaxis until further evidence is available.19  Characteristics of the systematic reviews are detailed in Table 1.

Table 1.

Systematic review characteristics

PopulationOutcomeStudy typesQuality assessment and gradingStatistical analysisRAMs identified
Huang et al (2013)18      
 Hospitalized nonsurgical patients (studies that focused primarily on children, pregnant women, psychiatric patients, surgical patients, or outpatients were excluded) VTE (DVT/PE); studies that only included patients with upper-extremity DVT were excluded Prognostic model studies where the model was developed either by analyzing individual patient data or by expert consensus • Studies that developed RAMs based on individual patient data: modified Downs and Black checklist
• Studies that developed RAMs based on expert consensus: modified AGREE instrument
• The quality score was expressed as a percentage of the total assigned score divided by the total maximum score of applicable items 
• Narrative synthesis
• No meta-analysis was done due to heterogeneous studies
• Reported ORs or HRs of risk factors included in category I RAMs
• C-statistics reported (model discrimination on derivation and/or validation data sets) 
RAMs based on individual patient data:
 • Woller 2011 (Intermountain/4 element)
 • Spyropoulos 2011 (IMPROVE)
 • Rothberg 2011 (Multivariable model)
 • Alikhan 2004 (MEDENOX)
 • Weil-Engerer 2004
 • Yale 2005
RAMs based on expert consensus:
 • Rocha 2007
 • McCaffrey 2007
 • Samama 2006
 • Cohen 2005
 • Caprini 2001 (Caprini) 
Stuck et al (2017)19      
 Acutely ill medical patients (studies in non-medical, pediatric, pregnant, or psychiatric patients were excluded) VTE (DVT/PE) Prognostic model studies developed based on individual patient data or consensus Not conducted • Narrative synthesis
• No meta-analysis was done 
RAMs developed by derivation by identifying factors with predictive power:
 • 4-Element RAM
 • IMPROVE-RAM (multicenter external validation)
 • Multivariable model
 • Full logistic model
RAMs generated empirically based on consensus approaches, published data, and clinical expertise:
 • Kucher RAM (multicenter external validation)
 • Geneva RAM (multicenter external validation)
 • Padua RAM (multicenter external validation)
 • Caprini RAM 
PopulationOutcomeStudy typesQuality assessment and gradingStatistical analysisRAMs identified
Huang et al (2013)18      
 Hospitalized nonsurgical patients (studies that focused primarily on children, pregnant women, psychiatric patients, surgical patients, or outpatients were excluded) VTE (DVT/PE); studies that only included patients with upper-extremity DVT were excluded Prognostic model studies where the model was developed either by analyzing individual patient data or by expert consensus • Studies that developed RAMs based on individual patient data: modified Downs and Black checklist
• Studies that developed RAMs based on expert consensus: modified AGREE instrument
• The quality score was expressed as a percentage of the total assigned score divided by the total maximum score of applicable items 
• Narrative synthesis
• No meta-analysis was done due to heterogeneous studies
• Reported ORs or HRs of risk factors included in category I RAMs
• C-statistics reported (model discrimination on derivation and/or validation data sets) 
RAMs based on individual patient data:
 • Woller 2011 (Intermountain/4 element)
 • Spyropoulos 2011 (IMPROVE)
 • Rothberg 2011 (Multivariable model)
 • Alikhan 2004 (MEDENOX)
 • Weil-Engerer 2004
 • Yale 2005
RAMs based on expert consensus:
 • Rocha 2007
 • McCaffrey 2007
 • Samama 2006
 • Cohen 2005
 • Caprini 2001 (Caprini) 
Stuck et al (2017)19      
 Acutely ill medical patients (studies in non-medical, pediatric, pregnant, or psychiatric patients were excluded) VTE (DVT/PE) Prognostic model studies developed based on individual patient data or consensus Not conducted • Narrative synthesis
• No meta-analysis was done 
RAMs developed by derivation by identifying factors with predictive power:
 • 4-Element RAM
 • IMPROVE-RAM (multicenter external validation)
 • Multivariable model
 • Full logistic model
RAMs generated empirically based on consensus approaches, published data, and clinical expertise:
 • Kucher RAM (multicenter external validation)
 • Geneva RAM (multicenter external validation)
 • Padua RAM (multicenter external validation)
 • Caprini RAM 

AGREE, Appraisal of Guidelines for Research and Evaluation; HR, hazard ratio; OR, odds ratio.

Risk-of-bias assessment of systematic reviews and included studies

We rated the overall risk of bias for the systematic review by Huang et al as low,18  and the review by Stuck et al as high.19  We judged both reviews as low risk in terms of study eligibility criteria (domain 1). However, we originally rated both systematic reviews at high risk for their identification and selection of studies for inclusion (domain 2), but each review came with different concerns. Stuck et al searched only 1 database,19  and did not conduct a sufficiently sensitive search to identify all potentially eligible studies.19  On the other hand, Huang et al made no mention of conducting an independent and duplicate screening process,18  or using standardized screening forms throughout the screening and abstraction processes.18  Also, both reviews placed restrictions on language that may have led to missing studies.18,19  Despite originally rating Huang et al as high risk of bias for the domain identification and selection of studies for inclusion, we considered it low risk of bias in our overall judgement.18  We made this decision because our results revealed that all relevant studies were captured within the search date, from inception through May 2011, despite the methodological limitations noted herein.18  Regarding data collection and study appraisal (domain 3), we rated the review by Huang et al as low risk of bias for appraising the included studies and the review by Stuck et al as high risk of bias for not conducting a critical appraisal.18,19  Neither of the reviews conducted a meta-analysis, which made it challenging to assess their methods of synthesis and presentation of findings domain (domain 4) using ROBIS. However, our rationale for rating Huang et al as low risk was due to the authors stating that the results were too heterogeneous and could not be pooled.18  We rated the review by Stuck et al as high risk because the authors did not provide a reasoning for not pooling the results.19  Supplemental eTable 2 provides the risk-of-bias assessment of the systematic reviews by phase and domain.

Huang et al described that the quality assessment, using the modified Downs and Black checklist, ranged between 55% and 88% for the RAMs derived using individual patient data.18  The quality appraisal score ranged between 48% and 77% for the RAMs developed by consensus based on the modified AGREE instrument.18 Tables 2 and 3 report the quality-assessment score for each of the RAMs.

Table 2.

Characteristics of the RAMs

Name of RAMNo. of risk factorsRisk factorsWeighing points of risk factorsDefined cutoffs for risk groups
RAMs developed based on individual patient data     
 MEDENOX RAM22  • Age, prior VTE, active cancer, acute infectious disease, chronic respiratory disease (primary diagnosis of COPD) Not described Not described 
 Weill Engerer RAM32  • Age, prior VTE, chronic edema of lower limbs, chronic heart failure, current lower-limb paralysis, bed rest/immobilized, congestive heart failure Not described Not described 
 Yale RAM34  • Age, oral contraceptive/HRT, varicose veins, type II diabetes mellitus, nursing home, chemotherapy, corticosteroids, angina Not described Not described 
 IMPROVE RAM31  • Previous VTE
• Known thrombophilia, current lower-limb paralysis, current cancer
• Immobilized ≥7 d, ICU/CCU stay, age >60 y 
• 3
• 2 points each
• 1 point each 
• According to Rosenberg et al28 : 0-2, low risk; ≥3, high risk
• According to Mahan et al20 : 0-1, low risk; 2-3, intermediate risk; ≥4, high risk 
 Multivariable RAM29  13 • Age, length of stay, sex, primary diagnosis, cancer, inflammatory bowel disease, obesity, central venous catheter, inherited thrombophilia, steroid use, mechanical ventilation, active chemotherapy, and urinary catheters None No cutoff available 
 4-Element RAM33  • Previous VTE, an order for bed rest, peripherally inserted central venous catheterization line, cancer diagnosis 1 point each • 0 low risk
• ≥1 high risk 
 Full logistic RAM33  86 Risk factors were not provided by systematic review; see reference for all risk factors None No cutoff available 
RAMs developed based on consensus approaches, data from the literature, and clinical expertise     
 Caprini RAM24  39 • Stroke; multiple trauma; elective major lower extremity arthroplasty; hip, pelvis or leg fracture; acute spinal cord injury (paralysis)
• Age (≥75 y); history of VTE; positive factor V Leiden; positive prothrombin G20210A; elevated serum homocysteine; positive lupus anticoagulant; other congenital or acquired thrombophilia; HIT; family history of VTE; elevated anticardiolipin antibodies
• Age (61-74 y); central venous access; arthroscopic surgery; major surgery; malignancy; laparoscopic procedure ≥45 min; patient confined to bed; immobilizing plaster cast
• Age (41-60 y); acute myocardial infarction; heart failure; varicose veins; obesity (BMI ≥ 25); inflammatory bowel disease; sepsis; COPD or abnormal pulmonary function; severe lung disease; oral contraceptives or HRT; pregnancy or postpartum; history of unexpected stillborn infant, recurrent spontaneous abortion (≥3), premature birth with toxemia or growth-restricted infant; medical patient currently at bed rest; minor surgery planned; history of prior major surgery; swollen legs 
• 5 points each
• 3 points each
• 2 points each
• 1 point each 
• According to Zhou et al36  and Zhou et al35 : 0-1, low risk; 2, intermediate risk; 3-4, high risk; >5, highest risk
• According to Abdel-Razeq et al21 : ≥2, low risk; 3-4, moderate risk high; ≥5, high risk 
 Cohen RAM25  Not described Not described Not described Not described 
 Samama RAM30  Not described Not described Not described Not described 
 Rocha RAM37  Not described Not described Not described Not described 
 McCaffrey RAM26  Not described Not described Not described Not described 
 Kucher RAM39  • Cancer, prior VTE, hypercoagulability
• Major surgery
• Bed rest, age >70 y, obesity (BMI > 30), HRT/oral contraceptive pill 
• 3 points each
• 2 points
• 1 point each 
• According to Kucher et al39 : 1-3, low risk; ≥4 points, high risk
• According to Woller et al33 : 1-2, low risk; ≥3, high risk 
 Padua RAM23  11 • Active cancer, previous VTE (with exclusion of superficial vein thrombosis), reduced mobility, known thrombophilia
• Recent (≥1 mo) trauma and/or surgery
• Elderly age (≥70 y), heart and/or respiratory failure, acute myocardial infarction or ischemic stroke, acute infection and/or rheumatologic disorder, obesity (BMI ≥ 30), ongoing hormonal treatment 
• 3 points each
• 2 points
• 1 point each 
• <4, low risk
• ≥4, high risk 
 Geneva RAM27  19 • Cardiac failure, respiratory failure, recent stroke (<3 mo), recent myocardial infarction (<4 wk), acute infectious disease (including sepsis), acute rheumatic disease, active cancer, myeloproliferative syndrome, nephrotic syndrome, prior VTE, known hypercoagulable state
• Immobilization (complete bed rest or inability to walk for >30 min/d) for >3 d, recent travel >6 h, age > 60 y, BMI > 30, chronic venous insufficiency, pregnancy, hormonal therapy, dehydration (assessed subjectively by the treating physician) 
• 2 points each
• 1 point each 
• 1-2, low risk
• ≥3, high risk 
Name of RAMNo. of risk factorsRisk factorsWeighing points of risk factorsDefined cutoffs for risk groups
RAMs developed based on individual patient data     
 MEDENOX RAM22  • Age, prior VTE, active cancer, acute infectious disease, chronic respiratory disease (primary diagnosis of COPD) Not described Not described 
 Weill Engerer RAM32  • Age, prior VTE, chronic edema of lower limbs, chronic heart failure, current lower-limb paralysis, bed rest/immobilized, congestive heart failure Not described Not described 
 Yale RAM34  • Age, oral contraceptive/HRT, varicose veins, type II diabetes mellitus, nursing home, chemotherapy, corticosteroids, angina Not described Not described 
 IMPROVE RAM31  • Previous VTE
• Known thrombophilia, current lower-limb paralysis, current cancer
• Immobilized ≥7 d, ICU/CCU stay, age >60 y 
• 3
• 2 points each
• 1 point each 
• According to Rosenberg et al28 : 0-2, low risk; ≥3, high risk
• According to Mahan et al20 : 0-1, low risk; 2-3, intermediate risk; ≥4, high risk 
 Multivariable RAM29  13 • Age, length of stay, sex, primary diagnosis, cancer, inflammatory bowel disease, obesity, central venous catheter, inherited thrombophilia, steroid use, mechanical ventilation, active chemotherapy, and urinary catheters None No cutoff available 
 4-Element RAM33  • Previous VTE, an order for bed rest, peripherally inserted central venous catheterization line, cancer diagnosis 1 point each • 0 low risk
• ≥1 high risk 
 Full logistic RAM33  86 Risk factors were not provided by systematic review; see reference for all risk factors None No cutoff available 
RAMs developed based on consensus approaches, data from the literature, and clinical expertise     
 Caprini RAM24  39 • Stroke; multiple trauma; elective major lower extremity arthroplasty; hip, pelvis or leg fracture; acute spinal cord injury (paralysis)
• Age (≥75 y); history of VTE; positive factor V Leiden; positive prothrombin G20210A; elevated serum homocysteine; positive lupus anticoagulant; other congenital or acquired thrombophilia; HIT; family history of VTE; elevated anticardiolipin antibodies
• Age (61-74 y); central venous access; arthroscopic surgery; major surgery; malignancy; laparoscopic procedure ≥45 min; patient confined to bed; immobilizing plaster cast
• Age (41-60 y); acute myocardial infarction; heart failure; varicose veins; obesity (BMI ≥ 25); inflammatory bowel disease; sepsis; COPD or abnormal pulmonary function; severe lung disease; oral contraceptives or HRT; pregnancy or postpartum; history of unexpected stillborn infant, recurrent spontaneous abortion (≥3), premature birth with toxemia or growth-restricted infant; medical patient currently at bed rest; minor surgery planned; history of prior major surgery; swollen legs 
• 5 points each
• 3 points each
• 2 points each
• 1 point each 
• According to Zhou et al36  and Zhou et al35 : 0-1, low risk; 2, intermediate risk; 3-4, high risk; >5, highest risk
• According to Abdel-Razeq et al21 : ≥2, low risk; 3-4, moderate risk high; ≥5, high risk 
 Cohen RAM25  Not described Not described Not described Not described 
 Samama RAM30  Not described Not described Not described Not described 
 Rocha RAM37  Not described Not described Not described Not described 
 McCaffrey RAM26  Not described Not described Not described Not described 
 Kucher RAM39  • Cancer, prior VTE, hypercoagulability
• Major surgery
• Bed rest, age >70 y, obesity (BMI > 30), HRT/oral contraceptive pill 
• 3 points each
• 2 points
• 1 point each 
• According to Kucher et al39 : 1-3, low risk; ≥4 points, high risk
• According to Woller et al33 : 1-2, low risk; ≥3, high risk 
 Padua RAM23  11 • Active cancer, previous VTE (with exclusion of superficial vein thrombosis), reduced mobility, known thrombophilia
• Recent (≥1 mo) trauma and/or surgery
• Elderly age (≥70 y), heart and/or respiratory failure, acute myocardial infarction or ischemic stroke, acute infection and/or rheumatologic disorder, obesity (BMI ≥ 30), ongoing hormonal treatment 
• 3 points each
• 2 points
• 1 point each 
• <4, low risk
• ≥4, high risk 
 Geneva RAM27  19 • Cardiac failure, respiratory failure, recent stroke (<3 mo), recent myocardial infarction (<4 wk), acute infectious disease (including sepsis), acute rheumatic disease, active cancer, myeloproliferative syndrome, nephrotic syndrome, prior VTE, known hypercoagulable state
• Immobilization (complete bed rest or inability to walk for >30 min/d) for >3 d, recent travel >6 h, age > 60 y, BMI > 30, chronic venous insufficiency, pregnancy, hormonal therapy, dehydration (assessed subjectively by the treating physician) 
• 2 points each
• 1 point each 
• 1-2, low risk
• ≥3, high risk 

BMI, body mass index; CCU, critical care unit; COPD, chronic obstructive pulmonary disease; HIT, heparin-induced thrombocytopenia; HRT, hormone replacement therapy; ICU, intensive care unit.

Table 3.

Characteristics of the studies describing the RAMs developed using individual patient data

First author, year (quality score by authors of SR)Reference systematic reviewStudy design and study type (sample size)SettingData sourcePopulation (sample size)Outcomes and methods of diagnosisFollow-up time
MEDENOX RAM        
 Alikhan et al, 2004 (82%)22  Huang et al • Double-masked RCT
• Derivation (1102 enrolled, 866 without missing data) 
• Countries, 9
• Sites, 60
• Time, 1996-1998 
Medical records (MEDENOX study) Age >75 y, 51.7; male, 49.7%; cancer, 13.6%; major surgery within 3 mo, 0%; % of VTE, 12%; % of PE, 4/102 (4%); in-hospital VTE prophylaxis, 288 (placebo; 287 enoxaparin 20 mg; 291 enoxaparin 40 mg) • DVT including below the knee DVT but not upper-extremity DVT (diagnosis based on venography of the legs or ultrasonography)
• PE (diagnosis confirmed by lung scanning, pulmonary angiography, helical CT; or at autopsy)
• Anticoagulant/ thrombolytic medication use was considered in model 
14 d since admission 
Weill-Engerer RAM        
 Weill-Engerer et al, 2004 (85%)32  Huang et al Prospective case control- derivation (310:310) • Countries, 1 (France)
• Sites, 10 (university hospitals with long, intermediate, and short-term care facilities)
• Time, 16 mo 
Medical records Geriatric and high-risk patients; mean age, 85.7 ± 7 y; male, 23.5%; cancer, 9%; major surgery within 1 mo, 4%; upper-limb DVT, ND; unknown site VTE, ND; in-hospital VTE prophylaxis, ND • Clinically confirmed DVT including below the knee DVT but not upper-extremity DVT (diagnosis based on ray-scale and Doppler sonography or venography)
• Anticoagulant/thrombolytic medication use was considered in model 
In hospital 
Yale RAM        
 Yale et al, 2005 (55%)34  Huang et al Case control- derivation (190:190) • Countries, 1 (USA)
• Sites, multiple
• Time, 1995-2002 
EMRs Medical patients discharged and rehospitalized; median age, ND; male, ND; cancer, ND; major surgery within 3 mo, 0%; upper-limb DVT, ND; unknown site VTE, ND; in-hospital VTE prophylaxis, ND • DVT, location not described (not specified on diagnosis and definition)
• Anticoagulant/thrombolytic medication use was considered in model 
60 d 
IMPROVE RAM        
 Spyropoulos et al, 2011 (86%)31  Huang et al and Stuck et al Prospective cohort derivation (N = 15156) • Countries, 12
• Sites, 52
• Time, 2002-2006 
Medical records review (IMPROVE Study) Median age, 68 y; male, 50%; cancer, 22%; major surgery within 3 mo, 0%; % of VTE, 184 (1.2%); % of PE, ND; in-hospital VTE prophylaxis, 44% • Symptomatic VTE excluding upper-extremity DVT
• No description if below the knee DVT was included (diagnosis based on diagnosis test result and treatment information)
• Anticoagulant/thrombolytic medication use was considered in model 
92 d 
 Mahan et al, 2014 (ND)20  Stuck et al Case control- external validation (ND) • Countries, ND
• Sites, 3
• Time, ND 
Not described Not described Not described Not described 
 Rosenberg et al, 2014 (ND)28  Stuck et al Retrospective study-external validation (ND) • Countries, ND
• Sites, 2
• Time, ND 
Not described Not described Not described Not described 
Multivariable RAM        
 Rothberg et al, 2011 (88%)29  Huang et al and Stuck et al Retrospective cohort- derivation and internal validation (242 738: 194 198 [80%] derivation set; 48 540 [20%] validation set) • Country, 1 (USA)
• Sites, 374
• Time, 2004-2005 
Premier’s Perspective database (measuring quality and health care utilization) Age ≥ 50 y, 87; male, 41%; cancer, 14%; major surgery before admission, ND; % VTE, 1052(0.4%); % of PE, ND; in-hospital VTE prophylaxis, 30% • Symptomatic VTE including below the knee DVT, upper-extremity DVT was excluded (second diagnosis based on ICD-9-CM and confirmed with diagnosis test result and treatment information)
• Anticoagulant/ thrombolytic medication use was considered in model 
30 d 
4-Element RAM        
 Woller et al, 2011 (68%)33  Huang et al and Stuck et al Retrospective cohort- derivation (143 975 + 46 856) • Country, 1 (USA)
• Sites, 22
• Time, 2000-2007 derivation; 2008-2009 validation 
Intermountain health care administrative and EMR system Mean age, 63 y; male, 44%; cancer, 44%; major surgery within 30 d, 1%; % of VTE, 3.7%; % of PE, ND • Symptomatic VTE including below the knee and upper-extremity DVT (diagnosis based on ICD-9-CM)
• No description whether anticoagulant/thrombolytic medication use was considered in model 
90 d post admission 
Full logistic RAM        
 Woller et al, 2011 (68%)33  Stuck et al Retrospective cohort- derivation (143 975 + 46 856) • Country, 1 (USA)
• Sites, 22
• Time, 2000-2007 derivation; 2008-2009 validation 
Intermountain health care administrative and EMR system Mean age, 63 y; male, 44%; cancer, 44%; major surgery within 30 d, 1%; % of VTE, 3.7%; % of PE, ND • Symptomatic VTE including below the knee and upper extremity DVT (diagnosis based on ICD-9-CM)
• No description whether anticoagulant/thrombolytic medication use was considered in model 
90 d post admission 
First author, year (quality score by authors of SR)Reference systematic reviewStudy design and study type (sample size)SettingData sourcePopulation (sample size)Outcomes and methods of diagnosisFollow-up time
MEDENOX RAM        
 Alikhan et al, 2004 (82%)22  Huang et al • Double-masked RCT
• Derivation (1102 enrolled, 866 without missing data) 
• Countries, 9
• Sites, 60
• Time, 1996-1998 
Medical records (MEDENOX study) Age >75 y, 51.7; male, 49.7%; cancer, 13.6%; major surgery within 3 mo, 0%; % of VTE, 12%; % of PE, 4/102 (4%); in-hospital VTE prophylaxis, 288 (placebo; 287 enoxaparin 20 mg; 291 enoxaparin 40 mg) • DVT including below the knee DVT but not upper-extremity DVT (diagnosis based on venography of the legs or ultrasonography)
• PE (diagnosis confirmed by lung scanning, pulmonary angiography, helical CT; or at autopsy)
• Anticoagulant/ thrombolytic medication use was considered in model 
14 d since admission 
Weill-Engerer RAM        
 Weill-Engerer et al, 2004 (85%)32  Huang et al Prospective case control- derivation (310:310) • Countries, 1 (France)
• Sites, 10 (university hospitals with long, intermediate, and short-term care facilities)
• Time, 16 mo 
Medical records Geriatric and high-risk patients; mean age, 85.7 ± 7 y; male, 23.5%; cancer, 9%; major surgery within 1 mo, 4%; upper-limb DVT, ND; unknown site VTE, ND; in-hospital VTE prophylaxis, ND • Clinically confirmed DVT including below the knee DVT but not upper-extremity DVT (diagnosis based on ray-scale and Doppler sonography or venography)
• Anticoagulant/thrombolytic medication use was considered in model 
In hospital 
Yale RAM        
 Yale et al, 2005 (55%)34  Huang et al Case control- derivation (190:190) • Countries, 1 (USA)
• Sites, multiple
• Time, 1995-2002 
EMRs Medical patients discharged and rehospitalized; median age, ND; male, ND; cancer, ND; major surgery within 3 mo, 0%; upper-limb DVT, ND; unknown site VTE, ND; in-hospital VTE prophylaxis, ND • DVT, location not described (not specified on diagnosis and definition)
• Anticoagulant/thrombolytic medication use was considered in model 
60 d 
IMPROVE RAM        
 Spyropoulos et al, 2011 (86%)31  Huang et al and Stuck et al Prospective cohort derivation (N = 15156) • Countries, 12
• Sites, 52
• Time, 2002-2006 
Medical records review (IMPROVE Study) Median age, 68 y; male, 50%; cancer, 22%; major surgery within 3 mo, 0%; % of VTE, 184 (1.2%); % of PE, ND; in-hospital VTE prophylaxis, 44% • Symptomatic VTE excluding upper-extremity DVT
• No description if below the knee DVT was included (diagnosis based on diagnosis test result and treatment information)
• Anticoagulant/thrombolytic medication use was considered in model 
92 d 
 Mahan et al, 2014 (ND)20  Stuck et al Case control- external validation (ND) • Countries, ND
• Sites, 3
• Time, ND 
Not described Not described Not described Not described 
 Rosenberg et al, 2014 (ND)28  Stuck et al Retrospective study-external validation (ND) • Countries, ND
• Sites, 2
• Time, ND 
Not described Not described Not described Not described 
Multivariable RAM        
 Rothberg et al, 2011 (88%)29  Huang et al and Stuck et al Retrospective cohort- derivation and internal validation (242 738: 194 198 [80%] derivation set; 48 540 [20%] validation set) • Country, 1 (USA)
• Sites, 374
• Time, 2004-2005 
Premier’s Perspective database (measuring quality and health care utilization) Age ≥ 50 y, 87; male, 41%; cancer, 14%; major surgery before admission, ND; % VTE, 1052(0.4%); % of PE, ND; in-hospital VTE prophylaxis, 30% • Symptomatic VTE including below the knee DVT, upper-extremity DVT was excluded (second diagnosis based on ICD-9-CM and confirmed with diagnosis test result and treatment information)
• Anticoagulant/ thrombolytic medication use was considered in model 
30 d 
4-Element RAM        
 Woller et al, 2011 (68%)33  Huang et al and Stuck et al Retrospective cohort- derivation (143 975 + 46 856) • Country, 1 (USA)
• Sites, 22
• Time, 2000-2007 derivation; 2008-2009 validation 
Intermountain health care administrative and EMR system Mean age, 63 y; male, 44%; cancer, 44%; major surgery within 30 d, 1%; % of VTE, 3.7%; % of PE, ND • Symptomatic VTE including below the knee and upper-extremity DVT (diagnosis based on ICD-9-CM)
• No description whether anticoagulant/thrombolytic medication use was considered in model 
90 d post admission 
Full logistic RAM        
 Woller et al, 2011 (68%)33  Stuck et al Retrospective cohort- derivation (143 975 + 46 856) • Country, 1 (USA)
• Sites, 22
• Time, 2000-2007 derivation; 2008-2009 validation 
Intermountain health care administrative and EMR system Mean age, 63 y; male, 44%; cancer, 44%; major surgery within 30 d, 1%; % of VTE, 3.7%; % of PE, ND • Symptomatic VTE including below the knee and upper extremity DVT (diagnosis based on ICD-9-CM)
• No description whether anticoagulant/thrombolytic medication use was considered in model 
90 d post admission 

CT, computed tomography; EMR, electronic medical record; ND, not described; SR, systematic review.

RAMs for VTE in hospitalized medical patients

Development of RAMs.

Table 2 provides a detailed description of the included RAMs. From the 15 included RAMs, 7 were derived by identifying risk factors with predictive power using individual patient data mainly from medical records. These include the RAMs by Alikhan et al, Weill-Engerer et al, Yale et al, Spyropoulos et al (IMPROVE RAM), Rothberg et al (Multivariable RAM), and Woller et al (4-Element and the full logistic RAMs).22,29,31-34  The individual studies included cohort studies (n = 3),29,31,33  case-control studies (n = 2),32,34  and a study based on a randomized controlled trial (RCT).22  The studies were all multicenter with 2 being multinational,22,31  and sample sizes ranged between 38034  to ∼243 000 patients.29  The eligibility criteria of individual studies differed mainly in the age cutoffs, ranging from >18 years to >65 years, the inclusion of history of surgery or trauma, length of hospital stay, known thrombophilia, and the exclusion of patients on pharmacologic thromboprophylaxis or anticoagulation. The proportion of cancer patients included in the derivation cohorts also varied widely ranging between 9%32  and 44%.33  Two studies reported on DVT alone (n = 2),32,34  whereas the rest of the studies used VTE (DVT, PE, and both) as their primary end point. All but 1 study considered symptomatic events only22 ; however, the definitions and the methods of diagnosis of the outcomes varied. Two studies used International Clinical Diagnosis Code version 9 (ICD-9) to define VTE.29,33  One of these 2 studies included codes for upper-extremity, superficial, and chronic DVT that were excluded from other studies, and did not use diagnostic test results and evidence of VTE treatment to validate the outcome.33  The other study validated the outcomes identified using ICD-9.29  Only 4 studies described their methods for diagnosis of DVT (ultrasonography or venography) or PE (lung scan, pulmonary angiography, or spiral computed tomography scanning).22,29,31,32  The follow-up time ranged from the index hospitalization up to 90 days postdischarge. The methods for selection of candidate risk factors also varied among the RAMs, with only 1 study using Kaplan-Meier and Cox multiple regression analyses to adjust for timing of events31  and another considering hospital clustering as a factor.29  The number of risk factors ranged between 4 and 14 in all the RAMs except the full logistic model, which included 86 risk factors.33  The measure of discrimination of the RAMs was reported in terms of the area under the curve (AUC) and ranged between 0.65 and 0.89.18,19  VTE prophylaxis administered in the hospital was considered as a potential confounder in only 2 of the included studies.29,31  Both studies reported no statistically significant impact of VTE prophylaxis on the outcome, and only 1 study included it as a prognostic factor in the RAM.29 Table 3 details the individual study characteristics related to these RAMs.

The other 8 RAMs included in the systematic review were empirically developed based on consensus approaches, risk factors identified from a literature review, and input from clinical experts (ranging between 3 and 24 members). These include the RAMs described by Caprini et al (Caprini RAM), Cohen et al, Samama et al, Rocha et al, McCaffrey et al, Kucher et al, Barbar et al (Padua RAM), and Nendaz et al (Geneva RAM).23-27,30,37,39  Only 1 of the 8 RAMs conducted a comprehensive systematic review of the literature to identify all potential risk factors prior to developing the model.37  One study focused on symptomatic DVT as the primary end point.26  Another study considered VTE as an outcome, but it was unclear whether it was symptomatic VTE only or symptomatic and asymptomatic VTE.23  The rest of the studies only considered symptomatic VTE as their primary end point. The number of risk factors included ranged between 8 and 39 for all except 4 RAMs whose risk factors were not described in the systematic reviews.25,26,30,37 Table 4 describes the individual study characteristics related to these RAMs.

Table 4.

Characteristics of the studies describing the RAMs developed using consensus approaches

First author, year (quality score by authors of SR)Reference systematic reviewStudy typeLiterature search and methods usedTarget population (sample size)OutcomeModel
Caprini RAM       
 Caprini et al, 2001 (64%)24  Huang et al Development Literature search ND; Consensus VTE prophylaxis among medical and surgical patients Symptomatic VTE; diagnosis, ND, varied among studies Checklist with risk-stratification score 
 Abdel- Razeq et al, 2010 (ND)21  Stuck et al External validation Validation cohort used Medically ill hospitalized cancer patients (n = 606) VTE (location, definition and diagnosis in systematic review) Checklist with risk-stratification score 
 Zhou et al, 2012 (ND)36  Stuck et al External validation Validation cohort used Hospitalized Chinese patients VTE (location, definition and diagnosis ND in systematic review) Checklist with risk-stratification score 
 Zhou et al, 2014 (ND)35  Stuck et al External validation Validation cohort used Hospitalized Chinese patients VTE (location, definition and diagnosis ND in systematic review) Checklist with risk-stratification score 
Cohen RAM       
 Cohen et al, 2005 (71%)25  Huang et al Development Literature search ND (2004 = latest literature included); consensus per literature review result VTE prophylaxis among acutely ill medical patients in hospital Symptomatic VTE diagnosis, ND, varied among studies Flowchart; no validation performed 
Samama RAM       
 Samama et al, 2006 (63%)30  Huang et al Development Literature search through 2002 (assess efficacy and effectiveness in combination with structured, quantitative techniques for incorporating the judgement of expert clinicians to produce appropriateness assessments for clinical conditions); RAND/UCLA appropriateness method to develop the risk matrix VTE prophylaxis among medical and surgical patients (pregnancy excluded) Symptomatic VTE diagnosis, ND, varied among studies Low/moderate/high/ very high grid case validation 
Rocha RAM       
 Rocha et al, 2007 (77%)37  Huang et al Development Systematic review search through 8/2004, RCT/cohort/case-control studies with at least 10 subjects evaluating risk factors or efficacy of prophylactic methods (LDUH, LMWH, mechanical) for VTE; classification of level of evidence per AHA/ACC/ESC guidelines for the management of patients VTE prophylaxis among acutely ill medical patients in hospital Symptomatic VTE; diagnosis, ND, varied Among studies Flowchart; no validation performed 
McCaffrey RAM       
 McCaffrey et al, 2007 (48%)26  Huang et al Development Literature search, 2000-2005 (OVID, ELSEVIER, CINAHL, Web of science); local hospitals and hospitals within the HCA network were asked to share DVT risk-assessment tools (total 15); consensus per literature review result VTE prophylaxis among all hospitalized patients Symptomatic DVT; diagnosis, ND Checklist with scores case validation (72 cases; 72 controls) ANOVA test used to compare the risk scores in 2 groups; Student t test used for each individual measure Interrater reliability measured (3 nurses; 144 charts at 32 different times) 
Kucher RAM       
 Woller et al, 2011 (ND)33  Stuck et al External validation Validation cohort used Medical patients (n = 143 975 [DC]; n =46 846 [VC]) VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score 
 Zhou et al, 2012 (ND)36  Stuck et al External validation Validation cohort used Hospitalized Chinese patients VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score 
 Kucher et al, 2005 (ND)39  Stuck et al Development and impact study A single center randomized trial where a computer-alert program was implemented to identify hospitalized patients at risk of VTE and assess impact on VTE and prophylaxis rates. Hospitalized patients VTE Computer-alert program 
 Baroletti et al, 2008 (ND)40  Stuck et al Impact study A prospective cohort study where electronic alerts were implemented to identify hospitalized patients at high risk of VTE not receiving prophylaxis Hospitalized patients VTE Electronic alerts 
Padua RAM       
 Barbar et al, 2010 (ND)23  Stuck et al Development Empirically generated based on consensus approaches, published data, and clinical expertise Hospitalized medical patients VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score 
 Zhou et al, 2012 (ND)36  Stuck et al External validation Validation cohort used Hospitalized Chinese patients VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score 
 Vardi et al, 2013 (ND)38  Stuck et al External validation Validation cohort used Patients with sepsis admitted to internal medicine department VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score 
 Zhou et al, 2014 (ND)35  Stuck et al External validation Validation cohort used Hospitalized Chinese patients VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score 
 Nendaz et al, 2014 (ND)27  Stuck et al External validation Validation cohort used to test the Padua prediction score that was developed through integration of additional empirically gained risk factors to the Kucher model Acutely ill medical patients (characteristics ND) (n = 1478) VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score 
 Rossetto et al, 2013 (ND)41  Stuck et al Impact study Physician compliance using Padua RAM in preventing VTE Hospitalized medical patients VTE Risk-stratification score 
Geneva RAM       
 Chopard et al, 2006 (not described in the systematic reviews)42  — Development Not described in the systematic reviews Not described in the systematic reviews Not described in the systematic reviews Not described in the systematic reviews 
 Nendaz et al, 2014 (ND)27  Stuck et al External validation Validation cohort used to test the Geneva risk score that was developed based on VTE prevention trials identified (search and search date ND) and recommendations from the ACCP guidelines Acutely ill medical patients (characteristics ND) (n = 1478) VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score; validated 
 Nendaz et al, 2010 (ND)43  Stuck et al Impact study A multicenter trial that implemented an e-Alert system with integration of the Geneva RAM Acutely ill medical patients VTE Electronic alerts and integrated RAMs 
First author, year (quality score by authors of SR)Reference systematic reviewStudy typeLiterature search and methods usedTarget population (sample size)OutcomeModel
Caprini RAM       
 Caprini et al, 2001 (64%)24  Huang et al Development Literature search ND; Consensus VTE prophylaxis among medical and surgical patients Symptomatic VTE; diagnosis, ND, varied among studies Checklist with risk-stratification score 
 Abdel- Razeq et al, 2010 (ND)21  Stuck et al External validation Validation cohort used Medically ill hospitalized cancer patients (n = 606) VTE (location, definition and diagnosis in systematic review) Checklist with risk-stratification score 
 Zhou et al, 2012 (ND)36  Stuck et al External validation Validation cohort used Hospitalized Chinese patients VTE (location, definition and diagnosis ND in systematic review) Checklist with risk-stratification score 
 Zhou et al, 2014 (ND)35  Stuck et al External validation Validation cohort used Hospitalized Chinese patients VTE (location, definition and diagnosis ND in systematic review) Checklist with risk-stratification score 
Cohen RAM       
 Cohen et al, 2005 (71%)25  Huang et al Development Literature search ND (2004 = latest literature included); consensus per literature review result VTE prophylaxis among acutely ill medical patients in hospital Symptomatic VTE diagnosis, ND, varied among studies Flowchart; no validation performed 
Samama RAM       
 Samama et al, 2006 (63%)30  Huang et al Development Literature search through 2002 (assess efficacy and effectiveness in combination with structured, quantitative techniques for incorporating the judgement of expert clinicians to produce appropriateness assessments for clinical conditions); RAND/UCLA appropriateness method to develop the risk matrix VTE prophylaxis among medical and surgical patients (pregnancy excluded) Symptomatic VTE diagnosis, ND, varied among studies Low/moderate/high/ very high grid case validation 
Rocha RAM       
 Rocha et al, 2007 (77%)37  Huang et al Development Systematic review search through 8/2004, RCT/cohort/case-control studies with at least 10 subjects evaluating risk factors or efficacy of prophylactic methods (LDUH, LMWH, mechanical) for VTE; classification of level of evidence per AHA/ACC/ESC guidelines for the management of patients VTE prophylaxis among acutely ill medical patients in hospital Symptomatic VTE; diagnosis, ND, varied Among studies Flowchart; no validation performed 
McCaffrey RAM       
 McCaffrey et al, 2007 (48%)26  Huang et al Development Literature search, 2000-2005 (OVID, ELSEVIER, CINAHL, Web of science); local hospitals and hospitals within the HCA network were asked to share DVT risk-assessment tools (total 15); consensus per literature review result VTE prophylaxis among all hospitalized patients Symptomatic DVT; diagnosis, ND Checklist with scores case validation (72 cases; 72 controls) ANOVA test used to compare the risk scores in 2 groups; Student t test used for each individual measure Interrater reliability measured (3 nurses; 144 charts at 32 different times) 
Kucher RAM       
 Woller et al, 2011 (ND)33  Stuck et al External validation Validation cohort used Medical patients (n = 143 975 [DC]; n =46 846 [VC]) VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score 
 Zhou et al, 2012 (ND)36  Stuck et al External validation Validation cohort used Hospitalized Chinese patients VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score 
 Kucher et al, 2005 (ND)39  Stuck et al Development and impact study A single center randomized trial where a computer-alert program was implemented to identify hospitalized patients at risk of VTE and assess impact on VTE and prophylaxis rates. Hospitalized patients VTE Computer-alert program 
 Baroletti et al, 2008 (ND)40  Stuck et al Impact study A prospective cohort study where electronic alerts were implemented to identify hospitalized patients at high risk of VTE not receiving prophylaxis Hospitalized patients VTE Electronic alerts 
Padua RAM       
 Barbar et al, 2010 (ND)23  Stuck et al Development Empirically generated based on consensus approaches, published data, and clinical expertise Hospitalized medical patients VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score 
 Zhou et al, 2012 (ND)36  Stuck et al External validation Validation cohort used Hospitalized Chinese patients VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score 
 Vardi et al, 2013 (ND)38  Stuck et al External validation Validation cohort used Patients with sepsis admitted to internal medicine department VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score 
 Zhou et al, 2014 (ND)35  Stuck et al External validation Validation cohort used Hospitalized Chinese patients VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score 
 Nendaz et al, 2014 (ND)27  Stuck et al External validation Validation cohort used to test the Padua prediction score that was developed through integration of additional empirically gained risk factors to the Kucher model Acutely ill medical patients (characteristics ND) (n = 1478) VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score 
 Rossetto et al, 2013 (ND)41  Stuck et al Impact study Physician compliance using Padua RAM in preventing VTE Hospitalized medical patients VTE Risk-stratification score 
Geneva RAM       
 Chopard et al, 2006 (not described in the systematic reviews)42  — Development Not described in the systematic reviews Not described in the systematic reviews Not described in the systematic reviews Not described in the systematic reviews 
 Nendaz et al, 2014 (ND)27  Stuck et al External validation Validation cohort used to test the Geneva risk score that was developed based on VTE prevention trials identified (search and search date ND) and recommendations from the ACCP guidelines Acutely ill medical patients (characteristics ND) (n = 1478) VTE (location, definition and diagnosis ND in systematic review) Risk-stratification score; validated 
 Nendaz et al, 2010 (ND)43  Stuck et al Impact study A multicenter trial that implemented an e-Alert system with integration of the Geneva RAM Acutely ill medical patients VTE Electronic alerts and integrated RAMs 

ACC, American College of Cardiology; ACCP, American College of Chest Physicians; AHA, American Heart Association; ANOVA, analysis of variance; DC, derivation cohort; ESC, European Society of Cardiology; HCA, Hospital Corporation of America; LDUH, low-dose unfractionated heparin; LMWH, low-molecular-weight heparin; VC, validation cohort.

Validation of RAMs.

The systematic reviews identified 5 RAMs that underwent internal validation without external validation. The RAMs by Woller et al (4-Element RAM and full logistic RAM) and the RAM by Rothberg et al (multivariable RAM) were developed based on individual patient data where risk factors were included based on their predictive ability identified in a regression analysis and compared in 1 retrospective study to one another.29,33  Comparing the 2 RAMs by Woller et al showed that the full logistic RAM reported a slightly higher AUC (0.86) compared with the 4-Element RAM (0.84).33  The study that assessed the RAM described by Rothberg et al (Multivariable RAM) in a validation cohort reported an AUC of 0.75.33  The other 2 RAMs developed empirically by Samama et al and McCaffrey et al were internally validated using clinical cases.18,26,30  Samama et al reported a 70% agreement between the levels of risk and judgments by clinicians in a validation effort.30  McCaffrey et al found that the mean total risk score in the VTE group was significantly higher than in the non-VTE group.26 

Only Stuck et al reported on external validation efforts of the RAMs.19  The authors of the systematic review reported the results of the validation studies conducted for the RAMs by Caprini et al (Caprini RAM), Nendaz et al (Geneva RAM), Spyropoulos et al (IMPROVE RAM), Kucher et al, and Barbar et al (Padua RAM), all of which were multicenter except for the Caprini RAM.20,21,27,28,31,33,35,36  The Caprini RAM was validated in 3 studies. The first study describes the validation of the Caprini RAM in a population including cancer patients and reported that the incidence rate of VTE in low-risk patients was 0% compared with 4.2% in high-risk patients.21  The second study compared the Caprini RAM to the RAMs by Kucher et al and Barbar et al (Padua RAM) and found that the Caprini RAM assigned more VTE patients into high- or highest-risk groups compared with the other 2 RAMs.36  The third study compared the cumulative risk of inpatients with VTE vs those without VTE using the Caprini and Padua RAMs. The authors reported that the high- to highest-risk group compared with the low- to moderate-risk group had similar odds ratios (ORs) using both RAMs with an OR of 3.01 for the Caprini RAM and 2.9 for the Padua RAM. However, 82.3% of VTE patients were found to be in the high- to highest-risk group according to the Caprini RAM whereas 30.1% of VTE patients were found to be in the high-risk group using the Padua RAM. A prospective multicenter study compared the RAM described by Nendaz et al (Geneva RAM) to the RAM by Barbar et al (Padua RAM) in ∼1500 patients with VTE and reported a favorable prediction of VTE and VTE-related mortality using the Geneva RAM.27  The study also found that the Geneva RAM more accurately identified low-risk patients who do not require thromboprophylaxis with a negative likelihood ratio of 0.28 compared with 0.51 for the RAM by Barbar et al (Padua RAM). The internal validation of the RAM by Spyropoulos et al (IMPROVE VTE RAM) reported that during hospitalization, the observed VTE rate for an IMPROVE-RAM score of 2 or 3 points (1.5%) and 4 points (5.7%) correlated with predicted VTE risk with an AUC of 0.69.31  The IMPROVE RAM was then externally validated in 2 studies. The first study reported an AUC of 0.77 based on high-, moderate-, and low-risk groups.20  The second study found good discrimination of low-risk and at-risk medical patients with an AUC of 0.70 despite using different cutoffs for risk classification, with more than two-thirds in the low-risk group not requiring prophylaxis.28  The RAM by Kucher et al was compared with the RAMs by Woller et al (4-Element and full logistic RAMs) in 1 study and was found to have the lowest AUC (0.76) among the 3 RAMs. In the internal validation cohort, the RAM by Barbar et al (Padua RAM) identified a 32-fold increased risk of VTE in the group of patients not on prophylaxis with a high score compared with a low score. It was then externally validated in 4 studies, 3 of which were described previously, including the study that compared the RAM by Barbar et al (Padua RAM) to the RAMs by Caprini et al (Caprini RAM) and Kucher et al, the study that compared Padua to the Caprini RAM alone, and the third that compared Padua to the Geneva RAM. A fourth study assessed the Padua RAM alone and found a correlation between the risk groups and in-hospital mortality but not with the incidence of VTE.38  The authors suggested that the Padua RAM is better used as a general comorbidity and disease severity index score than a VTE RAM.38 Table 5 describes the findings from comparative studies of risk assessment models identified in the systematic reviews.

Impact analysis of RAMs.

Stuck et al identified impact studies.19  The authors found that 3 RAMs described in the systematic reviews by Barbar et al (Padua RAM), Nendaz et al (Geneva RAM), and Kucher et al have been assessed in terms of thromboprophylaxis rates or clinical outcomes. The impact of the RAM by Barbar et al (Padua RAM) was assessed in a single-center study and was found to improve rates of adequate prophylaxis.23  The RAM described in the systematic reviews by Nendaz et al (Geneva RAM) was included in an e-Alert system, as part of a multicenter trial, and showed that its use increased appropriate prophylaxis rates.27  Two single-center studies tested the impact of the RAM described by Kucher et al. One was a randomized trial that included the RAM in a computer-alert program and showed an increase in the use of prophylaxis and a reduction in VTE rates among at-risk patients.39  The second study confirmed that implementing a computer alert program using the RAM may increase prophylaxis rates.40  No impact studies were conducted to assess the economic impact of the RAMs.

Discussion

Summary of findings

We conducted an overview of systematic reviews to identify RAMs for VTE and bleeding in hospitalized medical patients. We identified 15 unique RAMs for VTE, 7 of which were derived from individual participant data and 8 that were developed empirically using consensus approaches, risk factors identified from a literature review, and clinical expertise. The RAMs described in the systematic reviews by Caprini et al (Caprini RAM), Nendaz et al (Geneva RAM), Spyropoulos et al (IMPROVE RAM), Kucher et al, and Barbar et al (Padua RAM) have been externally validated. The RAMs described by Barbar et al (Padua RAM), Nendaz et al (Geneva RAM), and Kucher et al have been evaluated in terms of thromboprophylaxis rates or clinical outcomes and have been reported to increase appropriate prophylaxis rates. However, their economic impact has not been assessed.

Findings in relation to the literature

Following the publication of the most recent systematic review by Stuck et al in 2017,19  additional validation and impact studies were conducted. One study used the Kucher RAM along with electronic alerts and performance audits and reported increased rates of thromboprophylaxis in high-risk patients and decreased 90-day VTE rates without an observed increase in adverse events.44,45  Also, although our overview of systematic reviews did not identify any impact assessment of the Caprini RAM, we did identify a single-center study that used the Caprini RAM as part of a multifaceted quality-improvement initiative.46  The study reported increased VTE prophylaxis rates and a reduction in hospital-acquired VTE rates with the use of the Caprini RAM.46  Another prospective comparative study that was not included in the systematic reviews was conducted to assess the performance of the Geneva, Padua, and IMPROVE RAMs on thromboprophylaxis rates in acutely ill hospitalized medical patients.47  The study reported comparable discrimination abilities with a 90-day AUC of 0.71 for the Geneva RAM and an AUC of 0.70 for both the Padua and IMPROVE RAMs in patients not on thromboprophylaxis.47  The authors of the study highlighted that the IMPROVE RAM classified more patients as low risk (two-thirds of patients) compared with the Geneva RAM (one-third of patients), but with possibly lower sensitivity and greater VTE risks.47  Also, a secondary analysis of a cohort of acutely ill hospitalized medical patients participating in a cluster RCT (The Prevention of Venous Thromboembolism Disease in Emergency Departments [PREVENU] study) was not included in the systematic reviews.48  This study aimed to assess the Caprini, IMPROVE, and Padua RAMs and compared their performance to advanced age as a stand-alone predictor.48  The study reported poor discriminative ability of the RAMs to identify non–critically ill inpatients at risk of VTE and found that the RAMs did not perform better in comparison with risk assessment using advanced age as a sole predictor.48  Our search did not capture a systematic review reporting on bleeding RAMs in hospitalized medical patients. However, a primary study by Decousus et al in 2011 reported on the development of the IMPROVE bleeding RAM for in-hospital bleeding risk in acutely ill medical patients.14  Two studies externally validated the IMPROVE bleeding RAM and found similar results to the derivation effort in which hospitalized medical patients with a score >7 were shown to have over a twofold increased risk of any bleed or major bleed compared with those with lower bleeding-risk scores.49,50 

Strengths

Our study has several strengths, including the systematic and rigorous methods, a robust search strategy, the use of broad inclusion criteria, and our duplicate and independent screening and data-abstraction process. Another strength of our study is the applicability of our findings. The RAMs identified and the description of their clinical utility informed guideline developers. The RAMs can also aid health care practitioners and health care systems in selecting RAMs to optimize shared decision-making and provide appropriate prophylactic management.

Limitations and challenges

The limitations of this study result from the included studies themselves. Huang et al applied language restrictions during the initial search18  and Stuck et al applied language restrictions during the full text evaluation.19  Stuck et al also conducted searches in only 1 database.19  These limitations may have led to missing relevant studies. Some of the studies that were missed in the overview of reviews include a study by Zakai et al in 2014 in which the authors empirically derived the Medical Inpatients and Thrombosis (MITH) RAM to assess risk of VTE in medical patients on admission.51  Another study that the overview of reviews did not capture was the IMPROVEDD RAM, in which the variable D-dimer was added to the IMPROVE RAM and showed an enhanced VTE risk discrimination in hospitalized medical patients compared with the IMPROVE RAM.52  Also, we did not identify the external validation study by Greene et al in 2016. This study aimed to validate the Kucher, Padua, IMPROVE, and Intermountain RAMs.53  The authors reported that all RAMs showed good calibration but uniformly poor discrimination.53 

Implications for practice

The RAMs identified in our study can be used to estimate baseline risks of future health outcomes in people with a given disease or health condition54  and to aid health care practitioners in identifying an individual patient’s risk of VTE based on their individual characteristics. Also, many of these RAMs can be readily embedded in clinical decision aids to individualize the population-based recommendations. However, numerous shortcomings have limited the interpretation and clinical utility of the developed RAMs. First, the inability to more accurately identify medically ill patients at low or high risk of VTE may lead to overuse or underuse of prophylaxis and increased adverse events such as bleeding or thrombosis.7  Second, the complexity of some of the current RAMs (due to the large number of risk factors, such as 39 variables in Caprini and 86 in the full logistic RAM) limit their use to computer-based calculations.24,33  Third, the variability in the derivation and validation methods of the RAMs limit their comparability. Fourth, in the 2 systematic reviews, some RAMs were not found to be externally validated, limiting their use in clinical practice.29,31  Also, other RAMs have been validated in very specific medical subpopulations such as those with sepsis or cancer as noted in the Padua and Caprini RAMs, thus limiting the generalizability of their results to the general medical population.21,38  Fifth, there is a limited number of prospective comparative studies that assess the impact of applying different RAMs in clinical practice on outcomes.

Implications for future research

In this study, we provide the original and the update of the overview of systematic reviews evaluating VTE and bleeding RAMs used in hospitalized medical patients that informed the ASH guidelines. Findings from the original search directed us to conduct several follow-up studies. First, our work led us to conduct a systematic review of prognostic studies to identify all potential risk factors for VTE and bleeding in hospitalized medical patients.55  Second, we assessed the certainty of the evidence of the identified risk factors using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach.55  Third, we aimed to develop RAMs for VTE and bleeding that are accurate and usable in clinical practice.56  This was done by using the results of the systematic review and the clinical and methodological input of an expert panel.55,56  The expert panel made judgments on whether to include, potentially include, or exclude the identified risk factors from the final RAMs by considering GRADE evidence-to-decision framework criteria using the Delphi method.56  Fourth, we standardized the definitions of the identified included and potentially included risk factors based on our systematic review to decrease variability in methods of measurement across settings and to provide more clarity to health care professionals when evaluating patients’ risks of VTE and bleeding.56 

Conclusion

We conducted an overview of systematic reviews of VTE RAMs in hospitalized medical patients to inform the 2018 ASH guidelines on VTE prophylaxis.4  Our findings can assist experts in selecting a RAM to integrate in their health care systems, although further effort should be made to enhance these existing RAMs. This will allow for standardized approaches to estimating patients’ risks of VTE and individualization of population-based guideline recommendations for appropriate prevention strategies.

Table 5.

Findings from comparative studies of RAMs identified in the systematic reviews

Comparative studies (no. of centers)RAMAUCProportion of low-risk patients of VTE, %VTE incidence
 No. of patients in studyOverall, at 3 mo, %In low-risk patients, %In high-risk patients, %
Woller et al, 2011 (22 centers)32       
 • DC, 143 975
• VC, 46 846 Full logistic RAM DC, 0.893
VC, 0.861 
Not assessed DC, 3.7
VC, 4.5 
Not assessed Not assessed 
 • DC, 143 975
• VC, 46 846 4 Element RAM DC, 0.874
VC, 0.843 
Not assessed DC, 3.7
VC, 4.5 
Not assessed Not assessed 
 • DC, 143 975 
• VC, 46 846 Kucher RAM DC, 0.781
VC, 0.756 
Not assessed DC, 3.7
VC, 4.5 
Not assessed Not assessed 
Zhou et al, 201235    
 Not described Caprini RAM
Kucher RAM
Padua RAM 
Not described Caprini was found to classify more VTE patients into high or highest risk groups compared with the Kucher and Padua RAMs 
Zhou et al, 201434       
 Not described Caprini RAM
Padua RAM 
Not described 17.7
60.9 
Not described Not described Not described 
Nendaz et al, 2014 (8 centers)26       
 VC, 1478 Geneva RAM Negative likelihood ratio to identify low risk patients who do not require thromboprophylaxis, 0.28 35 2.3 0.6 3.25 
 VC, 1478 Padua RAM Negative likelihood ratio to identify low risk patients who do not require thromboprophylaxis, 0.51 52 2.3 1.1 3.5 
Comparative studies (no. of centers)RAMAUCProportion of low-risk patients of VTE, %VTE incidence
 No. of patients in studyOverall, at 3 mo, %In low-risk patients, %In high-risk patients, %
Woller et al, 2011 (22 centers)32       
 • DC, 143 975
• VC, 46 846 Full logistic RAM DC, 0.893
VC, 0.861 
Not assessed DC, 3.7
VC, 4.5 
Not assessed Not assessed 
 • DC, 143 975
• VC, 46 846 4 Element RAM DC, 0.874
VC, 0.843 
Not assessed DC, 3.7
VC, 4.5 
Not assessed Not assessed 
 • DC, 143 975 
• VC, 46 846 Kucher RAM DC, 0.781
VC, 0.756 
Not assessed DC, 3.7
VC, 4.5 
Not assessed Not assessed 
Zhou et al, 201235    
 Not described Caprini RAM
Kucher RAM
Padua RAM 
Not described Caprini was found to classify more VTE patients into high or highest risk groups compared with the Kucher and Padua RAMs 
Zhou et al, 201434       
 Not described Caprini RAM
Padua RAM 
Not described 17.7
60.9 
Not described Not described Not described 
Nendaz et al, 2014 (8 centers)26       
 VC, 1478 Geneva RAM Negative likelihood ratio to identify low risk patients who do not require thromboprophylaxis, 0.28 35 2.3 0.6 3.25 
 VC, 1478 Padua RAM Negative likelihood ratio to identify low risk patients who do not require thromboprophylaxis, 0.51 52 2.3 1.1 3.5 

Data-sharing requests may be e-mailed to the corresponding author, Holger J. Schünemann, at schuneh@mcmaster.ca.

Acknowledgments

This overview of systematic reviews was conducted to inform the American Society of Hematology (ASH) guidelines for VTE prophylaxis in hospitalized medical patients.

This work was supported by ASH.

Authorship

Contribution: A.J.D., E.A.A., and H.J.S. conceived and designed the study; A.J.D., A.B.R., R.Z.M., R.C., I.E.-I., S.G.K., A.A., T.L., R.W., E.A.A., and H.J.S. acquired data; A.J.D., A.B.R., and H.J.S. analyzed data and drafted the manuscript; and A.J.D., A.B.R., F.A.S., R.Z.M., R.C., I.E.-I., K.A.B., A.E.B., M.C., F.D., S.R.K., S.M.R., N.A.Z., A.A., S.G.K., T.L., W.W., R.W., A.I., E.A.A., and H.J.S. interpreted results, critically revised the manuscript, and approved the final version of the manuscript.

Conflict-of-interest disclosure: All authors were members of the 2018 ASH guideline panel, members of the overview of systematic review team, or both. A.J.D., I.E.-I., A.I., E.A.A., and H.J.S. reported that they are members of the GRADE Working Group. H.J.S. is cochair of the GRADE Working Group. A.J.D., F.A.S., M.C., N.A.Z., A.A., S.G.K., A.I., E.A.A., and H.J.S. were authors on a manuscript that conceptualized and tested a novel approach to selecting prognostic factors for the development or update of a RAM that has been accepted in the journal Blood Advances. H.J.S. reported being cochair of the ASH 2018 guidelines for prophylaxis for hospitalized and nonhospitalized medical patients and reported grant funding from the Centers for Disease Control and Prevention (CDC) for the project on identifying and selecting VTE and bleeding prognostic factors for the development or update of a RAM using a novel approach. M.C. reported being a former board director (2013-2017) of the American Heart Association and chairing the ASH 2018 guidelines for management of VTE: prophylaxis for hospitalized and nonhospitalized medical patients. F.A.S., A.B.R., K.A.B., A.E.B., F.D., S.R.K., S.M.R., and N.A.Z. reported participating as panel members for the ASH 2018 guidelines for management of VTE: prophylaxis for hospitalized and nonhospitalized medical patients, for which they disclosed their conflicts that are provided in the supplement of the published guideline. N.A.Z. also reported receiving honoraria in 2017 from ASH for the Highlights of ASH 2017 meeting (Dallas, New York, Latin America). N.A.Z. and M.C. reported intellectual conflicts as leads in the group that derived and validated the MITH RAM for VTE risk assessment in hospitalized medical patients. The remaining authors declare no competing financial interests.

Correspondence: Holger J. Schünemann, Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main St West; Hamilton, ON L8N 3Z5, Canada; e-mail: schuneh@mcmaster.ca.

References

References
1.
Cohoon
KP
,
Leibson
CL
,
Ransom
JE
, et al
.
Costs of venous thromboembolism associated with hospitalization for medical illness
.
Am J Manag Care
.
2015
;
21
(
4
):
e255
-
e263
.
2.
Heit
JA
,
Spencer
FA
,
White
RH
.
The epidemiology of venous thromboembolism
.
J Thromb Thrombolysis
.
2016
;
41
(
1
):
3
-
14
.
3.
Cushman
M
.
Epidemiology and risk factors for venous thrombosis
.
Semin Hematol
.
2007
;
44
(
2
):
62
-
69
.
4.
Schünemann
HJ
,
Cushman
M
,
Burnett
AE
, et al
.
American Society of Hematology 2018 guidelines for management of venous thromboembolism: prophylaxis for hospitalized and nonhospitalized medical patients
.
Blood Adv
.
2018
;
2
(
22
):
3198
-
3225
.
5.
Venous thromboembolism epidemiology: implications for prevention and management
.
Semin Thromb Hemost
.
2002
;
28
(
suppl 2
):
3
-
13
.
6.
Heit
JA
,
O’Fallon
WM
,
Petterson
TM
, et al
.
Relative impact of risk factors for deep vein thrombosis and pulmonary embolism: a population-based study
.
Arch Intern Med
.
2002
;
162
(
11
):
1245
-
1248
.
7.
Chan
NC
,
Gross
PL
,
Weitz
JI
.
Addressing the burden of hospital-related venous thromboembolism: the role of extended anticoagulant prophylaxis
.
J Thromb Haemost
.
2018
;
16
(
3
):
413
-
417
.
8.
Ye
F
,
Bell
LN
,
Mazza
J
,
Lee
A
,
Yale
SH
.
Variation in definitions of immobility in pharmacological thromboprophylaxis clinical trials in medical inpatients: a systematic review
.
Clin Appl Thromb Hemost
.
2018
;
24
(
1
):
13
-
21
.
9.
Grosse
SD
,
Nelson
RE
,
Nyarko
KA
,
Richardson
LC
,
Raskob
GE
.
The economic burden of incident venous thromboembolism in the United States: a review of estimated attributable healthcare costs
.
Thromb Res
.
2016
;
137
:
3
-
10
.
10.
Steyerberg
EW
,
Moons
KG
,
van der Windt
DA
, et al;
PROGRESS Group
.
Prognosis Research Strategy (PROGRESS) 3: prognostic model research
.
PLoS Med
.
2013
;
10
(
2
):
e1001381
.
11.
Schünemann
HJ
,
Wiercioch
W
,
Etxeandia
I
, et al
.
Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise
.
CMAJ
.
2014
;
186
(
3
):
E123
-
E142
.
12.
Brożek
JL
,
Akl
EA
,
Alonso-Coello
P
, et al;
GRADE Working Group
.
Grading quality of evidence and strength of recommendations in clinical practice guidelines. Part 1 of 3. An overview of the GRADE approach and grading quality of evidence about interventions
.
Allergy
.
2009
;
64
(
5
):
669
-
677
.
13.
Higgins
JPT
,
Green
S
,
eds
. Cochrane Handbook for Systematic Reviews of Interventions.
Chichester, United Kingdom
:
John Wiley & Sons
;
2011
.
14.
Decousus
H
,
Tapson
VF
,
Bergmann
J-F
, et al;
IMPROVE Investigators
.
Factors at admission associated with bleeding risk in medical patients: findings from the IMPROVE investigators
.
Chest
.
2011
;
139
(
1
):
69
-
79
.
15.
Whiting
P
,
Savović
J
,
Higgins
JP
, et al;
ROBIS group
.
ROBIS: a new tool to assess risk of bias in systematic reviews was developed
.
J Clin Epidemiol
.
2016
;
69
:
225
-
234
.
16.
Aromataris
E
,
Fernandez
R
,
Godfrey
CM
,
Holly
C
,
Khalil
H
,
Tungpunkom
P
.
Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach
.
Int J Evid Based Healthc
.
2015
;
13
(
3
):
132
-
140
.
17.
Moher
D
,
Shamseer
L
,
Clarke
M
, et al;
PRISMA-P Group
.
Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement
.
Syst Rev
.
2015
;
4
(
1
):
1
.
18.
Huang
W
,
Anderson
FA
,
Spencer
FA
,
Gallus
A
,
Goldberg
RJ
.
Risk-assessment models for predicting venous thromboembolism among hospitalized non-surgical patients: a systematic review
.
J Thromb Thrombolysis
.
2013
;
35
(
1
):
67
-
80
.
19.
Stuck
AK
,
Spirk
D
,
Schaudt
J
,
Kucher
N
.
Risk assessment models for venous thromboembolism in acutely ill medical patients. A systematic review
.
Thromb Haemost
.
2017
;
117
(
4
):
801
-
808
.
20.
Mahan
CE
,
Liu
Y
,
Turpie
AG
, et al
.
External validation of a risk assessment model for venous thromboembolism in the hospitalised acutely-ill medical patient (VTE-VALOURR)
.
Thromb Haemost
.
2014
;
112
(
4
):
692
-
699
.
21.
Abdel-Razeq
HN
,
Hijjawi
SB
,
Jallad
SG
,
Ababneh
BA
.
Venous thromboembolism risk stratification in medically-ill hospitalized cancer patients. A comprehensive cancer center experience
.
J Thromb Thrombolysis
.
2010
;
30
(
3
):
286
-
293
.
22.
Alikhan
R
,
Cohen
AT
,
Combe
S
, et al;
MEDENOX Study
.
Risk factors for venous thromboembolism in hospitalized patients with acute medical illness: analysis of the MEDENOX Study
.
Arch Intern Med
.
2004
;
164
(
9
):
963
-
968
.
23.
Barbar
S
,
Noventa
F
,
Rossetto
V
, et al
.
A risk assessment model for the identification of hospitalized medical patients at risk for venous thromboembolism: the Padua Prediction Score
.
J Thromb Haemost
.
2010
;
8
(
11
):
2450
-
2457
.
24.
Caprini
,
JA
,
Arcelus
,
JI
,
Reyna
J
.
Effective risk stratification of surgical and nonsurgical patients for venous thromboembolic disease
.
Semin Hematol
.
2001
;
38
(
2 suppl 5
):
12
-
19
.
25.
Cohen
AT
,
Alikhan
R
,
Arcelus
JI
, et al
.
Assessment of venous thromboembolism risk and the benefits of thromboprophylaxis in medical patients
.
Thromb Haemost
.
2005
;
94
(
4
):
750
-
759
.
26.
McCaffrey
R
,
Bishop
M
,
Adonis-Rizzo
M
, et al
.
Development and testing of a DVT risk assessment tool: providing evidence of validity and reliability
.
Worldviews Evid Based Nurs
.
2007
;
4
(
1
):
14
-
20
.
27.
Nendaz
M
,
Spirk
D
,
Kucher
N
, et al
.
Multicentre validation of the Geneva Risk Score for hospitalised medical patients at risk of venous thromboembolism. Explicit ASsessment of Thromboembolic RIsk and Prophylaxis for Medical PATients in SwitzErland (ESTIMATE)
.
Thromb Haemost
.
2014
;
111
(
3
):
531
-
538
.
28.
Rosenberg
D
,
Eichorn
A
,
Alarcon
M
,
McCullagh
L
,
McGinn
T
,
Spyropoulos
AC
.
External validation of the risk assessment model of the International Medical Prevention Registry on Venous Thromboembolism (IMPROVE) for medical patients in a tertiary health system
.
J Am Heart Assoc
.
2014
;
3
(
6
):
e001152
.
29.
Rothberg
MB
,
Lindenauer
PK
,
Lahti
M
,
Pekow
PS
,
Selker
HP
.
Risk factor model to predict venous thromboembolism in hospitalized medical patients
.
J Hosp Med
.
2011
;
6
(
4
):
202
-
209
.
30.
Samama
MM
,
Dahl
OE
,
Mismetti
P
, et al
.
An electronic tool for venous thromboembolism prevention in medical and surgical patients
.
Haematologica
.
2006
;
91
(
1
):
64
-
70
.
31.
Spyropoulos
AC
,
Anderson
FA
Jr.
,
FitzGerald
G
, et al;
IMPROVE Investigators
.
Predictive and associative models to identify hospitalized medical patients at risk for VTE
.
Chest
.
2011
;
140
(
3
):
706
-
714
.
32.
Weill-Engerer
S
,
Meaume
S
,
Lahlou
A
, et al
.
Risk factors for deep vein thrombosis in inpatients aged 65 and older: a case-control multicenter study
.
J Am Geriatr Soc
.
2004
;
52
(
8
):
1299
-
1304
.
33.
Woller
SC
,
Stevens
SM
,
Jones
JP
, et al
.
Derivation and validation of a simple model to identify venous thromboembolism risk in medical patients
.
Am J Med
.
2011
;
124
(
10
):
947
-
954.e2
.
34.
Yale
SH
,
Medlin
SC
,
Liang
H
,
Peters
T
,
Glurich
I
,
Mazza
JJ
.
Risk assessment model for venothromboembolism in post-hospitalized patients
.
Int Angiol
.
2005
;
24
(
3
):
250
-
254
.
35.
Zhou
H
,
Wang
L
,
Wu
X
, et al
.
Validation of a venous thromboembolism risk assessment model in hospitalized Chinese patients: a case-control study
.
J Atheroscler Thromb
.
2014
;
21
(
3
):
261
-
272
.
36.
Zhou
H-X
,
Peng
L-Q
,
Yan
Y
, et al
.
Validation of the Caprini risk assessment model in Chinese hospitalized patients with venous thromboembolism
.
Thromb Res
.
2012
;
130
(
5
):
735
-
740
.
37.
Rocha
AT
,
Paiva
EF
,
Lichtenstein
A
,
Milani
R
Jr.
,
Cavalheiro
CF
,
Maffei
FH
.
Risk-assessment algorithm and recommendations for venous thromboembolism prophylaxis in medical patients
.
Vasc Health Risk Manag
.
2007
;
3
(
4
):
533
-
553
.
38.
Vardi
M
,
Ghanem-Zoubi
NO
,
Zidan
R
,
Yurin
V
,
Bitterman
H
.
Venous thromboembolism and the utility of the Padua Prediction Score in patients with sepsis admitted to internal medicine departments
.
J Thromb Haemost
.
2013
;
11
(
3
):
467
-
473
.
39.
Kucher
N
,
Koo
S
,
Quiroz
R
, et al
.
Electronic alerts to prevent venous thromboembolism among hospitalized patients
.
N Engl J Med
.
2005
;
352
(
10
):
969
-
977
.
40.
Baroletti
S
,
Munz
K
,
Sonis
J
, et al
.
Electronic alerts for hospitalized high-VTE risk patients not receiving prophylaxis: a cohort study
.
J Thromb Thrombolysis
.
2008
;
25
(
2
):
146
-
150
.
41.
Rossetto
V
,
Barbar
S
,
Vedovetto
V
,
Milan
M
,
Prandoni
P
.
Physicians’ compliance with the Padua Prediction Score for preventing venous thromboembolism among hospitalized medical patients
.
J Thromb Haemost
.
2013
;
11
(
7
):
1428
-
1430
.
42.
Chopard
P
,
Spirk
D
,
Bounameaux
H
.
Identifying acutely ill medical patients requiring thromboprophylaxis
.
J Thromb Haemost
.
2006
;
4
(
4
):
915
-
916
.
43.
Nendaz
MR
,
Chopard
P
,
Lovis
C
, et al
.
Adequacy of venous thromboprophylaxis in acutely ill medical patients (IMPART): multisite comparison of different clinical decision support systems
.
J Thromb Haemost
.
2010
;
8
(
6
):
1230
-
1234
.
44.
Woller
SC
,
Stevens
SM
,
Evans
RS
, et al
.
Electronic alerts, comparative practitioner metrics, and education improve thromboprophylaxis and reduce venous thrombosis in community hospitals
.
Res Pract Thromb Haemost
.
2018
;
2
(
3
):
481
-
489
.
45.
Woller
SC
,
Stevens
SM
,
Evans
RS
, et al
.
Electronic alerts, comparative practitioner metrics, and education improves thromboprophylaxis and reduces thrombosis
.
Am J Med
.
2016
;
129
(
10
):
1124.e17
-
1124.e26
.
46.
Bullock-Palmer
RP
,
Weiss
S
,
Hyman
C
.
Innovative approaches to increase deep vein thrombosis prophylaxis rate resulting in a decrease in hospital-acquired deep vein thrombosis at a tertiary-care teaching hospital
.
J Hosp Med
.
2008
;
3
(
2
):
148
-
155
.
47.
Blondon
M
,
Spirk
D
,
Kucher
N
, et al
.
Comparative performance of clinical risk assessment models for hospital-acquired venous thromboembolism in medical patients
.
Thromb Haemost
.
2018
;
118
(
1
):
82
-
89
.
48.
Moumneh
T
,
Riou
J
,
Douillet
D
, et al
.
Validation of risk assessment models predicting venous thromboembolism in acutely ill medical inpatients: a cohort study
.
J Thromb Haemost
.
2020
;
18
(
6
):
1398
-
1407
.
49.
Hostler
DC
,
Marx
ES
,
Moores
LK
, et al
.
Validation of the International Medical Prevention Registry on Venous Thromboembolism Bleeding Risk Score
.
Chest
.
2016
;
149
(
2
):
372
-
379
.
50.
Rosenberg
DJ
,
Press
A
,
Fishbein
J
, et al
.
External validation of the IMPROVE Bleeding Risk Assessment Model in medical patients
.
Thromb Haemost
.
2016
;
116
(
3
):
530
-
536
.
51.
Zakai
NA
,
Callas
PW
,
Repp
AB
,
Cushman
M
.
Venous thrombosis risk assessment in medical inpatients: the medical inpatients and thrombosis (MITH) study
.
J Thromb Haemost
.
2013
;
11
(
4
):
634
-
641
.
52.
Gibson
CM
,
Spyropoulos
AC
,
Cohen
AT
, et al
.
The IMPROVEDD VTE risk score: incorporation of D-Dimer into the IMPROVE score to improve venous thromboembolism risk stratification
.
TH Open
.
2017
;
1
(
1
):
e56
-
e65
.
53.
Greene
MT
,
Spyropoulos
AC
,
Chopra
V
, et al
.
Validation of risk assessment models of venous thromboembolism in hospitalized medical patients
.
Am J Med
.
2016
;
129
(
9
):
1001.e9
-
1001.e18
.
54.
Hemingway
H
,
Croft
P
,
Perel
P
, et al;
PROGRESS Group
.
Prognosis research strategy (PROGRESS) 1: a framework for researching clinical outcomes
.
BMJ
.
2013
;
346
:
e5595
.
55.
Darzi
AJ
,
Karam
SG
,
Charide
R
, et al
.
Prognostic factors for VTE and bleeding in hospitalized medical patients: a systematic review and meta-analysis
.
Blood
.
2020
;
135
(
20
):
1788
-
1810
.
56.
Darzi
AJ
,
Karam
SG
,
Spencer
FA
, et al
.
Risk models for VTE and bleeding in medical inpatients: systematic identification and expert assessment
.
Blood Adv
.
2020
;
4
(
12
):
2557
-
2566
.

Author notes

*

A.J.D. and A.B.R. contributed equally.

The full-text version of this article contains a data supplement.

Supplemental data

Comments

0 Comments