Training programs to develop clinician scientists with a focus on patient-oriented research are important to ensure that research findings affect patient outcomes in a positive and meaningful manner.1,2  Traditionally, hematology training programs focused on basic and translational sciences. According to a survey of subspecialty training programs in adult and pediatric hematology and oncology, more than one third of programs devote at least 50% of the time to research efforts.3  Several National Institutes of Health working groups noted the lack of effective clinical research training programs and the lack of mentorship,4  and the American Society of Hematology (ASH) similarly identified this deficit more than a decade ago.3  In response to this need, ASH created the Clinical Research Training Institute (CRTI),5,6  and the inaugural class was held in 2003.

Measuring the impact of such training programs is necessary yet challenging. Rigorous and ongoing evaluation of a training program is important to ensure that the program is meeting its goals and that it is of value to trainees and stakeholders. Evaluation may also suggest areas in which the program needs to change or improve. However, little work has been conducted to evaluate training programs in general or in hematology more specifically. In the few published studies evaluating hematology training programs, the primary goal was to evaluate clinical training, not research outcomes.7,8 

CRTI has conducted several types of evaluations from program inception. However, a systematic evaluation of program outcomes has not been undertaken. The objective of this analysis was to evaluate the CRTI program through a single cross-sectional study of previous participants, 2 annual surveys focused on the mentorship experience and 2 annual evaluations conducted immediately before and after completion of the summer workshop. We then propose future directions to improve this evaluation process.

CRTI program

The CRTI program is a 1-year experience that begins with a weeklong workshop typically held in August. Eligibility criteria for entrance to the program include being a senior fellow or junior faculty member with an intended career in patient-oriented hematology research. Participants include clinicians focused on either malignant or benign hematology in pediatric or adult medicine. Participants primarily come from the United States or Canada, but more recently CRTI has included participants from Australia, Germany, the United Kingdom, and Singapore.

Each year, 20 participants are admitted to the program, although in some years, 1 or 2 additional participants have been included to encourage diversity. The summer workshop faculty includes ∼20 established clinical researchers, 5 or 6 biostatisticians, and representatives from key funding agencies including the National Heart, Lung, and Blood Institute and the National Cancer Institute. In recent years, a library scientist was invited.

Selection is a competitive process through a review panel of established hematology clinical researchers. Participants must submit a description of a research project with their application. The summer workshop begins and ends with a presentation of each participant’s proposal. The week consists of a mixture of didactic lecture and small group sessions that are focused on improving each proposal. Small groups are composed of 2 to 4 participants, 2 to 4 faculty members, and 1 biostatistician. Time is incorporated into the program to allow informal discussions to occur on daily hikes or evening sessions.5 

The CRTI program also involves the small groups reconvening twice: first during the annual ASH meeting in December, and then the following May at ASH headquarters. Small group reunions focus on participant presentations of updates related to their projects and career development. Recognizing the importance of mentorship, CRTI instituted a formal mentorship program in 2011 in which each participant is matched with a CRTI faculty member. Contact is prescribed to occur a minimum of 4 times throughout the year but may occur more frequently.

CRTI evaluation surveys

Three sets of surveys formed the basis of this analysis. First was a cross-sectional survey conducted in 2014 that was designed to capture outcomes (number of grants, publications, and current involvement in research) within the previous year from the date the survey was disseminated. Participants in CRTI over 8 years (2005-2012) were included in this survey and were asked 16 questions that captured demographic characteristics and the perceived influence of CRTI participation on his or her career in addition to outcomes. For this survey, our objectives were to describe academic success by CRTI graduates, defined as the number of peer-reviewed articles published, the number of research grants obtained, and whether the graduate was still involved in clinical research over the previous year. We also wanted to determine whether success was associated with strong agreement that CRTI facilitated his or her career development as an independent researcher and strong agreement that CRTI was instrumental to retaining him or her in hematology research in addition to demographic variables.

The second set of analyses consisted of annual pre– and post–summer workshop surveys conducted from 2009 to 2015. Not all questions were asked each year, and the response scale was not consistent; therefore, this analysis focused on data from 2014 to 2015. Questions about the impact of the summer workshop were asked on a 5-point Likert scale. For this analysis, we sought to determine whether confidence in research conduct improved immediately after the summer workshop in a variety of domains compared with baseline responses obtained at the beginning of the workshop.

The last analysis included surveys distributed in 2012 and 2013 that sought to evaluate the mentorship program established in 2011. This survey was not distributed in 2014 or 2015. Our objective was to describe products resulting from interaction with the mentor from the participant’s perspective in terms of academic productivity or career opportunities.

Covariates

The following demographic covariates were captured at the time of application to the program: position during CRTI (fellow vs faculty), sex, adult vs pediatric, and benign vs malignant.

Statistical analysis

For the cross-sectional survey, descriptive statistics including percentages and medians were calculated for all demographic, predictor, and outcome variables. To describe the relationship between the outcome of number of grants (0 vs ≥1) and the dichotomous demographic and predictor covariates, a χ2 test was performed. The Wilcoxon 2-sample test was used to examine the relationship between distribution of the number of articles and percent effort in research with the dichotomous demographic and predictor covariates. The pre/post questions assessing agreement or confidence in a variety of domains were dichotomized to strongly agree vs agree, neutral, disagree, and strongly disagree. Then the differences in the percentage of strongly agree between pre– and post–summer workshop were compared using a McNemar test. To describe the benefits of mentorship, the percentage that responded yes in each area was reported. P < .05 was used to define statistical significance. SAS version 9.4 was used for all statistical analysis (SAS Institute, Cary, NC).

Demographics

The 2014 cross-sectional survey was distributed to 160 participants from 2005 to 2012, and 115 responded (72% response rate; 6 responded anonymously; therefore, no demographic data are available for these respondents). Of the 109 respondents, 42% had a benign hematology focus, 65% treated adults, 48% were male, 69% were Caucasian, 28% were Asian, and 3% were African American (Table 1). At the time of CRTI, 79% were fellows, and 57% participated in 2009 or later. Ninety percent described having an academic career at the time of survey completion.

Table 1.

Demographics and current status of respondents to cross-sectional survey in 2014 (classes 2005-2012)

Characteristicn = 115 (%)
Demographic characteristics*  
 Benign vs malignant hematology focus 46/109 (42) 
 Adult vs pediatric focus 71/109 (65) 
 Male sex 52/109 (48) 
Race  
 African American 3/109 (3) 
 Asian 31/109 (28) 
 Caucasian 75/109 (69) 
 Hispanic 3/109 (3) 
Position at CRTI  
 Faculty 21/100 (21) 
 Fellow 79/100 (79) 
 Participated in 2009 or later 64/115 (57) 
Current status at survey  
 Changed positions in previous year 27 (23) 
 Received additional degree(s) 12 (10) 
 Serving as a study principal investigator or study chair 83 (72) 
 Serving as a study co-investigator 96 (83) 
 Serving as institutional principal investigator 80 (70) 
 Grant reviewer 39 (34) 
 Journal reviewer 81 (70) 
Current career  
 Academic 104 (90) 
 Government 1 (1) 
 Industry 2 (2) 
 Private 5 (4) 
 Other or missing 3 (3) 
Teaching research courses to any trainee 57 (50) 
Contact with CRTI mentor 66 (57) 
Characteristicn = 115 (%)
Demographic characteristics*  
 Benign vs malignant hematology focus 46/109 (42) 
 Adult vs pediatric focus 71/109 (65) 
 Male sex 52/109 (48) 
Race  
 African American 3/109 (3) 
 Asian 31/109 (28) 
 Caucasian 75/109 (69) 
 Hispanic 3/109 (3) 
Position at CRTI  
 Faculty 21/100 (21) 
 Fellow 79/100 (79) 
 Participated in 2009 or later 64/115 (57) 
Current status at survey  
 Changed positions in previous year 27 (23) 
 Received additional degree(s) 12 (10) 
 Serving as a study principal investigator or study chair 83 (72) 
 Serving as a study co-investigator 96 (83) 
 Serving as institutional principal investigator 80 (70) 
 Grant reviewer 39 (34) 
 Journal reviewer 81 (70) 
Current career  
 Academic 104 (90) 
 Government 1 (1) 
 Industry 2 (2) 
 Private 5 (4) 
 Other or missing 3 (3) 
Teaching research courses to any trainee 57 (50) 
Contact with CRTI mentor 66 (57) 
*

For 6 participants, only year of participation is known. For 9 additional participants, status at CRTI is unknown.

Facilitations owing to CRTI and description of academic success

Table 2 illustrates that 64% of respondents had some current collaborations as a result of CRTI, with most collaborations being with CRTI faculty (46%). Table 2 also illustrates that 63% of respondents strongly agreed that CRTI facilitated their career development as an independent researcher.

Table 2.

Outcomes reported in cross-sectional survey in 2014 (classes 2005-2012)

Activities facilitated because of CRTIn = 115
Collaborations created related CRTI, n (%)  
 With CRTI faculty 53 (46) 
 With CRTI coparticipants 38 (33) 
 With non-CRTI participants but because of CRTI 17 (15) 
 Any collaborations 74 (64) 
Agreement that CRTI facilitated career development as an independent researcher, n (%)  
 Strongly agree 73 (63) 
 Agree 33 (29) 
 Neutral 9 (8) 
 Disagree 
 Strongly disagree 
CRTI instrumental to retaining me in hematology research, n (%)  
 Strongly agree 61 (53) 
 Agree 32 (28) 
 Neutral 19 (17) 
 Disagree 3 (3) 
 Strongly disagree 
Outcomes  
 Number of grants in previous 1 year Median 0 (IQR 0, 1) 
  0 61 (53%) 
  1 32 (28%) 
  2-7 22 (19%) 
 Number of articles in previous 1 year Median 2 (IQR 1, 4) 
 Percent effort in research Median 50 (IQR 30, 70) 
 Percent effort in patient care Median 40 (IQR 20, 50) 
 Currently involved in research 109 (95%) 
Activities facilitated because of CRTIn = 115
Collaborations created related CRTI, n (%)  
 With CRTI faculty 53 (46) 
 With CRTI coparticipants 38 (33) 
 With non-CRTI participants but because of CRTI 17 (15) 
 Any collaborations 74 (64) 
Agreement that CRTI facilitated career development as an independent researcher, n (%)  
 Strongly agree 73 (63) 
 Agree 33 (29) 
 Neutral 9 (8) 
 Disagree 
 Strongly disagree 
CRTI instrumental to retaining me in hematology research, n (%)  
 Strongly agree 61 (53) 
 Agree 32 (28) 
 Neutral 19 (17) 
 Disagree 3 (3) 
 Strongly disagree 
Outcomes  
 Number of grants in previous 1 year Median 0 (IQR 0, 1) 
  0 61 (53%) 
  1 32 (28%) 
  2-7 22 (19%) 
 Number of articles in previous 1 year Median 2 (IQR 1, 4) 
 Percent effort in research Median 50 (IQR 30, 70) 
 Percent effort in patient care Median 40 (IQR 20, 50) 
 Currently involved in research 109 (95%) 

IQR, interquartile range.

In terms of academic success in the previous year, 47% had at least 1 grant, and the median number of peer-reviewed articles was 2. Typically, respondents spent 50% of their time conducting research, and 95% were still involved in research.

Factors associated with academic success

Table 3 illustrates the relationship between having at least 1 grant in the previous year, the number of published articles in the previous year, and the percent effort in research with the participant’s reported impact of CRTI on his or her career as well as demographic variables. Respondents who strongly agreed that CRTI facilitated their career development as independent researchers were more likely to have had at least one grant in the previous year (55% vs 33%, P = .0264), more published articles (median 3 vs 2, P = .0170), and greater percent effort in research (median 50% vs 30%, P = .0002) compared with the other respondents (Table 3). Similarly, respondents who strongly agreed that CRTI was instrumental to retaining them in hematology research reported a higher median number of articles (4 vs 2, P = .0046) and a higher percent effort in research (median 60% vs 30%, P < .0001). Male respondents reported a higher median number of articles (3 vs 2, P = .0092) and a higher median percent effort in research (50% vs 40%, P = .0029) than females. Respondents who were junior faculty at the time of CRTI participation reported a higher median percent effort in research (60% vs 40%, P = .0270) than those who were fellows. Last, respondents who reported any CRTI collaborations reported a higher median number of articles (3 vs 1, P = .0006) and a higher median percent effort in research (50% vs 40%, P = .0439).

Table 3.

Predictors of success (success defined by number of grants, publications, and percent effort in research) in cross-sectional survey in 2014 (classes 2005-2012)

Number of grants (≥1 vs 0)Number of peer-reviewed published articlesPercent effort research
Covariates% ≥1P*Median (IQR)PMedian (IQR)P
Facilitated career development as an independent researcher  .0264  .0170  .0002 
 Strongly agree (n = 73) 55  3 (1, 5)  50% (40, 70)  
 Agree, disagree, strongly disagree (n = 42) 33  2 (1, 4)  30% (20, 50)  
CRTI instrumental to retaining in hematology research  .1029  .0046  <.0001 
 Strongly agree (n = 61) 54  4 (2, 5)  60% (40, 70)  
 Agree, disagree, strongly disagree (n = 54) 39  2 (1, 3)  30% (20, 50)  
Malignant  .8389  .1351  .9395 
 Malignant (n = 63) 47  3 (1, 5)  40% (30, 60)  
 Benign (n = 46) 46  2 (1, 4)  50% (20, 70)  
Male  .3049  .0092  .0029 
 Male (n = 52) 52  3 (2, 6)  50% (35, 70)  
 Female (n = 57) 42  2 (1, 4)  40% (20, 50)  
Position at CRTI faculty  .1573  .4348  .0270 
 Faculty (n = 21) 62  3 (2, 4)  60% (40, 70)  
 Fellow (n = 79) 44  2 (1, 5)  40% (30, 60)  
Any CRTI collaborations  .0972  .0006  .0439 
 Yes (n = 74) 53  3 (2, 5)  50% (30, 70)  
 No (n = 41) 37  1 (0.5, 4)  40% (20, 50)  
Class year 2009 or later  .4479  .1391  .2909 
 2009 or later (n = 66) 50  2 (1, 4)  50% (30, 70)  
 Before 2009 (n = 49) 43  3 (1.5, 5)  40% (30, 70)  
Number of grants (≥1 vs 0)Number of peer-reviewed published articlesPercent effort research
Covariates% ≥1P*Median (IQR)PMedian (IQR)P
Facilitated career development as an independent researcher  .0264  .0170  .0002 
 Strongly agree (n = 73) 55  3 (1, 5)  50% (40, 70)  
 Agree, disagree, strongly disagree (n = 42) 33  2 (1, 4)  30% (20, 50)  
CRTI instrumental to retaining in hematology research  .1029  .0046  <.0001 
 Strongly agree (n = 61) 54  4 (2, 5)  60% (40, 70)  
 Agree, disagree, strongly disagree (n = 54) 39  2 (1, 3)  30% (20, 50)  
Malignant  .8389  .1351  .9395 
 Malignant (n = 63) 47  3 (1, 5)  40% (30, 60)  
 Benign (n = 46) 46  2 (1, 4)  50% (20, 70)  
Male  .3049  .0092  .0029 
 Male (n = 52) 52  3 (2, 6)  50% (35, 70)  
 Female (n = 57) 42  2 (1, 4)  40% (20, 50)  
Position at CRTI faculty  .1573  .4348  .0270 
 Faculty (n = 21) 62  3 (2, 4)  60% (40, 70)  
 Fellow (n = 79) 44  2 (1, 5)  40% (30, 60)  
Any CRTI collaborations  .0972  .0006  .0439 
 Yes (n = 74) 53  3 (2, 5)  50% (30, 70)  
 No (n = 41) 37  1 (0.5, 4)  40% (20, 50)  
Class year 2009 or later  .4479  .1391  .2909 
 2009 or later (n = 66) 50  2 (1, 4)  50% (30, 70)  
 Before 2009 (n = 49) 43  3 (1.5, 5)  40% (30, 70)  

Bold indicates statistically significant results.

IQR, interquartile range.

*

χ2.

Wilcoxon test.

Impact of summer workshop on confidence in research conduct

In 2014 and 2015, the same pre–post survey was distributed to 41 participants, and 38 responded (93% response rate). Table 4 illustrates that there were significant gains in confidence conducting clinical research immediately after the summer workshop compared with the baseline assessment in almost all measured domains. Areas in which there was a larger increase in confidence were: understanding the principals of research design and execution, preparing research for presentation, opportunities to collaborate, and ability to develop a sound research hypothesis.

Table 4.

Evaluations immediately before and after summer workshop for 2014 and 2015

Pre* % strongly agreePost* % strongly agreePre/post P
Questions asked in 2014 and 2015 (n = 35)    
 I feel I have a comprehension of the principles of clinical research design and execution, including clinical trials with correlative science objectives. 11 71 <.0001 
 I am confident in my ability to determine the ethical implications of clinical research. 26 60 .0073 
 I am confident in my knowledge to identify the regulatory requirements of clinical research. 49 .0005 
 I am confident in my comprehension of the fundamentals of competitive grant writing. 51 .0003 
 I feel confident in my current strategies for pursuing and developing a successful career in clinical research. 60 <.0001 
 I feel confident in my skills to formulate, develop, and sustain a multidisciplinary clinical research team. 54 .0003 
 I am confident in my skills in preparing research results and presenting the work to varied audiences. 69 <.0001 
 I frequently collaborate with other trainees/peers. 26 80 <.0001 
 I have opportunities to collaborate with leaders in clinical medicine and clinical research. 37 89 <.0001 
Questions asked in 2015 only (n = 14)    
 I feel confident conducting research in my field. 29 .1797 
 I am able to develop a sound, scientific hypothesis. 85 .0005 
 I have the competence to conduct and participate in collaborative research. 21 50 .0455 
 I know where to turn for assistance with my research. 14 79 .0027 
Pre* % strongly agreePost* % strongly agreePre/post P
Questions asked in 2014 and 2015 (n = 35)    
 I feel I have a comprehension of the principles of clinical research design and execution, including clinical trials with correlative science objectives. 11 71 <.0001 
 I am confident in my ability to determine the ethical implications of clinical research. 26 60 .0073 
 I am confident in my knowledge to identify the regulatory requirements of clinical research. 49 .0005 
 I am confident in my comprehension of the fundamentals of competitive grant writing. 51 .0003 
 I feel confident in my current strategies for pursuing and developing a successful career in clinical research. 60 <.0001 
 I feel confident in my skills to formulate, develop, and sustain a multidisciplinary clinical research team. 54 .0003 
 I am confident in my skills in preparing research results and presenting the work to varied audiences. 69 <.0001 
 I frequently collaborate with other trainees/peers. 26 80 <.0001 
 I have opportunities to collaborate with leaders in clinical medicine and clinical research. 37 89 <.0001 
Questions asked in 2015 only (n = 14)    
 I feel confident conducting research in my field. 29 .1797 
 I am able to develop a sound, scientific hypothesis. 85 .0005 
 I have the competence to conduct and participate in collaborative research. 21 50 .0455 
 I know where to turn for assistance with my research. 14 79 .0027 

Bold indicates statistically significant results.

*

Responses are the percentage of respondents who strongly agreed vs agreed, neutral, disagreed, or strongly disagreed with statement.

McNemar test.

Impact of mentorship

The mentoring survey was distributed to 40 of the participants from the 2012 and 2013 cohorts, and 38 responded (95% response rate). Over the 12-month period after the initial week of training, almost three-fourths of the respondents felt that the CRTI mentor played a role in increasing their knowledge or skills in conduct of research. Some of the respondents reported that the mentor interaction resulted in posters, presentations, or manuscripts. Mentors also facilitated job changes and promotions (Table 5). In free-text comments describing other benefits of the CRTI mentor, respondents cited overcoming hurdles in patient enrollment, setting short-term goals, improving presentation skills, refining their curriculum vitae, networking, work–life balance, role-modeling, reviewing offer letters and suggesting requests, and increased awareness about funding and training opportunities.

Table 5.

Benefits observed during a 1-year mentorship program (n = 38)

Arean (%)
Mentor interaction resulted in publication 6 (16) 
Mentor interaction resulted in presentation 7 (18) 
Mentor interaction resulted in poster 4 (11) 
Mentor interaction led to me learning new teaching method or approach 4 (11) 
Mentor resulted in increased knowledge or skills in conduct of research 28 (74) 
Mentor resulted in increasing new clinical knowledge 9 (24) 
Mentor facilitated job change or promotion 6 (16) 
Mentor facilitated involvement in a clinical trials cooperative group 6 (16) 
Arean (%)
Mentor interaction resulted in publication 6 (16) 
Mentor interaction resulted in presentation 7 (18) 
Mentor interaction resulted in poster 4 (11) 
Mentor interaction led to me learning new teaching method or approach 4 (11) 
Mentor resulted in increased knowledge or skills in conduct of research 28 (74) 
Mentor resulted in increasing new clinical knowledge 9 (24) 
Mentor facilitated job change or promotion 6 (16) 
Mentor facilitated involvement in a clinical trials cooperative group 6 (16) 

In this comprehensive evaluation of ASH’s 1-year CRTI program, we found that, within a range of 2 to 9 years after the original CRTI experience, 90% of respondents remained in academic hematology and 95% were still involved in hematology research. We also found that most respondents continued to have collaborations owing to CRTI, and most respondents strongly agreed that CRTI facilitated career development and contributed to retention in hematology research. We also found that strong agreement that CRTI contributed positively to career development was significantly associated with greater academic success, and that males and faculty member respondents were more likely to have more time dedicated to research. In almost every measure of confidence in conducting clinical research, respondents significantly improved after the summer workshop.

Our finding that 90% of respondents remained in academic hematology is higher than a previous survey of medical oncology fellows, which reported that 60% remained in an academic setting.9  These high rates of retention in academic hematology and hematology research are reassuring, although it is difficult to know how much CRTI directly influenced these positive outcomes.

When exploring factors related to measures of academic productivity, we found that sex was a significant variable. Males published one more paper per year and tended to spend 10% more of their reported effort in research. This finding is consistent with previously published research in academic medicine. Among a European evaluation of fellows, more of the female scientists were married or had partners who were also in academics compared with male scientists who were married or had partners. The females who had children also typically took 2 to 3 months of leave, whereas the males did not take leave periods when their children were born.10  It is unknown whether this modest decrease in academic success among female respondents is solely related to factors associated with having and raising children. Nonetheless, this gender disparity is important to emphasize in program planning. Although we have addressed work–life balance during current CRTI sessions, we did not directly focus on gender differences. With 50% of faculty being female, many of whom are working mothers, we should encourage discussions about approaches that have been helpful to academic success from more senior female faculty.

Respondents who collaborated with either faculty or other participants from CRTI published more manuscripts and spent more time in research. It is uncertain whether this finding is confounded by the underlying attributes of those more likely to be academically successful or whether CRTI directly contributes to positive outcomes. Nonetheless, these findings suggest that we should more proactively encourage collaboration with CRTI faculty and non-CRTI investigators.

A strength of this evaluation was the multiple approaches we took to evaluate the CRTI program, including short- and medium-term outcomes. Another strength was the high response rate for all surveys.

As with any survey, this study had limitations. Respondents may have been biased to report positive feelings or gratitude for a program that was provided at no cost to them. However, given the duration of time between CRTI participation and the 2014 cross-sectional survey, we anticipate that most of the “honeymoon period” positivity would have subsided. Another limitation was our lack of consistently worded surveys, particularly for the pre–post summer workshop survey, limiting our ability to evaluate the program over a prolonged period. However, as CRTI matures, we plan on improving the standardization of assessments going forward. Third, although we reported that 95% of respondents are still involved in research, our definition of involvement was not well defined, and future surveys will more carefully define and measure how respondents are involved in research. Fourth, although we were encouraged by our response rate of 72% to the cross-sectional survey, nonresponders may have differed from responders and biased our results. Finally, we lacked an appropriate control group, and a common criticism of an analysis of CRTI benefits is that CRTI respondents were destined to be successful because they applied to the program and were selected as the most promising applicants to the program. Nonetheless, our comparison within our cohort to identify factors associated with academic success is an important and novel approach to program evaluation and improvement.

Recommendations

In the future, we recommend several changes to increase participation in repeated and longer-term follow-up evaluations of CRTI participants. We are creating a CRTI “community.” For the past 2 years, we have emphasized the planned repeated interactions of CRTI participants for not only evaluation but also celebrations of success. Plans for social media groups, a newsletter, and gatherings at the annual meeting will allow us to check contact information and encourage participation. We have edited the surveys for uniformity in rating scales and rewritten questions to capture objectives of the training program (grant writing, research design, etc.). Recognizing that a 1-year follow-up may be too short to capture the impact of CRTI, we recommend that the interval for repeated follow-up evaluation occur 3 years after attendance of the summer didactic session. Within this timeframe, we anticipate that participants will be able to submit career development awards and conduct some of their proposed projects.

The diversity of CRTI participants and respondents was low, and we have made efforts to increase the participation of underrepresented minorities to 15%. We have included representatives from ASH’s Committees on Promoting Diversity on the CRTI Oversight Committee to disseminate information about CRTI to more diverse members. We have presented to the Committee on Training to reach training program directors and asked diverse alumni to recommend CRTI to diverse fellows and junior faculty. We recommend that these efforts continue and encourage the course directors to continue to maintain a similar proportion of underrepresented minorities on the faculty as well.

In conclusion, the ASH CRTI is a well-received program that is associated with an increased retention of academic hematologists and increased continued participation in patient-oriented research compared with typical fellowship-trained hematologists. This evaluation, despite its limitations, supports the continued efforts to mentor and train junior academic hematologists in a structured approach. Efforts should focus on reducing gender disparity and enhancing CRTI collaborations. In the future, more uniform and longitudinal assessments will inform our methods of training.

The publication costs of this article were defrayed in part by page charge payment. Therefore, and solely to indicate this fact, this article is hereby marked “advertisement” in accordance with 18 USC section 1734.

The views expressed in written conference materials or publications and by speakers and moderators do not necessarily reflect the official policies of the Department of Health and Human Services, nor does mention of trade names, commercial practices, or organizations imply endorsement by the US Government.

Funding for this conference is made possible in part by National Institutes of Health, National Heart, Lung, and Blood Institute grant 1R13HL129488-01 and National Cancer Institute grant 5R25CA168526‐03.

Contribution: A.A.K. organized the manuscript, assisted in planning the analysis, and wrote the manuscript; S.K.V. performed the analysis, wrote the analysis methods, and reviewed the manuscript; J.E. and J.B. collected the data and reviewed the manuscript; K.C. assisted with organizing the paper and planning the analysis and reviewed the manuscript; and L.S. organized, wrote and reviewed the manuscript, performed the analysis, and interpreted the data.

Conflict-of-interest disclosure: The authors declare no competing financial interests.

Correspondence: Allison A. King, Washington University School of Medicine, 4444 Forest Park Blvd, CB8505, St. Louis, MO, 63108; e-mail: king_a@wustl.edu.

1.
Lander
B
,
Hanley
GE
,
Atkinson-Grosjean
J
.
Clinician-scientists in Canada: barriers to career entry and progress
.
PLoS One
.
2010
;
5
(
10
):
e13168
.
2.
Shaywitz
DA
,
Martin
JB
,
Ausiello
DA
.
Patient-oriented research: principles and new approaches to training
.
Am J Med
.
2000
;
109
(
2
):
136
-
140
.
3.
Todd
RF
III
,
Gitlin
SD
,
Burns
LJ
;
Committee On Training Programs
.
Subspeciality training in hematology and oncology, 2003: results of a survey of training program directors conducted by the American Society of Hematology
.
Blood
.
2004
;
103
(
12
):
4383
-
4388
.
4.
Horn
LV
.
National Heart, Lung and Blood Strategic Plan Training and Career Development Implementation Workgroup Summary
.
2008
. http://www.nhlbi.nih.gov/research/training/2008-strategic-plan. Accessed 19 October 2016.
5.
Sung
L
,
Crowther
M
,
Byrd
J
,
Gitlin
SD
,
Basso
J
,
Burns
L
.
Challenges in measuring benefit of clinical research training programs—the ASH Clinical Research Training Institute example
.
J Cancer Educ
.
2015
;
30
(
4
):
754
-
758
.
6.
Burns
LJ
,
Clayton
CP
,
George
JN
,
Mitchell
BS
,
Gitlin
SD
.
The effect of an intense mentoring program on junior investigators’ preparation for a patient-oriented clinical research career
.
Acad Med
.
2015
;
90
(
8
):
1061
-
1066
.
7.
Leung
E
,
Dix
D
,
Ford
J
,
Barnard
D
,
McBride
E
.
The pediatric hematology/oncology educational laboratory in-training examination (PHOELIX): A formative evaluation of laboratory skills for Canadian pediatric hematology/oncology trainees
.
Pediatr Blood Cancer
.
2015
;
62
(
11
):
1952
-
1955
.
8.
Shumway
NM
,
Dacus
JJ
,
Lathrop
KI
,
Hernandez
EP
,
Miller
M
,
Karnad
AB
.
Use of milestones and development of entrustable professional activities in 2 hematology/oncology training programs
.
J Grad Med Educ
.
2015
;
7
(
1
):
101
-
104
.
9.
Horn
L
,
Koehler
E
,
Gilbert
J
,
Johnson
DH
.
Factors associated with the career choices of hematology and medical oncology fellows trained at academic institutions in the United States
.
J Clin Oncol
.
2011
;
29
(
29
):
3932
-
3938
.
10.
Ledin
A
,
Bornmann
L
,
Gannon
F
,
Wallon
G
.
A persistent problem. Traditional gender roles hold back female scientists
.
EMBO Rep
.
2007
;
8
(
11
):
982
-
987
.

Sign in via your Institution