ºìÐÓÊÓƵ

Guidelines for Systematic Reviews in the Context of Doctoral Education Background

The University of Victoria, School of Nursing (SON) is becoming a Joanna Briggs Collaborating Centre that promotes and supports the synthesis, transfer and utilization of evidence. Questions have arisen concerning the role of systematic reviews in doctoral dissertations. These guidelines provide direction for responding to the following questions: does a systematic review, in whole or in part, meet the criteria for a dissertation at the SON? Where and how might a systematic review fit within doctoral work? 

What is a Systematic Review?

A literature review is often required as a precursor to (or part of) graduate work to inform further research activities and identify potential gaps in the literature on a particular topic. However, the literature review landscape includes numerous terms referring to differing types of reviews which can create confusion for students who are planning to complete a particular type of literature review (Cooper, 2010; Hannes & Lockwood, 2012; Saini & Shlonsky, 2012). It is important to elucidate differences between a systematic review and other types of reviews that are commonly referred to in the literature (Dixon-Woods, Agarwal, Jones, Young & Sutton, 2005; Mallidou, 2014). Traditionally, reviews that have been completed (including those done for PhD dissertations) have varied in terms of the explicitness and transparency of the process of searching, appraising, and synthesizing the literature. Thus, there are concerns as to what literature might have not been accessed, and issues surrounding potential bias related to literature chosen for a particular review. However, there are differing ontological and epistemological underpinnings to various methods of review, which may be important for students to attend to when choosing a particular literature review method for their dissertation.

One particular type of review, a systematic review, can be understood as a specific type of review that graduate students may undertake during their studies. A systematic review is defined here as an approach to synthesizing primary research which has similar methodological rigor as a primary research study. A systematic review is a structured, comprehensive, transparent, and methodical process in which literature is rigorously identified, appraised, and synthesized (Aromataris & Pearson, 2014; Jones & Evans, 2000; Young & MacKinnon, n.d.). A systematic review requires extensive attention to various methodological details, including exhaustive literature searching and ensuring transparency of processes. Specific components of such a review include:

  • development of a clear, focused research question (PICO or PiCo),
  • preparation of a detailed protocol (which includes a preliminary review of the literature and inclusion/exclusion criteria),
  • conducting a comprehensive, replicable and transparent search of the literature,
  • screening and critically appraising the literature,
  • extracting data or findings,
  • synthesizing extracted data or findings, and
  • writing the report (Campbell Collaboration, 2014; Cochrane Collaboration, 2014; Joanna Briggs Institute, 2014; Salmond, 2012; Sandelowski & Barroso, 2007).

Significance of Systematic Reviews

The continual proliferation of primary research literature necessitates ways to synthesize a vast array of literature in order to provide comprehensive knowledge for both theoretical development and for practitioners and policy-makers (Joanna Briggs Institute, 2014; Mallidou, 2014; Cochrane Collaboration, 2014). Research synthesis also contributes to knowledge translation, as identified by the CIHR (2014). Various evidence-informed initiatives have stimulated the creation of various systematic review groups, including the Joanna Briggs Institute (2014), the Cochrane Collaboration (2014) , the EPPI-Centre (2014) and the Campbell Collaboration (2014), who provide guidance, education, and support for those engaging in systematically synthesizing literature. In the context of evidence-informed initiatives, systematic reviews contribute to development of policy and best practice documents for practitioners.

How might a systematic review fit within doctoral work?

The following benefits and limitations of utilizing systematic reviews for PhD dissertation work have been identified by students and faculty (Clark, personal communication, 23 November 2014; Daigneault, Jacob, & Ouimet, 2012; Minnie, van der Walt, , Klopper, & Cummings, 2010; Perry & Hammond, 2002; Sambunjak & Puljak, 2010) :

Benefits-Assists student to:

  • Gain a comprehensive understanding of current literature on a topic and in identifying literature gaps
  • Retain ability to engage in original research
  • Develop process-related and methodological expertise related to synthesis of primary studies
  • Develop networks of mentoring and research partnerships
  • Contribute to the global body of knowledge with publication of review
  • Foster acquisition of critical analytical skills in identifying strengths and limitations of various research designs
  • Can be incorporated into both a traditional and paper-based dissertation (i.e. chapter 1 of a dissertation)

Challenges:

  • Questions of independent work (if SR is the complete dissertation) since systematic reviews tend to be conducted by a team
  • The supervisory committee needs to work closely to determine whether the majority of the SR work is primarily done by the student. This includes ensuring that the student is the primary author of the review and implements the majority of the steps in a SR.
  • Questions of whether a systematic review builds substantially on nursing knowledge for a dissertation in nursing
  • Issue of ‘empty reviews’ – a concern arises when there are no primary studies available on a particular topic, or the inclusion criteria are too narrow. This will not create an effective learning opportunity for the student to complete a SR as dissertation work, in and of itself.
  • To support the PhD student, at least one committee member should be experienced and/or certified in the applicable SR methodology in which the student in engaging (i.e. JBI, Cochrane or another recognized approach to meta-analysis or meta-synthesis).
  • A SR requires a research team that consists of a librarian and two reviewers (e.g. one faculty and doctoral student).
  • For SRs that include a meta-analysis (not all SRs will include a meta-analysis) it may be helpful to also have a statistician on the team.
  • A pre-requisite course, training, or certification in systematic reviews should be completed by the student prior to engaging in a systematic review
  • Previous student work and experiences, including primary research or literature reviews (narrative, scoping, or critical/discursive reviews for example) conducted at the Master’s level could also inform future plans for a systematic review at the doctoral level.

Guidelines

The systematic review landscape continues to evolve, and it may be helpful for students who are considering undertaking this type of review to discuss potential benefits and limitations with their committee. Conducting a systematic review is a recognized research method for dissertation work in some disciplines. Within the PhD Program in Nursing, students have the option of completing a systematic review as part of their doctoral dissertation. While the benefits of conducting a systematic review are substantial and form a strong foundation for empirical research, not all students will need this experience as part of a program of doctoral research (i.e. a systematic review is encouraged but not required). The following guidelines can assist the doctoral student, supervisor and committee member(s) in deciding if, and how a systematic review would fit within doctoral work:

  1. A SR may become part of a doctoral student’s dissertation work. In a manuscript-based dissertation, a systematic review could constitute one of three papers. In a traditional form dissertation, a systematic review may replace the literature review section.
  2. The SR can be the methodological focus for candidacy exams (e.g. the student does not have a second methodological focus for candidacy).
  3. The supervisor and supervisory committee must agree to support the student in conducting a systematic review. The supervisor and supervisory committee are accountable for the process and quality of the systematic review.
  4. A SR contributing to a dissertation should be an original review, and not an updated review.
  5. The student will be the primary reviewer and the supervisor the secondary reviewer in the systematic review. The supervisory committee will be the advisory (support) group.
  6. The student must have solid knowledge on the research process and basic research methods (i.e. qualitative and/or quantitative) in order to be able to conduct a systematic review.
  7. Training in performing systematic reviews (e.g. JBI Training) might be required for the student prior to conducting a systematic review.
  8. The student should also engage in research involving primary data collection/analysis during their program, such as participating in a research internship.
  9. Other types of literature reviews may also be completed by the doctoral student in consultation with the supervisor and committee members.

References

Aromataris, E., & Pearson, A. (2014). The systematic review: An overview: Synthesizing evidence to inform nursing practice. American Journal of Nursing, 114(3), 53-58. 

Campbell Collaboration. (2014). What is a systematic review? Retrieved from 

Canadian Institutes of Health Research. (2014). More about knowledge translation at CIHR. Retrieved from

Cochrane Collaboration. (2014). The Cochrane collaboration: Cochrane reviews. Retrieved from

Cooper, H. (2010). Research synthesis and meta-analysis: A step-by-step approach (4th ed.). Los Angeles, CA: Sage.

Daigneault, P., Jacob, S., & Ouimet, M. (2012). Using systematic review methods within a PhD dissertation in political science: Challenges and lessons learned from practice. International Journal of Social Research Methodology, 1-17. Retrieved from

Evidence for Policy and Practice Information and Coordinating Centre (EPPI-Centre). (2014). What is a systematic review? Retrieved from:

Hannes, K., & Lockwood, C. (2012). Synthesizing qualitative research: Choosing the right approach. Hoboken, NJ: Wiley-Blackwell.

Joanna Briggs Institute. (2014). Reviewer’s manual. Adelaide, Australia: University of Adelaide/Joanna Briggs Institute.

Mallidou, A. (2014). Mapping the landscape of knowledge synthesis. Nursing Management, 21(5), 30-39.

Minnie, K., van der Walt, C., Klopper, H., & Cummings, C. (2010, July). Systematic or integrative review as research project for graduate study. Paper presented at the International Nursing Research Congress, Orlando, FL. Retrieved from

Perry, A., & Hammond, N. (2002). Systematic reviews: The experiences of a PhD student. Psychology Learning and Teaching, 2, 32-35. Retrieved from 

Saini, M., & Shlonsky, A. (2012). Systematic synthesis of qualitative research. New York: Oxford University Press.

Salmond, S.W. (2012). Steps in the systematic review process. In C. Holly, S. Salmond, & Saimbert (Eds.), Comprehensive systematic review for advanced nursing practice (pp. 13-31). New York: Springer.

Sambunjak, S., & Puljak, L. (2010). Cochrane systematic review as a PhD thesis: An alternative with numerous advantages. Biochemia Medica, 20(3), 319-326. Retrieved from http://dx.doi. org/10.11613/BM.2010.041

Sandelowski, M., & Barroso, J. (2007). Handbook for synthesizing qualitative research. New York: Springer.

Young, L., & MacKinnon, K.A. (n.d.). Developing capacity for evidence-informed practice: The Joanna Briggs Institute comes to Victoria. Victoria, BC: University of Victoria School of Nursing.