PROJECT METHODOLOGY 3

 Health Evidence Bulletins - Wales (logo)

Appaisal Checklists

Health Evidence Bulletins - Wales: Additional questions to assist with the critical appraisal of a qualitative study.
Adapted from the Critical Appraisal Skills Programme questions (CASP, Anglia and Oxford RHA, adapted from Mays N & Pope C. Rigour and qualitative research. British Medical Journal. 1995; 311: 109-112); North Thames Research Appraisal Group (NTRAG): 1998 Critical review form for reading a paper describing qualitative research and Barker, JM. Project for the enhancement of the Welsh Protocols for Investment in Health Gain. Project Methodology. Cardiff: Duthie Library. UWCM, 1996.

Paper details Authors:
Title:
Source:

A What is this paper about?
Screening questions
  Yes Can't tell No
1. Is the study relevant to the needs of the Project? continue   discard
2. Does the paper address a clearly focused issue?
Are the aims of the investigation clearly stated?		      

B/ Do I trust it?
  Yes Can't tell No
3. Is the choice of a qualitative method appropriate?
  • What was this study exploring (eg behaviour/reasoning/beliefs)?
  • Do you think a quantitative approach could have equally/better addressed this issue?
      
4. Was the author's position clearly stated?
  • Has the researcher described his/her perspective?
  • Has the researcher examined his/her role, potential bias and influence?
      
5. Was the sampling strategy clearly described and justified?
Check to see whether:
  • the method of sampling is stated or described
  • the investigators sampled the most useful or productive range of individuals and settings relevant to their question
  • the characteristics of those included in the study are defined (and are comparable to the wider population)
      
6. Was there an adequate description of the method of data collection given?
  • Is the method of data collection described and justified?
  • How were the data collected (eg audiotape/videotape/field notes)?
  • If interviews were used, were the questions pre-tested?
  • If observation was used, is the context described and were observations made in a variety of circumstances?
      
7. Were the procedures for data analysis/interpretation described and justified?
Check to see whether:
  • a description is given of how the themes and concepts were identified in the data
  • the analysis was performed by more than one researcher
  • negative/discrepant results were taken into account
  • the data were fed back to the participants for comment
      

Top

C/ What did they find?

8. What are the primary findings?
Consider whether the results:
  • address the research question
  • are likely to be clinically important
  
  Yes Can't tell No
9. Are the results credible?
  • Were sequences from the original data presented (eg quotations) and were these fairly selected?
  • Is it possible to determine the source of the data presented (eg numbering of extracts)?
  • How much of the information collected is available for independent assessment?
  • Are the explanations for the results plausible and coherent?
  • Are the results of the study compared with those from other studies?
      

Top

D/ Are the results relevant locally?

  Yes Can't tell No
10. Can the results be applied to the local situation?
  • Consider differences between the local and study populations (eg cultural, geographical, ethical) which could affect the relevance of the study.
      
11. Were all important outcomes/results considered?      
12. Accept for further use?   Refer to Team Leader  

Comments:

Draft Statement (if appropriate):
(Remember to include the relevant target group (age range, sex etc.); the measured outcomes/benefits and the health gain notation if appropriate)