Evaluating Research
Not all research is equal, and not all research applies to every client. Thus, it is critical that counselors be able to evaluate the quality of counseling outcome research.
Evaluating Qualitative Research
Evaluating qualitative research can be a difficult process, as there is not a single accepted format for reporting qualitative research (Merriam, 2009). This is because qualitative research is diverse and evolving. However, there are some general frameworks that can be used to guide the process of evaluating the extent to which qualitative research is useful to the consumer (i.e., you, the reader). Ellis (2000) and Tracy (2010) are examples of two such articles.
Trustworthiness, Transferability, Credibility, and Consistency
Qualitative researchers generally do not use quantitative terms like internal validity, external validity, and reliability when describing their research, as these terms are not congruent with the philosophies that underlie qualitative research. Instead, the terms trustworthiness, transferability, credibility, and consistency are likely to be used.
Trustworthiness
Trustworthiness in qualitative research refers to the extent to which one can trust the data in the project. Researchers can strengthen a study's trustworthiness through methods like triangulating data (using multiple data sources) and member checks (asking participants to review the findings to ensure they accurately represent participants' experiences).
Transferability
This concept is similar to external validity in quantitative research, as it refers to the extent to which a reader can transfer the researchers' implications or results to their own situation. While generalizability is not always a goal for qualitative researchers, researchers cans bolster transferability through providing readers with rich, thick descriptions, clearly describing the study's context, and using maximum variation sampling (intentionally drawing a diverse sample).
Credibility
Research data cannot speak for itself, rather it is interpreted through researchers who, like everyone else, have biases. Credibility is somewhat analogous to internal validity in quantitative research, as it refers to the extent to which research participants are credible as well as how accurately the researcher represented participants' experiences. Credibility is enhanced through triangulation, member checks, prolonged engagement (engaging with participants in multiple contexts and at multiple points in time), reflexivity (when a researcher reflects on their own biases and how their biases impacted the study), and peer review (asking another researcher to examine the congruence between raw data and the final report).
Consistency
Consistency, sometimes called dependability is similar to reliability in quantitative research. While most qualitative research cannot be perfectly replicated, it is possible for readers to evaluate how likely a researcher's results are across time. Consistency is strengthened through triangulation, peer review and keeping an audit trail (a detailed log of all of the research activities).
Evaluating Quantitative Research
There are a number of ways that researchers can misrepresent findings and that members of the lay public, including journalists, can misinterpret research. It is important that counselors know how to evaluate the quality of alleged research claims, so that they can use quality research to effectively guide client care while disregarding irrelevant, falsified, or fake research.
Check out Doctor Bergstrom and Dr. West's course, Calling Bullshit in the Age of Big Data, for some outstanding case studies that exemplify how quantitative research can be misrepresented and misinterpreted.