research limitations and barriers

Research Limitations 101 📖

A Plain-Language Explainer (With Practical Examples)

By: Derek Jansen (MBA) | Expert Reviewer: Dr. Eunice Rautenbach | May 2024

Research limitations are one of those things that students tend to avoid digging into, and understandably so. No one likes to critique their own study and point out weaknesses. Nevertheless, being able to understand the limitations of your study – and, just as importantly, the implications thereof – a is a critically important skill.

In this post, we’ll unpack some of the most common research limitations you’re likely to encounter, so that you can approach your project with confidence.

Overview: Research Limitations 101

• What are research limitations ?
• Access – based limitations
• Temporal & financial limitations
• Sample & sampling limitations
• Design limitations
• Researcher limitations
• Key takeaways

What (exactly) are “research limitations”?

At the simplest level, research limitations (also referred to as “the limitations of the study”) are the constraints and challenges that will invariably influence your ability to conduct your study and draw reliable conclusions .

Research limitations are inevitable. Absolutely no study is perfect and limitations are an inherent part of any research design. These limitations can stem from a variety of sources , including access to data, methodological choices, and the more mundane constraints of budget and time. So, there’s no use trying to escape them – what matters is that you can recognise them.

Acknowledging and understanding these limitations is crucial, not just for the integrity of your research, but also for your development as a scholar. That probably sounds a bit rich, but realistically, having a strong understanding of the limitations of any given study helps you handle the inevitable obstacles professionally and transparently, which in turn builds trust with your audience and academic peers.

Simply put, recognising and discussing the limitations of your study demonstrates that you know what you’re doing , and that you’ve considered the results of your project within the context of these limitations. In other words, discussing the limitations is a sign of credibility and strength – not weakness. Contrary to the common misconception, highlighting your limitations (or rather, your study’s limitations) will earn you (rather than cost you) marks.

So, with that foundation laid, let’s have a look at some of the most common research limitations you’re likely to encounter – and how to go about managing them as effectively as possible.

Need a helping hand?

Limitation #1: Access To Information

One of the first hurdles you might encounter is limited access to necessary information. For example, you may have trouble getting access to specific literature or niche data sets. This situation can manifest due to several reasons, including paywalls, copyright and licensing issues or language barriers.

To minimise situations like these, it’s useful to try to leverage your university’s resource pool to the greatest extent possible. In practical terms, this means engaging with your university’s librarian and/or potentially utilising interlibrary loans to get access to restricted resources. If this sounds foreign to you, have a chat with your librarian 🙃

In emerging fields or highly specific study areas, you might find that there’s very little existing research (i.e., literature) on your topic. This scenario, while challenging, also offers a unique opportunity to contribute significantly to your field , as it indicates that there’s a significant research gap .

All of that said, be sure to conduct an exhaustive search using a variety of keywords and Boolean operators before assuming that there’s a lack of literature. Also, remember to snowball your literature base . In other words, scan the reference lists of the handful of papers that are directly relevant and then scan those references for more sources. You can also consider using tools like Litmaps and Connected Papers (see video below).

Limitation #2: Time & Money

Almost every researcher will face time and budget constraints at some point. Naturally, these limitations can affect the depth and breadth of your research – but they don’t need to be a death sentence.

Effective planning is crucial to managing both the temporal and financial aspects of your study. In practical terms, utilising tools like Gantt charts can help you visualise and plan your research timeline realistically, thereby reducing the risk of any nasty surprises. Always take a conservative stance when it comes to timelines, especially if you’re new to academic research. As a rule of thumb, things will generally take twice as long as you expect – so, prepare for the worst-case scenario.

If budget is a concern, you might want to consider exploring small research grants or adjusting the scope of your study so that it fits within a realistic budget. Trimming back might sound unattractive, but keep in mind that a smaller, well-planned study can often be more impactful than a larger, poorly planned project.

If you find yourself in a position where you’ve already run out of cash, don’t panic. There’s usually a pivot opportunity hidden somewhere within your project. Engage with your research advisor or faculty to explore potential solutions – don’t make any major changes without first consulting your institution.

Limitation #3: Sample Size & Composition

As we’ve discussed before , the size and representativeness of your sample are crucial , especially in quantitative research where the robustness of your conclusions often depends on these factors. All too often though, students run into issues achieving a sufficient sample size and composition.

To ensure adequacy in terms of your sample size, it’s important to plan for potential dropouts by oversampling from the outset . In other words, if you aim for a final sample size of 100 participants, aim to recruit 120-140 to account for unexpected challenges. If you still find yourself short on participants, consider whether you could complement your dataset with secondary data or data from an adjacent sample – for example, participants from another city or country. That said, be sure to engage with your research advisor before making any changes to your approach.

A related issue that you may run into is sample composition. In other words, you may have trouble securing a random sample that’s representative of your population of interest. In cases like this, you might again want to look at ways to complement your dataset with other sources, but if that’s not possible, it’s not the end of the world. As with all limitations, you’ll just need to recognise this limitation in your final write-up and be sure to interpret your results accordingly. In other words, don’t claim generalisability of your results if your sample isn’t random.

Limitation #4: Methodological Limitations

As we alluded earlier, every methodological choice comes with its own set of limitations . For example, you can’t claim causality if you’re using a descriptive or correlational research design. Similarly, as we saw in the previous example, you can’t claim generalisability if you’re using a non-random sampling approach.

Making good methodological choices is all about understanding (and accepting) the inherent trade-offs . In the vast majority of cases, you won’t be able to adopt the “perfect” methodology – and that’s okay. What’s important is that you select a methodology that aligns with your research aims and research questions , as well as the practical constraints at play (e.g., time, money, equipment access, etc.). Just as importantly, you must recognise and articulate the limitations of your chosen methods, and justify why they were the most suitable, given your specific context.

Limitation #5: Researcher (In)experience 

A discussion about research limitations would not be complete without mentioning the researcher (that’s you!). Whether we like to admit it or not, researcher inexperience and personal biases can subtly (and sometimes not so subtly) influence the interpretation and presentation of data within a study. This is especially true when it comes to dissertations and theses , as these are most commonly undertaken by first-time (or relatively fresh) researchers.

When it comes to dealing with this specific limitation, it’s important to remember the adage “ We don’t know what we don’t know ”. In other words, recognise and embrace your (relative) ignorance and subjectivity – and interpret your study’s results within that context . Simply put, don’t be overly confident in drawing conclusions from your study – especially when they contradict existing literature.

Cultivating a culture of reflexivity within your research practices can help reduce subjectivity and keep you a bit more “rooted” in the data. In practical terms, this simply means making an effort to become aware of how your perspectives and experiences may have shaped the research process and outcomes.

As with any new endeavour in life, it’s useful to garner as many outsider perspectives as possible. Of course, your university-assigned research advisor will play a large role in this respect, but it’s also a good idea to seek out feedback and critique from other academics. To this end, you might consider approaching other faculty at your institution, joining an online group, or even working with a private coach .

Key Takeaways

Understanding and effectively navigating research limitations is key to conducting credible and reliable academic work. By acknowledging and addressing these limitations upfront, you not only enhance the integrity of your research, but also demonstrate your academic maturity and professionalism.

Whether you’re working on a dissertation, thesis or any other type of formal academic research, remember the five most common research limitations and interpret your data while keeping them in mind.

• Access to Information (literature and data)
• Time and money
• Sample size and composition
• Research design and methodology
• Researcher (in)experience and bias

If you need a hand identifying and mitigating the limitations within your study, check out our 1:1 private coaching service .

Psst… there’s more!

This post is an extract from our bestselling short course, Methodology Bootcamp . If you want to work smart, you don't want to miss this .

You Might Also Like:

Submit a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

• Print Friendly

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• Limitations of the Study
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

The limitations of the study are those characteristics of design or methodology that impacted or influenced the interpretation of the findings from your research. Study limitations are the constraints placed on the ability to generalize from the results, to further describe applications to practice, and/or related to the utility of findings that are the result of the ways in which you initially chose to design the study or the method used to establish internal and external validity or the result of unanticipated challenges that emerged during the study.

Price, James H. and Judy Murnan. “Research Limitations and the Necessity of Reporting Them.” American Journal of Health Education 35 (2004): 66-67; Theofanidis, Dimitrios and Antigoni Fountouki. "Limitations and Delimitations in the Research Process." Perioperative Nursing 7 (September-December 2018): 155-163. .

Importance of...

Always acknowledge a study's limitations. It is far better that you identify and acknowledge your study’s limitations than to have them pointed out by your professor and have your grade lowered because you appeared to have ignored them or didn't realize they existed.

Keep in mind that acknowledgment of a study's limitations is an opportunity to make suggestions for further research. If you do connect your study's limitations to suggestions for further research, be sure to explain the ways in which these unanswered questions may become more focused because of your study.

Acknowledgment of a study's limitations also provides you with opportunities to demonstrate that you have thought critically about the research problem, understood the relevant literature published about it, and correctly assessed the methods chosen for studying the problem. A key objective of the research process is not only discovering new knowledge but also to confront assumptions and explore what we don't know.

Claiming limitations is a subjective process because you must evaluate the impact of those limitations . Don't just list key weaknesses and the magnitude of a study's limitations. To do so diminishes the validity of your research because it leaves the reader wondering whether, or in what ways, limitation(s) in your study may have impacted the results and conclusions. Limitations require a critical, overall appraisal and interpretation of their impact. You should answer the question: do these problems with errors, methods, validity, etc. eventually matter and, if so, to what extent?

Price, James H. and Judy Murnan. “Research Limitations and the Necessity of Reporting Them.” American Journal of Health Education 35 (2004): 66-67; Structure: How to Structure the Research Limitations Section of Your Dissertation. Dissertations and Theses: An Online Textbook. Laerd.com.

Descriptions of Possible Limitations

All studies have limitations . However, it is important that you restrict your discussion to limitations related to the research problem under investigation. For example, if a meta-analysis of existing literature is not a stated purpose of your research, it should not be discussed as a limitation. Do not apologize for not addressing issues that you did not promise to investigate in the introduction of your paper.

Here are examples of limitations related to methodology and the research process you may need to describe and discuss how they possibly impacted your results. Note that descriptions of limitations should be stated in the past tense because they were discovered after you completed your research.

Possible Methodological Limitations

• Sample size -- the number of the units of analysis you use in your study is dictated by the type of research problem you are investigating. Note that, if your sample size is too small, it will be difficult to find significant relationships from the data, as statistical tests normally require a larger sample size to ensure a representative distribution of the population and to be considered representative of groups of people to whom results will be generalized or transferred. Note that sample size is generally less relevant in qualitative research if explained in the context of the research problem.
• Lack of available and/or reliable data -- a lack of data or of reliable data will likely require you to limit the scope of your analysis, the size of your sample, or it can be a significant obstacle in finding a trend and a meaningful relationship. You need to not only describe these limitations but provide cogent reasons why you believe data is missing or is unreliable. However, don’t just throw up your hands in frustration; use this as an opportunity to describe a need for future research based on designing a different method for gathering data.
• Lack of prior research studies on the topic -- citing prior research studies forms the basis of your literature review and helps lay a foundation for understanding the research problem you are investigating. Depending on the currency or scope of your research topic, there may be little, if any, prior research on your topic. Before assuming this to be true, though, consult with a librarian! In cases when a librarian has confirmed that there is little or no prior research, you may be required to develop an entirely new research typology [for example, using an exploratory rather than an explanatory research design ]. Note again that discovering a limitation can serve as an important opportunity to identify new gaps in the literature and to describe the need for further research.
• Measure used to collect the data -- sometimes it is the case that, after completing your interpretation of the findings, you discover that the way in which you gathered data inhibited your ability to conduct a thorough analysis of the results. For example, you regret not including a specific question in a survey that, in retrospect, could have helped address a particular issue that emerged later in the study. Acknowledge the deficiency by stating a need for future researchers to revise the specific method for gathering data.
• Self-reported data -- whether you are relying on pre-existing data or you are conducting a qualitative research study and gathering the data yourself, self-reported data is limited by the fact that it rarely can be independently verified. In other words, you have to the accuracy of what people say, whether in interviews, focus groups, or on questionnaires, at face value. However, self-reported data can contain several potential sources of bias that you should be alert to and note as limitations. These biases become apparent if they are incongruent with data from other sources. These are: (1) selective memory [remembering or not remembering experiences or events that occurred at some point in the past]; (2) telescoping [recalling events that occurred at one time as if they occurred at another time]; (3) attribution [the act of attributing positive events and outcomes to one's own agency, but attributing negative events and outcomes to external forces]; and, (4) exaggeration [the act of representing outcomes or embellishing events as more significant than is actually suggested from other data].

Possible Limitations of the Researcher

• Access -- if your study depends on having access to people, organizations, data, or documents and, for whatever reason, access is denied or limited in some way, the reasons for this needs to be described. Also, include an explanation why being denied or limited access did not prevent you from following through on your study.
• Longitudinal effects -- unlike your professor, who can literally devote years [even a lifetime] to studying a single topic, the time available to investigate a research problem and to measure change or stability over time is constrained by the due date of your assignment. Be sure to choose a research problem that does not require an excessive amount of time to complete the literature review, apply the methodology, and gather and interpret the results. If you're unsure whether you can complete your research within the confines of the assignment's due date, talk to your professor.
• Cultural and other type of bias -- we all have biases, whether we are conscience of them or not. Bias is when a person, place, event, or thing is viewed or shown in a consistently inaccurate way. Bias is usually negative, though one can have a positive bias as well, especially if that bias reflects your reliance on research that only support your hypothesis. When proof-reading your paper, be especially critical in reviewing how you have stated a problem, selected the data to be studied, what may have been omitted, the manner in which you have ordered events, people, or places, how you have chosen to represent a person, place, or thing, to name a phenomenon, or to use possible words with a positive or negative connotation. NOTE :   If you detect bias in prior research, it must be acknowledged and you should explain what measures were taken to avoid perpetuating that bias. For example, if a previous study only used boys to examine how music education supports effective math skills, describe how your research expands the study to include girls.
• Fluency in a language -- if your research focuses , for example, on measuring the perceived value of after-school tutoring among Mexican-American ESL [English as a Second Language] students and you are not fluent in Spanish, you are limited in being able to read and interpret Spanish language research studies on the topic or to speak with these students in their primary language. This deficiency should be acknowledged.

Aguinis, Hermam and Jeffrey R. Edwards. “Methodological Wishes for the Next Decade and How to Make Wishes Come True.” Journal of Management Studies 51 (January 2014): 143-174; Brutus, Stéphane et al. "Self-Reported Limitations and Future Directions in Scholarly Reports: Analysis and Recommendations." Journal of Management 39 (January 2013): 48-75; Senunyeme, Emmanuel K. Business Research Methods. Powerpoint Presentation. Regent University of Science and Technology; ter Riet, Gerben et al. “All That Glitters Isn't Gold: A Survey on Acknowledgment of Limitations in Biomedical Studies.” PLOS One 8 (November 2013): 1-6.

Structure and Writing Style

Information about the limitations of your study are generally placed either at the beginning of the discussion section of your paper so the reader knows and understands the limitations before reading the rest of your analysis of the findings, or, the limitations are outlined at the conclusion of the discussion section as an acknowledgement of the need for further study. Statements about a study's limitations should not be buried in the body [middle] of the discussion section unless a limitation is specific to something covered in that part of the paper. If this is the case, though, the limitation should be reiterated at the conclusion of the section.

If you determine that your study is seriously flawed due to important limitations , such as, an inability to acquire critical data, consider reframing it as an exploratory study intended to lay the groundwork for a more complete research study in the future. Be sure, though, to specifically explain the ways that these flaws can be successfully overcome in a new study.

But, do not use this as an excuse for not developing a thorough research paper! Review the tab in this guide for developing a research topic . If serious limitations exist, it generally indicates a likelihood that your research problem is too narrowly defined or that the issue or event under study is too recent and, thus, very little research has been written about it. If serious limitations do emerge, consult with your professor about possible ways to overcome them or how to revise your study.

When discussing the limitations of your research, be sure to:

• Describe each limitation in detailed but concise terms;
• Explain why each limitation exists;
• Provide the reasons why each limitation could not be overcome using the method(s) chosen to acquire or gather the data [cite to other studies that had similar problems when possible];
• Assess the impact of each limitation in relation to the overall findings and conclusions of your study; and,
• If appropriate, describe how these limitations could point to the need for further research.

Remember that the method you chose may be the source of a significant limitation that has emerged during your interpretation of the results [for example, you didn't interview a group of people that you later wish you had]. If this is the case, don't panic. Acknowledge it, and explain how applying a different or more robust methodology might address the research problem more effectively in a future study. A underlying goal of scholarly research is not only to show what works, but to demonstrate what doesn't work or what needs further clarification.

Aguinis, Hermam and Jeffrey R. Edwards. “Methodological Wishes for the Next Decade and How to Make Wishes Come True.” Journal of Management Studies 51 (January 2014): 143-174; Brutus, Stéphane et al. "Self-Reported Limitations and Future Directions in Scholarly Reports: Analysis and Recommendations." Journal of Management 39 (January 2013): 48-75; Ioannidis, John P.A. "Limitations are not Properly Acknowledged in the Scientific Literature." Journal of Clinical Epidemiology 60 (2007): 324-329; Pasek, Josh. Writing the Empirical Social Science Research Paper: A Guide for the Perplexed. January 24, 2012. Academia.edu; Structure: How to Structure the Research Limitations Section of Your Dissertation. Dissertations and Theses: An Online Textbook. Laerd.com; What Is an Academic Paper? Institute for Writing Rhetoric. Dartmouth College; Writing the Experimental Report: Methods, Results, and Discussion. The Writing Lab and The OWL. Purdue University.

Writing Tip

Don't Inflate the Importance of Your Findings!

After all the hard work and long hours devoted to writing your research paper, it is easy to get carried away with attributing unwarranted importance to what you’ve done. We all want our academic work to be viewed as excellent and worthy of a good grade, but it is important that you understand and openly acknowledge the limitations of your study. Inflating the importance of your study's findings could be perceived by your readers as an attempt hide its flaws or encourage a biased interpretation of the results. A small measure of humility goes a long way!

Another Writing Tip

Negative Results are Not a Limitation!

Negative evidence refers to findings that unexpectedly challenge rather than support your hypothesis. If you didn't get the results you anticipated, it may mean your hypothesis was incorrect and needs to be reformulated. Or, perhaps you have stumbled onto something unexpected that warrants further study. Moreover, the absence of an effect may be very telling in many situations, particularly in experimental research designs. In any case, your results may very well be of importance to others even though they did not support your hypothesis. Do not fall into the trap of thinking that results contrary to what you expected is a limitation to your study. If you carried out the research well, they are simply your results and only require additional interpretation.

Lewis, George H. and Jonathan F. Lewis. “The Dog in the Night-Time: Negative Evidence in Social Research.” The British Journal of Sociology 31 (December 1980): 544-558.

Yet Another Writing Tip

Sample Size Limitations in Qualitative Research

Sample sizes are typically smaller in qualitative research because, as the study goes on, acquiring more data does not necessarily lead to more information. This is because one occurrence of a piece of data, or a code, is all that is necessary to ensure that it becomes part of the analysis framework. However, it remains true that sample sizes that are too small cannot adequately support claims of having achieved valid conclusions and sample sizes that are too large do not permit the deep, naturalistic, and inductive analysis that defines qualitative inquiry. Determining adequate sample size in qualitative research is ultimately a matter of judgment and experience in evaluating the quality of the information collected against the uses to which it will be applied and the particular research method and purposeful sampling strategy employed. If the sample size is found to be a limitation, it may reflect your judgment about the methodological technique chosen [e.g., single life history study versus focus group interviews] rather than the number of respondents used.

Boddy, Clive Roland. "Sample Size for Qualitative Research." Qualitative Market Research: An International Journal 19 (2016): 426-432; Huberman, A. Michael and Matthew B. Miles. "Data Management and Analysis Methods." In Handbook of Qualitative Research . Norman K. Denzin and Yvonna S. Lincoln, eds. (Thousand Oaks, CA: Sage, 1994), pp. 428-444; Blaikie, Norman. "Confounding Issues Related to Determining Sample Size in Qualitative Research." International Journal of Social Research Methodology 21 (2018): 635-641; Oppong, Steward Harrison. "The Problem of Sampling in qualitative Research." Asian Journal of Management Sciences and Education 2 (2013): 202-210.

• << Previous: 8. The Discussion
• Next: 9. The Conclusion >>
• Last Updated: May 15, 2024 9:53 AM
• URL: https://libguides.usc.edu/writingguide

• Privacy Policy

Home » Limitations in Research – Types, Examples and Writing Guide

Limitations in Research – Types, Examples and Writing Guide

Table of Contents

Limitations in Research

Limitations in research refer to the factors that may affect the results, conclusions , and generalizability of a study. These limitations can arise from various sources, such as the design of the study, the sampling methods used, the measurement tools employed, and the limitations of the data analysis techniques.

Types of Limitations in Research

Types of Limitations in Research are as follows:

Sample Size Limitations

This refers to the size of the group of people or subjects that are being studied. If the sample size is too small, then the results may not be representative of the population being studied. This can lead to a lack of generalizability of the results.

Time Limitations

Time limitations can be a constraint on the research process . This could mean that the study is unable to be conducted for a long enough period of time to observe the long-term effects of an intervention, or to collect enough data to draw accurate conclusions.

Selection Bias

This refers to a type of bias that can occur when the selection of participants in a study is not random. This can lead to a biased sample that is not representative of the population being studied.

Confounding Variables

Confounding variables are factors that can influence the outcome of a study, but are not being measured or controlled for. These can lead to inaccurate conclusions or a lack of clarity in the results.

Measurement Error

This refers to inaccuracies in the measurement of variables, such as using a faulty instrument or scale. This can lead to inaccurate results or a lack of validity in the study.

Ethical Limitations

Ethical limitations refer to the ethical constraints placed on research studies. For example, certain studies may not be allowed to be conducted due to ethical concerns, such as studies that involve harm to participants.

Examples of Limitations in Research

Some Examples of Limitations in Research are as follows:

Research Title: “The Effectiveness of Machine Learning Algorithms in Predicting Customer Behavior”

Limitations:

• The study only considered a limited number of machine learning algorithms and did not explore the effectiveness of other algorithms.
• The study used a specific dataset, which may not be representative of all customer behaviors or demographics.
• The study did not consider the potential ethical implications of using machine learning algorithms in predicting customer behavior.

Research Title: “The Impact of Online Learning on Student Performance in Computer Science Courses”

• The study was conducted during the COVID-19 pandemic, which may have affected the results due to the unique circumstances of remote learning.
• The study only included students from a single university, which may limit the generalizability of the findings to other institutions.
• The study did not consider the impact of individual differences, such as prior knowledge or motivation, on student performance in online learning environments.

Research Title: “The Effect of Gamification on User Engagement in Mobile Health Applications”

• The study only tested a specific gamification strategy and did not explore the effectiveness of other gamification techniques.
• The study relied on self-reported measures of user engagement, which may be subject to social desirability bias or measurement errors.
• The study only included a specific demographic group (e.g., young adults) and may not be generalizable to other populations with different preferences or needs.

How to Write Limitations in Research

When writing about the limitations of a research study, it is important to be honest and clear about the potential weaknesses of your work. Here are some tips for writing about limitations in research:

• Identify the limitations: Start by identifying the potential limitations of your research. These may include sample size, selection bias, measurement error, or other issues that could affect the validity and reliability of your findings.
• Be honest and objective: When describing the limitations of your research, be honest and objective. Do not try to minimize or downplay the limitations, but also do not exaggerate them. Be clear and concise in your description of the limitations.
• Provide context: It is important to provide context for the limitations of your research. For example, if your sample size was small, explain why this was the case and how it may have affected your results. Providing context can help readers understand the limitations in a broader context.
• Discuss implications : Discuss the implications of the limitations for your research findings. For example, if there was a selection bias in your sample, explain how this may have affected the generalizability of your findings. This can help readers understand the limitations in terms of their impact on the overall validity of your research.
• Provide suggestions for future research : Finally, provide suggestions for future research that can address the limitations of your study. This can help readers understand how your research fits into the broader field and can provide a roadmap for future studies.

Purpose of Limitations in Research

There are several purposes of limitations in research. Here are some of the most important ones:

• To acknowledge the boundaries of the study : Limitations help to define the scope of the research project and set realistic expectations for the findings. They can help to clarify what the study is not intended to address.
• To identify potential sources of bias: Limitations can help researchers identify potential sources of bias in their research design, data collection, or analysis. This can help to improve the validity and reliability of the findings.
• To provide opportunities for future research: Limitations can highlight areas for future research and suggest avenues for further exploration. This can help to advance knowledge in a particular field.
• To demonstrate transparency and accountability: By acknowledging the limitations of their research, researchers can demonstrate transparency and accountability to their readers, peers, and funders. This can help to build trust and credibility in the research community.
• To encourage critical thinking: Limitations can encourage readers to critically evaluate the study’s findings and consider alternative explanations or interpretations. This can help to promote a more nuanced and sophisticated understanding of the topic under investigation.

When to Write Limitations in Research

Limitations should be included in research when they help to provide a more complete understanding of the study’s results and implications. A limitation is any factor that could potentially impact the accuracy, reliability, or generalizability of the study’s findings.

It is important to identify and discuss limitations in research because doing so helps to ensure that the results are interpreted appropriately and that any conclusions drawn are supported by the available evidence. Limitations can also suggest areas for future research, highlight potential biases or confounding factors that may have affected the results, and provide context for the study’s findings.

Generally, limitations should be discussed in the conclusion section of a research paper or thesis, although they may also be mentioned in other sections, such as the introduction or methods. The specific limitations that are discussed will depend on the nature of the study, the research question being investigated, and the data that was collected.

Examples of limitations that might be discussed in research include sample size limitations, data collection methods, the validity and reliability of measures used, and potential biases or confounding factors that could have affected the results. It is important to note that limitations should not be used as a justification for poor research design or methodology, but rather as a way to enhance the understanding and interpretation of the study’s findings.

Importance of Limitations in Research

Here are some reasons why limitations are important in research:

• Enhances the credibility of research: Limitations highlight the potential weaknesses and threats to validity, which helps readers to understand the scope and boundaries of the study. This improves the credibility of research by acknowledging its limitations and providing a clear picture of what can and cannot be concluded from the study.
• Facilitates replication: By highlighting the limitations, researchers can provide detailed information about the study’s methodology, data collection, and analysis. This information helps other researchers to replicate the study and test the validity of the findings, which enhances the reliability of research.
• Guides future research : Limitations provide insights into areas for future research by identifying gaps or areas that require further investigation. This can help researchers to design more comprehensive and effective studies that build on existing knowledge.
• Provides a balanced view: Limitations help to provide a balanced view of the research by highlighting both strengths and weaknesses. This ensures that readers have a clear understanding of the study’s limitations and can make informed decisions about the generalizability and applicability of the findings.

Advantages of Limitations in Research

Here are some potential advantages of limitations in research:

• Focus : Limitations can help researchers focus their study on a specific area or population, which can make the research more relevant and useful.
• Realism : Limitations can make a study more realistic by reflecting the practical constraints and challenges of conducting research in the real world.
• Innovation : Limitations can spur researchers to be more innovative and creative in their research design and methodology, as they search for ways to work around the limitations.
• Rigor : Limitations can actually increase the rigor and credibility of a study, as researchers are forced to carefully consider the potential sources of bias and error, and address them to the best of their abilities.
• Generalizability : Limitations can actually improve the generalizability of a study by ensuring that it is not overly focused on a specific sample or situation, and that the results can be applied more broadly.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

How to Cite Research Paper – All Formats and...

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...

Research Paper Format – Types, Examples and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

• Affiliate Program

• UNITED STATES
• 台灣 (TAIWAN)
• TÜRKIYE (TURKEY)
• Academic Editing Services
• - Research Paper
• - Journal Manuscript
• - Dissertation
• - College & University Assignments
• Admissions Editing Services
• - Application Essay
• - Personal Statement
• - Recommendation Letter
• - Cover Letter
• - CV/Resume
• Business Editing Services
• - Business Documents
• - Report & Brochure
• - Website & Blog
• Writer Editing Services
• - Script & Screenplay
• Our Editors
• Client Reviews
• Editing & Proofreading Prices
• Wordvice Points
• Partner Discount
• Plagiarism Checker

APA Citation Generator

MLA Citation Generator

Chicago Citation Generator

Vancouver Citation Generator

• - APA Style
• - MLA Style
• - Chicago Style
• - Vancouver Style
• Writing & Editing Guide
• Academic Resources
• Admissions Resources

How to Present the Limitations of the Study Examples

What are the limitations of a study?

The limitations of a study are the elements of methodology or study design that impact the interpretation of your research results. The limitations essentially detail any flaws or shortcomings in your study. Study limitations can exist due to constraints on research design, methodology, materials, etc., and these factors may impact the findings of your study. However, researchers are often reluctant to discuss the limitations of their study in their papers, feeling that bringing up limitations may undermine its research value in the eyes of readers and reviewers.

In spite of the impact it might have (and perhaps because of it) you should clearly acknowledge any limitations in your research paper in order to show readers—whether journal editors, other researchers, or the general public—that you are aware of these limitations and to explain how they affect the conclusions that can be drawn from the research.

In this article, we provide some guidelines for writing about research limitations, show examples of some frequently seen study limitations, and recommend techniques for presenting this information. And after you have finished drafting and have received manuscript editing for your work, you still might want to follow this up with academic editing before submitting your work to your target journal.

Why do I need to include limitations of research in my paper?

Although limitations address the potential weaknesses of a study, writing about them toward the end of your paper actually strengthens your study by identifying any problems before other researchers or reviewers find them.

Furthermore, pointing out study limitations shows that you’ve considered the impact of research weakness thoroughly and have an in-depth understanding of your research topic. Since all studies face limitations, being honest and detailing these limitations will impress researchers and reviewers more than ignoring them.

Where should I put the limitations of the study in my paper?

Some limitations might be evident to researchers before the start of the study, while others might become clear while you are conducting the research. Whether these limitations are anticipated or not, and whether they are due to research design or to methodology, they should be clearly identified and discussed in the discussion section —the final section of your paper. Most journals now require you to include a discussion of potential limitations of your work, and many journals now ask you to place this “limitations section” at the very end of your article. 

Some journals ask you to also discuss the strengths of your work in this section, and some allow you to freely choose where to include that information in your discussion section—make sure to always check the author instructions of your target journal before you finalize a manuscript and submit it for peer review .

Limitations of the Study Examples

There are several reasons why limitations of research might exist. The two main categories of limitations are those that result from the methodology and those that result from issues with the researcher(s).

Common Methodological Limitations of Studies

Limitations of research due to methodological problems can be addressed by clearly and directly identifying the potential problem and suggesting ways in which this could have been addressed—and SHOULD be addressed in future studies. The following are some major potential methodological issues that can impact the conclusions researchers can draw from the research.

Issues with research samples and selection

Sampling errors occur when a probability sampling method is used to select a sample, but that sample does not reflect the general population or appropriate population concerned. This results in limitations of your study known as “sample bias” or “selection bias.”

For example, if you conducted a survey to obtain your research results, your samples (participants) were asked to respond to the survey questions. However, you might have had limited ability to gain access to the appropriate type or geographic scope of participants. In this case, the people who responded to your survey questions may not truly be a random sample.

Insufficient sample size for statistical measurements

When conducting a study, it is important to have a sufficient sample size in order to draw valid conclusions. The larger the sample, the more precise your results will be. If your sample size is too small, it will be difficult to identify significant relationships in the data.

Normally, statistical tests require a larger sample size to ensure that the sample is considered representative of a population and that the statistical result can be generalized to a larger population. It is a good idea to understand how to choose an appropriate sample size before you conduct your research by using scientific calculation tools—in fact, many journals now require such estimation to be included in every manuscript that is sent out for review.

Lack of previous research studies on the topic

Citing and referencing prior research studies constitutes the basis of the literature review for your thesis or study, and these prior studies provide the theoretical foundations for the research question you are investigating. However, depending on the scope of your research topic, prior research studies that are relevant to your thesis might be limited.

When there is very little or no prior research on a specific topic, you may need to develop an entirely new research typology. In this case, discovering a limitation can be considered an important opportunity to identify literature gaps and to present the need for further development in the area of study.

Methods/instruments/techniques used to collect the data

After you complete your analysis of the research findings (in the discussion section), you might realize that the manner in which you have collected the data or the ways in which you have measured variables has limited your ability to conduct a thorough analysis of the results.

For example, you might realize that you should have addressed your survey questions from another viable perspective, or that you were not able to include an important question in the survey. In these cases, you should acknowledge the deficiency or deficiencies by stating a need for future researchers to revise their specific methods for collecting data that includes these missing elements.

Common Limitations of the Researcher(s)

Study limitations that arise from situations relating to the researcher or researchers (whether the direct fault of the individuals or not) should also be addressed and dealt with, and remedies to decrease these limitations—both hypothetically in your study, and practically in future studies—should be proposed.

Limited access to data

If your research involved surveying certain people or organizations, you might have faced the problem of having limited access to these respondents. Due to this limited access, you might need to redesign or restructure your research in a different way. In this case, explain the reasons for limited access and be sure that your finding is still reliable and valid despite this limitation.

Time constraints

Just as students have deadlines to turn in their class papers, academic researchers might also have to meet deadlines for submitting a manuscript to a journal or face other time constraints related to their research (e.g., participants are only available during a certain period; funding runs out; collaborators move to a new institution). The time available to study a research problem and to measure change over time might be constrained by such practical issues. If time constraints negatively impacted your study in any way, acknowledge this impact by mentioning a need for a future study (e.g., a longitudinal study) to answer this research problem.

Conflicts arising from cultural bias and other personal issues

Researchers might hold biased views due to their cultural backgrounds or perspectives of certain phenomena, and this can affect a study’s legitimacy. Also, it is possible that researchers will have biases toward data and results that only support their hypotheses or arguments. In order to avoid these problems, the author(s) of a study should examine whether the way the research problem was stated and the data-gathering process was carried out appropriately.

Steps for Organizing Your Study Limitations Section

When you discuss the limitations of your study, don’t simply list and describe your limitations—explain how these limitations have influenced your research findings. There might be multiple limitations in your study, but you only need to point out and explain those that directly relate to and impact how you address your research questions.

We suggest that you divide your limitations section into three steps: (1) identify the study limitations; (2) explain how they impact your study in detail; and (3) propose a direction for future studies and present alternatives. By following this sequence when discussing your study’s limitations, you will be able to clearly demonstrate your study’s weakness without undermining the quality and integrity of your research.

Step 1. Identify the limitation(s) of the study

• This part should comprise around 10%-20% of your discussion of study limitations.

The first step is to identify the particular limitation(s) that affected your study. There are many possible limitations of research that can affect your study, but you don’t need to write a long review of all possible study limitations. A 200-500 word critique is an appropriate length for a research limitations section. In the beginning of this section, identify what limitations your study has faced and how important these limitations are.

You only need to identify limitations that had the greatest potential impact on: (1) the quality of your findings, and (2) your ability to answer your research question.

Step 2. Explain these study limitations in detail

• This part should comprise around 60-70% of your discussion of limitations.

After identifying your research limitations, it’s time to explain the nature of the limitations and how they potentially impacted your study. For example, when you conduct quantitative research, a lack of probability sampling is an important issue that you should mention. On the other hand, when you conduct qualitative research, the inability to generalize the research findings could be an issue that deserves mention.

Explain the role these limitations played on the results and implications of the research and justify the choice you made in using this “limiting” methodology or other action in your research. Also, make sure that these limitations didn’t undermine the quality of your dissertation .

Step 3. Propose a direction for future studies and present alternatives (optional)

• This part should comprise around 10-20% of your discussion of limitations.

After acknowledging the limitations of the research, you need to discuss some possible ways to overcome these limitations in future studies. One way to do this is to present alternative methodologies and ways to avoid issues with, or “fill in the gaps of” the limitations of this study you have presented.  Discuss both the pros and cons of these alternatives and clearly explain why researchers should choose these approaches.

Make sure you are current on approaches used by prior studies and the impacts they have had on their findings. Cite review articles or scientific bodies that have recommended these approaches and why. This might be evidence in support of the approach you chose, or it might be the reason you consider your choices to be included as limitations. This process can act as a justification for your approach and a defense of your decision to take it while acknowledging the feasibility of other approaches.

P hrases and Tips for Introducing Your Study Limitations in the Discussion Section

The following phrases are frequently used to introduce the limitations of the study:

• “There may be some possible limitations in this study.”
• “The findings of this study have to be seen in light of some limitations.”
•  “The first is the…The second limitation concerns the…”
•  “The empirical results reported herein should be considered in the light of some limitations.”
• “This research, however, is subject to several limitations.”
• “The primary limitation to the generalization of these results is…”
• “Nonetheless, these results must be interpreted with caution and a number of limitations should be borne in mind.”
• “As with the majority of studies, the design of the current study is subject to limitations.”
• “There are two major limitations in this study that could be addressed in future research. First, the study focused on …. Second ….”

For more articles on research writing and the journal submissions and publication process, visit Wordvice’s Academic Resources page.

And be sure to receive professional English editing and proofreading services , including paper editing services , for your journal manuscript before submitting it to journal editors.

Wordvice Resources

Proofreading & Editing Guide

Writing the Results Section for a Research Paper

How to Write a Literature Review

Research Writing Tips: How to Draft a Powerful Discussion Section

How to Captivate Journal Readers with a Strong Introduction

Tips That Will Make Your Abstract a Success!

APA In-Text Citation Guide for Research Writing

Additional Resources

• Diving Deeper into Limitations and Delimitations (PhD student)
• Organizing Your Social Sciences Research Paper: Limitations of the Study (USC Library)
• Research Limitations (Research Methodology)
• How to Present Limitations and Alternatives (UMASS)

Article References

Pearson-Stuttard, J., Kypridemos, C., Collins, B., Mozaffarian, D., Huang, Y., Bandosz, P.,…Micha, R. (2018). Estimating the health and economic effects of the proposed US Food and Drug Administration voluntary sodium reformulation: Microsimulation cost-effectiveness analysis. PLOS. https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002551

Xu, W.L, Pedersen, N.L., Keller, L., Kalpouzos, G., Wang, H.X., Graff, C,. Fratiglioni, L. (2015). HHEX_23 AA Genotype Exacerbates Effect of Diabetes on Dementia and Alzheimer Disease: A Population-Based Longitudinal Study. PLOS. Retrieved from https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1001853

Writing Limitations of Research Study — 4 Reasons Why It Is Important!

It is not unusual for researchers to come across the term limitations of research during their academic paper writing. More often this is interpreted as something terrible. However, when it comes to research study, limitations can help structure the research study better. Therefore, do not underestimate significance of limitations of research study.

Allow us to take you through the context of how to evaluate the limits of your research and conclude an impactful relevance to your results.

Table of Contents

What Are the Limitations of a Research Study?

Every research has its limit and these limitations arise due to restrictions in methodology or research design.  This could impact your entire research or the research paper you wish to publish. Unfortunately, most researchers choose not to discuss their limitations of research fearing it will affect the value of their article in the eyes of readers.

However, it is very important to discuss your study limitations and show it to your target audience (other researchers, journal editors, peer reviewers etc.). It is very important that you provide an explanation of how your research limitations may affect the conclusions and opinions drawn from your research. Moreover, when as an author you state the limitations of research, it shows that you have investigated all the weaknesses of your study and have a deep understanding of the subject. Being honest could impress your readers and mark your study as a sincere effort in research.

Why and Where Should You Include the Research Limitations?

The main goal of your research is to address your research objectives. Conduct experiments, get results and explain those results, and finally justify your research question . It is best to mention the limitations of research in the discussion paragraph of your research article.

At the very beginning of this paragraph, immediately after highlighting the strengths of the research methodology, you should write down your limitations. You can discuss specific points from your research limitations as suggestions for further research in the conclusion of your thesis.

1. Common Limitations of the Researchers

Limitations that are related to the researcher must be mentioned. This will help you gain transparency with your readers. Furthermore, you could provide suggestions on decreasing these limitations in you and your future studies.

2. Limited Access to Information

Your work may involve some institutions and individuals in research, and sometimes you may have problems accessing these institutions. Therefore, you need to redesign and rewrite your work. You must explain your readers the reason for limited access.

3. Limited Time

All researchers are bound by their deadlines when it comes to completing their studies. Sometimes, time constraints can affect your research negatively. However, the best practice is to acknowledge it and mention a requirement for future study to solve the research problem in a better way.

4. Conflict over Biased Views and Personal Issues

Biased views can affect the research. In fact, researchers end up choosing only those results and data that support their main argument, keeping aside the other loose ends of the research.

Types of Limitations of Research

Before beginning your research study, know that there are certain limitations to what you are testing or possible research results. There are different types that researchers may encounter, and they all have unique characteristics, such as:

1. Research Design Limitations

Certain restrictions on your research or available procedures may affect your final results or research outputs. You may have formulated research goals and objectives too broadly. However, this can help you understand how you can narrow down the formulation of research goals and objectives, thereby increasing the focus of your study.

2. Impact Limitations

Even if your research has excellent statistics and a strong design, it can suffer from the influence of the following factors:

• Presence of increasing findings as researched
• Being population specific
• A strong regional focus.

3. Data or statistical limitations

In some cases, it is impossible to collect sufficient data for research or very difficult to get access to the data. This could lead to incomplete conclusion to your study. Moreover, this insufficiency in data could be the outcome of your study design. The unclear, shabby research outline could produce more problems in interpreting your findings.

How to Correctly Structure Your Research Limitations?

There are strict guidelines for narrowing down research questions, wherein you could justify and explain potential weaknesses of your academic paper. You could go through these basic steps to get a well-structured clarity of research limitations:

• Declare that you wish to identify your limitations of research and explain their importance,
• Provide the necessary depth, explain their nature, and justify your study choices.
• Write how you are suggesting that it is possible to overcome them in the future.

In this section, your readers will see that you are aware of the potential weaknesses in your business, understand them and offer effective solutions, and it will positively strengthen your article as you clarify all limitations of research to your target audience.

Know that you cannot be perfect and there is no individual without flaws. You could use the limitations of research as a great opportunity to take on a new challenge and improve the future of research. In a typical academic paper, research limitations may relate to:

1. Formulating your goals and objectives

If you formulate goals and objectives too broadly, your work will have some shortcomings. In this case, specify effective methods or ways to narrow down the formula of goals and aim to increase your level of study focus.

2. Application of your data collection methods in research

If you do not have experience in primary data collection, there is a risk that there will be flaws in the implementation of your methods. It is necessary to accept this, and learn and educate yourself to understand data collection methods.

3. Sample sizes

This depends on the nature of problem you choose. Sample size is of a greater importance in quantitative studies as opposed to qualitative ones. If your sample size is too small, statistical tests cannot identify significant relationships or connections within a given data set.

You could point out that other researchers should base the same study on a larger sample size to get more accurate results.

4. The absence of previous studies in the field you have chosen

Writing a literature review is an important step in any scientific study because it helps researchers determine the scope of current work in the chosen field. It is a major foundation for any researcher who must use them to achieve a set of specific goals or objectives.

However, if you are focused on the most current and evolving research problem or a very narrow research problem, there may be very little prior research on your topic. For example, if you chose to explore the role of Bitcoin as the currency of the future, you may not find tons of scientific papers addressing the research problem as Bitcoins are only a new phenomenon.

It is important that you learn to identify research limitations examples at each step. Whatever field you choose, feel free to add the shortcoming of your work. This is mainly because you do not have many years of experience writing scientific papers or completing complex work. Therefore, the depth and scope of your discussions may be compromised at different levels compared to academics with a lot of expertise. Include specific points from limitations of research. Use them as suggestions for the future.

Have you ever faced a challenge of writing the limitations of research study in your paper? How did you overcome it? What ways did you follow? Were they beneficial? Let us know in the comments below!

Frequently Asked Questions

Setting limitations in our study helps to clarify the outcomes drawn from our research and enhance understanding of the subject. Moreover, it shows that the author has investigated all the weaknesses in the study.

Scope is the range and limitations of a research project which are set to define the boundaries of a project. Limitations are the impacts on the overall study due to the constraints on the research design.

Limitation in research is an impact of a constraint on the research design in the overall study. They are the flaws or weaknesses in the study, which may influence the outcome of the research.

1. Limitations in research can be written as follows: Formulate your goals and objectives 2. Analyze the chosen data collection method and the sample sizes 3. Identify your limitations of research and explain their importance 4. Provide the necessary depth, explain their nature, and justify your study choices 5. Write how you are suggesting that it is possible to overcome them in the future

Excellent article ,,,it has helped me big

This is very helpful information. It has given me an insight on how to go about my study limitations.

Good comments and helpful

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Diversity and Inclusion
• Trending Now

The Silent Struggle: Confronting gender bias in science funding

In the 1990s, Dr. Katalin Kariko’s pioneering mRNA research seemed destined for obscurity, doomed by…

• Promoting Research

Plain Language Summary — Communicating your research to bridge the academic-lay gap

Science can be complex, but does that mean it should not be accessible to the…

Addressing Barriers in Academia: Navigating unconscious biases in the Ph.D. journey

In the journey of academia, a Ph.D. marks a transitional phase, like that of a…

• Manuscripts & Grants
• Reporting Research

Unraveling Research Population and Sample: Understanding their role in statistical inference

Research population and sample serve as the cornerstones of any scientific inquiry. They hold the…

• Manuscript Preparation
• Publishing Research

Research Problem Statement — Find out how to write an impactful one!

What Is a Research Problem Statement? A research problem statement is a clear, concise, and…

How to Develop a Good Research Question? — Types & Examples

5 Effective Ways to Avoid Ghostwriting for Busy Researchers

Top 5 Key Differences Between Methods and Methodology

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

As a researcher, what do you consider most when choosing an image manipulation detector?

Organizing Academic Research Papers: Limitations of the Study

• Purpose of Guide
• Design Flaws to Avoid
• Glossary of Research Terms
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Executive Summary
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tertiary Sources
• What Is Scholarly vs. Popular?
• Qualitative Methods
• Quantitative Methods
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Annotated Bibliography
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• How to Manage Group Projects
• Multiple Book Review Essay
• Reviewing Collected Essays
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Research Proposal
• Acknowledgements

The limitations of the study are those characteristics of design or methodology that impacted or influenced the application or interpretation of the results of your study. They are the constraints on generalizability and utility of findings that are the result of the ways in which you chose to design the study and/or the method used to establish internal and external validity.

Importance of...

Always acknowledge a study's limitations. It is far better for you to identify and acknowledge your study’s limitations than to have them pointed out by your professor and be graded down because you appear to have ignored them.

Keep in mind that acknowledgement of a study's limitations is an opportunity to make suggestions for further research. If you do connect your study's limitations to suggestions for further research, be sure to explain the ways in which these unanswered questions may become more focused because of your study.

Acknowledgement of a study's limitations also provides you with an opportunity to demonstrate to your professor that you have thought critically about the research problem, understood the relevant literature published about it, and correctly assessed the methods chosen for studying the problem. A key objective of the research process is not only discovering new knowledge but also to confront assumptions and explore what we don't know.

Claiming limitiations is a subjective process because you must evaluate the impact of those limitations . Don't just list key weaknesses and the magnitude of a study's limitations. To do so diminishes the validity of your research because it leaves the reader wondering whether, or in what ways, limitation(s) in your study may have impacted the findings and conclusions. Limitations require a critical, overall appraisal and interpretation of their impact. You should answer the question: do these problems with errors, methods, validity, etc. eventually matter and, if so, to what extent?

Structure: How to Structure the Research Limitations Section of Your Dissertation . Dissertations and Theses: An Online Textbook. Laerd.com.

Descriptions of Possible Limitations

All studies have limitations . However, it is important that you restrict your discussion to limitations related to the research problem under investigation. For example, if a meta-analysis of existing literature is not a stated purpose of your research, it should not be discussed as a limitation. Do not apologize for not addressing issues that you did not promise to investigate in your paper.

Here are examples of limitations you may need to describe and to discuss how they possibly impacted your findings. Descriptions of limitations should be stated in the past tense.

Possible Methodological Limitations

• Sample size -- the number of the units of analysis you use in your study is dictated by the type of research problem you are investigating. Note that, if your sample size is too small, it will be difficult to find significant relationships from the data, as statistical tests normally require a larger sample size to ensure a representative distribution of the population and to be considered representative of groups of people to whom results will be generalized or transferred.
• Lack of available and/or reliable data -- a lack of data or of reliable data will likely require you to limit the scope of your analysis, the size of your sample, or it can be a significant obstacle in finding a trend and a meaningful relationship. You need to not only describe these limitations but to offer reasons why you believe data is missing or is unreliable. However, don’t just throw up your hands in frustration; use this as an opportunity to describe the need for future research.
• Lack of prior research studies on the topic -- citing prior research studies forms the basis of your literature review and helps lay a foundation for understanding the research problem you are investigating. Depending on the currency or scope of your research topic, there may be little, if any, prior research on your topic. Before assuming this to be true, consult with a librarian! In cases when a librarian has confirmed that there is a lack of prior research, you may be required to develop an entirely new research typology [for example, using an exploratory rather than an explanatory research design]. Note that this limitation can serve as an important opportunity to describe the need for further research.
• Measure used to collect the data -- sometimes it is the case that, after completing your interpretation of the findings, you discover that the way in which you gathered data inhibited your ability to conduct a thorough analysis of the results. For example, you regret not including a specific question in a survey that, in retrospect, could have helped address a particular issue that emerged later in the study. Acknowledge the deficiency by stating a need in future research to revise the specific method for gathering data.
• Self-reported data -- whether you are relying on pre-existing self-reported data or you are conducting a qualitative research study and gathering the data yourself, self-reported data is limited by the fact that it rarely can be independently verified. In other words, you have to take what people say, whether in interviews, focus groups, or on questionnaires, at face value. However, self-reported data contain several potential sources of bias that should be noted as limitations: (1) selective memory (remembering or not remembering experiences or events that occurred at some point in the past); (2) telescoping [recalling events that occurred at one time as if they occurred at another time]; (3) attribution [the act of attributing positive events and outcomes to one's own agency but attributing negative events and outcomes to external forces]; and, (4) exaggeration [the act of representing outcomes or embellishing events as more significant than is actually suggested from other data].

Possible Limitations of the Researcher

• Access -- if your study depends on having access to people, organizations, or documents and, for whatever reason, access is denied or otherwise limited, the reasons for this need to be described.
• Longitudinal effects -- unlike your professor, who can literally devote years [even a lifetime] to studying a single research problem, the time available to investigate a research problem and to measure change or stability within a sample is constrained by the due date of your assignment. Be sure to choose a topic that does not require an excessive amount of time to complete the literature review, apply the methodology, and gather and interpret the results. If you're unsure, talk to your professor.
• Cultural and other type of bias -- we all have biases, whether we are conscience of them or not. Bias is when a person, place, or thing is viewed or shown in a consistently inaccurate way. It is usually negative, though one can have a positive bias as well. When proof-reading your paper, be especially critical in reviewing how you have stated a problem, selected the data to be studied, what may have been omitted, the manner in which you have ordered events, people, or places and how you have chosen to represent a person, place, or thing, to name a phenomenon, or to use possible words with a positive or negative connotation. Note that if you detect bias in prior research, it must be acknowledged and you should explain what measures were taken to avoid perpetuating bias.
• Fluency in a language -- if your research focuses on measuring the perceived value of after-school tutoring among Mexican-American ESL [English as a Second Language] students, for example, and you are not fluent in Spanish, you are limited in being able to read and interpret Spanish language research studies on the topic. This deficiency should be acknowledged.

Brutus, Stéphane et al. Self-Reported Limitations and Future Directions in Scholarly Reports: Analysis and Recommendations. Journal of Management 39 (January 2013): 48-75; Senunyeme, Emmanuel K. Business Research Methods . Powerpoint Presentation. Regent University of Science and Technology.

Structure and Writing Style

Information about the limitations of your study are generally placed either at the beginning of the discussion section of your paper so the reader knows and understands the limitations before reading the rest of your analysis of the findings, or, the limitations are outlined at the conclusion of the discussion section as an acknowledgement of the need for further study. Statements about a study's limitations should not be buried in the body [middle] of the discussion section unless a limitation is specific to something covered in that part of the paper. If this is the case, though, the limitation should be reiterated at the conclusion of the section.

If you determine that your study is seriously flawed due to important limitations , such as, an inability to acquire critical data, consider reframing it as a pilot study intended to lay the groundwork for a more complete research study in the future. Be sure, though, to specifically explain the ways that these flaws can be successfully overcome in later studies.

But, do not use this as an excuse for not developing a thorough research paper! Review the tab in this guide for developing a research topic . If serious limitations exist, it generally indicates a likelihood that your research problem is too narrowly defined or that the issue or event under study  is too recent and, thus, very little research has been written about it. If serious limitations do emerge, consult with your professor about possible ways to overcome them or how to reframe your study.

When discussing the limitations of your research, be sure to:

• Describe each limitation in detailed but concise terms;
• Explain why each limitation exists;
• Provide the reasons why each limitation could not be overcome using the method(s) chosen to gather the data [cite to other studies that had similar problems when possible];
• Assess the impact of each limitation in relation to  the overall findings and conclusions of your study; and,
• If appropriate, describe how these limitations could point to the need for further research.

Remember that the method you chose may be the source of a significant limitation that has emerged during your interpretation of the results [for example, you didn't ask a particular question in a survey that you later wish you had]. If this is the case, don't panic. Acknowledge it, and explain how applying a different or more robust methodology might address the research problem more effectively in any future study. A underlying goal of scholarly research is not only to prove what works, but to demonstrate what doesn't work or what needs further clarification.

Brutus, Stéphane et al. Self-Reported Limitations and Future Directions in Scholarly Reports: Analysis and Recommendations. Journal of Management 39 (January 2013): 48-75; Ioannidis, John P.A. Limitations are not Properly Acknowledged in the Scientific Literature. Journal of Clinical Epidemiology 60 (2007): 324-329; Pasek, Josh. Writing the Empirical Social Science Research Paper: A Guide for the Perplexed . January 24, 2012. Academia.edu; Structure: How to Structure the Research Limitations Section of Your Dissertation . Dissertations and Theses: An Online Textbook. Laerd.com; What Is an Academic Paper? Institute for Writing Rhetoric. Dartmouth College; Writing the Experimental Report: Methods, Results, and Discussion. The Writing Lab and The OWL. Purdue University.

Writing Tip

Don't Inflate the Importance of Your Findings! After all the hard work and long hours devoted to writing your research paper, it is easy to get carried away with attributing unwarranted importance to what you’ve done. We all want our academic work to be viewed as excellent and worthy of a good grade, but it is important that you understand and openly acknowledge the limitiations of your study. Inflating of the importance of your study's findings in an attempt hide its flaws is a big turn off to your readers. A measure of humility goes a long way!

Another Writing Tip

Negative Results are Not a Limitation!

Negative evidence refers to findings that unexpectedly challenge rather than support your hypothesis. If you didn't get the results you anticipated, it may mean your hypothesis was incorrect and needs to be reformulated, or, perhaps you have stumbled onto something unexpected that warrants further study. Moreover, the absence of an effect may be very telling in many situations, particularly in experimental research designs. In any case, your results may be of importance to others even though they did not support your hypothesis. Do not fall into the trap of thinking that results contrary to what you expected is a limitation to your study. If you carried out the research well, they are simply your results and only require additional interpretation.

Yet Another Writing Tip

A Note about Sample Size Limitations in Qualitative Research

Sample sizes are typically smaller in qualitative research because, as the study goes on, acquiring more data does not necessarily lead to more information. This is because one occurrence of a piece of data, or a code, is all that is necessary to ensure that it becomes part of the analysis framework. However, it remains true that sample sizes that are too small cannot adequately support claims of having achieved valid conclusions and sample sizes that are too large do not permit the deep, naturalistic, and inductive analysis that defines qualitative inquiry. Determining adequate sample size in qualitative research is ultimately a matter of judgment and experience in evaluating the quality of the information collected against the uses to which it will be applied and the particular research method and purposeful sampling strategy employed. If the sample size is found to be a limitation, it may reflect your judgement about the methodological technique chosen [e.g., single life history study versus focus group interviews] rather than the number of respondents used.

Huberman, A. Michael and Matthew B. Miles. Data Management and Analysis Methods. In Handbook of Qualitative Research. Norman K. Denzin and Yvonna S. Lincoln, eds. (Thousand Oaks, CA: Sage, 1994), pp. 428-444.

• << Previous: 8. The Discussion
• Next: 9. The Conclusion >>
• Last Updated: Jul 18, 2023 11:58 AM
• URL: https://library.sacredheart.edu/c.php?g=29803
• QuickSearch
• Library Catalog
• Databases A-Z
• Publication Finder
• Course Reserves
• Citation Linker
• Digital Commons
• Our Website

Research Support

• Ask a Librarian
• Appointments
• Interlibrary Loan (ILL)
• Research Guides
• Databases by Subject
• Citation Help

Using the Library

• Reserve a Group Study Room
• Renew Books
• Honors Study Rooms
• Off-Campus Access
• Library Policies
• Library Technology

User Information

• Grad Students
• Online Students
• COVID-19 Updates
• Staff Directory
• News & Announcements
• Library Newsletter

My Accounts

• Interlibrary Loan
• Staff Site Login

FIND US ON  

• Cookies & Privacy
• GETTING STARTED
• Introduction
• FUNDAMENTALS
• Acknowledgements
• Research questions & hypotheses
• Concepts, constructs & variables
• Research limitations
• Getting started
• Sampling Strategy
• Research Quality
• Research Ethics
• Data Analysis

How to structure the Research Limitations section of your dissertation

There is no "one best way" to structure the Research Limitations section of your dissertation. However, we recommend a structure based on three moves : the announcing , reflecting and forward looking move. The announcing move immediately allows you to identify the limitations of your dissertation and explain how important each of these limitations is. The reflecting move provides greater depth, helping to explain the nature of the limitations and justify the choices that you made during the research process. Finally, the forward looking move enables you to suggest how such limitations could be overcome in future. The collective aim of these three moves is to help you walk the reader through your Research Limitations section in a succinct and structured way. This will make it clear to the reader that you recognise the limitations of your own research, that you understand why such factors are limitations, and can point to ways of combating these limitations if future research was carried out. This article explains what should be included in each of these three moves :

• THE ANNOUNCING MOVE: Identifying limitations and explaining how important they are
• THE REFLECTING MOVE: Explaining the nature of the limitations and justifying the choices you made
• THE FORWARD LOOKING MOVE: Suggesting how such limitations could be overcome in future

THE ANNOUNCING MOVE Identifying limitations, and explaining how important they are

There are many possible limitations that your research may have faced. However, is not necessary for you to discuss all of these limitations in your Research Limitations section. After all, you are not writing a 2000 word critical review of the limitations of your dissertation, just a 200-500 word critique that is only one section long (i.e., the Research Limitations section within your Conclusions chapter). Therefore, in this first announcing move , we would recommend that you identify only those limitations that had the greatest potential impact on: (a) the quality of your findings; and (b) your ability to effectively answer your research questions and/or hypotheses.

We use the word potential impact because we often do not know the degree to which different factors limited our findings or our ability to effectively answer our research questions and/or hypotheses. For example, we know that when adopting a quantitative research design, a failure to use a probability sampling technique significantly limits our ability to make broader generalisations from our results (i.e., our ability to make statistical inferences from our sample to the population being studied). However, the degree to which this reduces the quality of our findings is a matter of debate. Also, whilst the lack of a probability sampling technique when using a quantitative research design is a very obvious example of a research limitation, other limitations are far less clear. Therefore, the key point is to focus on those limitations that you feel had the greatest impact on your findings, as well as your ability to effectively answer your research questions and/or hypotheses.

Overall, the announcing move should be around 10-20% of the total word count of the Research Limitations section.

THE REFLECTING MOVE Explaining the nature of the limitations and justifying the choices you made

Having identified the most important limitations to your dissertation in the announcing move , the reflecting move focuses on explaining the nature of these limitations and justifying the choices that you made during the research process. This part should be around 60-70% of the total word count of the Research Limitations section.

It is important to remember at this stage that all research suffers from limitations, whether it is performed by undergraduate and master's level dissertation students, or seasoned academics. Acknowledging such limitations should not be viewed as a weakness, highlighting to the person marking your work the reasons why you should receive a lower grade. Instead, the reader is more likely to accept that you recognise the limitations of your own research if you write a high quality reflecting move . This is because explaining the limitations of your research and justifying the choices you made during the dissertation process demonstrates the command that you had over your research.

We talk about explaining the nature of the limitations in your dissertation because such limitations are highly research specific. Let's take the example of potential limitations to your sampling strategy. Whilst you may have a number of potential limitations in sampling strategy, let's focus on the lack of probability sampling ; that is, of all the different types of sampling technique that you could have used [see Types of probability sampling and Types of non-probability sampling ], you choose not to use a probability sampling technique (e.g., simple random sampling , systematic random sampling , stratified random sampling ). As mentioned, if you used a quantitative research design in your dissertation, the lack of probability sampling is an important, obvious limitation to your research. This is because it prevents you from making generalisations about the population you are studying (e.g. Facebook usage at a single university of 20,000 students) from the data you have collected (e.g., a survey of 400 students at the same university). Since an important component of quantitative research is such generalisation, this is a clear limitation. However, the lack of a probability sampling technique is not viewed as a limitation if you used a qualitative research design. In qualitative research designs, a non-probability sampling technique is typically selected over a probability sampling technique.

And this is just part of the puzzle?

Even if you used a quantitative research design, but failed to employ a probability sampling technique, there are still many perfectly justifiable reasons why you could have made such a choice. For example, it may have been impossible (or near on impossible) to get a list of the population you were studying (e.g., a list of all the 20,000 students at the single university you were interested in). Since probability sampling is only possible when we have such a list, the lack of such a list or inability to attain such a list is a perfectly justifiable reason for not using a probability sampling technique; even if such a technique is the ideal.

As such, the purpose of all the guides we have written on research limitations is to help you: (a) explain the nature of the limitations in your dissertation; and (b) justify the choices you made.

In helping you to justifying the choices that you made, these articles explain not only when something is, in theory , an obvious limitation, but how, in practice , such a limitation was not necessarily so damaging to the quality of your dissertation. This should significantly strengthen the quality of your Research Limitations section.

THE FORWARD LOOKING MOVE Suggesting how such limitations could be overcome in future

Finally, the forward looking move builds on the reflecting move by suggesting how the limitations you have discuss could be overcome through future research. Whilst a lot could be written in this part of the Research Limitations section, we would recommend that it is only around 10-20% of the total word count for this section.

Research Limitations

It is for sure that your research will have some limitations and it is normal. However, it is critically important for you to be striving to minimize the range of scope of limitations throughout the research process.  Also, you need to provide the acknowledgement of your research limitations in conclusions chapter honestly.

It is always better to identify and acknowledge shortcomings of your work, rather than to leave them pointed out to your by your dissertation assessor. While discussing your research limitations, don’t just provide the list and description of shortcomings of your work. It is also important for you to explain how these limitations have impacted your research findings.

Your research may have multiple limitations, but you need to discuss only those limitations that directly relate to your research problems. For example, if conducting a meta-analysis of the secondary data has not been stated as your research objective, no need to mention it as your research limitation.

Research limitations in a typical dissertation may relate to the following points:

1. Formulation of research aims and objectives . You might have formulated research aims and objectives too broadly. You can specify in which ways the formulation of research aims and objectives could be narrowed so that the level of focus of the study could be increased.

2. Implementation of data collection method . Because you do not have an extensive experience in primary data collection (otherwise you would not be reading this book), there is a great chance that the nature of implementation of data collection method is flawed.

3. Sample size. Sample size depends on the nature of the research problem. If sample size is too small, statistical tests would not be able to identify significant relationships within data set. You can state that basing your study in larger sample size could have generated more accurate results. The importance of sample size is greater in quantitative studies compared to qualitative studies.

4. Lack of previous studies in the research area . Literature review is an important part of any research, because it helps to identify the scope of works that have been done so far in research area. Literature review findings are used as the foundation for the researcher to be built upon to achieve her research objectives.

However, there may be little, if any, prior research on your topic if you have focused on the most contemporary and evolving research problem or too narrow research problem. For example, if you have chosen to explore the role of Bitcoins as the future currency, you may not be able to find tons of scholarly paper addressing the research problem, because Bitcoins are only a recent phenomenon.

5. Scope of discussions . You can include this point as a limitation of your research regardless of the choice of the research area. Because (most likely) you don’t have many years of experience of conducing researches and producing academic papers of such a large size individually, the scope and depth of discussions in your paper is compromised in many levels compared to the works of experienced scholars.

You can discuss certain points from your research limitations as the suggestion for further research at conclusions chapter of your dissertation.

My e-book,  The Ultimate Guide to Writing a Dissertation in Business Studies: a step by step assistance  offers practical assistance to complete a dissertation with minimum or no stress. The e-book covers all stages of writing a dissertation starting from the selection to the research area to submitting the completed version of the work within the deadline. John Dudovskiy

Reducing Suicide: A National Imperative (2002)

Chapter: 10 barriers to research and promising approaches, 10 barriers to research and promising approaches.

Given its unique nature, research on suicide faces a series of obstacles that limit progress in the understanding, prevention, and treatment of the problem. Because the field is a conglomeration of several disciplines that grew up independently, issues of interdisciplinary research pose problems of communication, jargon, and disciplinary rivalries (see IOM, 2000). Furthermore, recruiting researchers to the field is difficult because of the many obstacles that the field faces, as discussed in this chapter. As indicated in Chapter 1 , the terminology used among suicide researchers is inconsistent. Consequently, it is difficult to obtain reliable numbers about the incidence and prevalence of suicide and suicide attempts. Working with patients that present a risk of suicide presents ethical and safety concerns that can be difficult to resolve. Special measures must be taken to increase the statistical power of intervention and prevention studies, since suicide is a relatively infrequent event. These approaches range from using alternate endpoints such as suicidal ideation to finding ways to increase the size of the population under study. Each has limitations.

This chapter first explores the methodological issues that affect the collection of data. Next, it addresses the ethical and safety issues surrounding research protocols with suicidal participants. Statistical approaches to addressing some of these barriers are presented. Options for working with the limitation of suicide’s low base-rate are presented at various points in the chapter. Finally, the chapter presents a center-based approach that can be used to advance the study of suicide.

METHODOLOGY

Research on suicide is plagued with many methodological problems that limit progress in the field. Definitions lack uniformity, proximal measures are not always predictive of suicide, reporting of suicide is inaccurate, and its low frequency exacerbates all of these problems.

Terminology

There is a need for researchers and clinicians in suicidology to use a common language or set of terms in describing suicidal phenomena. Thirty years ago, NIMH convened a conference on suicide prevention at which a committee was charged with recommending a system for defining and communicating about suicidal behaviors (Beck et al., 1973). As a result of this committee’s work, operational definitions for basic terms such as suicidal ideation, suicide attempts, and completed suicide were proposed. Definitional issues were revisited in the mid-1990s at workshops held by the American Association of Suicidology, NIMH, and the Center for Mental Health Services, and through informal discussions among suicidologists (O’Carroll et al., 1996). Once again, the difficulties caused by lack of efficient communication and cross-talk were described, and a specific nomenclature with objective definitions of suicidal behaviors was proposed. Interestingly, many of the definitions proposed in this article were not appreciably different from those proposed for researchers more than a quarter of a century ago by the NIMH committee. Despite this seeming consensus, terminology continues to be an obstacle (see also Chapter 1 ). For example, “suicide attempt” does not uniformly include the intent to die. Since some who harm themselves do not actually intend to die (Linehan, 1986), assessing suicidal behavior is difficult. Not only are terms used differently across the field, they only infrequently are operationally defined in studies. Furthermore, often researchers do not reliably assess behavioral intent, since interviews can be unreliable (Linehan, 1997). Comparisons across studies also are complicated by differences in scales and instruments used to measure suicidality (see also Chapter 7 ). Many studies use only selected questions from questionnaires instead of the complete validated tool. Many of the studies do not report validity and reliability of instruments used.

Low Base-Rate Event

The base-rate of completed suicide is sufficiently low to preclude all but the largest of studies. When such studies are performed, resultant comparisons are between extremely small and large groups of individu-

als (suicide completers versus non-suicide completers, or suicide attempters versus non-suicide attempters). Use of suicidal ideation as an outcome can increase incidence and alleviate the problem to some extent; however, it is unclear whether suicidal ideation is a strong predictor of suicide completion. Using both attempts and completions can confound the analysis since attempters may account for some of the suicides completed within the study period. Because the duration of the prevention studies is frequently too brief to collect sufficient data on the low frequency endpoints of suicide or suicide attempt, proximal measures such as changes in knowledge or attitude are used. Yet the predictive value of these variables is unconfirmed. Statistical approaches (see Appendix A ) and proximal endpoints may provide solutions, but a large population base is preferable.

Psychological Autopsy

A psychological autopsy is the reconstruction of the events leading up to the death; ascertainment of the circumstances of the death, including suicidal intent; and an in-depth exploration of other significant risk factors for suicide (Beskow et al., 1991; Brent et al., 1988; Brent et al., 1993; Cooper, 1999; Hawton et al., 1998; Kelly and Mann, 1996; Velting et al., 1998). The psychological autopsy is the standard approach to augmenting the information obtained from a death certificate. Information is gathered through a semi-structured interview with key informants, and discrepancies are resolved by re-interviewing informants and through a case conference using “best estimate” procedures (Mitchell, 1982). Among the key issues and risk factors addressed are:

Circumstances and method of suicide

Psychopathology

Family history of psychopathology and suicidal behavior

Social adjustments and functioning

Personality characteristics, especially aggression/impulsivity

Life stressors and supports, including religion

Characteristics of treatment, especially in the 90 days prior to death.

Physical health and medical history

Socioeconomic background and family constellation

Communication of suicidal intent

Other record linkage (birth records, child welfare, school, criminal justice)

Most studies have found that the optimal time to conduct this kind of investigation is between 2 and 6 months after the death. Informants’ emo-

tions may be too raw to conduct an extensive interview prior to 2 months after the death. Longer than 6 months after the death, many informants want closure on the suicide and no longer are willing to open up and discuss emotionally difficult topics. The quality of information, measured by the number of diagnoses generated, did not vary as a function of the amount of time since the death (Brent et al., 1988). Caution should be used when interpreting information gathered from friends and relatives; one experimental study found that subjects’ descriptions of psychological distress varied with characteristics of the deceased and aspects of the manner of death (Telcser, 1996). Use of a comparison group of individuals who died accidentally by similar means could strengthen validity of findings. In general, when case-control methods are used in psychological autopsy studies, the comparisons are made to individuals who died by natural causes matched on demographic variables or psychiatric diagnoses.

The psychological autopsy has many similarities to the Family History-Research Diagnostic Criteria or any other indirect interview. The interview is less informative than a direct interview (Andreasen et al., 1977; 1986) but improves with the number of informants. Certain informants may provide specific information that may not be available from others. For example, friends of adolescent suicide victims may be more aware of substance use and abuse than parents (Brent et al., 1988). Employers and co-workers may be able to describe the victim’s functional ability on the job; for younger victims, interview of teachers and review of school records may play an analogous role.

Certain types of information are very difficult, or even impossible, to obtain with a psychological autopsy approach. For example, sexual orientation is information that the victim may have been subliminally aware of, or may not have confided to a friend or parent. Information processing style, or other laboratory-based measures obviously cannot be obtained without the victim’s self-report. However, psychological autopsy studies can help to identify living individuals whose characteristics closely resemble suicide victims who can then be studied using more dynamic assessments.

Combining biological findings with information about psychopathology, personality, family history, treatment history, and history of family adversity may provide a much more complete picture about the neurobiology of suicidal behavior. For example, altered serotonin in the brain may be a consequence of adverse rearing environments (Kaufman et al., 1998; Kraemer et al., 1989; Pine et al., 1997), and may very well be a consistent finding across different mental disorders. Conversely, psychological autopsy data may allow for the selection of relatively homoge-

neous sub-samples that can be subjected to genetic analyses. Complementary concurrent methods with intense, highly focused ethnography can improve knowledge about setting, process, motivations, and outcome, and thereby increase validity of data.

SURVEILLANCE OF SUICIDE AND SUICIDE ATTEMPTS

To address suicide as a public health problem requires the sustained and systematic collection, analysis and dissemination of accurate information on the incidence, prevalence and characteristics of suicide and suicide attempts. Surveillance is a cornerstone of public health, allowing realistic priority setting, the design of effective prevention initiatives, and the ability to evaluate such programs (IOM, 1999). Official suicide rates have been used to chart trends in suicide; monitor the impact of change in legislation, treatment policies, and social change; and to compare suicides across regions, both within and across countries. In addition, suicide rates have offered a way to assess risk and protective factors for geographical areas (counties, states and countries). However, there exist serious inadequacies in the availability and quality of information. The sources of data that are currently available remain “fragmentary and unlinked” (Berman, 2001). The need for improved and expanded surveillance systems is highlighted as one of the central goals of the National Strategy for Suicide Prevention (PHS, 2001).

Completed Suicide: Sources of Variability in Suicide Statistics

The suicide rate information available on a national level is derived from state vital records systems that collect data from local death certificate registries. States forward the information to the National Center for Health Statistics of the CDC which maintains the National Vital Statistics System (Davies et al., 2001). The utility and accuracy of these data are constrained by the variability in suicides statistics. As described in Chapter 2 , there are at least four sources of this variability (Jobes et al., 1987; O’Carroll, 1989), including:

regional differences in the definition of suicide and how ambiguous cases are classified

regional differences in the requirements and political arrangements for the office of coroner or medical examiner

differences in terms of the extent to which cases are investigated

variations that have to do with the quality of data management involved in preparing official statistics.

Ambiguous Cases

Classifications of deaths vary regionally (see also Chapter 6 ). Some jurisdictions, for example, require a suicide note in order to render a verdict of suicide, yet fewer than half of all suicide victims leave a note. Russian roulette deaths are called suicides in some jurisdictions but accidents in others (Keck et al., 1998). Religious traditions, life insurance policies, or actual legal sanctions may motivate underreports of suicide. Some jurisdictions tend to call any deaths with prominent intoxication an accident. All of these differences interfere in cross-site comparisons (Brent et al., 1987; McCarthy and Walsh, 1975).

The verdict of “undetermined” (also known as “open verdict” in the United Kingdom) harbors many unreported suicides, with estimates ranging from 50–100 percent of all undetermined cases being true suicides (Brent et al., 1987; Cavanagh et al., 1999; Holding and Barraclough, 1975; 1978; Ovenstone, 1973). Undetermined verdicts appear to be more likely if the victim is older, died by poisoning, and is female, perhaps because this profile may not fit the archetypal suicide completer (Ohberg and Lönnqvist, 1998; Ovenstone, 1973). Studies suggest that the official suicide rate underestimates the true rate by about 30 percent, but that time trends are unaffected by classification errors (Brent et al., 1987; Gist and Welch, 1989; Sainsbury and Jenkins, 1982).

There are other types of ambiguous cases that may be misclassified as accidents or homicides. For instance, some controversy exists about the degree to which vehicular deaths might be due to suicide (Jenkins and Sainsbury, 1980; Phillips and Ruth, 1993; Schmidt et al., 1977). “Victim precipitated suicides,” often in the context of “suicide by cop,” (Mohandie and Meloy, 2000; Wolfgang, 1958) are difficult to determine definitively, but may contribute to the underestimation of suicide. The availability of routine toxicology, physical evidence, autopsy, and psychological data can influence the classification of suicide. Larger jurisdictions may be able to investigate most comprehensively, making inter-jurisdiction comparisons unreliable (Nelson et al., 1978). Differential investigation by ethnic group and differences in willingness to share information with an investigator can also distort the picture of suicide.

Training and Background of Coroner/Medical Examiner

There are marked differences in the training and background of the persons who by law certify a death as a suicide among states within the United States (O’Carroll, 1989) and internationally (see also Chapter 6 ). In the United States, the qualifications range from simply having an interest in the job (e.g., Indiana) to specialized training in forensic pathology (e.g.,

Oklahoma). Medico-legal officials may be elected, appointed or serve exofficio (e.g., elected county sheriffs). Investigations may be centralized within a state (e.g., Rhode Island) or organized by each county (e.g., Utah). Each of these factors affects the nature, extent, and quality of the investigation and the classification of deaths as suicide.

Danish and English coroners differ significantly by their threshold for certification of suicide (Atkinson et al., 1975). Yet, one British study showed that changes in coroners (within a country) did not result in significant changes in certification practices (Sainsbury and Jenkins, 1982). While some have reported that examiners with a medical background are more willing to certify a suicide as such, others found that more highly trained medical examiners were more likely to classify a death as “undetermined” (Murphy et al., 1986; Pescosolido and Mendelsohn, 1986). More recent studies have suggested substantial variability between different coroners’ courts and even within courts (O’Donnell and Farmer, 1995). Furthermore, reported rates in countries that are predominantly Catholic, such as Ireland, may be artificially lower because of a greater stigma associated with suicide and consequent greater reluctance to certify a death as a suicide (Jobes et al., 1987; Myers and Farquhar, 1998). Similarly, the low rates of suicide reported by Muslim countries may reflect possible diagnosis or reporting bias due to stigma (Wasserman and Varnik, 1998). Ongoing concern exists about the lack of quality monitoring of the persistently idiosyncratic death certification process (Maudsley and Williams, 1996).

Local, State, and National Surveillance

National data provide perspective on the scale of the problem of suicide, and permit the evaluation of the impact of federal laws. Given the low-base rate of completed suicide, national level data are necessary to aggregate enough cases to identify patterns of suicide across populations. National data also allow for the analysis of variations in the suicide rate by regions of the country and by different environments (e.g., urban vs. rural). However, state and local data are essential in order to examine suicide as it occurs in specific communities. The map found later in this chapter ( Figure 10.1 ) demonstrates how suicide rates can vary widely across relatively small geographical areas. Understanding which specific qualities of the areas and populations tend to influence the suicide rate is critical for designing programs to enhance protective factors and reduce risk factors. Since many suicide prevention programs are implemented in community and school settings, more precise data are needed at these levels to be able to evaluate their effectiveness, recognize what services

FIGURE 10-1. Bayes Estimates of County-Level Deviations from the National Annual Suicide Rates per 100,000 (1996–1998). Adjusted for Age, Sex, and Race.

may still be needed, and identify populations that have not been targeted. In addition, the evaluation of state and local policies and laws can only occur through state and local data collected over time.

There is no precedent for federal law to require the reporting of health conditions to the national government. However, state regulations frequently mandate that details of various diseases and conditions be reported to the Centers for Disease Control and Prevention (CDC). For example, confirmed diagnoses of tuberculosis and various sexually transmitted diseases including AIDS are required by law to be reported in all states (Bunk, 1997). In the case of AIDS reporting, the CDC encouraged the states to pass such statutes by requiring the existence of surveillance regulation in order to receive funding for state AIDS prevention and treatment programs (Gostin et al., 1997). Similarly, for suicide surveillance, data should be collected at the local and state levels in a standardized manner so that it can be aggregated for a national reporting system.

Fatality Analysis Reporting System (FARS)

The potential benefits of a state-based, national reporting system for suicides are great. Such systems have successfully been used to monitor the incidence and characteristics of public health concerns such as infectious diseases and motor-vehicle injuries. For example, the National Highway Traffic Safety Administration (NHTSA) maintains the longstanding Fatality Analysis Reporting System (FARS) to track the circumstances and incidence of motor-vehicle related deaths, which are similar to suicide in number (~40,000/year) (Barber et al., 2000; NHTSA, 2001). The system collects detailed information from the 50 states, the District of Columbia, and Puerto Rico within 30 days of the occurrence. A FARS report includes over 100 coded pieces of data on each crash and the vehicle and people involved (Davies et al., 2001). A state employed FARS analyst collects the required information from a variety of sources: police accident reports, state vehicle registration files, state driver licensing files, state highway department data, vital statistics death certificates, coroner/ medical examiner reports, hospital medical records and emergency medical service reports (NHTSA, 2001).

Since its inception in 1975, surveillance data from FARS has improved our understanding of motor-vehicle injuries and the state and federal laws that affect traffic safety. For example, FARS data and vital statistics data were used to assess the effects of establishing 21 (vs. 18) as the minimum age to purchase alcoholic beverages (Cook and Tauchen, 1984; GAO, 1987). Based on their results, a federal law was passed that made federal highway funding to states contingent upon the establishment of 21 as the minimum age for purchasing alcohol (Wagenaar, 1993), and this policy is estimated to have saved 16,513 lives between 1975 and 1996 (NHTSA, 1996).

National Violent Death Reporting System (NVDRS)

A National Violent Death Reporting System (NVDRS) has been designed by researchers at the Harvard Injury Control and Research Center to collect information on homicides and suicides as well as other firearm deaths. It is based on FARS and on a pilot called the National Violent Injury Statistics System (NVISS) (Azrael et al., 2001).

Currently 11 states and metropolitan areas are collaborating with NVISS to design and pilot test NVDRS. Ten sites have received grants from the project and collect data covering the states of Connecticut, Maine, Maryland, Michigan, Utah, and Wisconsin and in Allegheny County (PA), Miami-Dade County, metropolitan Atlanta, and San Francisco (HICRC, 2001). Legislation for Fiscal Year 2002 funding programs under the De-

partments of Labor, Health and Human Services, and Education (P.L. 107-116) included designated funding to the CDC’s National Center for Injury Prevention and Control of $1.5 million for continued planning and preliminary implementation of the NVDRS in selected states (AAST, 2002). It is estimated that a fully implemented system covering every state would require approximately $20 million per year (AAST, 2002).

NVDRS collects information from four sources: death certificates, coroner/medical examiner reports, police Uniform Crime Reports (and, in some jurisdictions, police incident reports), and crime laboratories (HICRC, 2001). This diversity of sources is expected to allay some of the quality of data limitations that exist due to the irregular information available from the medical examiner/coroner system (see above, and IOM, 1999). NVDRS will collect detailed information on both victims and offenders, including basic demographics, substance use, relationship to one another, the circumstances leading to the injury, whether the event occurred at their home or work, specific of the incidents (e.g., date and location), and weapon type. For suicide deaths, this information will be supplemented by data on physical and mental health, treatment status, and possible precipitating life stresses. In the case of firearm involvement, the weapon’s type, make, model, and caliber will also be collected, and for deaths involving under-age shooters, information regarding how the weapon was obtained will be sought (HICRC, 2001). The researchers and pilot sites have developed uniform data elements, reporting protocols, and software for the reporting system (NFFIRS Workgroup, 2001; NVISS Workgroup, 2002). These technical details are centrally important to the future success of implementing this system on a national level; other concerns that must be addressed include ongoing technical assistance and extensive training to ensure quality and consistency of data (Gallagher, 2001).

Surveillance of Attempted Suicides

The quality of the data on suicide attempts is even more tenuous than that of completed suicides. The concerns about nomenclature (Garrison et al., 1991; O’Carroll et al., 1996) and accurate reporting (PHS, 2001) apply here even more than with suicide deaths. There is neither systematic nor mandatory reporting of suicide attempts in the United States. The two major sources of data on suicide attempts comes from the National Comorbidity Survey conducted between 1990 and 1992 (Kessler et al., 1999) and the Epidemiological Catchment Area study conducted in the 1980s (Moscicki et al., 1988). Other epidemiological data on attempts are available from small surveys in localized areas. Risk factors for attempts,

especially clinical factors, are surveyed, but often not through population-based surveys that would avoid the bias and lack of generalizability of clinical populations (Feinstein, 1977). Most information regarding suicide attempts must be collected from data systems designed for other purposes (PHS, 2001). This section describes a few of the potential models and sources of information on suicide attempts: the National Electronic Injury Surveillance System, the Youth Risk Behavior Survey, and the Oregon State Adolescent Suicide Attempt Data System.

National Electronic Injury Surveillance System (NEISS).

The NEISS has been operated by the U.S. Consumer Product Safety Commission (CPSC) for almost 30 years. In 2000, the system was expanded to collect data on all injuries, and since 1992 NEISS has collected information on all nonfatal firearm-related injuries seen in NEISS emergency rooms (Annest et al., 1995; Davis et al., 1996); some of these incidences may represent suicide attempts. NEISS is based on injury data gathered from the emergency departments of 100 representative hospitals selected as a probability sample of all 5,300+ U.S. hospitals with emergency departments (EDs) (grouped into 5 “strata,” four representing EDs of differing sizes and one from children’s hospitals) (CPSC, 2001).

Given its current sampling system, the utility of NEISS is limited; the data can only be used for national estimates and are invalid at regional, state and local levels (GAO, 1997). In addition, because it does not use the International Classification of Diseases (ICD) coding system both the detail of data collected and the ease with which the data can be shared with other systems are limited (AdvanceMed, 2001).

Youth Risk Behavior Survey (YRBS).

The YRBS is managed by the CDC and includes national, state, territorial, and local school-based surveys of representative samples of students in grades 9–12 in participating jurisdictions. Its intent is to monitor risk behaviors associated with the leading causes of injury and death among adolescents (Kann et al., 1998). In 1997, the YRBS was conducted by 38 states, 4 territories and 17 large cities, in addition to the national-level representative survey (STIPDA, 1999). Concerns regarding the validity of self-reports presents particular problems for collecting information on suicide attempts by the YRBS (Ivarsson et al., 2002). In addition, some jurisdictions, especially less populous ones, choose not to include the items about suicidality out of concerns for liability and imitation. This introduces bias into the results when comparing geographical areas. How-

ever, given that some suicide attempts may never come to medical attention or result in hospitalization, self-report measures like YRBS remain a valuable source of information.

Oregon State Adolescent Suicide Attempt Data System (ASADS)

In 1987, Oregon became the only state with a law requiring the reporting of suicide attempts by youth under 18 to the state health department (Hopkins et al., 1995). Failure to comply with this regulation is a Class A misdemeanor; however, it has been unnecessary to charge any hospital thus far (personal communication, D. Hopkins, Oregon Department of Human Services, April 19, 2002). Oregon law also specifies that the treating hospital must refer attempters to “in-patient or out-patient community resources, crisis intervention or other appropriate intervention by the patient’s attending physician, hospital social work staff or other appropriate staff 1 ”.

ASADS contains information such as demographics, date, county, method, place of attempt, living arrangement, psychological history, drug/alcohol use, previous attempt(s), reason(s), and seriousness and intent of attempt. Data are collected from emergency department records for all youth treated for a suicide attempt. Under-reporting is thought to occur. Training of staff and consistency of information is also an issue since only the information included in the patient’s medical chart can be collected. The system could be improved through better documentation by the health care provider (personal communication, L. Millet, Oregon Department of Human Services, April 22, 2002).

Limitations of the Model Systems

There exist many potential sources of information, but most often these are unlinked and in some cases represent redundant efforts. Most of what is available is based on hospitalization records or self-reports; however, studies have shown that injury surveillance based on hospitalization information alone may underestimate incidence by as much as 65 percent (Washington Department of Health, 1997). Currently, there is no systematic way of following repeat attempters over time. Elderly patients have among the highest rates of suicide completion, yet they are not included in some of the attempt surveillance systems that exist. Addi-

tional sources of information that could be consulted include emergency medical services data, school health services records, community and private health care providers data, etc. However, there are serious technical and practical limitations to integrating these sources.

External Cause of Injury codes (E-codes) were developed by the World Health Organization (WHO) as a supplemental code for use with the International Classification of Diseases (ICD). These codes provide a systematic way to classify diagnostic information that health care providers have entered into the medical record. They are standardized internationally and thus permit comparisons of data among communities, states, and countries (Educational Development Center, 1999). Since 1999 mortality data in the United States has been coded using the 10 th Revision of the ICD (ICD-10), while morbidity data is coded using a clinical modification of the 9 th Revision of the ICD (ICD-9CM) (Annest et al., 1998). ICD-10CM is currently being developed, and is expected to improve the specificity and accuracy for descriptions of non-fatal injuries (Annest et al., 1998). Currently, 26 states either mandate or have rates over 90 percent for use of E-codes in their Hospital Discharge Data Systems (ICRIN, 2001), and 11 states require their use in Emergency Department Data Systems (Annest et al., 1998).

The usefulness of E-codes for a surveillance system rests on the consistency of their use, and technical concerns regarding the compatibility of the format and type of different systems (CDC, 1995). For example, the number of permitted fields on reporting forms would need to be standardized since more than one field allows much more detailed and informative coding. The CDC is currently pilot-testing its National Electronic Disease Surveillance System (NEDSS), an initiative that will standardize public health data systems for infectious disease to allow integrated and electronically compatible national, state, and local surveillance systems. NEDSS also will support surveillance of other public health concerns including causes of injury (CDC, 2001b). In the future, use of a NEDSS compatible system will be a requirement for CDC surveillance funding of infectious diseases (CDC, 2001a). An ongoing international effort seeks to develop a new multi-axial classification system for external causes of injury which is intended to be used in both mortality and morbidity databases (Annest and Pogostin, 2000). Such a system needs to be compatible with existing data coding systems in order to maintain consistency of monitoring and to increase the feasibility of large-scale implementation (IOM, 1999).

Issues of Confidentiality

Surveillance systems are usually organized either by the name of the individuals, by a unique identification number, or by a record identification number for each incident. Each of these approaches presents methodological and ethical concerns. With a name-based system concerns of privacy and the possibility of reluctance to report could limit effectiveness and compliance. To avoid this, the Oregon ASADS system (discussed above) assigns a record number to each attempt (personal communication, D. Hopkins, Oregon Department of Human Services, April 19, 2002). Surveillance for HIV/AIDS provides a valuable precedent for use of a name-based system. The organization of the HIV/AIDS reporting system was and remains a highly contentious issue. This debate echoes a larger ongoing discussion regarding the privacy of health information in the computerized age. Unique identification numbers presented particular problems in pilot programs for expanding from AIDS to HIV/AIDS reporting with the recounting of cases at diagnosis and again with onset of the syndrome, and with incomplete reporting due to technical and operational difficulties (for review, see CDC, 1999). Critics of name-based systems cite concerns about the stigma of HIV/AIDS and the ensuing discrimination. Anecdotal evidence is cited that providing names would discourage individuals from getting tested. However, the CDC and six state health departments determined that rates of testing did not decrease when name-based testing was instituted (Nakashima et al., 1998).

Many of the same issues exist with suicide attempts, particularly with regard to tracking individuals over time. Because a previous suicide attempts is one of the strongest predictors of completed suicide, and repeat attempters are at higher risk for completed suicide, it is important to be able to track individuals over time.

All of these factors contribute to the tension that exists between the need for quality surveillance to promote the public’s health and an individual’s right to privacy. In the surveillance of many other health conditions (e.g., HIV/AIDS), state health departments remove identifying information prior to sending the data to the CDC for national reporting (CDC, 1995). The CDC also maintains specific administrative policies and technical program procedures to protect the security of both paper-based and electronic records. Reports are reviewed before public dissemination to ensure that potential individual identifiers are not released (CDC, 1995). Oregon’s ASADS does not collect identifying information such as the name or school of the attempter. In addition, data from ASADS are not released when there is a possibility of identification of a particular individual; more explicitly, data from a jurisdication are withheld if it reports fewer than 10 attempts or if the population at risk numbers less

than 50. There is limited staff access to the database, and they have not experienced any inappropriate releases of information (personal communication, D. Hopkins, Oregon Department of Human Services, April 19, 2002).

ETHICS AND SAFETY

Since suicide, by definition, involves intent to die, including people at risk of suicide in research on prevention and intervention presents a number of unique and complex ethical dilemmas. Ethical arguments can be made for both inclusion and exclusion of those at risk for suicide in clinical trials. Excluding suicidal participants has been standard industry practice for trials of psychoactive medicines in an effort to reduce the risk of death. On the other hand, excluding those at risk for suicide can be considered unethical since it precludes evaluation of treatments for this population. Furthermore, such screening is impractical since it is not possible to totally screen out people at risk for suicide.

Intervention research with suicidal patients is a complex and risky undertaking. The elements required for ethical intervention research with suicidal patients are similar to those for other types of clinical research: social and scientific value, scientific rigor and validity, fair participant selection, favorable risk–benefit ratio, independent review by a data and safety monitoring board, informed consent, and respect for potential and enrolled participants (Emanuel et al., 2000).

This section reviews the issues of informed consent and safe conduct of clinical trials and presents a statistical approach that can facilitate clinical research with suicidal participants.

Informed Consent

As reviewed earlier ( Chapter 3 ), most suicide is associated with a diagnosis of a major mental disorder. The National Bioethics Advisory Commission (1998) recently issued a report providing guidelines for research on persons with impaired decision making that focused on those with mental disorders. Depression, for example, was included because of the resultant impairment in information processing (Hartlage et al., 1993), reasoning (Baker and Channon, 1995) and possibly decision making (Elliot, 1997; Lee and Ganzini, 1992). Before a person can consent to be part of a clinical trial, they must understand the purpose, the risks, and the possible benefits of the research (National Bioethics Advisory Commission, 1998). To give informed consent requires the ability to express choice, to understand, to reason, and to appreciate the relevance to oneself of the research and any intervention it entails (Appelbaum and Grisso,

1995). It has been argued that a depressed person might understand the risks of a research protocol but may not care about, or may even welcome, the risks. (Elliot, 1997). On the other hand, one might argue that a desperately ill cancer patient or a patient with end-stage cardiac disease awaiting mechanical assistance may be equally unconcerned about the risks. We have no direct data regarding the decision-making capacity of suicidal patients. For some people, as seen with depressed patients, the hopelessness and despair may impair their reasoning about risks and benefits. There is a clear need for empirical research to test hypotheses on the capacity of suicidal patients to give informed consent.

Some mental disorders are accompanied by fluctuating decision-making ability (like bipolar disorder) or by progressive impairment (such as Alzheimer’s dementia). It is important to note that suicidal ideation also fluctuates and that suicidal acts, especially in the young, are often impulsive (Brent et al., 1999; Hawton et al., 1982). There are alternatives to providing consent at the time of the clinical trial (National Bioethics Advisory Commission, 1998). These options include advance directives 2 or the consent of a legally authorized representative. Another option is assent/ objection, which means that a person who may be partially impaired but still functional can enroll in a low-risk study from which he/she could withdraw at any time. Such approaches would be useful in studies with suicidal participants. Here, it may be appropriate and reasonable to utilize advance research directives or to reevaluate the patient’s understanding of the research at some point after the patient has entered the trial and is better able to understand, reason, and appreciate what is being asked of him/her.

In the absence of direct, empirical research on decision-making capacity in suicidal patients, additional safeguards to ensure safety and ethical conduct of research must entail involving family members or other surrogates. This is especially true in cases where there may be reduced capacity for consent such as with highly suicidal participants and/or members of special populations, including minors and elders, patients who are severely mentally ill or psychotic, and prisoners. Community consultation, the process of conferring with representatives who can adequately reflect the concerns of the prospective research participants, is also important under these circumstances. As risk rises, increased formality, objectivity, and documentation of capacity assessment and the informed consent process is wise.

No Undue Risk

Conducting clinical trials with suicidal patients involves managing risk at many levels and in many ways. As with all clinical trials, investigators need to take appropriate precautions to protect their participants in order to prevent exposure to unnecessary risk. Risk should not be greater than under ordinary, usual, or standard care. Commonly used precautions in clinical trials include frequent and ongoing team-based clinical monitoring of participants, an explicit protocol to follow in the face of acutely increased risk, the specification of rules for participant withdrawal (either temporarily or permanently), and the use of Data and Safety Monitoring Boards to assess: (1) the risk to benefit ratio for participants, (2) investigator adherence to protocol, and (3) the need for and appropriateness of study continuation versus termination (National Bioethics Advisory Commission, 2001). Other standards of protocol safety should include a process of informed consent and avoidance of coercion (see above), documentation of discussions about the clinical and protocol management of participants, adherence to national and local standards of care as appropriate, provision of a safety net for participants and their care givers, avoidance of conflict of interest on the part of investigators, and clarity about who pays the costs of care and injury (e.g., sponsor, institution, insurer; National Bioethics Advisory Commission, 2001). Finally, other risk reduction strategies may include experimental procedures such as the use of adaptive randomization, 3 as well as early termination of a study when the null hypothesis has been disconfirmed. These practices are standard for all clinical trials and can apply to suicidal patients as well. To address some of these concerns, NIMH issued a report on “Issues to Consider in Intervention Research with Persons at High Risk for Suicidality” (Pearson et al., 2000).

One example of how safety and ethical concerns for patients at risk for suicide can be addressed in an intervention trial is the PROSPECT study (Reynolds et al., 2001). In this study, primary care practices are randomly assigned to either the intervention arm of the study (which utilizes depression care managers to improve the recognition and treatment of depression) or to the usual care arm of the practice (which provides screening and assessment services, but no treatment; treatment re-

mains the prerogative of the primary care physician [PCP]). Treatment as usual in older primary care patients has been linked to under-recognition of depression and elevated rates of suicide completion. Although treatment as usual is a necessary and credible control condition, the ethical requirement not to expose participants to undue risk requires that patients and their PCP’s be informed of the results of psychiatric assessments performed as part of the PROSPECT protocol. Thus, if a patient in a treatment as usual practice is found to have suicidal and/or homicidal ideation, the patient’s PCP is promptly informed, as are the patient’s caregivers. Although this type of information enhances what is actually and usually available in usual care, it is necessary to meet the ethical demand to do no harm by withholding such crucial information. At the same time, however, this ethically necessary practice potentially prejudices a fair test of the main study hypothesis (intervention practices will lower rates of suicidal ideation, hopelessness, and suicidal behavior to a greater extent than treatment as usual practices). Nonetheless, ethical issues inevitably attend the conduct of intervention studies addressing life and death issues like suicide and must be dealt with forthrightly.

Exclusion from Trials

Because of the concerns about death by suicide, however, people who exhibit suicidal behaviors are often excluded from clinical trials (Pearson et al., 2000). The practice of excluding these patients from trials limits the opportunity for this population to benefit from such research. The Food and Drug Administration’s (FDA) requirement to prevent undue risk is frequently noted as the reason for the exclusion. However, suicidal behavior is not expressly excluded by the regulations (personal communication, P. David, FDA, November 15, 2001).

Institutional review boards (IRBs) 4 are charged with protecting participants in research trials to ensure that they are provided with all the information they need for informed consent, protected from unnecessary harm, provided with maximal benefit possible, and provided with appropriate (or usual) medical care if not part of the experimental arm of the protocol. 5 All research protocols using human participants are required to

have IRB approval. In carrying out this charge, IRBs often require patients at risk for suicide to be excluded from trials (Ethical considerations, 2001). The concerns are that the participants’ risk exceeds their benefit and that the investigators and/or the treatment protocols are inadequate to monitor and address suicide (Pearson et al., 2000).

Excluding suicidal patients from clinical trials has serious repercussions. The number of studies that assess changes in suicidal behavior with new pharmacological treatments is extremely limited (Linehan, 1997). Treatments for suicidality have not been, for the most part, subject to controlled clinical trials. The evidence base for care is lacking. Linehan in 1997 could locate only 20 clinical trials that selected suicidal participants. Among the 13 outpatient studies in this list, 6 excluded those at high risk for suicide. These 6 studies showed no significant effects of experimental treatment. In contrast, 6 of the 7 remaining studies that included high-risk patients were able to demonstrate effectiveness of an intervention. This analysis clearly demonstrates how critical it is to include suicidal patients in a trial if we are to develop effective treatment protocols for those at risk.

As discussed in Chapter 3 , suicide is, unfortunately, a medically expectable outcome of many mental illnesses. Death in a cancer clinical trial may be predictable, or even inevitable, but trials do not exclude terminally ill patients. Likewise, suicide attempts and completed suicide are also expectable, if unpredictable, events in severely mentally ill persons. One might logically reframe the perspective in this way: the outcome of suicide is a result of the mental illness, not the research or therapeutic intervention. Similarly, one might argue that a suicide attempt (or relapse) should not automatically exclude continuing participation, since it is necessary and desirable from a public health perspective to establish whether a particular intervention is effective in preventing further attempts or completed suicide in high-risk participants.

RESEARCH DESIGN AND ANALYSIS ISSUES

Statistical analysis and display of suicide data are complex problems, and there are many different approaches to their solution. This section considers several statistical issues in the design and analysis of studies of suicide. Many of the issues presented are well known in the suicide literature (e.g., estimating prevalence) whereas some are new to the study of suicide (e.g., use of empirical Bayes estimates in studying geographic variation in suicide rates). Throughout the chapter, a distinction is made between the analysis of suicide rates, where the unit of observation is typically a geographic area (e.g., a county) and the analysis of suicide as an outcome where the unit of analysis is an individual. The section dis-

cusses 1) the problem of computing lifetime risk of suicide and describes an appropriate methodology, 2) the problem of identifying suicide clusters, 3) statistical approaches that can inform suicide research, and 4) issues in the design of suicide studies. These issues include, case-control studies, risk-based allocation, and sample size and statistical power. Technical details of the statistical models are presented in Appendix A . The methods described here are by no means an exhaustive list of potentially useful approaches in the analysis of suicide data. It is hoped that these examples will provide a perspective on the power of appropriate statistical methodologies in suicide research.

Lifetime Risk of Suicide

Based on the work of Guze and Robins (1970), much of the psychiatric literature purports that 15 percent of depressed patients will die by suicide. To better understand the foundation of this estimate it is important to understand the various ways in which lifetime risk can be computed. In the case of Guze and Robins (1970), lifetime risk is defined as the proportion of the dead who died by suicide, often termed “proportionate mortality” (see also Goodwin and Jamison, 1990). As pointed out by Bostwick and Pankratz (2000), proportionate mortality is a reasonable estimator of lifetime risk only when the participants are followed until death. In general, however, the studies synthesized in the report by Guze and Robins, typically followed patients for no more than a few years. Furthermore, the participants were hospitalized psychiatric patients, often hospitalized as a precaution for suicide. Both this selection effect and the use of proportionate mortality as an estimator of lifetime risk, lead to an increase in the estimated lifetime prevalence. To obtain a more accurate assessment, Inskip, Harris, Barraclough (1998) calculated percent death by suicide to percent dead overall in a large number of studies. Analyses were stratified by diagnostic group (alcohol dependence, affective disorder, schizophrenia). Unfortunately, the majority of these studies had overall mortality rates of less than 50 percent, so the estimates of lifetime risk (i.e., 100 percent mortality) were extrapolated from the available data. Nevertheless, the lifetime suicide risk estimates were 7 percent for alcohol dependence, 6 percent for affective disorder, and 4 percent for schizophrenia.

In the most statistically rigorous approach to date, Bostwick and Pankratz (2000) compared proportionate mortality to “case fatality prevalence,” which is the number of suicides divided by the total number of patients at risk. Based on a synthesis of 29 studies of hospitalized affective disorder inpatients (19,723 patients), the pooled estimate of proportionate mortality prevalence was 20.0 percent, but only 4.1 percent for case fatal-

ity prevalence. As one might expect, in outpatients the rate for case fatality prevalence decreased to 2.0 percent in the analysis of 7 studies of 7,444 affective disorder outpatients. In contrast, the rate actually increased slightly to 24.6 percent for proportionate mortality prevalence.

Bostwick and Pankratz (2000) computed lifetime risk of suicide, using Bayes theorem, as the probability of suicide given death times the probability of death. For example, the overall probability of death in the 29 studies of affective disorder inpatients was 20 percent and of those, 20 percent died by suicide. The product of these two probabilities (i.e., the conditional probability of suicide given death and the prior probability of death) is the Bayes estimate of lifetime risk, which in this case is 4 percent. The Bayes estimate is remarkably close to the case fatality prevalence of 4.1 percent. This finding was consistent for all of the groups examined in their study (affective disorder outpatients = 2.2 percent, affective disorder inpatients = 4.0 percent, Guze and Robins data = 4.8 percent, Goodwin and Jamison data = 3 percent, and the general population = 0.5 percent).

Suicide Clustering

Suicidal behavior in adolescents is a major public health problem (NCHS, 1988). Data suggest that teen suicides often occur in temporal and geographic proximity of one another. This phenomenon is not unlike the concept of an outbreak of a disease in a particular community. Naturally, some clustering of suicides occurs by chance alone even if suicides occur at random. In the study of suicide clusters, the goal is to determine whether or not the outbreaks are occurring to an extent greater than would be expected by chance variation. Past studies have used various populations, such as psychiatric in-patients, high school and college students, marine troops, prison inmates, religious sects etc. (Gould et al., 1990). However, county of residence may be a more sensitive space unit to define a cluster (Gould et al., 1990).

Several statistical methods have been used to detect and statistically assess the time-space clustering of disease (see Gould et al., 1990). The Ederer-Myers-Mantel method (Ederer et al., 1964) is found to be sensitive to temporal clustering as well as time-space clustering. A method proposed by Knox (1964) considers all possible pairs of cases and the time and space distances between them. It establishes clustering by demonstrating a positive relation between the time and space distances of a pair, but required specification of the critical values for time and space to define closeness. This approach was modified by Smith (1982) to define “close in space” as occurring within the same geographic area. Wallenstein and colleagues (1989) provided a formula to assess the practical significance of clusters as well as the statistical significance. Gibbons et al. (1990)

took this further to decompose the overall distribution of suicide rates into a mixture of two Poisson distributions, the first to characterize the normal rate and the second to characterize the elevated rate, possibly due to one or more “suicide epidemics.” When they fit the model to 10 years (1977–1987) of monthly suicide rate data from Cook County (Chicago area), they found no evidence for a contribution of the second distribution. However, as described in Appendix A , using this analysis on the spatial distribution of suicide has identified qualitatively distinct geographic groupings of suicide rates across the United States. During the past decade, statistical research on finite mixture distributions has developed greatly (for review, see Böhning, 1999) and holds great promise for application to suicide. Appendix A describes the general statistical theory and developments.

Statistical Models for Assessment of Suicide Rates

Poisson regression models.

In the analysis of suicide rate data, Poisson regression models are a natural choice. With this approach, the data are modeled as Poisson counts whose means are expressed as a function of covariates. For example, the data may consist of yearly county-level suicide rates, broken down by age, sex, and race for that year. For these type of rate estimates a fixed-effects model is usually used. When there is a mixture of fixed (e.g., age, sex, and race) and random effects (e.g., unobservable county-specific effects), the more general mixed-effects Poisson regression model is used. In the case of suicide, the rates are considered to be nested within geographic locations (e.g., counties) and can represent multiple rates obtained over time (e.g., yearly suicide rates for a given county) or rates for different strata within a given county (e.g., males and females) or both. The random effects would modify the rate for each county from the population average.

Often, it is of interest to estimate values of the random effects within a sample. In the present context, these estimates would represent the deviation of the suicide rate for a given county from the national mean suicide rate, conditional on model covariates such as age, race, and sex, which may be either fixed or random effects. This can be done by using an empirical Bayes estimator of cluster -specific effects. 6 Thomas et al. (1992)

have used this kind of analysis to describe hospital mortality rates where cluster -specific (hospital-specific) effects represent how much the death rates for patients at a particular hospital differ from the national rates for patients with the same covariate values (i.e., matched patients). Longford (1994) provides extensive references to applications involving empirical Bayes estimates of random effects.

An alternative approach to the analysis of suicide rate data is based on Generalized Estimating Equations (GEEs) models, which were introduced by Liang and Zeger (1986) and Zeger and Liang (1986). The GEE method models the marginal expectation (i.e., average response for observations having the same covariates) of outcomes as a function of the explanatory variables. In this approach, the coefficients measure differences in the average response for a unit change in the predictor; in contrast the mixed-effect model produces predictions that are cluster -specific. An important property of the GEE method is that the parameter estimates are consistent even if the working correlation matrix is misspecified as long as the model for the mean is correct. A disadvantage of GEE is that it does not provide cluster -specific (e.g., county-level) suicide rate estimates adjusted for case mix (i.e., covariate effects). Appendix A outlines the statistical foundations of both fixed-effects and mixed-effects Poisson regression models, as well as the alternative approach based on GEE.

To illustrate how Poisson regression models can be used to estimate the effects of age, race, and sex on clustered (i.e., within counties) suicide rate data, this example considers the effects of age divided into five categories (5–14, 15–24, 25–44, 45–64, and 65 and older), sex, and race (African American versus Other) in the prediction of suicide rates across the United States for the period of 1996–1998. These categories were used so that there would be sufficient sample sizes available to compare observed and expected annual suicide rates for both GEE and mixed-effects Poisson regression models. In general, the GEE and mixed effect parameter estimates were remarkably similar.

Table 10-1 displays observed and expected annual suicide rates for both methods of estimation, broken down by age, sex, and race calculated from the parameter estimates. Inspection of Table 10-1 reveals several interesting results. In general, suicide increases with age, is higher in males, and is lower in African Americans. Black females have the lowest suicide rates across the age range. In non-Black males, the suicide rate increases with age whereas in all other groups, the suicide rate either is constant or decreases after age 65. Comparison of the expected frequencies for the GEE and mixed-effects models reveal that they are quite similar and the GEE does a slightly better job of predicting the observed rates.

A special feature of the mixed-effects model is the ability of estimating county-specific rates using empirical Bayes estimates of the random

TABLE 10-1 Observed and Expected Suicide Rates by Age, Race, and Sex

effects as described in the previous section. This allows an estimate of county-specific, expected suicide rates, which directly incorporate the effects race, sex, and age of that county. Table 10-2 provides a comparison of observed and expected numbers of suicides (1996-1998) for 100 randomly selected counties. Inspection of Table 10-2 reveals remarkably close agreement between observed and expected numbers of suicides.

This approach also allows the use of Bayes estimates directly to obtain county-level suicide rates adjusted for the effects of race, sex, and age. For example, a Bayes estimate of 1.0 represents an adjusted rate that is equal to the national rate, while a Bayes estimate of 2.0 represents a doubling of the national rate, and a Bayes estimate of 0.5 represents one-half of the national rate. Figure 10-1 (found on page 382) displays the Bayes estimates by county across the United States and reveals that even after accounting for these important demographic variables, considerable spatial variability remains. This map provides a useful tool for qualitative research into the etiology of suicide through an assessment of the spatial

TABLE 10-2 Observed and Expected Number of Suicides for 100 Randomly Selected Counties

distribution of Bayes estimates for outliers. For example, in the western continental United States and Alaska where suicide rates are typically high, a few counties have Bayes estimates consistent with the national average. Similarly, in the central United States where there is a high concentration of counties with the lowest suicide rates, a few counties exhibit the highest suicide rates. What are the risk and protective factors that have produced these spatial anomalies? Are these spatial anomalies simply due to reporting bias or some other unmeasured characteristic? Based on a review of the literature, it does not appear that this type of statistical approach to this problem has been previously considered. Examining these spatial anomalies in greater detail is a fruitful area for further research.

Mixed-effects Ordinal Regression Models

To study suicidal ideation, attempts, and completion in individual participants under various conditions, mixed-effects ordinal and nominal regression models can be used. The basic concept is to develop an ordinal scale of suicidal behavior, ranging from no suicidal ideation, low, medium, and high suicidal ideation, suicide attempt, and ending at suicide completion. Several authors have described models including both random and fixed effects (e.g., Agresti and Lang, 1993; Ezzet and Whitehead, 1991; Harville and Mee, 1984; Hedeker and Gibbons, 1994; Jansen, 1990; Ten Have, 1996). Statistical details are presented in Appendix A .

A reanalysis of the longitudinal data from Rudd et al. (1996) on suicidal ideation and attempts in a sample of 300 suicidal young adults (personal communication, Dr. M. David Rudd, Professor of Psychology, Baylor University) serves as an illustration of an application of the mixed-effects ordinal logistic regression model. In the original study, 180 participants were assigned to an outpatient intervention group therapy condition and 120 participants received treatment as usual. This re-analysis assigns the ordinal outcome measure of 0=low suicidal ideation, 1=clinically significant suicidal ideation, and 2=suicide attempt. Suicidal ideation was defined as a score of 11 or more on the Modified Scale for Suicide Ideation (MSSI, Miller et al., 1986). Model specification included main effects of month (0, 1, and 6) and treatment (0=control, 1=intervention), and the treatment by month interaction. Although data at 12, 18 and 24 months were also available, the dropout rates at these later months were too large for a meaningful analysis. In addition, to illustrate the flexibility of the model, depression as measured by the Beck Depression Inventory (BDI, Beck and Steer, 1987) and anxiety as measured by the Millon Clinical Multiaxial Inventory (MCMI-A, Millon, 1983) were treated as time-varying covariates in the model, to relate fluctuations in depressed

mood and anxiety to shifts in suicidality. Details of the analysis are in Appendix A .

Briefly, the analysis reveals that both of the time-varying covariates, depression and anxiety, were significantly associated with suicidality. By contrast, the treatment by time interaction was not significant, indicating that the intervention did not significantly effect the rate of suicide ideation or attempts over time. Treatment was found to be ineffective even after excluding the effects of anxiety and depression, which could mask treatment and treatment by time interactions. The random month effect was significant, indicating appreciable inter-individual variability in the rates of change over time.

Interval Estimation

Three types of statistical intervals, confidence intervals, prediction intervals, and tolerance intervals, can provide information relevant to suicide. A confidence interval can be used to describe our uncertainty in the overall suicide rate. A prediction limit can estimate an upper bound on a future rate. A statistical tolerance limit can set an upper bound on a specified proportion of all future monthly or yearly suicides incidences, with a specified level of confidence. General discussion of Poisson confidence, prediction, and tolerance limits are presented in Hahn and Meeker (1991) and Gibbons (1994), and in Appendix A .

As an example, consider the case in which after 61 months of observation for a particular county, with a population of 100,000 people, 123 suicides are recorded. The 95 percent confidence interval for the true population suicide rate is between 1.659 and 2.373 suicides per month per 100,000 (details of this computation are provided in Appendix A ). In contrast, a 95 percent upper prediction limit for the actual number of suicides in the next month is 4 suicides. Finally, to have 95 percent confidence that the limit will not be exceeded in 99 percent of all future months (not just the next single month), the 95 percent confidence 99 percent coverage tolerance limit is 7 suicides per month (see Appendix A for computational details). These calculations are useful to determine if an event (e.g., a television program about teen suicide) has an impact on suicide rates. If the number of suicides in the month following the event exceeds the calculated prediction (in the example, 4 suicides), then the television program may have been related to an increase in the suicide rate that is inconsistent with chance expectation based on the previous 61 months of data. If the tolerance limit is exceeded (in the example 7 suicides in the month), there is evidence that the rate is beyond chance expectations given the 61 months of historical data. The advantage of the

tolerance limit over the prediction limit is that the tolerance limit preserves the confidence level over a large number of future comparisons, whereas the prediction limit applies to a single future time period. Complete computational details are provided in Appendix A .

Study Design Issues

The following sections touch on some experimental design issues that may be useful in future studies of suicide either at the level of the individual or at the population level.

Case-Control Studies

Retrospective case-control studies are often used to examine risk factors of completed suicide. Logistic regression is typically the standard method for analysis of these case-control studies where multiple risk factors are assessed. For low-base rate conditions such as suicide, risk factors such as previous suicide attempts are often not seen in matched controls. For example, in a study by Brent et al. (1999) that attempted to relate past suicide attempts to completed suicide in female adolescents, 13 of 21 completed suicides, but none of the 39 controls, had a previous suicide attempt. The problem has been termed “complete separation” by Hosmer and Lemeshow (1989). As a consequence, parameter estimation becomes difficult and often the logistic regression model fails to converge. Some investigators (e.g., Shaffer et al., 1996) have simply eliminated previous attempts as a predictor. Chen, Iyengar, and Brent (unpublished) developed a hybrid model to handle the case of zero cells caused by variables like previous suicide attempts. The model is essentially a mixture of a standard logistic regression model estimated from both cases and controls and a risk estimate for previous attempts (or some other low-rate risk factor), which is the conditional probability of suicide given a past attempt. Parameters are estimated by subtracting the risk attributable to previous attempts from the overall risk and then modeling the residual risk over the rest of the risk factors using a logistic regression model. Innovative statistical approaches such as this are needed to deal with some of the special problems associated with the modeling of low-rate events such as completed suicide.

Risk-Based Allocation

Risk-based allocation, a non-randomized design could be quite useful in the study of suicide. It allows participants at higher risk, or with greater disease severity, to benefit from the potentially superior experimental

treatment, assuring that all of the sickest patients will receive the experimental treatment. Consequently, the design is sometimes called an “assured allocation” design (Finkelstein et al., 1996a; 1996b). Because the design is non-randomized, it should only be considered in those situations where a randomized trial would not be possible.

The design first requires a quantitative measure of risk, disease severity, or prognosis, which is observed at or before enrollment into the study, together with a pre-specified threshold for receiving the experimental therapy. All participants above the threshold receive the experimental treatment, while all participants below the threshold receive the standard treatment. The risk-based design also requires a prediction of what the outcomes would have been in the sicker patients if they had received the standard treatment. One example of such a model might be an appropriate regression model of the relationship between pre-treatment suicidal ideation on post-treatment suicidal ideation in a group of depressed patients treated with the standard antidepressant therapy. The validity of this model can then be tested by comparing the observed and predicted levels of the of suicidal ideation in the low-risk control participants that were given the standard treatment. This is the basis of another novel feature of the risk-based design: to estimate the difference in average outcome between the high-risk participants who received the experimental treatment, compared with what the same participants would have experienced on the standard treatment.

The model for the standard treatment (but only the standard treatment) needs to relate the average or expected outcome to specific values of the baseline measure of risk used for the allocation. Because the parameters of the model will be estimated from the concurrent control data and extrapolated to the high-risk patients, only the functional form of the model is required, not specific values of the model parameters. This offers a real advantage over historical estimates. All one needs to assume for the risk-based design is that the mathematical form of the model relating outcome to risk is correctly specified throughout the entire range of the risk measure. This is a strong assumption, to be sure, but with sufficient experience and prior data on the standard treatment, the form of the model can be validated.

The risk-based allocation clearly creates a “biased” allocation and, obviously, the statistical analysis appropriate to estimate the treatment effect is not the simple comparison of mean outcomes in the two groups, as it would be in a randomized trial. Instead, the theory of general empirical Bayes estimation can be applied (Robbins, 1993; Robbins and Zhang, 1988; Robbins and Zhang, 1989; Robbins and Zhang, 1991). There are several cautions to observe. The population of participants entering a trial with risk-based allocation should be the same as that for which the model

was validated, so that the form of the assumed model is correct. Clinicians enrolling patients into the trial need to be comfortable with the allocation rule, because protocol violations raise difficulties just as they do in randomized trials. Finally, the standard error of estimates will reflect the effect of extrapolation of the model predictions for the higher-risk patients based on the data from the lower-risk patients. Because of this, a randomized design with balanced arms will have smaller standard errors than a risk-based design with the same number of patients.

Sample Size Considerations

When designing studies for low-base rate events, such as suicide incidence, attempts, and ideation, statistical power considerations can lead to studies with quite different sample size requirements. The lower the base-rate and the greater the need for precision, the larger the population size necessary to achieve significance. In addition, large samples are required to increase the confidence limits. Table 10-3 provides a summary of required sample sizes for computing incidence rates and corresponding levels of precision, with three levels of confidence. For a 90 percent confidence interval for a rate of 10 per 100,000 (plus or minus 5 per 100,000), approximately 100,000 participants are needed. By contrast, if the assessment is in a high-risk population where the mean incidence is 50 per

TABLE 10-3 Sample Size Estimates for Estimating the Population Proportion of Suicide or Suicide Attempts

100,000 with a 95 percent confidence interval from 25 to 75 per 100,000, 30,717 participants are required. For measurement of risk of suicide attempts in a high-risk population (estimated at 10 percent, for instance), a sample size of 3458 is required for precision of 1 percent, but only 139 participants are needed for precision of 5 percent.

Study designs frequently compare suicide, or suicide attempts, or suicidal ideation as an outcome in two groups of individuals. Several factors affect the necessary sample size. Table 10-4 illustrates required sample sizes for a two-group comparison of suicide (completion, attempts, or ideation) rates based on a Type I error rate of 5 percent, power of 0.8, for one to ten repeated evaluations, assuming either intra-class (i.e., intrasubject) correlations (ICC) of 0.3, 0.5, or 0.7. The larger the ICC, the less independent information available, and the larger the required sample size for a given effects size. Furthermore, the larger the ICC, the less impact taking multiple repeated measurements has on the required number of participants. If there is only one repeated measurement, the sample

TABLE 10-4 Sample Size Estimates for Comparison of Two Groups

sizes are for a cross-sectional study. For more than one repeated measurement, the strength of the correlation of events over time must be considered and based on preliminary data. Obviously, for completed suicide, there would be only one repeated measurement and consequently the required sample size would be larger. Note that Table 10-4 provides an example of study sizes needed when the incidence of the outcome is at least 1 percent (i.e., 1000/100,000). As demonstrated in both Table 10-3 and Table 10-4 , population sizes must be much larger when working with a low base-rate event. The estimates in Table 10-4 are based on the method of Hsieh (1988).

Because of its low base rate, the difficulties in assessment, and the long-term, interdisciplinary nature of the risk and protective factors, the optimal approach to learn about suicide is to use large populations with cultural and genetic diversity for long-duration, interdisciplinary studies. A centrally coordinated, population-based approach would provide the infrastructure necessary to estimate more reliably the incidence of suicidal behavior (including attempts and completions) in different racial and ethnic groups and in different groups of mentally ill persons. Such an approach would also facilitate longitudinal studies of risk and protective factors (biological and psychosocial) and of preventive interventions. Research centers have often been the mechanism used to address similar research objectives. Large research centers have the additional advantage of being able to provide training opportunities and thereby attract new researchers to a difficult field. Furthermore, centers provide the opportunities for tissue banks and registries that are necessary resources to advance the field. At large research centers, the collaboration of researchers from different but complementary disciplines can significantly enhance progress. For example, integration of ethnographic assessments with psychological and biological evaluations can considerably deepen and contextually validate psychological autopsies to provide a better understanding of the interplay between community and individual risk, and to more successfully develop interdisciplinary preventive interventions that can be rigorously assessed.

In the early 1960s, there was a drive to understand the reasons for and the consequences of the rapidly increasing world population. To address these issues, the Ford Foundation funded the creation of several population research centers around the world, including centers at Georgetown University (2001), the University of Michigan (2002), and the University of the Philippines (2002), which are still flourishing today. The scientific questions required a prospective approach with many years of follow up

and a large population base to ensure statistical significance (Garenne et al., 1997). In these characteristics, the study of populations encountered some of the same challenges as the study of suicide. With the creation of the population research centers and ongoing support from the Ford Foundation, Rockefeller Foundation, and the Population Council, the field of demography blossomed. Research efforts created new links among biomedical research, economic analyses, and cultural context. It brought population issues to the attention of policy makers, especially in developing countries (Nagelberg, 1985).

More recently, research centers have been established to tackle difficult public health issues. Many different approaches have been used and some examples are described below.

To study the prevention and treatment of tobacco use, Transdisciplinary Tobacco Use Research Centers were created through a collaborative funding effort of the National Cancer Institute, NIDA and the Robert Woods Johnson Foundation. They created seven academic institutions with a commitment of $84 million over 5 years (NIDA, 1999). Each center is organized around a particular theme, such as Relapse, Tobacco Dependence, or Biobehavioral Basis of Use. Using cultural, genetic, behavioral, and other approaches, scientists at each center focus their research on their particular theme.

The Centers for Disease Control and Prevention (CDC) has established Injury Control Research Centers to explore the prevention, care and rehabilitation needs presented by various types of injury. In fiscal year 2001, the CDC had plans to fund ten centers (Injury prevention, 2001) at an estimated cost of $900,000 each (Grants for injury control, 2001) for up to 5 years each. The Centers are asked to conduct research in prevention, acute care, and rehabilitation and to serve as training centers and information centers for the public. Studies may be organized around a single theme but this is not a requirement. One such center at the University of North Carolina was funded during its first seven years (1987– 1993) at an average of $3 million per year. The CDC provided the core funding, which was 20–25 percent of the total (University of North Carolina, 2001).

The National Institute on Aging (NIA) supports about 30 Alzheimer’s Disease Research Centers to foster basic and clinical studies of Alzheimer’s Disease and related disorders. According to NIA’s request for proposals (NIA, 2000), the Centers are intended to “provide financial, intellectual, patient and tissue resources to support research projects that have been reviewed and supported on an individual basis … [and] to provide an environment that will strengthen research, increase productivity, and generate new ideas through formal interdisciplinary collaborative efforts.” The centers are mandated to include an administrative core, a clinical core

to recruit patients, a neuropathology core to archive specimens, and an educational core. The centers support short pilot projects as well as multiyear research efforts. NIA funds each center at a level up to $1.4 million each year for five years, with competitive renewals (NIA, 1998; NIA, 2000). In 1994, NIA was committing $35 million per year on 28 centers (Benowitz, 1996). By supporting both basic and clinical research, the centers enhance the translation of research advances into improved care and diagnosis. With the advent of the centers, progress in understanding neurodegenerative dementias has been remarkable (see IOM, 2000).

The National Cancer Institute (NCI) has established 60 cancer centers throughout the United States to provide a broad-based, interdisciplinary effort in cancer research (NCI, 2001a). NCI has found that the centers provide the opportunity to apply complex research strategies and to undertake novel multidisciplinary approaches to the critical questions facing prevention, assessment, and treatment of cancer (NCI, 2001b). Because of the long duration of funding and centralized support that centers provide, it is possible to follow through on studies of etiology and treatment that would be difficult to undertake in a different setting (NCI, 2001b). Many of the same issues facing researchers in suicide (assessment of risk factors, the necessity for large clinical studies, multidisciplinary issues, etc.) have been successfully addressed through these centers. The Cancer Centers vary greatly in their structure and scope. The centers might encompass basic science, clinical trials, population studies, and/or clinical care. They might be part of a university, freestanding institutes, or jointly run by multiple institutions (NCI, 2000a). In Fiscal Year 2000, $169 million was allocated for funding these 60 centers through NCI (NCI, 2000b). This averages to almost $2.8 million for each center that year. Because of the many sources from which cancer researchers can obtain funds, the general guideline of NCI is that NCI funding should be no more than 20 percent of the total funding of the center (NCI, 2001c). This means that if NCI provides $2.5 million per year to a center, it is expected that the total annual funding is about $12.5 million. Although the support from NCI is only a small part of the total, it is key to the support of the infrastructure, provides shared resources, and allows flexibility in the use of funds. Through these means, NCI funding of the centers helps to stimulate innovation and collaboration in cancer research (NCI, 2000a).

The Conte Center for the Neuroscience of Mental Disorders, funded by the National Institute of Mental Health, is an example of another approach to research centers. In 2000, there were eleven Conte centers (Hyman, 2000). Each of these centers is organized around a single hypothesis. The Centers are designed to optimize the use of resources to address a specific scientific problem. Center funding is not intended to provide full research support for investigators; they are expected also to have

individual grant support. To coordinate the activities among these centers, the directors are expected to attend an annual meeting (NIMH, 1998). Funding varies with each center’s needs. In 2001, Washington University was awarded $2 million over 3 years to establish a center for brain-mapping projects to enhance understanding of schizophrenia and other psychiatric disorders (Washington University, 2001). In 2000, Emory University received $13 million over 5 years to support their activities on clinical depression (Emory University, 2000).

The committee believes that suicide is best served by a similar mechanism that would allow a nationally coordinated effort that can be extended to global populations through international collaborations. As the discussion above points out, there are many options for organizing centers: core funding, hypothesis, theme, or flexible support. The committee did not find evidence that any one approach was preferable to another. Whatever structure is chosen, such centers would allow better estimates of incidence, improved evaluation of risk and protective factors, longitudinal studies that could assess intervention and prevention, and development of tissue repositories for genetic and other biological analyses. Ancillary benefits of such a center would be the wealth of data that would be collected on mental illness and substance abuse, major risk factors for suicide, and correlation with detailed social data. Because schizophrenia, mood disorders, substance abuse, and personality disorders are associated with increased incidence of suicide, studies of suicide will also address these important psychiatric conditions and lead to a better understanding of and possible interventions for the mental disorders. These are also the appropriate settings to conduct more rigorous evaluation of suicide prevention projects that link community- and individual-level evaluations.

Although suicide is less common than cancer, injury, Alzheimer’s Disease, or tobacco use, suicide research has parallels to these other center-based research efforts. Like these other areas of investigation, suicide centers would focus an effort in an area of high significance for public health but where inroads toward solutions have been heretofore limited. While research in suicide has been stymied by inadequate populations for investigation, there is a wealth of understanding about many risk and protective factors including behavioral, psychological, and cultural aspects of the problem. As with other health areas, integration of multiple perspectives will generate new questions and hypotheses that can be expected to jump-start an understanding of mechanisms and possible interventions. Particularly because of the limits of our current knowledge about suicide, these centers are especially important for future progress. The same benefits that centers have provided to other disease initiatives can be expected for suicide.

Currently there are a small number of suicide research centers around the world. In the United States, for example, these include the Suicide Prevention Research Center at the University of Nevada School of Medicine funded by the CDC (NCIPC, 2002) and the University of Rochester’s Center for the Study and Prevention of Suicide funded by NIMH (University of Rochester, 2001). Internationally there are centers in Oxford, England (University of Oxford, 2002); Gent, Belgium (Ghent University, 2001); Hamburg, Germany (University of Hamburg, 2000); Stockholm, Sweden (IPM, 2001); and others. The scale of these centers is relatively small; their interactions are limited. As proposed, the population research centers would enhance the population database for research among these investigators and provide an integrated network for coordination of effort and collaborative study.

Other alternatives might achieve some of these goals. Broad assessments of risk factors can be accomplished by including suicidal endpoints in all large epidemiological studies and this is an important approach to enhancing the research effort on suicide. Adding suicide cores to centers focussed on mental illness might also increase the collection of data regarding suicide. But these less expensive alternatives will not address the most serious obstacle to understanding suicide: the need for a large, a well-characterized population that would allow links to be made about causes, risks, protective factors, and successful interventions. Such a wellcharacterized and carefully studied population is essential for evaluating biological markers and genetic bases of suicide as well as the social and cultural influences. Currently the database is inadequate. Additionally, individual projects cannot access the large populations necessary to come to significant conclusions regarding this low base-rate event. As the statistical analysis above points out, at a suicide rate of 10 per 100,000 population, approximately 100,000 participants are needed to achieve statistical significance. In studying suicide among low-risk groups, the numbers needed are even greater.

While each center might be able to obtain a sufficiently large sample for studies in the general population, a consortium of centers will be necessary to fully explore differences based on region, economic environment, culture, urbanization, and other factors that vary across the country. In addition, one center might be responsible for coordinating international data to increase the understanding of suicide on a global scale. Furthermore, certain subpopulations may be sufficiently small or low risk to require broader recruitment than one center could access. The coordinated network of centers is an optimal way to achieve these goals. In addition, the centers will allow a comprehensive approach to understanding suicide from a truly interdisciplinary perspective with both cross sectional and longitudinal studies.

Suicide is responsible for over 30,000 deaths each year. Mental illness, the primary risk factor, afflicts over 80 million people in the United States; almost 15 million people have a serious mental illness (i.e, a mental disorder that leads to a functional impairment). For comparison, breast cancer claims the lives of about 40,000 women per year and between 10–15 million people are living with the diagnosis of breast cancer. In 1998 over $400 million from NCI and the Army programs was allocated to research into the prevention, treatment, and cure of breast cancer (IOM, 2001). From estimates of the portfolios of SAMHSA, CDC, and NIMH, the funding for suicide was less than $40 million in 2000. The committee finds that this is disproportionately low, given the magnitude of the problem of suicide. A substantial investment of funds is needed to make meaningful progress.

Suicidology has faced serious methodological limitations, including inconsistencies in definitions and misclassification of deaths by medical examiners or coroners. The quality of the data on suicide attempts is even less reliable than that for completed suicide. The low base-rate of suicide has necessitated the use of proximal endpoints in lieu of suicide itself and has created difficulties in computing the risk of suicide and statistically analyzing results.

The field should endorse common definitions and psychometrically sound measures. Better training of coroners and medical examiners in certification of suicide needs to be developed and provided. Registries of suicide attempts for clinical surveillance and research purposes that have the capacity to foster, among other things, international community-oriented studies that integrate clinical, biological, and social data should be created. Finally, novel applications of statistical analyses and research designs should be employed to advance research on suicide.

Large sample sizes are required in order to provide statistical power for studies of events with a low base rate. For example, to determine the overall incidence rate of suicide in a general population within plus or minus 5 per 100,000 with 90 percent confidence, would require about 100,000 participants. Many statistical approaches exist that can be effectively applied to the field of suicide.

Researchers in suicide should seek out appropriate methodologies and statistical consultation to most effectively design protocols and analyze data.

Suicidal individuals have been largely excluded from clinical trials. Evidence suggests, however, that high-risk participants can benefit from treatment trials and that certain research designs can increase the safety of participants. Evidence-based treatment protocols for suicidality are seriously lacking, making clinical trials critical.

Clinical trials of drug and psychotherapy treatment should include suicidal patients. Appropriate safeguards, including safe study designs, should be used with such populations.

Treatment trials often exclude suicidal individuals because of liability concerns. Re-framing this mortality risk to include comparison with risks in other types of medical trials underscores that suicide is an unfortunately expectable outcome of mental illness for some percentage of afflicted individuals. Suicide is not a consequence of the research or therapeutic intervention.

Inform the public and Institutional Review Boards that mental illnesses are potentially fatal, and that for some percentage of individuals, suicide represents an unfortunately expectable outcome of mental illness.

Lack of longitudinal and prospective studies remains a critical barrier to understanding and preventing suicide. Understanding the interactive risk and protective factors and their developmental course necessitates prospective, transactional research. Long-term assessment must occur for preventive interventions to be properly evaluated. Large, interconnected research centers have helped advance the science base regarding other societal problems, and are expected to do so for suicide, as well.

A national network of suicide research population laboratories devoted to interdisciplinary research on suicide and suicide prevention across the life cycle should be developed. Such an approach would redress many of the current methodological limitations and resultant lags in progress.

AAST (American Association for the Surgery of Trauma). 2002. National Violent Death Reporting System Receives Initial Funding . [Online]. Available: http://www.aast.org/nvdrs.html [accessed April 15, 2002].

AdvanceMed. 2001. Audit of the National Electronic Injury Surveillance System (NEISS) Hospitals. Final System-Wide Analysis Report to the U.S. Consumer Product Safety Commission .

Agresti A, Lang JB. 1993. A proportional odds model with subject-specific effects for repeated ordered categorical responses. Biometrika , 80: 527-534.

Andreasen NC, Endicott J, Spitzer RL, Winokur G. 1977. The family history method using diagnostic criteria. Reliability and validity. Archives of General Psychiatry , 34(10): 1229-1235.

Andreasen NC, Rice J, Endicott J, Reich T, Coryell W. 1986. The family history approach to diagnosis. How useful is it? Archives of General Psychiatry , 43(5): 421-429.

Annest JL, Fingerhut LA, Conn JM, Pickett D, McLoughlin E, Gallagher S. 1998. How States Are Collecting and Using Cause of Injury Data: A Survey on State-Based Injury Surveillance, External Causeof Injury Coding Practices, and Coding Guidelines in the 50 States, the District of Columbia, and Puerto Rico . Washington, DC: American Public Health Association.

Annest JL, Mercy JA, Gibson DR, Ryan GW. 1995. National estimates of nonfatal firearm-related injuries. Beyond the tip of the iceberg. Journal of the American Medical Association , 273(22): 1749-1754.

Annest JL, Pogostin CL. 2000. A Pilot Test of the Centers for Disease Control and Prevention Short Version of the International Classification of External Causes of Injury (ICECI). Report to the World Health Organization Collaborating Centers on the Classification of Disease . Atlanta, GA: Office of Statistics and Programming, National Center for Injury Prevention and Control, Centers for Disease Control and Prevention.

Appelbaum PS, Grisso T. 1995. The MacArthur Treatment Competence Study. I: Mental illness and competence to consent to treatment. Law and Human Behavior , 19(2): 105-126.

Atkinson MW, Kessel N, Dalgaard JB. 1975. The comparability of suicide rates. British Journal of Psychiatry , 127: 247-256.

Azrael D, Barber C, Mercy J. 2001. Linking data to save lives: Recent progress in establishing a National Violent Death Reporting System. Harvard Health Policy Review , 2(2).

Baker JE, Channon S. 1995. Reasoning in depression: Impairment on a concept discrimination learning task. Cognition and Emotion , 9: 579-597.

Barber C, Hemenway D, Hargarten S, Kellermann A, Azrael D, Wilt S. 2000. A “call to arms” for a national reporting system on firearm injuries. American Journal of Public Health , 90(8): 1191-1193.

Beck AT, Davis JH, Frederick CJ, Perlin S, Pokorny AD, Schulman RE, Seiden RH, Wittlin BJ. 1973. Classification and nomenclature. In: Resnick HLP, Hathorne BC, Editors. Suicide Prevention in the Seventies . (pp. 7-12). Washington, DC: U.S. Government Printing Office.

Beck AT, Steer RA. 1987. Beck Depression Inventory: Manual . New York: Psychological Corporation.

Benowitz S. 1996. Gerontologist’s provocative questions: Does NIA spend too much on Alzheimer’s? The Scientist , 10(1): 5.

Berman AL. 2001. Assessing and recording violent injuries: The need for a national violent death reporting system. A panel discussion held at the 20th Annual American Medical Association Science Reporters Conference, October 28-29, 2001, San Francisco, CA. Summary available in: American Medical Association. National Violent Death Reporting System Needed: Data From a National Reporting System Could Help Prevent Deaths [Online]. Available: http://www.ama-assn.org/ama/pub/print/article/4197-5468.html [accessed April 15, 2002].

Beskow J, Runeson B, Asgard U. 1991. Ethical aspects of psychological autopsy. Acta Psychiatrica Scandinavica , 84(5): 482-487.

Böhning D. 1999. Computer-Assisted Anaysis of Mixtures and Applications. Meta-Analysis, Disease Mapping and Others . Boca Raton, FL: Chapman and Hall CRC.

Bostwick JM, Pankratz VS. 2000. Affective disorders and suicide risk: A reexamination. American Journal of Psychiatry , 157(12): 1925-1932.

Brent DA, Baugher M, Bridge J, Chen T, Chiappetta L. 1999. Age- and sex-related risk factors for adolescent suicide. Journal of the American Academy of Child and Adolescent Psychiatry , 38(12): 1497-1505.

Brent DA, Perper JA, Allman CJ. 1987. Alcohol, firearms, and suicide among youth: Temporal trends in Allegheny County, Pennsylvania, 1960 to 1983. Journal of the American Medical Association , 257(24): 3369-3372.

Brent DA, Perper JA, Kolko DJ, Zelenak JP. 1988. The psychological autopsy: Methodological considerations for the study of adolescent suicide. Journal of the American Academy of Child and Adolescent Psychiatry , 27(3): 362-366.

Brent DA, Perper JA, Moritz G, Allman CJ, Roth C, Schweers J, Balach L. 1993. The validity of diagnoses obtained through the psychological autopsy procedure in adolescent suicide victims: Use of family history. Acta Psychiatrica Scandinavica , 87(2): 118-122.

Bunk S. 1997. National HIV reporting approaches, but privacy remains paramount. The Scientist , 11(22): 1.

Cavanagh JT, Owens DG, Johnstone EC. 1999. Suicide and undetermined death in south east Scotland. A case-control study using the psychological autopsy method. Psychological Medicine , 29(5): 1141-1149.

CDC (Centers for Disease Control and Prevention). 1995. Integrating Public Health Information and Surveillance Systems: A Report and Recommendations . [Online]. Available: http://www.cdc.gov/od/hissb/docs/katz.htm [accessed April 24, 2002].

CDC (Centers for Disease Control and Prevention). 2001a. National Electronic Disease Surveillance System: Frequently Asked Questions . [Online]. Available: http://www.cdc.gov/nedss/About/faq.htm#injury_surveillance [accessed April 16, 2002].

CDC (Centers for Disease Control and Prevention). 2001b. Supporting Public Health Surveillance Through the National Electronic Disease Surveillance System . [Online]. Available: http://www.cdc.gov/od/hissb/docs.htm#nedss [accessed April 16, 2002].

CDC (Centers for Disease Control and Prevention). 1999. CDC guidelines for national human immunodeficiency virus case surveillance, including monitoring for human immunodeficiency virus infection and acquired immunodeficiency syndrome. Morbidity and Mortality Weekly Report , 48(RR-13): 1-29.

Chen T, Iyengar S, Brent D. unpublished. A Hybrid Logistic Model for Case-Control Studies . Unpublished manuscript, Bowling Green State University.

Cook PJ, Tauchen G. 1984. The effect of minimum drinking age legislation on youthful auto fatalities. Journal of Legal Studies , 13: 169-190.

Cooper J. 1999. Ethical issues and their practical application in a psychological autopsy study of suicide. Journal of Clinical Nursing , 8(4): 467-475.

CPSC (Consumer Product Safety Commission). 2001. National Electronic Injury Surveillance System . CPSC Document #3002. [Online]. Available: http://www.cpsc.gov/cpscpub/pubs/3002.html [accessed April 15, 2002].

Davies M, Connolly A, Horan J, Editors. 2001. State Injury Indicators Report . Atlanta, GA: Centers for Disease Control and Prevention, National Center for Injury Prevention and Control.

Davis Y, Annest JL, Powell KE, Mercy J.A. 1996. Evaluation of the National Electronic Injury Surveillance System for use in monitoring nonfatal firearm injuries and obtaining national estimates. Journal of Safety Research , 27: 83-91.

Ederer F, Meyers MH, Mantel N. 1964. A statistical problem in space and time: Do leukemia cases come in clusters? Biometrics , 20: 626-638.

Educational Development Center. 1999. E Codes . [Online]. Available: http://www.edc.org/HHD/csn/buildbridges/bb2.2/ECODES.html [accessed April 15, 2002].

Elliot C. 1997. Caring about risks: Are severely depressed patients competent to consent to research? Archives of General Psychiatry , 54: 113-116.

Emanuel EJ, Wendler D, Grady C. 2000. What makes clinical research ethical? Journal of the American Medical Association , 283(20): 2701-2711.

Emory University. 2000. Does Abuse Breed Depression? Emory Magazine , 75(4): [Online]. Available: http://medicine.wustl.edu/~wumpa/news/2001/conte.html [accessed May 10, 2002].

Ethical considerations in suicide research debated. 2001. [Online]. Available: http://www.suicidology.org/newslinksum42001.htm [accessed December 4, 2001].

Ezzet F, Whitehead J. 1991. A random effects model for ordinal responses from a crossover trial. Statistics in Medicine , 10: 901-907.

Feinstein AR. 1977. Clinical biostatistics. XLI. Hard science, soft data, and the challenges of choosing clinical variables in research. Clinical Pharmacology and Therapeutics , 22(4): 485-498.

Finkelstein MO, Levin B, Robbins H. 1996a. Clinical and prophylactic trials with assured new treatment for those at greater risk: II. Examples. American Journal of Public Health , 86(5): 696-705.

Finkelstein MO, Levin B, Robbins H. 1996b. Clinical and prophylactic trials with assured new treatment for those at greater risk: I. A design proposal. American Journal of Public Health , 86(5): 691-695.

Gallagher SS. 2001. Critical Needs in Implementing a National Violent Death Reporting System— Results From a Process Evaluation . Paper presented at the 129th annual meeting of the American Public Health Association in Atlanta, Georgia, October 21-25, 2001.

GAO (General Accounting Office). 1987. Drinking-Age Laws: An Evaluation Synthesis of Their Impact on Highway Safety . GAO/PEMD-87-10. Washington, DC: GAO.

GAO (General Accounting Office). 1997. Consumer Product Safety Commission: Better Data Needed to Help Identify and Analyze Potential Hazards . GAO/HEHS-97-147. Washington, DC: GAO.

Garenne M, Das Gupta M, Pison G, Aaby P. 1997. Introduction. In: Das Gupta M, Aaby P, Garenne M, Pison G, Editors. Prospective Community Studies in Developing Countries . (pp. 1-15). New York: Oxford University Press.

Garrison CZ, Lewinsohn PM, Marsteller F, Langhinrichsen J, Lann I. 1991. The assessment of suicidal behavior in adolescents. Suicide and Life-Threatening Behavior , 21(3): 217-230.

Georgetown University. 2001. History of Demography Department . [Online]. Available: http://www.georgetown.edu/departments/demography/main/AboutDept/history.htm [accessed May 10, 2002].

Ghent University. 2001. Unit for Suicide Research . [Online]. Available: http://allserv.rug.ac.be/~cvheerin/ [accessed May 10, 2002].

Gibbons RD. 1994. Statistical Methods for Groundwater Monitoring . New York: Wiley.

Gibbons RD, Clark DC, Fawcet J. 1990. A statistical method for evaluating suicide clusters and implementing cluster surveillance. American Journal of Epidemiology , 132 (Suppl 1): S183-S191.

Gist R, Welch QB. 1989. Certification change versus actual behavior change in teenage suicide rates, 1955-1979. Suicide and Life-Threatening Behavior , 19(3): 277-288.

Goodwin FK, Jamison KR. 1990. Manic-Depressive Illness . New York: Oxford University Press.

Gostin LO, Ward JW, Baker AC. 1997. National HIV case reporting for the United States. A defining moment in the history of the epidemic. New England Journal of Medicine , 337(16): 1162-1167.

Gould MS, Wallenstein S, Kleinman M. 1990. Time-space clustering of teenage suicide. American Journal of Epidemiology , 131(1): 71-78.

Grants for injury control research centers. Program announcement 01007. 2001. HIVDENT. [Online]. Available: http://www.hivdent.org/infctl/ic1679092000.htm [accessed December 19, 2001].

Guze SB, Robins E. 1970. Suicide and primary affective disorders. British Journal of Psychiatry , 117(539): 437-438.

Hahn GJ, Meeker WQ. 1991. Statistical Intervals: A Guide for Practitioners . New York: Wiley.

Hartlage S, Alloy LB, Vazquez C, Dykman B. 1993. Automatic and effortful processing in depression. Psychological Bulletin , 113(2): 247-278.

Harville DA, Mee RW. 1984. A mixed-model procedure for analyzing ordered categorical data. Biometrics , 40: 393-408.

Hawton K, Appleby L, Platt S, Foster T, Cooper J, Malmberg A, Simkin S. 1998. The psychological autopsy approach to studying suicide: A review of methodological issues. Journal of Affective Disorders , 50(2-3): 269-276.

Hawton K, Cole D, O’Grady J, Osborn M. 1982. Motivational aspects of deliberate self-poisoning in adolescents. British Journal of Psychiatry , 141: 286-291.

Hedeker D, Gibbons RD. 1994. A random effects ordinal regression model for multilevel analysis. Biometrics , 50: 933-944.

HICRC (Harvard Injury Control Research Center). 2001. National Violent Injury Statistics System: Linking Data to Save Lives . [Online]. Available: http://www.hsph.harvard.edu/hicrc/nviss/ [accessed April 15, 2002].

Holding TA, Barraclough BM. 1975. Psychiatric morbidity in a sample of a London coroner’s open verdicts. British Journal of Psychiatry , 127: 133-143.

Holding TA, Barraclough BM. 1978. Undetermined deaths—suicide or accident? British Journal of Psychiatry , 133: 542-549.

Hopkins DD, Grant-Worley JA, Fleming DW. 1995. Fatal and nonfatal suicide attempts among adolescents—Oregon, 1988–1993. Morbidity and Mortality Weekly Report , 44(16): 312-315; 321-323.

Hosmer D, Lemeshow S. 1989. Applied Logistic Regression . New York: Wiley.

Hsieh FY. 1988. Sample size formulae for intervention studies with the cluster as unit of randomization. Statistics in Medicine , 7(11): 1195-1201.

Hyman SE. 2000. Director’s Column . [Online]. Available: http://apu.sfn.org/NL/2000/May-June/articles/director.html [accessed May 10, 2002].

ICRIN (Injury Contol Research Information Network). 2001. Status of Hospital Discharge ECoding by State, U.S. [Online]. Available: http://www.injurycontrol.com/icrin/Ecode%20tracking.htm [accessed April 22, 2002].

Injury prevention and control research and state and community based programs. 2001. Catalog of Federal Domestic Assistance. [Online]. Available: http://aspe.os.dhhs.gov/cfda/P93136.htm [accessed December 19, 2001].

Inskip HM, Harris EC, Barraclough B. 1998. Lifetime risk of suicide for affective disorder, alcoholism and schizophrenia. British Journal of Psychiatry , 172: 35-37.

IOM (Institute of Medicine). 1999. Bonnie RJ, Fulco CE, Liverman CT, Editors. Reducing the Burden of Injury: Advancing Prevention and Treatment . Washington, DC: National Academy Press.

IOM (Institute of Medicine). 2000. Pellmar TC, Eisenberg L, Editors. Bridging Disciplines in the Brain, Behavioral, and Clinical Sciences . Washington, DC: National Academy Press.

IOM (Institute of Medicine). 2001. Nass SJ, Henderson IC, Lashof JC, Editors. Mammography and Beyond: Developing Technologies for the Early Detection of Breast Cancer . Washington, DC: National Academy Press.

IPM (National Swedish Institute for Psychosocial Medicine)/Karolinska Institute. 2001. National Centre for Suicide Research and Prevention of Mental Ill-Health . [Online]. Available: http://www.ki.se/ipm/enheter/engSui.html [accessed May 10, 2002].

Ivarsson T, Gillberg C, Arvidsson T, Broberg AG. 2002. The Youth Self-Report (YSR) and the Depression Self-Rating Scale (DSRS) as measures of depression and suicidality among adolescents. European Child and Adolescent Psychiatry , 11(1): 31-37.

Jansen J. 1990. On the statistical analysis of ordinal data when extravariation is present. Applied Statistics , 39: 75-84.

Jenkins J, Sainsbury P. 1980. Single-car road deaths—disguised suicides? British Medical Journal , 281(6247): 1041.

Jobes DA, Berman AL, Josselson AR. 1987. Improving the validity and reliability of medical–legal certifications of suicide. Suicide and Life-Threatening Behavior , 17(4): 310-325.

Kann L, Kinchen SA, Williams BI, Ross JG, Lowry R, Hill CV, Grunbaum JA, Blumson PS, Collins JL, Kolbe LJ. 1998. Youth Risk Behavior Surveillance—United States, 1997. State and Local YRBSS Coordinators. Journal of School Health , 68(9): 355-369.

Kaufman J, Birmaher B, Perel J, Dahl RE, Stull S, Brent D, Trubnick L, al-Shabbout M, Ryan ND. 1998. Serotonergic functioning in depressed abused children: Clinical and familial correlates. Biological Psychiatry , 44(10): 973-981.

Keck PE Jr, McElroy SL, Strakowski SM, West SA, Sax KW, Hawkins JM, Bourne ML, Haggard P. 1998. 12-month outcome of patients with bipolar disorder following hospitalization for a manic or mixed episode. American Journal of Psychiatry , 155(5): 646-652.

Kelly TM, Mann JJ. 1996. Validity of DSM-III-R diagnosis by psychological autopsy: A comparison with clinician ante-mortem diagnosis. Acta Psychiatrica Scandinavica , 94(5): 337-343.

Kessler RC, Borges G, Walters EE. 1999. Prevalence of and risk factors for lifetime suicide attempts in the National Comorbidity Survey. Archives of General Psychiatry , 56(7): 617-626.

Knox G. 1964. The detection of space-time interactions. Applied Statistics , 3: 25-29.

Kraemer GW, Ebert MH, Schmidt DE, McKinney WT. 1989. A longitudinal study of the effect of different social rearing conditions on cerebrospinal fluid norepinephrine and biogenic amine metabolites in rhesus monkeys. Neuropsychopharmacology , 2(3): 175-189.

Lee MA, Ganzini L. 1992. Depression in the elderly: Effect on patient attitudes toward lifesustaining therapy. Journal of the American Geriatrics Society , 40: 983-988.

Liang KY, Zeger SL. 1986. Longitudinal data analysis using generalized linear models. Biometrika , 73: 13-22.

Linehan MM. 1986. Suicidal people. One population or two? Annals of the New York Academy of Sciences , 487: 16-33.

Linehan MM. 1997. Behavioral treatments of suicidal behaviors. Definitional obfuscation and treatment outcomes. Annals of the New York Academy of Sciences , 836: 302-328.

Longford NT. 1994. Logistic regression with random coefficients. Computational Statistics and Data Analysis , 17: 1-15.

Maudsley G, Williams EM. 1996. “Inaccuracy” in death certification—where are we now? Journal of Public Health Medicine , 18(1): 59-66.

McCarthy PD, Walsh D. 1975. Suicide in Dublin: I. The under-reporting of suicide and the consequences for national statistics. British Journal of Psychiatry , 126: 301-308.

Miller IW, Norman WH, Bishop SB, Dow MG. 1986. The Modified Scale for Suicidal Ideation: Reliability and validity. Journal of Consulting and Clinical Psychology , 54(5): 724-725.

Millon T. 1983. Millon Clinical Multiaxial Inventory . 2nd ed. Minneapolis, MN: National Computer Systems.

Mitchell RE. 1982. Social networks and psychiatric clients: The personal and environmental context. American Journal Community Psychology , 10(4): 387-401.

Mohandie K, Meloy JR. 2000. Clinical and forensic indicators of “suicide by cop”. Journal of Forensic Sciences , 45(2): 384-389.

Moscicki EK, O’Carroll P, Rae DS, Locke BZ, Roy A, Regier DA. 1988. Suicide attempts in the Epidemiologic Catchment Area study. The Yale Journal of Biology and Medicine , 61: 259-268.

Murphy E, Lindesay J, Grundy E. 1986. 60 years of suicide in England and Wales. A cohort study. Archives of General Psychiatry , 43(10): 969-976.

Myers KA, Farquhar DR. 1998. Improving the accuracy of death certification. Canadian Medical Association Journal , 158(10): 1317-1323.

Nagelberg J. 1985. Promoting Population Policy: The Activities of the Rockefeller Foundation, the Ford Foundation and the Population Council: 1959–1966 . Dissertation Abstract, Columbia University.

Nakashima AK, Horsley R, Frey RL, Sweeney PA, Weber JT, Fleming PL. 1998. Effect of HIV reporting by name on use of HIV testing in publicly funded counseling and testing programs. Journal of the American Medical Association , 280(16): 1421-1426.

National Bioethics Advisory Commission. 1998. Research Involving Persons With Mental Disorders That May Affect Decisionmaking Capacity . Rockville, MD. [Online]. Available: http://www.bioethics.gov/capacity/TOC.htm [accessed December 19, 2001].

National Bioethics Advisory Commission. 2001. Ethical and Policy Issues in Research Involving Human Participants . Rockville, MD. [Online]. Available: http://bioethics.georgetown.edu/nbac/human/overvol1.html [accessed December 19, 2001].

NCHS (National Center for Health Statistics). 1988. Vital Statistics of the United States, 1985. Volume II. Mortality, Part A . DHHS publication number (PHS) 88-1101. Washington, DC: US Government Printing Office.

NCI (National Cancer Institute). 2000a. The Cancer Center Branch of the National Cancer Institute. Policies and Guidelines Relating to the Cancer–Center Support Grant . [Online]. Available: http://www3.cancer.gov/cancercenters/download.html [accessed May 10, 2002].

NCI (National Cancer Institute). 2000b. Fact Book: National Cancer Institute . Bethesda, MD: U.S. Department of Health and Human Services, National Institutes of Health, Public Health Service. [Online]. Available: http://www.nci.nih.gov/admin/fmb/Factbook2000.pdf [accessed December 19, 2001].

NCI (National Cancer Institute). 2001a. Description of the Cancer Centers Program . [Online]. Available: http://www.nci.nih.gov/cancercenters/description.html [accessed December 19, 2001].

NCI (National Cancer Institute). 2001b. History of the NCI Cancer Centers . [Online]. Available: http://www.nci.nih.gov/cancercenters/ccsg_comp_pt1_1to2.html#1history [accessed December 18, 2001].

NCI (National Cancer Institute). 2001c. Major Policies on Budget . [Online]. Available: http://www.nci.nih.gov/cancercenters/ccsg_comp_pt1_13to15.html#13major [accessed December 19, 2001].

NCIPC (National Center for Injury Prevention and Control). 2002. Suicide Prevention Research Center. Announcement Number 98067 . [Online]. Available: http://www.cdc.gov/ncipc/dvp/suinevada.htm [accessed May 9, 2001].

Nelson FL, Farberow NL, MacKinnon DR. 1978. The certification of suicide in eleven western states: An inquiry into the validity of reported suicide rates. Suicide and Life-Threatening Behavior , 8(2): 75-88.

NFFIRS (National Fatal Firearm Injury Reporting System) Workgroup. 2001. Uniform Data Elements: National Fatal Firearm Injury Reporting System. Release 1.1 . [Online]. Available: http://www.hsph.harvard.edu/hicrc/nviss/about_parent_tech.htm [accessed May 14, 2002].

NHTSA (National Highway Traffic Safety Administration). 1996. Traffic Safety Facts, 1996. Alcohol . Washington, DC: NHTSA.

NHTSA (National Highway Traffic Safety Administration). 2001. Fatality Analysis Reporting System . [Online]. Available: http://www.nhtsa.dot.gov/people/ncsa/fars.html [accessed April 19, 2002].

NIA (National Institute on Aging). 1998. NIH Guide: Amendment to Alzheimer’s Disease Research Centers . [Online]. Available: http://grants.nih.gov/grants/guide/notice-files/not98-025.html [accessed January 8, 2002].

NIA (National Institute on Aging). 2000. NIH Guide: Alzheimer’s Disease Core Centers . [Online]. Available: http://grants.nih.gov/grants/guide/rfa-files/RFA-AG-00-002.html [accessed January 8, 2002].

NIDA (National Institute on Drug Abuse). 1999. Federal Institutes and The Robert Wood Johnson Foundation Create Tobacco Use Research Centers . [Online]. Available: http://www.nida.nih.gov/MedAdv/99/NR-1018.html [accessed May 10, 2002].

NIMH (National Institute of Mental Health). 1998. Silvio O. Conte Centers for the Neuroscience of Mental Disorders . [Online]. Available: http://grants1.nih.gov/grants/guide/pafiles/PAR-98-058.html [accessed May 10, 2002].

NVISS (National Violent Injury Statistics System) Workgroup. 2002. Violent Death Reporting System Training Manual . [Online]. Available: http://www.hsph.harvard.edu/hicrc/nviss/about_parent_tech.htm [accessed May 14, 2002].

O’Carroll PW. 1989. A consideration of the validity and reliability of suicide mortality data. Suicide and Life-Threatening Behavior , 19(1): 1-16.

O’Carroll PW, Berman AL, Maris RW, Moscicki EK, Tanney BL, Silverman MM. 1996. Beyond the tower of Babel: A nomenclature for suicidology. Suicide and Life-Threatening Behavior , 26(3): 237-252.

O’Donnell I, Farmer R. 1995. The limitations of official suicide statistics. British Journal of Psychiatry , 166(4): 458-461.

Ohberg A, Lönnqvist J. 1998. Suicides hidden among undetermined deaths. Acta Psychiatrica Scandinavica , 98(3): 214-218.

Ovenstone IM. 1973. A psychiatric approach to the diagnosis of suicide and its effect upon the Edinburgh statistics. British Journal of Psychiatry , 123(572): 15-21.

Pearson JL, King C, Stanley B, Fisher C. 2000. Issues to Consider in Intervention Research With Persons at High Risk for Suicidality : National Institute of Mental Health. [Online] Available: http://www.nimh.nih.gov/research/highrisksuicide.cfm [accessed December 19, 2001].

Pescosolido BA, Mendelsohn R. 1986. Social causation or social construction of suicide? An investigation into the social organization of official rates. American Sociological Review , 51(1): 80-100.

Phillips DP, Ruth TE. 1993. Adequacy of official suicide statistics for scientific research and public policy. Suicide and Life-Threatening Behavior , 23(4): 307-319.

PHS (Public Health Service). 2001. National Strategy for Suicide Prevention: Goals and Objectives for Action . Rockville, MD: U.S. Department of Health and Human Services.

Pine DS, Coplan JD, Wasserman GA, Miller LS, Fried JE, Davies M, Cooper TB, Greenhill L, Shaffer D, Parsons B. 1997. Neuroendocrine response to fenfluramine challenge in boys. Associations with aggressive behavior and adverse rearing. Archives of General Psychiatry , 54(9): 839-846.

Reynolds CF 3rd, Degenholtz H, Parker LS, Schulberg HC, Mulsant BH, Post E, Rollman B. 2001. Treatment as usual (TAU) control practices in the PROSPECT Study: Managing the interaction and tension between research design and ethics. International Journal of Geriatric Psychiatry , 16(6): 602-608.

Robbins H. 1993. Comparing two treatments under biased allocation. La Gazette Des Sciences Mathematique De Quebec , 15: 35-41.

Robbins H, Zhang CH. 1988. Estimating a treatment effect under biased sampling. Proceedings of the National Academy of Sciences , 85(11): 3670-3672.

Robbins H, Zhang CH. 1989. Estimating the superiority of a drug to a placebo when all and only those patients at risk are treated with the drug. Proceedings of the National Academy of Sciences , 86(9): 3003-3005.

Robbins H, Zhang CH. 1991. Estimating a multiplicative treatment effect under biased allocation. Biometrika , 78: 349-354.

Rudd MD, Rajab MH, Orman DT, Joiner T, Stulman DA, Dixon W. 1996. Effectiveness of an outpatient problem-solving intervention targeting suicidal young adults: Preliminary results. Journal of Consulting and Clinical Psychology , 64(1): 179-190.

Sainsbury P, Jenkins JS. 1982. The accuracy of officially reported suicide statistics for purposes of epidemiological research. Journal of Epidemiology and Community Health , 36(1): 43-48.

Schmidt CW Jr, Shaffer JW, Zlotowitz HI, Fisher RS. 1977. Suicide by vehicular crash. American Journal of Psychiatry , 134(2): 175-178.

Shaffer D, Gould MS, Fisher P, Trautman P, Moreau D, Kleinman M, Flory M. 1996. Psychiatric diagnosis in child and adolescent suicide. Archives of General Psychiatry , 53: 339-348.

Smith PG. 1982. Spatial and temporal clustering. In: Schottenfeld D, Fraumeni JF, Editors. Cancer Epidemiology and Prevention . (pp. 391-407). Philadelphia: WB Saunders.

STIPDA (State and Territorial Injury Prevention Directors Association). 1999. STIPDA Resolutions: CSTE Position Statement #INJ-6 . [Online]. Available: http://www.stipda.org/resol/99nphss-intent.htm [accessed April 17, 2002].

Telcser SL. 1996. The psychological autopsy study of completed suicide: An experimental test of the impact of knowledge of the suicide upon informants’ reports of psychopathology in the victim. Thesis (Ph.D.)—Loyola University of Chicago. Dissertation Abstracts International . Vol. 57-03. (pp. 2167).

Ten Have TR. 1996. A mixed effects model for multivariate ordinal response data including correlated failure times with ordinal responses. Biometrics , 52: 473-491.

Thomas N, Longford NT, Rolph J. 1992. A Statistical Framework for Severity Adjustment of Hospital Mortality Rates . Working paper. Santa Monica, CA: RAND.

University of Hamburg. 2000. Center for Therapy and Studies of Suicidal Behavior . [Online]. Available: http://www.uke.uni-hamburg.de/Clinics/Psych/TZS/TZS_e.html [accessed May 10, 2002].

University of Michigan. 2002. Introduction to PSC (Population Studies Center) . [Online]. Available: http://www.psc.isr.umich.edu/intro.html [accessed May 10, 2002].

University of North Carolina. 2001. UNC Injury Prevention Research Center: Funding Description . [Online]. Available: http://www.sph.unc.edu/iprc/aboutiprc/description.html [accessed December 19, 2001].

University of Oxford. 2002. Centre for Suicide Research . [Online]. Available: http://cebmh.warne.ox.ac.uk/csr/ [accessed May 10, 2002].

University of Rochester. 2001. Suicide study awarded $3.2 million. Currents , 29(21): [Online]. Available: http://www.rochester.edu/pr/Currents/V29/V29N21/story04.html [accessed May 10, 2002].

University of the Philippines. 2002. Population Institute . [Online]. Available: http://www.geocities.com/popinst/ [accessed May 10, 2002].

Velting DM, Shaffer D, Gould MS, Garfinkel R, Fisher P, Davies M. 1998. Parent-victim agreement in adolescent suicide research. Journal of American Academy of Child and Adolescent Psychiatry , 37(11): 1161-1166.

Wagenaar AC. 1993. Research affects public policy: The case of the legal drinking age in the United States . Addiction , 88(Supplement): S75-S81.

Wallenstein S, Gould M, Kleinman M. 1989. Use of the scan statistic to detect time-space clustering. American Journal of Epidemiology , 130: 1057-1064.

Washington Department of Health. 1997. New statewide system tracks firearm-related injuries to gather information for prevention strategies. EpiTrends , 2(3): 1-2.

Washington University. 2001. Washington University Researchers Launch Silvio Conte Center . [Online]. Available: http://medicine.wustl.edu/~wumpa/news/2001/conte.html [accessed May 10, 2002].

Wasserman D, Varnik A. 1998. Reliability of statistics on violent death and suicide in the former USSR, 1970-1990. Acta Psychiatrica Scandinavica Supplement , 394: 34-41.

Wolfgang ME. 1958. An analysis of homicide–suicide. Journal of Clinical and Experimental Psychopathology , 19: 208-218.

Zeger SL, Liang KY. 1986. Longitudinal data analysis for discrete and continuous outcomes. Biometrics , 42(1): 121-130.

To be, or not to be: that is the Question:

Whether ’tis Nobler in the Mind to suffer

The Slings and Arrows of outrageous Fortune,

Or to take Arms against a Sea of Troubles,

And by opposing end them; to die to sleep

No more, and by a Sleep to say we end

The Hart-ache, and the thousand Natural Shocks

That Flesh is heir to; ’tis a Consummation

Devoutly to be wish’d to die to sleep,

To Sleep, perchance to dream; ay there’s the

For in that Sleep of Death what Dreams may

Come…

—W ILLIAM S HAKESPEARE

Hamlet , Act III, scene I.

Every year, about 30,000 people die by suicide in the U.S., and some 650,000 receive emergency treatment after a suicide attempt. Often, those most at risk are the least able to access professional help.

Reducing Suicide provides a blueprint for addressing this tragic and costly problem: how we can build an appropriate infrastructure, conduct needed research, and improve our ability to recognize suicide risk and effectively intervene. Rich in data, the book also strikes an intensely personal chord, featuring compelling quotes about people's experience with suicide. The book explores the factors that raise a person's risk of suicide: psychological and biological factors including substance abuse, the link between childhood trauma and later suicide, and the impact of family life, economic status, religion, and other social and cultural conditions. The authors review the effectiveness of existing interventions, including mental health practitioners' ability to assess suicide risk among patients. They present lessons learned from the Air Force suicide prevention program and other prevention initiatives. And they identify barriers to effective research and treatment.

This new volume will be of special interest to policy makers, administrators, researchers, practitioners, and journalists working in the field of mental health.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

Switch between the Original Pages , where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

Rethinking Barriers and Enablers in Qualitative Health Research: Limitations, Alternatives, and Enhancements

Affiliations.

• 1 Institute for Musculoskeletal Health, Sydney Musculoskeletal Health, The University of Sydney and Sydney Local Health District, Sydney, NSW, Australia.
• 2 School of Public Health, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia.
• 3 Brain and Mind Centre, The University of Sydney, Sydney, NSW, Australia.
• PMID: 38459909
• DOI: 10.1177/10497323241230890

Explorations of barriers and enablers (or barriers and facilitators) to a desired health practice, implementation process, or intervention outcome have become so prevalent that they seem to be a default in much health services and public health research. In this article, we argue that decisions to frame research questions or analyses using barriers and enablers (B&Es) should not be default. Contrary to the strengths of qualitative research, the B&Es approach often bypasses critical reflexivity and can lead to shallow research findings with poor understanding of the phenomena of interest. The B&Es approach is untheorised, relying on assumptions of linear, unidirectional processes, universally desirable outcomes, and binary thinking which are at odds with the rich understanding of context and complexity needed to respond to the challenges faced by health services and public health. We encourage researchers to develop research questions using informed deliberation that considers a range of approaches and their implications for producing meaningful knowledge. Alternatives and enhancements to the B&Es approach are explored, including using 'whole package' methodologies; theories, conceptual frameworks, and sensitising ideas; and participatory methods. We also consider ways of advancing existing research on B&Es rather than doing 'more of the same': researchers can usefully investigate how a barrier or enabler works in depth; develop and test implementation strategies for addressing B&Es; or synthesise the B&Es literature to develop a new model or theory. Illustrative examples from the literature are provided. We invite further discussion on this topic.

Keywords: complexity; context; facilitators; research questions; theory.

• Help & FAQ

Intervention Strategies to Address Barriers and Facilitators to a Healthy Lifestyle Using the Behaviour Change Wheel: A Qualitative Analysis of the Perspectives of Postpartum Women

• Eastern Health Clinical School
• Monash Centre for Health Research & Implementation
• School of Public Health and Preventive Medicine
• Evidence Synthesis Qual & Implt Methods
• Epidemiology and Preventive Medicine Alfred Hospital

Research output : Contribution to journal › Article › Research › peer-review

Postpartum women experience unique barriers to maintaining healthy lifestyles after birth. Theory-based behaviour change techniques and intervention strategies can be integrated into postpartum lifestyle interventions to enable women to overcome barriers to change. This study aims to explore barriers and facilitators to engaging in healthy postpartum lifestyle behaviours and develop intervention strategies for integration in a postpartum lifestyle intervention using the Behaviour Change Wheel (BCW). Semi-structured interviews were conducted with women up to two years postpartum (n = 21). Interviews were thematically analysed, themes were mapped to the Capability, Opportunity, and Motivation Model of Behaviour Change and intervention strategies were developed using the BCW. Findings suggest that women face barriers and facilitators within capability (sleep deprivation, mental exhaustion, ability to plan), opportunity (support of friends, partners and extended families) and motivation (challenges with prioritising self, exercise to cope with stress). Intervention strategies included supporting behaviour regulation and sleep to enhance capability, engaging partners, strengthening peer support to create opportunities and highlighting the mental health benefits of healthy lifestyles to inspire motivation. Integrating targeted evidence-based behaviour change strategies into postpartum lifestyle interventions may support women in overcoming commonly reported barriers to a healthy lifestyle.

• behaviour change
• behaviour therapy
• dietary intervention
• healthy lifestyle
• mental health
• sleep deprivation

Access to Document

• 10.3390/nu16071046 Licence: CC BY

Other files and links

• Link to publication in Scopus

Projects per year

Weight gain prevention in women with polycystic ovary syndrome

Lim, S. & Moran, L.

National Health and Medical Research Council (NHMRC) (Australia)

1/02/18 → 31/01/23

Project : Research

Assessment and management of obesity in women with polycystic ovary syndrome and during pregnancy

National Heart Foundation of Australia

1/01/17 → 31/12/22

T1 - Intervention Strategies to Address Barriers and Facilitators to a Healthy Lifestyle Using the Behaviour Change Wheel

T2 - A Qualitative Analysis of the Perspectives of Postpartum Women

AU - Lim, Siew

AU - Lang, Sarah

AU - Savaglio, Melissa

AU - Skouteris, Helen

AU - Moran, Lisa J.

PY - 2024/4

Y1 - 2024/4

N2 - Postpartum women experience unique barriers to maintaining healthy lifestyles after birth. Theory-based behaviour change techniques and intervention strategies can be integrated into postpartum lifestyle interventions to enable women to overcome barriers to change. This study aims to explore barriers and facilitators to engaging in healthy postpartum lifestyle behaviours and develop intervention strategies for integration in a postpartum lifestyle intervention using the Behaviour Change Wheel (BCW). Semi-structured interviews were conducted with women up to two years postpartum (n = 21). Interviews were thematically analysed, themes were mapped to the Capability, Opportunity, and Motivation Model of Behaviour Change and intervention strategies were developed using the BCW. Findings suggest that women face barriers and facilitators within capability (sleep deprivation, mental exhaustion, ability to plan), opportunity (support of friends, partners and extended families) and motivation (challenges with prioritising self, exercise to cope with stress). Intervention strategies included supporting behaviour regulation and sleep to enhance capability, engaging partners, strengthening peer support to create opportunities and highlighting the mental health benefits of healthy lifestyles to inspire motivation. Integrating targeted evidence-based behaviour change strategies into postpartum lifestyle interventions may support women in overcoming commonly reported barriers to a healthy lifestyle.

AB - Postpartum women experience unique barriers to maintaining healthy lifestyles after birth. Theory-based behaviour change techniques and intervention strategies can be integrated into postpartum lifestyle interventions to enable women to overcome barriers to change. This study aims to explore barriers and facilitators to engaging in healthy postpartum lifestyle behaviours and develop intervention strategies for integration in a postpartum lifestyle intervention using the Behaviour Change Wheel (BCW). Semi-structured interviews were conducted with women up to two years postpartum (n = 21). Interviews were thematically analysed, themes were mapped to the Capability, Opportunity, and Motivation Model of Behaviour Change and intervention strategies were developed using the BCW. Findings suggest that women face barriers and facilitators within capability (sleep deprivation, mental exhaustion, ability to plan), opportunity (support of friends, partners and extended families) and motivation (challenges with prioritising self, exercise to cope with stress). Intervention strategies included supporting behaviour regulation and sleep to enhance capability, engaging partners, strengthening peer support to create opportunities and highlighting the mental health benefits of healthy lifestyles to inspire motivation. Integrating targeted evidence-based behaviour change strategies into postpartum lifestyle interventions may support women in overcoming commonly reported barriers to a healthy lifestyle.

KW - behaviour change

KW - behaviour therapy

KW - dietary intervention

KW - healthy lifestyle

KW - mental health

KW - postpartum

KW - sleep deprivation

UR - http://www.scopus.com/inward/record.url?scp=85190486311&partnerID=8YFLogxK

U2 - 10.3390/nu16071046

DO - 10.3390/nu16071046

M3 - Article

C2 - 38613079

AN - SCOPUS:85190486311

SN - 2072-6643

JO - Nutrients

JF - Nutrients

Mobile Menu Overlay

The White House 1600 Pennsylvania Ave NW Washington, DC 20500

NSTC: Best Practices for Reducing Organizational, Cultural, and Institutional Barriers in STEM   Research

NSTC: Best Practices for Reducing Organizational, Cultural, and Institutional Barriers in STEM Research can be found here .

Stay Connected

We'll be in touch with the latest information on how President Biden and his administration are working for the American people, as well as ways you can get involved and help our country build back better.

Opt in to send and receive text messages from President Biden.

A .gov website belongs to an official government organization in the United States.

A lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Signs and Symptoms
• How It Spreads
• Testing and Diagnosis
• Surveillance and Data
• Educational Materials
• Clinical Testing and Diagnosis
• Clinical Care and Treatment
• Continuing Education
• Clinical Resources
• Vector-Borne Diseases

Lyme Disease Surveillance and Data

Over 63,000 cases of Lyme disease were reported to CDC by state health departments and the District of Columbia in 2022. This number reflects cases reported through routine national surveillance, which is only one way public health officials track diseases. Recent estimates using other methods suggest that approximately 476,000 people may be diagnosed and treated for Lyme disease each year in the United States. This number likely includes patients who are treated based on clinical suspicion but do not actually have Lyme disease.

Surveillance data explained

Lyme disease has been a nationally notifiable condition in the United States since 1991. Reports of Lyme disease are routinely collected and verified by state and local health departments in accordance with their legal mandate and surveillance practices. After removal of personal identifiers, selected information on cases is shared with CDC through the National Notifiable Diseases Surveillance System (NNDSS) . Policies regarding case definitions, reporting, confidentiality, and data release are determined by states and territories under the auspices of the Council of State and Territorial Epidemiologists (CSTE). Surveillance data have a number of limitations that need to be considered in the analysis, interpretation, and reporting of results.

Limitations of surveillance data

• Under-reporting and misclassification are features common to all surveillance systems. Not every case of Lyme disease is reported to CDC, and some cases that are reported may be due to another cause.
• Surveillance data are captured by county of residence, not county of exposure.
• States may close their annual surveillance dataset at a different time than CDC. Thus, the final case counts published by CDC may not exactly match numbers published by each state agency for a given year.
• Following its implementation in 1991, the national surveillance case definition for Lyme disease was modified in 1996, 2008, 2011, 2017, and again in 2022. Some of these changes impacted surveillance data and must be considered when attempting to interpret trends. Case definitions for each period are available .

Available data

• Interactive graphs and tables of reported Lyme disease data
• Interactive map of reported Lyme disease data
• CDC WONDER tables

Alternative data sources

CDC is currently working to establish enhanced surveillance and research platforms for Lyme disease using electronic health records (EHRs) from large healthcare systems in areas of the U.S. with a high incidence of Lyme disease. The Surveillance Based Lyme Disease Network is comprised of partners in healthcare systems from Maine, Massachusetts, Pennsylvania, and Wisconsin. CDC will use these EHR data in concert with traditional public health surveillance data to better understand how Lyme disease affects the American public.

Lyme Disease

Lyme disease is caused by Borrelia bacteria spread to people by the bite of an infected blacklegged tick.

For Everyone

Health care providers.

Justice Department takes 'major step' toward rescheduling marijuana

WASHINGTON — The Justice Department took a significant step toward rescheduling marijuana Thursday, formalizing its process to reclassify the drug as lower-risk and remove it from a category in which it has been treated as more dangerous than fentanyl and meth.

President Joe Biden announced the “major” move in a direct-to-camera video posted to his official account on X. “This is monumental,” Biden said in the message. “It’s an important move towards reversing long-standing inequities. … Far too many lives have been upended because of a failed approach to marijuana, and I’m committed to righting those wrongs. You have my word on it.”

The Biden administration has been signaling that it would move to reschedule the drug from Schedule I — a strict classification including drugs like heroin — to the less-stringent Schedule III, which would for the first time acknowledge the drug’s medical benefits at the federal level. The Drug Enforcement Administration submitted a notice of proposed rulemaking in the Federal Register on Thursday afternoon, triggering a 60-day comment period that will allow members of the public to submit remarks regarding the rescheduling proposal before it is finalized.

Biden first directed federal agencies to review how marijuana is scheduled in October 2022, weeks before that year’s midterm elections. The process was led by the DOJ and the Department of Health and Human Services.

“Look folks, no one should be in jail for merely using or possessing marijuana. Period,” Biden said in Thursday’s video, his third time speaking extensively on the topic since his directive two years ago.

The second time Biden addressed the issue was during this year’s State of the Union address, making history by referring to marijuana from the dais in the House chamber. “No one should be jailed for using or possessing marijuana,” he said at the time.

Vice President Kamala Harris also released a video Thursday, hailing the progress.

“Currently marijuana is classified on the same level as heroin and more dangerous than fentanyl. We are finally changing that,” Harris said. “We are on the road to getting it done.”

During the first 30 days of the comment period, interested parties could request a hearing regarding the rescheduling proposal. Under the statute, the DEA would be required to hold a hearing before an administrative law judge.

After the DEA reviews and considers the public comments, and at the conclusion of any requested hearing, the DEA will issue a final order to reschedule marijuana. (The DEA could decline to reschedule the drug but that’s unlikely given the administration’s strong support).

The entire process can take anywhere from a few months to up to a year.

Once completed, federal scientists will be able to research and study the potential medical benefits of the drug for the first time since the Controlled Substances Act was enacted in 1971. It could also open the door for pharmaceutical companies to get involved with the sale and distribution of medical marijuana in states where it is legal.

For the $34 billion cannabis industry, the move would also eliminate significant tax burdens for businesses in states where the drug is legal, notably removing it from the IRS code’s Section 280E, which prohibits legal cannabis companies from deducting what would otherwise be ordinary business expenses.

The Justice Department’s rescheduling decision could also help shrink the black market, which has thrived despite legalization in states like New York and California, and has undercut legal markets, which are fiercely regulated and highly taxed.

Dr. Kevin Sabet, president of the anti-marijuana legalization group Smart Approaches to Marijuana, blasted the decision. “It’s become undeniable that politics, not science, is driving this decision and has been since the very beginning. This decision won’t legalize marijuana, and it won’t release anyone from prison or jail,” Sabet said. “This is setting the stage to create the Big Tobacco of our time.”

During his time in office, Biden issued pardons for prior federal offenses of simple possession of marijuana and issued a proclamation granting additional pardons for simple possession, attempted simple possession and use of the drug.

The White House has also urged governors to do the same in their states and some have heeded the call, including in Oregon and Massachusetts.

Democrats in Congress are pursuing a partisan effort to remove cannabis entirely from the Controlled Substances Act, empowering states to create their own cannabis laws and prioritize restorative and economic justice for those affected by the “war on drugs.”

“Congress must do everything we can to end the federal prohibition on cannabis and address long-standing harms caused by the War on Drugs,” Senate Majority Leader Chuck Schumer, D-N.Y., said earlier this month.

Julie Tsirkin is a correspondent covering Capitol Hill.

Monica Alba is a White House correspondent for NBC News.

An official website of the United States government

Here's how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock Locked padlock ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Breaking Barriers to Affordable and Abundant Housing: A German-U.S. Comparison of Publicly Led Development Projects

Report Acceptance Date: May 2024 (2 pages)

Posted Date: May 16, 2024

This publication identifies ways cities can support the development of mixed-used, transit-adjacent housing. The author compares urban development policy approaches in the United States and Germany and highlights publicly-led projects in three cities in the United States—Atlanta, Georgia; St. Louis, Missouri; and Seattle, Washington—and three cities in Germany—Berlin, Frankfurt, and Munich. After identifying barriers to publicly-led development, such as resident opposition to new construction, the author identifies promising ways the highlighted cities advanced significant new housing construction.

The report Foreword from Todd Richardson, the General Deputy Assistant Secretary for Policy Development and Research, is available here from HUD User.

The full report is available at the German Marshall Fund website: https://www.gmfus.org/news/breaking-barriers-affordable-and-abundant-housing

Download the Foreword by Todd Richardson here

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• SAGE Open Nurs
• v.8; Jan-Dec 2022

Barriers to Research Utilization in Nursing: A Systematic Review (2002–2021)

Fritz gerald v. jabonete.

1 National University, Manila, Philippines

Rachel Edita O. Roxas

Introduction.

There is an existing gap between what people learned from theory and what they clinically practiced, as revealed in research studies in nursing. This gap is primarily due to identified barriers in utilizing the research findings in actual nursing practice.

To present a scientific mapping of the Scopus-indexed literature published from 2002 to 2021, which studied barriers to research utilization in nursing using the BARRIER scale.

This systematic review utilized bibliometric analysis. One hundred seventy-nine extracted literature from Scopus was manually reviewed, and the study included 53 documents for further analysis.

Remarkably, almost three-fourths of the documents identified setting-related factors as the most common barrier to research utilization in nursing (n = 39, 73.58%). This is followed by presentation-related factors (n = 16.98%) and nurse-related factors (n = 5, 9.43%), respectively. Findings revealed that insufficient time at work in implementing new ideas was perceived as the top barrier in research utilization in nursing.

It is crucial to determine the hindrances to the utilization of research findings. The results of this study establish the connection between research and evidence-based practice which stimulates in meeting the gap in the current nursing practice. Future studies must include research utilization studies that apply tools other than the BARRIER scale.

There is a discrepancy between the knowledge gained from theoretical research and actual clinical practice ( Benton et al., 2020 ; Mackey & Bassendowski, 2017 ). A significant number of research studies have focused on developing and applying practical research ideas in practice from the past five years ( Estabrooks, 1999a ). The clinical application coexists with evidence-based practice ( Mackey & Bassendowski, 2017 ). It is asserted that health interventions provided by professionals went through a selective process of choosing the best available scientific research evidence in making decisions about the care of the patient. As a result, developing, evaluating, and implementing research poses a challenge. The relevance of clinical nursing practice that is evidence-based is gaining recognition; however, clinical application poses a challenge and would frequently fail ( Estabrooks et al., 2008 ).

With limited tools to measure barriers in research utilization, the Barriers to Research Utilization (BARRIER) scale was developed by Funk et al. (1991) by identifying the common barriers cited in the literature. Since then, the tool is now being widely used by nursing practitioners, clinicians, administrators, and academicians in determining the barriers to the utilization of research findings in practice in the United States ( Atkinson et al., 2008 ; Baernholdt & Lang, 2007 ; Brown et al., 2009, 2010 ; Cline et al., 2017 ; Fink et al., 2005 ; Karkos & Peters, 2006 ; Niederhauser & Kohr, 2005 ; Phillips, 2015 ; Schoonover, 2009 ; Stichler et al., 2011 ), the United Kingdom ( Bryar et al., 2003 ; Carrion et al., 2004 ; Kirshbaum et al., 2004 ), Finland ( Kuuppelomäki & Tuomi, 2003 ; Oranta et al., 2002 ), Sweden ( Andersson et al., 2007 ; Boström et al., 2008 ), Australia ( Hutchinson & Johnston, 2004 ), Turkey ( Kocaman et al., 2010 ; Tan et al., 2012 ; Uysal et al., 2010 ; Yava et al., 2009 ), China ( Chien et al., 2013 ; Wang et al., 2013 ; Zhou et al., 2015 ), Saudi Arabia ( Aboshaiqah et al., 2014 ; Aljezawi et al., 2019 ; Omer, 2012 ), Ireland ( Brenner, 2005 ; Glacken & Chaney, 2004 ), Spain ( Cidoncha-Moreno & Ruíz de Alegría-Fernandez de Retana, 2017 ; Sarabia-Cobo et al., 2015 ), Canada ( McCleary & Brown, 2003 ), Greece ( Patiraki et al., 2004 ), and Taiwan ( Chen et al., 2013 ).

From 2006 to 2010, three distinct quantitative reviews of research that used the BARRIER scale were conducted ( Carlson & Plonczynski, 2008 ; Hutchinson & Johnston, 2006 ; Kajermo et al., 2010 ). These reviews overlapped its literature and findings conducted in a similar time frame from 1991 to 2009. The said reviews covered the literature for more than 10 years ago. Thus, an updated review of BARRIER scale studies is essential to ensure the relevance and applicability of research findings in the evolving clinical practice. Moreover, Estabrooks et al. (2004) mapped out the literature on barriers to research utilization in the scientific community and the recent network of researchers for articles published only from 1972 to 2001.

Literature Review

Research utilization.

Research utilization is described as applying scientific research findings to clinical practice. Scientific evidence and conclusions in this field are relevant to practitioners to make optimal decisions and improve patient conditions and outcomes ( Da'seh & Rababa, 2021 ). However, there is a limited study on the barriers of research utilization in nursing, including the organization and expansion of this field of study.

Research utilization studies marked history in the 1980 and gained popularity in the 1990s. Estabrooks (1999b) identified no empirical methods in the health literature that attempted to map the field. As a result, she mapped factors hindering research utilization in nursing using bibliometric methods. Her study determined the structure of this scientific community and the recent network of researchers, which the study was published from 1972 to 2001. Those studies were the last attempt at bibliometric research.

Bibliometrics

Bibliometric analyses contribute to the growth and exchange of knowledge within a specified field of academic research ( Estabrooks et al., 2004 ). The bibliometric approach uses empirical data and quantitative analysis to utilize the published literature and publication patterns within a field. Pritchard (1969) first coined the term bibliometrics that “employs mathematical and statistical methods in bibliometric to determine and analyze the growth and trend of a particular research theme” (p. 349).

BARRIER Scale

The BARRIER scale was developed by Funk et al. (1991) and is a widely accepted and utilized tool in determining perceived barriers as research findings in the United States, the United Kingdom, Finland, Sweden, Norway, and Australia. This tool established validity and reliability ( Bryar et al., 2003 ; Carrion et al., 2004 ; Closs et al., 2000 ; Funk et al., 1995 ; Gerrish & Clayton, 2004 ; Nilsson Kajermo et al., 1998 ; Oranta et al., 2002 ; Parahoo, 2000 ; Retsas, 2000 ). The tool includes factors that interfere with research utilization, as follows: (1) the characteristics of the adopter, which consider the nurse's research values, skills, and awareness; (2) the characteristics of the organization, which include the settings, barriers, and limitations; (3) the characteristics of the innovation that apply to the research qualities produced; and (4) the c haracteristics of the communication , which relates on how the accessibility and manner of research presentation.

The BARRIER scale is a 29-item and 4-point Likert-type questionnaire. The respondents are asked to appraise how the identified barriers affect research utilization. The tool ranges from 1 (“to no extent”) to 4 (“to a great extent”). The tool is reliable with Cronbach's alpha index scores: nurse (α = .80), setting (α = .80), presentation (α = .72), and research (α = .65) by ( Funk et al., 1991 ).

Purpose of the Study

The purpose of the study is to identify the perception of nurses on these barriers from 2002 to 2021 using the Barriers to Research Utilization (BARRIER) scale. Specifically, it will answer the following research questions: (1) What is the current state of research utilization in nursing in terms of research productivity and impact and citation per document? (2) What intellectual structure can be derived in the study in nursing, including its networks of co-citation, keywords co-occurrence, and co-authorship? and (3) What are the top barriers to research utilization in nursing?

Theoretical Framework

The Theory of Diffusion (RTD) of Innovations by Rogers (1995) was utilized in this study. RTD is a behavioral theory that describes how the user decides whether to adopt or reject new ideas, response patterns, or technology. Rogers (1995) defined the notion of diffusion as a process that includes the exchange of new ideas among individuals. The primary components of the RTD are channels, innovation, time, social systems, and communication. The innovation-decision integrates these components, explaining how the adopter learns to put the evaluation and innovation into practice. Furthermore, Rogers (1995) specified that the innovation-decision process has five stages: knowledge, persuasion, decision, implementation, and confirmation. Therefore, this study focused on knowledge and persuasion stages, emphasizing “innovation” to apply research knowledge.

Conceptual Framework

Scientific mapping is the process of identifying the intellectual structure and classifying the scientific output of a given field of study over a period (Hallinger & Kovačević, 2019). Moreover, Zupic and Čater (2015) defined intellectual structure as the product of examining scientific domains, such as the research tradition, the disciplinary composition, persuasive research topics, and the pattern of their interrelationships. The intellectual structure described scientific activity's size, timing, space, and composition. Specifically, citations are seen as a metric of research productivity and quality. This implies that the more articles are cited, the higher their influence or quality contributes to the body of knowledge ( Geisler, 2000 ).

Study Design

This study employed a quantitative bibliometric analysis of Scopus-indexed nursing research utilization articles published from 2002 to 2021. The approach utilizes published literature and identifies publication patterns in nursing using empirical data and quantitative analysis. Moreover, it determines and analyzes the growth and trend of research utilization in the field.

Sampling Criteria

The inclusion criteria required Scopus-indexed full-text articles and reviews related to research utilization from 2002 to the present. This research utilized the BARRIER scale in determining the barriers to nursing research utilization. This excluded professions other than nursing; reviews which are not written in English or with no English translation; and included only the specific literature on research barriers; articles on research implementation, journal clubs, and evidence-based practice; notes, letters and correspondence, dissertations; and discussions about policy, management, governmental, or organizational concerns.

Data Collection

The Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) procedure was adopted ( Moher et al., 2009 ). PRISMA follows four stages: identification, screening, eligibility, and inclusion (see Figure 1 ). The study focused on Scopus-indexed publications and publications that are written in English.

Preferred reporting items for systematic reviews and meta-analysis (PRISMA).

For the accuracy of the search process, during the identification stage, documents were retrieved using the Scopus database and the Boolean query in Scopus’ advanced search function: ALL (“Research Use” and “Research utilization”) AND (“barriers”) AND (“nursing”) AND (“bibliometric”) AND (“systematic review”). The oldest review was published in 1972, and the most recent was in 2021. Only studies and review papers published from 2002 to the present were considered. After applying the exclusion criteria, only 53 articles were subjected to further analysis.

Data Analysis

A “Scopus-Analyze Search Result and Citation Overview” was used to calculate various bibliometric indicators, such as research productivity, research impact, and citations per document. The h-index was identified to measure the quantitative index involving productivity, citation impact, and influence of authors, institutions, and countries or regions in research utilization.

The VOS viewer software developed by van Eck and Waltman (2010) was used to obtain graphical visualizations of the knowledge flow network of nursing research utilization. It generated a map of the co-citation, keywords co-occurrence, co-authorship network published in research utilization in nursing. Larger nodes represent the more frequent citations. Moreover, the distance between the nodes indicates more frequent co-citation and keywords co-occurrence and authorship. Furthermore, node proximity reflects variations in the content of the scholarly works of the author. The more that studies are mentioned together signifies that work is conceptually comparable.

Visualizations on the intellectual structure

Visualizations included those presented by Linnenluecke et al. (2020) , Moral-Muñoz et al. (2020) , Hallinger and Kovačević (2019) , and Cobo et al. (2011) . The metrics used are research productivity, research impact, and citations per document.

Research productivity is represented by the number of documents per year by a source identified in research utilization in nursing and its barriers. This includes the number of publications produced each year, affiliation, source, author's country or locality, and funding sponsor. The researchers consider the primary author's country or locality and authorship.

Research impact is based on the SCImago journal rank and source normalized impact per paper. A citation per document refers to the top documents’ total and yearly citations. This includes source citations and documents not cited by year.

The Current State of Research Utilization in Nursing in Terms of Research Productivity and Impact, and Citations per Document

Using the “Scopus Analyze Search Results” function, 53 documents were retrieved from 2002 to 2021. It shows the frequency of publications about research utilization in nursing. The majority of the documents were published in 2004, 2005, 2008, and 2013 with 5 documents each. It was observed that there is a minimal drop in the number of publications from 2015 to 2018. No study on nursing research utilization using the BARRIER scale was conducted after 2019.

The top authors in nursing research utilization studies are affiliated with Karolinska Institute and Karolinska University Hospital, Sweden. The remaining studies come from the United States, the United Kingdom, and Australia. The Journal of Clinical Nursing (JCN) had the highest number of publications in research utilization studies. However, this publication trend differed in the bibliometric study conducted from 1972 to 2001 by Estabrooks et al. (2004). The Journal of Advanced Nursing (JAN) was recorded as the highest number of publications in the same field.

The researchers bibliometrically identified the research performance of the top publication sources, using the metrics as defined by Scopus. Research impact is based on the SCImago journal rank by year and by year's source normalized impact per paper. It is observed that the International Journal of Nursing Studies (IJNS) tops among the other journals in terms of research impact. This means that this journal is frequently cited in high-impact publications. This is followed by the Journal of Advanced Nursing , which is almost tied with the Journal of Clinical Nursing in its SCImago journal rank and the normalized impact per paper each year.

The Journal of Advanced Nursing ranked first in source citation, followed by the Journal of Clinical Nursing . There was an increasing trend in the number of source citations in the past two decades until a noticeable sudden drop of citations in 2021 in all journals. The Journal of Nurses in Staff Development ranked first among those not cited documents.

Kajermo, who published 3 documents as a principal author, was marked first to contribute to nursing research utilization. Among other authors in the field are Cline et al. (2019) and Andersson et al. (2007) , whom each have 2 published documents. The majority of the authors come from the United States, followed by Spain, Australia, and Turkey. The rest of the authors come from Canada, Saudi Arabia, and the United Kingdom.

The identified sponsors had funded at least one study in terms of funding sponsors. These sponsors are the European Oncology Nursing Society, Ministry of Science, ICT and Future Planning, National Research Foundation of Korea, and Sigma Theta Tau International. The rest of the articles did not disclose funding sponsors.

Intellectual Structure and Networks of Co-citation, Keywords Co-occurrence, and Co-authorship

Using the “View Citation Overview” function of Scopus, a remarkable increase in the citation was noted from 2002 to 2008. Then, the fluctuating trend started in 2009 onwards. There was a considerable dramatic drop in citation, which was observed in 2018. The h-index of the 53 documents is 23, which means there are 23 documents of the 53 documents cited at least 23 times.

The selected 53 documents had total citations of 1,894 over the 15 years. Those documents published in 2010 and 2016 received the highest number of citations. Moreover, the document “Nursing practice, knowledge, attitudes and perceived barriers to evidence-based practice at an academic medical center,” in 2009 with 273 citations as the highest. It was closely followed by the document “Overcoming barriers and promoting the use of research in practice” in 2005 with 233 citations.

Top Barriers to Research Utilization in Nursing

Table 1 presents the development of barriers to research utilization in nursing from 2002 to 2021. Remarkably, almost three-fourths of the documents identified setting-related factors as the most common barrier to research utilization in nursing (n = 39, 73.58%). This is followed by presentation-related factors (n = 16.98%) and nurse-related factors (n = 5, 9.43%), respectively. Among the documents, research-related factors are not identified as barriers to research utilization in nursing. No research utilization studies utilized the BARRIER scale in 2018 and 2020 up to the present.

Table 1.

Development of Barriers to Research Utilization in Nursing from 2002 to 2021.

The items related to setting were identified as the most common barriers. These include items, “There is insufficient time on the job to implement new ideas” was perceived as the top barrier to nursing research utilization (n = 18, 36%); followed by items, “The facilities are inadequate for implementation,” “The nurse does not feel he or she has enough authority to change patient care procedures,” and “The nurses do not have enough time to read research” with (n = 6, 12%); and item, “Implications for practice are not made clear” was also included as one of the top barriers (n = 3, 6%).

Co-citation network

Figure 2 presents the co-citation network based on cited authors with a minimum number of citations of 20 for each author. Out of 1,660 authors, only 26 authors met the threshold. Red nodes and lines represent the first cluster, the second cluster in green, and the third cluster in blue on the map. Champagne and Funk have the most co-citations more than any other possible co-authorship. Kajermo and Nordstrom are the most co-cited authors in the green cluster, while Krusebrant and Bjorvent are in the blue cluster. Furthermore, the works of Champagne, Tornquist, and Funk are aligned conceptually.

Co-citation network.

Keyword co-occurrence network

Figure 3 shows the keyword co-occurrence network using the keywords with a minimum number of co-occurrences set to 8. There were 378 keywords identified; however, only 35 keywords with strong links are shown in the network. The most common keywords are “human,” “humans,” “adult,” “articles,” “attitude of health personnel,” “nursing research,” and “female.”

Keywords co-occurrence network.

Co-authorship network

The co-authorship network is generated from 53 documents (see Figure 4 ). Each author in the network has a minimum of 1 document and formed 2 clusters. The circle size represents the number of documents made by the author, and the lines represent the strong links between the authors. Out of 180 authors, only 12 authors formed a strong connection in the network. Thompson had the highest co-authorship. He co-authored with Bryar, Baum, Cooke, and Griffith on the red cluster and Lopez and Chau on the green cluster.

Co-authorship network.

Fifty-three studies that utilized the BARRIER scale were included in this review. Setting-related barriers and limitations were identified as the top barriers to research utilization in nursing (n = 39). This is followed by presentation-related barrier (n = 9); and the awareness and nurse-related skills (n = 5). None of the barriers identified research-related barriers. Furthermore, the setting-related barriers accounted for 73% of the top 5 barriers listed. The BARRIER scale's setting-related barriers include insufficient time at work to implement new ideas, insufficient facilities for execution, a lack of power to improve patient care, and inadequate time to read the research. The identified barriers consistently emerge as the main factor affecting research knowledge translation to clinical practice. Regardless of diverse specialization, the absence or presence of support within an organization influences the kind of research culture that promotes a healthy environment for clinical nursing practice ( Berthelsen & Hølge-Hazelton, 2017 ). The nurses’ professionalism, academic reflection, and incorporation of nursing research into daily routine in a supportive environment are essential facets of sustaining nursing research culture toward efficient utilization of research findings to practice. Thus, nursing managers and stakeholders must advocate for initiatives that promote research utilization ( Berthelsen & Hølge-Hazelton, 2017 ).

Significant findings using the most frequently occurring keywords suggest that the discussions on research utilization studies focused on the attitude of health personnel toward understanding adults’ behavior and humanity. Notably, global networks of authors working on research utilization studies were noticeable. The evidence suggests that authors from different countries share related concepts on research utilization. The Middle Eastern authors showed exclusive inter-region collaboration but not with authors from Western countries such as the United States, Canada, and Australia. This could be attributed to the local contextual variation in nursing practice and research utilization.

Research-related barriers were not seen as the top barrier to research utilization. The technology could have contributed to the faster dissemination of research findings. Furthermore, there are multiple avenues for disseminating research findings nowadays, such as journal publishing, conferences, and research colloquia. These provide nurses access to reputable and scientific sources that generate high-quality research.

The theory of diffusion supports the idea that barriers to research utilization start with determining an individual's knowledge on the existence of a challenging situation. Then, this individual is persuaded that change is necessary and will decide to identify appropriate interventions to answer the problem. As an adopter of research findings, nurses must be convinced of applying the new ideas to current practice.

The study's research findings are beneficial to bring improvement to patient outcomes and delivery of care. Therefore, applying theory-derived, research-based knowledge to inform decisions about care delivery is essential. Research utilization is evident in policy and decision-making processes ( Walugembe et al., 2015 ), clinical decisions in patient care units ( Estabrooks et al., 2008 ); in developing practice guidelines for patient referral in an emergency setting ( Sukwatjanee, 2018 ); in reviewing evidence-based management in intrapartum care ( Gennaro et al., 2007 ); use of research to inform practice in pediatric settings ( Cummings et al., 2010 ); in organizing framework for knowledge translation in a public health setting ( Wilson et al., 2011 ), and in documenting HIV research-utilization activities, outputs, and outcomes ( Kalibala et al., 2021 ).

This research has several implications. First, there is a need to reflect on the setting-related factors that hinder research utilization and provide a concrete solution to fill the inadequacies. Providing enough time for nurses to read research findings will allow them to implement relevant research findings and scientific knowledge into clinical practice. Some strategies can be employed to promote the culture of research utilization, such as hiring more skilled and efficient personnel to reduce the workload of the nurse, organizing nurse's work shifts, providing adequate staffing between patients and nurses in the wards, and work deloading for nurses involved in research or issuing a directive requiring nurses to devote part of their time to utilize and implement the research findings. Furthermore, employees reward system, recognition, and availability of conducive, non-threatening, and facilitative environment to implement relevant results to clinical practice. It is therefore indispensable to engage nurses in negotiation and apply decision-making skills, utilizing their bargaining power to request the organization's needs in support of research utilization.

The study provided valuable insights into the status of research utilization, while some limitations need to be noted. First, this bibliometric did not include comments on the quality or content of the articles included in the study. Other methods (such as content analysis) are preferable if such analysis is desired. Second, the BARRIER scale has been examined, with low validity and bias concerning construct validity ( Kajermo et al., 2010 ). However, the scale addressed the general barriers to nursing research utilization; it was beneficial in data collection ( Carlson & Plonczynski, 2008 ; Hutchinson & Kajermo et al., 2010). Third, the BARRIER scale has been criticized for variety of reasons, including not being comprehensive enough ( Carlson & Plonczynski, 2008 ), incorporating generalized notions ( Kajermo et al., 2010 ), and being obsolete in terms of technological integration in research methodologies ( Kajermo et al., 2010 ). It is suggested that an upgraded scale is necessary to incorporate new themes related to nurses’ understanding of barriers. Carlson and Plonczynski (2008) supported the latter idea and proposed the addition of items reflecting the contextual factors in relation to the organizational environment and nursing practice ( Hutchinson & Johnston, 2004 ). Moreover, Rycroft-Malone et al. (2004) added that a local contextualization is vital from an organizational standpoint when creating and enhancing an evidence-based approach. Berthelsen and Hølge-Hazelton (2021) suggest excluding the BARRIER scale because it is outdated or to modify it based on current challenges and focal points in clinical nursing practice. Moreover, emerging tools could have resulted in fewer researches that used the BARRIER scale from 2019 to the present. Fourth, some of the excluded studies that used other scales to measure barriers to research utilization with different parameters other than the BARRIER scale could be a source of selection bias. However, the researchers of this present study believe that the BARRIER scale is still helpful in identifying, exploring, and evaluating the barriers to nursing research utilization. Moreover, they also think the identified barriers from the study will pinpoint the most tailored fit interventions in creating the culture of research utilization.

Strengths & Limitations: The study provides valuable insights into the status of research utilization studies which will be the basis for devising and enhancing interventions utilizing an evidence-based approach. Since the design was purely quantitative, the findings do not provide deeper insights into nurses’ perspectives, which can be done in qualitative studies.

Implication for Practice: The use of research findings in clinical practice to improve care is imperative. Therefore, determining the barriers to research utilization is a significant reference, which will aid for implementing future interventions to foster quality nursing care through the utilization of available scientific research findings.

Despite its limitations, the present findings provide an updated overview of the current research utilization studies conducted in nursing. It is crucial to determine the hindrances to the utilization of research findings. The results of this study establish the connection between research and evidence-based practice which stimulate in meeting the gap in the current nursing practice. Time is a limited resource for implementing research findings; yet, a strong political will by nursing administrators must be imposed. The nurses should feel the ownership of their research, and they are supported and recognized by the administration for their efforts. It is recommended that future studies include research utilization studies that will utilize other tools than the BARRIER scale.

Acknowledgments

The authors thank Ms. Sheryl Nuevo-Jabonete, Mr. Rogelio Ruzco Tobias and Asst Professor Angelica H. De La Cruz for their assistance.

Declaration of Conflicting Interests: The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding: The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD: Fritz Gerald V. Jabonete https://orcid.org/0000-0002-0654-3618

• Aboshaiqah A. E., Qasim A., AlBashaireh A., Patalagsa J. G. (2014). Nurses` perception of barriers to research utilization in a public hospital in Saudi Arabia . Saudi Medical Journal , 35 ( 9 ), 1136–1139. [ PubMed ] [ Google Scholar ]
• Aljezawi M., Al Qadire M., Alhajjy M. H., Tawalbeh L. I., Alamery A. H., Aloush S., ALBashtawy M. (2019). Barriers to integrating research into clinical nursing practice . Journal of Nursing Care Quality , 34 ( 3 ), E7–E11. 10.1097/NCQ.0000000000000371 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Andersson N., Jylli L., Kajermo K. N., Klang B. (2007). Nurses in pediatric care – self-reported professional self and perceived research utilization . Scandinavian Journal of Caring Sciences , 21 ( 4 ), 426–433. 10.1111/j.1471-6712.2006.00486.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Atkinson M., Turkel M., Cashy J. (2008). Overcoming barriers to research in a magnet community hospital . Journal of Nursing Care Quality , 23 ( 4 ), 362–368. 10.1097/01.NCQ.0000336675.48466.37 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Baernholdt M., Lang N. M. (2007). Government chief nursing officers’ perceptions of barriers to using research on staffing . International Nursing Review , 54 ( 1 ), 49–55. 10.1111/j.1466-7657.2007.00511.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Benton D. C., Watkins M. J., Beasley C. J., Ferguson S. L., Holloway A. (2020). Evidence-based policy: Nursing now and the importance of research synthesis . International Nursing Review , 67 ( 1 ), 52–60. 10.1111/inr.12572 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Berthelsen C. B., Hølge-Hazelton B. (2017). Nursing research culture in the context of clinical nursing practice: Addressing a conceptual problem . Journal of Advanced Nursing , 73 (5), 1066–1074. 10.1111/jan.13229 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Berthelsen C., Hølge-Hazelton B. (2021). The importance of context and organization culture in the understanding of nurses’ barriers against research utilization: A systematic review . Worldviews on Evidence-Based Nursing , 18 ( 2 ), 111–117. 10.1111/wvn.12488 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Boström A. M., Kajermo K. N., Nordström G., Wallin L. (2008). Barriers to research utilization and research use among registered nurses working in the care of older people: Does the BARRIERS scale discriminate between research users and non-research users on perceptions of barriers? Implementation Science , 3 (24). 10.1186/1748-5908-3-24 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Brenner M. (2005). Children's nursing in Ireland: Barriers to, and facilitators of, research utilisation . Paediatric Nursing , 17 ( 4 ), 40–45. 10.7748/paed2005.05.17.4.40.c99015906899 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Brown C. E., Ecoff L., Kim S. C., Wickline M. A., Rose B., Klimpel K., Glaser D. (2010). Multi-institutional study of barriers to research utilisation and evidence-based practice among hospital nurses . Journal of Clinical Nursing , 19 ( 13–14 ), 1944–1951. 10.1111/j.1365-2702.2009.03184.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Brown C. E., Wickline M. A., Ecoff L., Glaser D. (2009). Nursing practice, knowledge, attitudes and perceived barriers to evidence-based practice at an academic medical center . Journal of Advanced Nursing , 65 ( 2 ), 371–381. 10.1111/j.1365-2648.2008.04878.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Bryar R. M., Closs S. J., Baum G., Cooke J., Griffiths J., Hostick T., Kelly S., Knight S., Marshall K., Thompson D. R. (2003). The Yorkshire BARRIERS project: Diagnostic analysis of barriers to research utilization . International Journal of Nursing Studies , 40 ( 1 ), 73–84. 10.1016/s0020-7489(02)00039-1 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Carlson C. L., Plonczynski D. J. (2008). Has the BARRIERS scale changed nursing practice? An integrative review . Journal of Advanced Nursing , 63 ( 4 ), 322–333. 10.1111/j.1365-2648.2008.04705.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Carrion M., Woods P., Norman I. (2004). Barriers to research utilisation among forensic mental health nurses . International Journal of Nursing Studies , 41 ( 6 ), 613–619. 10.1016/j.ijnurstu.2004.01.006 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Chen S. H., Shao J. H., Hsiao Y. C., Lee H. C. (2013). Barriers to research utilization by registered nurses in Taiwan . Research in Nursing & Health , 36 ( 2 ), 191–202. 10.1002/nur.21521 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Chien W. T., Bai Q., Wong W. K., Wang H., Lu X. (2013). Nurses’ perceived barriers to and facilitators of research utilization in mainland China: A cross-sectional survey . The Open Nursing Journal , 7 , 96–106. 10.2174/1874434601307010096 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cidoncha-Moreno M. Á., Ruíz de Alegría-Fernandez de Retana B. (2017). Percepción de barreras para la utilización de la investigación en enfermeras de osakidetza [ Barriers to the implementation of research perceived by nurses from osakidetza ]. Enfermeria Clinica , 27 ( 5 ), 286–293. 10.1016/j.enfcli.2017.03.015 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cline G. J., Burger K. J., Amankwah E. K., Goldenberg N. A., Ghazarian S. R. (2017). Promoting the utilization of science in healthcare (PUSH) project: A description of the perceived barriers and facilitators to research utilization among pediatric nurses . Journal for Nurses in Professional Development , 33 ( 3 ), 113–119. 10.1097/NND.0000000000000345 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Cline G. J., Burger K. J., Amankwah E. K., Goldenberg N. A., Ghazarian S. R. (2019). Targeted education and trends in pediatric nurses perceptions of barriers, facilitators, confidence, and attitudes toward research and evidence-based practice over time . Journal for Nurses in Professional Development , 35 ( 2 ), 76–84. 10.1097/NND.0000000000000529 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Closs SJ, Baum G, Bryar RM, Griffiths J, Knight S. (2000). Barriers to research implementation in two Yorkshire hospitals . Clinical Effectiveness in Nursing , 4 (1): 3–10. 10.1054/cein.2000.0100 [ CrossRef ] [ Google Scholar ]
• Cobo M., López-Herrera A., Herrera-Viedma E., Herrera F. (2011). Science mapping software tools: Review, analysis, and cooperative study among tools . journal of American Society for Information Science , 62 (7): 1382–1402. 10.1002/asi.21525 [ CrossRef ] [ Google Scholar ]
• Cummings G. G., Hutchinson A. M., Scott S. D., Norton P. G., Estabrooks C. A. (2010). The relationship between characteristics of context and research utilization in a pediatric setting . BMC Health Services Research , 10 , 168. 10.1186/1472-6963-10-168 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Da'seh A., Rababa M. (2021). Military nurses’ perspectives towards research utilization barriers . Heliyon , 7 ( 10 ), e08181. 10.1016/j.heliyon.2021.e08181 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Estabrooks C. A. (1999a). The conceptual structure of research utilization . Research in Nursing & Health , 22 ( 3 ), 203–216. 10.1002/(sici)1098-240x(199906)22:3<203::aid-nur3>3.0.co;2-9 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Estabrooks C. A. (1999b). Mapping the research utilization field in nursing . The Canadian Journal of Nursing Research = Revue Canadienne de Recherche en Sciences Infirmieres , 31 ( 1 ), 53–72. [ PubMed ] [ Google Scholar ]
• Estabrooks C. A., Scott S., Squires J. E., Stevens B., O'Brien-Pallas L., Watt-Watson J., Profetto-McGrath J., McGilton K., Golden-Biddle K., Lander J., Donner G., Boschma G., Humphrey C. K., Williams J. (2008). Patterns of research utilization on patient care units . Implementation Science , 3 , 31. 10.1186/1748-5908-3-31 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Estabrooks C. A., Winther C., Derksen L. (2004). Mapping the field: A bibliometric analysis of the research utilization literature in nursing . Nursing Research , 53 ( 5 ), 293–303. 10.1097/00006199-200409000-00003 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Fink R., Thompson C. J., Bonnes D. (2005). Overcoming barriers and promoting the use of research in practice . The Journal of Nursing Administration , 35 ( 3 ), 121–129. 10.1097/00005110-200503000-00005 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Funk S. G., Champagne M. T., Wiese R. A., Tornquist E. M. (1991). BARRIERS: The barriers to research utilization scale . Applied Nursing Research , 4 ( 1 ), 39–45. 10.1016/s0897-1897(05)80052-7 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Funk S. G., Tornquist E. M., Champagne M. T. (1995). Barriers and facilitators of research utilization. An integrative review . The Nursing Clinics of North America , 30 ( 3 ), 395–407. [ PubMed ] [ Google Scholar ]
• Geisler E. (2000). The metric of science and technology . Quorum Books. [ Google Scholar ]
• Gennaro S., Mayberry L. J., Kafulafula U. (2007). The evidence supporting nursing management of labor . Journal of Obstetric, Gynecologic, and Neonatal Nursing , 36 ( 6 ), 598–604. 10.1111/j.1552-6909.2007.00194.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Gerrish K., Clayton J. (2004). Promoting evidence-based practice: An organizational approach . Journal of Nursing Management , 12 ( 2 ), 114–123. 10.1111/j.1365-2834.2004.00454.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Glacken M., Chaney D. (2004). Perceived barriers and facilitators to implementing research findings in the Irish practice setting . Journal of Clinical Nursing , 13 ( 6 ), 731–740. 10.1111/j.1365-2702.2004.00941.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hallinger P., Kovačević J. (2019). A bibliometric review of research on educational administration: Science mapping the literature, 1960 to 2018 . Review of Educational Research , 89 (3), 335–369. 10.3102/0034654319830380 [ CrossRef ] [ Google Scholar ]
• Hutchinson A. M., Johnston L. (2004). Bridging the divide: A survey of nurses’ opinions regarding barriers to, and facilitators of, research utilization in the practice setting . Journal of Clinical Nursing , 13 ( 3 ), 304–315. 10.1046/j.1365-2702.2003.00865.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hutchinson A. M., Johnston L. (2006). Beyond the BARRIERS scale: Commonly reported barriers to research use . The Journal of Nursing Administration , 36 ( 4 ), 189–199. 10.1097/00005110-200604000-00008 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kalibala S., Sinai I., Nutley T. (2021). Documenting HIV research-utilization activities, outputs and outcomes: Examples and lessons learned from project SOAR . Archives of Public Health 79 , 99. 10.1186/s13690-021-00628-x [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kajermo K. N., Boström A. M., Thompson D. S., Hutchinson A. M., Estabrooks C. A., Wallin L. (2010). The BARRIERS scale – the barriers to research utilization scale: A systematic review . Implementation Science 5, 32. 10.1186/1748-5908-5-32 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Karkos B., Peters K. (2006). A magnet community hospital: Fewer barriers to nursing research utilization . The Journal of Nursing Administration , 36 ( 7–8 ), 377–382. 10.1097/00005110-200607000-00011 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kirshbaum M., Beaver K., Luker K. (2004). Perspectives of breast care nurses on research dissemination and utilisation . Clinical Effectiveness in Nursing , 8 ( 1 ). 47–58. 10.1016/j.cein.2004.01.001 [ CrossRef ] [ Google Scholar ]
• Kocaman G., Seren S., Lash A. A., Kurt S., Bengu N., Yurumezoglu H. A. (2010). Barriers to research utilisation by staff nurses in a university hospital . Journal of Clinical Nursing , 19 ( 13–14 ), 1908–1918. 10.1111/j.1365-2702.2009.03032.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kuuppelomäki M., Tuomi J. (2003). Finnish nurses’ views on their research activities . Journal of Clinical Nursing , 12 ( 4 ), 589–600. 10.1046/j.1365-2702.2003.00756.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Linnenluecke M. K., Marrone M., Singh A. K. (2020). Conducting systematic literature reviews and bibliometric analyses . Australian Journal of Management , 45 (2), pp. 175–194. https://journals.sagepub.com/doi/full/10.1177/0312896219877678 [ Google Scholar ]
• Mackey A., Bassendowski S. (2017). The history of evidence-based practice in nursing education and practice . Journal of Professional Nursing , 33 ( 1 ), 51–55. 10.1016/j.profnurs.2016.05.00928131148 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• McCleary L., Brown G. T. (2003). Barriers to paediatric nurses’ research utilization . Journal of Advanced Nursing , 42 ( 4 ), 364–372. 10.1046/j.1365-2648.2003.02628.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Moher D., Liberati A., Tetzlaff J., Altman D. G., & PRISMA Group (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement . Annals of Internal Medicine , 151 ( 4 ), 264–W64. 10.7326/0003-4819-151-4-200908180-00135 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Moral-Muñoz José A., Herrera-Viedma Enrique, Santisteban-Espejo Antonio, Cobo Manuel J. (2020). “Software tools for conducting bibliometric analysis in science: An up-to-date review” . El Profesional de la Información , 29 (1), e290103. 10.3145/epi.2020.ene.03 [ CrossRef ]
• Niederhauser V. P., Kohr L. (2005). Research endeavors among pediatric nurse practitioners (REAP) study . Journal of Pediatric Health Care , 19 ( 2 ), 80–89. 10.1016/j.pedhc.2004.08.007 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Nilsson Kajermo K., Nordström G., Krusebrant A., Björvell, H. (1998). Barriers to and facilitators of research utilization, as perceived by a group of registered nurses in Sweden . Journal of Advanced Nursing , 27 ( 4 ), 798–807. 10.1046/j.1365-2648.1998.00614.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Omer T. (2012). Research utilization in a multicultural nursing setting in Saudi Arabia: Barriers and facilitators . The Journal of Nursing Research , 20 ( 1 ), 66–73. 10.1097/JNR.0b013e31824777d8 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Oranta O., Routasalo P., Hupli M. (2002). Barriers to and facilitators of research utilization among Finnish registered nurses . Journal of Clinical Nursing , 11 ( 2 ), 205–213. 10.1046/j.1365-2702.2002.00587.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Parahoo K. (2000). Barriers to, and facilitators of, research utilization among nurses in Northern Ireland . Journal of Advanced Nursing , 31 ( 1 ), 89–98. 10.1046/j.1365-2648.2000.01256.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Patiraki E., Karlou C., Papadopoulou D., Spyridou A., Kouloukoura C., Bare E., Merkouris A. (2004). Barriers in implementing research findings in cancer care: The Greek registered nurses perceptions . European Journal of Oncology Nursing 8 ( 3 ), 245–256. 10.1016/j.ejon.2003.12.002 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Phillips C. (2015). Relationships between duration of practice, educational level, and perception of barriers to implement evidence-based practice among critical care nurses . International Journal of Evidence-Based Healthcare , 13 ( 4 ), 224–232. 10.1097/XEB.0000000000000044 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Pritchard A. (1969). Statistical bibliography or bibliometrics . Journal of Documentation , 25 ( 4 ), 348–349. [ Google Scholar ]
• Retsas A. (2000). Barriers to using research evidence in nursing practice . Journal of Advanced Nursing , 31 ( 3 ), 599–606. 10.1046/j.1365-2648.2000.01315.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rogers E. (1995). Diffusion of innovations (4th ed.). The Free Press, 25–38. [ Google Scholar ]
• Rycroft-Malone J., Seers K., Titchen A., Harvey G., Kitson A., McCormack B. (2004). What counts as evidence in evidence-based practice? Journal of Advanced Nursing , 47 ( 1 ), 81–90. 10.1111/j.1365-2648.2004.03068.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sarabia-Cobo C. M., Sarabia-Cobo A. B., Pérez V., Hermosilla C., Nuñez M. J., de Lorena P. (2015). Barriers in implementing research among registered nurses working in the care of the elderly: A multicenter study in Spain . Applied Nursing Research , 28 ( 4 ), 352–355. 10.1016/j.apnr.2015.03.003 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Schoonover H. (2009). Barriers to research utilization among registered nurses practicing in a community hospital . Journal for Nurses in Staff Development , 25 ( 4 ), 199–212. 10.1097/NND.0b013e3181ae145f [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Stichler J. F., Fields W., Kim S. C., Brown C. E. (2011). Faculty knowledge, attitudes, and perceived barriers to teaching evidence-based nursing . Journal of Professional Nursing , 27 ( 2 ), 92–100. 10.1016/j.profnurs.2010.09.01221420041 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sukwatjanee A. (2018). Practice guideline development for referral patients with acute stroke at primary level hospital . Walailak Journal of Science and Technology , 16 ( 12 ), 921–930. 10.48048/wjst.2019.4058 [ CrossRef ] [ Google Scholar ]
• Tan M., Akgün Sahin Z., Kardas Özdemir F. (2012). Barriers of research utilization from the perspective of nurses in eastern Turkey . Nursing Outlook , 60 ( 1 ), 44–50. 10.1016/j.outlook.2011.07.002 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Uysal A., Temel A. B., Ardahan M., Ozkahraman S. (2010). Barriers to research utilisation among nurses in Turkey . Journal of Clinical Nursing , 19 ( 23–24 ), 3443–3452. 10.1111/j.1365-2702.2010.03318.x [ PubMed ] [ CrossRef ] [ Google Scholar ]
• van Eck N. J., Waltman L. (2010). Software survey: VOSviewer, a computer program for bibliometric mapping . Scientometrics , 84 ( 2 ), 523–538. 10.1007/s11192-009-0146-3 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Walugembe D. R., Kiwanuka S. N., Matovu J. K., Rutebemberwa E., Reichenbach L. (2015). Utilization of research findings for health policy making and practice: Evidence from three case studies in Bangladesh . Health Research Policy and Systems , 13 , 26. 10.1186/s12961-015-0015-x [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Wang L. P., Jiang X. L., Wang L., Wang G. R., Bai Y. J. (2013). Barriers to and facilitators of research utilization: A survey of registered nurses in China . PLoS One , 8 ( 11 ), e81908. 10.1371/journal.pone.0081908 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Wilson K. M., Brady T. J., Lesesne C., & NCCDPHP Work Group on Translation (2011). An organizing framework for translation in public health: The knowledge to action framework . Preventing Chronic Disease , 8 ( 2 ), A46. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Yava A., Tosun N., Ciçek H., Yavan T., Terakye G., Hatipoğlu S. (2009). Nurses’ perceptions of the barriers to and the facilitators of research utilization in Turkey . Applied Nursing Research , 22 ( 3 ), 166–175. 10.1016/j.apnr.2007.11.003 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Zhou F., Maier M., Hao Y., Tang L., Guo H., Liu H., Liu Y. (2015). Barriers to research utilization among registered nurses in traditional Chinese medicine hospitals: A cross-sectional survey in China . Evidence-based Complementary and Alternative Medicine , 2015 , 475340. 10.1155/2015/475340 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Zupic I., Čater T. (2015). Bibliometric methods in management and organization . Organizational Research Methods , 18 ( 3 ), 429–472. 10.1177/1094428114562629 [ CrossRef ] [ Google Scholar ]

Rural Victims of Intimate-Partner Violence Need More Resources and Support, Study Finds

W hile intimate-partner violence is a problem in all areas of the country, victims in rural communities need more resources and support, a new study has found.

The study from the University of Minnesota’ Rural Health Research Center found that rural victims of intimate-partner violence, or IPV, face more barriers and resource limitations that could affect their health and well-being. Attempts to address intimate-partner violence in rural areas should be tailored to the specific needs of the people and places in those areas, the study said.

Alyssa Fritz, the lead researcher on the study, said her team spoke with 15 state and national advocacy organizations, some serving rural communities directly, to determine what barriers rural victims face and what opportunities exist to address those challenges.

“The advocates that we talked to said, across the board, increasing funding that is accessible to smaller and rural organizations is needed,” she said in an interview with the Daily Yonder. “And I think funding training, specifically in detecting and responding to IPV, training for health care providers, and law enforcement and judges, is important.”

All of the respondents said rural victims lack access to services like shelters, advocacy, legal services and law enforcement. Most frequently, the organizations said there was a shortage of support services and shelters in rural communities, and if programs that address intimate-partner violence exist, they are underfunded and understaffed.

Another issue facing rural victims, Fritz said, was a lack of access to health care. Many respondents also said poor health care access, especially for victims who are pregnant and postpartum, was a challenge in IPV intervention. Many times, the respondents said, if victims were able to access health care, there was a lack of IPV-specific and trauma-informed knowledge or training amongst the professionals who interacted with IPV victims in rural areas.

“We did find that pregnant victims of IPV were less likely to be screened (for IPV),” Fritz said. “Anecdotally, we’ve also heard that to be the case (for non-pregnant victims). It’s possibly driven by the fact that there might be less training for rural health care providers… They’re covering lots of different conditions and they might not see things like IPV as often, and don’t have training in it.”

And rural residents tend not to report IPV, the researchers found, because of the lack of anonymity. Nearly half of the organizations brought up a lack of privacy and confidentiality in small communities as an extra challenge that rural victims have to consider when they weigh whether or not to reach out for help or leave. In other cases, attitudes and societal norms in some rural communities may justify or normalize violence and victim-blaming.

“So you [a rural victim] may be much more enmeshed in your community, you may know the lawyer, the judge, the police officer, the abusive partner’s job, family, associations and things like that,” one national advocacy organization was quoted as responding in the study. 

In a different study focused on IPV survivors in Vermont, Anna Mullany, a postdoctoral student at Emory University, found that even organizations that were supposed to help victims sometimes did harm. Prior to her doctoral studies she worked as a crisis counselor for many years. Many of the IPV survivors she talked to said those attitudes were sometimes more damaging than the violence, creating further barriers for victims in getting help.

“I ended up doing 32 interviews with survivors of violence in rural Vermont and a number of them would tell me instances of violence happening out in the community,” Mullany said in an interview with the Daily Yonder. “Not just something that was gossip or that people knew about, but that they actually witnessed… There’s a stigma and judgment they experience that is so detrimental. I actually had one survivor tell me that the judgment she felt after she left the relationship… almost did more damage than the actual abuse.”

Mullany said that, in many cases, victims could not rely on law enforcement to help, either. In remote areas, law enforcement may take 15 or more minutes to respond. And if the officers know the accused, they may choose not to arrest them. 

“One of my interviewees was in an extremely abusive situation and she had called the police on him a number of times,” Mullany said. “One of the times they showed up, they did not arrest him even though there had been a violent incident present. As soon as the police left, she was thrown up against a wall. So, sometimes, if the police are not skilled in that situation or don’t see the level of danger, it can actually cause more harm to the victim because a perpetrator will retaliate.”

Other studies have shown that IPV is more prevalent in rural communities. A 2011 study supported by the University of Iowa Social Research Center and the University of Iowa Injury Prevention Research Center found that women in small rural and isolated areas are more likely to experience IPV than women in urban areas, and generally are three times as far away from IPV resources than urban women. Over 25% of women in rural areas lived more than 40 miles from the closest program compared to less than 1% of urban women. 

Respondents to the University of Minnesota research said IPV doesn’t happen in a vacuum, but instead happens because of the societal structure that exists and will be difficult to undo. 

“People that hold power, you know, are perceived to have more rights to behave the way they want to, and how do you hold power accountable when you’re not in power?” one state advocacy coalition told researchers. 

But combatting all those challenges, Fritz said, means more funding for entities outside of the advocacy organizations as well, she said.

“The American College of Obstetricians and Gynecologists recommends that all pregnant people be screened throughout their pregnancy and postpartum (for IPV),” she said. “But research shows that it does not happen nearly as often as it should. So, a response to that could be looking at reimbursement mechanisms to make sure that providers are incentivized to screen.”

Respondents to the study said IPV prevention initiatives, direct and discretionary financial support for victims, and policy making in rural communities that includes IPV victims and survivors are also needed. 

“Imagine just a very low barrier, means-tested access to income supports, instead of the kind of hostile system that we have, where folks are having to get denied and then reapply; you know, SSI (Supplemental Security Income), those kinds of things, to be able to have their basic needs met. Absolutely fundamental, particularly for folks in rural areas,” one national advocacy organization told researchers. 

But respondents said it was just as imperative to invest in rural community infrastructure to ensure that IPV victims have the resources they need to leave their abusers and to heal in safety. From rural housing access to affordable child care to investment in broadband internet and transportation infrastructure, providing rural IPV victims with resources, services and information was a key factor in helping ensure their safety and health, the study, Intimate-partner Violence in Rural Communities: Perspectives from Key Informant Interviews, published in March 2024 found.

The post Rural Victims of Intimate-Partner Violence Need More Resources and Support, Study Finds appeared first on The Daily Yonder .