results and discussion research paper

When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

• PLOS Biology
• PLOS Climate
• PLOS Complex Systems
• PLOS Computational Biology
• PLOS Digital Health
• PLOS Genetics
• PLOS Global Public Health
• PLOS Medicine
• PLOS Mental Health
• PLOS Neglected Tropical Diseases
• PLOS Pathogens
• PLOS Sustainability and Transformation
• PLOS Collections
• How to Write Discussions and Conclusions

The discussion section contains the results and outcomes of a study. An effective discussion informs readers what can be learned from your experiment and provides context for the results.

What makes an effective discussion?

When you’re ready to write your discussion, you’ve already introduced the purpose of your study and provided an in-depth description of the methodology. The discussion informs readers about the larger implications of your study based on the results. Highlighting these implications while not overstating the findings can be challenging, especially when you’re submitting to a journal that selects articles based on novelty or potential impact. Regardless of what journal you are submitting to, the discussion section always serves the same purpose: concluding what your study results actually mean.

A successful discussion section puts your findings in context. It should include:

• the results of your research,
• a discussion of related research, and
• a comparison between your results and initial hypothesis.

Tip: Not all journals share the same naming conventions.

You can apply the advice in this article to the conclusion, results or discussion sections of your manuscript.

Our Early Career Researcher community tells us that the conclusion is often considered the most difficult aspect of a manuscript to write. To help, this guide provides questions to ask yourself, a basic structure to model your discussion off of and examples from published manuscripts. 

Questions to ask yourself:

• Was my hypothesis correct?
• If my hypothesis is partially correct or entirely different, what can be learned from the results? 
• How do the conclusions reshape or add onto the existing knowledge in the field? What does previous research say about the topic? 
• Why are the results important or relevant to your audience? Do they add further evidence to a scientific consensus or disprove prior studies? 
• How can future research build on these observations? What are the key experiments that must be done? 
• What is the “take-home” message you want your reader to leave with?

How to structure a discussion

Trying to fit a complete discussion into a single paragraph can add unnecessary stress to the writing process. If possible, you’ll want to give yourself two or three paragraphs to give the reader a comprehensive understanding of your study as a whole. Here’s one way to structure an effective discussion:

Writing Tips

While the above sections can help you brainstorm and structure your discussion, there are many common mistakes that writers revert to when having difficulties with their paper. Writing a discussion can be a delicate balance between summarizing your results, providing proper context for your research and avoiding introducing new information. Remember that your paper should be both confident and honest about the results! 

• Read the journal’s guidelines on the discussion and conclusion sections. If possible, learn about the guidelines before writing the discussion to ensure you’re writing to meet their expectations. 
• Begin with a clear statement of the principal findings. This will reinforce the main take-away for the reader and set up the rest of the discussion. 
• Explain why the outcomes of your study are important to the reader. Discuss the implications of your findings realistically based on previous literature, highlighting both the strengths and limitations of the research. 
• State whether the results prove or disprove your hypothesis. If your hypothesis was disproved, what might be the reasons? 
• Introduce new or expanded ways to think about the research question. Indicate what next steps can be taken to further pursue any unresolved questions. 
• If dealing with a contemporary or ongoing problem, such as climate change, discuss possible consequences if the problem is avoided. 
• Be concise. Adding unnecessary detail can distract from the main findings. 

Don’t

• Rewrite your abstract. Statements with “we investigated” or “we studied” generally do not belong in the discussion. 
• Include new arguments or evidence not previously discussed. Necessary information and evidence should be introduced in the main body of the paper. 
• Apologize. Even if your research contains significant limitations, don’t undermine your authority by including statements that doubt your methodology or execution. 
• Shy away from speaking on limitations or negative results. Including limitations and negative results will give readers a complete understanding of the presented research. Potential limitations include sources of potential bias, threats to internal or external validity, barriers to implementing an intervention and other issues inherent to the study design. 
• Overstate the importance of your findings. Making grand statements about how a study will fully resolve large questions can lead readers to doubt the success of the research. 

Snippets of Effective Discussions:

Consumer-based actions to reduce plastic pollution in rivers: A multi-criteria decision analysis approach

Identifying reliable indicators of fitness in polar bears

• How to Write a Great Title
• How to Write an Abstract
• How to Write Your Methods
• How to Report Statistics
• How to Edit Your Work

The contents of the Peer Review Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

The contents of the Writing Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

There’s a lot to consider when deciding where to submit your work. Learn how to choose a journal that will help your study reach its audience, while reflecting your values as a researcher…

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Int J Endocrinol Metab
• v.17(2); 2019 Apr

The Principles of Biomedical Scientific Writing: Results

Zahra bahadoran.

1 Nutrition and Endocrine Research Center, Research Institute for Endocrine Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran

Parvin Mirmiran

2 Department of Clinical Nutrition and Diet Therapy, Faculty of Nutrition Sciences and Food Technology, National Nutrition and Food Technology Research Institute, Shahid Beheshti University of Medical Sciences, Tehran, Iran

Azita Zadeh-Vakili

3 Cellular and Molecular Endocrine Research Center, Research Institute for Endocrine Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran

Farhad Hosseinpanah

4 Obesity Research Center, Research Institute for Endocrine Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran

Asghar Ghasemi

5 Endocrine Physiology Research Center, Research Institute for Endocrine Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran

The “results section” of a scientific paper provides the results related to all measurements and outcomes that have been posted earlier in the materials and methods section. This section consists of text, figures, and tables presenting detailed data and facts without interpretation and discussion. Results may be presented in chronological order, general to specific order, most to least important order, or may be organized according to the topic/study groups or experiment/measured parameters. The primary content of this section includes the most relevant results that correspond to the central question stated in the introduction section, whether they support the hypothesis or not. Findings related to secondary outcomes and subgroup analyses may be reported in this section. All results should be presented in a clear, concise, and sensible manner. In this review, we discuss the function, content, and organization of the “results section,” as well as the principles and the most common tips for the writing of this section.

The “results section” is the heart of the paper, around which the other sections are organized ( 1 ). Research is about results and the reader comes to the paper to discover the results ( 2 ). In this section, authors contribute to the development of scientific literature by providing novel, hitherto unknown knowledge ( 3 ). In addition to the results, this section contains data and statistical information for supporting or refuting the hypothesis proposed in the introduction ( 4 ).

“Results section” should provide an objective description of the main findings, clearly and concisely, without interpretation ( 5 , 6 ). The authors need to use an interesting combination of text, tables, and figures to answer the study questions and to tell the story without diversions ( 7 ). The systemic assessment of published articles highlights the fact that the literature frequently suffers from selective reporting of results only for certain assessed outcomes, selective reporting of statistical analyses, and confused, ambiguous, incomplete, or misleading presentation of data ( 8 , 9 ).

In this section of our series on the principles of biomedical scientific writing ( 10 , 11 ), we describe the function, content, and organization of the “results section” in a scientific paper (mostly for hypothesis-testing papers) and provide common recommendations that can help authors to write this section more effectively.

2. The Function of the Results Section

The function of the “results section” is to present the main results of experiments described in the materials and methods section ( 12 , 13 ) and to present the supporting data in the form of text, tables, and figures ( 13 ). This section should answer the basic question: “What did the authors find in research?” By providing the results, authors try to elucidate the research data, making it to the point and meaningful ( 13 ).

3. Content of the Results Section

The “results section” includes both results and data that are presented in text, tables, and figures. Results are presented in the text; data (the most important) are presented in figures and tables, with a limited amount presented in the text ( 13 ). Statistically relevant parameters including sample size, P values, and the type of statistics used are also presented in this section ( 13 ).

3.1. Difference Between Data and Results

Data and results are not the same ( 14 ); providing results but no data vs. data but no results should be avoided ( 14 , 15 ). Results are general statements in the main text that summarize or explain what the data (facts and numbers) show ( 13 , 14 ); in other words, results are text descriptions of what is important about data ( 16 ) and give meaning to the data ( 15 ). When reporting data or results, make sure that they are logical ( 2 ). See Box 1 for more differences between results and data.

a The text presented in square brackets is data and the remainder is a result.

3.2. The Appropriate Format for Presenting Data/Results

Depending on how the data best support the findings of the study, the “results section” is structured as text, tables, and figures ( 12 ) and should consist of a dynamic interplay between text and figures/tables; the most important data are usually presented in both formats ( 17 ). The reader should select the mode of presentation in a way that optimizes comprehension of the data; however, as a general rule, if you want to present three or fewer numbers, you should use a sentence; otherwise, you consider a table or a graph ( 18 ).

Selecting the best format for presenting results/data depends on the level of details (exact values or patterns) to present ( 19 ). Tables are useful to present specific information or exact values ( 19 ), and function as reference tools for readers ( 20 ) whereas figures are useful to show comparisons and patterns ( 19 ), functioning as analytic tools ( 20 ).

Tables are meant to summarize large amounts of data, to organize and display data more clearly than words, to compare groups of data, to simplify found information, and to facilitate calculations ( 19 ). A table typically has three or more interrelated columns and three or more interrelated rows; otherwise, presenting the information in the text may be more appropriate ( 19 ).

The functions of figures include: (1) showing the underlying patterns of data that are not presentable in text or tables, (2) displaying data more clearly than they can be done in text or tables, (3) more summarizing a large amount of data than they can be done in text or tables, and (4) improving the understanding and locating the specific information easily and rapidly ( 21 ).

3.3. Results

The primary content of this section includes the most relevant (but not all) results corresponding to the central question posed in the introduction section, whether they support the hypothesis or not ( 12 , 13 ). The secondary findings, e.g., results related to secondary outcomes and subgroup analyses, may also be reported in this section ( 22 ). Results must be presented for both experimental and control groups ( 13 ). Results of each item mentioned in the materials and methods should be given in the results section ( 12 , 15 ).

The text of the “results section” should state and summarize the main results and explain the data presented within tables and/or figures ( 23 ); reiteration of all numbers presented in tables and figures is not recommended ( 22 ); however, readers must be given the main messages derived from a table or figure without having to interpret the data themselves ( 7 ). It means that if there is a large amount of data in a table or figure, restating a key piece of data in the text is acceptable and helps the reader zero in on important data ( 14 ).

3.3.1. Reporting Negative Findings

Authors are highly recommended excluding irrelevant results but not ignoring valid anomalous results that contradict the research hypothesis or do not support the current scientific literature ( 22 ). The Feynman, says “if you are doing an experiment, you should report everything that you think might make it invalid-not only what you think is right about it” ( 24 ). Although reporting null or negative findings is not as straightforward as positive findings, it may lead to reexamining current scientific thinking, and guide scientists towards unabridged science ( 25 ). Reporting negative findings can also prevent the replication of the study and prevent the waste of time and resources ( 25 ). The ignorance of null or negative findings also leads to an overestimation of an effect size or treatment effect in available data ( 9 ).

3.3.2. Referring to Unpublished Results

Referring to unpublished results is not recommend unless there is a strong argument supporting their inclusion ( 14 ); therefore, authors are advised to avoid using the term “data not shown” ( 4 ).

3.3.3. Methods or Interpretation in the Results Section

Generally, the “results section” is not the place for presenting methods and experimental details or interpreting data ( 14 ). When experiments are described in this section, if a result leads to additional experiments, it is better to report the new experimental details in the “results section” ( 14 ). Sometimes authors want to refer to a specific experiment or method in results; in these cases, they should not repeat experimental details, but preferably use a transition phrase to link methods with results ( 14 ). To justify the rationale behind the experiment, using topic sentences/phrases (e.g. in order to determine whether…) provides an overview before giving details ( 12 ); however, in this case, the method statement should not be used as a topic sentence and the main verbs should describe results, not methods (e.g., “ when propranolol was administered during normal ventilation, phospholipids decreased ”; here “ method ” is subordinated in a transition clause and result is the main clause) ( 13 ). Two patterns of sentence structure are recommended for including methods in a result statement: making the method the subject of the sentence or stating the method using a transition phrase or clause and the result in the main clause ( 13 ).

The traditional view of writing the “results section” is just to report data and results without any interpretation; accordingly, the result is not expected to contain statements that need to be referenced (comparisons of findings) ( 13 , 26 ). In another view, some interpretation or brief comparisons that do not fit into the discussion may be included ( 13 , 27 ).

Data are facts and numbers, mostly presented as non-textual elements (usually in tables and figures) where they are easy to read ( 13 , 14 , 28 ). A limited amount of data may also be presented in the text, following a result statement ( 13 ) although too much data in the text make it too long ( Box 1 ) ( 28 ). Data may be in the form of raw data, summarized data, or transformed data ( 13 ); however, it is suggested that raw data (i.e. patients’ records, individual observations) not be presented in results ( 12 ). Note that numerical data are absolute while some data, e.g. microscopic data, are subjective ( 2 ).

3.4.1. Non-Textual Elements

Providing study findings visually, rather than entire textualizing, enables authors to summarize a great deal of data compactly within the text with an appropriate reference; some images convey more than words ( 29 ). The primary purpose of non-textual elements, i.e. tables, graphs, figures, and maps, is to present data such that they can be easily and quickly grasped ( 23 ) while being more informative than when appearing in the text ( 6 ). Tables and figures should be complete/comprehensible, being able to stand alone without the text ( 5 , 12 ).

Non-textual elements should be referred to in the text at the appropriate point ( 5 , 6 , 12 ). Location statements, i.e. statements referring to non-textual elements, may be presented in different patterns (e.g., A. X is shown in table/figure; B. table/figure shows; C. see table/figure; D. as shown in table/figure); pattern B is more and pattern C is less common ( 27 ).

Some general tips about using non-textual elements in the “results section” are reviewed in Box 2 . The most common rules in organizing tables and figures are given in the following. For more information about designing different types of tables/figures/graphs, please refer to additional references ( 7 , 19 , 20 , 30 , 31 ).

3.4.1.1. Tables

The use of tables is an effective way to summarize demographic information and descriptive statistics ( 23 ). Note that tables must have a purpose and be integrated into the text ( 21 ). Tables are most useful to present counts, proportions, and percentages ( 8 ), and are appropriate also for presenting details especially when exact values matter ( 32 ), being are more informative than graphs ( 29 ). However, limited information should be presented in tables; otherwise, most readers find them difficult to read and thus, may ignore them ( 5 , 23 ). Data in tables can be arranged horizontally or vertically; whenever possible, primary comparisons are preferably presented horizontally from left to right ( 19 ).

3.4.1.1.1. Basic Elements of Tables

Tables usually have at least six elements: (1) table number, (2) table title, (3) row headings (stubs), and (4) column headings (boxes), identifying information in rows and columns, (5) data in data field, and (6) horizontal lines (rules). Most also have footnotes, row subheadings, spanner headings (identifying subgroups in column headings), and expanded forms of abbreviations in the table ( 19 , 21 , 31 , 33 ).

The table title should clearly state what appears in it and provide sufficient information on the study, i.e. provide a context helping readers interpret the table information ( 19 ). Some specific details may also be provided including the type and number of subjects or the period of study ( 30 ). For developing the title of a table, one can describe the main cell entries, followed by qualification or more description ( 32 ). The table’s title is presented as a phrase not a full sentence ( 19 ). Authors need to refer to the journal’s style for rules on which words in titles are capitalized.

As a rule, comparing two (or even three) numbers should be side-by-side rather than above and below ( 30 ). Column and row headings help readers find information and they should be included group sizes and measurement units ( 19 ). Tables should be in borderless grids of rows and columns ( 5 , 32 ) with no vertical rule and limited horizontal rules ( 32 ). The first column of a table includes usually a list of variables that are presented in the table; although the first column usually does not need a header, sometimes a simple description of what appears in each row may be provided as the heading of the first column. Units for variables may be placed in parentheses immediately below the row descriptions ( 30 ).

Headings for other columns should also be informative without vague labels, e.g. group A, group B, group C, etc.; instead, a brief description summarizing group characteristics is used ( 30 ). The last column may show P values for comparison between study groups ( 34 ), except for randomized clinical trials, where P values are not needed to compare baseline characteristics of participants ( 7 ). The first letters of lines and column headings in tables should be capitalized.

The fields of tables are points at which columns and rows intersect ( 19 ). Cells of a table are the data field of the table, other than those containing row and column headings ( 21 ). Cells contain information as numerals, text, or symbols ( 19 ). Every cell must contain information; if no information is available, one can use NA in the cell and define it in the footnote as not available or not applicable; alternatively, a dash mark may be inserted ( 19 ). The content of columns need to be aligned ( 19 ); words are usually left aligned, numerals are aligned at decimals, parenthesis, and factors of 10 ( 19 , 21 ).

Table footnotes should be brief, and define abbreviations, provide statistical results, and explain discrepancies in data, e.g., “percentages do not total 100 because of rounding” ( 19 , 30 ). In addition to asterisks usually used to show statistical significance ( 33 ), the following symbols are used, in sequence, for further notes: †, ‡, §, ¶, #, ††, ‡‡ ( 30 ).

3.4.1.1.2. Different Types of Tables

Table of lists, table of baseline or clinical characteristics of subjects, table of comparisons, and table of multivariable results are various types of tables that may be used ( 30 ). The table’s format should be selected according to the purpose of the table ( 30 ). A table of lists just presents a list of items including diagnostic criteria or causes of a disease; it is critical to arrange such tables based on their contents by order (e.g., alphabetical order) or their importance (most to least) ( 30 ). Tables of study participants’ characteristics usually provide a general overview of the essential characteristics of subjects, such as age, sex, race, disease stage, and selected risk factors ( 30 ). The table of comparisons (≥ two groups) provides details for each group and differences between the groups. Tables of multivariable results elaborate results of statistical analyses assessing relationships between predictor (independent) and outcome (dependent) variables, and usually include regression coefficients, standard errors, slopes, partial correlation coefficients, and P values or odds ratio, hazard ratios, and 95% confidence intervals for regression models ( 30 ).

3.4.1.2. Figures

Graphical elements convey the important messages of research ( 20 ). A figure is “any graphical display to present information or data” ( 20 ), and it effectively presents complicated patterns ( 32 ), best used for presenting an important point at a glance or indicating trends or relationships ( 20 ). Like tables, figures should have a purpose and be integrated with the rest of the text ( 21 ).

3.4.1.2.1. Basic Elements of Figures

Most figures that present quantitative information (charts and graphs) have at least seven elements, including figure number, figure caption/legend, data field, vertical scale, horizontal scale, labels, and data (plotting symbols, lines, and so on) ( 21 ). Some figures also have reference lines in the data field to help orient readers and keys that identify data ( 21 ).

Figure caption/legend, usually given below the figure, describes the figure and must reflect the figure entirely, independent of the main text ( 21 , 31 ). For the figure to stand alone, a figure legend needs to be included four parts (a brief title, experimental or statistical information/details, definitions of symbols, line, or bar patterns, and abbreviations) ( 31 ).

Data field is a space in the figure in which data are presented; it is usually bordered on the left by the X-axis (abscissa) and on the bottom by the Y-axis (ordinate) ( 20 , 21 ). Labels identify the variables graphed and the units of measurement ( 21 ). Figure lines should be broad and the labeling text should be large enough to be legible after reduction to a single- or two-column size ( 32 ). Appropriate font size should be used to maintain legibility after fitting figures to publication size ( 31 ).

Scales on each axis should match the data range and be slightly above the highest value ( 20 ). Symbols should be uniform across the figures ( 20 ). The data point symbols should be easily distinguishable; using black and white circles (● - ∘) is the easiest way when two are needed ( 31 ); if more are needed, using up-pointing triangles (▲ - Δ) and squares (■ - □) is suggested ( 31 ). Using symbols, line types, and colors is also effective in differentiating important strata in figures ( 8 ).

3.4.1.2.2. Emphasizing Important Data on Figures

To make figures visually efficient, the subordination of all non-data elements vs. data elements is advised (gridlines should be used as thin as possible and very faint). Directly labeling objects, instead of legends, may keep readers’ attention on the most important parts of the figure ( 8 ). Using different line weights may also be helpful to emphasize the important information/data in figures ( 31 ). The use of color, shading, or 3D perspectives is not suggested unless they serve a specific explanatory function in figure ( 8 ).

3.4.1.2.3. Different Types of Figures

Two major categories of figures are statistical figures (graphs) and non-statistical figures (clinical images, photographs, diagrams, illustrations, and textual figures) ( 20 ). Graphs are suitable for presenting relationships whereas non-statistical figures are used to confirm findings or provide explanatory information ( 20 ).

In statistical figures, selecting a graphical format (bar graph, line graph, dot plot, and scatterplot) is done according to the type of relationship that authors wish to communicate ( 20 ); for example, line graphs are appropriate for showing trends and bar graphs for magnitudes ( 20 ). Using a graphing format that is easy to interpret is preferred ( 20 ); pie graphs are sparingly used because comparing different angles is complicated with them ( 20 ). Graphs should accurately represent findings; when possible, scales should start at zero, and figure axes should not be altered in order to make data more meaningful ( 20 ).

Non-statistical figures are those that visually present information that does not contain data ( 20 ). Clinical images and photographs [ultrasonograms, computed tomographic scans (CT scans), magnetic resonance images (MRI), images of patients, tissue samples, microscopic findings, and so on] provide absolute proof of findings ( 20 ). Illustrations are used for explaining structures (parts of a cell), mechanisms, and relationships ( 20 ). Diagrams (flowcharts, algorithms, pedigrees, and maps) are useful for displaying complex relations ( 20 ). Textual figures, containing only text, are mostly used for describing steps of a procedure or summarizing guidelines ( 20 ). For photographs, patient information or identifiers should be removed ( 20 ).

3.5. Statistics in the Results Section

Statistics in the “results section” must report data in a way that enables readers to assess the degree of experimental variation and to estimate the variability or precision of the findings ( 22 ). For more details, one can see SAMPL (Statistical Analysis and methods in the Published Literature) guidelines ( 35 ). To report normally distributed data, the mean and estimated variation from mean should be stated ( 13 ). Variability should be reported using standard deviation (SD), which is a descriptive statistic ( 36 ) and reflects the dispersion of individual sample observation of the sample mean ( 37 ). The standard error (SE), an inferential statistic ( 36 ) reflecting the theoretical dispersion of sample means about some population means, characterizes uncertainty about true values of population means ( 37 ). It is useful for assessing the precision of an estimator ( 36 ) and is not an appropriate estimate of the variability in observations ( 37 ). Using “mean (SD or SE)” is preferred to “mean ± SD or SE” because the “±” sign can cause confusion ( 22 ). Increasing sample size decreases SE but not SD ( 36 ). To report data with a skewed distribution, the median and the interquartile range (between 25th and 75th percentiles) should be provided ( 22 ).

To report risk, rates, and ratios, one should use a type of rate (incidence rate, survival rate), ratio (odds ratio, hazards ratio), or risk (absolute risk, relative risk, relative risk reduction) ( 35 ). The measure of precision (95% CI) for estimated risks, rates, and ratios should also be provided ( 35 ). For correlation analysis, the exact values of the correlation coefficient and 95% CI should be reported. Describing correlation using qualitative words (low, moderate, high) without providing a clear definition is not acceptable ( 35 ). Results of regression analysis should include regression coefficients (β) of each explanatory variable, corresponding 95% CI and/or P value and a measure of the “goodness-of-fit” of the model ( 35 ).

3.5.1. Significance Levels

A P value is the probability of consistency between data and the hypothesis being tested ( 38 ). Reporting the exact P values ( P = 0.34 or P = 0.02) rather than the conventional P ( P < 0.05) is recommended for all primary analyses ( 12 , 37 ) as it conveys more information ( 37 ). The use of the term “partially significant” or “marginally significant”, where the P value is almost significant (e.g. P = 0.057) is not acceptable if the significance level is defined as P = 0.05 ( 39 ). Some, however, argue that it is not always necessary to stick to P = 0.05 for the interpretation of results and it is better to report the exact P value and confidence interval for the estimator ( 40 ).

The use of the 95% confidence interval (95% CI) can provide further information compared to P values per se, and prefigures the direction of the effect size (negative or positive), its magnitude, and the degree of precision ( 17 ). A confidence interval characterizes uncertainty about the true value of population parameters ( 37 ). It is essential to provide the sample size (n) and probability values for tests of statistical significance ( 13 ).

Statements about significance must be qualified numerically ( 41 ). In the text, it is suggested that P values be reported as equalities rather than as inequalities in relation to the alpha criterion ( 41 ). In tables and figures, inequalities may be useful for groups of data ( 41 ) where asterisks *, **, and *** are usually used to show statistical significance at 0.05, 0.01, and 0.001 probability levels, respectively ( 33 ).

Although not consistent, P values < 0.001 are reported as P < 0.001; for 0.001 ≤ P values < 0.01, a three-significant digit is recommended, e.g. P = 0.003; for 0.01 ≤ P values < 0.1, a two-significant digit is sufficient (e.g. P = 0.05); for 0.1 ≤ P values ≤ 0.9, a one-significant digit is sufficient (e.g. P = 0.4); and P values > 0.9 are reported as P > 0.9 ( 42 ). For genome-wide association studies, the power of 10 is used for reporting P values, e.g. 6 × 10 -9 ( 42 ). It is generally suggested that zero be used before a decimal point when the value is below one, e.g. 0.37 ( 43 ). According to the American Psychological Association, zero before a decimal point is used for numbers that are below one, but it can also be used for values that may exceed one (e.g. 0.23 cm). Therefore, when statistics cannot be greater than one (e.g. correlations, proportions, and P values), do not use a zero before decimal fraction, e.g. P = .028 not P = 0.028 ( 18 ); this recommendation, however, is not always adopted by everyone. The international standard is P (large italic) although both ‘p’ and ‘P’ are allowed ( 40 ).

4. Organization of the Results Section

There are different ways for organizing the “results section” including ( 1 , 12 , 14 , 22 , 44 ): (1) chronological order, (2) general to specific, (3) most to least important, and (4) grouping results by topic/study groups or experiment/measured parameters. Authors decide which format is more appropriate for the presentation of their data ( 12 ); anyway, results should be presented in a logical manner ( 4 ).

4.1. Different Ways of Organizing the Results Section

4.1.1. chronological order.

The best order for organizing “results section” may be the chronological order ( 22 ). It is considered as the most straightforward approach using subheadings that parallel methods ( 14 ). This order facilitates referring to a method associated with a given result ( 14 ) such that results are presented in the same order as methods ( 15 ).

4.1.2. General to Specific

This format is mostly used in clinical studies involving multiple groups of individuals receiving different treatments ( 14 ). The “results section” usually proceeds from general to more specific findings ( 1 ). Characteristics of the overall study population (sex and age distribution and dropouts) are first given ( 14 ), followed by data and results for each group starting with the control group or the group receiving the standard treatment ( 14 ); finally, the disease group or group receiving the experimental treatment are addressed ( 14 ). As a general rule, secondary results should be given after presenting more important (primary) results, followed by any supporting information ( 22 ). A common order is stating recruitment/response, characteristics of the sample/study participants, findings from the primary analyses, findings from secondary analyses, and any additional or unexpected findings ( 17 ). In other words, the “results section” should be initiated by univariate statistics, followed by bivariate analyses to describe associations between explanatory and outcome variables; finally, it gets through by any multivariate analyses ( 7 ).

4.1.3. Most to Least Important

This format is used in case that the order of presenting results is not critical to their being comprehendible and allows the author to immediately highlight important findings ( 14 ). Results that answer the main question are presented at the beginning of the “results section,” followed by other results in next paragraphs ( 13 ).

4.1.4. Grouping by Topic or Experiment

Comparison of the diagnostic and analytical performance of a number of assays for analytes is an example of using this format ( 14 ).

4.2. Paragraphing of the Results Section

The “results section” may be initiated by two approaches: (1) by giving a general (not detailed) overview of the experiment and (2) by going directly to the results by referring to tables or figures ( 44 ). The first paragraph of this section, along with table 1, describes the characteristics of the study population (number, sex, age, and symptoms) ( 23 ). These data show the comparability of the study groups at baseline and the distribution of potential confounders between groups, as a source of bias that can affect the study findings ( 7 ). It allows the reader to decide whether or not the case and control groups are similar and represent the patient population in their private practice ( 23 ).

For clinical trials, the number of patients completing the protocol in each treatment/study group, the number of patients lost to follow-up, and the number and reasons for excluded/withdrawn subjects should be given. Commenting on whether baseline characteristics of study groups are statistically similar or different is also important ( 1 ). For further information, authors can consult reporting guidelines for the main study types available at http://www.equator-network.org.

The number of the middle paragraphs depends on the number of research questions/hypotheses and the types of statistical analyses; each hypothesis or specific analysis typically devotes at least a paragraph to itself ( 1 ). Figure legends, description of the methods and results for control groups should not be given at the beginning of paragraphs, as they do not narrate the story ( 28 ). However, sometimes, it is needed that results of the control group are presented first (e.g. for establishing the stability of baseline) ( 13 ).

5. Emphasizing Important Results

Since not all results are equally important, the reader must be able to distinguish important results and authors have to emphasize important information and de-emphasize less important information ( 13 ). There are various techniques for emphasizing important information, including condensing or omitting less important information, subordinating less important information, placing important results at the power position, and labeling, stating, and repeating important information ( 13 ).

For condensing or omitting less important information, you should be careful not to duplicate/repeat data in tables and figures or repeat them in the text ( 4 , 6 , 12 ); one or two values from tables/figures can be repeated in the text for emphasis ( 13 ).

For subordinating less important information, one should not use table titles, figure legends or methods statement as a topic sentence in the text ( 13 , 22 ). Instead, after stating the first result relevant to the table/figure, you can cite it in parenthesis ( 13 ). Since a result states a message and creates an expectation, it is a more powerful topic sentence than a figure legend or table title ( 13 ). Sometimes, control results can be subordinated by incorporating them into experimental results ( 13 ).

To highlight more important results (those that help answer questions), authors can put these results at the beginning of paragraphs, the strongest power position ( 12 , 22 , 28 ), followed by supporting details and control results ( 28 ).

Moreover, key findings may receive more attention by using a signal (e.g. we found or we observed) at the beginning of the sentence ( 13 ).

6. Other Considerations

6.1. length and paragraphing.

To see the forest for the tree, the “results section” should be as brief and uncluttered as possible ( 13 ), which can be accomplished by having a well-organized “materials and methods” section ( 3 ) and avoiding unnecessary repetition ( 13 ); for example, similar results for several variables can be reported together. The “results section” of an original manuscript usually includes 2 - 3 pages (~1000 words) with a 1.5 line spacing, font size 11 (including tables and figures) ( 45 ), and 4 - 9 paragraphs (each 130 words) on average ( 45 ); a paragraph should be devoted to one or more closely related figures ( 4 ).

Presenting additional results/data as supplementary materials is a suggestion for keeping the “results section” brief ( 17 ). In addition to save the text space, supplementary materials improve the presentation and facilitate communications among scientists ( 46 , 47 ). According to Springer, supplementary materials can be used for presenting data that are not needed to support the major conclusions but are still interesting. However, keep in mind that the unregulated use of supplementary materials is harmful to science ( 47 ). Supplementary materials should be referred to at the appropriate points in the main text.

For referring to results obtained in hypothesis testing studies, using past tenses is recommended ( 4 , 12 - 14 ); non-textual elements should be referred using present tenses, e.g. “as seen in table 1 …” or “table 1 shows …” in descriptive studies, results are reported in the present tense ( 13 ).

6.3. Word Choice

Although adverbs/adjectives are commonly used to highlight the importance of results, it is recommended altogether avoiding the use of such qualitative/emotive words in the “results section” ( 7 , 13 ). Some believe that qualitative words should not be used because they may imply an interpretation of findings ( 17 ). In biomedical publications, the terms ‘significant, significance, and significantly’ (followed by P values) are used to show statistical relationships and should not be used for other purposes for which, other terms such as substantial, considerable, or noteworthy can be used ( 14 ). See Box 3 for appropriate word choice for the “results section.”

In the “results section,” to make a comparison between the results, i.e. stating the similarity/equivalence or difference/non-equivalence, using appropriate signals is recommended ( 27 ). To show a similarity, a signal to the reader may be used such as “like”, “alike”, “similar to”, and “the same as”; to show differences, the following signals can be used: “but”, “while”, “however”, “in contrast”, “more likely than”, and “less likely than” ( 27 ).

6.4. Reporting Numbers

Numbers play an important role in scientific communication and there are some golden rules for reporting numbers in a scientific paper ( 43 , 48 ). Significant figures (significant digits) should reflect the degree of precision of the original measurement ( 12 ). The number of digits reported for a quantity should be consistent with scientific relevance ( 37 ); for example, a resolution to 0.001 units is necessary for pH but a resolution of < 1 mm Hg is unimportant for blood pressure ( 37 ). Avoid using “about” or “approximately” to qualify a measurement or calculation ( 12 ). The use of percentage for sample sizes of < 20 and decimal for sample sizes of < 100 is not recommended ( 43 ).

The numbers should be spelled out at the beginning of a sentence or when they are less than 10, e.g., twelve students improved… ( 43 ). In a sentence, the authors should be consistent where they use numbers as numerals or spelled-out ( 43 ). Before a unit of a measure, time, dates, and points, numbers should be used as numerals, e.g. 12 cm; 1 h 34 min; at 12:30 A.M., and on a 7-point scale ( 18 ).

A space between the numeral and the unit should be considered, except in the case of %. Because the terms “billion,” “trillion,” and “quadrillion” imply different numbers in Europe and the USA, they should not be used ( 48 ). To express ranges in text, the terms “to” or “through” are preferred to dashes; in tables, the use of dashes or hyphens is recommended ( 48 ).

7. Conclusions

The “results section” of a biomedical manuscript should clearly present findings of the study using an effective combination of results and data. Some dos and don’ts of writing the “results section” are provided in Box 4 . Authors should try to find the best format using a dynamic interplay between text and figures/tables. Results can be organized in different ways including chronological order or most to least important; however, results should be presented in a manner that makes sense.

Acknowledgments

The authors wish to acknowledge Ms. Niloofar Shiva for critical editing of English grammar and syntax of the manuscript.

Conflict of Interests: It is not declared by the authors.

Funding/Support: Research Institute for Endocrine Sciences supported the study.

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 7. The Results
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

The results section is where you report the findings of your study based upon the methodology [or methodologies] you applied to gather information. The results section should state the findings of the research arranged in a logical sequence without bias or interpretation. A section describing results should be particularly detailed if your paper includes data generated from your own research.

Annesley, Thomas M. "Show Your Cards: The Results Section and the Poker Game." Clinical Chemistry 56 (July 2010): 1066-1070.

Importance of a Good Results Section

When formulating the results section, it's important to remember that the results of a study do not prove anything . Findings can only confirm or reject the hypothesis underpinning your study. However, the act of articulating the results helps you to understand the problem from within, to break it into pieces, and to view the research problem from various perspectives.

The page length of this section is set by the amount and types of data to be reported . Be concise. Use non-textual elements appropriately, such as figures and tables, to present findings more effectively. In deciding what data to describe in your results section, you must clearly distinguish information that would normally be included in a research paper from any raw data or other content that could be included as an appendix. In general, raw data that has not been summarized should not be included in the main text of your paper unless requested to do so by your professor.

Avoid providing data that is not critical to answering the research question . The background information you described in the introduction section should provide the reader with any additional context or explanation needed to understand the results. A good strategy is to always re-read the background section of your paper after you have written up your results to ensure that the reader has enough context to understand the results [and, later, how you interpreted the results in the discussion section of your paper that follows].

Bavdekar, Sandeep B. and Sneha Chandak. "Results: Unraveling the Findings." Journal of the Association of Physicians of India 63 (September 2015): 44-46; Brett, Paul. "A Genre Analysis of the Results Section of Sociology Articles." English for Specific Speakers 13 (1994): 47-59; Go to English for Specific Purposes on ScienceDirect;Burton, Neil et al. Doing Your Education Research Project . Los Angeles, CA: SAGE, 2008; Results. The Structure, Format, Content, and Style of a Journal-Style Scientific Paper. Department of Biology. Bates College; Kretchmer, Paul. Twelve Steps to Writing an Effective Results Section. San Francisco Edit; "Reporting Findings." In Making Sense of Social Research Malcolm Williams, editor. (London;: SAGE Publications, 2003) pp. 188-207.

Structure and Writing Style

I.  Organization and Approach

For most research papers in the social and behavioral sciences, there are two possible ways of organizing the results . Both approaches are appropriate in how you report your findings, but use only one approach.

• Present a synopsis of the results followed by an explanation of key findings . This approach can be used to highlight important findings. For example, you may have noticed an unusual correlation between two variables during the analysis of your findings. It is appropriate to highlight this finding in the results section. However, speculating as to why this correlation exists and offering a hypothesis about what may be happening belongs in the discussion section of your paper.
• Present a result and then explain it, before presenting the next result then explaining it, and so on, then end with an overall synopsis . This is the preferred approach if you have multiple results of equal significance. It is more common in longer papers because it helps the reader to better understand each finding. In this model, it is helpful to provide a brief conclusion that ties each of the findings together and provides a narrative bridge to the discussion section of the your paper.

NOTE :   Just as the literature review should be arranged under conceptual categories rather than systematically describing each source, you should also organize your findings under key themes related to addressing the research problem. This can be done under either format noted above [i.e., a thorough explanation of the key results or a sequential, thematic description and explanation of each finding].

II.  Content

In general, the content of your results section should include the following:

• Introductory context for understanding the results by restating the research problem underpinning your study . This is useful in re-orientating the reader's focus back to the research problem after having read a review of the literature and your explanation of the methods used for gathering and analyzing information.
• Inclusion of non-textual elements, such as, figures, charts, photos, maps, tables, etc. to further illustrate key findings, if appropriate . Rather than relying entirely on descriptive text, consider how your findings can be presented visually. This is a helpful way of condensing a lot of data into one place that can then be referred to in the text. Consider referring to appendices if there is a lot of non-textual elements.
• A systematic description of your results, highlighting for the reader observations that are most relevant to the topic under investigation . Not all results that emerge from the methodology used to gather information may be related to answering the " So What? " question. Do not confuse observations with interpretations; observations in this context refers to highlighting important findings you discovered through a process of reviewing prior literature and gathering data.
• The page length of your results section is guided by the amount and types of data to be reported . However, focus on findings that are important and related to addressing the research problem. It is not uncommon to have unanticipated results that are not relevant to answering the research question. This is not to say that you don't acknowledge tangential findings and, in fact, can be referred to as areas for further research in the conclusion of your paper. However, spending time in the results section describing tangential findings clutters your overall results section and distracts the reader.
• A short paragraph that concludes the results section by synthesizing the key findings of the study . Highlight the most important findings you want readers to remember as they transition into the discussion section. This is particularly important if, for example, there are many results to report, the findings are complicated or unanticipated, or they are impactful or actionable in some way [i.e., able to be pursued in a feasible way applied to practice].

NOTE:   Always use the past tense when referring to your study's findings. Reference to findings should always be described as having already happened because the method used to gather the information has been completed.

III.  Problems to Avoid

When writing the results section, avoid doing the following :

• Discussing or interpreting your results . Save this for the discussion section of your paper, although where appropriate, you should compare or contrast specific results to those found in other studies [e.g., "Similar to the work of Smith [1990], one of the findings of this study is the strong correlation between motivation and academic achievement...."].
• Reporting background information or attempting to explain your findings. This should have been done in your introduction section, but don't panic! Often the results of a study point to the need for additional background information or to explain the topic further, so don't think you did something wrong. Writing up research is rarely a linear process. Always revise your introduction as needed.
• Ignoring negative results . A negative result generally refers to a finding that does not support the underlying assumptions of your study. Do not ignore them. Document these findings and then state in your discussion section why you believe a negative result emerged from your study. Note that negative results, and how you handle them, can give you an opportunity to write a more engaging discussion section, therefore, don't be hesitant to highlight them.
• Including raw data or intermediate calculations . Ask your professor if you need to include any raw data generated by your study, such as transcripts from interviews or data files. If raw data is to be included, place it in an appendix or set of appendices that are referred to in the text.
• Be as factual and concise as possible in reporting your findings . Do not use phrases that are vague or non-specific, such as, "appeared to be greater than other variables..." or "demonstrates promising trends that...." Subjective modifiers should be explained in the discussion section of the paper [i.e., why did one variable appear greater? Or, how does the finding demonstrate a promising trend?].
• Presenting the same data or repeating the same information more than once . If you want to highlight a particular finding, it is appropriate to do so in the results section. However, you should emphasize its significance in relation to addressing the research problem in the discussion section. Do not repeat it in your results section because you can do that in the conclusion of your paper.
• Confusing figures with tables . Be sure to properly label any non-textual elements in your paper. Don't call a chart an illustration or a figure a table. If you are not sure, go here .

Annesley, Thomas M. "Show Your Cards: The Results Section and the Poker Game." Clinical Chemistry 56 (July 2010): 1066-1070; Bavdekar, Sandeep B. and Sneha Chandak. "Results: Unraveling the Findings." Journal of the Association of Physicians of India 63 (September 2015): 44-46; Burton, Neil et al. Doing Your Education Research Project . Los Angeles, CA: SAGE, 2008;  Caprette, David R. Writing Research Papers. Experimental Biosciences Resources. Rice University; Hancock, Dawson R. and Bob Algozzine. Doing Case Study Research: A Practical Guide for Beginning Researchers . 2nd ed. New York: Teachers College Press, 2011; Introduction to Nursing Research: Reporting Research Findings. Nursing Research: Open Access Nursing Research and Review Articles. (January 4, 2012); Kretchmer, Paul. Twelve Steps to Writing an Effective Results Section. San Francisco Edit ; Ng, K. H. and W. C. Peh. "Writing the Results." Singapore Medical Journal 49 (2008): 967-968; Reporting Research Findings. Wilder Research, in partnership with the Minnesota Department of Human Services. (February 2009); Results. The Structure, Format, Content, and Style of a Journal-Style Scientific Paper. Department of Biology. Bates College; Schafer, Mickey S. Writing the Results. Thesis Writing in the Sciences. Course Syllabus. University of Florida.

Writing Tip

Why Don't I Just Combine the Results Section with the Discussion Section?

It's not unusual to find articles in scholarly social science journals where the author(s) have combined a description of the findings with a discussion about their significance and implications. You could do this. However, if you are inexperienced writing research papers, consider creating two distinct sections for each section in your paper as a way to better organize your thoughts and, by extension, your paper. Think of the results section as the place where you report what your study found; think of the discussion section as the place where you interpret the information and answer the "So What?" question. As you become more skilled writing research papers, you can consider melding the results of your study with a discussion of its implications.

Driscoll, Dana Lynn and Aleksandra Kasztalska. Writing the Experimental Report: Methods, Results, and Discussion. The Writing Lab and The OWL. Purdue University.

• << Previous: Insiderness
• Next: Using Non-Textual Elements >>
• Last Updated: May 9, 2024 11:05 AM
• URL: https://libguides.usc.edu/writingguide

How to Write the Discussion Section of a Research Paper

The discussion section of a research paper analyzes and interprets the findings, provides context, compares them with previous studies, identifies limitations, and suggests future research directions.

Updated on September 15, 2023

Structure your discussion section right, and you’ll be cited more often while doing a greater service to the scientific community. So, what actually goes into the discussion section? And how do you write it?

The discussion section of your research paper is where you let the reader know how your study is positioned in the literature, what to take away from your paper, and how your work helps them. It can also include your conclusions and suggestions for future studies.

First, we’ll define all the parts of your discussion paper, and then look into how to write a strong, effective discussion section for your paper or manuscript.

Discussion section: what is it, what it does

The discussion section comes later in your paper, following the introduction, methods, and results. The discussion sets up your study’s conclusions. Its main goals are to present, interpret, and provide a context for your results.

What is it?

The discussion section provides an analysis and interpretation of the findings, compares them with previous studies, identifies limitations, and suggests future directions for research.

This section combines information from the preceding parts of your paper into a coherent story. By this point, the reader already knows why you did your study (introduction), how you did it (methods), and what happened (results). In the discussion, you’ll help the reader connect the ideas from these sections.

Why is it necessary?

The discussion provides context and interpretations for the results. It also answers the questions posed in the introduction. While the results section describes your findings, the discussion explains what they say. This is also where you can describe the impact or implications of your research.

Adds context for your results

Most research studies aim to answer a question, replicate a finding, or address limitations in the literature. These goals are first described in the introduction. However, in the discussion section, the author can refer back to them to explain how the study's objective was achieved. 

Shows what your results actually mean and real-world implications

The discussion can also describe the effect of your findings on research or practice. How are your results significant for readers, other researchers, or policymakers?

What to include in your discussion (in the correct order)

A complete and effective discussion section should at least touch on the points described below.

Summary of key findings

The discussion should begin with a brief factual summary of the results. Concisely overview the main results you obtained.

Begin with key findings with supporting evidence

Your results section described a list of findings, but what message do they send when you look at them all together?

Your findings were detailed in the results section, so there’s no need to repeat them here, but do provide at least a few highlights. This will help refresh the reader’s memory and help them focus on the big picture.

Read the first paragraph of the discussion section in this article (PDF) for an example of how to start this part of your paper. Notice how the authors break down their results and follow each description sentence with an explanation of why each finding is relevant. 

State clearly and concisely

Following a clear and direct writing style is especially important in the discussion section. After all, this is where you will make some of the most impactful points in your paper. While the results section often contains technical vocabulary, such as statistical terms, the discussion section lets you describe your findings more clearly. 

Interpretation of results

Once you’ve given your reader an overview of your results, you need to interpret those results. In other words, what do your results mean? Discuss the findings’ implications and significance in relation to your research question or hypothesis.

Analyze and interpret your findings

Look into your findings and explore what’s behind them or what may have caused them. If your introduction cited theories or studies that could explain your findings, use these sources as a basis to discuss your results.

For example, look at the second paragraph in the discussion section of this article on waggling honey bees. Here, the authors explore their results based on information from the literature.

Unexpected or contradictory results

Sometimes, your findings are not what you expect. Here’s where you describe this and try to find a reason for it. Could it be because of the method you used? Does it have something to do with the variables analyzed? Comparing your methods with those of other similar studies can help with this task.

Context and comparison with previous work

Refer to related studies to place your research in a larger context and the literature. Compare and contrast your findings with existing literature, highlighting similarities, differences, and/or contradictions.

How your work compares or contrasts with previous work

Studies with similar findings to yours can be cited to show the strength of your findings. Information from these studies can also be used to help explain your results. Differences between your findings and others in the literature can also be discussed here. 

How to divide this section into subsections

If you have more than one objective in your study or many key findings, you can dedicate a separate section to each of these. Here’s an example of this approach. You can see that the discussion section is divided into topics and even has a separate heading for each of them. 

Limitations

Many journals require you to include the limitations of your study in the discussion. Even if they don’t, there are good reasons to mention these in your paper.

Why limitations don’t have a negative connotation

A study’s limitations are points to be improved upon in future research. While some of these may be flaws in your method, many may be due to factors you couldn’t predict.

Examples include time constraints or small sample sizes. Pointing this out will help future researchers avoid or address these issues. This part of the discussion can also include any attempts you have made to reduce the impact of these limitations, as in this study .

How limitations add to a researcher's credibility

Pointing out the limitations of your study demonstrates transparency. It also shows that you know your methods well and can conduct a critical assessment of them.  

Implications and significance

The final paragraph of the discussion section should contain the take-home messages for your study. It can also cite the “strong points” of your study, to contrast with the limitations section.

Restate your hypothesis

Remind the reader what your hypothesis was before you conducted the study. 

How was it proven or disproven?

Identify your main findings and describe how they relate to your hypothesis.

How your results contribute to the literature

Were you able to answer your research question? Or address a gap in the literature?

Future implications of your research

Describe the impact that your results may have on the topic of study. Your results may show, for instance, that there are still limitations in the literature for future studies to address. There may be a need for studies that extend your findings in a specific way. You also may need additional research to corroborate your findings. 

Sample discussion section

This fictitious example covers all the aspects discussed above. Your actual discussion section will probably be much longer, but you can read this to get an idea of everything your discussion should cover.

Our results showed that the presence of cats in a household is associated with higher levels of perceived happiness by its human occupants. These findings support our hypothesis and demonstrate the association between pet ownership and well-being. 

The present findings align with those of Bao and Schreer (2016) and Hardie et al. (2023), who observed greater life satisfaction in pet owners relative to non-owners. Although the present study did not directly evaluate life satisfaction, this factor may explain the association between happiness and cat ownership observed in our sample.

Our findings must be interpreted in light of some limitations, such as the focus on cat ownership only rather than pets as a whole. This may limit the generalizability of our results.

Nevertheless, this study had several strengths. These include its strict exclusion criteria and use of a standardized assessment instrument to investigate the relationships between pets and owners. These attributes bolster the accuracy of our results and reduce the influence of confounding factors, increasing the strength of our conclusions. Future studies may examine the factors that mediate the association between pet ownership and happiness to better comprehend this phenomenon.

This brief discussion begins with a quick summary of the results and hypothesis. The next paragraph cites previous research and compares its findings to those of this study. Information from previous studies is also used to help interpret the findings. After discussing the results of the study, some limitations are pointed out. The paper also explains why these limitations may influence the interpretation of results. Then, final conclusions are drawn based on the study, and directions for future research are suggested.

How to make your discussion flow naturally

If you find writing in scientific English challenging, the discussion and conclusions are often the hardest parts of the paper to write. That’s because you’re not just listing up studies, methods, and outcomes. You’re actually expressing your thoughts and interpretations in words.

• How formal should it be?
• What words should you use, or not use?
• How do you meet strict word limits, or make it longer and more informative?

Always give it your best, but sometimes a helping hand can, well, help. Getting a professional edit can help clarify your work’s importance while improving the English used to explain it. When readers know the value of your work, they’ll cite it. We’ll assign your study to an expert editor knowledgeable in your area of research. Their work will clarify your discussion, helping it to tell your story. Find out more about AJE Editing.

Adam Goulston, PsyD, MS, MBA, MISD, ELS

Science Marketing Consultant

See our "Privacy Policy"

Ensure your structure and ideas are consistent and clearly communicated

Pair your Premium Editing with our add-on service Presubmission Review for an overall assessment of your manuscript.

• SpringerLink shop

Discussion and Conclusions

Your Discussion and Conclusions sections should answer the question: What do your results mean?

In other words, the majority of the Discussion and Conclusions sections should be an interpretation of your results. You should:

• Discuss your conclusions in order of  most to least important.
• Compare  your results with those from other studies: Are they consistent? If not, discuss possible reasons for the difference.
• Mention any  inconclusive results  and explain them as best you can. You may suggest additional experiments needed to clarify your results.
• Briefly describe the  limitations  of your study to show reviewers and readers that you have considered your experiment’s weaknesses. Many researchers are hesitant to do this as they feel it highlights the weaknesses in their research to the editor and reviewer. However doing this actually makes a positive impression of your paper as it makes it clear that you have an in depth understanding of your topic and can think objectively of your research.
• Discuss  what your results may mean  for researchers in the same field as you, researchers in other fields, and the general public. How could your findings be applied?
• State how your results  extend the findings  of previous studies.
• If your findings are preliminary, suggest  future studies  that need to be carried out.
• At the end of your Discussion and Conclusions sections,  state your main conclusions once again .

Back │ Next

• Langson Library
• Science Library
• Grunigen Medical Library
• Law Library
• Connect From Off-Campus
• Accessibility
• Gateway Study Center

Email this link

Writing a scientific paper.

• Writing a lab report
• INTRODUCTION

Writing a "good" discussion section

"discussion and conclusions checklist" from: how to write a good scientific paper. chris a. mack. spie. 2018., peer review.

• LITERATURE CITED
• Bibliography of guides to scientific writing and presenting
• Presentations
• Lab Report Writing Guides on the Web

This is is usually the hardest section to write. You are trying to bring out the true meaning of your data without being too long. Do not use words to conceal your facts or reasoning. Also do not repeat your results, this is a discussion.

• Present principles, relationships and generalizations shown by the results
• Point out exceptions or lack of correlations. Define why you think this is so.
• Show how your results agree or disagree with previously published works
• Discuss the theoretical implications of your work as well as practical applications
• State your conclusions clearly. Summarize your evidence for each conclusion.
• Discuss the significance of the results
•  Evidence does not explain itself; the results must be presented and then explained.
• Typical stages in the discussion: summarizing the results, discussing whether results are expected or unexpected, comparing these results to previous work, interpreting and explaining the results (often by comparison to a theory or model), and hypothesizing about their generality.
• Discuss any problems or shortcomings encountered during the course of the work.
• Discuss possible alternate explanations for the results.
• Avoid: presenting results that are never discussed; presenting discussion that does not relate to any of the results; presenting results and discussion in chronological order rather than logical order; ignoring results that do not support the conclusions; drawing conclusions from results without logical arguments to back them up. 

CONCLUSIONS

• Provide a very brief summary of the Results and Discussion.
• Emphasize the implications of the findings, explaining how the work is significant and providing the key message(s) the author wishes to convey.
• Provide the most general claims that can be supported by the evidence.
• Provide a future perspective on the work.
• Avoid: repeating the abstract; repeating background information from the Introduction; introducing new evidence or new arguments not found in the Results and Discussion; repeating the arguments made in the Results and Discussion; failing to address all of the research questions set out in the Introduction. 

WHAT HAPPENS AFTER I COMPLETE MY PAPER?

 The peer review process is the quality control step in the publication of ideas.  Papers that are submitted to a journal for publication are sent out to several scientists (peers) who look carefully at the paper to see if it is "good science".  These reviewers then recommend to the editor of a journal whether or not a paper should be published. Most journals have publication guidelines. Ask for them and follow them exactly.    Peer reviewers examine the soundness of the materials and methods section.  Are the materials and methods used written clearly enough for another scientist to reproduce the experiment?  Other areas they look at are: originality of research, significance of research question studied, soundness of the discussion and interpretation, correct spelling and use of technical terms, and length of the article.

• << Previous: RESULTS
• Next: LITERATURE CITED >>
• Last Updated: Aug 4, 2023 9:33 AM
• URL: https://guides.lib.uci.edu/scientificwriting

Off-campus? Please use the Software VPN and choose the group UCIFull to access licensed content. For more information, please Click here

Software VPN is not available for guests, so they may not have access to some content when connecting from off-campus.

Organizing Academic Research Papers: 8. The Discussion

• Purpose of Guide
• Design Flaws to Avoid
• Glossary of Research Terms
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Executive Summary
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tertiary Sources
• What Is Scholarly vs. Popular?
• Qualitative Methods
• Quantitative Methods
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Annotated Bibliography
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• How to Manage Group Projects
• Multiple Book Review Essay
• Reviewing Collected Essays
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Research Proposal
• Acknowledgements

The purpose of the discussion is to interpret and describe the significance of your findings in light of what was already known about the research problem being investigated, and to explain any new understanding or fresh insights about the problem after you've taken the findings into consideration. The discussion will always connect to the introduction by way of the research questions or hypotheses you posed and the literature you reviewed, but it does not simply repeat or rearrange the introduction; the discussion should always explain how your study has moved the reader's understanding of the research problem forward from where you left them at the end of the introduction.

Importance of a Good Discussion

This section is often considered the most important part of a research paper because it most effectively demonstrates your ability as a researcher to think critically about an issue, to develop creative solutions to problems based on the findings, and to formulate a deeper, more profound understanding of the research problem you are studying.

The discussion section is where you explore the underlying meaning of your research , its possible implications in other areas of study, and the possible improvements that can be made in order to further develop the concerns of your research.

This is the section where you need to present the importance of your study and how it may be able to contribute to and/or fill existing gaps in the field. If appropriate, the discussion section is also where you state how the findings from your study revealed new gaps in the literature that had not been previously exposed or adequately described.

This part of the paper is not strictly governed by objective reporting of information but, rather, it is where you can engage in creative thinking about issues through evidence-based interpretation of findings. This is where you infuse your results with meaning.

Kretchmer, Paul. Fourteen Steps to Writing to Writing an Effective Discussion Section . San Francisco Edit, 2003-2008.

Structure and Writing Style

I.  General Rules

These are the general rules you should adopt when composing your discussion of the results :

• Do not be verbose or repetitive.
• Be concise and make your points clearly.
• Avoid using jargon.
• Follow a logical stream of thought.
• Use the present verb tense, especially for established facts; however, refer to specific works and references in the past tense.
• If needed, use subheadings to help organize your presentation or to group your interpretations into themes.

II.  The Content

The content of the discussion section of your paper most often includes :

• Explanation of results : comment on whether or not the results were expected and present explanations for the results; go into greater depth when explaining findings that were unexpected or especially profound. If appropriate, note any unusual or unanticipated patterns or trends that emerged from your results and explain their meaning.
• References to previous research : compare your results with the findings from other studies, or use the studies to support a claim. This can include re-visiting key sources already cited in your literature review section, or, save them to cite later in the discussion section if they are more important to compare with your results than being part of the general research you cited to provide context and background information.
• Deduction : a claim for how the results can be applied more generally. For example, describing lessons learned, proposing recommendations that can help improve a situation, or recommending best practices.
• Hypothesis : a more general claim or possible conclusion arising from the results [which may be proved or disproved in subsequent research].

III. Organization and Structure

Keep the following sequential points in mind as you organize and write the discussion section of your paper:

• Think of your discussion as an inverted pyramid. Organize the discussion from the general to the specific, linking your findings to the literature, then to theory, then to practice [if appropriate].
• Use the same key terms, mode of narration, and verb tense [present] that you used when when describing the research problem in the introduction.
• Begin by briefly re-stating the research problem you were investigating and answer all of the research questions underpinning the problem that you posed in the introduction.
• Describe the patterns, principles, and relationships shown by each major findings and place them in proper perspective. The sequencing of providing this information is important; first state the answer, then the relevant results, then cite the work of others. If appropriate, refer the reader to a figure or table to help enhance the interpretation of the data. The order of interpreting each major finding should be in the same order as they were described in your results section.
• A good discussion section includes analysis of any unexpected findings. This paragraph should begin with a description of the unexpected finding, followed by a brief interpretation as to why you believe it appeared and, if necessary, its possible significance in relation to the overall study. If more than one unexpected finding emerged during the study, describe each them in the order they appeared as you gathered the data.
• Before concluding the discussion, identify potential limitations and weaknesses. Comment on their relative importance in relation to your overall interpretation of the results and, if necessary, note how they may affect the validity of the findings. Avoid using an apologetic tone; however, be honest and self-critical.
• The discussion section should end with a concise summary of the principal implications of the findings regardless of statistical significance. Give a brief explanation about why you believe the findings and conclusions of your study are important and how they support broader knowledge or understanding of the research problem. This can be followed by any recommendations for further research. However, do not offer recommendations which could have been easily addressed within the study. This demonstrates to the reader you have inadequately examined and interpreted the data.

IV.  Overall Objectives

The objectives of your discussion section should include the following: I.  Reiterate the Research Problem/State the Major Findings

Briefly reiterate for your readers the research problem or problems you are investigating and the methods you used to investigate them, then move quickly to describe the major findings of the study. You should write a direct, declarative, and succinct proclamation of the study results.

II.  Explain the Meaning of the Findings and Why They are Important

No one has thought as long and hard about your study as you have. Systematically explain the meaning of the findings and why you believe they are important. After reading the discussion section, you want the reader to think about the results [“why hadn’t I thought of that?”]. You don’t want to force the reader to go through the paper multiple times to figure out what it all means. Begin this part of the section by repeating what you consider to be your most important finding first.

III.  Relate the Findings to Similar Studies

No study is so novel or possesses such a restricted focus that it has absolutely no relation to other previously published research. The discussion section should relate your study findings to those of other studies, particularly if questions raised by previous studies served as the motivation for your study, the findings of other studies support your findings [which strengthens the importance of your study results], and/or they point out how your study differs from other similar studies. IV.  Consider Alternative Explanations of the Findings

It is important to remember that the purpose of research is to discover and not to prove . When writing the discussion section, you should carefully consider all possible explanations for the study results, rather than just those that fit your prior assumptions or biases.

V.  Acknowledge the Study’s Limitations

It is far better for you to identify and acknowledge your study’s limitations than to have them pointed out by your professor! Describe the generalizability of your results to other situations, if applicable to the method chosen, then describe in detail problems you encountered in the method(s) you used to gather information. Note any unanswered questions or issues your study did not address, and.... VI.  Make Suggestions for Further Research

Although your study may offer important insights about the research problem, other questions related to the problem likely remain unanswered. Moreover, some unanswered questions may have become more focused because of your study. You should make suggestions for further research in the discussion section.

NOTE: Besides the literature review section, the preponderance of references to sources in your research paper are usually found in the discussion section . A few historical references may be helpful for perspective but most of the references should be relatively recent and included to aid in the interpretation of your results and/or linked to similar studies. If a study that you cited disagrees with your findings, don't ignore it--clearly explain why the study's findings differ from yours.

V.  Problems to Avoid

• Do not waste entire sentences restating your results . Should you need to remind the reader of the finding to be discussed, use "bridge sentences" that relate the result to the interpretation. An example would be: “The lack of available housing to single women with children in rural areas of Texas suggests that...[then move to the interpretation of this finding].”
• Recommendations for further research can be included in either the discussion or conclusion of your paper but do not repeat your recommendations in the both sections.
• Do not introduce new results in the discussion. Be wary of mistaking the reiteration of a specific finding for an interpretation.
• Use of the first person is acceptable, but too much use of the first person may actually distract the reader from the main points.

Analyzing vs. Summarizing. Department of English Writing Guide. George Mason University; Discussion . The Structure, Format, Content, and Style of a Journal-Style Scientific Paper. Department of Biology. Bates College; Hess, Dean R. How to Write an Effective Discussion. Respiratory Care 49 (October 2004); Kretchmer, Paul. Fourteen Steps to Writing to Writing an Effective Discussion Section . San Francisco Edit, 2003-2008; The Lab Report . University College Writing Centre. University of Toronto; Summary: Using it Wisely . The Writing Center. University of North Carolina; Schafer, Mickey S. Writing the Discussion . Writing in Psychology course syllabus. University of Florida; Yellin, Linda L. A Sociology Writer's Guide. Boston, MA: Allyn and Bacon, 2009.

Writing Tip

Don’t Overinterpret the Results!

Interpretation is a subjective exercise. Therefore, be careful that you do not read more into the findings than can be supported by the evidence you've gathered. Remember that the data are the data: nothing more, nothing less.

Another Writing Tip

Don't Write Two Results Sections!

One of the most common mistakes that you can make when discussing the results of your study is to present a superficial interpretation of the findings that more or less re-states the results section of your paper. Obviously, you must refer to your results when discussing them, but focus on the interpretion of those results, not just the data itself.

Azar, Beth. Discussing Your Findings.  American Psychological Association gradPSYCH Magazine (January 2006)

Yet Another Writing Tip

Avoid Unwarranted Speculation!

The discussion section should remain focused on the findings of your study. For example, if you studied the impact of foreign aid on increasing levels of education among the poor in Bangladesh, it's generally not appropriate to speculate about how your findings might apply to populations in other countries without drawing from existing studies to support your claim. If you feel compelled to speculate, be certain that you clearly identify your comments as speculation or as a suggestion for where further research is needed. Sometimes your professor will encourage you to expand the discussion in this way, while others don’t care what your opinion is beyond your efforts to interpret the data.

• << Previous: Using Non-Textual Elements
• Next: Limitations of the Study >>
• Last Updated: Jul 18, 2023 11:58 AM
• URL: https://library.sacredheart.edu/c.php?g=29803
• QuickSearch
• Library Catalog
• Databases A-Z
• Publication Finder
• Course Reserves
• Citation Linker
• Digital Commons
• Our Website

Research Support

• Ask a Librarian
• Appointments
• Interlibrary Loan (ILL)
• Research Guides
• Databases by Subject
• Citation Help

Using the Library

• Reserve a Group Study Room
• Renew Books
• Honors Study Rooms
• Off-Campus Access
• Library Policies
• Library Technology

User Information

• Grad Students
• Online Students
• COVID-19 Updates
• Staff Directory
• News & Announcements
• Library Newsletter

My Accounts

• Interlibrary Loan
• Staff Site Login

FIND US ON  

• Manuscript Preparation

How to write the results section of a research paper

• 3 minute read
• 63.4K views

Table of Contents

At its core, a research paper aims to fill a gap in the research on a given topic. As a result, the results section of the paper, which describes the key findings of the study, is often considered the core of the paper. This is the section that gets the most attention from reviewers, peers, students, and any news organization reporting on your findings. Writing a clear, concise, and logical results section is, therefore, one of the most important parts of preparing your manuscript.

Difference between results and discussion

Before delving into how to write the results section, it is important to first understand the difference between the results and discussion sections. The results section needs to detail the findings of the study. The aim of this section is not to draw connections between the different findings or to compare it to previous findings in literature—that is the purview of the discussion section. Unlike the discussion section, which can touch upon the hypothetical, the results section needs to focus on the purely factual. In some cases, it may even be preferable to club these two sections together into a single section. For example, while writing  a review article, it can be worthwhile to club these two sections together, as the main results in this case are the conclusions that can be drawn from the literature.

Structure of the results section

Although the main purpose of the results section in a research paper is to report the findings, it is necessary to present an introduction and repeat the research question. This establishes a connection to the previous section of the paper and creates a smooth flow of information.

Next, the results section needs to communicate the findings of your research in a systematic manner. The section needs to be organized such that the primary research question is addressed first, then the secondary research questions. If the research addresses multiple questions, the results section must individually connect with each of the questions. This ensures clarity and minimizes confusion while reading.

Consider representing your results visually. For example, graphs, tables, and other figures can help illustrate the findings of your paper, especially if there is a large amount of data in the results.

Remember, an appealing results section can help peer reviewers better understand the merits of your research, thereby increasing your chances of publication.

Practical guidance for writing an effective results section for a research paper

• Always use simple and clear language. Avoid the use of uncertain or out-of-focus expressions.
• The findings of the study must be expressed in an objective and unbiased manner. While it is acceptable to correlate certain findings in the discussion section, it is best to avoid overinterpreting the results.
• If the research addresses more than one hypothesis, use sub-sections to describe the results. This prevents confusion and promotes understanding.
• Ensure that negative results are included in this section, even if they do not support the research hypothesis.
• Wherever possible, use illustrations like tables, figures, charts, or other visual representations to showcase the results of your research paper. Mention these illustrations in the text, but do not repeat the information that they convey.
• For statistical data, it is adequate to highlight the tests and explain their results. The initial or raw data should not be mentioned in the results section of a research paper.

The results section of a research paper is usually the most impactful section because it draws the greatest attention. Regardless of the subject of your research paper, a well-written results section is capable of generating interest in your research.

For detailed information and assistance on writing the results of a research paper, refer to Elsevier Author Services.

• Research Process

Writing a good review article

Why is data validation important in research?

You may also like.

Make Hook, Line, and Sinker: The Art of Crafting Engaging Introductions

Can Describing Study Limitations Improve the Quality of Your Paper?

A Guide to Crafting Shorter, Impactful Sentences in Academic Writing

6 Steps to Write an Excellent Discussion in Your Manuscript

How to Write Clear and Crisp Civil Engineering Papers? Here are 5 Key Tips to Consider

The Clear Path to An Impactful Paper: ②

The Essentials of Writing to Communicate Research in Medicine

Changing Lines: Sentence Patterns in Academic Writing

Input your search keywords and press Enter.

• Walden University
• Faculty Portal

General Research Paper Guidelines: Discussion

Discussion section.

The overall purpose of a research paper’s discussion section is to evaluate and interpret results, while explaining both the implications and limitations of your findings. Per APA (2020) guidelines, this section requires you to “examine, interpret, and qualify the results and draw inferences and conclusions from them” (p. 89). Discussion sections also require you to detail any new insights, think through areas for future research, highlight the work that still needs to be done to further your topic, and provide a clear conclusion to your research paper. In a good discussion section, you should do the following:

• Clearly connect the discussion of your results to your introduction, including your central argument, thesis, or problem statement.
• Provide readers with a critical thinking through of your results, answering the “so what?” question about each of your findings. In other words, why is this finding important?
• Detail how your research findings might address critical gaps or problems in your field
• Compare your results to similar studies’ findings
• Provide the possibility of alternative interpretations, as your goal as a researcher is to “discover” and “examine” and not to “prove” or “disprove.” Instead of trying to fit your results into your hypothesis, critically engage with alternative interpretations to your results.

For more specific details on your Discussion section, be sure to review Sections 3.8 (pp. 89-90) and 3.16 (pp. 103-104) of your 7 th edition APA manual

*Box content adapted from:

University of Southern California (n.d.). Organizing your social sciences research paper: 8 the discussion . https://libguides.usc.edu/writingguide/discussion

Limitations

Limitations of generalizability or utility of findings, often over which the researcher has no control, should be detailed in your Discussion section. Including limitations for your reader allows you to demonstrate you have thought critically about your given topic, understood relevant literature addressing your topic, and chosen the methodology most appropriate for your research. It also allows you an opportunity to suggest avenues for future research on your topic. An effective limitations section will include the following:

• Detail (a) sources of potential bias, (b) possible imprecision of measures, (c) other limitations or weaknesses of the study, including any methodological or researcher limitations.
• Sample size: In quantitative research, if a sample size is too small, it is more difficult to generalize results.
• Lack of available/reliable data : In some cases, data might not be available or reliable, which will ultimately affect the overall scope of your research. Use this as an opportunity to explain areas for future study.
• Lack of prior research on your study topic: In some cases, you might find that there is very little or no similar research on your study topic, which hinders the credibility and scope of your own research. If this is the case, use this limitation as an opportunity to call for future research. However, make sure you have done a thorough search of the available literature before making this claim.
• Flaws in measurement of data: Hindsight is 20/20, and you might realize after you have completed your research that the data tool you used actually limited the scope or results of your study in some way. Again, acknowledge the weakness and use it as an opportunity to highlight areas for future study.
• Limits of self-reported data: In your research, you are assuming that any participants will be honest and forthcoming with responses or information they provide to you. Simply acknowledging this assumption as a possible limitation is important in your research.
• Access: Most research requires that you have access to people, documents, organizations, etc.. However, for various reasons, access is sometimes limited or denied altogether. If this is the case, you will want to acknowledge access as a limitation to your research.
• Time: Choosing a research focus that is narrow enough in scope to finish in a given time period is important. If such limitations of time prevent you from certain forms of research, access, or study designs, acknowledging this time restraint is important. Acknowledging such limitations is important, as they can point other researchers to areas that require future study.
• Potential Bias: All researchers have some biases, so when reading and revising your draft, pay special attention to the possibilities for bias in your own work. Such bias could be in the form you organized people, places, participants, or events. They might also exist in the method you selected or the interpretation of your results. Acknowledging such bias is an important part of the research process.
• Language Fluency: On occasion, researchers or research participants might have language fluency issues, which could potentially hinder results or how effectively you interpret results. If this is an issue in your research, make sure to acknowledge it in your limitations section.

University of Southern California (n.d.). Organizing your social sciences research paper: Limitations of the study . https://libguides.usc.edu/writingguide/limitations

In many research papers, the conclusion, like the limitations section, is folded into the larger discussion section. If you are unsure whether to include the conclusion as part of your discussion or as a separate section, be sure to defer to the assignment instructions or ask your instructor.

The conclusion is important, as it is specifically designed to highlight your research’s larger importance outside of the specific results of your study. Your conclusion section allows you to reiterate the main findings of your study, highlight their importance, and point out areas for future research. Based on the scope of your paper, your conclusion could be anywhere from one to three paragraphs long. An effective conclusion section should include the following:

• Describe the possibilities for continued research on your topic, including what might be improved, adapted, or added to ensure useful and informed future research.
• Provide a detailed account of the importance of your findings
• Reiterate why your problem is important, detail how your interpretation of results impacts the subfield of study, and what larger issues both within and outside of your field might be affected from such results

University of Southern California (n.d.). Organizing your social sciences research paper: 9. the conclusion . https://libguides.usc.edu/writingguide/conclusion

• Previous Page: Results
• Next Page: References
• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Research and Doctoral Services
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

• Privacy Policy

Home » Research Results Section – Writing Guide and Examples

Research Results Section – Writing Guide and Examples

Table of Contents

Research Results

Research results refer to the findings and conclusions derived from a systematic investigation or study conducted to answer a specific question or hypothesis. These results are typically presented in a written report or paper and can include various forms of data such as numerical data, qualitative data, statistics, charts, graphs, and visual aids.

Results Section in Research

The results section of the research paper presents the findings of the study. It is the part of the paper where the researcher reports the data collected during the study and analyzes it to draw conclusions.

In the results section, the researcher should describe the data that was collected, the statistical analysis performed, and the findings of the study. It is important to be objective and not interpret the data in this section. Instead, the researcher should report the data as accurately and objectively as possible.

Structure of Research Results Section

The structure of the research results section can vary depending on the type of research conducted, but in general, it should contain the following components:

• Introduction: The introduction should provide an overview of the study, its aims, and its research questions. It should also briefly explain the methodology used to conduct the study.
• Data presentation : This section presents the data collected during the study. It may include tables, graphs, or other visual aids to help readers better understand the data. The data presented should be organized in a logical and coherent way, with headings and subheadings used to help guide the reader.
• Data analysis: In this section, the data presented in the previous section are analyzed and interpreted. The statistical tests used to analyze the data should be clearly explained, and the results of the tests should be presented in a way that is easy to understand.
• Discussion of results : This section should provide an interpretation of the results of the study, including a discussion of any unexpected findings. The discussion should also address the study’s research questions and explain how the results contribute to the field of study.
• Limitations: This section should acknowledge any limitations of the study, such as sample size, data collection methods, or other factors that may have influenced the results.
• Conclusions: The conclusions should summarize the main findings of the study and provide a final interpretation of the results. The conclusions should also address the study’s research questions and explain how the results contribute to the field of study.
• Recommendations : This section may provide recommendations for future research based on the study’s findings. It may also suggest practical applications for the study’s results in real-world settings.

Outline of Research Results Section

The following is an outline of the key components typically included in the Results section:

I. Introduction

• A brief overview of the research objectives and hypotheses
• A statement of the research question

II. Descriptive statistics

• Summary statistics (e.g., mean, standard deviation) for each variable analyzed
• Frequencies and percentages for categorical variables

III. Inferential statistics

• Results of statistical analyses, including tests of hypotheses
• Tables or figures to display statistical results

IV. Effect sizes and confidence intervals

• Effect sizes (e.g., Cohen’s d, odds ratio) to quantify the strength of the relationship between variables
• Confidence intervals to estimate the range of plausible values for the effect size

V. Subgroup analyses

• Results of analyses that examined differences between subgroups (e.g., by gender, age, treatment group)

VI. Limitations and assumptions

• Discussion of any limitations of the study and potential sources of bias
• Assumptions made in the statistical analyses

VII. Conclusions

• A summary of the key findings and their implications
• A statement of whether the hypotheses were supported or not
• Suggestions for future research

Example of Research Results Section

An Example of a Research Results Section could be:

• This study sought to examine the relationship between sleep quality and academic performance in college students.
• Hypothesis : College students who report better sleep quality will have higher GPAs than those who report poor sleep quality.
• Methodology : Participants completed a survey about their sleep habits and academic performance.

II. Participants

• Participants were college students (N=200) from a mid-sized public university in the United States.
• The sample was evenly split by gender (50% female, 50% male) and predominantly white (85%).
• Participants were recruited through flyers and online advertisements.

III. Results

• Participants who reported better sleep quality had significantly higher GPAs (M=3.5, SD=0.5) than those who reported poor sleep quality (M=2.9, SD=0.6).
• See Table 1 for a summary of the results.
• Participants who reported consistent sleep schedules had higher GPAs than those with irregular sleep schedules.

IV. Discussion

• The results support the hypothesis that better sleep quality is associated with higher academic performance in college students.
• These findings have implications for college students, as prioritizing sleep could lead to better academic outcomes.
• Limitations of the study include self-reported data and the lack of control for other variables that could impact academic performance.

V. Conclusion

• College students who prioritize sleep may see a positive impact on their academic performance.
• These findings highlight the importance of sleep in academic success.
• Future research could explore interventions to improve sleep quality in college students.

Example of Research Results in Research Paper :

Our study aimed to compare the performance of three different machine learning algorithms (Random Forest, Support Vector Machine, and Neural Network) in predicting customer churn in a telecommunications company. We collected a dataset of 10,000 customer records, with 20 predictor variables and a binary churn outcome variable.

Our analysis revealed that all three algorithms performed well in predicting customer churn, with an overall accuracy of 85%. However, the Random Forest algorithm showed the highest accuracy (88%), followed by the Support Vector Machine (86%) and the Neural Network (84%).

Furthermore, we found that the most important predictor variables for customer churn were monthly charges, contract type, and tenure. Random Forest identified monthly charges as the most important variable, while Support Vector Machine and Neural Network identified contract type as the most important.

Overall, our results suggest that machine learning algorithms can be effective in predicting customer churn in a telecommunications company, and that Random Forest is the most accurate algorithm for this task.

Example 3 :

Title : The Impact of Social Media on Body Image and Self-Esteem

Abstract : This study aimed to investigate the relationship between social media use, body image, and self-esteem among young adults. A total of 200 participants were recruited from a university and completed self-report measures of social media use, body image satisfaction, and self-esteem.

Results: The results showed that social media use was significantly associated with body image dissatisfaction and lower self-esteem. Specifically, participants who reported spending more time on social media platforms had lower levels of body image satisfaction and self-esteem compared to those who reported less social media use. Moreover, the study found that comparing oneself to others on social media was a significant predictor of body image dissatisfaction and lower self-esteem.

Conclusion : These results suggest that social media use can have negative effects on body image satisfaction and self-esteem among young adults. It is important for individuals to be mindful of their social media use and to recognize the potential negative impact it can have on their mental health. Furthermore, interventions aimed at promoting positive body image and self-esteem should take into account the role of social media in shaping these attitudes and behaviors.

Importance of Research Results

Research results are important for several reasons, including:

• Advancing knowledge: Research results can contribute to the advancement of knowledge in a particular field, whether it be in science, technology, medicine, social sciences, or humanities.
• Developing theories: Research results can help to develop or modify existing theories and create new ones.
• Improving practices: Research results can inform and improve practices in various fields, such as education, healthcare, business, and public policy.
• Identifying problems and solutions: Research results can identify problems and provide solutions to complex issues in society, including issues related to health, environment, social justice, and economics.
• Validating claims : Research results can validate or refute claims made by individuals or groups in society, such as politicians, corporations, or activists.
• Providing evidence: Research results can provide evidence to support decision-making, policy-making, and resource allocation in various fields.

How to Write Results in A Research Paper

Here are some general guidelines on how to write results in a research paper:

• Organize the results section: Start by organizing the results section in a logical and coherent manner. Divide the section into subsections if necessary, based on the research questions or hypotheses.
• Present the findings: Present the findings in a clear and concise manner. Use tables, graphs, and figures to illustrate the data and make the presentation more engaging.
• Describe the data: Describe the data in detail, including the sample size, response rate, and any missing data. Provide relevant descriptive statistics such as means, standard deviations, and ranges.
• Interpret the findings: Interpret the findings in light of the research questions or hypotheses. Discuss the implications of the findings and the extent to which they support or contradict existing theories or previous research.
• Discuss the limitations : Discuss the limitations of the study, including any potential sources of bias or confounding factors that may have affected the results.
• Compare the results : Compare the results with those of previous studies or theoretical predictions. Discuss any similarities, differences, or inconsistencies.
• Avoid redundancy: Avoid repeating information that has already been presented in the introduction or methods sections. Instead, focus on presenting new and relevant information.
• Be objective: Be objective in presenting the results, avoiding any personal biases or interpretations.

When to Write Research Results

Here are situations When to Write Research Results”

• After conducting research on the chosen topic and obtaining relevant data, organize the findings in a structured format that accurately represents the information gathered.
• Once the data has been analyzed and interpreted, and conclusions have been drawn, begin the writing process.
• Before starting to write, ensure that the research results adhere to the guidelines and requirements of the intended audience, such as a scientific journal or academic conference.
• Begin by writing an abstract that briefly summarizes the research question, methodology, findings, and conclusions.
• Follow the abstract with an introduction that provides context for the research, explains its significance, and outlines the research question and objectives.
• The next section should be a literature review that provides an overview of existing research on the topic and highlights the gaps in knowledge that the current research seeks to address.
• The methodology section should provide a detailed explanation of the research design, including the sample size, data collection methods, and analytical techniques used.
• Present the research results in a clear and concise manner, using graphs, tables, and figures to illustrate the findings.
• Discuss the implications of the research results, including how they contribute to the existing body of knowledge on the topic and what further research is needed.
• Conclude the paper by summarizing the main findings, reiterating the significance of the research, and offering suggestions for future research.

Purpose of Research Results

The purposes of Research Results are as follows:

• Informing policy and practice: Research results can provide evidence-based information to inform policy decisions, such as in the fields of healthcare, education, and environmental regulation. They can also inform best practices in fields such as business, engineering, and social work.
• Addressing societal problems : Research results can be used to help address societal problems, such as reducing poverty, improving public health, and promoting social justice.
• Generating economic benefits : Research results can lead to the development of new products, services, and technologies that can create economic value and improve quality of life.
• Supporting academic and professional development : Research results can be used to support academic and professional development by providing opportunities for students, researchers, and practitioners to learn about new findings and methodologies in their field.
• Enhancing public understanding: Research results can help to educate the public about important issues and promote scientific literacy, leading to more informed decision-making and better public policy.
• Evaluating interventions: Research results can be used to evaluate the effectiveness of interventions, such as treatments, educational programs, and social policies. This can help to identify areas where improvements are needed and guide future interventions.
• Contributing to scientific progress: Research results can contribute to the advancement of science by providing new insights and discoveries that can lead to new theories, methods, and techniques.
• Informing decision-making : Research results can provide decision-makers with the information they need to make informed decisions. This can include decision-making at the individual, organizational, or governmental levels.
• Fostering collaboration : Research results can facilitate collaboration between researchers and practitioners, leading to new partnerships, interdisciplinary approaches, and innovative solutions to complex problems.

Advantages of Research Results

Some Advantages of Research Results are as follows:

• Improved decision-making: Research results can help inform decision-making in various fields, including medicine, business, and government. For example, research on the effectiveness of different treatments for a particular disease can help doctors make informed decisions about the best course of treatment for their patients.
• Innovation : Research results can lead to the development of new technologies, products, and services. For example, research on renewable energy sources can lead to the development of new and more efficient ways to harness renewable energy.
• Economic benefits: Research results can stimulate economic growth by providing new opportunities for businesses and entrepreneurs. For example, research on new materials or manufacturing techniques can lead to the development of new products and processes that can create new jobs and boost economic activity.
• Improved quality of life: Research results can contribute to improving the quality of life for individuals and society as a whole. For example, research on the causes of a particular disease can lead to the development of new treatments and cures, improving the health and well-being of millions of people.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

How to Cite Research Paper – All Formats and...

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...

Research Paper Format – Types, Examples and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

• Open access
• Published: 09 May 2024

A systematic review of telemedicine for neuromuscular diseases: components and determinants of practice

• Deniz Senyel 1 , 2 ,
• Katja Senn 1 ,
• James Boyd 2 &
• Klaus Nagels 1  

BMC Digital Health volume  2 , Article number:  17 ( 2024 ) Cite this article

78 Accesses

Metrics details

Introduction

Neuromuscular diseases (NMDs) entail a group of mostly inherited genetic disorders with heterogeneous phenotypes impacting muscles, the central or peripheral nervous system. They can lead to severe disabilities and shortened lifespans. Despite their severity, NMDs often lack in public awareness and appropriate medical and social support. Telemedicine can improve patients’ and caregivers’ lives by enhancing continuity of and access to care. The first aim of this systematic review was to summarise the status quo of telemedicine services for patients with NMDs. Secondly, barriers and facilitators of the respective implementation processes should be analysed.

The databases PubMed, Web of Science and CENTRAL by Cochrane were searched in May 2022. To be truly explorative, any original evidence from any setting was included. Two independent researchers completed the screening process. Data was extracted and analysed using the taxonomy of Bashshur et al. (2011) and the Consolidated Framework for Implementation Research (CFIR).

Fifty-seven original papers were included in the systematic review. The results showed a high representation of teleconsultations and remote monitoring studies. Teleconsultations replaced in person appointments and telemonitoring mostly focused on ventilation. Physical therapy, pulmonology, neurology, and psychology were the most represented medical specialties. We found barriers and facilitators relating to implementation mainly referred to the intervention and the individuals involved. Technical errors and inaccessibility due to a lack of technical devices or the patient’s disability were stated as hindrances. A positive mindset of users as well as patient empowerment were necessary for the adoption of new technology. Technophobia or uncertainty around technology negatively impacted the implementation process.

This systematic review provides an overview of the current use of telemedicine in patients with NMDs. The distribution of telemedicine interventions between the defined domains was very heterogenous. Previous research has neglected to fully describe the implementation process of telemedicine for NMDs.

The evidence shows that telemedicine can benefit patients with NMDs in a multitude of ways. Therefore, health policies should endorse and incentivise the uptake of telemedicine by institutions and health care workers. Further research needs to be conducted to confirm the current evidence and close existing research gaps.

Peer Review reports

Neuromuscular diseases (NMDs) are a heterogeneous group of disorders, that affect the nerves controlling muscles, leading to muscle weakness, wasting, and other related symptoms [ 1 ]. NMDs are often hereditary and have been linked to 500 different affected genes [ 2 , 3 ]. Most NMDs are classified as rare diseases. The prevalence of NMDs can vary widely and, even for common diagnostic groups, the prevalence ranges between 0.1 to 60 per 100,000 [ 4 ]. The onset, cause, and course of the disease vary widely between disorders [ 5 ]. While each individual's experience is unique, there are common disability-related challenges faced by patients with NMDs. Acknowledging these commonalities and addressing the unique needs of each person are essential for providing comprehensive care and support to individuals and their families living with NMDs. NMDs are highly complex diseases defined by a degenerative course and progressive muscle weakness as the main symptom. Their impact extends beyond the musculoskeletal system, affecting various organs and systems throughout the body, such as eyes, lungs or the brain [ 1 , 2 ]. As a result, patients suffer from a reduced quality of life and a significant disease burden [ 2 , 6 ]. Multidisciplinary care is often considered the optimal approach for providing holistic treatment and symptomatic management for individuals with NMDs [ 7 , 8 , 9 , 10 , 11 ]. The needs of patients during disease progression are ever changing based on disease stage, symptom burden, and personal priorities. General practitioners, specialists, and allied health professionals each bring unique expertise to the care team, allowing for comprehensive, patient-centred care that adapts to changing needs and priorities throughout the course of the disease and ensures continuity and quality of care [ 1 , 12 , 13 ]. Recognising and supporting caregivers is crucial in the care of NMD patients. Most NMD patients receive informal care, often provided by their partner or family members. The caregiver burden increases with the progression of the patient’s disease. In severe cases, it can lead to psychological distress and burnout, a state of physical and emotional exhaustion [ 14 , 15 , 16 , 17 ].

Mobile health apps, teleconsultation and telemonitoring have been proven to be useful tools in the management and treatment of chronic diseases such as diabetes, heart failure, asthma, chronic obstructive pulmonary disease, and cancer. They have the potential to increase treatment adherence, support self-management, and promote continuity of care [ 18 , 19 , 20 ]. They have the potential to reduce hospital admissions, decrease mortality rates, and lessen health services usage [ 21 , 22 , 23 , 24 ]. The research focus in telemedicine for NMDs varies between disorders. A recent systematic review by Helleman et al. showed telemedicine for ALS patients to be a useful option for remote monitoring, consultations, and follow-ups [ 25 ]. From a patient’s perspective it can be time- and cost-saving while reducing stress and fatigue. While telemedicine has demonstrated its value in certain NMDs like ALS, its usage in the care of other NMDs have not been as extensively studied or described.

This systematic review aims to identify telemedicine interventions for patients with NMDs and analyse the barriers and facilitators of the implementation process associated with telemedicine for NMD patients. The taxonomy by Bashshur et al. will be used to standardise terminology and make it easier to categorise and study the various telehealth interventions and services [ 26 ]. The term “Telemedicine” will be used as an umbrella term to encompass a broad range of remote healthcare services and technologies. This is done to avoid the potential ambiguities and unclarities that can arise from newer terms like "e-health" or "telehealth". This review will provide an overview of the status quo and will offer recommendations for future innovations.

This systematic review followed the PRISMA [ 27 ] checklist. The study protocol was registered on PROSPERO (ID: CRD42022325481).

Databases and search strategy

For the literature search PubMed, Web of Science, and the Cochrane database CENTRAL were used as sources. If full text could not be found, the authors were contacted. The final search was conducted in May of 2022.

The search strategy consisted of two major themes: Firstly, synonyms for NMDs and secondly, synonyms and subcategories for telemedicine. The full search strings can be found in the supplementary file 1 .

Study selection

The study selection was conducted by two reviewers KS and DS. The following inclusion criteria were applied: Studies from any country with any healthcare and insurance system were eligible to maximise the diversity and inclusivity of the evidence base. No restrictions regarding cultural or socio-economic context were made to be truly explorative. Articles were eligible for inclusion if their study population consisted of patients with one or more types of NMDs. Since a single comprehensive list of all NMDs could not be found, the list of NMDs by the Muscular Dystrophy Association (MDA) was used as a reference [ 28 ]. If a disease could not be found under the listed disorders, the International Classification of Diseases (ICD) was consulted [ 29 ]. No limitations regarding sex, age, race, or nationality were made. All types of telemedicine were eligible for inclusion. The taxonomy by Bashshur et al. was used as a guiding definition [ 26 ]. Bashshur uses telemedicine in his paper as the original term for ICT in healthcare. The domains include the following components:

Telehealth : Health behavior & education; Health & disease epidemiology; Environmental/Industrial health; Health management & policy.

E-health : Electronic health record; Health information; Clinical decision support system; Physician order entry.

M-health : Clinical support; Health worker support; Remote data collection; Helplines.

Interventions could be implemented on a national, communal, or institutional level. The users could include patients, caregivers, and healthcare workers. Only primary research was included. Due to the explorative nature of the systematic review, no major restrictions regarding study types were made. Only articles written in English or German were included. Due to the rapid pace of technological progress, only studies from the last ten years were considered. This ensured that the telemedicine interventions were not out-of-date or obsolete.

Studies were excluded if no specific diagnostic group was mentioned. Further reviews, study protocols and commentaries were excluded.

Data extraction and analysis

The data extraction and analysis were done by DS. From the included studies the following data points were extracted: authors, year of publication, country, included NMDs, intervention type and analysed outcomes. Additionally, barriers and facilitators of the implementation process were collected. The Consolidated Framework for Implementation Research (CFIR) was used to guide the extraction process [ 30 ]. The CFIR is an established framework for the analysis of implementation processes. Based on this structure, a detailed coding manual with operationalised definitions for each construct was created. This manual served as a reference guide to ensure that the extraction and coding process was systematic and reproducible.

The data synthesis was done narratively. Since no effect measures were used, a quantitative analysis was not applicable. Firstly, the types of telemedicine interventions were clustered according to the domains described by Bashshur et al., to gain a comprehensive understanding of the current landscape of telemedicine applications [ 26 ]. Secondly, the CFIR was used to label quotes on implementation barriers and facilitators [ 30 ].

No meta-analysis was conducted as there are no quantitative outcomes to analyse. Further, the heterogeneity of the studies was not assessed. Due to the broad inclusion criteria, a high heterogeneity could be expected. Since the focus of this systematic review lies on the intervention types, rather than on their effectiveness, subgroup analyses were not performed. Equally no sensitivity analyses were conducted. The focus of the systematic review was not to summarise evidence regarding a specific intervention, it was an exploration of the current telemedicine options for patients with NMDs.

Risk of bias

The study protocol stated a risk of bias assessment using the RoB 2 and ROBINS-I tools [ 31 , 32 ]. This was later changed to the JBI’s critical appraisal tools as they offered a wider selection of checklists [ 33 ]. No meta-bias was analyzed since the outcomes of the studies were not a point of interest.

Included studies

Figure  1 depicts the study selection process for the systematic review, including a total of 57 reports. These included four report pairs with interlinked content. Ando et al. published two papers on the Intervention Careportal in 2019 and 2021 [ 34 , 35 ]. Hobson et al. conducted one study with results disseminated across two publications [ 36 , 37 ]. Martinet et al. conducted two studies utilising the same intervention but with distinct comparison groups and study populations [ 38 , 39 ]. Lastly, Sobierajska-Rek et al. and Wasilewska et al. published two articles addressing different subsections of one main study [ 40 , 41 ]. Studies excluded during the full text screening process can be found in supplementary file 2 .

Flow diagram of the identified studies (Source: own depiction)

Study characteristics

Table 1 presents an overview of the study characteristics. A total of 25 studies were carried out using a cross-sectional design [ 34 , 35 , 40 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 ]. Additionally, the review included two case series [ 64 , 65 ] and one case–control study [ 66 ]. Among the studies, 16 adopted a cohort study design [ 41 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 , 80 , 81 ], while ten employed an experimental design [ 36 , 37 , 38 , 39 , 82 , 83 , 84 , 85 , 86 , 87 ]. The remaining three reports were method papers [ 88 , 89 , 90 ]. Geographically, the majority of the studies took place in Europe [ 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 47 , 48 , 49 , 53 , 58 , 59 , 60 , 62 , 65 , 66 , 68 , 69 , 72 , 73 , 74 , 76 , 80 , 81 , 82 , 84 , 87 , 88 , 89 , 90 ] and the USA [ 44 , 45 , 46 , 51 , 52 , 54 , 56 , 57 , 63 , 67 , 71 , 77 , 78 , 79 , 85 , 86 ]. Two studies were conducted in Canada [ 50 , 64 ] and one in each of the following countries: Japan [ 70 ], India [ 75 ], Brazil [ 83 ], and Australia [ 55 ]. One study included participants from around the globe [ 61 ].

A total of twenty-nine studies exclusively focussed on ALS patients [ 34 , 35 , 36 , 37 , 42 , 46 , 48 , 51 , 52 , 55 , 56 , 57 , 62 , 69 , 70 , 71 , 72 , 73 , 76 , 77 , 78 , 79 , 80 , 81 , 85 , 86 , 87 , 88 , 89 , 90 ], while another four studies included ALS patients alongside other NMD diagnostic groups [ 53 , 54 , 58 , 60 ]. The study outcomes assessed in these studies varied widely. Clinical outcomes, such as physical and cognitive function, as well as mental health, were often used. Further, user satisfaction and utilisation measurements were applied to evaluate interventions. For patient registry studies, epidemiological statistics, including prevalence, were commonly employed as outcome measures.

While the primary focus has been on exploring the availability of telemedicine interventions for patients with NMDS, it is crucial not to overlook the evaluation of individual study quality and the potential impact of bias. In summary, most studies demonstrated a low risk of bias and employed sound methods and procedures. However, certain limitations, such as the lack of comparison groups, insufficient follow up time, and some inadequate reporting, should be noted. Visual depictions and the complete analysis can be found in supplementary material 3 . Three reports were not assessed as they only presented a method paper without empirical results [ 88 , 89 , 90 ].

Telemedicine domains of included interventions

In the following sections the telemedicine interventions included in the analysis will be examined, guided by the taxonomy by Bashshur et al [ 26 ]. According to their definition, telemedicine comprises of three major domains: telehealth, e-health, and m-health . Eight studies were categorised under the telehealth domain, encompassing all traditional public health areas. E-health, mainly describing the online storage of information and supporting tools for physicians, was represented by ten studies. The majority of studies fell within the m-health domain, a rapidly growing field that leverages mobile devices like smartphones and tablets to deliver healthcare services, monitor patients remotely, and support self-management. Given that interventions could encompass elements from different domains, multiple mentions or references to different domains is possible. As stated, there were instances where multiple reports featured identical telemedicine interventions [ 34 , 35 , 36 , 37 , 38 , 39 ]. In order not to bias the results, identical interventions were counted as one during the analysis of the telemedicine domains and components. The distribution of telemedicine domains is illustrated in Fig.  2 a.

Distribution of the telemedicine ( a ) and telehealth domains ( b ) (Source: own depiction)

The studies within the telehealth domain were mostly epidemiological studies. Six studies described online patient registries for one or more NMDs [ 43 , 44 , 47 , 56 , 58 , 72 ]. The remaining two studies were categorised under health education. One study introduced a blended curriculum focusing on physical examinations for patients with NMDs [ 45 ] while another detailed a virtual neuromuscular ultrasound course [ 61 ]. The distribution of the telehealth domain can be seen in Fig.  2 b.

The second smallest domain was e-health (Fig.  3 ). Within this domain, three studies incorporated electronic health records [ 69 , 89 , 90 ]. Health information was the subject of five studies, with two of these not providing an intervention but instead investigating patients’ computer use and information seeking behaviour [ 42 , 50 ]. Only two interventions described clinical decision support systems, one supporting physicians during the diagnostic phase [ 53 ] and another supporting patients with advanced care planning [ 71 ]. A singular app used a function for physician order entries, specifically for nutrition plan entries [ 86 ].

Distribution of the e-Health ( a ) and m-health domains ( b ) (Source: own depiction)

Most included studies contained m-health components (Fig.  3 ). Among the various m-health interventions analysed, helplines represented the smallest category. Specifically, four interventions provided emergency telephone support, and one included useful helpline numbers in their app [ 35 , 60 , 81 , 89 , 90 ].

The predominant categories within the m-health domain were clinical support and remote data collection. Nine studies reported interventions with synchronous consultations and data collection [ 40 , 51 , 57 , 62 , 65 , 73 , 75 , 81 , 85 ]. To illustrate, Christodoulou et al. conducted telephone-based cognitive-behavioural screening in ALS patients [ 85 ], demonstrating how telemedicine can seamlessly combine remote data collection processes with distance consultations. Another example was the remote application of the ALS Functioning Rating Scale during teleconsultations [ 62 ]. An alternative approach identified involving clinical support and remote data collection occurring asynchronously, utilising specially designed devices or mobile applications for data collection [ 35 , 66 , 68 , 69 , 70 , 88 , 90 ]. In this approach, clinical consultation was offered either on demand or automatically triggered based on the collected data.

Fourteen studies used clinical support without remote data collection, including home exercise programs [ 40 , 82 , 83 , 84 ], psychological interventions [ 39 , 87 ] and pure teleconsultation [ 52 , 54 , 55 , 60 , 77 , 78 , 79 ]. In contrast, 12 studies focussed on pure remote data collection without clinical support. This included, accelerometers [ 74 , 80 ], physical assessments [ 63 , 64 , 67 , 76 ] or the assessment of the patient’s nutritional status [ 86 ] or disease-related health [ 37 , 46 , 89 ]. Additionally, Cesareo et al. as well as Wasilewska et al. examined remote pulmonary monitoring devices [ 41 , 49 ].

Barriers and facilitators for the implementation of telemedicine

CFIR was used to assess factors that may facilitate or hinder the implementation of telemedicine. This framework consists of five domains: the inner setting, the outer setting, the implementation process, the intervention characteristics, and the characteristics of the individuals. Relevant information was found in 22 studies, with a predominant focus on patient and carer perspectives [ 34 , 36 , 37 , 41 , 42 , 48 , 49 , 50 , 51 , 52 , 55 , 62 , 63 , 66 , 69 , 73 , 74 , 75 , 76 , 77 , 78 , 83 ]. As a result, no information regarding the inner/outer setting or the implementation process was gathered. All statements focused on the intervention characteristics or the characteristics of the individuals. Thus, the following section is structured according to the two domains and their constructs.

Intervention characteristics

A summary of mentioned barriers and facilitators can be seen in Table  2 .

General characteristics

This category summarises all barriers and facilitators directly linked to the intervention that could not be categorised elsewhere. The most common barrier encountered during the implementation of telemedicine interventions were malfunctions related to internet connectivity or end devices. Examples included software errors [ 51 ], faulty data transmission [ 34 ] or a poor internet connection [ 83 ]. Additionally, it was reported, that the internet and necessary end devices, such as smartphones, tablets, or computers, were often not available [ 48 , 50 , 63 ].

Relative advantage

A major factor for patients was the reduced time and travel burden [ 34 , 51 , 52 , 62 , 76 , 78 ]. In more advanced stages of the diseases travelling with medical equipment became almost impossible, making telemedicine vital for house-bound patients [ 78 ].

Telemonitoring and the remote data collection provided multiple advantages, with patients and caregivers highlighting the timeliness of actions in case of alerts [ 34 , 73 ]. Continuous monitoring also proved beneficial for in-person visits, as medical staff stated that appointments could be used more efficiently with data being analysed beforehand [ 69 ]. Some disadvantages regarding telemedicine were acknowledged. Caregivers and physicians noted the lack of physical evaluation as problematic [ 51 , 52 ]. Additionally, an emotional distance and a lack of informal encounters between patients and healthcare workers was reported [ 52 , 55 ].

Adaptability

Patients appreciated the flexibility of online exercise programs, which were easier to integrate into their daily routines [ 83 ]. It was seen as important to be able to choose the main form of communication [ 55 , 62 ]. For example, patients with speech difficulties communicating via E-Mail was preferred.

Interventions were easier implemented if participants were thoroughly informed about the telemedicine service and if a computer-literate person was on-site [ 78 ]. The duration and frequency of sessions was another major point. Overall, more frequent, and shorter sessions were perceived as less fatiguing [ 78 ].

Design and quality

Critical considerations included the presentation, design, and quality of telemedicine products, emphasizing features like accessible closing mechanisms for wearable devices and age-appropriate designs [ 49 , 74 ].

From a patient’s perspective telemedicine was cost-saving due to reduced travel [ 34 , 48 ]. Nevertheless, acquisition costs could be a barrier for some. Institutional perspectives indicated potential savings, ranging from 20 to 89%, depending on the approach, making costs a crucial factor [ 50 , 77 ].

Characteristics of individuals

The second domain related to the characteristics of individuals. This includes all stakeholders such as patients, caregivers, and healthcare workers. Table 3 depicts the barriers and facilitators relating to the characteristics of individuals.

Knowledge and Beliefs about the Intervention

The CFIR highlights the importance of an individual’s pre-existing knowledge and beliefs about the intervention [ 30 ]. Trust in the intervention was vital for patients using telemonitoring [ 34 , 36 , 52 , 69 , 78 ]. This includes being confident that the transmitted data was monitored and that providers would act in the case of abnormalities.

Self-efficacy

Easy to use technology was seen as an enabler for telemedicine implementation, as it reassured the user in their abilities. Accordingly, barriers arose if patients could not or did not feel confident in using technological devices [ 50 , 51 , 69 ]. Lack of confidence led patients to use technology on rare occasions and only if deemed necessary [ 36 ].

Other personal attributes

Lastly, this category summarises all personal traits of stakeholders that might impact the implementation of the intervention [ 30 ]. Younger, higher-educated patients embraced technology more readily [ 42 , 75 ]. Another enabler was telemonitoring improving patient empowerment, symptom awareness, and communication [ 34 , 36 , 51 , 69 ]. However, some found constant disease confrontation challenging [ 69 ]. Lastly, a personal connection with medical staff enabled telemedicine use [ 36 ].

This systematic review presents a comprehensive overview of the current status of telemedicine applications for patients with NMDs. The primary objective was to classify the identified interventions according to the dimensions of telemedicine. While some studies within this review explored the epidemiology of NMDs, and two interventions provided education for clinical staff, it's clear that certain aspects of telemedicine in public health remain under-studied.

E-health, encompassing health information, an electronic health record or physician order entries/treatment instructions, was comparatively underutilised, with only a subset of interventions included. Moreover, decision support systems were rarely investigated. The predominant focus of most interventions was on clinical support and remote data collection.

The second phase of the analysis concentrated on the implementation process, with a specific focus on identifying barriers and facilitators associated with both the intervention itself and the individuals involved. In comparison to traditional care, telemedicine often demonstrated a relative advantage. The high motivation demonstrated by NMD patients and their caregivers in integrating telemedicine into their care plan is a testament to the potential of telemedicine as a transformative force in healthcare.

Telemedicine was often perceived as a resource-saving, less fatiguing alternative, particularly offering increased accessibility for homebound patients. The lack of physical touch and reduced personal connections emerged as significant barriers. Additionally, the accessibility of technology played a pivotal role, as inadequate design hindered some patients from using telemedicine services. The acceptance and uptake of telemedicine services often depended on the readiness of patients and their caregivers to embrace and adapt to new digital solutions. Recognising the importance of patient empowerment, fostering the development of essential skills and confidence in utilising technology is crucial for enabling patients to actively engage in their healthcare.

Clinical and policy implications

The COVID-19 pandemic created an unprecedented opportunity for the development and implementation of telehealth. Disruptions in healthcare access, caused by social distancing and hygiene guidelines, led healthcare practitioners to expand telemedicine services to ensure the continuity of care [ 91 , 92 ]. This trend extended to the field of neuromuscular disease care as well [ 48 , 62 , 91 , 93 , 94 ]. The American Academy of Neurology's "Telehealth Position Statement" endorsed telemedicine, citing benefits such as improved access, reduced costs, and enhanced comfort, aligning with findings in this review [ 95 ].

Our findings further highlighted important considerations for the successful implementation of telemedicine. Firstly, it is essential to recognise that not all geographic locations are equally suited for telehealth. Remote areas with insufficient internet or cell phone coverage, as well as low-income households with a lack of digital technologies, may encounter difficulties in participating in telemedicine interventions [ 96 ]. Secondly, careful selection of the target population is vital, as the attitude and willingness of users significantly impact technology uptake [ 34 , 36 , 52 , 69 , 78 ]. The acceptance and efficacy of telemedicine interventions are inherently intertwined with diverse cultural attitudes towards healthcare and technology.

Therefore, understanding cultural factors is critical to discern how these variables may influence the successful integration of telehealth programs across diverse patient populations. A systematic analysis of cultural competence would provide valuable insights to refine and customise approaches, meeting the distinctive needs of diverse communities. Such considerations not only enhance the inclusivity of telemedicine but also contribute to its overall effectiveness and acceptance among a broad spectrum of individuals.

As the results have shown, it is vital to adapt telemedicine to the specific and evolving needs of patients with NMDs. These needs not only vary from patient to patient but also change over time as the disease progresses [ 5 ]. Therefore, when designing telemedicine technology for patients with NMDs, emphasis should be placed on adaptability, flexibility and accessibility [ 49 , 55 , 62 , 74 , 83 ].

Designing telemedicine technology that caters for the unique challenges faced by patients with physical disabilities and cognitive impairments is crucial for fostering inclusive healthcare [ 49 , 74 ]. User interfaces need to incorporate accessibility features, such as voice commands, large fonts, and intuitive navigation, to accommodate individuals with motor challenges or cognitive limitations. Additionally, instructions and information must be presented in various accessible formats, accommodating diverse learning needs [ 97 ].

Prioritising plain language and ensuring readability at lower literacy levels is essential. This approach not only makes instructions universally accessible but also empowers all patients to effectively participate in telemedicine interactions. By incorporating these considerations into the design, telemedicine can better serve the needs of patients with NMDs, promoting inclusivity and enhancing the overall effectiveness of healthcare delivery [ 97 ].

Health policies and regulatory frameworks play a significant role in influencing the development and adoption of telehealth practices. A nuanced understanding of these regulations, encompassing aspects such as licensure, reimbursement, and liability, is essential for gaining comprehensive insights into the complex landscape that shapes and governs telemedicine [ 96 ]. The intricate web of reimbursement policies directly influences the economic viability of telemedicine services, impacting both healthcare providers and patients. By navigating and understanding these policy and regulatory intricacies, stakeholders in the telemedicine ecosystem can strategically address and potentially overcome barriers, facilitating a more widespread and effective implementation of telehealth services [ 96 ].

This review reveals that telemedicine interventions for patients with NMDs exist but have yet to realise their full protentional. Firstly, the heavy focus on ALS care should be expanded to encompass all diagnostic groups within the NMD spectrum. Especially the high availability of mHealth applications, which could be seamlessly integrated into care plans. This integration has the potential to enhance continuity of care, simultaneously easing the burden on the healthcare system and reducing appointment frequency for patients [ 69 ].

The incorporation of long-term patient data through remote monitoring holds numerous advantages [ 98 , 99 ]. Continuous data collection could offer enhanced insights into disease progression, thereby improving disease management. Given the degenerative nature of most NMDs, there is a speculation that long-term data could help in detecting early signs of deterioration, facilitating quicker adaption of treatments. Furthermore, detailed information about disease progression could contribute to health prognosis, empowering both patients and healthcare professionals to better plan and coordinate care [ 98 , 99 ]. It is evident that the full benefits of telemonitoring remain undiscovered, making it an important and interesting area for future research. The exploration of these untapped potentials could significantly advance the effectiveness and scope of telemedicine in the context of NMDs.

Research and evaluation opportunities

The current telemedicine landscape yields promising results, particularly in its role in supporting rare disease research through the establishment of disease registries. These registries systematically collect patient data related to disease progression and treatment, forming the foundation for observational studies [ 100 , 101 ]. These studies offer critical insights into the management and progression of rare disease, contributing to evidence-based clinical decisions and facilitating the recruitment of participants for clinical trial.

National and international patient registries are pivotal for studying prevalence and incidence, enhancing our understanding of rare diseases like neuromuscular disorders [ 100 , 101 ]. The establishment of global patient registries becomes especially important for pooling data on rare diseases. International collaborations can help bridge the gap in research for understudied NMDs. By fostering collaboration and sharing data on a global scale, telemedicine-supported registries contribute significantly to advancing our understanding and management of rare diseases.

The results of our systematic review highlight a gap in the research on telemedicine for NMDs. Except for ALS, most NMDs are underrepresented in the current body of literature. Future research should include a more diverse range of diagnostic groups and undertake a comparative analysis of challenges and solutions. This would lead to a higher external validity and faster adaption of telemedicine solutions.

While teleconsultation and remote monitoring for NMDs are well described, other critical domains within telemedicine have received comparatively limited attention. These research gaps should be addressed in the future. Most importantly, implementation science has a critical role in the successful deployment of telemedicine interventions for NMDs. As seen in this systematic review studies, the focus needs to be on patients, caregivers, and health care practitioners, as well as the intervention itself.

It is noteworthy that there is underreporting of crucial aspects, such as the inner and outer settings, as well as the implementation process, in telemedicine interventions for NMDs. Additionally, there is need for research examining the impact of health policies and clinical guidelines on the adoption and implementation of telemedicine. The lack of implementation research has been described in the systematic review by Helleman et al., who analysed telemedicine for ALS patients [ 25 ]. Implementation science is needed to improve the efficiency and uptake of future telemedicine interventions for NMDs [ 102 ].

While our systematic review focused on highlighting the barriers and facilitators of telemedicine, we fully recognise the importance of addressing the validation challenges associated with digital health data. Future research and healthcare policies should emphasise the need for robust validation processes to ensure the reliability and clinical relevance of digital outcomes in telemedicine interventions.

Limitations

Despite an extensive search string, additional search terms might have yielded more results, especially considering synonyms for neuromuscular diseases. A more specific search for individual diagnostic groups would have been more inclusive, but the sheer number of NMDs made this unfeasible.

The literature databases used represent common sources of clinical evidence, but they may not comprehensively cover health policies, management, and health education related to NMDs, which might be found in other types of databases.

The absence of experimental study designs in the individual studies was notable, with most included studies being cross-sectional or observational. However, as this review aims to provide an overview of interventions, this description suffices.

The majority of included studies are from high-income countries, and the extent of telemedicine utilisation in low- and middle-income countries remains unclear. The variation in target population size and time horizon in NMD research reflects the complexity and rarity of these conditions, suggesting a need for longer follow-up times in future studies to better describe long-term outcomes.

This systematic review offers a comprehensive view of the telemedicine landscape in the context of NMDs. While domains like teleconsultation and telemonitoring have received extensive attention and reporting in the literature, other critical domains, such as decision support tools and informational support, are notably lacking in research and documentation. To further understand, develop and implement telemedicine solutions and to close existing gaps in NMD-specific healthcare provision, policies and guidelines are needed. By actively integrating telemedicine into existing healthcare plans and maintaining a commitment to ongoing updates and improvements, healthcare systems can optimise care delivery, enhance patient outcomes, and ensure that individuals with NMDs receive the high-quality care they deserve. In addition, more high-quality studies are needed to close research gaps concerning the implementation process of telemedicine and prove the respective efficiency and effectiveness in the long run.

Availability of data and materials

Due to the nature of the paper, no primary data was generated. All data analysed during this study are included in this published article and its supplementary information files.

Abbreviations

Amyotrophic lateral sclerosis

Anti-synthetase syndrome

Becker muscular dystrophy

Consolidated framework for implementation research

Congenital myopathy

Congenital muscular dystrophy

Charcot-Marie-tooth disease

Dermatomyositis

Duchenne muscular dystrophy

Emery-Dreifuss muscular dystrophy

Facioscapulohumeral muscular dystrophy

Hereditary spastic paraparesis

International classification of diseases

Juvenile dermatomyositis

Lambert-Eaton-myasthenic-syndrome

Limb-Girdle muscular dystrophy

Myotonic dystrophy

Muscular dystrophy

Muscular dystrophy association

Myofibrillar myopathies

Myasthenia gravis

Muscle glycogenosis

Necrotizing myositis

Neuromuscular disease

Overlap myositis

Pompe disease

Polymyositis

Post-Polio syndrome

Randomized controlled trial

Spinal and bulbar muscular atrophy

Spinal muscular atrophy

Transthyretin familial amyloid polyneuropathy

Mary P, Servais L, Vialle R. Neuromuscular diseases: diagnosis and management. Orthopaed Traumatol: Surg Res. 2018;104(1, Supplement):89-S95. Available from: https://www.sciencedirect.com/science/article/pii/S187705681730333X .

Google Scholar  

Aitken M, Mercer EJ, Mckemey A. Understanding neuromuscular disease care: current state and future prospects. IQVIA Institute; 2018.

Bonne G, Rivier F, Hamroun D. The 2018 version of the gene table of monogenic neuromuscular disorders (nuclear genome). Neuromusc Disord: NMD 2017; 27(12). Available from: https://pubmed.ncbi.nlm.nih.gov/29961566/ .

Deenen JC, Horlings CG, Verschuuren JJ, Verbeek AL, van Engelen BG. The epidemiology of neuromuscular disorders: a comprehensive overview of the literature. J Neuromuscul Dis. 2015;2(1):73–85.

Article   PubMed   Google Scholar  

Feldman EL, Russell JW, Löscher WN, Grisold W, Meng S. Atlas of Neuromuscular diseases: a practical guideline. 3rd ed. Cham: Springer International Publishing; Imprint Springer; 2021. Springer eBook Collection.

Book   Google Scholar  

Craig BM, Hartman JD, Owens MA, Brown DS. Prevalence and losses in quality-adjusted life years of child health conditions: a burden of disease analysis. Matern Child Health J. 2016;20(4):862–9.

Article   PubMed   PubMed Central   Google Scholar  

Birnkrant DJ, Bushby K, Bann CM, Alman BA, Apkon SD, Blackwell A, et al. Diagnosis and management of Duchenne muscular dystrophy, part 2: respiratory, cardiac, bone health, and orthopaedic management. Lancet Neurol. 2018;17(4):347–61. Available from: https://pubmed.ncbi.nlm.nih.gov/29395990/ .

Birnkrant DJ, Bushby K, Bann CM, Apkon SD, Blackwell A, Brumbaugh D, et al. Diagnosis and management of Duchenne muscular dystrophy, part 1: diagnosis, and neuromuscular, rehabilitation, endocrine, and gastrointestinal and nutritional management. Lancet Neurol. 2018;17(3):251–67. Available from: https://pubmed.ncbi.nlm.nih.gov/29395989/ .

Hulisz D. Amyotrophic lateral sclerosis: disease state overview. The American journal of managed care. 2018;24(15 Suppl):S320–6. Available from: https://pubmed.ncbi.nlm.nih.gov/30207670/ .

PubMed   Google Scholar  

Khan F, Ng L, Amatya B, Brand C, Turner-Stokes L. Multidisciplinary care for Guillain-Barré syndrome. Europ J Phys Rehabil Med. 2011;47(4):607–12. Available from: https://pubmed.ncbi.nlm.nih.gov/21912364/ .

CAS   Google Scholar  

Mercuri E, Sumner CJ, Muntoni F, Darras BT, Finkel RS. Spinal muscular atrophy. Nat Rev Dis Prim. 2022;8(1):52. Available from: https://pubmed.ncbi.nlm.nih.gov/35927425/ .

Birnkrant DJ, Bushby K, Bann CM, Apkon SD, Blackwell A, Colvin MK, et al. Diagnosis and management of Duchenne muscular dystrophy, part 3: primary care, emergency management, psychosocial care, and transitions of care across the lifespan. Lancet Neurol. 2018;17(5):445–55.

Johnson NE. Myotonic muscular dystrophies. Continuum (Minneap Minn). 2019;25(6):1682–95.

Brandt M, Johannsen L, Inhestern L, Bergelt C. Parents as informal caregivers of children and adolescents with spinal muscular atrophy: a systematic review of quantitative and qualitative data on the psychosocial situation, caregiver burden, and family needs. Orphanet J Rare Dis. 2022;17(1):274. Available from: https://pubmed.ncbi.nlm.nih.gov/35854387/ .

Landfeldt E, Edström J, Buccella F, Kirschner J, Lochmüller H. Duchenne muscular dystrophy and caregiver burden: a systematic review. Dev Med Child Neurol. 2018;60(10):987–96.

Tramonti F, Bonfiglio L, Bongioanni P, Belviso C, Fanciullacci C, Rossi B, et al. Caregiver burden and family functioning in different neurological diseases. Psychol Health Med. 2019;24(1):27–34.

de Wit J, Bakker LA, van Groenestijn AC, van den Berg LH, SchrÖder CD, Visser-Meily JMA, et al. Caregiver burden in amyotrophic lateral sclerosis: a systematic review. Palliat Med. 2018;32(1):231–45. Available from: https://pubmed.ncbi.nlm.nih.gov/28671483/ .

Hamine S, Gerth-Guyette E, Faulx D, Green BB, Ginsburg AS. Impact of mHealth chronic disease management on treatment adherence and patient outcomes: a systematic review. J Med Internet Res. 2015;17(2):e52. Available from: https://pubmed.ncbi.nlm.nih.gov/25803266/ .

Hanlon P, Daines L, Campbell C, McKinstry B, Weller D, Pinnock H. Telehealth interventions to support self-management of long-term conditions: a systematic metareview of diabetes, heart failure, asthma, chronic obstructive pulmonary disease, and cancer. J Med Internet Res. 2017;19(5):e172.

Whitehead L, Seaton P. The effectiveness of self-management mobile phone and tablet apps in long-term condition management: a systematic review. J Med Internet Res. 2016;18(5):e97.

Bashshur RL, Shannon GW, Smith BR, Alverson DC, Antoniotti N, Barsan WG, et al. The empirical foundations of telemedicine interventions for chronic disease management. Telemed J e-health. 2014;20(9):769–800.

Eze ND, Mateus C, Cravo Oliveira Hashiguchi T. Telemedicine in the OECD: An umbrella review of clinical and cost-effectiveness, patient experience and implementation. PLos One. 2020;15(8):0237585.

Article   Google Scholar  

León-Salas B, González-Hernández Y, Infante-Ventura D, de Armas-Castellano A, García-García J, García-Hernández M, et al. Telemedicine for neurological diseases: a systematic review and meta-analysis. Eur J Neurol. 2023;30(1):241–54.

Wang H, Yuan X, Wang J, Sun C, Wang G. Telemedicine maybe an effective solution for management of chronic disease during the COVID-19 epidemic. Prim Health Care Res Dev. 2021;22:e48.

Helleman J, Kruitwagen ET, van den Berg LH, Visser-Meily JMA, Beelen A. The current use of telehealth in ALS care and the barriers to and facilitators of implementation: a systematic review. Amyotroph Lateral Scler Frontotemporal Degener. 2020;21(3–4):167–82.

Article   CAS   PubMed   Google Scholar  

Bashshur R, Shannon G, Krupinski E, Grigsby J. The taxonomy of telemedicine. Telemed J E-health 2011;17(6). Available from: https://pubmed.ncbi.nlm.nih.gov/21718114/ .

Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.

Muscular Dystrophy Association. Find a neuromuscular disease | Muscular Dystrophy Association; 2018 [cited 2022 Mar 28.526Z]. Available from: https://www.mda.org/disease/list .

ICD-11 for Mortality and Morbidity Statistics; 2022 [cited 2022 Mar 31.331Z]. Available from: https://icd.who.int/browse11/l-m/en .

Damschroder LJ, Aron DC, Keith RE, Kirsh, SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009;4. Available from: https://pubmed.ncbi.nlm.nih.gov/19664226/ .

Sterne JAC, Savović J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:l4898.

Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919.

Critical Appraisal Tools | JBI; 2022 [cited 2022 Sep 8]. Available from: https://jbi.global/critical-appraisal-tools .

Ando H, Ashcroft-Kelso H, Halhead R, Chakrabarti B, Young CA, Cousins R, et al. Experience of telehealth in people with motor neurone disease using noninvasive ventilation. Disabil Rehabil Assist Technol. 2021;16(5):490–6.

Ando H, Ashcroft-Kelso H, Halhead R, Young CA, Chakrabarti B, Levene P, et al. Incorporating self-reported questions for telemonitoring to optimize care of patients with MND on noninvasive ventilation (MND OptNIVent). Amyotroph Lateral Scler Frontotemporal Degener. 2019;20(5–6):336–47.

Hobson E, Baird W, Bradburn M, Cooper C, Mawson S, Quinn A, et al. Process evaluation and exploration of telehealth in motor neuron disease in a UK specialist centre. BMJ Open. 2019;9(10):e028526. Available from: https://pubmed.ncbi.nlm.nih.gov/31640994/ .

Hobson EV, Baird WO, Bradburn M, Cooper C, Mawson S, Quinn A, et al. Using telehealth in motor neuron disease to increase access to specialist multidisciplinary care: a UK-based pilot and feasibility study. BMJ Open. 2019;9(10):e028525. Available from: https://bmjopen.bmj.com/content/9/10/e028525 .

Martínez O, Amayra I, López-Paz JF, Lázaro E, Caballero P, García I, et al. Effects of teleassistance on the quality of life of people with rare neuromuscular diseases according to their degree of disability. Front Psychol. 2021;12:637413.

Martínez O, Jometón A, Pérez M, Lázaro E, Amayra I, López-Paz JF et al. Effectiveness of teleassistance at improving quality of life in people with neuromuscular diseases. Spanish J Psychol 2014;17. Available from: https://pubmed.ncbi.nlm.nih.gov/26055393/ .

Sobierajska-Rek A, Mański Ł, Jabłońska-Brudło J, Śledzińska K, Ucińska A, Wierzba J. Establishing a telerehabilitation program for patients with Duchenne muscular dystrophy in the COVID-19 pandemic. Wien Klin Wochenschr. 2020;133(7–8):344–50. Available from: https://link.springer.com/article/10.1007/s00508-020-01786-8 .

PubMed   PubMed Central   Google Scholar  

Wasilewska E, Sobierajska-Rek A, Małgorzewicz S, Soliński M, Szalewska D, Jassem E. Is it possible to have home e-monitoring of pulmonary function in our patients with duchenne muscular dystrophy in the covid-19 pandemic?-A one center pilot study. Int J Environ Res Public Health 2021;18(17).

Abdulla S, Vielhaber S, Machts J, Heinze H-J, Dengler R, Petri S. Information needs and information-seeking preferences of ALS patients and their carers. Amyotroph Lateral Scler Frontotemporal Degener. 2014;15(7–8):505–12.

Ambrosini A, Calabrese D, Avato FM, Catania F, Cavaletti G, Pera MC, et al. The Italian neuromuscular registry: a coordinated platform where patient organizations and clinicians collaborate for data collection and multiple usage. Orphanet J Rare Dis. 2018;13(1):176.

Anil R, Kumar A, Alaparthi S, Sharma A, Nye JL, Roy B, et al. Exploring outcomes and characteristics of myasthenia gravis: Rationale, aims and design of registry - The EXPLORE-MG registry. J Neurol Sci. 2020;414:116830.

Benjamin JC, Groner J, Walton J, Noritz G, Gascon GM, Mahan JD. A blended curriculum to improve resident physical exam skills for patients with neuromuscular disability. MedEdPORTAL. 2019;15:10792.

Berry JD, Paganoni S, Carlson K, Burke K, Weber H, Staples P, et al. Design and results of a smartphone-based digital phenotyping study to quantify ALS progression. Ann Clin Transl Neurol. 2019;6(5):873–81.

Bettio C, Salsi V, Orsini M, Calanchi E, Magnotta L, Gagliardelli L, et al. The Italian National Registry for FSHD: an enhanced data integration and an analytics framework towards smart health care and precision medicine for a rare disease. Orphanet J Rare Dis. 2021;16(1):470.

Capozzo R, Zoccolella S, Musio M, Barone R, Accogli M, Logroscino G. Telemedicine is a useful tool to deliver care to patients with Amyotrophic Lateral Sclerosis during COVID-19 pandemic: results from Southern Italy. Amyotroph Lateral Scler Frontotemporal Degener. 2020;21(7–8):542–8.

Cesareo A, Nido SA, Biffi E, Gandossini S, D’Angelo MG, Aliverti A. A Wearable device for breathing frequency monitoring: a pilot study on patients with muscular dystrophy. Sensors (Basel) 2020;20(18).

Climans SA, Piechowicz C, Koopman WJ, Venance SL. Survey of Canadian myotonic dystrophy patients’ access to computer technology. Can J Neurol Sci. 2017;44(5):567–71.

Geronimo A, Simmons Z. Evaluation of remote pulmonary function testing in motor neuron disease. Amyotroph Lateral Scler Frontotemporal Degener. 2019;20(5–6):348–55.

Geronimo A, Wright C, Morris A, Walsh S, Snyder B, Simmons Z. Incorporation of telehealth into a multidisciplinary ALS Clinic: feasibility and acceptability. Amyotroph Lateral Scler Frontotemporal Degener. 2017;18(7–8):555–61.

Grigull L, Lechner W, Petri S, Kollewe K, Dengler R, Mehmecke S, et al. Diagnostic support for selected neuromuscular diseases using answer-pattern recognition and data mining techniques: a proof of concept multicenter prospective trial. BMC Med Inform Decis Mak. 2016;16:31.

Hooshmand S, Cho J, Singh S, Govindarajan R. Satisfaction of telehealth in patients with established neuromuscular disorders. Front Neurol. 2021;12:667813.

James N, Power E, Hogden A, Vucic S. Patients’ perspectives of multidisciplinary home-based e-Health service delivery for motor neurone disease. Disabil Rehabil Assist Technol. 2019;14(7):737–43.

Malek AM, Stickler DE, Antao VC, Horton DK. The National ALS Registry: a recruitment tool for research. Muscle Nerve. 2014;50(5):830–4.

Pulley MT, Brittain R, Hodges W, Frazier C, Miller L, Matyjasik-Liggett M, et al. Multidisciplinary amyotrophic lateral sclerosis telemedicine care: The store and forward method. Muscle Nerve. 2019;59(1):34–9.

Roy AJ, van den Bergh P, van Damme P, Doggen K, van Casteren V. Early stages of building a rare disease registry, methods and 2010 data from the Belgian Neuromuscular Disease Registry (BNMDR). Acta Neurol Belg. 2015;115(2):97–104.

Sobierajska-Rek A, Mański Ł, Jabłońska-Brudło J, Śledzińska K, Wasilewska E, Szalewska D. Respiratory telerehabilitation of boys and young men with Duchenne muscular dystrophy in the COVID-19 Pandemic. Int J Environ Res Public Health 2021;18(12).

Spiliopoulos KC, Kasdaglis N, Veltsista D, Lykouras D, Lagadinou M, Chroni E. Teleneurology in a center for neuromuscular diseases during the COVID-19 pandemic. Acta Neurol Belg. 2022;122(3):721–4.

Tawfik EA, van Alfen N, Cartwright MS, Inkpen P, Kerasnoudis A, Lieba-Samal D, et al. Virtual neuromuscular ultrasound courses during COVID-19 pandemic: Leveraging technology to enhance learning opportunities. Muscle Nerve. 2021;65(1):29–33.

Vasta R, Moglia C, D’Ovidio F, Di Pede F, de Mattei F, Cabras S, et al. Telemedicine for patients with amyotrophic lateral sclerosis during COVID-19 pandemic: an Italian ALS referral center experience. Amyotroph Lateral Scler Frontotemporal Degener. 2021;22(3–4):308–11.

White MK, Leffler M, Rychlec K, Jones C, McSherry C, Walker L, et al. Adapting traditional content validation methods to fit purpose: an example with a novel video assessment and training materials in Duchenne muscular dystrophy (DMD). Qual Life Res. 2019;28(11):2979–88.

Menon D, Alnajjar S, Barnett C, Vijayan J, Katzberg H, Fathi D, et al. Telephone consultation for myasthenia gravis care during the COVID-19 pandemic: Assessment of a novel virtual myasthenia gravis index. Muscle Nerve. 2021;63(6):831–6.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Portaro S, Calabrò RS, Bramanti P, Silvestri G, Torrisi M, Conti-Nibali V, et al. Telemedicine for Facio-Scapulo-Humeral Muscular Dystrophy: A multidisciplinary approach to improve quality of life and reduce hospitalization rate? Disabil Health J. 2017;11(2):306–9.

Trucco F, Pedemonte M, Racca F, Falsaperla R, Romano C, Wenzel A, et al. Tele-monitoring in paediatric and young home-ventilated neuromuscular patients: A multicentre case-control trial. J Telemed Telecare. 2019;25(7):414–24.

Contesse MG, Sapp ATL, Apkon SD, Lowes LP, Dalle Pazze L, Leffler MG. Reliability and construct validity of the Duchenne Video Assessment. Muscle Nerve. 2021;64(2):180–9.

Garuti G, Bagatti S, Verucchi E, Massobrio M, Spagnolatti L, Vezzani G, et al. Pulmonary rehabilitation at home guided by telemonitoring and access to healthcare facilities for respiratory complications in patients with neuromuscular disease. Eur J Phys Rehabil Med. 2013;49(1):51–7. Available from: https://pubmed.ncbi.nlm.nih.gov/22820817/ .

CAS   PubMed   Google Scholar  

Helleman J, van Eenennaam R, Kruitwagen ET, Kruithof WJ, Slappendel MJ, van den Berg LH, et al. Telehealth as part of specialized ALS care: feasibility and user experiences with “ALS home-monitoring and coaching.” Amyotroph Lateral Scler Frontotemporal Degener. 2020;21(3–4):183–92.

Kamei T, Yamamoto Y, Kanamori T, Nakayama Y, Porter SE. Detection of early-stage changes in people with chronic diseases: A telehome monitoring-based telenursing feasibility study. Nurs Health Sci. 2018;20(3):313–22.

Levi BH, Simmons Z, Hanna C, Brothers A, Lehman E, Farace E, et al. Advance care planning for patients with amyotrophic lateral sclerosis. Amyotroph Lateral Scler Frontotemporal Degener. 2017;18(5–6):388–96.

Longinetti E, Regodón Wallin A, Samuelsson K, Press R, Zachau A, Ronnevi L-O, et al. The Swedish motor neuron disease quality registry. Amyotroph Lateral Scler Frontotemporal Degener. 2018;19(7–8):528–37.

de Marchi F, Sarnelli MF, Serioli M, de Marchi I, Zani E, Bottone N, et al. Telehealth approach for amyotrophic lateral sclerosis patients: the experience during COVID-19 pandemic. Acta Neurol Scand. 2021;143(5):489–96.

McErlane F, Davies EH, Ollivier C, Mayhew A, Anyanwu O, Harbottle V, et al. Wearable technologies for children with chronic illnesses: an exploratory approach. Ther Innov Regul Sci. 2021;55(4):799–806.

Naveen R, Sundaram TG, Agarwal V, Gupta L. Teleconsultation experience with the idiopathic inflammatory myopathies: a prospective observational cohort study during the COVID-19 pandemic. Rheumatol Int. 2020;41(1):67–76.

Newton J, Jayaprakash K, Glasmacher SA, McEleney A, Bethell A, Fraser E, et al. Excellent reliability of the ALSFRS-R administered via videoconferencing: a study of people with motor neuron disease in Scotland. J Neurol Sci. 2020;416:116991.

Paganoni S, van de Rijn M, Drake K, Burke K, Doyle M, Ellrodt AS, et al. Adjusted cost analysis of video televisits for the care of people with amyotrophic lateral sclerosis. Muscle Nerve. 2019;60(2):147–54.

Roman A, Baylor C, Johnson L, Barton M. Expanding Availability of Speech-Generating Device Evaluation and Treatment to People With Amyotrophic Lateral Sclerosis (pALS) Through Telepractice: Perspectives of pALS and Communication Partners. Am J Speech Lang Pathol. 2021;30(5):2098–114.

Selkirk SM, Washington MO, McClellan F, Flynn B, Seton JM, Strozewski R. Delivering tertiary centre specialty care to ALS patients via telemedicine: a retrospective cohort analysis. Amyotroph Lateral Scler Frontotemporal Degener. 2017;18(5–6):324–32.

van Eijk RPA, Bakers JNE, Bunte TM, de Fockert AJ, Eijkemans MJC, van den Berg LH. Accelerometry for remote monitoring of physical activity in amyotrophic lateral sclerosis: a longitudinal cohort study. J Neurol. 2019;266(10):2387–95.

Vitacca M, Comini L, Assoni G, Fiorenza D, Gilè S, Bernocchi P, et al. Tele-assistance in patients with amyotrophic lateral sclerosis: long term activity and costs. Disabil Rehabil Assist Technol. 2012;7(6):494–500.

Alexanderson H, Munters LA, Dastmalchi M, Loell I, Heimbürger M, Opava CH, et al. Resistive home exercise in patients with recent-onset polymyositis and dermatomyositis—a randomized controlled single-blinded study with a 2-year followup. J Rheumatol. 2014;41(6):1124–32.

Astley C, Sieczkowska SM, Marques IG, Ihara BP, Lindoso L, Lavorato SSM, et al. Home-based exercise program for adolescents with juvenile dermatomyositis quarantined during COVID-19 pandemic: a mixed methods study. Pediatr Rheumatol Online J. 2021;19(1):159.

Bankolé L-C, Millet GY, Temesi J, Bachasson D, Ravelojaona M, Wuyam B, et al. Safety and efficacy of a 6-month home-based exercise program in patients with facioscapulohumeral muscular dystrophy: A randomized controlled trial. Medicine (Baltimore). 2016;95(31): e4497.

Christodoulou G, Gennings C, Hupf J, Factor-Litvak P, Murphy J, Goetz RR, et al. Telephone based cognitive-behavioral screening for frontotemporal changes in patients with amyotrophic lateral sclerosis (ALS). Amyotroph Lateral Scler Frontotemporal Degener. 2016;17(7–8):482–8.

Wills AM, Garry J, Hubbard J, Mezoian T, Breen CT, Ortiz-Miller C et al. Nutritional counseling with or without mobile health technology: a randomized open-label standard-of-care-controlled trial in ALS. BMC Neurol 2019; 19.

de Wit J, Beelen A, Drossaert CHC, Kolijn R, van den Berg LH, SchrÖder CD, et al. Blended psychosocial support for partners of patients with ALS and PMA: results of a randomized controlled trial. Amyotroph Lateral Scler Frontotemporal Degener. 2020;21(5–6):344–54.

Palumbo A, Ielpo N, Calabrese B, Corchiola D, Garropoli R, Gramigna V et al. SIMpLE: A mobile cloud-based system for health monitoring of people with ALS. Sensors (Basel) 2021; 21(21).

Ricci G, Baldanzi S, Seidita F, Proietti C, Carlini F, Peviani S, et al. A mobile app for patients with Pompe disease and its possible clinical applications. Neuromusc Disord: NMD. 2018;28(6):471–5.

Scalvini S, Bernocchi P, Zanelli E, Comini L, Vitacca M. Maugeri centre for telehealth and telecare: a real-life integrated experience in chronic patients. J Telemed Telecare. 2018;24(7):500–7.

El-Hassar L, Amara A, Sanson B, Lacatus O, Amir Belhouchet A, Kroneman M, et al. Telemedicine in neuromuscular diseases during covid-19 pandemic: ERN-NMD European survey. J Neuromuscul Dis. 2022;Preprint(Preprint):1–12.

Monaghesh E, Hajizadeh A. The role of telehealth during COVID-19 outbreak: a systematic review based on current evidence. BMC Public Health. 2020;20(1):1193.

Bombaci A, Abbadessa G, Trojsi F, Leocani L, Bonavita S, Lavorgna L. Telemedicine for management of patients with amyotrophic lateral sclerosis through COVID-19 tail. Neurol Sci. 2021;42(1):9–13.

Giannotta M, Petrelli C, Pini A. Telemedicine applied to neuromuscular disorders: focus on the COVID-19 pandemic era. Acta myologica: myopathies and cardiomyopathies. 2022;41(1):30–6. Available from: https://pubmed.ncbi.nlm.nih.gov/35465343/ .

Hatcher-Martin JM, Busis NA, Cohen BH, Wolf RA, Jones EC, Anderson ER, et al. American academy of neurology telehealth position statement. Neurology. 2021;97(7):334–9. Available from: https://n.neurology.org/content/97/7/334 .

Kruse CS, Williams K, Bohls J, Shamsi W. Telemedicine and health policy: a systematic review. Health Policy Technol. 2021;10(1):209–29.

Phuong J, Ordóñez P, Cao J, Moukheiber M, Moukheiber L, Caspi A, et al. Telehealth and digital health innovations: a mixed landscape of access. PLOS Digit Health. 2023;2(12):e0000401.

Beswick E, Fawcett T, Hassan Z, et al. A systematic review of digital technology to evaluate motor function and disease progression in motor neuron disease. J Neurol. 2022;269:6254–68.

Serrano LP, Maita KC, Avila FR, Torres-Guzman RA, Garcia JP, Eldaly AS, Haider CR, Felton CL, Paulson MR, Maniaci MJ, Forte AJ. Benefits and challenges of remote patient monitoring as perceived by health care practitioners: a systematic review. Permanente J. 2023;27(4):100.

Lacaze P, Millis N, Fookes M, Zurynski Y, Jaffe A, Bellgard M, Winship I, McNeil J, Bittles AH. Rare disease registries: a call to action. Intern Med J. 2017;47(9):1075–9.

Kölker S, Gleich F, Mütze U, Opladen T. Rare disease registries are key to evidence-based personalized medicine: highlighting the European experience. Front Endocrinol. 2022;4(13):832063.

Peters DH, Adam T, Alonge O, Akua Agyepong I, Tran N. Implementation research: what it is and how to do it. BMJ 2013;347. Available from: https://www.bmj.com/content/347/bmj.f6753.full

Download references

Acknowledgements

Not applicable.

The authors did not receive any funding for this paper.

Author information

Authors and affiliations.

Chair of Healthcare Management and Health Services Research, University of Bayreuth, Bayreuth, Germany

Deniz Senyel, Katja Senn & Klaus Nagels

Chair of Digital Health and Innovation, Department of Public Health, School of Psychology and Public Health, La Trobe University, Melbourne, Australia

Deniz Senyel & James Boyd

You can also search for this author in PubMed   Google Scholar

Contributions

DS and KS prepared the study protocol as well as performed the literature search and study selection. DS conducted the data extraction and analysis. The report was written by DS with contributions by KS. JB and KN supervised the complete process. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Deniz Senyel .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Senyel, D., Senn, K., Boyd, J. et al. A systematic review of telemedicine for neuromuscular diseases: components and determinants of practice. BMC Digit Health 2 , 17 (2024). https://doi.org/10.1186/s44247-024-00078-9

Download citation

Received : 05 June 2023

Accepted : 22 February 2024

Published : 09 May 2024

DOI : https://doi.org/10.1186/s44247-024-00078-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Telemedicine
• Neuromuscular diseases
• Digitalization

BMC Digital Health

ISSN: 2731-684X

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• Open access
• Published: 13 May 2024

What are the strengths and limitations to utilising creative methods in public and patient involvement in health and social care research? A qualitative systematic review

• Olivia R. Phillips 1 , 2   na1 ,
• Cerian Harries 2 , 3   na1 ,
• Jo Leonardi-Bee 1 , 2 , 4   na1 ,
• Holly Knight 1 , 2 ,
• Lauren B. Sherar 2 , 3 ,
• Veronica Varela-Mato 2 , 3 &
• Joanne R. Morling 1 , 2 , 5  

Research Involvement and Engagement volume  10 , Article number:  48 ( 2024 ) Cite this article

1 Altmetric

Metrics details

There is increasing interest in using patient and public involvement (PPI) in research to improve the quality of healthcare. Ordinarily, traditional methods have been used such as interviews or focus groups. However, these methods tend to engage a similar demographic of people. Thus, creative methods are being developed to involve patients for whom traditional methods are inaccessible or non-engaging.

To determine the strengths and limitations to using creative PPI methods in health and social care research.

Electronic searches were conducted over five databases on 14th April 2023 (Web of Science, PubMed, ASSIA, CINAHL, Cochrane Library). Studies that involved traditional, non-creative PPI methods were excluded. Creative PPI methods were used to engage with people as research advisors, rather than study participants. Only primary data published in English from 2009 were accepted. Title, abstract and full text screening was undertaken by two independent reviewers before inductive thematic analysis was used to generate themes.

Twelve papers met the inclusion criteria. The creative methods used included songs, poems, drawings, photograph elicitation, drama performance, visualisations, social media, photography, prototype development, cultural animation, card sorting and persona development. Analysis identified four limitations and five strengths to the creative approaches. Limitations included the time and resource intensive nature of creative PPI, the lack of generalisation to wider populations and ethical issues. External factors, such as the lack of infrastructure to support creative PPI, also affected their implementation. Strengths included the disruption of power hierarchies and the creation of a safe space for people to express mundane or “taboo” topics. Creative methods are also engaging, inclusive of people who struggle to participate in traditional PPI and can also be cost and time efficient.

‘Creative PPI’ is an umbrella term encapsulating many different methods of engagement and there are strengths and limitations to each. The choice of which should be determined by the aims and requirements of the research, as well as the characteristics of the PPI group and practical limitations. Creative PPI can be advantageous over more traditional methods, however a hybrid approach could be considered to reap the benefits of both. Creative PPI methods are not widely used; however, this could change over time as PPI becomes embedded even more into research.

Plain English Summary

It is important that patients and public are included in the research process from initial brainstorming, through design to delivery. This is known as public and patient involvement (PPI). Their input means that research closely aligns with their wants and needs. Traditionally to get this input, interviews and group discussions are held, but this can exclude people who find these activities non-engaging or inaccessible, for example those with language challenges, learning disabilities or memory issues. Creative methods of PPI can overcome this. This is a broad term describing different (non-traditional) ways of engaging patients and public in research, such as through the use or art, animation or performance. This review investigated the reasons why creative approaches to PPI could be difficult (limitations) or helpful (strengths) in health and social care research. After searching 5 online databases, 12 studies were included in the review. PPI groups included adults, children and people with language and memory impairments. Creative methods included songs, poems, drawings, the use of photos and drama, visualisations, Facebook, creating prototypes, personas and card sorting. Limitations included the time, cost and effort associated with creative methods, the lack of application to other populations, ethical issues and buy-in from the wider research community. Strengths included the feeling of equality between academics and the public, creation of a safe space for people to express themselves, inclusivity, and that creative PPI can be cost and time efficient. Overall, this review suggests that creative PPI is worthwhile, however each method has its own strengths and limitations and the choice of which will depend on the research project, PPI group characteristics and other practical limitations, such as time and financial constraints.

Peer Review reports

Introduction

Patient and public involvement (PPI) is the term used to describe the partnership between patients (including caregivers, potential patients, healthcare users etc.) or the public (a community member with no known interest in the topic) with researchers. It describes research that is done “‘with’ or ‘by’ the public, rather than ‘to,’ ‘about’ or ‘for’ them” [ 1 ]. In 2009, it became a legislative requirement for certain health and social care organisations to include patients, families, carers and communities in not only the planning of health and social care services, but the commissioning, delivery and evaluation of them too [ 2 ]. For example, funding applications for the National Institute of Health and Care Research (NIHR), a UK funding body, mandates a demonstration of how researchers plan to include patients/service users, the public and carers at each stage of the project [ 3 ]. However, this should not simply be a tokenistic, tick-box exercise. PPI should help formulate initial ideas and should be an instrumental, continuous part of the research process. Input from PPI can provide unique insights not yet considered and can ensure that research and health services are closely aligned to the needs and requirements of service users PPI also generally makes research more relevant with clearer outcomes and impacts [ 4 ]. Although this review refers to both patients and the public using the umbrella term ‘PPI’, it is important to acknowledge that these are two different groups with different motivations, needs and interests when it comes to health research and service delivery [ 5 ].

Despite continuing recognition of the need of PPI to improve quality of healthcare, researchers have also recognised that there is no ‘one size fits all’ method for involving patients [ 4 ]. Traditionally, PPI methods invite people to take part in interviews or focus groups to facilitate discussion, or surveys and questionnaires. However, these can sometimes be inaccessible or non-engaging for certain populations. For example, someone with communication difficulties may find it difficult to engage in focus groups or interviews. If individuals lack the appropriate skills to interact in these types of scenarios, they cannot take advantage of the participation opportunities it can provide [ 6 ]. Creative methods, however, aim to resolve these issues. These are a relatively new concept whereby researchers use creative methods (e.g., artwork, animations, Lego), to make PPI more accessible and engaging for those whose voices would otherwise go unheard. They ensure that all populations can engage in research, regardless of their background or skills. Seminal work has previously been conducted in this area, which brought to light the use of creative methodologies in research. Leavy (2008) [ 7 ] discussed how traditional interviews had limits on what could be expressed due to their sterile, jargon-filled and formulaic structure, read by only a few specialised academics. It was this that called for more creative approaches, which included narrative enquiry, fiction-based research, poetry, music, dance, art, theatre, film and visual art. These practices, which can be used in any stage of the research cycle, supported greater empathy, self-reflection and longer-lasting learning experiences compared to interviews [ 7 ]. They also pushed traditional academic boundaries, which made the research accessible not only to researchers, but the public too. Leavy explains that there are similarities between arts-based approaches and scientific approaches: both attempts to investigate what it means to be human through exploration, and used together, these complimentary approaches can progress our understanding of the human experience [ 7 ]. Further, it is important to acknowledge the parallels and nuances between creative and inclusive methods of PPI. Although creative methods aim to be inclusive (this should underlie any PPI activity, whether creative or not), they do not incorporate all types of accessible, inclusive methodologies e.g., using sign language for people with hearing impairments or audio recordings for people who cannot read. Given that there was not enough scope to include an evaluation of all possible inclusive methodologies, this review will focus on creative methods of PPI only.

We aimed to conduct a qualitative systematic review to highlight the strengths of creative PPI in health and social care research, as well as the limitations, which might act as a barrier to their implementation. A qualitative systematic review “brings together research on a topic, systematically searching for research evidence from primary qualitative studies and drawing the findings together” [ 8 ]. This review can then advise researchers of the best practices when designing PPI.

Public involvement

The PHIRST-LIGHT Public Advisory Group (PAG) consists of a team of experienced public contributors with a diverse range of characteristics from across the UK. The PAG was involved in the initial question setting and study design for this review.

Search strategy

For the purpose of this review, the JBI approach for conducting qualitative systematic reviews was followed [ 9 ]. The search terms were (“creativ*” OR “innovat*” OR “authentic” OR “original” OR “inclu*”) AND (“public and patient involvement” OR “patient and public involvement” OR “public and patient involvement and engagement” OR “patient and public involvement and engagement” OR “PPI” OR “PPIE” OR “co-produc*” OR “co-creat*” OR “co-design*” OR “cooperat*” OR “co-operat*”). This search string was modified according to the requirements of each database. Papers were filtered by title, abstract and keywords (see Additional file 1 for search strings). The databases searched included Web of Science (WoS), PubMed, ASSIA and CINAHL. The Cochrane Library was also searched to identify relevant reviews which could lead to the identification of primary research. The search was conducted on 14/04/23. As our aim was to report on the use of creative PPI in research, rather than more generic public engagement, we used electronic databases of scholarly peer-reviewed literature, which represent a wide range of recognised databases. These identified studies published in general international journals (WoS, PubMed), those in social sciences journals (ASSIA), those in nursing and allied health journals (CINAHL), and trials of interventions (Cochrane Library).

Inclusion criteria

Only full-text, English language, primary research papers from 2009 to 2023 were included. This was the chosen timeframe as in 2009 the Health and Social Reform Act made it mandatory for certain Health and Social Care organisations to involve the public and patients in planning, delivering, and evaluating services [ 2 ]. Only creative methods of PPI were accepted, rather than traditional methods, such as interviews or focus groups. For the purposes of this paper, creative PPI included creative art or arts-based approaches (e.g., e.g. stories, songs, drama, drawing, painting, poetry, photography) to enhance engagement. Titles were related to health and social care and the creative PPI was used to engage with people as research advisors, not as study participants. Meta-analyses, conference abstracts, book chapters, commentaries and reviews were excluded. There were no limits concerning study location or the demographic characteristics of the PPI groups. Only qualitative data were accepted.

Quality appraisal

Quality appraisal using the Critical Appraisal Skills Programme (CASP) checklist [ 10 ] was conducted by the primary authors (ORP and CH). This was done independently, and discrepancies were discussed and resolved. If a consensus could not be reached, a third independent reviewer was consulted (JRM). The full list of quality appraisal questions can be found in Additional file 2 .

Data extraction

ORP extracted the study characteristics and a subset of these were checked by CH. Discrepancies were discussed and amendments made. Extracted data included author, title, location, year of publication, year study was carried out, research question/aim, creative methods used, number of participants, mean age, gender, ethnicity of participants, setting, limitations and strengths of creative PPI and main findings.

Data analysis

The included studies were analysed using inductive thematic analysis [ 11 ], where themes were determined by the data. The familiarisation stage took place during full-text reading of the included articles. Anything identified as a strength or limitation to creative PPI methods was extracted verbatim as an initial code and inputted into the data extraction Excel sheet. Similar codes were sorted into broader themes, either under ‘strengths’ or ‘limitations’ and reviewed. Themes were then assigned a name according to the codes.

The search yielded 9978 titles across the 5 databases: Web of Science (1480 results), PubMed (94 results), ASSIA (2454 results), CINAHL (5948 results) and Cochrane Library (2 results), resulting in 8553 different studies after deduplication. ORP and CH independently screened their titles and abstracts, excluding those that did not meet the criteria. After assessment, 12 studies were included (see Fig.  1 ).

PRISMA flowchart of the study selection process

Study characteristics

The included studies were published between 2018 and 2022. Seven were conducted in the UK [ 12 , 14 , 15 , 17 , 18 , 19 , 23 ], two in Canada [ 21 , 22 ], one in Australia [ 13 ], one in Norway [ 16 ] and one in Ireland [ 20 ]. The PPI activities occurred across various settings, including a school [ 12 ], social club [ 12 ], hospital [ 17 ], university [ 22 ], theatre [ 19 ], hotel [ 20 ], or online [ 15 , 21 ], however this information was omitted in 5 studies [ 13 , 14 , 16 , 18 , 23 ]. The number of people attending the PPI sessions varied, ranging from 6 to 289, however the majority (ten studies) had less than 70 participants [ 13 , 14 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 ]. Seven studies did not provide information on the age or gender of the PPI groups. Of those that did, ages ranged from 8 to 76 and were mostly female. The ethnicities of the PPI group members were also rarely recorded (see Additional file 3 for data extraction table).

Types of creative methods

The type of creative methods used to engage the PPI groups were varied. These included songs, poems, drawings, photograph elicitation, drama performance, visualisations, Facebook, photography, prototype development, cultural animation, card sorting and creating personas (see Table  1 ). These were sometimes accompanied by traditional methods of PPI such as interviews and focus group discussions.

The 12 included studies were all deemed to be of good methodological quality, with scores ranging from 6/10 to 10/10 with the CASP critical appraisal tool [ 10 ] (Table  2 ).

Thematic analysis

Analysis identified four limitations and five strengths to creative PPI (see Fig.  2 ). Limitations included the time and resource intensity of creative PPI methods, its lack of generalisation, ethical issues and external factors. Strengths included the disruption of power hierarchies, the engaging and inclusive nature of the methods and their long-term cost and time efficiency. Creative PPI methods also allowed mundane and “taboo” topics to be discussed within a safe space.

Theme map of strengths and limitations

Limitations of creative PPI

Creative ppi methods are time and resource intensive.

The time and resource intensive nature of creative PPI methods is a limitation, most notably for the persona-scenario methodology. Valaitis et al. [ 22 ] used 14 persona-scenario workshops with 70 participants to co-design a healthcare intervention, which aimed to promote optimal aging in Canada. Using the persona method, pairs composed of patients, healthcare providers, community service providers and volunteers developed a fictional character which they believed represented an ‘end-user’ of the healthcare intervention. Due to the depth and richness of the data produced the authors reported that it was time consuming to analyse. Further, they commented that the amount of information was difficult to disseminate to scientific leads and present at team meetings. Additionally, to ensure the production of high-quality data, to probe for details and lead group discussion there was a need for highly skilled facilitators. The resource intensive nature of the creative co-production was also noted in a study using the persona scenario and creative worksheets to develop a prototype decision support tool for individuals with malignant pleural effusion [ 17 ]. With approximately 50 people, this was also likely to yield a high volume of data to consider.

To prepare materials for populations who cannot engage in traditional methods of PPI was also timely. Kearns et al. [ 18 ] developed a feedback questionnaire for people with aphasia to evaluate ICT-delivered rehabilitation. To ensure people could participate effectively, the resources used during the workshops, such as PowerPoints, online images and photographs, had to be aphasia-accessible, which was labour and time intensive. The author warned that this time commitment should not be underestimated.

There are further practical limitations to implementing creative PPI, such as the costs of materials for activities as well as hiring a space for workshops. For example, the included studies in this review utilised pens, paper, worksheets, laptops, arts and craft supplies and magazines and took place in venues such as universities, a social club, and a hotel. Further, although not limited to creative PPI methods exclusively but rather most studies involving the public, a financial incentive was often offered for participation, as well as food, parking, transport and accommodation [ 21 , 22 ].

Creative PPI lacks generalisation

Another barrier to the use of creative PPI methods in health and social care research was the individual nature of its output. Those who participate, usually small in number, produce unique creative outputs specific to their own experiences, opinions and location. Craven et al. [ 13 ], used arts-based visualisations to develop a toolbox for adults with mental health difficulties. They commented, “such an approach might still not be worthwhile”, as the visualisations were individualised and highly personal. This indicates that the output may fail to meet the needs of its end-users. Further, these creative PPI groups were based in certain geographical regions such as Stoke-on-Trent [ 19 ] Sheffield [ 23 ], South Wales [ 12 ] or Ireland [ 20 ], which limits the extent the findings can be applied to wider populations, even within the same area due to individual nuances. Further, the study by Galler et al. [ 16 ], is specific to the Norwegian context and even then, maybe only a sub-group of the Norwegian population as the sample used was of higher socioeconomic status.

However, Grindell et al. [ 17 ], who used persona scenarios, creative worksheets and prototype development, pointed out that the purpose of this type of research is to improve a certain place, rather than apply findings across other populations and locations. Individualised output may, therefore, only be a limitation to research wanting to conduct PPI on a large scale.

If, however, greater generalisation within PPI is deemed necessary, then social media may offer a resolution. Fedorowicz et al. [ 15 ], used Facebook to gain feedback from the public on the use of video-recording methodology for an upcoming project. This had the benefit of including a more diverse range of people (289 people joined the closed group), who were spread geographically around the UK, as well as seven people from overseas.

Creative PPI has ethical issues

As with other research, ethical issues must be taken into consideration. Due to the nature of creative approaches, as well as the personal effort put into them, people often want to be recognised for their work. However, this compromises principles so heavily instilled in research such as anonymity and confidentiality. With the aim of exploring issues related to health and well-being in a town in South Wales, Byrne et al. [ 12 ], asked year 4/5 and year 10 pupils to create poems, songs, drawings and photographs. Community members also created a performance, mainly of monologues, to explore how poverty and inequalities are dealt with. Byrne noted the risks of these arts-based approaches, that being the possibility of over-disclosure and consequent emotional distress, as well as people’s desire to be named for their work. On one hand, the anonymity reduces the sense of ownership of the output as it does not portray a particular individual’s lived experience anymore. On the other hand, however, it could promote a more honest account of lived experience. Supporting this, Webber et al. [ 23 ], who used the persona method to co-design a back pain educational resource prototype, claimed that the anonymity provided by this creative technique allowed individuals to externalise and anonymise their own personal experience, thus creating a more authentic and genuine resource for future users. This implies that anonymity can be both a limitation and strength here.

The use of creative PPI methods is impeded by external factors

Despite the above limitations influencing the implementation of creative PPI techniques, perhaps the most influential is that creative methodologies are simply not mainstream [ 19 ]. This could be linked to the issues above, like time and resource intensity, generalisation and ethical issues but it is also likely to involve more systemic factors within the research community. Micsinszki et al. [ 21 ], who co-designed a hub for the health and well-being of vulnerable populations, commented that there is insufficient infrastructure to conduct meaningful co-design as well as a dominant medical model. Through a more holistic lens, there are “sociopolitical environments that privilege individualism over collectivism, self-sufficiency over collaboration, and scientific expertise over other ways of knowing based on lived experience” [ 21 ]. This, it could be suggested, renders creative co-design methodologies, which are based on the foundations of collectivism, collaboration and imagination an invalid technique in the research field, which is heavily dominated by more scientific methods offering reproducibility, objectivity and reliability.

Although we acknowledge that creative PPI techniques are not always appropriate, it may be that their main limitation is the lack of awareness of these methods or lack of willingness to use them. Further, there is always the risk that PPI, despite being a mandatory part of research, is used in a tokenistic or tick-box fashion [ 20 ], without considering the contribution that meaningful PPI could make to enhancing the research. It may be that PPI, let alone creative PPI, is not at the forefront of researchers’ minds when planning research.

Strengths of creative PPI

Creative ppi disrupts power hierarchies.

One of the main strengths of creative PPI techniques, cited most frequently in the included literature, was that they disrupt traditional power hierarchies [ 12 , 13 , 17 , 19 , 23 ]. For example, the use of theatre performance blurred the lines between professional and lay roles between the community and policy makers [ 12 ]. Individuals created a monologue to portray how poverty and inequality impact daily life and presented this to representatives of the National Assembly of Wales, Welsh Government, the Local Authority, Arts Council and Westminster. Byrne et al. [ 12 ], states how this medium allowed the community to engage with the people who make decisions about their lives in an environment of respect and understanding, where the hierarchies are not as visible as in other settings, e.g., political surgeries. Creative PPI methods have also removed traditional power hierarchies between researchers and adolescents. Cook et al. [ 13 ], used arts-based approaches to explore adolescents’ ideas about the “perfect” condom. They utilised the “Life Happens” resource, where adolescents drew and then decorated a person with their thoughts about sexual relationships, not too dissimilar from the persona-scenario method. This was then combined with hypothetical scenarios about sexuality. A condom-mapping exercise was then implemented, where groups shared the characteristics that make a condom “perfect” on large pieces of paper. Cook et al. [ 13 ], noted that usually power imbalances make it difficult to elicit information from adolescents, however these power imbalances were reduced due to the use of creative co-design techniques.

The same reduction in power hierarchies was noted by Grindell et al. [ 17 ], who used the person-scenario method and creative worksheets with individuals with malignant pleural effusion. This was with the aim of developing a prototype of a decision support tool for patients to help with treatment options. Although this process involved a variety of stakeholders, such as patients, carers and healthcare professionals, creative co-design was cited as a mechanism that worked to reduce power imbalances – a limitation of more traditional methods of research. Creative co-design blurred boundaries between end-users and clinical staff and enabled the sharing of ideas from multiple, valuable perspectives, meaning the prototype was able to suit user needs whilst addressing clinical problems.

Similarly, a specific creative method named cultural animation was also cited to dissolve hierarchies and encourage equal contributions from participants. Within this arts-based approach, Keleman et al. [ 19 ], explored the concept of “good health” with individuals from Stoke-on Trent. Members of the group created art installations using ribbons, buttons, cardboard and straws to depict their idea of a “healthy community”, which was accompanied by a poem. They also created a 3D Facebook page and produced another poem or song addressing the government to communicate their version of a “picture of health”. Public participants said that they found the process empowering, honest, democratic, valuable and practical.

This dissolving of hierarchies and levelling of power is beneficial as it increases the sense of ownership experienced by the creators/producers of the output [ 12 , 17 , 23 ]. This is advantageous as it has been suggested to improve its quality [ 23 ].

Creative PPI allows the unsayable to be said

Creative PPI fosters a safe space for mundane or taboo topics to be shared, which may be difficult to communicate using traditional methods of PPI. For example, the hypothetical nature of condom mapping and persona-scenarios meant that adolescents could discuss a personal topic without fear of discrimination, judgement or personal disclosure [ 13 ]. The safe space allowed a greater volume of ideas to be generated amongst peers where they might not have otherwise. Similarly, Webber et al. [ 23 ], , who used the persona method to co-design the prototype back pain educational resource, also noted how this method creates anonymity whilst allowing people the opportunity to externalise personal experiences, thoughts and feelings. Other creative methods were also used, such as drawing, collaging, role play and creating mood boards. A cardboard cube (labelled a “magic box”) was used to symbolise a physical representation of their final prototype. These creative methods levelled the playing field and made personal experiences accessible in a safe, open environment that fostered trust, as well as understanding from the researchers.

It is not only sensitive subjects that were made easier to articulate through creative PPI. The communication of mundane everyday experiences were also facilitated, which were deemed typically ‘unsayable’. This was specifically given in the context of describing intangible aspects of everyday health and wellbeing [ 11 ]. Graphic designers can also be used to visually represent the outputs of creative PPI. These captured the movement and fluidity of people and well as the relationships between them - things that cannot be spoken but can be depicted [ 21 ].

Creative PPI methods are inclusive

Another strength of creative PPI was that it is inclusive and accessible [ 17 , 19 , 21 ]. The safe space it fosters, as well as the dismantling of hierarchies, welcomed people from a diverse range of backgrounds and provided equal opportunities [ 21 ], especially for those with communication and memory difficulties who might be otherwise excluded from PPI. Kelemen et al. [ 19 ], who used creative methods to explore health and well-being in Stoke-on-Trent, discussed how people from different backgrounds came together and connected, discussed and reached a consensus over a topic which evoked strong emotions, that they all have in common. Individuals said that the techniques used “sets people to open up as they are not overwhelmed by words”. Similarly, creative activities, such as the persona method, have been stated to allow people to express themselves in an inclusive environment using a common language. Kearns et al. [ 18 ], who used aphasia-accessible material to develop a questionnaire with aphasic individuals, described how they felt comfortable in contributing to workshops (although this material was time-consuming to make, see ‘Limitations of creative PPI’ ).

Despite the general inclusivity of creative PPI, it can also be exclusive, particularly if online mediums are used. Fedorowicz et al. [ 15 ], used Facebook to create a PPI group, and although this may rectify previous drawbacks about lack of generalisation of creative methods (as Facebook can reach a greater number of people, globally), it excluded those who are not digitally active or have limited internet access or knowledge of technology. Online methods have other issues too. Maintaining the online group was cited as challenging and the volume of responses required researchers to interact outside of their working hours. Despite this, online methods like Facebook are very accessible for people who are physically disabled.

Creative PPI methods are engaging

The process of creative PPI is typically more engaging and produces more colourful data than traditional methods [ 13 ]. Individuals are permitted and encouraged to explore a creative self [ 19 ], which can lead to the exploration of new ideas and an overall increased enjoyment of the process. This increased engagement is particularly beneficial for younger PPI groups. For example, to involve children in the development of health food products, Galler et al. [ 16 ] asked 9-12-year-olds to take photos of their food and present it to other children in a “show and tell” fashion. They then created a newspaper article describing a new healthy snack. In this creative focus group, children were given lab coats to further their identity as inventors. Galler et al. [ 16 ], notes that the methods were highly engaging and facilitated teamwork and group learning. This collaborative nature of problem-solving was also observed in adults who used personas and creative worksheets to develop the resource for lower back pain [ 23 ]. Dementia patients too have been reported to enjoy the creative and informal approach to idea generation [ 20 ].

The use of cultural animation allowed people to connect with each other in a way that traditional methods do not [ 19 , 21 ]. These connections were held in place by boundary objects, such as ribbons, buttons, fabric and picture frames, which symbolised a shared meaning between people and an exchange of knowledge and emotion. Asking groups to create an art installation using these objects further fostered teamwork and collaboration, both at an individual and collective level. The exploration of a creative self increased energy levels and encouraged productive discussions and problem-solving [ 19 ]. Objects also encouraged a solution-focused approach and permitted people to think beyond their usual everyday scope [ 17 ]. They also allowed facilitators to probe deeper about the greater meanings carried by the object, which acted as a metaphor [ 21 ].

From the researcher’s point of view, co-creative methods gave rise to ideas they might not have initially considered. Valaitis et al. [ 22 ], found that over 40% of the creative outputs were novel ideas brought to light by patients, healthcare providers/community care providers, community service providers and volunteers. One researcher commented, “It [the creative methods] took me on a journey, in a way that when we do other pieces of research it can feel disconnected” [ 23 ]. Another researcher also stated they could not return to the way they used to do research, as they have learnt so much about their own health and community and how they are perceived [ 19 ]. This demonstrates that creative processes not only benefit the project outcomes and the PPI group, but also facilitators and researchers. However, although engaging, creative methods have been criticised for not demonstrating academic rigour [ 17 ]. Moreover, creative PPI may also be exclusive to people who do not like or enjoy creative activities.

Creative PPI methods are cost and time efficient

Creative PPI workshops can often produce output that is visible and tangible. This can save time and money in the long run as the output is either ready to be implemented in a healthcare setting or a first iteration has already been developed. This may also offset the time and costs it takes to implement creative PPI. For example, the prototype of the decision support tool for people with malignant pleural effusion was developed using personas and creative worksheets. The end result was two tangible prototypes to drive the initial idea forward as something to be used in practice [ 17 ]. The use of creative co-design in this case saved clinician time as well as the time it would take to develop this product without the help of its end-users. In the development of this particular prototype, analysis was iterative and informed the next stage of development, which again saved time. The same applies for the feedback questionnaire for the assessment of ICT delivered aphasia rehabilitation. The co-created questionnaire, designed with people with aphasia, was ready to be used in practice [ 18 ]. This suggests that to overcome time and resource barriers to creative PPI, researchers should aim for it to be engaging whilst also producing output.

That useable products are generated during creative workshops signals to participating patients and public members that they have been listened to and their thoughts and opinions acted upon [ 23 ]. For example, the development of the back pain resource based on patient experiences implies that their suggestions were valid and valuable. Further, those who participated in the cultural animation workshop reported that the process visualises change, and that it already feels as though the process of change has started [ 19 ].

The most cost and time efficient method of creative PPI in this review is most likely the use of Facebook to gather feedback on project methodology [ 15 ]. Although there were drawbacks to this, researchers could involve more people from a range of geographical areas at little to no cost. Feedback was instantaneous and no training was required. From the perspective of the PPI group, they could interact however much or little they wish with no time commitment.

This systematic review identified four limitations and five strengths to the use of creative PPI in health and social care research. Creative PPI is time and resource intensive, can raise ethical issues and lacks generalisability. It is also not accepted by the mainstream. These factors may act as barriers to the implementation of creative PPI. However, creative PPI disrupts traditional power hierarchies and creates a safe space for taboo or mundane topics. It is also engaging, inclusive and can be time and cost efficient in the long term.

Something that became apparent during data analysis was that these are not blanket strengths and limitations of creative PPI as a whole. The umbrella term ‘creative PPI’ is broad and encapsulates a wide range of activities, ranging from music and poems to prototype development and persona-scenarios, to more simplistic things like the use of sticky notes and ordering cards. Many different activities can be deemed ‘creative’ and the strengths and limitations of one does not necessarily apply to another. For example, cultural animation takes greater effort to prepare than the use of sticky notes and sorting cards, and the use of Facebook is cheaper and wider reaching than persona development. Researchers should use their discretion and weigh up the benefits and drawbacks of each method to decide on a technique which suits the project. What might be a limitation to creative PPI in one project may not be in another. In some cases, creative PPI may not be suitable at all.

Furthermore, the choice of creative PPI method also depends on the needs and characteristics of the PPI group. Children, adults and people living with dementia or language difficulties all have different engagement needs and capabilities. This indicates that creative PPI is not one size fits all and that the most appropriate method will change depending on the composition of the group. The choice of method will also be determined by the constraints of the research project, namely time, money and the research aim. For example, if there are time constraints, then a method which yields a lot of data and requires a lot of preparation may not be appropriate. If generalisation is important, then an online method is more suitable. Together this indicates that the choice of creative PPI method is highly individualised and dependent on multiple factors.

Although the limitations discussed in this review apply to creative PPI, they are not exclusive to creative PPI. Ethical issues are a consideration within general PPI research, especially when working with more vulnerable populations, such as children or adults living with a disability. It can also be the case that traditional PPI methods lack generalisability, as people who volunteer to be part of such a group are more likely be older, middle class and retired [ 24 ]. Most research is vulnerable to this type of bias, however, it is worth noting that generalisation is not always a goal and research remains valid and meaningful in its absence. Although online methods may somewhat combat issues related to generalisability, these methods still exclude people who do not have access to the internet/technology or who choose not to use it, implying that online PPI methods may not be wholly representative of the general population. Saying this, however, the accessibility of creative PPI techniques differs from person to person, and for some, online mediums may be more accessible (for example for those with a physical disability), and for others, this might be face-to-face. To combat this, a range of methods should be implemented. Planning multiple focus group and interviews for traditional PPI is also time and resource intensive, however the extra resources required to make this creative may be even greater. Although, the rich data provided may be worth the preparation and analysis time, which is also likely to depend on the number of participants and workshop sessions required. PPI, not just creative PPI, often requires the provision of a financial incentive, refreshments, parking and accommodation, which increase costs. These, however, are imperative and non-negotiable, as they increase the accessibility of research, especially to minority and lower-income groups less likely to participate. Adequate funding is also important for co-design studies where repeated engagement is required. One barrier to implementation, which appears to be exclusive to creative methods, however, is that creative methods are not mainstream. This cannot be said for traditional PPI as this is often a mandatory part of research applications.

Regarding the strengths of creative PPI, it could be argued that most appear to be exclusive to creative methodologies. These are inclusive by nature as multiple approaches can be taken to evoke ideas from different populations - approaches that do not necessarily rely on verbal or written communication like interviews and focus groups do. Given the anonymity provided by some creative methods, such as personas, people may be more likely to discuss their personal experiences under the guise of a general end-user, which might be more difficult to maintain when an interviewer is asking an individual questions directly. Additionally, creative methods are by nature more engaging and interactive than traditional methods, although this is a blanket statement and there may be people who find the question-and-answer/group discussion format more engaging. Creative methods have also been cited to eliminate power imbalances which exist in traditional research [ 12 , 13 , 17 , 19 , 23 ]. These imbalances exist between researchers and policy makers and adolescents, adults and the community. Lastly, although this may occur to a greater extent in creative methods like prototype development, it could be suggested that PPI in general – regardless of whether it is creative - is more time and cost efficient in the long-term than not using any PPI to guide or refine the research process. It must be noted that these are observations based on the literature. To be certain these differences exist between creative and traditional methods of PPI, direct empirical evaluation of both should be conducted.

To the best of our knowledge, this is the first review to identify the strengths and limitations to creative PPI, however, similar literature has identified barriers and facilitators to PPI in general. In the context of clinical trials, recruitment difficulties were cited as a barrier, as well as finding public contributors who were free during work/school hours. Trial managers reported finding group dynamics difficult to manage and the academic environment also made some public contributors feel nervous and lacking confidence to speak. Facilitators, however, included the shared ownership of the research – something that has been identified in the current review too. In addition, planning and the provision of knowledge, information and communication were also identified as facilitators [ 25 ]. Other research on the barriers to meaningful PPI in trial oversight committees included trialist confusion or scepticism over the PPI role and the difficulties in finding PPI members who had a basic understanding of research [ 26 ]. However, it could be argued that this is not representative of the average patient or public member. The formality of oversight meetings and the technical language used also acted as a barrier, which may imply that the informal nature of creative methods and its lack of dependency on literacy skills could overcome this. Further, a review of 42 reviews on PPI in health and social care identified financial compensation, resources, training and general support as necessary to conduct PPI, much like in the current review where the resource intensiveness of creative PPI was identified as a limitation. However, others were identified too, such as recruitment and representativeness of public contributors [ 27 ]. Like in the current review, power imbalances were also noted, however this was included as both a barrier and facilitator. Collaboration seemed to diminish hierarchies but not always, as sometimes these imbalances remained between public contributors and healthcare staff, described as a ‘them and us’ culture [ 27 ]. Although these studies compliment the findings of the current review, a direct comparison cannot be made as they do not concern creative methods. However, it does suggest that some strengths and weaknesses are shared between creative and traditional methods of PPI.

Strengths and limitations of this review

Although a general definition of creative PPI exists, it was up to our discretion to decide exactly which activities were deemed as such for this review. For example, we included sorting cards, the use of interactive whiteboards and sticky notes. Other researchers may have a more or less stringent criteria. However, two reviewers were involved in this decision which aids the reliability of the included articles. Further, it may be that some of the strengths and limitations cannot fully be attributed to the creative nature of the PPI process, but rather their co-created nature, however this is hard to disentangle as the included papers involved both these aspects.

During screening, it was difficult to decide whether the article was utilising creative qualitative methodology or creative PPI , as it was often not explicitly labelled as such. Regardless, both approaches involved the public/patients refining a healthcare product/service. This implies that if this review were to be replicated, others may do it differently. This may call for greater standardisation in the reporting of the public’s involvement in research. For example, the NIHR outlines different approaches to PPI, namely “consultation”, “collaboration”, “co-production” and “user-controlled”, which each signify an increased level of public power and influence [ 28 ]. Papers with elements of PPI could use these labels to clarify the extent of public involvement, or even explicitly state that there was no PPI. Further, given our decision to include only scholarly peer-reviewed literature, it is possible that data were missed within the grey literature. Similarly, the literature search will not have identified all papers relating to different types of accessible inclusion. However, the intent of the review was to focus solely on those within the definition of creative.

This review fills a gap in the literature and helps circulate and promote the concept of creative PPI. Each stage of this review, namely screening and quality appraisal, was conducted by two independent reviewers. However, four full texts could not be accessed during the full text reading stage, meaning there are missing data that could have altered or contributed to the findings of this review.

Research recommendations

Given that creative PPI can require effort to prepare, perform and analyse, sufficient time and funding should be allocated in the research protocol to enable meaningful and continuous PPI. This is worthwhile as PPI can significantly change the research output so that it aligns closely with the needs of the group it is to benefit. Researchers should also consider prototype development as a creative PPI activity as this might reduce future time/resource constraints. Shifting from a top-down approach within research to a bottom-up can be advantageous to all stakeholders and can help move creative PPI towards the mainstream. This, however, is the collective responsibility of funding bodies, universities and researchers, as well as committees who approve research bids.

A few of the included studies used creative techniques alongside traditional methods, such as interviews, which could also be used as a hybrid method of PPI, perhaps by researchers who are unfamiliar with creative techniques or to those who wish to reap the benefits of both. Often the characteristics of the PPI group were not included, including age, gender and ethnicity. It would be useful to include such information to assess how representative the PPI group is of the population of interest.

Creative PPI is a relatively novel approach of engaging the public and patients in research and it has both advantages and disadvantages compared to more traditional methods. There are many approaches to implementing creative PPI and the choice of technique will be unique to each piece of research and is reliant on several factors. These include the age and ability of the PPI group as well as the resource limitations of the project. Each method has benefits and drawbacks, which should be considered at the protocol-writing stage. However, given adequate funding, time and planning, creative PPI is a worthwhile and engaging method of generating ideas with end-users of research – ideas which may not be otherwise generated using traditional methods.

Data availability

No datasets were generated or analysed during the current study.

Abbreviations

Critical Appraisal Skills Programme

The Joanna Briggs Institute

National Institute of Health and Care Research

Public Advisory Group

Public and Patient Involvement

Web of Science

National Institute for Health and Care Research. What Is Patient and Public Involvement and Public Engagement? https://www.spcr.nihr.ac.uk/PPI/what-is-patient-and-public-involvement-and-engagement Accessed 01 Sept 2023.

Department of Health. Personal and Public Involvement (PPI) https://www.health-ni.gov.uk/topics/safety-and-quality-standards/personal-and-public-involvement-ppi#:~:text=The Health and Social Care Reform Act (NI) 2009 placed,delivery and evaluation of services . Accessed 01 Sept 2023.

National Institute for Health and Care Research. Policy Research Programme – Guidance for Stage 1 Applications https://www.nihr.ac.uk/documents/policy-research-programme-guidance-for-stage-1-applications-updated/26398 Accessed 01 Sept 2023.

Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, Chant A. Frameworks for supporting patient and public involvement in research: systematic review and co-design pilot. Health Expect. 2019. https://doi.org/10.1111/hex.12888

Article   PubMed   PubMed Central   Google Scholar  

Street JM, Stafinski T, Lopes E, Menon D. Defining the role of the public in health technology assessment (HTA) and HTA-informed decision-making processes. Int J Technol Assess Health Care. 2020. https://doi.org/10.1017/S0266462320000094

Article   PubMed   Google Scholar  

Morrison C, Dearden A. Beyond tokenistic participation: using representational artefacts to enable meaningful public participation in health service design. Health Policy. 2013. https://doi.org/10.1016/j.healthpol.2013.05.008

Leavy P. Method meets art: arts-Based Research Practice. New York: Guilford; 2020.

Google Scholar  

Seers K. Qualitative systematic reviews: their importance for our understanding of research relevant to pain. Br J Pain. 2015. https://doi.org/10.1177/2049463714549777

Lockwood C, Porritt K, Munn Z, Rittenmeyer L, Salmond S, Bjerrum M, Loveday H, Carrier J, Stannard D. Chapter 2: Systematic reviews of qualitative evidence. Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis JBI. 2020. https://synthesismanual.jbi.global . https://doi.org/10.46658/JBIMES-20-03

CASP. CASP Checklists https://casp-uk.net/images/checklist/documents/CASP-Qualitative-Studies-Checklist/CASP-Qualitative-Checklist-2018_fillable_form.pdf (2022).

Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Res Psychol. 2006. https://doi.org/10.1191/1478088706qp063oa

Article   Google Scholar  

Byrne E, Elliott E, Saltus R, Angharad J. The creative turn in evidence for public health: community and arts-based methodologies. J Public Health. 2018. https://doi.org/10.1093/pubmed/fdx151

Cook S, Grozdanovski L, Renda G, Santoso D, Gorkin R, Senior K. Can you design the perfect condom? Engaging young people to inform safe sexual health practice and innovation. Sex Educ. 2022. https://doi.org/10.1080/14681811.2021.1891040

Craven MP, Goodwin R, Rawsthorne M, Butler D, Waddingham P, Brown S, Jamieson M. Try to see it my way: exploring the co-design of visual presentations of wellbeing through a workshop process. Perspect Public Health. 2019. https://doi.org/10.1177/1757913919835231

Fedorowicz S, Riley V, Cowap L, Ellis NJ, Chambers R, Grogan S, Crone D, Cottrell E, Clark-Carter D, Roberts L, Gidlow CJ. Using social media for patient and public involvement and engagement in health research: the process and impact of a closed Facebook group. Health Expect. 2022. https://doi.org/10.1111/hex.13515

Galler M, Myhrer K, Ares G, Varela P. Listening to children voices in early stages of new product development through co-creation – creative focus group and online platform. Food Res Int. 2022. https://doi.org/10.1016/j.foodres.2022.111000

Grindell C, Tod A, Bec R, Wolstenholme D, Bhatnagar R, Sivakumar P, Morley A, Holme J, Lyons J, Ahmed M, Jackson S, Wallace D, Noorzad F, Kamalanathan M, Ahmed L, Evison M. Using creative co-design to develop a decision support tool for people with malignant pleural effusion. BMC Med Inf Decis Mak. 2020. https://doi.org/10.1186/s12911-020-01200-3

Kearns Á, Kelly H, Pitt I. Rating experience of ICT-delivered aphasia rehabilitation: co-design of a feedback questionnaire. Aphasiology. 2020. https://doi.org/10.1080/02687038.2019.1649913

Kelemen M, Surman E, Dikomitis L. Cultural animation in health research: an innovative methodology for patient and public involvement and engagement. Health Expect. 2018. https://doi.org/10.1111/hex.12677

Keogh F, Carney P, O’Shea E. Innovative methods for involving people with dementia and carers in the policymaking process. Health Expect. 2021. https://doi.org/10.1111/hex.13213

Micsinszki SK, Buettgen A, Mulvale G, Moll S, Wyndham-West M, Bruce E, Rogerson K, Murray-Leung L, Fleisig R, Park S, Phoenix M. Creative processes in co-designing a co-design hub: towards system change in health and social services in collaboration with structurally vulnerable populations. Evid Policy. 2022. https://doi.org/10.1332/174426421X16366319768599

Valaitis R, Longaphy J, Ploeg J, Agarwal G, Oliver D, Nair K, Kastner M, Avilla E, Dolovich L. Health TAPESTRY: co-designing interprofessional primary care programs for older adults using the persona-scenario method. BMC Fam Pract. 2019. https://doi.org/10.1186/s12875-019-1013-9

Webber R, Partridge R, Grindell C. The creative co-design of low back pain education resources. Evid Policy. 2022. https://doi.org/10.1332/174426421X16437342906266

National Institute for Health and Care Research. A Researcher’s Guide to Patient and Public Involvement. https://oxfordbrc.nihr.ac.uk/wp-content/uploads/2017/03/A-Researchers-Guide-to-PPI.pdf Accessed 01 Nov 2023.

Selman L, Clement C, Douglas M, Douglas K, Taylor J, Metcalfe C, Lane J, Horwood J. Patient and public involvement in randomised clinical trials: a mixed-methods study of a clinical trials unit to identify good practice, barriers and facilitators. Trials. 2021 https://doi.org/10.1186/s13063-021-05701-y

Coulman K, Nicholson A, Shaw A, Daykin A, Selman L, Macefield R, Shorter G, Cramer H, Sydes M, Gamble C, Pick M, Taylor G, Lane J. Understanding and optimising patient and public involvement in trial oversight: an ethnographic study of eight clinical trials. Trials. 2020. https://doi.org/10.1186/s13063-020-04495-9

Ocloo J, Garfield S, Franklin B, Dawson S. Exploring the theory, barriers and enablers for patient and public involvement across health, social care and patient safety: a systematic review of reviews. Health Res Policy Sys. 2021. https://doi.org/10.1186/s12961-020-00644-3

National Institute for Health and Care Research. Briefing notes for researchers - public involvement in NHS, health and social care research. https://www.nihr.ac.uk/documents/briefing-notes-for-researchers-public-involvement-in-nhs-health-and-social-care-research/27371 Accessed 01 Nov 2023.

Download references

Acknowledgements

With thanks to the PHIRST-LIGHT public advisory group and consortium for their thoughts and contributions to the design of this work.

The research team is supported by a National Institute for Health and Care Research grant (PHIRST-LIGHT Reference NIHR 135190).

Author information

Olivia R. Phillips and Cerian Harries share joint first authorship.

Authors and Affiliations

Nottingham Centre for Public Health and Epidemiology, Lifespan and Population Health, School of Medicine, University of Nottingham, Clinical Sciences Building, City Hospital Campus, Hucknall Road, Nottingham, NG5 1PB, UK

Olivia R. Phillips, Jo Leonardi-Bee, Holly Knight & Joanne R. Morling

National Institute for Health and Care Research (NIHR) PHIRST-LIGHT, Nottingham, UK

Olivia R. Phillips, Cerian Harries, Jo Leonardi-Bee, Holly Knight, Lauren B. Sherar, Veronica Varela-Mato & Joanne R. Morling

School of Sport, Exercise and Health Sciences, Loughborough University, Epinal Way, Loughborough, Leicestershire, LE11 3TU, UK

Cerian Harries, Lauren B. Sherar & Veronica Varela-Mato

Nottingham Centre for Evidence Based Healthcare, School of Medicine, University of Nottingham, Nottingham, UK

Jo Leonardi-Bee

NIHR Nottingham Biomedical Research Centre (BRC), Nottingham University Hospitals NHS Trust, University of Nottingham, Nottingham, NG7 2UH, UK

Joanne R. Morling

You can also search for this author in PubMed   Google Scholar

Contributions

Author contributions: study design: ORP, CH, JRM, JLB, HK, LBS, VVM, literature searching and screening: ORP, CH, JRM, data curation: ORP, CH, analysis: ORP, CH, JRM, manuscript draft: ORP, CH, JRM, Plain English Summary: ORP, manuscript critical review and editing: ORP, CH, JRM, JLB, HK, LBS, VVM.

Corresponding author

Correspondence to Olivia R. Phillips .

Ethics declarations

Ethics approval and consent to participate.

The Ethics Committee of the Faculty of Medicine and Health Sciences, University of Nottingham advised that approval from the ethics committee and consent to participate was not required for systematic review studies.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

40900_2024_580_MOESM1_ESM.docx

Additional file 1: Search strings: Description of data: the search strings and filters used in each of the 5 databases in this review

Additional file 2: Quality appraisal questions: Description of data: CASP quality appraisal questions

40900_2024_580_moesm3_esm.docx.

Additional file 3: Table 1: Description of data: elements of the data extraction table that are not in the main manuscript

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Phillips, O.R., Harries, C., Leonardi-Bee, J. et al. What are the strengths and limitations to utilising creative methods in public and patient involvement in health and social care research? A qualitative systematic review. Res Involv Engagem 10 , 48 (2024). https://doi.org/10.1186/s40900-024-00580-4

Download citation

Received : 28 November 2023

Accepted : 25 April 2024

Published : 13 May 2024

DOI : https://doi.org/10.1186/s40900-024-00580-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Public and patient involvement
• Creative PPI
• Qualitative systematic review

Research Involvement and Engagement

ISSN: 2056-7529

• General enquiries: [email protected]

1. Introduction

2. results and discussion, 3. conclusions, 4. related literature, supporting information.

research papers \(\def\hfill{\hskip 5em}\def\hfil{\hskip 3em}\def\eqno#1{\hfil {#1}}\)

Analysis of COF-300 synthesis: probing degradation processes and 3D electron diffraction structure

a XStruct, Department of Chemistry, Ghent University, Krijgslaan 281–S3, 9000 Ghent, Belgium, b COMOC – Center for Ordered Materials, Organometallics and Catalysis – Department of Chemistry, Ghent University, Krijgslaan 281–S3, 9000 Ghent, Belgium, c Rigaku Corporation, Haijima, Tokyo, Japan, and d Rigaku Europe SE, Neu-Isenburg, Germany * Correspondence e-mail: [email protected]

Although COF-300 is often used as an example to study the synthesis and structure of (3D) covalent organic frameworks (COFs), knowledge of the underlying synthetic processes is still fragmented. Here, an optimized synthetic procedure based on a combination of linker protection and modulation was applied. Using this approach, the influence of time and temperature on the synthesis of COF-300 was studied. Synthesis times that were too short produced materials with limited crystallinity and porosity, lacking the typical pore flexibility associated with COF-300. On the other hand, synthesis times that were too long could be characterized by loss of crystallinity and pore order by degradation of the tetrakis(4-aminophenyl)methane (TAM) linker used. The presence of the degradation product was confirmed by visual inspection, Raman spectroscopy and X-ray photoelectron spectroscopy (XPS). As TAM is by far the most popular linker for the synthesis of 3D COFs, this degradation process might be one of the reasons why the development of 3D COFs is still lagging compared with 2D COFs. However, COF crystals obtained via an optimized procedure could be structurally probed using 3D electron diffraction (3DED). The 3DED analysis resulted in a full structure determination of COF-300 at atomic resolution with satisfying data parameters. Comparison of our 3DED-derived structural model with previously reported single-crystal X-ray diffraction data for this material, as well as parameters derived from the Cambridge Structural Database, demonstrates the high accuracy of the 3DED method for structure determination. This validation might accelerate the exploitation of 3DED as a structure determination technique for COFs and other porous materials.

Keywords: 3D electron diffraction ; 3DED ; microcrystal electron diffraction ; microED ; covalent organic frameworks ; Cambridge Structural Database ; porous organic solids ; crystallization and crystal growth .

CCDC reference: 2321626

In Figs. S6–S8, the effect of time on I 65°C (as discussed earlier) is compared with the effect on the other samples. Note that, as expected, I RT is slower to form a crystalline material compared with I 65°C due to the reduced error correction at room temperature, with reflections appearing after 1 d, and fully developed crystallinity after 5 d. However, the pore structure never fully establishes, as indicated by the broad, late and small second step in the N 2 -sorption isotherm. The appearance of crystallinity in C 65°C is even more delayed, with no crystalline reflections observed after 1 d of reaction time, indicating the superiority of the intermediate-assisted procedure. Here, maximal crystallinity is observed after 5 d, as peaks start to broaden significantly after 7 d. Surprisingly, the best N 2 -sorption behaviour was observed for the 7 d sample, indicating that the relationship between crystallinity and porosity is not always straightforward. Finally, using the conditions of C RT, we were unable to form any crystalline material, even after 7 d of reaction time. We also checked if the scale of the synthesis had any influence on the material. Therefore, a sample (I 65°C ×5) was prepared in an identical way to I 65°C but with every quantity used multiplied by 5. The resulting PXRD patterns and the N 2 -sorption isotherms are presented in Fig. S9 and show no significant influence on the crystallinity and a small decrease of porosity (with a BET surface area of 1180 m 2  g −1 and V p of 0.71 obtained for I 65°C ×5).

The response of COF-300 to an intermediate-assisted synthesis protocol was studied by careful evaluation of the evolution of both crystallinity and porosity as functions of reaction time and temperature. Kinetic studies among four different synthesis conditions revealed three distinct stages in the synthesis of COF-300, namely a network build-up phase at short synthetic times (≤1 d) with low crystallinity and no pore flexibility, followed by an optimal stage (3 d) characterized by high crystallinity and porosity before partial breakdown by TAM degradation (≥5 d). This degradation process could be confirmed in both control experiments as well as the obtained COF materials and can easily be estimated by the observation of magenta-coloured reaction mixtures. As a pronounced influence of this degradation reaction on both crystallinity and porosity was observed and most 3D COFs are based on the TAM linker, knowledge of TAM degradation in a acidic environment is of utmost importance for the synthesis of high-quality 3D COFs. Knowledge of this degradation process might help to increase the synthetic toolbox for 3D COFs (which are mainly based on the TAM linker), which is still lacking compared with 2D COFs. However, using the optimized conditions, a reliable crystal structure of COF-300 could be readily obtained via 3DED analysis, indicating single crystallinity of the synthesized materials. The structure model obtained showed high completeness and comparable resolution and R values. Comparison with an SCXRD structure model as well as with data for similar chemical functionalities in the CSD database showed no significant differences, supporting that 3DED is a reliable and fast technique for the structure solution of COFs. As SCXRD structure solution is hardly possible and PXRD models often show ambiguity in structure determination, 3DED might play an important role in the future of COFs with better accessibility of 3DED diffraction equipment and improving dynamic refinement algorithms.

Crystal structure: contains datablock 1. DOI: https://doi.org/10.1107/S2052252524003713/vq5005sup1.cif

Structure factors: contains datablock 1. DOI: https://doi.org/10.1107/S2052252524003713/vq5005sup2.hkl

Supporting Information - revised - highlighted. DOI: https://doi.org/10.1107/S2052252524003713/vq5005sup3.pdf

Acknowledgements

The authors thank Karen Leus for the XPS measurements and Dieter Buyst for the solid-state NMR measurements.

Funding information

LB acknowledges Ghent University (UGent) for funding. PVDV acknowledges financial support through UGent concerted action (grant No. 01G01017) and the Fonds Wetenschappelijk Onderzoek (FWO)–Vlaanderen project (grant Nos. 3G020521 awarded to PVDV; 1275221N awarded to SB and KVH). Gas sorption and powder X-ray diffraction were made possible through UGent (grant Nos. 01B00215; BOF20/BAS/015 awarded to PVDV). The spectrometer electronics, magnet and accessories used for solid-state NMR measurements, including the BBI and high-gradient diffusion probe, were funded by the Hercules foundation (grant No. AUGE/09/2006); the solid-state (CP-MAS) and HR-MAS expansion were made possible by FWO (grant No. I006920N).

This is an open-access article distributed under the terms of the Creative Commons Attribution (CC-BY) Licence , which permits unrestricted use, distribution, and reproduction in any medium, provided the original authors and source are cited.

• Open access
• Published: 13 May 2024

Sexual and reproductive health implementation research in humanitarian contexts: a scoping review

• Alexandra Norton 1 &
• Hannah Tappis 2  

Reproductive Health volume  21 , Article number:  64 ( 2024 ) Cite this article

Metrics details

Meeting the health needs of crisis-affected populations is a growing challenge, with 339 million people globally in need of humanitarian assistance in 2023. Given one in four people living in humanitarian contexts are women and girls of reproductive age, sexual and reproductive health care is considered as essential health service and minimum standard for humanitarian response. Despite growing calls for increased investment in implementation research in humanitarian settings, guidance on appropriate methods and analytical frameworks is limited.

A scoping review was conducted to examine the extent to which implementation research frameworks have been used to evaluate sexual and reproductive health interventions in humanitarian settings. Peer-reviewed papers published from 2013 to 2022 were identified through relevant systematic reviews and a literature search of Pubmed, Embase, PsycInfo, CINAHL and Global Health databases. Papers that presented primary quantitative or qualitative data pertaining to a sexual and reproductive health intervention in a humanitarian setting were included.

Seven thousand thirty-six unique records were screened for inclusion, and 69 papers met inclusion criteria. Of these, six papers explicitly described the use of an implementation research framework, three citing use of the Consolidated Framework for Implementation Research. Three additional papers referenced other types of frameworks used in their evaluation. Factors cited across all included studies as helping the intervention in their presence or hindering in their absence were synthesized into the following Consolidated Framework for Implementation Research domains: Characteristics of Systems, Outer Setting, Inner Setting, Characteristics of Individuals, Intervention Characteristics, and Process.

This review found a wide range of methodologies and only six of 69 studies using an implementation research framework, highlighting an opportunity for standardization to better inform the evidence for and delivery of sexual and reproductive health interventions in humanitarian settings. Increased use of implementation research frameworks such as a modified Consolidated Framework for Implementation Research could work toward both expanding the evidence base and increasing standardization.

Plain English summary

Three hundred thirty-nine million people globally were in need of humanitarian assistance in 2023, and meeting the health needs of crisis-affected populations is a growing challenge. One in four people living in humanitarian contexts are women and girls of reproductive age, and provision of sexual and reproductive health care is considered to be essential within a humanitarian response. Implementation research can help to better understand how real-world contexts affect health improvement efforts. Despite growing calls for increased investment in implementation research in humanitarian settings, guidance on how best to do so is limited. This scoping review was conducted to examine the extent to which implementation research frameworks have been used to evaluate sexual and reproductive health interventions in humanitarian settings. Of 69 papers that met inclusion criteria for the review, six of them explicitly described the use of an implementation research framework. Three used the Consolidated Framework for Implementation Research, a theory-based framework that can guide implementation research. Three additional papers referenced other types of frameworks used in their evaluation. This review summarizes how factors relevant to different aspects of implementation within the included papers could have been organized using the Consolidated Framework for Implementation Research. The findings from this review highlight an opportunity for standardization to better inform the evidence for and delivery of sexual and reproductive health interventions in humanitarian settings. Increased use of implementation research frameworks such as a modified Consolidated Framework for Implementation Research could work toward both expanding the evidence base and increasing standardization.

Peer Review reports

Over the past few decades, the field of public health implementation research (IR) has grown as a means by which the real-world conditions affecting health improvement efforts can be better understood. Peters et al. put forward the following broad definition of IR for health: “IR is the scientific inquiry into questions concerning implementation – the act of carrying an intention into effect, which in health research can be policies, programmes, or individual practices (collectively called interventions)” [ 1 ].

As IR emphasizes real-world circumstances, the context within which a health intervention is delivered is a core consideration. However, much IR implemented to date has focused on higher-resource settings, with many proposed frameworks developed with particular utility for a higher-income setting [ 2 ]. In recognition of IR’s potential to increase evidence across a range of settings, there have been numerous reviews of the use of IR in lower-resource settings as well as calls for broader use [ 3 , 4 ]. There have also been more focused efforts to modify various approaches and frameworks to strengthen the relevance of IR to low- and middle-income country settings (LMICs), such as the work by Means et al. to adapt a specific IR framework for increased utility in LMICs [ 2 ].

Within LMIC settings, the centrality of context to a health intervention’s impact is of particular relevance in humanitarian settings, which present a set of distinct implementation challenges [ 5 ]. Humanitarian responses to crisis situations operate with limited resources, under potential security concerns, and often under pressure to relieve acute suffering and need [ 6 ]. Given these factors, successful implementation of a particular health intervention may require different qualities than those that optimize intervention impact under more stable circumstances [ 7 ]. Despite increasing recognition of the need for expanded evidence of health interventions in humanitarian settings, the evidence base remains limited [ 8 ]. Furthermore, despite its potential utility, there is not standardized guidance on IR in humanitarian settings, nor are there widely endorsed recommendations for the frameworks best suited to analyze implementation in these settings.

Sexual and reproductive health (SRH) is a core aspect of the health sector response in humanitarian settings [ 9 ]. Yet, progress in addressing SRH needs has lagged far behind other services because of challenges related to culture and ideology, financing constraints, lack of data and competing priorities [ 10 ]. The Minimum Initial Service Package (MISP) for SRH in Crisis Situations is the international standard for the minimum set of SRH services that should be implemented in all crisis situations [ 11 ]. However, as in other areas of health, there is need for expanded evidence for planning and implementation of SRH interventions in humanitarian settings. Recent systematic reviews of SRH in humanitarian settings have focused on the effectiveness of interventions and service delivery strategies, as well as factors affecting utilization, but have not detailed whether IR frameworks were used [ 12 , 13 , 14 , 15 ]. There have also been recent reviews examining IR frameworks used in various settings and research areas, but none have explicitly focused on humanitarian settings [ 2 , 16 ].

Given the need for an expanded evidence base for SRH interventions in humanitarian settings and the potential for IR to be used to expand the available evidence, a scoping review was undertaken. This scoping review sought to identify IR approaches that have been used in the last ten years to evaluate SRH interventions in humanitarian settings.

This review also sought to shed light on whether there is a need for a common framework to guide research design, analysis, and reporting for SRH interventions in humanitarian settings and if so, if there are any established frameworks already in use that would be fit-for-purpose or could be tailored to meet this need.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for scoping reviews was utilized to guide the elements of this review [ 17 ]. The review protocol was retrospectively registered with the Open Science Framework ( https://osf.io/b5qtz ).

Search strategy

A two-fold search strategy was undertaken for this review, which covered the last 10 years (2013–2022). First, recent systematic reviews pertaining to research or evaluation of SRH interventions in humanitarian settings were identified through keyword searches on PubMed and Google Scholar. Four relevant systematic reviews were identified [ 12 , 13 , 14 , 15 ] Table 1 .

Second, a literature search mirroring these reviews was conducted to identify relevant papers published since the completion of searches for the most recent review (April 2017). Additional file 1 includes the search terms that were used in the literature search [see Additional file 1 ].

The literature search was conducted for papers published from April 2017 to December 2022 in the databases that were searched in one or more of the systematic reviews: PubMed, Embase, PsycInfo, CINAHL and Global Health. Searches were completed in January 2023 Table 2 .

Two reviewers screened each identified study for alignment with inclusion criteria. Studies in the four systematic reviews identified were considered potentially eligible if published during the last 10 years. These papers then underwent full-text review to confirm satisfaction of all inclusion criteria, as inclusion criteria were similar but not fully aligned across the four reviews.

Literature search results were exported into a citation manager (Covidence), duplicates were removed, and a step-wise screening process for inclusion was applied. First, all papers underwent title and abstract screening. The remaining papers after abstract screening then underwent full-text review to confirm satisfaction of all inclusion criteria. Title and abstract screening as well as full-text review was conducted independently by both authors; disagreements after full-text review were resolved by consensus.

Data extraction and synthesis

The following content areas were summarized in Microsoft Excel for each paper that met inclusion criteria: publication details including author, year, country, setting [rural, urban, camp, settlement], population [refugees, internally displaced persons, general crisis-affected], crisis type [armed conflict, natural disaster], crisis stage [acute, chronic], study design, research methods, SRH intervention, and intervention target population [specific beneficiaries of the intervention within the broader population]; the use of an IR framework; details regarding the IR framework, how it was used, and any rationale given for the framework used; factors cited as impacting SRH interventions, either positively or negatively; and other key findings deemed relevant to this review.

As the focus of this review was on the approach taken for SRH intervention research and evaluation, the quality of the studies themselves was not assessed.

Twenty papers underwent full-text review due to their inclusion in one or more of the four systematic reviews and meeting publication date inclusion criteria. The literature search identified 7,016 unique papers. After full-text screening, 69 met all inclusion criteria and were included in the review. Figure  1 illustrates the search strategy and screening process.

Flow chart of paper identification

Papers published in each of the 10 years of the review timeframe (2013–2022) were included. 29% of the papers originated from the first five years of the time frame considered for this review, with the remaining 71% papers coming from the second half. Characteristics of included publications, including geographic location, type of humanitarian crisis, and type of SRH intervention, are presented in Table  3 .

A wide range of study designs and methods were used across the papers, with both qualitative and quantitative studies well represented. Twenty-six papers were quantitative evaluations [ 18 , 19 , 20 , 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 ], 17 were qualitative [ 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 ], and 26 used mixed methods [ 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 , 80 , 81 , 82 , 83 , 84 , 85 , 86 ]. Within the quantitative evaluations, 15 were observational, while five were quasi-experimental, five were randomized controlled trials, and one was an economic evaluation. Study designs as classified by the authors of this review are summarized in Table  4 .

Six papers (9%) explicitly cited use of an IR framework. Three of these papers utilized the Consolidated Framework for Implementation Research (CFIR) [ 51 , 65 , 70 ]. The CFIR is a commonly used determinant framework that—in its originally proposed form in 2009—is comprised of five domains, each of which has constructs to further categorize factors that impact implementation. The CFIR domains were identified as core content areas influencing the effectiveness of implementation, and the constructs within each domain are intended to provide a range of options for researchers to select from to “guide diagnostic assessments of implementation context, evaluate implementation progress, and help explain findings.” [ 87 ] To allow for consistent terminology throughout this review, the original 2009 CFIR domains and constructs are used.

Guan et al. conducted a mixed methods study to assess the feasibility and effectiveness of a neonatal hepatitis B immunization program in a conflict-affected rural region of Myanmar. Guan et al. report mapping data onto the CFIR as a secondary analysis step. They describe that “CFIR was used as a comprehensive meta-theoretical framework to examine the implementation of the Hepatitis B Virus vaccination program,” and implementation themes from multiple study data sources (interviews, observations, examination of monitoring materials) were mapped onto CFIR constructs. They report their results in two phases – Pre-implementation training and community education, and Implementation – with both anchored in themes that they had mapped onto CFIR domains and constructs. All but six constructs were included in their analysis, with a majority summarized in a table and key themes explored further in the narrative text. They specify that most concerns were identified within the Outer Setting and Process domains, while elements identified within the Inner Setting domain provided strength to the intervention and helped mitigate against barriers [ 70 ].

Sarker et al. conducted a qualitative study to assess provision of maternal, newborn and child health services to Rohingya refugees residing in camps in Cox’s Bazar, Bangladesh. They cite using CFIR as a guide for thematic analysis, applying it after a process of inductive and deductive coding to index these codes into the CFIR domains. They utilized three of the five CFIR domains (Outer Setting, Inner Setting, and Process), stating that the remaining two domains (Intervention Characteristics and Characteristics of Individuals) were not relevant to their analysis. They then proposed two additional CFIR domains, Context and Security, for use in humanitarian contexts. In contrast to Guan et al., CFIR constructs are not used nor mentioned by Sarker et al., with content under each domain instead synthesized as challenges and potential solutions. Regarding the CFIR, Sarker et al. write, “The CFIR guided us for interpretative coding and creating the challenges and possible solutions into groups for further clarification of the issues related to program delivery in a humanitarian crisis setting.” [ 51 ]

Sami et al. conducted a mixed methods case study to assess the implementation of a package of neonatal interventions at health facilities within refugee and internally displaced persons camps in South Sudan. They reference use of the CFIR earlier in the study than Sarker et al., basing their guides for semi-structured focus group discussions on the CFIR framework. They similarly reference a general use of the CFIR framework as they conducted thematic analysis. Constructs are referenced once, but they do not specify whether their application of the CFIR framework included use of domains, constructs, or both. This may be in part because they then applied an additional framework, the World Health Organization (WHO) Health System Framework, to present their findings. They describe a nested approach to their use of these frameworks: “Exploring these [CFIR] constructs within the WHO Health Systems Framework can identify specific entry points to improve the implementation of newborn interventions at critical health system building blocks.” [ 65 ]

Three papers cite use of different IR frameworks. Bolan et al. utilized the Theoretical Domains Framework in their mixed methods feasibility study and pilot cluster randomized trial evaluating pilot use of the Safe Delivery App by maternal and newborn health workers providing basic emergency obstetric and newborn care in facilities in the conflict-affected Maniema province of the Democratic Republic of the Congo (DRC). They used the Theroetical Domains Framework in designing interview questions, and further used it as the coding framework for their analysis. Similar to the CFIR, the Theoretical Domains Framework is a determinant framework that consists of domains, each of which then includes constructs. Bolan et al. utilized the Theoretical Domains Framework at the construct level in interview question development and at the domain level in their analysis, mapping interview responses to eight of the 14 domains [ 83 ]. Berg et al. report using an “exploratory design guided by the principles of an evaluation framework” developed by the Medical Research Council to analyze the implementation process, mechanisms of impact, and outcomes of a three-pillar training intervention to improve maternal and neonatal healthcare in the conflict-affected South Kivu province of the DRC [ 67 , 88 ]. Select components of this evaluation framework were used to guide deductive analysis of focus group discussions and in-depth interviews [ 67 ]. In their study of health workers’ knowledge and attitudes toward newborn health interventions in South Sudan, before and after training and supply provision, Sami et al. report use of the Conceptual Framework of the Role of Attitudes in Evidence-Based Practice Implementation in their analysis process. The framework was used to group codes following initial inductive coding analysis of in-depth interviews [ 72 ].

Three other papers cite use of specific frameworks in their intervention evaluation [ 19 , 44 , 76 ]. As a characteristic of IR is the use of an explicit framework to guide the research, the use of the frameworks in these three papers meets the intention of IR and serves the purpose that an IR framework would have in strengthening the analytical rigor. Castle et al. cite use of their program’s theory of change as a framework for a mixed methods evaluation of the provision of family planning services and more specifically uptake of long-acting reversible contraception use in the DRC. They describe use of the theory of change to “enhance effectiveness of [long-acting reversible contraception] access and uptake.” [ 76 ] Thommesen et al. cite use of the AAAQ (Availability, Accessibility, Acceptability and Quality) framework in their qualitative study assessing midwifery services provided to pregnant women in Afghanistan. This framework is focused on the “underlying elements needed for attainment of optimum standard of health care,” but the authors used it in this paper to evaluate facilitators and barriers to women accessing midwifery services [ 44 ]. Jarrett et al. cite use of the Centers for Disease Control and Prevention’s (CDC) Guidelines for Evaluating Public Health Surveillance Systems to explore the characteristics of a population mobility, mortality and birth surveillance system in South Kivu, DRC. Use of these CDC guidelines is cited as one of four study objectives, and commentary is included in the Results section pertaining to each criteria within these guidelines, although more detail regarding use of these guidelines or the authors’ experience with their use in the study is not provided [ 19 ].

Overall, 22 of the 69 papers either explicitly or implicitly identified IR as relevant to their work. Nineteen papers include a focus on feasibility (seven of which did not otherwise identify the importance of exploring questions concerning implementation), touching on a common outcome of interest in implementation research [ 89 ].

While a majority of papers did not explicitly or implicitly use an IR framework to evaluate their SRH intervention of focus, most identified factors that facilitated implementation when they were present or served as a barrier when absent. Sixty cite factors that served as facilitators and 49 cite factors that served as barriers, with just three not citing either. Fifty-nine distinct factors were identified across the papers.

Three of the six studies that explicitly used an IR framework used the CFIR, and the CFIR is the only IR framework that was used by multiple studies. As previously mentioned, Means et al. put forth an adaptation of the CFIR to increase its relevance in LMIC settings, proposing a sixth domain (Characteristics of Systems) and 11 additional constructs [ 2 ]. Using the expanded domains and constructs as proposed by Means et al., the 59 factors cited by papers in this review were thematically grouped into the six domains: Characteristics of Systems, Outer Setting, Inner Setting, Characteristics of Individuals, Intervention Characteristics, and Process. Within each domain, alignment with CFIR constructs was assessed for, and alignment was found with 29 constructs: eight of Means et al.’s 11 constructs, and 21 of the 39 standard CFIR constructs. Three factors did not align with any construct (all fitting within the Outer Setting domain), and 14 aligned with a construct label but not the associated definition. Table 5 synthesizes the mapping of factors affecting SRH intervention implementation to CFIR domains and constructs, with the construct appearing in italics if it is considered to align with that factor by label but not by definition.

Table 6 lists the CFIR constructs that were not found to have alignment with any factor cited by the papers in this review.

This scoping review sought to assess how IR frameworks have been used to bolster the evidence base for SRH interventions in humanitarian settings, and it revealed that IR frameworks, or an explicit IR approach, are rarely used. All four of the systematic reviews identified with a focus on SRH in humanitarian settings articulate the need for more research examining the effectiveness of SRH interventions in humanitarian settings, with two specifically citing a need for implementation research/science [ 12 , 13 ]. The distribution of papers across the timeframe included in this review does suggest that more research on SRH interventions for crisis-affected populations is taking place, as a majority of relevant papers were published in the second half of the review period. The papers included a wide range of methodologies, which reflect the differing research questions and contexts being evaluated. However, it also invites the question of whether there should be more standardization of outcomes measured or frameworks used to guide analysis and to facilitate increased comparison, synthesis and application across settings.

Three of the six papers that used an IR framework utilized the CFIR. Guan et al. used the CFIR at both a domain and construct level, Sarker et al. used the CFIR at the domain level, and Sami et al. did not specify which CFIR elements were used in informing the focus group discussion guide [ 51 , 65 , 70 ]. It is challenging to draw strong conclusions about the applicability of CFIR in humanitarian settings based on the minimal use of CFIR and IR frameworks within the papers reviewed, although Guan et al. provides a helpful model for how analysis can be structured around CFIR domains and constructs. It is worth considering that the minimal use of IR frameworks, and more specifically CFIR constructs, could be in part because that level of prescriptive categorization does not allow for enough fluidity in humanitarian settings. It also raises questions about the appropriate degree of standardization to pursue for research done in these settings.

The mapping of factors affecting SRH intervention implementation provides an example of how a modified CFIR framework could be used for IR in humanitarian contexts. This mapping exercise found factors that mapped to all five of the original CFIR domains as well as the sixth domain proposed by Means et al. All factors fit well within the definition for the selected domain, indicating an appropriate degree of fit between these existing domains and the factors identified as impacting SRH interventions in humanitarian settings. On a construct level, however, the findings were more variable, with one-quarter of factors not fully aligning with any construct. Furthermore, over 40% of the CFIR constructs (including the additional constructs from Means et al.) were not found to align with any factors cited by the papers in this review, also demonstrating some disconnect between the parameters posed by the CFIR constructs and the factors cited as relevant in a humanitarian context.

It is worth noting that while the CFIR as proposed in 2009 was used in this assessment, as well as in the included papers which used the CFIR, an update was published in 2022. Following a review of CFIR use since its publication, the authors provide updates to construct names and definitions to “make the framework more applicable across a range of innovations and settings.” New constructs and subconstructs were also added, for a total of 48 constructs and 19 subconstructs across the five domains [ 90 ]. A CFIR Outcomes Addendum was also published in 2022, based on recommendations for the CFIR to add outcomes and intended to be used as a complement to the CFIR determinants framework [ 91 ]. These expansions to the CFIR framework may improve applicability of the CFIR in humanitarian settings. Several constructs added to the Outer Setting domain could be of particular utility – critical incidents, local attitudes, and local conditions, each of which could help account for unique challenges faced in contexts of crisis. Sub-constructs added within the Inner Setting domain that seek to clarify structural characteristics and available resources would also be of high utility based on mapping of the factors identified in this review to the original CFIR constructs. As outcomes were not formally included in the CFIR until the 2022 addendum, a separate assessment of implementation outcomes was not undertaken in this review. However, analysis of the factors cited by papers in this review as affecting implementation was derived from the full text of the papers and thus captures content relevant to implementation determinants that is contained within the outcomes.

Given the demonstrated need for additional flexibility within an IR framework for humanitarian contexts, while not a focus of this review, it is worth considering whether a different framework could provide a better fit than the CFIR. Other frameworks have differing points of emphasis that would create different opportunities for flexibility but that do not seem to resolve the challenges experienced in applying the CFIR to a humanitarian context. As one example, the EPIS (Exploration, Preparation, Implementation, Sustainment) Framework considers the impact of inner and outer context on each of four implementation phases; while the constructs within this framework are broader than the CFIR, an emphasis on the intervention characteristics is missing, a domain where stronger alignment within the CFIR is also needed [ 92 ]. Alternatively, the PRISM (Practical, Robust Implementation and Sustainability Model) framework is a determinant and evaluation framework that adds consideration of context factors to the RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) outcomes framework. It has a stronger emphasis on intervention aspects, with sub-domains to account for both organization and patient perspectives within the intervention. While PRISM does include aspects of context, external environment considerations are less robust and intentionally less comprehensive in scope, which would not provide the degree of alignment possible between the Characteristics of Systems and Outer Setting CFIR domains for the considerations unique to humanitarian environments [ 93 ].

Reflecting on their experience with the CFIR, Sarker et al. indicate that it can be a “great asset” in both evaluating current work and developing future interventions. They also encourage future research of humanitarian health interventions to utilize the CFIR [ 51 ]. The other papers that used the CFIR do not specifically reflect on their experience utilizing it, referring more generally to having felt that it was a useful tool [ 65 , 70 ]. On their use of an evaluation framework, Berg et al. reflected that it lent useful structure and helped to identify aspects affecting implementation that otherwise would have gone un-noticed [ 67 ]. The remaining studies that utilized an IR framework did not specifically comment on their experience with its use [ 72 , 83 ]. While a formal IR framework was not engaged by other studies, a number cite a desire for IR to contribute further detail to their findings [ 21 , 37 ].

In their recommendations for strengthening the evidence base for humanitarian health interventions, Ager et al. speak to the need for “methodologic innovation” to develop methodologies with particular applicability in humanitarian settings [ 7 ]. As IR is not yet routinized for SRH interventions, this could be opportune timing for the use of a standardized IR framework to gauge its utility. Using an IR framework to assess factors influencing implementation of the MISP in initial stages of a humanitarian response, and interventions to support more comprehensive SRH service delivery in protracted crises, could lend further rigor and standardization to SRH evaluations, as well as inform strategies to improve MISP implementation over time. Based on categorizing factors identified by these papers as relevant for intervention evaluation, there does seem to be utility to a modified CFIR approach. Given the paucity of formal IR framework use within SRH literature, it would be worth conducting similar scoping exercises to assess for explicit use of IR frameworks within the evidence base for other health service delivery areas in humanitarian settings. In the interim, the recommended approach from this review for future IR on humanitarian health interventions would be a modified CFIR approach with domain-level standardization and flexibility for constructs that may standardize over time with more use. This would enable use of a common analytical framework and vocabulary at the domain level for stakeholders to describe interventions and the factors influencing the effectiveness of implementation, with constructs available to use and customize as most appropriate for specific contexts and interventions.

This review had a number of limitations. As this was a scoping review and a two-part search strategy was used, the papers summarized here may not be comprehensive of those written pertaining to SRH interventions over the past 10 years. Papers from 2013 to 2017 that would have met this scoping review’s inclusion criteria may have been omitted due to being excluded from the systematic reviews. The review was limited to papers available in English. Furthermore, this review did not assess the quality of the papers included or seek to assess the methodology used beyond examination of the use of an IR framework. It does, however, serve as a first step in assessing the extent to which calls for implementation research have been addressed, and identify entry points for strengthening the science and practice of SRH research in humanitarian settings.

With one in 23 people worldwide in need of humanitarian assistance, and financing required for response plans at an all-time high, the need for evidence to guide resource allocation and programming for SRH in humanitarian settings is as important as ever [ 94 ]. Recent research agenda setting initiatives and strategies to advance health in humanitarian settings call for increased investment in implementation research—with priorities ranging from research on effective strategies for expanding access to a full range of contraceptive options to integrating mental health and psychosocial support into SRH programming to capturing accurate and actionable data on maternal and perinatal mortality in a wide range of acute and protracted emergency contexts [ 95 , 96 ]. To truly advance guidance in these areas, implementation research will need to be conducted across diverse humanitarian settings, with clear and consistent documentation of both intervention characteristics and outcomes, as well as contextual and programmatic factors affecting implementation.

Conclusions

Implementation research has potential to increase impact of health interventions particularly in crisis-affected settings where flexibility, adaptability and context-responsive approaches are highlighted as cornerstones of effective programming. There remains significant opportunity for standardization of research in the humanitarian space, with one such opportunity occurring through increased utilization of IR frameworks such as a modified CFIR approach. Investing in more robust sexual and reproductive health research in humanitarian contexts can enrich insights available to guide programming and increase transferability of learning across settings.

Availability of data and materials

The datasets analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

Availability, Accessibility, Acceptability and Quality

Centers for Disease Control and Prevention

Consolidated Framework for Implementation Research

Democratic Republic of the Congo

Exploration, Preparation, Implementation, Sustainment

• Implementation research

Low and middle income country

Minimum Initial Service Package

Practical, Robust Implementation and Sustainability Model

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Reach, Effectiveness, Adoption, Implementation, Maintenance

• Sexual and reproductive health

World Health Organization

Peters DH, et al. Implementation research: what it is and how to do it. RESEARCH METHODS. 2013;347:7.

Means AR, et al. Evaluating and optimizing the consolidated framework for implementation research (CFIR) for use in low- and middle-income countries: a systematic review. Implement Sci. 2020;15(1):17.

Article   PubMed   PubMed Central   Google Scholar  

Alonge O, et al. How is implementation research applied to advance health in low-income and middle-income countries? BMJ Glob Health. 2019;4(2):e001257.

Ridde V, Pérez D, Robert E. Using implementation science theories and frameworks in global health. BMJ Glob Health. 2020;5(4):e002269.

Gaffey MF, et al. Delivering health and nutrition interventions for women and children in different conflict contexts: a framework for decision making on what, when, and how. Lancet (London, England). 2021;397(10273):543–54.

Article   PubMed   Google Scholar  

Singh NS, et al. Delivering health interventions to women, children, and adolescents in conflict settings: what have we learned from ten country case studies? The Lancet. 2021;397(10273):533–42.

Article   Google Scholar  

Ager A, et al. Strengthening the evidence base for health programming in humanitarian crises. Science. 2014;345(6202):1290–2.

Article   CAS   PubMed   Google Scholar  

Blanchet K, et al. Evidence on public health interventions in humanitarian crises. The Lancet. 2017;390(10109):2287–96.

Sphere A. The Sphere Handbook | Standards for quality humanitarian response. 2018.

Google Scholar  

Barot S. In a State of Crisis: Meeting the Sexual and Reproductive Health Needs of Women in Humanitarian Situations. Guttmacher Policy Rev. 2017;20:7.

Crisis, I.-A.W.G.f.R.H.i., Minimum Initial Service Package. 2020: https://www.unfpa.org/resources/minimum-initial-service-package-misp-srh-crisis-situations .

Casey SE. Evaluations of reproductive health programs in humanitarian settings: a systematic review. Confl Heal. 2015;9(1):S1.

Singh NS, et al. A long way to go: a systematic review to assess the utilisation of sexual and reproductive health services during humanitarian crises. BMJ Glob Health. 2018;3(2):e000682.

Singh NS, et al. Evaluating the effectiveness of sexual and reproductive health services during humanitarian crises: A systematic review. PLoS ONE. 2018;13(7):e0199300.

Warren E, et al. Systematic review of the evidence on the effectiveness of sexual and reproductive health interventions in humanitarian crises. BMJ Open. 2015;5(12):e008226.

Dadich A, Piper A, Coates D. Implementation science in maternity care: a scoping review. Implement Sci. 2021;16(1):16.

Tricco AC, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467–73.

Devine A, et al. Strategies for the prevention of perinatal hepatitis B transmission in a marginalized population on the Thailand-Myanmar border: a cost-effectiveness analysis. BMC Infect Dis. 2017;17(1):552.

Jarrett P, et al. Evaluation of a population mobility, mortality, and birth surveillance system in South Kivu. Democratic Republic of the Congo Disasters. 2020;44(2):390–407.

PubMed   Google Scholar  

Logie CH, et al. A Psycho-Educational HIV/STI Prevention Intervention for Internally Displaced Women in Leogane, Haiti: Results from a Non-Randomized Cohort Pilot Study. PLoS ONE. 2014;9(2):e89836.

O’Laughlin KN, et al. A cohort study to assess a communication intervention to improve linkage to HIV care in Nakivale Refugee Settlement. Uganda Glob Public Health. 2021;16(12):1848–55.

Adam I. The influence of maternal health education on the place of delivery in conflict settings of Darfur. Sudan Conflict and Health. 2015;9:31.

Adam IF, et al. Relationship between implementing interpersonal communication and mass education campaigns in emergency settings and use of reproductive healthcare services: evidence from Darfur, Sudan. BMJ Open. 2015;5(9):e008285.

Edmond K, et al. Mobile outreach health services for mothers and children in conflict-affected and remote areas: a population-based study from Afghanistan. Arch Dis Child. 2020;105(1):18–25.

Nasir S, et al. Dissemination and implementation of the e-MCHHandbook, UNRWA’s newly released maternal and child health mobile application: a cross-sectional study. BMJ Open. 2020;10(3):e034885.

O’Laughlin KN, et al. Feasibility and acceptability of home-based HIV testing among refugees: a pilot study in Nakivale refugee settlement in southwestern Uganda. BMC Infect Dis. 2018;18(1):332.

Adam I. Evidence from cluster surveys on the association between home-based counseling and use of family planning in conflict-affected Darfur. Int J Gynecol Obstet. 2016;133(2):221–5.

Casey S, et al. Availability of long-acting and permanent family-planning methods leads to increase in use in conflict-affected northern Uganda: Evidence from cross-sectional baseline and endline cluster surveys. Glob Public Health. 2013;8(3):284–97.

Corna F, et al. Supporting maternal mental health of Rohingya refugee women during the perinatal period to promote child health and wellbeing: a field study in Cox’s Bazar. Intervention, the Journal of Mental Health & Psychosocial Support in Conflict Affected Areas. 2019;17(2):160–8.

Glass N, et al. Effectiveness of the Communities Care programme on change in social norms associated with gender-based violence (GBV) with residents in intervention compared with control districts in Mogadishu, Somalia. BMJ Open. 2019;9(3):e023819.

James LE, et al. Development and Testing of a Community-Based Intervention to Address Intimate Partner Violence among Rohingya and Syrian Refugees: A Social Norms-Based Mental Health-Integrated Approach. Int J Environ Res Public Health. 2021;18(21):11674.

Le Roux E, et al. Engaging with faith groups to prevent VAWG in conflict-affected communities: results from two community surveys in the DRC. BMC Int Health Hum Rights. 2020;20(1):27.

Morris CN, et al. When political solutions for acute conflict in Yemen seem distant, demand for reproductive health services is immediate: a programme model for resilient family planning and post-abortion care services. Sex Reprod Health Matters. 2019;27(2):1610279.

Anibueze AU, et al. Impact of counseling visual multimedia on use of family planning methods among displaced Nigerian families. Health Promot Int. 2022;37(3):daac060.

Doocy S, et al. Cash-based assistance and the nutrition status of pregnant and lactating women in the Somalia food crisis: A comparison of two transfer modalities. PLoS ONE. 2020;15(4):e0230989.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Draiko CV, et al. The effect of umbilical cord cleansing with chlorhexidine gel on neonatal mortality among the community births in South Sudan: a quasi-experimental study. Pan Afr Med J. 2021;38:78.

Edmond KM, et al. Can community health worker home visiting improve care-seeking and maternal and newborn care practices in fragile states such as Afghanistan? A population-based intervention study. BMC Med. 2018;16(1):106.

Edmond KM, et al. Conditional cash transfers to improve use of health facilities by mothers and newborns in conflict affected countries, a prospective population based intervention study from Afghanistan. BMC Pregnancy Childbirth. 2019;19(1):193.

Bakesiima R, et al. Effect of peer counselling on acceptance of modern contraceptives among female refugee adolescents in northern Uganda: A randomised controlled trial. PLoS ONE. 2021;16(9):e0256479.

Greene MC, et al. Evaluation of an integrated intervention to reduce psychological distress and intimate partner violence in refugees: Results from the Nguvu cluster randomized feasibility trial. PLoS ONE. 2021;16(6):e0252982.

Gupta J, et al. Gender norms and economic empowerment intervention to reduce intimate partner violence against women in rural Côte d’Ivoire: a randomized controlled pilot study. BMC Int Health Hum Rights. 2013;13(1):46.

Hossain M, et al. Working with men to prevent intimate partner violence in a conflict-affected setting: a pilot cluster randomized controlled trial in rural Côte d’Ivoire. BMC Public Health. 2014;14(1):339.

Vaillant J, et al. Engaging men to transform inequitable gender attitudes and prevent intimate partner violence: a cluster randomised controlled trial in North and South Kivu, Democratic Republic of Congo. BMJ Glob Health. 2020;5(5):e002223.

Thommesen T, et al. “The midwife helped me … otherwise I could have died”: women’s experience of professional midwifery services in rural Afghanistan - a qualitative study in the provinces Kunar and Laghman. BMC Pregnancy Childbirth. 2020;20(1):140.

Awasom-Fru A, et al. Doctors’ experiences providing sexual and reproductive health care at Catholic Hospitals in the conflict-affected North-West region of Cameroon: a qualitative study. Reprod Health. 2022;19(1):126.

Kabakian-Khasholian T, Makhoul J, Ghusayni A. “A person who does not have money does not enter”: a qualitative study on refugee women’s experiences of respectful maternity care. BMC Pregnancy and Childbirth. 2022;22(1):748.

Lilleston P, et al. Evaluation of a mobile approach to gender-based violence service delivery among Syrian refugees in Lebanon. Health Policy Plan. 2018;33(7):767–76.

Mugo NS, et al. Barriers Faced by the Health Workers to Deliver Maternal Care Services and Their Perceptions of the Factors Preventing Their Clients from Receiving the Services: A Qualitative Study in South Sudan. Matern Child Health J. 2018;22(11):1598–606.

Persson M, et al. A qualitative study on health care providers’ experiences of providing comprehensive abortion care in Cox’s Bazar, Bangladesh. Conflict and Health. 2021;15(1):6.

Phanwichatkul T, et al. The perceptions and practices of Thai health professionals providing maternity care for migrant Burmese women: An ethnographic study. Women Birth. 2022;35(4):e356–68.

Sarker M, et al. Effective maternal, newborn and child health programming among Rohingya refugees in Cox’s Bazar, Bangladesh: Implementation challenges and potential solutions. PLoS ONE. 2020;15(3):e0230732.

Tousaw E, et al. “Without this program, women can lose their lives”: migrant women’s experiences with the Safe Abortion Referral Programme in Chiang Mai. Thailand Reprod Health Matters. 2017;25(51):58–68.

Tousaw E, et al. “It is just like having a period with back pain”: exploring women’s experiences with community-based distribution of misoprostol for early abortion on the Thailand-Burma border. Contraception. 2018;97(2):122–9.

West L, et al. Factors in use of family planning services by Syrian women in a refugee camp in Jordan. Journal of Family Planning and Reproductive Health Care. 2017;43(2):96–102.

O’Connell KA, et al. Meeting the Sexual and Reproductive Health Needs of Internally Displaced Persons in Ethiopia’s Somali Region: A Qualitative Process Evaluation. Glob Health Sci Pract. 2022;10(5):e2100818.

Orya E, et al. Strengthening close to community provision of maternal health services in fragile settings: an exploration of the changing roles of TBAs in Sierra Leone and Somaliland. BMC Health Serv Res. 2017;17(1):460.

Perera SM, et al. Barriers to seeking post-abortion care in Paktika Province, Afghanistan: a qualitative study of clients and community members. BMC Womens Health. 2021;21(1):390.

Tanabe M, et al. Piloting community-based medical care for survivors of sexual assault in conflict-affected Karen State of eastern Burma. Confl Heal. 2013;7(1):12.

Tran NT, et al. Clinical outreach refresher trainings in crisis settings (S-CORT): clinical management of sexual violence survivors and manual vacuum aspiration in Burkina Faso, Nepal, and South Sudan. Reprod Health Matters. 2017;25(51):103–13.

Yankah E, et al. Feasibility and acceptability of mobile phone platforms to deliver interventions to address gender-based violence among Syrian adolescent girls and young women in Izmir. Turkey Vulnerable Children and Youth Studies. 2020;15(2):133–43.

Muuo S, et al. Barriers and facilitators to care-seeking among survivors of gender-based violence in the Dadaab refugee complex. Sex Reprod Health Matters. 2020;28(1):1722404.

Amsalu R, et al. Essential newborn care practice at four primary health facilities in conflict affected areas of Bossaso, Somalia: a cross-sectional study. Conflict and Health. 2019;13(13):27.

Myers A, et al. Facilitators and barriers in implementing the Minimum Initial Services Package (MISP) for reproductive health in Nepal post-earthquake. Conflict and Health. 2018;12:35.

Santo L.C.d, et al. Feasibility and acceptability of a video library tool to support community health worker counseling in rural Afghan districts: a cross-sectional assessment. Conflict and Health. 2020;14:56.

Sami S, et al. Understanding health systems to improve community and facility level newborn care among displaced populations in South Sudan: a mixed methods case study. BMC Pregnancy Childbirth. 2018;18(1):325.

Amsalu R, et al. Effectiveness of clinical training on improving essential newborn care practices in Bossaso, Somalia: a pre and postintervention study. BMC Pediatr. 2020;20(1):215.

Berg M, Mwambali SN, Bogren M. Implementation of a three-pillar training intervention to improve maternal and neonatal healthcare in the Democratic Republic Of Congo: a process evaluation study in an urban health zone. Glob Health Action. 2022;15(1):2019391.

Castillo M, et al. Turning Disaster into an Opportunity for Quality Improvement in Essential Intrapartum and Newborn Care Services in the Philippines: Pre- to Posttraining Assessments. Biomed Res Int. 2016;2016:1–9.

Foster AM, Arnott G, Hobstetter M. Community-based distribution of misoprostol for early abortion: evaluation of a program along the Thailand-Burma border. Contraception. 2017;96(4):242–7.

Guan TH, et al. Implementation of a neonatal hepatitis B immunization program in rural Karenni State, Myanmar: A mixed-methods study. PLoS ONE. 2021;16(12):e0261470.

Logie, C.H., et al., Mixed-methods findings from the Ngutulu Kagwero (agents of change) participatory comic pilot study on post-rape clinical care and sexual violence prevention with refugee youth in a humanitarian setting in Uganda. Global Public Health, 2022((Logie C.H., [email protected]) Factor-Inwentash Faculty of Social Work, University of Toronto, Toronto, Canada(Logie C.H., [email protected]) Women’s College Research Institute, Women’s College Hospital, Toronto, Canada(Logie C.H., carmen.l).

Sami S, et al. “You have to take action”: changing knowledge and attitudes towards newborn care practices during crisis in South Sudan. Reprod Health Matters. 2017;25(51):124–39.

Smith JR, et al. Clinical care for sexual assault survivors multimedia training: a mixed-methods study of effect on healthcare providers’ attitudes, knowledge, confidence, and practice in humanitarian settings. Confl Heal. 2013;7(1):14.

Stevens A, et al. Folate supplementation to prevent birth abnormalities: evaluating a community-based participatory action plan for refugees and migrant workers on the Thailand-Myanmar border. Public Health. 2018;161:83–9.

Nguyen Toan T, et al. Strengthening healthcare providers’ capacity for safe abortion and postabortion care services in humanitarian settings: lessons learned from the clinical outreach refresher training model (S-CORT) in Uganda, Nigeria, and the Democratic Republic of Congo. Conflict and Health. 2021;15(1):20.

Castle S, et al. Successful programmatic approaches to facilitating IUD uptake: CARE’s experience in DRC. BMC Womens Health. 2019;19(1):104.

Deitch J, et al. “They Love Their Patients”: Client Perceptions of Quality of Postabortion Care in North and South Kivu, the Democratic Republic of the Congo. Global health, science and practice. 2019;7(Suppl 2):S285–98.

Ferreyra C, et al. Evaluation of a community-based HIV test and start program in a conflict affected rural area of Yambio County, South Sudan. PLoS ONE. 2021;16(7):e0254331.

Ho LS, Wheeler E. Using Program Data to Improve Access to Family Planning and Enhance the Method Mix in Conflict-Affected Areas of the Democratic Republic of the Congo. Glob Health Sci Pract. 2018;6(1):161–77.

Klabbers RE, et al. Health Worker Perspectives on Barriers and Facilitators of Assisted Partner Notification for HIV for Refugees and Ugandan Nationals: A Mixed Methods Study in West Nile Uganda. AIDS Behav. 2021;25(10):3206–22.

Turner C, et al. Neonatal Intensive Care in a Karen Refugee Camp: A 4 Year Descriptive Study. PLoS ONE. 2013;8(8):e72721.

Vries Id, et al. Key lessons from a mixed-method evaluation of a postnatal home visit programme in the humanitarian setting of Gaza. Eastern Mediterr Health J. 2021;27(6):546–52.

Bolan NE, et al. mLearning in the Democratic Republic of the Congo: A Mixed-Methods Feasibility and Pilot Cluster Randomized Trial Using the Safe Delivery App. Global health, science and practice. 2018;6(4):693–710.

Khan MN, et al. Evaluating feasibility and acceptability of a local psycho-educational intervention for pregnant women with common mental problems affected by armed conflict in Swat, Pakistan: A parallel randomized controlled feasibility trial. Int J Soc Psychiatry. 2017;63(8):724–35.

Hynes M, et al. Using a quality improvement approach to improve maternal and neonatal care in North Kivu, Democratic Republic of Congo. Reprod Health Matters. 2017;25(51):140–50.

Gibbs A, et al. The impacts of combined social and economic empowerment training on intimate partner violence, depression, gender norms and livelihoods among women: an individually randomised controlled trial and qualitative study in Afghanistan. BMJ Glob Health. 2020;5(3):e001946.

Damschroder L, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation science: IS; 2009.

Moore GF, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.

Proctor E, et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Adm Policy Ment Health. 2011;38(2):65–76.

Damschroder LJ, et al. The updated Consolidated Framework for Implementation Research based on user feedback. Implement Sci. 2022;17(1):75.

Damschroder LJ, et al. Conceptualizing outcomes for use with the Consolidated Framework for Implementation Research (CFIR): the CFIR Outcomes Addendum. Implement Sci. 2022;17(1):7.

Aarons GA, Hurlburt M, Horwitz SM. Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38(1):4–23.

Feldstein AC, Glasgow RE. A Practical, Robust Implementation and Sustainability Model (PRISM) for Integrating Research Findings into Practice. The Joint Commission Journal on Quality and Patient Safety. 2008;34(4):228–43.

OCHA. Global Humanitarian Overview 2023. 2022 [cited 2023 8/3/2023]; Available from: https://humanitarianaction.info/node/13073/article/glance-0 . Accessed 8 Mar 2023.

Kobeissi L, et al. Setting research priorities for sexual, reproductive, maternal, newborn, child and adolescent health in humanitarian settings. Confl Heal. 2021;15(1):16.

Save the, C., et al. Roadmap to Accelerate Progress for Every Newborn in Humanitarian Settings 2020 – 2024. 2020. p. 52.

Inter-Agency Working Group on Reproductive Health in, C. Inter-Agency Field Manual on Reproductive Health in Humanitarian Settings. 2018.

Download references

Acknowledgements

Not applicable.

The authors received no funding for this study.

Author information

Authors and affiliations.

Duke University School of Medicine, 40 Duke Medicine Circle, Durham, NC, 27710, USA

Alexandra Norton

Johns Hopkins Bloomberg School of Public Health, 615 N. Wolfe St, Baltimore, MD, 21205, USA

Hannah Tappis

You can also search for this author in PubMed   Google Scholar

Contributions

AN and HT designed the scoping review. AN conducted the literature search. AN and HT screened records for inclusion. AN extracted data from included studies. Both authors contributed to synthesis of results. AN drafted the manuscript and both authors contributed to editorial changes.

Corresponding author

Correspondence to Alexandra Norton .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

. Literature search terms: Exact search terms used in literature search, with additional detail on the methodology to determine search terms and definitions used for each component of the search

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Norton, A., Tappis, H. Sexual and reproductive health implementation research in humanitarian contexts: a scoping review. Reprod Health 21 , 64 (2024). https://doi.org/10.1186/s12978-024-01793-2

Download citation

Received : 06 November 2023

Accepted : 12 April 2024

Published : 13 May 2024

DOI : https://doi.org/10.1186/s12978-024-01793-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Humanitarian settings

Reproductive Health

ISSN: 1742-4755

• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 07 November 2023

Social virtual reality helps to reduce feelings of loneliness and social anxiety during the Covid-19 pandemic

• Keith Kenyon   ORCID: orcid.org/0000-0002-5084-9024 1 ,
• Vitalia Kinakh 2 &
• Jacqui Harrison 1  

Scientific Reports volume  13 , Article number:  19282 ( 2023 ) Cite this article

4165 Accesses

2 Citations

2 Altmetric

Metrics details

• Human behaviour
• Quality of life

Evidence shows that the Covid-19 pandemic caused increased loneliness, anxiety and greater social isolation due to social distancing policies. Virtual reality (VR) provides users with an easy way to become engaged in social activities without leaving the house. This study focused on adults, who were socialising in Altspace VR, a social VR platform, during the Covid-19 pandemic and it explored whether social VR could alleviate feelings of loneliness and social anxiety. A mixed-methods research design was applied. Participants (n = 74), aged 18–75, completed a questionnaire inside the social VR platform to measure levels of loneliness (UCLA 20-item scale) and social anxiety (17-item SPIN scale) in the social VR platform (online condition) and real world (offline condition). Subsequently, a focus group (n = 9) was conducted to gather insights into how and why participants were using the social VR platform. Findings from the questionnaire revealed significantly lower levels of loneliness and social anxiety when in the social VR platform. Lower levels of loneliness and social anxiety were also associated with participants who socialised with a regular group of friends. In addition, findings from the focus group suggested that being part of an online group facilitates stronger feelings of belonging. Social VR can be used as a valuable intervention to reduce feelings of loneliness and social anxiety. Future studies should continue to establish whether social VR can help to encourage group formation and provide people with enhanced social opportunities beyond the COVID-19 pandemic.

Similar content being viewed by others

Having more virtual interaction partners during COVID-19 physical distancing measures may benefit mental health

Daily exposure to virtual nature reduces symptoms of anxiety in college students

Contextual modulation of preferred social distance during the Covid-19 pandemic

Introduction.

On the 11th March 2020 the World Health Organisation declared the rapidly spreading Corona virus outbreak a pandemic 1 and world governments began to impose enforced social isolation rules. Throughout 2020/2021 the majority of countries imposed lengthy periods of lockdown. The first UK lockdown lasted almost 4 months and during this time only essential travel was permitted and interaction with others from outside the direct household was forbidden 2 . The lock-down caused disruption to daily routines, social activities, education and work. Social distancing measures led to a collapse in social contact. When people experience a reduction in social contact or when the quality of interaction with others is diminished, they can suffer feelings of loneliness. Nearly 7.5 million adults experienced "lockdown loneliness," which is the equivalent to around 14% of the population. 3 Additionally, the percentage of the UK population reporting loneliness increased from 10% in March 2020 to 26% in February 2021 4 .

Social isolation and loneliness

Social isolation and loneliness are different. Social isolation is commonly defined as “the state in which the individual or group expresses a need or desire for contact with others but is unable to make that contact” 5 , p. 731 . Social isolation can occur due to quarantine or physical separation. Due to quarantine measures enforced during lockdown, people faced involuntary social isolation or at least a reduction in their social interactions to the point that their social network was quantitatively diminished 6 . Loneliness is a subjective experience that arises when a person feels that they are isolated and deprived of companionship, lack a sense of belonging, or that their social interactions with others are diminished in either quantity or quality 7 .

Social isolation, loneliness and detrimental implications for physical and mental health

The rise of loneliness during lockdown also increased the prevalence of anxiety 3 and such health problems as depressive symptoms and insomnia, reconfirming findings from earlier research 8 that explored the relationship between social isolation and loneliness and the effect it has on our physical and mental health. Loneliness can lead to stress and high blood pressure, a sedentary or less active lifestyle, and a reduction in cognitive function 9 , 10 , 11 . Loneliness can also lead to less healthy behaviours e.g. an increase in alcohol consumption and smoking 12 , a poor diet 13 and poor sleeping patterns 14 . Loneliness has been found to have an impact on a person’s social wellbeing leading to feelings of low self-esteem and worthlessness as well as increased anxiety and decreased levels of happiness, resulting in depression 11 , 15 , 16 , 17 .

Technology-based interventions to reduce social isolation and loneliness

Within the last decade several systematic reviews have focused on technology-based interventions for people who are experiencing or who are at risk of experiencing loneliness and social isolation 18 , 19 , 20 , 21 . Masi et al. 18 in their meta-analysis, explored the efficacy of technology-based vs non-technology-based interventions across all population groups, notably, the mean size effect for technology-based interventions was − 1.04 (N = 6; 95% CI  − 1.68, − 0.40; p  < 0.01), as opposed to − 0.21 (N = 12; 95% CI  − 0.43, 0.01; p = 0.05) for non-technology-based interventions. Choi et al. 19 reported a significant pooled reduction in loneliness in older adults after implementing technology-based interventions (Z = 2.085, p  = 0.037). Early technology-based interventions consisted of conference calls/video conferencing, text-based Inter Relay Chat and Emails 18 , 19 , 20 . Subsequent systematic reviews 21 , 22 found that video conferencing was able to reduce loneliness in older particpants, however, this technology only helped to facilitate communication between existing, rather than new contacts. These types of intervention are therefore less beneficial for individuals who are socially isolated and struggling to establish connections with others.

During the Covid-19 lockdowns there was no possibility to provide or continue providing face-to-face individual or group interventions for lonely people. Moreover, even non-lonely people found themselves in situations where they could not maintain their social relationships through face-to-face interactions. Thus, the Department of Primary Care and Public Health in England recommended that avenues for mitigating feelings of loneliness should look to include web- and smartphone-based interventions 23 .

Virtual reality (VR) using a head mounted display (HMD) is considered qualitatively different from other technologies in that it has the ability to provide a sensation of immersiveness or ‘being there’ 24 . VR technologies are becoming more accessible and comfortable with the creation of lighter more portable HMDs at a more affordable cost. This allows the technology to be used by a greater range of adults and members of vulnerable groups, e.g. adults with mobility impairments and older adults with age-related impairments. VR users, often represented as avatars, are able to meet and communicate in real-time with each other within a range of different scenarios. People are able to participate in social activities with new people, e.g. venturing off into new and exciting worlds (with nature scenes) 24 , travelling to different destinations around the world 25 , 26 without leaving their homes and escaping their confined realties or engaging in horticultural therapeutic interactions 27 . Older adults are able to engage in social networking activities, including playing games with other people and attending family events through VR, users spoke very positively and expressed visible signs of enjoyment about their experience 28 , 29 , 30 . Virtual gaming is very popular among younger users with 31 , 32 reporting that players experience significantly lower levels of loneliness and social anxiety when playing VR games compared within the real world.

Users taking part in VR interventions report being less socailly isolated, show less signs of depression, and demonstrate greater levels of overal well-being 24 , 25 , 26 , 27 , 33 , 34 . Widow(er)s in a VR support group showed a significant improvement during an 8-week intervention 35 . While both systematic reviews 33 , 34 reported useful insights regarding the positive impact of VR technology on loneliness, most studies on VR environments included a small number of participants from specific populations, thus the reported findings have limited generalisability.

When VR is used as an intervention to reduce social and public speaking anxiety, it is found to be most effective as a mode of delivery for alternative therapeutic interventions such as Acceptance and Commitment Therapy 36 . Furthermore, Kim et al. 37 found that patients with Social Anxiety Disorder (SAD) benefitted from the use of VR as an intervention, evidenced by short-term neuronal changes during exposure. They concluded that VR is useful as a first intervention for SAD patients who are unable to access formal treatment.

Various social VR platforms have emerged since 2013, e.g. VRChat, Altspace VR and RecRoom, however, the use of social VR as an intervention for reducing social isolation and loneliness is still a relatively new and unexplored field. Therefore, whilst there is research to support the effectiveness of VR as a tool to deliver therapeutic interventions and improve social well-being, there is limited research on the use of social VR as an online mechanism to decrease social isolation and improve group belonging.

Innovation and contributions of this study

The current study is a cross-sectional study of the general population, socially isolated during the Covid-19 pandemic and who were using social VR platforms to interact with each other. This study addresses the limitations of previous studies, which have focused exclusively on specific groups within the population, i.e. older adults or VR gamers, or explored general well-being rather that loneliness and social anxiety. In previous studies the HMDs were often provided by the research team, meaning that there was a time restrain (frequency or length) in relation to the use of the VR technology by participants. This study is novel as it explores the effects of loneliness and social anxiety on a wider demographic of people, who have unrestricted access to HMDs and have been socialising in Altspace VR during the Covid-19 pandemic. This study is of an international character and utilises a mixed methods approach to explore the benefits of social VR to help reduce feelings of loneliness and social anxiety and to provide additional means by which social contact can be enhanced for vulnerable populations who may remain isolated post-pandemic.

Research hypotheses

The following hypotheses were explored:

Lower levels of loneliness and social anxiety are experienced when participants are in the social VR platform (online) compared with in the real-world condition (offline).

Lower levels of loneliness and social anxiety are experienced by participants who are part of a group in social VR, i.e. members of a Virtual Social Group (VSG), than those who are not.

Lower levels of loneliness and social anxiety are experienced by participants who have a group of friends in the social VR in comparison with those who do not.

Lower levels of loneliness and social anxiety are experienced by participants who spend greater amounts of time in social VR.

The study used a convergent parallel mixed-methods research design 38 to collect both diverse quantitative and qualitative data (see Fig.  1 ). The study complied will relevant ethical regulations and was approved by the Research Ethics Committee of the University of Bolton, UK. Written informed consent was obtained from all participants.

A convergent parallel mixed-methods model of the current research.

Collection of quantitative data

Participants.

Participants were required to be English speaking, over the age of 18 and users of Altspace VR. A message of invitation was posted on different Discord community channels/message boards: Official Altspace VR; Educators In VR; Spatial Network; Humanism; Computer Science in VR; VR Church. 87 participants were recruited via an opportunity sampling method.

Materials and measures

A private research room was created inside Altspace VR to ensure that participants were able to complete the questionnaire undisturbed (see Fig.  5 ). The online questionnaire was created in Qualtrics XM and could be accessed across multiple devices: Oculus Quest, Oculus GO, Oculus Rift, HTC Vive and PC. The online questionnaire included sections about demographics, details of Altspace VR usage and sections assessing participant’s subjective feelings of loneliness and social anxiety. Measures of loneliness and social anxiety were collected for both conditions—real world (offline condition), followed by social VR (online condition).

The UCLA Loneliness Scale version 3 39 was used to measure the subjective level of loneliness. This 20-item self-reporting questionnaire uses a four-point Likert scale, with 0 = “Never”, 1 = “Rarely”, 2 = “Sometimes”, 3 = “Often”. The loneliness score for each participant (range from 0 to 60) was determined as the sum of responses to all 20 items—higher scores reflecting greater loneliness. The UCLA Loneliness scale was adapted to include the word Altspace in the online condition as it was felt that this would further help participants to focus specifically on the online experience. No further adaptations were made to this questionnaire. The Social Phobia Inventory (SPIN) scale 40 was used to measure the subjective level of social anxiety as it is effective in measuring the severity of social anxiety. This 17-item self-reporting questionnaire uses a five-point Likert scale, with 0 = “Not at all”, 1 = "A little”, 2 = “Somewhat”, 3 = “Very much”, 4 = “Extremely”. Adding the scores from each item produced a SPIN score for each participant. A higher SPIN score indicates more severe symptoms of social anxiety. No adaptations were made to the SPIN questionnaire.

Participants who were interested in taking part in the survey were taken to the research room inside Altspace VR where they were sent a message with a link to the online questionnaire. Participants who clicked on the link were then presented with a browser window inside the room that only they could see. Participants who opened the questionnaire were first presented with the participant information sheet giving full details of the study. Information regarding withdrawal from the study and a list of additional support services were also provided in line with the University of Bolton’s ethical guidelines. After reading the study information sheet, participants were presented with the consent form for which full consent was required before they were able to move onto the survey.

The strategy for dealing with incomplete cases was to remove any participants who did not answer all of the questions, thus analysis was conducted on 74 participants. Exported data from the Qualtrics system was imported into the Statistical Package for Social Sciences (IBM SPSS, version 25). A Kolmogorov–Smirnov test ( p  > 0.5) was carried out to test for a normal distribution and histograms, nominal Q-Q plots and box plots were used to identify any outliers. Two outliers were found in the data for Social Anxiety in the offline condition and these were replaced with the mean of 17.54 .

Characteristics of the sample

Of the total sample (n = 74), 46 were males and 28 females. The age range of respondents was 18–75 years (the split of valid participants is shown in Table 1 ). Participants were recruited globally (the geographical demographic is shown in Fig.  2 ). Out of these 74 participants, 31 participants (15 males, 16 females) were new to Altspace VR, having joined Altspace VR during the Covid-19 pandemic. 43 participants indicated that they had used Altspace VR before the outbreak of Covid-19.

Participant’s location.

Change in loneliness and social anxiety

Figure  3 shows the breakdown of social anxiety scores in both the online and offline conditions. The data shows that the severity of social anxiety is higher in the offline condition, whereas participant’s levels of anxiety reduce when they are online.

Participant’s SPIN Scores.

The UCLA loneliness scale uses continuous scoring and so it is not possible to provide a similar breakdown for participant’s levels of loneliness. The effect that social VR has on the participant will be discussed in greater detail later.

It was anticipated that during the Covid-19 pandemic and as a direct result of social distancing rules being imposed that general usage in Altspace VR would increase. Figure  4 shows that 76% of participants felt that their usage had increased and after calculating the average difference in usage (before and during Covid-19) an average increase per user of 11 h per week was reported.

Participants usage of Altspace VR since Covid-19.

Hypothesis 1

Hypothesis 1 predicted lower levels of loneliness and social anxiety are experienced when participants are in social VR (online) compared with in the real-world condition (offline) A paired-samples t-test was carried out to compare online (inside social VR) and offline (real-world) conditions for both loneliness and social anxiety. The results in Table 2 demonstrate a statistically significant decrease in the scores for loneliness from the offline condition (M = 20.53, SD = 14.80) to the online condition (M = 16.32, SD = 11.04), t  = − 2.573, p  < 0.05. A statistically significant decrease in social anxiety was found in the offline condition (M = 23.01, SD = 16.65) compared to the online condition (M = 16.34, SD = 13.09), t  = − 5.80, p  < 0.05. A small to moderate effect size 41 was found for both variables (i.e. d loneliness = 0.32 and d social anxiety = 0.45).

Hypotheses 2, 3 and 4

H2 predicted that lower levels of loneliness and social anxiety are experienced by participants who are part of a group in social VR than those who are not.

Being a member of a VSG means that the participant meets with a group or number of groups on a regular basis to take part in scheduled events, e.g. regular church services for members of VR Church; discussions around education each week for members of Educators in VR; mediation and relaxation sessions for members of the EvolVR group; and discussions on a whole range of matters relating to life in the Humanism group. 75.7% of participants (n = 56) indicated that they were a member of a VSG and 24.3% (n = 18) were not affiliated with any groups.

A one-way between participants ANOVA was carried out to compare the effect of being a member of a VSG separately for each of the dependent variables. No significant effect was found for loneliness in both the online condition F(1,72) = 0.17, p  = 0.68 and offline condition F(1,72) = 1.63, p  = 0.20. No significant effect was found for social anxiety in the online condition F(1,72) = 2.22, p  = 0.14, however, a significant effect was found for social anxiety in the offline condition F(1,72) = 4.23, p  < 0.05, η 2  = 0.06 (a medium effect size). This finding suggests that participants who are part of a VSG experience less social anxiety (M = 20.80, SD = 15.64) than those who are not (M = 29.89, SD = 18.26) when in the real world (offline) condition.

H3 predicted that lower levels of loneliness and social anxiety are experienced by participants who have a group of friends in social VR in comparison with those who do not. This differs from Hypothesis 2 in that having friends in Altspace VR is seen as a deeper connection than simply taking part in group events where connections may not have been formed. Participants were grouped on whether they have a circle of friends in social VR with whom they regularly socialise with (52.7%, n = 39) and not (47.3%, n = 35).

A one-way between participants ANOVA was carried out to compare the effect of having a circle of friends separately for each of the dependent variables. A significant effect was found for loneliness in the online condition F(1,72) = 6.75, p  < 0.05, η 2  = 0.08 (a medium effect size), whereas no significant effect was found for loneliness in the offline condition F(1,72) = 0.03, p  = 0.86. This suggests that participants who have a circle of online friends experience less loneliness (M = 13.28, SD = 11.02) than those who do not (M = 19.71, SD = 10.17). A significant effect was found for social anxiety in both the online condition F(1,72) = 6.82, p  < 0.05, η 2  = 0.09 (a medium effect size) and offline condition F(1,72) = 9.18, p  < 0.01, η 2  = 0.11 (a large effect size). This suggests that participants who have a circle of online friends experience less social anxiety (M = 12.72, SD = 12.64) than those who do not (M = 20.37, SD = 12.54) in both online and offline conditions.

H4 predicted that lower levels of loneliness and social anxiety are experienced by participants who spend greater amounts of time in social VR. There was a reasonable balance of participants who have been members of Altspace VR for more than 6 months prior to (n = 43) and who joined during (n = 31) the Covid-19 pandemic.

A one-way between participants ANOVA shows a significant effect for loneliness in the online condition F(1,72) = 4.68, p  < 0.05, η 2  = 0.06 (a medium effect size), whereas no significant effect was found for loneliness in the offline condition F(1,72) = 0.08, p  = 0.93. This suggests that participants who have been members of Altspace VR for more than 6 months experienced less loneliness (M = 14.02, SD = 11.63) than those who joined during the Covid-19 pandemic (M = 19.52, SD = 09.43). No significant effect was found for social anxiety in the online condition F(1,72) = 2.13, p  = 0.15, however, a significant effect was found for social anxiety in the offline condition F(1,72) = 4.77, p  < 0.05, η 2  = 0.06 (a medium effect size). This suggests that participants who have been members of Altspace VR for more than 6 months experienced less social anxiety (M = 19.51, SD = 16.82) than those who recently joined (M = 27.87, SD = 15.38).

Discussion of quantitative results

Research into the use of web-based technologies and virtual worlds has consistently demonstrated positive effects of such interventions on an individual’s subjective feelings of loneliness and social anxiety. Hypothesis 1 of this study is therefore supported and is consistent with the earlier findings 31 , 32 , 42 , 43 and a recent review 44 .

The results of this study in relation to hypothesis 2 were unable to support the assumption that being part of a VSG will reduce feelings of loneliness. The study was therefore unable to support findings from 32 which reported that VR gamers who played as part of a guild were less likely to experience feelings of loneliness. Social identity theory 45 provides a possible explanation for this. Teaming up with a specific VR gaming guild with the common purpose of defeating an enemy for example exerts a stronger sense of identity and group attachment compared to belonging to multiple virtual social groups, where an individual could have several social identities, thus group attachment is less salient. Furthermore, group attachment takes time to develop and within Altspace VR new VSGs are being created all the time. Future studies should look to explore the relationship between the membership duration and the strength of group attachment and the effect this has on subjective feelings of loneliness.

The results of this study support hypothesis 3 in that participants, who have a circle of friends with who they regularly socialise in social VR, experience lower levels of loneliness and social anxiety. This is consistent with the findings of 32 who found that playing with known people helps to reduce feelings of loneliness and social anxiety. This also further supports the findings of 46 who found that half of participants considered their gamer friends to be comparable to their real-life friends. As pointed out by 47 in the Need to Belong Theory, people need frequent and meaningful interactions to feel fulfilled. The ability to form positive social interactions with people with which we feel most connected, i.e. a circle of friends that share our goals or with which we have a common purpose, promotes greater levels of satisfaction and generates greater feelings of belonginess, which in turn reduces our feelings of loneliness and social anxiety 48 .

The results of this study in relation to hypothesis 4 support the assumption that the longer a person has been in social VR the lower will be their feelings of loneliness. There was a significant reduction in feelings of loneliness in the online condition, but not in the offline condition. The explanation for the divergence is that both new and existing Altspace VR users were experiencing similarly high levels of loneliness in the real-world condition, due to the sudden enforced period of lockdown that was imposed upon them, and that whilst being in social VR for a longer period of time showed a greater reduction in feelings of loneliness, in the real world the length of time they had been using social VR was not significant. A possible explanation for this is that when returning to the real world a person is again faced with the challenges of the imposed social isolation and will therefore continue to experience greater levels of loneliness. The reverse situation was found for social anxiety with a significant reduction in social anxiety being found in the offline condition for participants who had been using social VR for longer. This is a useful finding because it shows that using social VR for longer periods of time can help to reduce feelings of social anxiety in the real world. As is suggested by 42 social VR can be used to build up social capital and thereby help to improve a person’s social skills in the real world.

Focus group

Nine participants (6 male, 3 female) who took part in the online questionnaire were later recruited to take part in a focus group. The demographics of this group are shown in Table 3 . The focus group was made up of a wide mix of people from around the world. Participants were a mix of educators, students, developers and other professionals. Four of the participants were new to Altspace VR, having joined during the Covid-19 pandemic, whilst five had been in Altspace VR for more than 6 months. All the participants had previously attended at least one Educators in VR research event.

The focus group study took place in a private research room inside of Altspace VR (see Fig.  5 ), purposely created by the researcher. Only selected participants were able to join this room via a portal link provided by the researcher. The interview was recorded using OBS screen recording software on the researcher’s computer.

Virtual research room.

Prompts were kept to a minimum and questions were open-ended to elicit rich responses from participants. The focus group was later transcribed verbatim by the researcher. The transcript was analysed using a thematic data analysis approach as per the Braun and Clarke framework 49 . Thematic analysis is a suitable analytic approach to systematically establish patterns of meaning within qualitative data sets 50 . Microsoft Word was used to facilitate data management and the coding of themes. Participants’ responses were coded and themes identified.

Qualitative results

Four superordinate themes with several subordinate themes were identified (see Table 4 ).

Theme 1. Why the participant visits the social VR platform

Participants spoke freely about how they got involved in Altspace VR and what they believe to be the main reason they visit Altspace VR. Three sub-themes were discovered, although from the discussions it was clear that most, if not all, participants, valued the group interaction and attendance at events very highly.

Socialising in VR

What was interesting about the group of participants in the focus group was that they were all connected due to their involvement with the Educators in VR community and not through friendship ties. Some participants highlighted that they initially joined Altspace VR to meet new people and then started building a network of professional relationships.

Participant quotes from the transcripts are given within the results section for each subordinate theme. For confidentiality purposes quotes from participants will be referenced as: Participant (P), followed by a number 1–9 and the participant’s gender M (male), F (female) e.g. “P1M”.

“In VR I hang out with friends and of course the [Educators in VR] research team, but I don’t hang out around the campfire as much anymore” (31-33,P3F).

The campfire in Altspace VR is a meeting place for new users to mingle, chat and make friends. New users to Altspace VR tend to levitate towards the campfire until they establish friendship groups and events in which to take part in. This participant has already established a network of meaningful friendships and they are now spending less unstructured time in social zones.

All participants highlighted that they had seen an increase in their usage during the Covid-19 pandemic. The imposed restrictions on physical meetups led to several participants using social VR to meet with real-world friends to satisfy their social needs.

“During this pandemic I have probably come in an hour or two more per day. Part of that was to connect with some of my friends. I got some friends to start coming into Altspace VR so we were able actually hang out in Altspace” (52-55,P5F). “more recently, in the last month or so, because I work in the VR community and a lot of my personal friends have VR headsets, the people that I work with at the university, The people that are in my groups and in my sphere so to speak at the university are some of my best friends and so we have started having social meet-ups in VR for nothing other than social, like just for social meet-ups” (125-132,P1M)

Attending community events and learning new skills

All of the focus group participants recognised the value of taking part in regular events in social VR. In particular, participants were positive about the opportunities that exists within Altspace VR to collaborate with others to expand and learn new skills. Community involvement within Altspace VR generates a strong sense of belonging thus reducing feelings of loneliness and social anxiety.

“I got inspired by the Covid situation to host events, so it inspired me to bring people together. I think if the Covid situation did not happen I wouldn’t have organised these research meetings to be honest, so it was pretty much the catalyst to hosting events” (161-165,P3F) “One thing I love about the Altspace environment is the Educators forum because I have joined philosophy classes, I’ve done Psychology classes, I’ve really interacted. In fact, I started a talk show, [ ] my own event, and that’s one thing that I love about Altspace, so I do love this place” (72-78,P7M)

Sharing ideas with professionals and like-minded people

Altspace VR allows users to create their own events and to share knowledge with other users. There are a wide range of different interest groups within Altspace VR. Establishing common interests with others is a cornerstone to forming positive and meaningful relationships. Establishing a network of contacts is also beneficial by encouraging, giving advice and supporting each other in difficult times 51 . Several of the participants commented that social VR is a useful tool not least during periods of enforced social isolation, but also to those who find themselves unable to form such relationships within their existing real-world social networks.

“I entered Altspace mainly for the Educators in VR conference and after that, during the Covid crisis obviously I stayed because it is a perfect place to find people that have a similar interest with mine” (62-64,P6F). “It’s almost impossible where I live to find people with similar interests like mine, so this is probably the only way for me to find people with similar interests” (188-190,P6F) “I love coming here because there are so many truly brilliant people with so much to learn and so many interesting things to hear and see” (105-107,P9M)

Theme 2. How the participant sees their current situation

Although participants were not specifically asked, they took it upon themselves to reflect how they see the current situation and their specific circumstance in terms of being socially isolated. Participants felt that they were socially isolated and less social for several reasons. These have been broken down into the following sub-themes.

Introverted/anti-social

Several participants stated that they are socially inhibited and anxious individuals, who find socialising in the real world more challenging, whereas social VR offers a less intimidating way for them to meet and make friends.

“If you struggle with social interaction, VR is a little less intimidating, I would say. I really think these platforms are a great way to connect and less intimidating as well” (240-245,P3F) “Prior to Covid I was actually pretty like unsocial, I still kind of am unsocial, but it seems as though now society is kind of like bending towards introverts so in a sense it’s like the market’s benefiting my type so like in a sense I’m becoming increasingly more social” (18-22,P2M).

Socially isolated due to remote location and work/life balance

Some participants lamented that their geographic location or work/life balance in the real world made it very difficult for them to meet and to have frequent interactions with people with similar interests to theirs. This aspect makes them at a greater risk of loneliness to others. Social interaction within social VR is not restricted by geographic location and so these participants feel that this has helped to enhance their social interaction with others.

“I use VR to socialise because I live in a little village so for me it’s the only way to meet people, to communicate with people etc because normally I don’t meet people in the real life. With my friends and with my brother etc so I use the VR to socialise okay” (40-43,P4M) “I went on sabbatical in September this academic year I spent my entire summer, last year outside hiking and camping and all of that and then all of a sudden I was inside doing research and I was isolated from my community. I feel like my work community is my community, you know, and I felt like I lost my community and I felt like I found a new one in Altspace” (259-265,P1M)

Theme 3. How the participant sees the social VR platform

Several participants elaborated in detail on how they felt that social VR helped them to connect with people in ways that were better than alternative digital communication methods such as video conferencing, text chat or social media.

Greater immersion/presence

Immersion and presence are important characteristics within VR because the aim after all is to replicate, to some degree, the feelings of being within the real world. The more this is made possible the more useful VR will be in combating feelings of loneliness and social anxiety during periods of prolonged isolation in the real world.

“I’ve been in here with students for tutorials and […] students have said that they feel more presence with other students in this environment” (108-111,P9M) “I’m a perceptual psychologist so I even think about it from the view of like it feels like some of the spaces that I go into now in Altspace really regularly feel in my head like real spaces that I go to so when I feel like I go to a couple of events in the afternoon in Altspace and then I take the headset off it kind of feels like I left my house and I went out and did something and then came back, it doesn’t feel like I was in my house the whole time” (154-160,P1M)

More ways to connect

In addition to the greater immersion and presence that VR can create, Altspace VR also gives individuals the ability to control and create their own environments for social interaction. It is not possible within the real world for most of us to simply create our own hang-outs or to control our environments so easily. This allows people to therefore interact in ways that up until now have not been possible. Several participants linked the ability to create stimulating and exciting environments in the Altspace VR to something that they can feel proud of, and this gives them social capital over other users with less advanced skills in world creation. This in turn helps to improve their ability to socialise and build further friendships in social VR that they would not have been able to build in the real world.

“I made a beach environment, a beach world and there are other ones out there, but I made a custom private one for me and my friends to meet in and so we meet in there and other places and we bounce around and look at different places but we often find somewhere like a private room where we can actually have a nice private conversation and we don’t have to worry about anyone interfering and everyone said its fantastic it really allows us to connect in ways, you know like those personal chats you have with close friends that it’s hard to do in any other medium, it feels a little more natural in VR to do that and so it’s been fantastic, we’ve been really enjoying it” (132-142,P1M) “Since coming in here now [my friends] are like world building and have created some really awesome spaces in here and so we go in and check out the space that they just created and so I’m still kind of doing project oriented hang-outs as far as like we will be like oh that lighting needs to be a little different and stuff like that but it’s been a really fun way to hang out with people that I already may have been friends with before all this happened but now that this happened they are starting to come into this space so we can connect even more often” (214-222,P5F)

Theme 4. How social VR is helping during the Covid-19 pandemic

In the second part of the focus group, participants were asked to think about how they thought Altspace VR was helping them specifically during the Covid-19 pandemic and whether they thought that others could benefit from this experience too. The responses were very positive and provided a great deal of insight into how Altspace VR is helping them to deal with loneliness and social anxiety during Covid-19. A number of key sub-themes emerged from this category.

Helps people feel less lonely

Several participants said that social VR helps them to feel connected with a circle of friends and that this helps to reduce feelings of loneliness and depression.

“I feel it really does help me in social isolation. I have been on sabbatical this last year so my whole year has been about isolation even before Covid-19, I’ve been working a lot on my own and that sort of thing so yeah becoming part of the community in Altspace, collectively in the different ways that I have has had a huge impact on my mental health. I was getting a little depressed in the fall and having this community has really felt like that it brought me out of it a bit” (147-154,P1M) “By the second semester I only had like one course and we were like really concentrating on a specific project and everything and it was like really limiting me to go outside and do some other stuff. Even though I’m an introvert but I do feel like I really wanted to go outside and have some fun. I really like to see other stuff around me and doing all this stuff here in VR kept me really engaged with the communities” (191-197,P8M)

Helps to motivate and provide structure

Having a purpose and being occupied with an interesting project and subsequently conversing about its progress/issues with others in social VR were perceived as motivational factors, which helped them to deal with the imposed social isolation.

“Events really motivated me to keep busy also when I was in social isolation for two months. Yeah, two months is a long time you know to not get out of your house so that was great I created some sense of purpose and it was really heart-warming to see everybody come together and really interesting people as well. Everybody has something cool to share and was very helpful so that gave me some energy, you know to just keep on going and make the best out of the situation” (166-173,P3F) “I finally have a structure for a project that I have been thinking about for over a year now and having these interactions in here and talking to people allowed me to bring a clear picture of how I can start a project I have been thinking about and start building it inside Altspace, so that’s a big plus for me” (178-182,P6F)

Helps people to be less anti-social and reduced social anxiety

Several participants explained that social VR is “a great way to connect and less intimidating as well” for socially anxious, i.e. “unsocial” and “introverted” people, who as a result often feel lonely. In addition, social VR is a convenient tool for social interactions as it brings people closer “especially during these situations, but not only during like pandemics”. (240–243,P3F)

“In my case the Covid increased my social interaction with people because I’m a pretty anti-social person in real life so for me this has increased ten-fold my social interaction in general” (174-176,P6F). “Covid pushed people inside spaces like VR and made my social interactions far easier to have” (186-188,P6F). “I am in sort of a group, let’s say of people who have problems with connecting with people, this is awesome. This is definitely a big plus and I would like more of this” (322-324,P6F) “I was, I guess, somewhat socially isolated before coming in Altspace I tend to just like to work on projects and stay at home or be at work, but since coming in Altspace I’ve definitely started experiencing more of the social aspect of living like making connections with other people in ways that aren’t strictly like a project that I’m working on and so that’s been nice” (202-208,P5F). “I do think that VR can help us, those of us who are socially isolated or have social anxieties of some sort. It does make it more accessible for us to be able to go into a space and interact with people. For instance in real life, if you were to have social anxiety and you start feeling almost like a panic attack coming on, that would prevent you from going into a real life space, whereas in VR you […] can say, oh I have to go really easily and you’re back in your home and you can work through whatever may have come up with social anxiety. So I do think it makes social interactions more accessible in those cases” (307-316,P5F)

Helps to socialise with real life friends during lock-down

Another idea that surfaced among the participants is the potential to use social VR as a mode of interaction/engagement with real-life friends/family members who live afar. Participants expressed the view that the current restriction on face-to-face contact could to some extent be counterbalanced by inviting real-world friends into social VR to socialise.

“The fully social part of VR has happened because of the Covid-19 situation, because I used to go for dinners with people like every month, […] and we can’t do the real world social, so we are trying to do the VR social” (142-146,P1M) “Once everyone went into social isolation for Covid I actually started hanging out with a friend that lives 3 hours away from me more than before because before it would be a 3 hour drive, but then once all this happened, I actually convinced them to come into Altspace” (208-212,P5F) “It’s been a really fun way to hang out with people that I already may have been friends with before all this happened but now that this happened they are starting to come into this space so we can connect even more often. (218-222,P5F).

Discussion of qualitative findings

Overall, participants’ commentaries to Theme 1 reconfirm that their usage of social VR has increased during the period of imposed social isolation and restrictions on physical meetups due to the Covid-19 pandemic. They were using social VR to meet with real-world friends to satisfy their social needs and continue to receive support from people they are close to; or to mix socially with other users who they meet either at a “campfire” or whilst taking part in regular events inside of the social VR platform, thus expanding their social network of non-intimate contacts. As a result, they felt less lonely online (whilst being in Altspace VR) as they felt like they were in the same space together. Interestingly, participants noted that they also benefited emotionally from meeting like-minded people/professionals and sharing ideas with them, getting support and advice, and working together in real-time. This is a new explanation why people use VR technology, which did not surface in the earlier research studies. Nonetheless this reason ties with the Need to Belong Theory 47 . This is useful to help us to understand why users visit Altspace VR in general and during the enforced social isolation period.

In theme 2 participants’ responses reiterate what has already been explained in the literature that shy, socially inhibited and anxious individuals find online anonymity liberating and less inhibited than the real world 52 . Moreover, in Altspace VR it is also possible to make use of non-verbal communication such as emojis or emoticons (see Fig.  6 ).

Use of emojis to communicate in Altspace VR.

Some participants commented that their geographic location or work/life balance in the real world made it very difficult for them to meet people with similar interests. The social internet, e.g. Facebook 53 and video conferencing 54 have long been used to socialise with friends and family and have been found to be an affective intervention for reducing loneliness. Theme 3 considers that social VR could be regarded as the latest endeavour within this field as individuals are able to create their own exciting hangouts, e.g. a beach or a city from Ancient Greece. Furthermore users are able to easily control environments and restrict entry. This allows people to interact in ways that up until now have not been possible.

Findings in Theme 4 give a clear indication that social VR helps to reduce feelings of loneliness, and this further supports the findings of 32 . Social interactions in social VR are also particularly attractive to those who are lonely or shy/socially anxious/self-conscious or have poor social skills, etc. as they feel more in control of their online interactions and feel that they have a broader range of topics that they are able to discuss compared with in the real world 55 . Lonelier people also feel that they can be more themselves in online social interactions than in the real world 56 .

General discussion

People use social VR for many different reasons: to socialise with new and existing friends; to join social interest groups; to learn new skills and generally to be part of a larger community of people (including other professionals) than those that they are part of in the real world. Social VR attracts a wide range of people because of the ease in which people can meet people with similar interests to their own, although it could be argued that up until the recent Covid-19 pandemic social VR tended to attract a greater amount of people who found real-life social interaction difficult. The results of this study show a reduction in social anxiety in individuals with moderate, severe and very severe social anxiety in the online condition, i.e. when using social VR. The increase in availability of VR headsets in recent years has led to an expansion in usage of social VR and the recent Covid-19 pandemic and subsequent social distancing rules led to more people and organisations making a greater use of VR to communicate and carry out their daily business and routines during the prolonged period of social isolation. Social VR also enables people to collaborate in ways not possible within the real world, reducing geographic restrictions and breaking through communication barriers by using visually stimulating content creation tools to enhance the process of human interaction through world-building and event hosting.

The main objective of this study was to explore whether social VR could be used to help reduce feelings of loneliness and social anxiety amongst people confined to their homes and away from their regular friendship groups and social connections, i.e. when the quantity and quality of their social network is gravely affected. Overall, the synthesised results of the present study show that participants experience a statistically significant reduction in loneliness and social anxiety when in social VR than in the real world during prolonged periods of imposed social isolation. Qualitative findings support/validate the quantitative results for H1. Thus, the evidence shows that social VR can decrease the sense of loneliness and social anxiety with users and have an overall positive effect on their emotional and social wellbeing.

The qualitative data diverges from the quantitative results presented for H2 that addressed the effect of being part of a VSG separately for loneliness and social anxiety. The quantitative results showed no significant effect for loneliness in the online and the offline conditions, whereas participants’ views showed that being a member of a VSG created a sense of belongingness and helped them to feel less lonely and depressed. Quantitative data showed no significant effect for social anxiety when an individual is a member of a VSG or not; but revealed a medium effect for social anxiety in the offline condition indicating that users, who are part of a VSG and subsequently take part in regular group events, experience less social anxiety in real world (i.e. offline), than those who are not part of a VSG. Participants who are part of a VSG were positive about the possibilities of social VR and being part of a VSG, because this setup helped shy and socially inhibited individuals to observe conversations, use emojis to show emotions rather than speak, use the online anonymity to get over the discomfort of social interactions and gradually become more connected and accepted by other members of the VSG. This prepares socially anxious individuals to handle being out there (in online and the real world).

Qualitative findings are in line with the quantitative results for H3 in that the degree of loneliness and social anxiety is also further reduced by factors such as having a circle of online friends. Social VR allows people to meet others who share similar interests, this is more difficult within the real world for people who struggle with social anxiety or who live in remote locations for example, or as was the case with this study, people who were confined to their homes due to social distancing rules during a pandemic. The qualitative data helps to produce a better understanding in relation to ‘online friends’ as these include individuals who were met in social VR and real-life friends who currently live afar and were invited to join the social VR platform.

The qualitative findings somewhat converge with quantitative results for H4 in that online loneliness reduces with the length of time the participant has been using social VR, i.e. participants who had been using social VR for greater than 6 months experienced less loneliness than those who joined during the Covid-19 pandemic. The length of time the participant had been using social VR had no effect on their feelings of loneliness in the real world. Comments from participants who have been members of Altspace VR for more than 6 months revealed that finding a new (online) community that supports their need to belong and provides meaningful and positive social interactions acted as an antidote to the loneliness that they experience in the real world. Individuals who struggle to build meaningful relationships in the real world due to social anxiety and other social phobias turn to social VR as it provides a less confrontational way in which to form and maintain social relationships with others and therefore help to reduce feelings of loneliness and social anxiety.

Research limitations and implications

The heterogeneity of the sample for the quantitative survey enabled conclusions to be drawn regarding the participant experience in Altspace VR, their subjective feelings of loneliness and social during the Covid-19 pandemic. However, in interpreting the views of participants in the focus group it should be stressed that the sample of participants was solely recruited from the Educators in VR research event and that this may not represent the views of others who do not take part in such events. Although the reported themes were clearly identified, there remains a possibility that additional themes would be detected should the views of participants from a wider pool be collected.

It is the researcher’s understanding that this is the first study that has exclusively focused on participant’s feelings of loneliness and social anxiety during a period of enforced prolonged isolation whereby social VR has been utilized as an intervention to help reduce such feelings. The results offered here, should therefore be taken as a starting point upon which further empirical studies could be built. Longitudinal investigations could be carried out to further assess the suitability of social VR as an intervention to help reduce loneliness and social anxiety amongst specific communities, e.g. remote learners/workers, people living alone or in care, the less physically able, prisoners and other sub-groups of people facing loneliness and social anxiety whereby their ability to socialise with other is in some way restricted. Future research would also need to provide accurate estimates of the prevalence of loneliness and social anxiety in these sub-groups.

The COVID-19 pandemic forced people to change the way in which they connected with others during lockdown. Social VR helped to improve social connectedness during the COVID-19 pandemic and reduce “lockdown loneliness”. Post-pandemic it is necessary to recognise the additional needs that face society, especially vulnerable people and those struggling with mental health issues resulting from lockdown. Social VR can, therefore, be a way of further supporting people facing social isolation, loneliness and social anxiety. Social VR platforms may be virtual, but the relationships we build in them are very real.

Data availability

All data generated or analysed during this study are included in this published article or in the accompanying Supplementary Information file.

WHO. WHO Director-General’s opening remarks at the media briefing on COVID-19- 11 March 2020. www.who.int . https://www.who.int/dg/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020 (2020).

Gov.UK. PM Address to the Nation on coronavirus: 23 March 2020. GOV.UK. https://www.gov.uk/government/speeches/pm-address-to-the-nation-on-coronavirus-23-march-2020 (2020).

Office for national statistics. Mapping loneliness during the coronavirus pandemic. onsgovuk. https://www.ons.gov.uk/peoplepopulationandcommunity/wellbeing/articles/mappinglonelinessduringthecoronaviruspandemic/2021-04-07 (2021).

Pandemic one year on: Landmark mental health study reveals mixed picture. www.mentalhealth.org.uk . Available from: https://www.mentalhealth.org.uk/about-us/news/pandemic-one-year-mental-health-study

Schuch, R. Nursing diagnosis-application to clinical practice. Gastroenterol. Nurs. 14 (5), 275 (1992).

Article   Google Scholar  

Victor, C. R. & Yang, K. The prevalence of loneliness among adults: A case study of the United Kingdom. J. Psychol. 146 (1–2), 85–104. https://doi.org/10.1080/00223980.2011.613875 (2012).

Article   PubMed   Google Scholar  

Mansfield, L., Daykin, N., Meads, C., Tomlinson, A., Gray, K., Lane, J. et al. A conceptual review of loneliness across the adult life course (16+ years). https://whatworkswellbeing.org/wp-content/uploads/2020/02/V3-FINAL-Loneliness-conceptual-review.pdf (2019).

Torales, J., O’Higgins, M., Castaldelli-Maia, J. M. & Ventriglio, A. The outbreak of COVID-19 coronavirus and its impact on global mental health. Int. J. Soc. Psychiatry 66 (4), 317–320. https://doi.org/10.1177/0020764020915212 (2020).

Hawkley, L. C., Masi, C. M., Berry, J. D. & Cacioppo, J. T. Loneliness is a unique predictor of age-related differences in systolic blood pressure. Psychol. Aging 21 (1), 152–164 (2006).

Hawkley, L. C., Thisted, R. A., Masi, C. M. & Cacioppo, J. T. Loneliness predicts increased blood pressure: 5-year cross-lagged analyses in middle-aged and older adults. Psychol. Aging 25 (1), 132–141 (2010).

Article   PubMed   PubMed Central   Google Scholar  

British Library. www.bl.uk . Available from: https://www.bl.uk/collection-items/loneliness-and-social-isolationamong-older-people-in-north-yorkshire-project-commissioned-by-north-yorkshireolder-peoples-partnership-board-executive-summary . Accessed 9 May 2023

Liffe, S. et al. Health risk appraisal in older people 1: Are older people living alone an “at-risk” group?. Br. J. Gen. Pract. 57 (537), 277–282 (2007).

Google Scholar  

Locher, J. L. et al. Social isolation, support, and capital and nutritional risk in an older sample: Ethnic and gender differences. Soc. Sci. Med. 60 (4), 747–761 (2005).

Matthews, T. et al. Sleeping with one eye open: Loneliness and sleep quality in young adults. Psychol. Med. 47 (12), 2177–2186 (2017).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Cacioppo, J. T. et al. Loneliness within a nomological net: An evolutionary perspective. J. Res. Pers. 40 (6), 1054–1085 (2006).

Wei, M., Russell, D. W. & Zakalik, R. A. Adult attachment, social self-efficacy, self-disclosure, loneliness, and subsequent depression for freshman college students: A longitudinal study. J. Couns. Psychol. 52 (4), 602–614 (2005).

Coyle, C. E. & Dugan, E. Social Isolation, loneliness and health among older adults. J. Aging Health 24 (8), 1346–1363 (2012).

Masi, C. M., Chen, H. Y., Hawkley, L. C. & Cacioppo, J. T. A meta-analysis of interventions to reduce loneliness. Personal. Soc. Psychol. Rev. 15 (3), 219–266 (2010).

Choi, M., Kong, S. & Jung, D. Computer and internet interventions for loneliness and depression in older adults: A meta-analysis. Healthc. Inform. Res. 18 (3), 191 (2012).

Hagan, R., Manktelow, R., Taylor, B. J. & Mallett, J. Reducing loneliness amongst older people: A systematic search and narrative review. Aging Ment. Health 18 (6), 683–693 (2014).

Morris, M. E. et al. Smart technologies to enhance social connectedness in older people who live at home. Australas J. Ageing 33 (3), 142–152 (2014).

Poscia, A. et al. Interventions targeting loneliness and social isolation among the older people: An update systematic review. Exp. Gerontol. 102 , 133–144 (2018).

BMJ. The effects of isolation on the physical and mental health of older adults. The BMJ . https://blogs.bmj.com/bmj/2020/04/09/the-effects-of-isolation-on-the-physical-and-mental-health-of-older-adults (2020).

Appel, L. et al. Older adults with cognitive and/or physical impairments can benefit from immersive virtual reality experiences: A feasibility study. Front. Med. 15 , 6 (2020).

Tussyadiah, I. P., Wang, D., Jung, T. H. & Dieck, M. C. Virtual reality, presence, and attitude change: Empirical evidence from tourism. Tour. Manag. 66 , 140–154 (2018).

Lin, C. X., Lee, C., Lally, D. & Coughlin, J. Impact of Virtual Reality (VR) Experience on Older Adults’ Well-Being. In Human Aspects of IT for the Aged Population. Applications in Health, Assistance, and Entertainment Vol. 10927 (eds Zhou, J. & Salvendy, G.) (Springer International Publishing, 2018). https://doi.org/10.1007/978-3-319-92037-5_8 .

Chapter   Google Scholar  

Lin, T. Y. et al. Effects of a combination of three-dimensional virtual reality and hands-on horticultural therapy on institutionalized older adults’ physical and mental health: Quasi-experimental design. J. Med. Internet Res. 22 (11), e19002 (2020).

Brown, J. A. An exploration of virtual reality use and application among older adult populations. Gerontol. Geriatr. Med. 5 , 233372141988528. https://doi.org/10.1177/2333721419885287 (2019).

Baker, S. et al. Interrogating social virtual reality as a communication medium for older adults. Proc. ACM Human-Comput. Interact. 3 (CSCW), 1–24 (2019).

Baker, S. et al. Evaluating the use of interactive virtual reality technology with older adults living in residential aged care. Inf. Proc. Manag. 57 (3), 102105 (2020).

Mandryk, R. L., Frommel, J., Armstrong, A. & Johnson, D. How passion for playing world of warcraft predicts in-game social capital, loneliness, and wellbeing. Front. Psychol. 11 , 2165 (2020).

Martončik, M. & Lokša, J. Do World of Warcraft (MMORPG) players experience less loneliness and social anxiety in online world (virtual environment) than in real world (offline)?. Comput. Human Behav. 56 , 127–134 (2016).

Miller, K. J. et al. Effectiveness and feasibility of virtual reality and gaming system use at home by older adults for enabling physical activity to improve health-related domains: A systematic review. Age Ageing 43 (2), 188–195 (2013).

Lee, L. N., Kim, M. J. & Hwang, W. J. Potential of augmented reality and virtual reality technologies to promote wellbeing in older adults. Appl. Sci. 9 (17), 3556 (2019).

Knowles, L. M., Stelzer, E. M., Jovel, K. S. & O’Connor, M. F. A pilot study of virtual support for grief: Feasibility, acceptability, and preliminary outcomes. Comput. Human Behav. 73 , 650–658 (2017).

Gorinelli, S., Gallego, A., Lappalainen, P. & Lappalainen, R. Virtual reality acceptance and commitment therapy intervention for social and public speaking anxiety: A randomized controlled trial. J. Context. Behav. Sci. 28 , 289–299 (2023).

Kim, M. K., Eom, H., Kwon, J. H., Kyeong, S. & Kim, J. J. Neural effects of a short-term virtual reality self-training program to reduce social anxiety. Psychol. Med. 52 (7), 1296–1305 (2022).

Creswell, J. W. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches 4th edn. (Sage Publications Ltd, 2014).

Russell, D. W. UCLA loneliness scale (Version 3): Reliability, validity, and factor structure. J. Personal. Assess. 66 (1), 20–40. https://doi.org/10.1207/s15327752jpa6601_2 (1996).

Article   CAS   Google Scholar  

Connor, K. M. et al. Psychometric properties of the social phobia inventory (SPIN). Br. J. Psychiatry 176 (4), 379–386 (2000).

Article   CAS   PubMed   Google Scholar  

Cohen, J. Statistical Power Analysis for the Behavioral Sciences 2nd edn. (Routledge, 1988).

MATH   Google Scholar  

Nowland, R., Necka, E. A. & Cacioppo, J. T. Loneliness and social internet use: Pathways to reconnection in a digital world?. Perspect. Psychol. Sci. 13 (1), 70–87. https://doi.org/10.1177/1745691617713052 (2017).

Hwang, T. J., Rabheru, K., Peisah, C., Reichman, W. & Ikeda, M. Loneliness and social isolation during the COVID-19 pandemic. Int. Psychogeriatr. 32 (10), 1–15 (2020).

Döring, N. et al. Can communication technologies reduce loneliness and social isolation in older people? A scoping review of reviews. Int. J. Environ. Res. Public Health 19 (18), 11310 (2022).

Tajfel, H. & Turner, J. C. An Integrative Theory of Inter-Group Conflict. In The Social Psychology of Inter-Group Relations (eds Austin, W. G. & Worchel, S.) 33–47 (Brooks/Cole, 1979).

Cole, H. & Griffiths, M. D. Social interactions in massively multiplayer online role-playing gamers. Cyberpsyhol. Behav. 10 (4), 575–583 (2007).

Baumeister, R. F. & Leary, M. R. The need to belong: Desire for interpersonal attachments as a fundamental human motivation. Psychol. Bull. 117 (3), 497–529 (1995).

de Gierveld, J. J., van Tilburg, T. G. & Dykstra, P. A. New Ways of Theorizing and Conducting Research in the Field of Loneliness and Social Isolation. In The Cambridge Handbook of Personal Relationships (eds Vangelisti, A. L. & Perlman, D.) 391–404 (Cambridge University Press, 2018).

Braun, V. & Clarke, V. Using thematic analysis in psychology. Qual. Res. Psychol. 3 (2), 77–101. https://doi.org/10.1191/1478088706qp063oa (2006).

Braun, V. & Clarke. V. Thematic analysis . APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological 2 (2), 57–71 (2012).

Ernst CPH. The Influence of Perceived Belonging on Massively Multiplayer Online Role-Playing Games. Proceedings of the 50th Hawaii International Conference on System Sciences (2017).

Morahan-Martin, J. & Schumacher, P. Loneliness and social uses of the Internet. Comput. Human Behav. 19 (6), 659–671 (2003).

Teppers, E., Luyckx, K. A., Klimstra, T. & Goossens, L. Loneliness and Facebook motives in adolescence: A longitudinal inquiry into directionality of effect. J. Adolesc. 37 (5), 691–699 (2014).

Tsai, H. H. & Tsai, Y. F. Changes in depressive symptoms, social support, and loneliness over 1 year after a minimum 3-month videoconference program for older nursing home residents. J. Med. Internet Res. 13 (4), e93 (2011).

Valkenburg, P. M. & Peter, J. Preadolescents’ and adolescents’ online communication and their closeness to friends. Dev. Psychol. 43 (2), 267–277 (2007).

Leung, L. Loneliness, social support, and preference for online social interaction: The mediating effects of identity experimentation online among children and adolescents. Chin. J. Commun. 4 (4), 381–399 (2011).

Download references

Author information

Authors and affiliations.

School of Psychology, Faculty of Health and Wellbeing, University of Bolton, Deane Road, Bolton, BL3 5AB, UK

Keith Kenyon & Jacqui Harrison

Faculty of Biology, Medicine and Health, The University of Manchester, Coupland Building 3, Oxford Road, Manchester, M13 9PL, UK

Vitalia Kinakh

You can also search for this author in PubMed   Google Scholar

Contributions

The authors confirm contribution to the paper as follows: study conception: K.K.; design: K.K. and V.K.; data collection and analysis: K.K. and J.H.; interpretation of results: K.K. and J.H.; draft manuscript preparation: K.K.; critically revising draft manuscript: V.K. and J.H. All authors reviewed the results and approved the final version of the manuscript.

Corresponding authors

Correspondence to Keith Kenyon , Vitalia Kinakh or Jacqui Harrison .

Ethics declarations

Competing interests.

The authors declare no competing interests. 

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Kenyon, K., Kinakh, V. & Harrison, J. Social virtual reality helps to reduce feelings of loneliness and social anxiety during the Covid-19 pandemic. Sci Rep 13 , 19282 (2023). https://doi.org/10.1038/s41598-023-46494-1

Download citation

Received : 22 May 2023

Accepted : 31 October 2023

Published : 07 November 2023

DOI : https://doi.org/10.1038/s41598-023-46494-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.