If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Correspondence: Address to Dr. Agniva Majumdar, MD, Scientist C (Medical), Division of Regional Virus Research and Diagnostic Laboratory, ICMR-National Institute of Cholera and Enteric Diseases, Kolkata, West Bengal, India.
To examine the availability, characteristics, content, reliability, and quality of YouTube videos on the recent monkeypox outbreak.
Methods
A systematic search was carried out on YouTube using the keywords “Monkeypox” on May 31, 2022. Video-related metrics were entered into the EpiCollect5 application simultaneously while watching the videos. A total of 599 videos were taken for final analysis. On the basis of content, the videos were categorized as useful, misleading, and news updates. Standardized tools were utilized to assess the content, quality, and reliability of the useful videos.
Results
Videos attracted cumulative 16,825,261 views and 591,932 likes. A total of 272 (45.4%) videos were categorized as useful, 122 (20.4%) videos as misleading, and 205 (34.2%) videos as news reports, with a 0.96 kappa coefficient of agreement between the 2 accessors. Mass media contributed to approximately 69% of the total videos. The mean Global Quality Scale, medical information and content index, and modified DISCERN scores were 2.8±0.7, 2.2±0.8, and 3.07±0.7 respectively.
Conclusion
The present study found that nearly half of the selected videos on recent monkeypox outbreaks contained useful information. Health care professionals and health-related organizations should be encouraged to disseminate scientifically correct information during public health emergencies. One-fifth of the videos contained misleading information, which may be due to a lack of quality assessment and peer review processes. Hence, strict caution and strategies for proper verification procedures should be exercised before uploading any health-related video on this platform.
The ongoing outbreak was first reported from the United Kingdom by the UK Health Security Agency on May 7, 2022. From early May 2022 to September 21, 2022, 61,753 confirmed cases and 23 deaths have been confirmed from 105 nonendemic countries.
Re-emergence of zoonotic infectious diseases with epidemic potential has raised the importance of quick dissemination of precise information related to preventive measures and practice-specific behaviors among the general population. Social media platforms have flourished as the most influential source of medical information at the current time, and the general public usually prefers audio-visual presentation rather than only audio or text information.
According to a recent survey on Americans, YouTube and Facebook were found to be the most dominant social media platforms. As per the survey, 81% of Americans use YouTube and 69% use Facebook.
This increasing popularity is due to several factors, ie, ease of use and access, large volume, and variety of content. Because of its huge popularity and easy accessibility, YouTube can play an important role in relieving public anxiety and imposing public health measures to contain the outbreaks.
Are internet videos useful sources of information during global public health emergencies? A case study of YouTube videos during the 2015-16 Zika virus pandemic.
have examined the content of YouTube videos on occurrence of earlier epidemics, to the best of our knowledge, no prior studies have assessed YouTube as a source of information in the current monkeypox outbreak. We conducted this cross-sectional analytical study to evaluate YouTube videos linked to present monkeypox outbreak and to measure the accuracy, reliability, and quality of the information delivered through these videos.
Methods
Data Extraction and Data Entry
Four independent members (B.G., S.K., P.K.J., and S.P.) of our research team conducted a search on YouTube on May 31, 2022, by entering the keyword “Monkeypox.” The search word was selected on the basis of discussion by the investigation team, which comprised physicians, microbiologists, scientists, and public health specialists. The data collection form was created on the web-based application EpiCollect5 (five.epicollect.net). Videos were selected as per our inclusion and exclusion criteria (Figure 1) and, simultaneously, data entry was conducted in EpiCollect5 (android application) by members of the team while watching the videos. The uniform resource locator (URL) field was kept at the beginning of the data entry form and was made unique for all the entries. This technique of data entry prevented duplicate entries in the portal and saved time by avoiding viewing of same videos more than once. The team members entered video-related information in the EpiCollect5 application of their android phones in the following order: 1) serial number (1-200 for B, 201-300 for S.K., 301-400 for P.K.J., and 401-600 for S.P.); 2) URL; 3) views; 4) likes; 5) number of comments; 6) channel subscribers; 7) channel authorship type [indicating whether an individual or organization uploaded the videos, categorized into 5 types: a) mass media; b) health-related organization; c) health professionals; d) people and blogs; and e) science and technology related organizations or professionals]; 8) video publishing date; 9) duration of videos (in seconds); and 10) country of origin. The changeable variables (variables that may change over the course of time, ie, views, likes, number of comments, and channel subscribers) were captured on EpiCollect5 application on May 31, 2022, for ease of analysis and interpretation. The rest of the data was collected at the convenience of the accessors.
FIGURE 1Flow diagram for review of YouTube videos on Monkeypox.
); 2) available on May 31, 2022; and 3) related to recent “Monkeypox” outbreak (May 2022 onward).
Exclusion Criteria for Videos
1) Videos with English titles but non-English content; 2) non-English videos with English subtitles; 3) mute videos; 4) advertisements; 5) animations or clips from movies; 6) videos with more than 20-minute duration [Under the Duration tab, YouTube has 3 categories of videos, ie <4 minutes, 4-20 minutes, and >20 minutes. The first 2 categories of videos were included for the convenience of data collection]. Videos of more than 20-minute duration were filtered out, as the video ranking system
graphs found increased popularity after 2 minutes and touched a peak value at 15-16 minutes; and 7) In-case of several series videos, only the first video (according to the upload date) of the series was included in the analysis.
Video Evaluation
After initial data entry, the Microsoft Excel file was extracted from the EpiCollect5 application and shared among two public health specialists (S.P. and P.KR.) to evaluate the content, reliability, and quality of the videos and to review the viewer-posted comments. S.P. and P.K.R. were approached to keep the study bias-free as both were working in different organizations and were from different states. Any doubts or differences were resolved via telephonic conversation by a public health specialist (P.K.J.) and microbiologist (A.M.) from the study center. Based on the central message, our accessors categorized the videos into 1) Useful (covering scientifically right information on epidemiology, prevalence, sign and symptoms, treatment, and diagnosis prevention of monkeypox disease); 2) Misleading (having at least 1 scientifically unproven datum, eg, “Well, it was only a matter of time before someone started blaming the COVID-19 vaccines for the monkeypox outbreak”); and 3) News update (videos uploaded by news agencies with information regarding the present situation of the disease in terms of first case detection, mortality, and spread only, without any information about prevention, treatment, or public health intervention to combat the disease). This classification was performed on the basis of reporting from other similar studies.
After watching complete videos, the reviewer also categorized the videos on the basis of their content: 1) Description of the disease; 2) Question answers (interviews); 3) Reporting of the disease; 4) Next pandemic; 5) Vaccine-related; 6) Prevention of the disease; 7) lesbian, gay, bisexual and transgender stigma, and 8) Stock market insecurities.
Medical information and content index (MICI), Global Quality Scale (GQS), and modified DISCERN (mDISCERN) tool were used for evaluating the content, quality, and reliability of the videos respectively. DISCERN tool is a 5-point scale that has been used previously for similar studies.
Are internet videos useful sources of information during global public health emergencies? A case study of YouTube videos during the 2015-16 Zika virus pandemic.
Are internet videos useful sources of information during global public health emergencies? A case study of YouTube videos during the 2015-16 Zika virus pandemic.
also used the GQS to categorize videos on the basis of quality, information flow, and usefulness of the videos. On the basis of current knowledge on monkeypox, we developed the MICI scale (5-point Likert scale) to assess medical information from useful videos. Previous studies
also used this scale during previous outbreaks for content analysis. The full list of criteria and scoring system for MICI, mDISCERN, and GQS is available in Supplemental Tables 1, 2, and 3, respectively (available online at http://www.mcpdigitalhealth.org/). Inter-accessor agreement between the categorization of videos, mDISCERN, GQS, and MICI scores were also analyzed. If the MICI, GQS, or mDISCERN scores of any particular video given by the 2 accessors (S.P. and P.K.R.) differed, then the final score was given by accessors (P.K.J. and A.M.) from the study center. All 3 scores were averaged to obtain a final score, which was considered for data analysis.
Ethical Consideration
As the present study dealt with only public access data, we did not obtain any approval from the Institutional Ethics Committee. Previous studies
on YouTube data have also not received any ethics approval.
Statistical Analyses
The analysis was executed using IBM SPSS Statistics for Windows, Version 20.0. (Armonk, NY: IBM Corp). The quantitative and categorical variables were summarized as median with range and percentage with 95% CIs, respectively. Cohen’s kappa coefficient was used to assess the degree of agreement between the 2 assessors. Shapiro–Wilk test was used to check the normality of data. The Kruskal-Wallis statistical test was performed for intergroup comparisons, and the Mann-Whitney U test was used for 2-group comparisons. P<.05 was considered statistically significant. Unadjusted odds ratios (ORs) with 95% CIs were calculated. Variables with a P value of <.05 were considered for multivariable logistic regression model to get an adjusted OR.
Results
After applying the exclusion criteria, a total of 599 videos were considered for statistical analyses (Figure 1). The total length of all the videos was 51.5 hours, with a cumulative number of 16,825,261 views, 591,932 likes, and 90,714 comments. A total of 272 (45.4%; 95% CI, 41.4-49.4%) videos were categorized as useful, 122 (20.4%; 95% CI, 17.3-23.8%) videos as misleading, and 205 (34.2%; 95% CI, 30.5-38.1%) videos as news reports (Figure 2), with a 0.96 kappa coefficient (P<.001) of agreement between the 2 accessors. Figure 2 shows the contribution of different video-producing sources for uploading useful videos. Statements of the misleading videos were transcribed and further classified into 4 thematic areas, ie, related to 1) stigmatization, 2) trepidation, 3) vaccination, and 4) miscellaneous, as shown in Supplemental Table 4 (available online at https://www.mcpdigitalhealth.org/). The news channel contributed to approximately 69.2% (n=415) of the total videos, followed by people and blogs (22.5%, n=135). Table 1 shows the summary characteristics of video-related metrics. The mean number of views, views/days, likes, likes/days, and days on YouTube, video length, comments, and channel’s subscribers for 3 categories of videos are described in Table 2. The mean number of likes and comments were found to be 2.8 times (2385 vs 856; P≤.001) and 2.2 times (246 vs 112; P≤.001) higher in misleading videos, respectively, compared with useful videos. There was no significant difference in the mean number of views, views/days, and days on YouTube (P=.06, P=.14, and P=.41, respectively) among the 3 groups (Table 2). The mean GQS, MICI, and mDISCERN scores of useful videos were found to be 2.82±0.78, 2.26±0.85, and 3.07±0.76, respectively (Table 1). The kappa coefficient of agreement between 2 reviewers for mean GQS, MICI, and mDISCERN scores were 0.81, 0.89, and 0.85, respectively, with a P value of <.001. As per our medical information and content analysis, more than 80% of useful videos addressed the disease prevalence, transmission, clinical symptoms, and treatment, whereas only 8% of videos discussed the screening or testing section (Table 1). Table 3 shows a comparison of mean mDISCERN, MICI, and GQS scores with different video-related metrics. Multivariable logistic regression model to predict misleading information of the videos using several video-related metrics is described in Table 4. Videos uploaded by people and blogs were more likely to be misleading (adjusted OR=11.5; 95% CI, 6.08-21.73; P<.001) compared with the other video-uploading source. YouTube channels having less than 50,000 subscribers were significantly producing 2.7 times more misleading (adjusted OR=2.71; 95% CI, 1.01-7.24; P<.05) videos compared with channels having ≥10 lakhs subscribers. Videos which got less than 50 likes and 50 to <500 likes were 0.2 (adjusted OR=0.26; 95% CI, 0.10-0.64; P<.005) and 0.4 (adjusted OR=0.45; 95% CI, 0.22-0.95; P<.05) times misleading, respectively, compared with videos with ≥500 likes.
FIGURE 2Distribution chart showing contribution of various video-producing sources for uploading different categories videos on Monkeypox.
TABLE 1Summary of Characteristics of the Videos on Monkeypox (N=599)
Characteristic
Parameter
Total count
Median (range)=$, % (95% CI)=¥, Mean (SD)=€
Video metrics (N=599)
Number of views
16,825,261
4402 (2-1,507,777)$
Likes
591,932
66 (0-41,000)$
Comments
90,714
7 (0-3400)$
Subscribers
1,358,644,894
365,000 (0-221,000,000)$
No. of days in YouTube
4874
8.00 (1-22)$
Video length (minutes)
3090.25
4.05 (0.33 -19.53)$
Source of video (descriptive data) (N=599)
Mass media (including television and journals)
415
69.2 (65.45-72.85)¥
People and blogs (user-generated content)
135
22.5 (19.35-26.06)¥
Health professional
36
6.01 (4.3-8.2)¥
Science and technology
10
1.6 (0.89- 3.07)¥
Health-related organizations (World Health Organization [WHO], Centers for Disease Control and Prevention[ CDC], Indian Council of Medical Research [ICMR] etc.)
3
0.5 (0.16-1.54)¥
Origin of the videos according to WHO region (N=598
Country origin of one video could not be possible to retrieve
African Region
23
3.85 (2.56-5.72)¥
Region of the Americas
360
60.20 (56.20-64.06)¥
South-East Asian Region
51
8.53 (6.53-11.05)¥
European Region
99
16.56 (13.77-19.76)¥
Eastern Mediterranean Region
7
1.17 (0.55-2.43)¥
Western Pacific Region
58
9.70 (7.56-12.35)¥
Content of the video (N=599)
Description of the disease
188
31.39 (27.78-35.22)¥
Question answer (Interviews)
142
23.71 (20.46 -27.28)¥
Reporting of the disease
137
22.87 (19.67-26.41)¥
Next pandemic
43
7.17 (5.36-9.54)¥
Vaccine related
27
4.51 (3.10-6.49)¥
Prevention of the disease
26
4.34 (2.96-6.30)¥
Lesbian, gay, bisexual and transgender (LGBT) stigma
TABLE 3Comparison of the DISCERN, MICI, and GQS Scores for Each of the Useful Videos (N=272) According to Uploading Sources, Video Length, Origin, Views, Chanel’s Subscription, Comments, and Likes
Characteristic
Number n (%)
Views n (%)
Modified DISCERNscore
MICI score
GQS score
Mean ± SD
P value
Mean ± SD
P value
Mean ± SD
P value
Uploading Source
Mass media
184 (67.65%)
3,504,885 (49.48%)
3.02 (0.72)
<.001
2.02 (0.73)
<.001
2.63 (0.70)
.001
People and blogs
42 (15.45%)
896,220 (12.65%)
2.83 (0.93)
2.47 (0.70)
2.90 (0.79)
Health professional
33 (12.13%)
2,189,073 (30.90%)
3.51 (0.61)
3.15 (0.87)
3.57 (0.66)
Science and technology
10 (3.68%)
420,783 (5.94%)
3.5 (0.70)
2.9 (1.10)
3.4 (0.69)
Health-related organization
3 (1.10%)
71,572 (1.01%)
3 (0)
2.33 (1.15)
3 (1)
Video Length
<4 minutes
100 (36.76%)
1,993,548 (28.14%)
3.02 (0.77)
.29
2.07 (0.78)
.005
2.56 (0.74)
<.001
4-20 minutes
172 (63.24%)
5,088,985 (71.85%)
3.10 (0.76)
2.38 (.088)
2.97 (0.76)
Origin of the videos according to WHO region
African Region
16 (5.90%)
42,849 (0.60%)
2.93 (0.68)
.67
2.43 (0.89)
.33
2.93 (0.77)
.76
Region of the Americas
144 (53.14%)
4,853,878 (68.53%)
3.02 (0.78)
2.34 (0.84)
2.84 (0.77)
South-East Asian Region
23 (8.49%)
291,255 (4.11%)
3.17 (0.83)
2.30 (0.82)
2.96 (0.82)
European Region
53 (19.56%)
1,525,648 (21.54%)
3.18 (0.76)
2.07 (0.87)
2.75 (0.75)
Eastern Mediterranean Region
4 (1.48%)
35,597 (0.50%)
2.75 (0.95)
2.5 (0.57)
3 (0.81)
Western Pacific Region
31 (11.4%)
332,754 (4.69%)
3.16 (0.68)
2.12 (0.92)
2.66 (0.83)
Views
<1000 views
95 (34.93%)
27,007 (0.38%)
3.12 (0.80)
.06
2.31(0.91)
.28
2.86 (0.75)
.27
1000 to <5000 views
62 (22.79%)
164,198 (2.31%)
3.22 (0.63)
2.12(0.87)
2.67(0.80)
≥5000 views
115 (42.28%)
6,891,328 (97.30%)
2.94 (0.79)
2.31 (0.79)
2.86 (0.78)
Channel’s subscription
<50000 subscription
74 (27.21%)
222,722 (3.14%)
3.12 (0.87)
.49
2.55 (0.87)
.003
3.09(0.72)
<.001
50000 to <10 lakh subscription
113 (41.54%)
2,269,125 (32.03%)
3.09 (0.68)
2.12 (0.84)
2.71 (0.81)
≥10 lakh subscription
85 (31.25%)
4,590,686 (64.81%)
3 (0.78)
2.21 (0.80)
2.72 (0.73)
Comments
Zero comment or comments not available
63 (23.16%)
887,655 (12.53%)
2.98 (0.81)
.28
2.06 (0.85)
.10
2.61 (0.77)
.09
<50 comments
149 (54.78%)
2,990,927 (42.22%)
3.14 (0.73)
2.35 (0.87)
2.87 (0.78)
≥50 comments
60 (22.06%)
3,203,951 (45.23%)
3 (0.80)
2.26 (0.78) `
2.91 (0.76)
Likes
<50 likes
141 (51.84%)
185,576 (2.62-%)
3.13 (0.73)
.42
2.21 (0.85)
.04
2.78 (0.75)
.22
50 to < 500 likes
79 (29.04%)
1,139,714 (16.09%)
3.02 (0.76)
2.18 (0.81)
2.77 (0.79)
≥500 likes
52 (19.12%)
5,757,243 (81.28%)
2.98 (0.85)
2.51 (0.89)
3 (0.81)
MICI, medical information and content index; GQS, Global Quality Scale.
In the midst of the worldwide fight against the COVID-19 pandemic, the World Health Organization (WHO) declared the ongoing monkeypox outbreak a Public Health Emergency of International Concern on July 23, 2022.
Electronic multimedia penetrates each part of medicine and health care. YouTube is the second most visited website globally, with one billion hours of video watching by its 1.24 billion monthly users.
The performance of an uploaded video depends on the number of views, likes, and comments. In the present study, the total number of views, likes, and comments were 16,825,261, 591,932, and 90,714, respectively, which is much higher than that in previous studies
Are internet videos useful sources of information during global public health emergencies? A case study of YouTube videos during the 2015-16 Zika virus pandemic.
on outbreaks. The reason may be that our study evaluated 4-5 times more videos than previous studies, the increasing popularity of YouTube, and the worldwide spread of the disease. However, viewership of the present study appears to be lower than that in studies
on 2019 SARS-CoV-2 outbreaks. The reason may be that the data collection of the present study was completed very early in the outbreak (the mean number of days on YouTube is approximately 7 days) and COVID-19 is the largest pandemic in history.
Most of the videos were produced by mass media (69%), followed by people and blogs (22%), and this finding is similar to results from earlier studies
Are internet videos useful sources of information during global public health emergencies? A case study of YouTube videos during the 2015-16 Zika virus pandemic.
outbreaks reported that mass media and independent users contributed an equal number of videos. However, the higher number of videos was uploaded by an independent user during the West Nile virus infection.
At present, there is no peer review system; anyone can upload videos on YouTube without any restrictions. Hence, a structured review system should be in place before uploading medical information-related videos on this knowledge sharing platform.
Our study revealed that one-fifth of YouTube videos on the recent monkeypox outbreak contained misleading information. A significant proportion of misleading videos were uploaded during previous outbreaks as well, ie, 27.5% during the novel coronavirus outbreak,
Are internet videos useful sources of information during global public health emergencies? A case study of YouTube videos during the 2015-16 Zika virus pandemic.
The reason may be that they evaluated very few videos compared with that in our study and they did not consider news updates as a separate category. We considered 34% of the total evaluated videos as news updates, as those were only disseminating information regarding case detection, disease’s current situation, and mortality-related data without any misleading information or any helpful medical information for viewers. D'Souza et al
created a separate category for this type of videos as ‘neither useful nor misleading’. During the previous outbreak, YouTube studies on influenza pandemic,
also reported 22.5%, 32%, and 21% videos as news updates, respectively.
The mean GQS, mDISCERN, and MICI scores in the current study were 2.82, 3.07, and 2.26, respectively. Videos uploaded by health professionals scored the highest points. Previous studies
also revealed that high-quality videos are predominantly produced by health care professionals. As reliability, quality, and medical information content scores completely depend on the video-uploading source, it is highly recommended to consider video-uploading sources while acquiring medical information-related knowledge on this platform. The study also found a positive association between the length of the videos with mDISCERN, GQS, and MICI scores. YouTube studies on delayed ejaculation
also revealed a positive correlation between the duration of video with DISCERN and GQS scores. The MICI scores of the current study were calculated to be lower than that in earlier studies,
which may be due to limited data available worldwide on monkeypox. In the case of newly emerging pathogens, time is crucial to optimize scientific evidence-related data. Rishu et al
reported that a median of 335 days was needed to conduct an outbreak-oriented observational study. Most of the useful videos portrayed epidemiological characteristics, transmission, sign-symptoms, and treatment of the disease, whereas diagnosis and prevention are less discussed. Only 8% of useful videos focused on screening or testing. Previous studies
during the coronavirus outbreak reported the same findings. However, discussion on the prevention of disease on this social media platform is more desirable to halt an epidemic.
Strengths
We evaluated a total of 589 videos, which is 4-9 times higher than that in most previous studies
Are internet videos useful sources of information during global public health emergencies? A case study of YouTube videos during the 2015-16 Zika virus pandemic.
evaluating disease outbreak information on YouTube. To the best of our knowledge, this is the first study in which EpiCollect5 (android application) was used for checking the uniqueness of URL links of videos. This data entry technique saved a lot of time, avoiding unnecessary duplicate video removal such as in previous studies
during analysis. The innovative data entry procedure using EpiCollect5 application will show the way to future research projects on YouTube. Standardized tools were utilized to evaluate the quality, reliability, and content of the videos, in contrast with mostly descriptive YouTube studies published earlier. The study was conducted in the very early stage of the outbreak; hence, the findings could guide health care professionals and health-related organizations in collaboration with content makers on YouTube to convey high-quality useful information to stop disease spread.
Limitations
The study could not account for the varying content of YouTube over time as it was a cross-sectional snapshot study and evaluated videos on the recent monkeypox outbreak. Videos posted during the previous outbreaks or after May 31, 2022, were not evaluated. The present study was restricted to a direct search of YouTube and did not explore YouTube videos accessed from different links or other websites. We evaluated only English language videos, and videos of more than 20-minute duration were not included in our study. Video power index was not measured due to the nonavailability of dislike data on YouTube. Subjective methods were executed to evaluate the reliability, quality, and usefulness of the videos, although a high level of agreement was found between the 2 reviewers.
Conclusion
Although YouTube has pros and cons, responsible use can disseminate scientifically correct information and avoid unnecessary panic during a pandemic. The present study found that nearly half of the total videos contained useful information on the monkeypox outbreak. One-fifth of the videos were found to be disseminating misleading information; hence, there should be a strict caution and verification process before these videos are made available for the public. This study also revealed that videos uploaded by health professionals scored highest points in accordance with the quality, reliability, and medical content of the videos. These findings can stimulate health professionals to produce more videos for the society during such public health emergencies. Only 7% of the total videos were uploaded by health professionals and health-related organizations according to the present study. Health professionals and health-related organizations should utilize this social media platform to positively influence the preventive behaviors of the general population by disseminating outbreak-specific useful information.
Potential Competing Interests
The authors report no competing interests.
Acknowledgments
PKJ, AM, and SD were responsible for the concept and design. PKJ, AM, SP, PKR, AD, SM, and AC developed the search strategy for the video selection and also developed the EpiCollect5 form for data collection. PKJ, SP, PKR, BG, and SK did the data extraction, critical appraisal, and coding. PKJ, AM, and SD contributed to conflict resolution during scoring the videos. PKJ, SP, and PKR contributed to the statistical analyses. PKJ drafted the manuscript. All authors critically revised the manuscript for intellectually important content and had access to the data. SD, AM, AD, SM, and AC verified the underlying data. PKJ and AM were responsible for the overall monitoring of the project.
Are internet videos useful sources of information during global public health emergencies? A case study of YouTube videos during the 2015-16 Zika virus pandemic.
00006-8&id=gr1.jpg){kind=link}
00006-8&id=gr2.jpg){kind=link}