Wikipedia as a Reliable Information Source: A Comparision of Chinese and English Versions | Blog scientifique de l'Institut Confucius de l'Université de Genève

Peiyuan SUN

In the digital age, Wikipedia has grown to be the biggest online encyclopedia in terms of comprehensiveness, reach and coverage. It has more than fifty-five million articles in around 300 languages, making it one of the most-widely visited websites around the globe (Mostafa, 2023). Most importantly, Wikipedia has a large amount of volunteers who produce the content for free and edit existing posts through the adoption of an online “volunteer crowdsourcing” model (Jones, 2018; Wikipedia, 2023). This open editing policy has long been a point of contention in academic circles, which raised questions about its reliability and suitability as a reference in educational contexts. Some researchers suggest that “the exponential growth and reliability of Wikipedia make it an ideal knowledge resource for information retrieval” (Mostafa, 2023). However, there continues to be much debate about whether Wikipedia should be used in academia or not (Okoli et al., 2012; Arroyo-Machado et al., 2020).[1] The impermanence of encyclopedic entries and concerns about the quality of content have posed major challenges for years (Peoples, 2009).

The reliability of information sources in Chinese Wikipedia has been a subject of ongoing debate, particularly when compared to its English counterpart. Chinese Wikipedia has been blocked since May 2015 in mainland China.[2] As a result, the users who contribute the most to the Chinese Wikipedia are from Taiwan, Hong Kong, Macau, Singapore, Malaysia, and the large Chinese diaspora.[3] In addition, there are also some Wikipedia administrators from China who have permitted IP block exemption and that are recruited to modify Wikipedia’s editorial content to align with China’s perspectives and/or to back the election of administrators who are sympathetic to the Chinese government. This effort is believed as part of the Chinese government’s broader strategy to influence widely respected platforms and promote their preferred narratives.[4]

Recent research indicates significant disparities in the reliability of references across different language editions of Wikipedia, with the Chinese version demonstrating a higher prevalence of non-authoritative sources. This issue is predominantly linked to the frequent citation of sources such as the Chinese tabloid Huanqiu.com, or the international media outlet Epoch Times (known for its anti-Chinese Communist Party stance and controversial reporting practices), which contributes to a relatively lower proportion of reliable references in the Chinese edition (Baigutanova et al., 2023).

A valuable resource for assessing the reliability of Chinese information sources is China Fact Check (Chinese: 有据), an independent fact-checking organization founded by Wei Xing that is known for its hybrid model combining professional journalists and university collaboration (China Media Project, 2023).[5] Its assessment is that while Chinese Wikipedia’s entry explanations sometimes cite credible media sources, they also sometimes cite ordinary user-generated content – something to be explored further below.

Wikipedia’s Chinese content vulnerabilities are underscored by incidents like the Zhemao hoaxes, where a Chinese housewife fabricated extensive historical content over several years, exploiting gaps in the editorial review process.[6] This case not only damaged the credibility of the platform but also highlighted the ongoing challenges in ensuring source reliability and the integrity of information in Chinese.

This study will be a preliminary exploration into the reliability of Wikipedia as an information source for academic content, and will compare the reliability assessment of information sources between Chinese Wikipedia and English Wikipedia. This work aims to address two primary research questions:

To what extent can Wikipedia be considered a reliable source of information for academic content?
Based on the comparison with Wikipedia in English, is Wikipedia in Chinese a reliable source of information? How does its fact-checking mechanism work?

The first question will evaluate Wikipedia’s reliability by examining academic perspectives on its accuracy and completeness. This approach will provide insights into Wikipedia’s strengths and limitations as an information source, as well as its evolution over time. The second question explores the differences in reliability assessment between Chinese and English Wikipedia, focusing on their respective fact-checking mechanisms and source evaluation processes. We will examine how these two language versions of Wikipedia compare in terms of source categorization, reliability labeling, and the types of sources they rely on. Additionally, we will investigate the potential dependency of Chinese Wikipedia on English Wikipedia’s source reliability assessments.

By addressing these questions, this study seeks to contribute to the ongoing discourse on the reliability of Wikipedia as an information source, with a particular focus on the Chinese language version. Our goal is to provide insights into the complexities of maintaining consistent standards of reliability across linguistic and cultural boundaries in the context of a global, collaborative encyclopedia.

Academic Perspectives on Wikipedia’s Reliability

The reliability of Wikipedia as an information source has been a subject of extensive academic scrutiny over the past decade. This overview of the literature synthesizes some of the findings from various studies between 2010 and 2024, and offers an assessment of Wikipedia’s reliability and its evolving role in academic contexts.

The foundational question of Wikipedia’s reliability has been approached from multiple angles, with particular attention paid to specialized fields. In the medical domain, studies have consistently highlighted concerns about the platform’s accuracy and completeness. A study examining cardiovascular entries in Wikipedia found significant errors of omission, rendering it unsuitable as a primary resource for medical students (Azer et al., 2015). This finding was corroborated by another study in the same year focusing on respiratory topics, which also noted knowledge deficiencies and inaccuracies (Azer, 2015). Extending beyond medicine, a study in pharmacology compared Wikipedia to a specialized drug information database. It revealed that Wikipedia had a narrower scope, was less complete, and contained more errors of omission (Clauson et al., 2016). These studies collectively suggest that while Wikipedia may serve as a useful starting point for general information, it falls short as an authoritative source in specialized fields. This is also the case for encyclopedias in general, and it is important to remember when thinking about it as a teaching tool.

Wikipedia’s reliability is limited in specialized areas, and the user community’s perception of Wikipedia’s reliability is also generally more cautious. A study on young adults’ credibility assessment found that while students frequently use Wikipedia, their trust in its accuracy varies based on their familiarity with the topic and the presence of peripheral cues such as references and citations (Menchen-Trevino & Hargittai, 2011). Another study on university students’ perception also showed that while most respondents used Wikipedia regularly for both academic and leisure purposes, they harbored reservations about its accuracy and stability. (Amina & Warraich, 2022). In addition, the perception of Wikipedia’s reliability is not uniform across different user groups. These findings emphasize the complexity of user interactions with Wikipedia and highlight the need for a multifaceted approach to understanding its perceived reliability.

In academic contexts, attitudes towards Wikipedia have evolved significantly. The research done by Malik, Rafiq and Mahmood (2023) on university faculty patterns of use and perceptions found that faculty views have grown more favorable since 2012 regarding Wikipedia’s use, usefulness, and quality of information. Many faculty members are now actively integrating Wikipedia into their teaching, learning, and research endeavors, encouraging students to be critical and proficient users of the platform. This shift in academic perception is partially supported by another study in Estonian general education schools, which revealed that while teachers consider Wikipedia suitable for teaching alongside other sources, they still express some caution, largely due to attitudes inherited from Wikipedia’s early days (Reinsalu et al., 2023).

Efforts to improve Wikipedia’s reliability have been ongoing and have shown some positive outcomes. For example, a 2018 study comparing Wikipedia to Encyclopedia Britannica in U.S. politics found Wikipedia articles more biased towards Democratic views, but this bias decreased with more revisions (Greenstein & Zhu, 2018). A recent study confirmed a moderate liberal bias in Wikipedia’s news media sources (Yang & Colavizza, 2024). Additionally, Baigutanova et al. (2023) found that the quality of references has significantly improved over the past decade, with more than half of verifiable statements now accompanied by references, and the reference risk remaining low. These findings suggest that Wikipedia’s collaborative nature contributes to its ongoing improvement and reliability.

In addition to Wikipedia’s self-improvement mechanisms, users’ perceptions of its reliability are also improving, including a growing recognition of Wikipedia’s value as a starting point for research and its potential as a teaching tool when used critically. For instance, a 2024 study exploring how to explain Wikipedia’s credibility mechanisms to university students found that after experiencing content creation on Wikipedia, students’ perception of its credibility increased, with 70% believing Wikipedia to be reliable or more reliable than expected (Bidegain, Egaña, & Zuberogoitia, 2024). This suggests that hands-on engagement with Wikipedia’s processes can enhance users’ understanding of its reliability mechanisms.

In conclusion, a preliminary review of the academic literature shows that Wikipedia is still not accurate and reliable enough as a source of information in specialized fields. However, its reliability has improved over time, especially in terms of referencing, but it still requires careful use and verification. The academic community’s perception of Wikipedia has become more positive, with increasing efforts to integrate it into educational processes while teaching critical evaluation skills. Future research could focus on exploring how best to incorporate Wikipedia into educational curricula to capitalise on its strengths while reducing its limitations.

Chinese vs. English Wikipedia

In order to compare the reliability of Chinese and English Wikipedia, particularly in terms of their fact-checking mechanisms, this study employs a comparative analysis approach. According to China Fact Check (2023), the classification of fact-checking rating systems worldwide can be broadly divided into four categories based on the information carriers they primarily use: text, image, color and digital. Generally, both the Chinese and English Wikipedia’s ratings of reliable sources use a hybrid approach that incorporates elements of text, images and color. This approach not only visually presents the reliability levels of different sources in an intuitive manner, but also provides a relatively unified rating format, making it easier for users to compare the reliability of the same source across different language versions.

Given that the Chinese and English Wikipedias operate within different information ecologies, with unique editorial practices and available sources (Lei, 2016; Zeng, 2020), direct comparisons may highlight differences in source evaluation and reliability standards. The methodology consists of four primary steps: data collection, data cleaning and organization, data analysis, and results presentation and interpretation.

The data collection process involves gathering information from both English and Chinese Wikipedia platforms. For English Wikipedia, we accessed the “Reliable sources/Perennial sources” page[7] and extracted the list of reliable and controversial sources. Similarly, for Chinese Wikipedia, we accessed the equivalent page containing source reliability information list.[8] Due to the easy availability of data on Wikipedia, data collection was done by manual replication.

Following data collection, a cleaning and organization process was undertaken to ensure the data is suitable for analysis. This involves standardizing the format of both English and Chinese source lists to ensure consistency and organizing the source lists into a unified tabular format to facilitate subsequent comparison and analysis.

The data analysis phase comprises three main components. First, a quantitative comparison is conducted to calculate the number of sources in each reliability label for both English and Chinese Wikipedia and creating comparative charts to visualize these quantitative differences. Second, a qualitative comparison was made by categorizing sources by type and analyze differences in source types (e.g., news agency, website and newspaper, social media, government) between the two platforms. Lastly, a dependency analysis compares Chinese Wikipedia sources with those of English Wikipedia and identifies the sources of information shared in the two languages Wikipediaes to assess the degree of dependency of Chinese Wikipedia on English Wikipedia’s source rating system.

Table 1: List of Information Source Categories

Overall, we aim to assess the reliability of Chinese Wikipedia as an information source and evaluate the effectiveness of its fact-checking mechanism in comparison to English Wikipedia. This approach allows us to systematically compare the sources of information in the two-language versions of Wikipedia, and to gain a preliminary understanding of their characteristics and potential areas of improvement in ensuring the reliability of the information.

Findings

Our quantitative analysis firstly revealed differences in source categorization between English and Chinese Wikipedia. The English version employs five reliability labels for a total of 433 sources,[9] while the Chinese version uses six labels for 174 sources,[10] indicating a more extensive classification system in the English Wikipedia. Besides, a key difference lies in the proportion of “Generally reliable” sources. In English Wikipedia, this category is the largest, comprising 140 entries (32.3% of the total). In contrast, Chinese Wikipedia’s equivalent category “通常可靠” (Generally reliable) accounts for only 28 entries (16.1%), which suggests that Chinese Wikipedia might be more conservative in labelling whether a source is reliable. Another significant difference is in the treatment of sources without clear consensus. English Wikipedia groups these under “No consensus, unclear, or additional considerations apply,” making up 97 sources (22.4%). Chinese Wikipedia, however, has a distinct “无共识” (No consensus) category with 29 sources (16.7%), as well as a larger proportion of “半可靠” (Semi-reliable) sources at 41 (23.6%). This suggests that Chinese Wikipedia may have a more nuanced approach to sources that fall between clearly reliable and unreliable.[11] These quantitative differences highlight varying approaches to source evaluation between the two platforms, and potentially reflect differences in editorial policies, available sources, or cultural approaches to information reliability.

Figure 1: Distribution of Source Reliability Labels on Chinese Wikipedia

Figure 2: Distribution of Source Reliability Labels on English Wikipedia

Our qualitative analysis shows that both platforms rely on news agency/media/newspaper sources, but the English Wikipedia has a much higher usage rate. Chinese Wikipedia generally rates its sources as less reliable compared to English Wikipedia, with only 23% of news agencies, websites, and newspapers categorized as “通常可靠” (generally reliable), compared to 38% in English Wikipedia. Interestingly, while Chinese Wikipedia has a smaller proportion of unreliable/deprecated sources, it shows a higher percentage of sources lacking consensus on their reliability. This is evident in the significant number of sources categorized as “无共识” (No consensus) in Chinese Wikipedia, contrasting with English Wikipedia’s more definitive categorization system. This pattern suggests two key points: firstly, the credibility of news media in the Chinese-speaking world may still have room for improvement; [12] and secondly, the Chinese Wikipedia editing community may benefit from more constructive and in-depth communication to reach consensus on source reliability.

Figure 3: Comparison of Source Types between English and Chinese Wikipedia

Figure 4: Reliability Distribution of News Agency, Website, and Newspaper Sources on English Wikipedia

Figure 5: Reliability Distribution of News Agency, Website, and Newspaper Sources on Chinese Wikipedia

It is also worth noting that English Wikipedia has a wider range of source types, with significantly more entries in categories such as companies/institutions/organizations and online encyclopedias/databases. Interestingly, the Chinese Wikipedia relies more on social media sources. Regarding social media sources, the Chinese Wikipedia exhibits a notably stricter approach. Figure 6 illustrates that the majority of social media sources are classified as either “通常不可靠” (usually unreliable) or “列入黑名单” (blacklisted). This stringent evaluation of social media reliability stands in contrast to the English Wikipedia’s more varied categorization of sources overall. The Chinese Wikipedia’s approach to social media sources may reflect a heightened awareness of the potential unreliability of user-generated content. Such results are also consistent with China Fact Check’s monitoring and verification mainly in Chinese social networks (China Fact Check, 2023). Moreover, this reflects the fact that in the Chinese information ecosystem, social media are extremely influential in the dissemination of information.

Figure 6: Reliability Distribution of Social Media Sources on Chinese Wikipedia

In terms of dependency analysis, we observed that there is limited overlap in the sources used by both platforms. The shared sources primarily consist of internationally renowned news agencies, major Chinese news outlets, and some of the world’s largest social media platforms. For these common sources, we found general similarities in reliability assessments. For instance, both versions approach sources like Xinhua News Agency with caution, acknowledging potential bias in reporting related to Chinese government interests. This alignment suggests a shared understanding of certain sources’ limitations across linguistic boundaries.

However, significant differences also emerged in the evaluation of some shared sources. The Chinese Wikipedia community often lacks consensus on the reliability of certain sources, reflected in their “无共识” (No consensus) category. In contrast, English Wikipedia tends to provide more definitive reliability ratings, often leaning towards “Generally unreliable” for sources that Chinese Wikipedia classifies as unclear. This discrepancy points to a more developed and possibly more stringent evaluation process in English Wikipedia. It suggests that while Chinese Wikipedia doesn’t heavily depend on English Wikipedia’s source reliability assessments, it may have a less robust system for reaching consensus on complex source evaluations.

In conclusion, in our quantitative analysis, differences in the diversity and number of sources suggest that English Wikipedia may have access to a wider range of sources, whereas Chinese Wikipedia seems to be more selective or restricted in its choice of sources. Due to the existence of “无共识” (no consensus) label, the rating of reliable sources in Chinese Wikipedia is more ambiguous than in the English version. The qualitative differences in source assessment between the English and Chinese versions of Wikipedia also reflect the need for further discussions among the editorial community on Chinese Wikipedia. In addition, the differences in features found in the dependency analyses highlight the challenges of maintaining consistent standards of reliability across different language versions of Wikipedia, while also reflecting the unique information environments and editorial cultures of each language community.

Conclusion

This study has provided a preliminary examination of the reliability of Wikipedia as an information source, with a particular focus on comparing the Chinese and English versions of the platform. Our overview of the literature demonstrated that Wikipedia’s reliability varies significantly across different fields of study. While Wikipedia may lack authority in specialist areas, its overall accuracy and completeness are notable and have improved over time. The collaborative nature of Wikipedia was once seen as its main weakness, but it is now recognized as a potential strength if properly understood and exploited.

The quantitative and qualitative analyses emphasized that English Wikipedia employs a more extensive classification system, has a wider range of source types, and generally provides more definitive reliability ratings compared to Chinese Wikipedia. The Chinese version appears to take a more conservative approach in labeling sources as reliable, particularly for news media and social media sources, and seems to have a higher proportion of sources lacking consensus on their reliability. The dependency analysis indicates limited overlap in sources used by both platforms, suggesting that while Chinese Wikipedia doesn’t heavily rely on English Wikipedia’s assessments, it may have a less robust system for reaching consensus on complex source evaluations. These findings underscore the challenges of maintaining consistent reliability standards across different language versions of Wikipedia and reflect the unique information environments and editorial cultures of each language community.

While the reliability of Wikipedia across different language versions remains a complex issue, this preliminary study underscores its potential as a valuable tool for fostering global information literacy. As we navigate the challenges of misinformation and digital literacy in the 21st century, the lessons learned from comparing different language versions of Wikipedia could prove beneficial. Future research could explore how these insights might be applied to enhance cross-cultural information literacy education and to develop more robust, culturally sensitive fact-checking mechanisms for collaborative knowledge platforms. Ultimately, this work contributes to fostering a more informed, critical-thinking global citizenry capable of navigating our increasingly complex and interconnected information environment.

This contribution was reviewed by Steven J. BARELA.

Suggested Citation:

SUN, Peiyuan (2024). « Wikipedia as a Reliable Information Source: A Comparision of Chinese and English Versions ». In Blog Scientifique de l’Institut Confucius, Université de Genève. Permanent link: https://ic.unige.ch/?p=2223, accessed 12/04/2024.

References

Amina, W., & Warraich, N. F. (2022). Use and Trustworthiness of Wikipedia Information: Students’ Perceptions and Reflections. In Digital Library Perspectives, 38(1), pp. 16-32.
Arroyo-Machado, W., Torres-Salinas, D., Herrera-Viedma, E., & Romero-Frías, E. (2020). Science through Wikipedia: A Novel Representation of Open Knowledge through Co-Citation Networks. In PloS ONE, 15(2), e0228713.
Azer, S. A. (2015). Is Wikipedia A Reliable Learning Resource for Medical Students? Evaluating Respiratory Topics. In Advances in Physiology Education, 39(1), pp. 5-14.
Azer, S. A., AlSwaidan, N. M., Alshwairikh, L. A., & AlShammari, J. M. (2015). Accuracy and Readability of Cardiovascular Entries on Wikipedia: Are They Reliable Learning Resources for Medical Students?. In BMJ open, 5(10), e008187.
Baigutanova, A., Myung, J., Saez-Trumper, D., Chou, A. J., Redi, M., Jung, C., & Cha, M. (2023, April). Longitudinal Assessment of Reference Quality on Wikipedia. In Proceedings of the ACM Web Conference 2023 (pp. 2831-2839).
Bidegain, E., Egaña, T., & Zuberogoitia, A. (2024). Explaining How Wikipedia Deals with Credibility to University Students: The Case of Wikipedia in the Basque Language. In Journal of Information, Communication and Ethics in Society, 22(1), pp. 159-173.
China Fact-Check. (2023). Fact-Check Manual. Retrieved from https://chinafactcheck.com/wp-content/themes/youju/assets/fact-check-manua-PC.pdf
Clauson, K. A., Polen, H. H., Boulos, M. N. K., & Dzenowagis, J. H. (2008). Scope, Completeness, and Accuracy of Drug Information in Wikipedia. In Annals of Pharmacotherapy, 42(12), pp. 1814-1821.
Fang, K., & Wei, X. (2021, September 30). Checking in on Fact-Checking in China. In China Media Project. Retrieved from https://chinamediaproject.org/2021/09/30/checking-in-on-fact-checking-in-china/
Greenstein, S., & Zhu, F. (2018). Do Experts or Crowd-Based Models Produce More Bias? Evidence from Encyclopedia Britannica and Wikipedia. In MIS Quarterly, 42(3), pp. 945-960.
Huang, J., Shi, S., Chen, Y., & Chow, W. S. (2016). How Do Students Trust Wikipedia? An Examination across Genders. In Information Technology & People, 29(4), pp. 750-773.
Jones, H. (2018). “Wikipedia”, Translation, and the Collaborative Production of Spatial Knowledge. In Alif: Journal of Comparative Poetics, 38, pp. 264–297. http://www.jstor.org/stable/26496377
Lei, Y. W. (2016). Freeing the Press: How Field Environment Explains Critical News Reporting in China. In American Journal of Sociology, 122(1), pp. 1-48.
Lichtenstein, S., & Parker, C. M. (2009). Wikipedia Model for Collective Intelligence: A Review of Information Quality. In International Journal of Knowledge and Learning, 5(3-4), pp. 254-272.
Malik, A., Rafiq, M., & Mahmood, K. (2023). Wikipedia and Academia: University Faculty Patterns of Use and Perceptions of Credibility. In Journal of Librarianship and Information Science. https://doi.org/10.1177/09610006231190652
Menchen-Trevino, E., & Hargittai, E. (2011). Young Adults’ Credibility Assessment of Wikipedia. In Information, Communication & Society, 14(1), pp. 24-51.
Mostafa, M. M. (2023). Twenty Years of Wikipedia in Scholarly Publications: A Bibliometric Network Analysis of the Thematic and Citation Landscape. In Quality & Quantity, 57, pp. 5623–5653. https://doi.org/10.1007/s11135-023-01626-7
Okoli, C., Mehdi, M., Mesgari, M., Nielsen, F. Å., & Lanamäki, A. (2012). The People’s Encyclopedia Under the Gaze of the Sages: A Systematic Review of Scholarly Research on Wikipedia. Available at SSRN 2021326.
Peoples, L. F. (2009). The Citation of Wikipedia in Judicial Opinions. In Yale Journal of Law & Technology, 12(1), pp. 1-51.
Reinsalu, R., Vija, M., Org, A., Siiman, A., & Remmik, M. (2023). With or Without Wikipedia? Integrating Wikipedia into the Teaching Process in Estonian General Education Schools. In Education Sciences, 13(6), 583, pp. 1-17.
Wikipedia. In Wikipedia, The Free Encyclopedia. Retrieved from https://en.wikipedia.org/wiki/Wikipedia
Yang, P., & Colavizza, G. (2024). Polarization and Reliability of News Sources in Wikipedia. In Online Information Review. https://doi.org/10.1108/OIR-02-2023-0084
Zeng, W. (2020). Reframing News by Different Agencies: A Case Study of Translations of News on the US-China Trade Dispute. In Babel, 66(4-5), pp. 847-866.