Assessing factuality in Spanish: do experts and non-experts agree?
DOI:
https://doi.org/10.58859/resla.1545Keywords:
corpus, factuality markers, polarity, news, SpanishAbstract
Event veridicality and factuality are key to natural language understanding. Deciding whether the events mentioned in a text are viewed as happening or not is one of the key features in fake news detection. That is one of the reasons why factuality has been widely studied and enormous efforts have been made to annotate corpora and develop tools that automatically identify factuality and veridicality. Most of the studies carried out use English corpora annotated by non-experts; however, little has been said about factuality in Spanish and on how appropriate the use of non-expert annotators is.
Based on the discussion above, this study aims to a) shed some light on how non-expert Spanish speakers interpret the various factuality markers, b) determine if the age of the informants is a relevant factor when assessing the degree of factuality, and c) explore whether the experts and the non-expert’s interpretations differ.
Global Statistics ℹ️
|
22
Views
|
4
Downloads
|
|
26
Total
|
|
References
Alonso, L., Castellón, I., Curell, H., Fernández-Montraveta, A., Oliver, S., Vázquez, G. (2018). Proyecto TAGFACT: Del texto al conocimiento. Factualidad y grados de certeza en español, Procesamiento del Lenguaje Natural, 61, p. 151-154. http://dx.doi.org/10.26342/2018-61-20
Barrios, L. (2022). La factualidad en las oraciones adversativas, concesivas y condicionales en español: El papel de los tiempos verbales en la anotación automática de corpus. [Doctoral Dissertation, Universitat de Lleida]. https://www.tdx.cat/handle/10803/673898#page=1
Box, G.E., Hunter, J.S., & Hunter, W.G. (2008). Estadística para Investigadores. Editorial Reverté.
Colomer, J.P., Keyu , L., & Rambow. O. (2016). Detecting level of belief in Chinese and Spanish. In Proceedings of the Workshop on Extra-Propositional Aspects of Meaning in Computational Linguistics (ExProM), 22-30. https://aclanthology.org/W16-5003.pdf
de Marneffe, M. C., Manning, C. D., & Potts, C. (2012). Did It Happen? The Pragmatic Complexity of Veridicality Assessment. Computational Linguistics, 38(2), 301–333. DOI: 10.1162/COLI_a_00097 DOI: https://doi.org/10.1162/COLI_a_00097
Fernández-Montraveta, A., Vàzquez, G., Curell, H., & Castellón, I. (2020a). The TAGFACT annotator and editor: A versatile tool. Research in Corpus Linguistics 8, 131-146. http://dx.doi.org/10.32714/ricl.08.01.08 DOI: https://doi.org/10.32714/ricl.08.01.08
Lee, K., Artzi, Y., Choi, Y., & Zettlemoyer, L. (2015). Event Detection and Factuality Assessment with Non-Expert Supervision. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 1643–1648. DOI: 10.18653/v1/D15-1189 DOI: https://doi.org/10.18653/v1/D15-1189
Martín, F. J. & Ruiz-Maya, L. (2001). Estadística II: Inferencia. Libros Aula Magna. Alfa Centauro.
Miller, B. (2020). Reading Certainty: Evidence from a Large Study on NLP and Witness Testimony. Digital Humanities 2020 Conference. July 20-24, 2020.
Monge, J.F., & Juan, A.A. (2002). Estadística no paramétrica: Prueba Chi-cuadrado. Proyecto e-Math. ET-M05. Universitat Oberta de Catalunya.
Peña, D. (1988). Estadística, Modelos y Métodos. Editorial Alianza Universidad.
Prabhakaran, V., By, T., Hirschberg, J., Rambow, O., Shaikh, S., Strzalkowski, T., Tracey, J., Arrigo, M., Basu, R., Clark, M., Dalton, A., Diab, M., Guthrie, L., Prokofieva, A., Strassel, S., Werner, G., Wiebe, J., & Wilks, Y. (2015). A New Dataset and Evaluation for Belief/Factuality. In Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics (*SEM 2015), 82-91. DOI: 10.18653/v1/S15-1009 DOI: https://doi.org/10.18653/v1/S15-1009
Rosá, A., Alonso, L., Castellón, I., Chiruzzo, L., Curell, H., Fernández-Montraveta, A., Góngora, S., Malcouri, M., Vázquez, G., & Wonsever, D. (2020b). Overview of FACT at IBERLEF 2020: Events detection and classification. In CEUR Workshop Proceedings, 2664, 197-205. https://ceur-ws.org/Vol-2664/fact_overview.pdf
Rudinger, R., White, A.S., & Van Durme, B. (2018). Neural Models of Factuality. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), 731–744. DOI: 10.18653/v1/N18-1067 DOI: https://doi.org/10.18653/v1/N18-1067
Saurí, R. (2008). A Factuality Profiler for Eventualities in Text. [Doctoral Dissertation, Brandeis University] https://www.cs.brandeis.edu/~roser/pubs/sauriDiss_1.5.pdf
Saurí, R., & Pustejovsky, J. (2009). FactBank: a corpus annotated with event factuality. Language Resources and Evaluation 43, 227–268. https://doi.org/10.1007/s10579-009-9089-9 DOI: https://doi.org/10.1007/s10579-009-9089-9
Saurí, R., & Pustejovsky, J. (2012). Are You Sure That This Happened? Assessing the Factuality Degree of Events in Text. Computational Linguistics, 38(2), 261–299. DOI: 10.1162/COLI_a_00096 DOI: https://doi.org/10.1162/COLI_a_00096
Soni, S., Mitra, T., Gilbert, E., & Eisenstein, J. (2014). Modeling Factuality Judgments in Social Media Text. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 415–420. DOI: 10.3115/v1/P14-2068 DOI: https://doi.org/10.3115/v1/P14-2068
Stanovsky, G., Eckle-Kohler, J., Puzikov, Y., Dagan, I., & Gurevych, I. (2017). Integrating Deep Linguistic Features in Factuality Prediction over Unified Datasets. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 352–357. DOI: 10.18653/v1/P17-2056 DOI: https://doi.org/10.18653/v1/P17-2056
Tonelli, S., Sprugnoli, R., Speranza, M. & Minard, A. (2014). NewsReader Guidelines for Annotation at Document Level. Extension of Deliverable D3. Technical Report NWR- 2014-2. Trento.
Tracey, J., Arrigo, M., & Strassel, S. (2019). DEFT Spanish Commited Belief Annotation. Linguistic Data Consortium LDG2019T09. Web download. Philadelphia: Linguistic Data Consortium. https://catalog.ldc.upenn.edu/LDC2019T09
Tracey, J., Rambow, O., Cardie, C., Dalton, A., Trang Dang, H., Diab, M., Dorr, B., Guthrie, L., Markowska, M., Muresan, S., Prabhakaran, V., Shaikh, S.,
& Strzalkowski, T. (2022). BeSt: The Belief and Sentiment Corpus. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, 2460–2467. https://aclanthology.org/2022.lrec-1.262 DOI: https://doi.org/10.63317/4run4u9jq8mx
Uzzaman, N., Llorens, H., Derczynski, L., Verhagen, M., Allen, J., & Pustejovsky, J. (2013). Semeval-2013 task 1: Tempeval-3: Evaluating time expressions, events, and temporal relations. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), 1–9.
van Son, C., van Erp, M., Fokkens, A., and Vossen, P. (2014). Hope and Fear: Interpreting Perspectives by Integrating Sentiment and Event Factuality. In Proceedings of the 9th Language Resources and Evaluation Conference, 3857-3864. DOI: https://doi.org/10.63317/2voodkchtwdx
White, A. S., Reisinger, D., Sakaguchi, K., Vieira, T., Zhang, S., Rudinger,R., Rawlins, K., & Van Durme, J. (2016). Universal decompositional semantics on universal dependencies. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 1713–1723. DOI: https://doi.org/10.18653/v1/D16-1177
Wonsever, D., Aiala, R. & Malcuori, M. (2016). Factuality Annotation and Learning in Spanish Texts. In Proceedings of the 10th Edition of the Language Resources and Evaluation Conference, 2076-2080. DOI: https://doi.org/10.63317/3zbqbkmidsff
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Elisabet Comelles

This work is licensed under a Creative Commons Attribution 4.0 International License.
Public licensing terms for published work
All articles published in {$contextName} are distributed under a Creative Commons Attribution 4.0 International (CC BY 4.0) License, unless otherwise stated.
This means that authors retain copyright and that anyone is free to:
- Share – copy and redistribute the material in any medium or format.
- Adapt – remix, transform, and build upon the material for any purpose, even commercially.
Provided that the following conditions are met:
- Attribution – Proper credit must be given to the authors and the journal, a link to the license must be provided, and any changes must be indicated.
- No additional restrictions – No legal terms or technological measures may be applied that restrict others from using the work.
For more information, please refer to the full license text at: https://creativecommons.org/licenses/by/4.0/








