The use of generative artificial intelligence in historical writing exposes a distinction between citing sources and justifying historical claims. This article examines the conditions under which support judgments used in language model evaluation can inform historical source criticism. It combines a comparative conceptual analysis of ten purposively selected scholarly and technical publications with a bounded examination of the historical text of Turkey’s Law on the Unification of Education. The publications are compared in terms of their objects of evaluation, evidential grounds, inferential scope, and unresolved questions. The analysis shows that identifying sources, tracing production processes, establishing support for propositions, and justifying historical explanations are distinct operations. It develops algorithmic source criticism as a framework connecting source provenance, textual transformation, and claim justification. The framework separates the mode of a claim’s relationship to its source from its evidential status and records the conditions for revising an assessment. The documentary examination demonstrates that a legal text can directly support claims about its provisions while requiring additional evidence for implementation, public reception, and causal effects. The example does not measure model performance. The article contributes a method for examining the relationship between the scope of a historical claim and its available grounds without reducing credibility to a single accuracy score. It offers implications for historical scholarship, education, and editorial assessment while preserving interpretive plurality and specifying responsibility for correction.
Bender, E. M., & Koller, A. (2020). Climbing towards NLU: On meaning, form, and understanding in the age of data. In D. Jurafsky, J. Chai, N. Schluter, & J. Tetreault (Eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 5185–5198). Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.463
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957
Elangovan, A., Liu, L., Xu, L., Bodapati, S. B., & Roth, D. (2024). ConSiDERS-The-Human evaluation framework: Rethinking human evaluation for generative large language models. In L.-W. Ku, A. Martins, & V. Srikumar (Eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 1137–1160). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.acl-long.63
Fickers, A. (2020). Update für die Hermeneutik: Geschichtswissenschaft auf dem Weg zur digitalen Forensik? [An update for hermeneutics: Historical scholarship on the way to digital forensics?] Zeithistorische Forschungen/Studies in Contemporary History, 17(1), 157–168. https://doi.org/10.14765/zzf.dok-1765
Gao, T., Yen, H., Yu, J., & Chen, D. (2023). Enabling large language models to generate text with citations. In H. Bouamor, J. Pino, & K. Bali (Eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 6465–6488). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.emnlp-main.398
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), Article 248. https://doi.org/10.1145/3571730
Judd, C. M., Westfall, J., & Kenny, D. A. (2012). Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem. Journal of Personality and Social Psychology, 103(1), 54–69. https://doi.org/10.1037/a0028347
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. https://doi.org/10.1111/jedm.12000
Lebo, T., Sahoo, S., & McGuinness, D. (Eds.). (2013). PROV-O: The PROV ontology. World Wide Web Consortium. https://www.w3.org/TR/prov-o/
Lévesque, S. (2008). Thinking historically: Educating students for the twenty-first century. University of Toronto Press.
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, & H. Lin (Eds.), Advances in Neural Information Processing Systems 33 (pp. 9459–9474). Curran Associates. https://papers.nips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html
Milligan, I. (2019). History in the age of abundance? How the web is transforming historical research. McGill-Queen’s University Press.
Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-T., Koh, P. W., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023). FActScore: Fine-grained atomic evaluation of factual precision in long form text generation. In H. Bouamor, J. Pino, & K. Bali (Eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 12076–12100). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.emnlp-main.741
Moreau, L., & Missier, P. (Eds.). (2013). PROV-DM: The PROV data model. World Wide Web Consortium. https://www.w3.org/TR/prov-dm/
Putnam, L. (2016). The transnational and the text-searchable: Digitized sources and the shadows they cast. The American Historical Review, 121(2), 377–402. https://doi.org/10.1093/ahr/121.2.377
Reber, R., & Schwarz, N. (1999). Effects of perceptual fluency on judgments of truth. Consciousness and Cognition, 8(3), 338–342. https://doi.org/10.1006/ccog.1999.0386
Trouillot, M.-R. (1995). Silencing the past: Power and the production of history. Beacon Press.
van Drie, J., & van Boxtel, C. (2008). Historical reasoning: Towards a framework for analyzing students’ reasoning about the past. Educational Psychology Review, 20(2), 87–110. https://doi.org/10.1007/s10648-007-9056-1
Walters, W. H., & Wilder, E. I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, 13(1), Article 14045. https://doi.org/10.1038/s41598-023-41032-5
Westfall, J., Kenny, D. A., & Judd, C. M. (2014). Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. Journal of Experimental Psychology: General, 143(5), 2020–2045. https://doi.org/10.1037/xge0000014
Wineburg, S. (2001). Historical thinking and other unnatural acts: Charting the future of teaching the past. Temple University Press.
Zhao, J. X., Liu, J. Z. J., Hooi, B., & Ng, S.-K. (2025). How does response length affect long-form factuality. In W. Che, J. Nabende, E. Shutova, & M. T. Pilehvar (Eds.), Findings of the Association for Computational Linguistics: ACL 2025 (pp. 3102–3125). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.findings-acl.161
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., & Stoica, I. (2023). Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, & S. Levine (Eds.), Advances in Neural Information Processing Systems 36 (pp. 46595–46623). Curran Associates. https://proceedings.neurips.cc/paper_files/paper/2023/hash/91f18a1287b398d378ef22505bf41832-Abstract.html
Select and copy the citation format of your choice to cite this work in your academic publications.
Işıldak, R. A. D. S. (2026). The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence. AI-Driven Journal of Interdisciplinary Social Research, 1(2), 101-125. https://doi.org/10.5281/zenodo.24174687
Işıldak, R. A. D. S. (2026). The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence. AI-Driven Journal of Interdisciplinary Social Research, 1(2), 101-125. https://doi.org/10.5281/zenodo.24174687
Işıldak, R. A. D. S. (2026). The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence. AI-Driven Journal of Interdisciplinary Social Research, 1(2), 101-125. doi:10.5281/zenodo.24174687
Işıldak, R. A. D. S. (2026). The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence. AI-Driven Journal of Interdisciplinary Social Research, 1(2), 101-125. doi:10.5281/zenodo.24174687
Işıldak, Res. Assist. Dr. Sinan. "The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence." AI-Driven Journal of Interdisciplinary Social Research, vol. 1, no. 2, 2026, pp. 101-125, https://doi.org/10.5281/zenodo.24174687.
Işıldak, Res. Assist. Dr. Sinan. "The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence." AI-Driven Journal of Interdisciplinary Social Research, vol. 1, no. 2, 2026, pp. 101-125, https://doi.org/10.5281/zenodo.24174687.
Işıldak, Res. Assist. Dr. Sinan. 2026. "The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence." AI-Driven Journal of Interdisciplinary Social Research 1, no. 2: 101-125. https://doi.org/10.5281/zenodo.24174687.
Işıldak, Res. Assist. Dr. Sinan. 2026. "The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence." AI-Driven Journal of Interdisciplinary Social Research 1, no. 2: 101-125. https://doi.org/10.5281/zenodo.24174687.
Işıldak, R. A. D. S. (2026) 'The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence', AI-Driven Journal of Interdisciplinary Social Research, 1(2), pp. 101-125. Available at: https://doi.org/10.5281/zenodo.24174687.
Işıldak, R. A. D. S. (2026) 'The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence', AI-Driven Journal of Interdisciplinary Social Research, 1(2), pp. 101-125. Available at: https://doi.org/10.5281/zenodo.24174687.
R. A. D. S. Işıldak, "The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence," AI-Driven J. Interdiscip. Soc. Res., vol. 1, no. 2, pp. 101-125, 2026, doi: 10.5281/zenodo.24174687.
R. A. D. S. Işıldak, "The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence," AI-Driven J. Interdiscip. Soc. Res., vol. 1, no. 2, pp. 101-125, 2026, doi: 10.5281/zenodo.24174687.
Işıldak RADS. The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence. AI-Driven J. Interdiscip. Soc. Res.. 2026;1(2):101-125.
Işıldak RADS. The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence. AI-Driven J. Interdiscip. Soc. Res.. 2026;1(2):101-125.
@article{ildak2026aijisr,
title = {{The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence}},
author = {{Işıldak, Res. Assist. Dr. Sinan}},
journal = {{AI-Driven Journal of Interdisciplinary Social Research}},
volume = {1},
number = {2},
pages = {101-125},
year = {2026},
doi = {10.5281/zenodo.24174687},
url = {https://aijisr.org/articles/AI-JISR-2026-0022}
}
TY - JOUR
TI - The Algorithmic Limits of Historical Evidence and Responsibility for Justification in Generative Artificial Intelligence
AU - Işıldak, Res. Assist. Dr. Sinan
JO - AI-Driven Journal of Interdisciplinary Social Research
VL - 1
IS - 2
SP - 101
EP - 125
PY - 2026
DO - 10.5281/zenodo.24174687
UR - https://aijisr.org/articles/AI-JISR-2026-0022
ER -