Hostname: page-component-cd9895bd7-gxg78 Total loading time: 0 Render date: 2024-12-27T14:07:27.287Z Has data issue: false hasContentIssue false
  • English
  • Français

Quantifying Quality: A Computational Approach to Literary Value in North Korea

Published online by Cambridge University Press:  24 February 2022

Benoit Berthelier Open the ORCID record for Benoit Berthelier [Opens in a new window]
Benoit Berthelier*
Affiliation:
Faculty of Arts and Social Sciences, University of Sydney, Australia
*
Email: [email protected]
Get access

Abstract

This article analyzes questions of literary production and literary value in the Democratic People's Republic of Korea (DPRK) using quantitative methods and data sets covering a significant part of the country's literary and intellectual output over forty years (1977–2016). The first part of the article adopts a thematic approach, using topic modeling and time series, to map the DPRK's literary output and investigate the influence of genre on literary consecration. The second part looks at questions of style and literary quality in the DPRK through a comparative analysis of multiple novels grouped by degree of literary prestige. It introduces a method to evaluate the originality of an author's writing style using a masked language model (BERT) and uses it to assess the role of conformity and originality in the DPRK's literary field. Even though writers consecrated by the state display a high level of stylistic conformity when writing mandated biographies of the ruling family, their independent output ranks higher in originality than nonconsecrated writers.

Keywords

authoritarian fictionBERTdigital humanitiesKorean literatureliterary originalityliterary valuelanguage modelNorth Koreatopic modeling
Type
Research Article
Information
The Journal of Asian Studies , Volume 81 , Issue 2 , May 2022 , pp. 267 - 288
Check for updates
Copyright
Copyright © The Association for Asian Studies, Inc. 2022

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

List of References

An Mak. 1946. “Sin chŏngse wa minjujuŭi munhwa yesul chŏnsŏn kanghwa ŭi immu” 신정세와 민주주의문화예술전선 강화의 의무 [The new conjuncture and the duty of reinforcing the democratic fronts of the arts and letters]. Munhwa Chŏnsŏn 2:2–12.Google Scholar
Antoniak, Maria, Walsh, Melanie, and Mimno, David. 2021. “Tags, Borders, and Catalogs: Social Re-Working of Genre on LibraryThing.” Proceedings of the ACM on Human-Computer Interaction 5 (CSCW1): Article 29.CrossRefGoogle Scholar
Barthes, Roland. 1964. “Éléments de sémiologie” [Elements of semiology]. Communications 4:91–135.CrossRefGoogle Scholar
Beecroft, Alexander. 2014. “World Literature without a Hyphen.” In World Literature in Theory, edited by Damrosch, David, 180–90. Oxford: Wiley Blackwell.Google Scholar
Emily M., Bender, Gebru, Timnit, McMillan-Major, Angelina, and Shmitchell, Shmargaret. 2021. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21), 610–23. New York: Association for Computing Machinery.Google Scholar
Blei, David. 2012. “Topic Modeling and Digital Humanities.” Journal of Digital Humanities 2 (1). http://journalofdigitalhumanities.org/2-1/topic-modeling-and-digital-humanities-by-david-m-blei/ (accessed September 14, 2020).Google Scholar
Blodgett, Su Lin, Barocas, Solon, Daumé, Hal III, and Wallach, Hanna. 2020. “Language (Technology) Is Power: A Critical Survey of ‘Bias’ in NLP.” https://arxiv.org/abs/2005.14050 (accessed December 12, 2020).Google Scholar
Bourdieu, Pierre. 1979. La Distinction [Distinction]. Paris: Minuit.Google Scholar
Bourdieu, Pierre. 1991. “Le champ littéraire” [The literary field]. Actes de la Recherche en Sciences Sociales [Proceedings of Research in the Social Sciences] 89:3–46.CrossRefGoogle Scholar
Bourdieu, Pierre. 1992. Les Règles de l'art [The rules of art]. Paris: Seuil.Google Scholar
Bradley, Efron, and Tibshirani, R. J.. 1994. An Introduction to the Bootstrap. New York: Chapman and Hall/CRC.Google Scholar
Broadwell, Peter, Chen, Jack, and Shepard, David. 2019. “Reading the Quan Tang shi: Literary History, Topic Modeling, Divergence Measures.” Digital Humanities Quarterly 13 (4). http://www.digitalhumanities.org/dhq/vol/13/4/000434/000434.html (accessed September 1, 2020).Google Scholar
Buchholz, Larissa. 2017. “What Is a Global Field? Theorizing Fields beyond the Nation-State.” Sociological Review 64 (2): 31–60.CrossRefGoogle Scholar
Carnap, Rudolf. 1964. The Logical Syntax of Language. London: Routledge and Kegan Paul.Google Scholar
Casanova, Pascale. 1999. La République mondiale des lettres [The world republic of letters]. Paris: Seuil.Google Scholar
Casanova, Pascale. 2005. “Literature as a World.” New Left Review 31:71–90.Google Scholar
Cho, Heekyoung. 2018. “Rethinking World Literature through the Relations between Russian and East Asian Literatures.” Cross-Currents: East Asian History and Culture Review 28:7–26. https://cross-currents.berkeley.edu/e-journal/issue-28/cho (accessed December 11, 2020).Google Scholar
Chae-bong, Choi. 2005. “Kyŏngk'wae han munch'e ro namnyŏ sarang kŭmgi kkaeda” 경쾌한 문체로 남녀사랑 ‘금기’ 깨다 [Breaking the taboo of girl-boy love with a light-hearted style]. Hankyoreh, August 4.Google Scholar
Choi, Yearn Hong. 1994. “North Korean literature in the Chosun Munhak.” Global Economic Review 23 (1): 75–98.Google Scholar
Clark, Katerina. 1981. The Soviet Novel: History as Ritual. Bloomington: Indiana University Press.Google Scholar
Clark, Kevin, Khandelwal, Urvashi, Levy, Omer, and Manning, Christopher D.. 2019. “What Does BERT Look At? An Analysis of BERT's Attention.” BlackBoxNLP@ACL. https://arxiv.org/abs/1906.04341 (accessed November 1, 2020).CrossRefGoogle Scholar
Dällenbach, Lucien, and Tomarken, Annette. 1980. “Reflexivity and Reading.” New Literary History 11 (3): 435–49.CrossRefGoogle Scholar
David-West, Alzo. 2007. “An Inquiry of Intentions in Kim Hye-yŏng's First Meeting: A North Korean Short Story in Korea Today.” Transnational Literature 6 (1). http://dspace.flinders.edu.au/jspui/bitstream/2328/27124/1/Inquiry_of_Intentions.pdf (accessed January 14, 2022).Google Scholar
De Ceuster, Koen. 2011. “To be an Artist in North Korea: Talent and Then Some More.” In Exploring North Korean Arts, edited by Frank, Rüdiger, 51–71. Nürnberg: Verlag für moderne Kunst.Google Scholar
Devlin, Jacob, Chang, Ming-Wai, Lee, Kenton, and Toutanova, Kristina. 2019. “BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding.” In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1, edited by Jill Burstein, Christy Doran, and Thamar Solorio, 4171–86. Minneapolis, MN: NAACL.Google Scholar
Djagalov, Rossen. 2018. “Literary Monopolists and the Forging of the Post-WWII People's Republic of Letters.” In Socialist Realism in Central and Eastern European Literatures: Institutions, Dynamics, Discourses, edited by Skradol, Natalia and Dobrenko, Evgeny, 25–37. London: Anthem Press.CrossRefGoogle Scholar
Epstein, Stephen. 2002. “North Korean Short Stories on the Cusp of the New Millennium.” Acta Koreana 5 (1): 33–55.Google Scholar
Fricke, Harald. 2008. “How Scientific Can Literary Evaluation Be?” In The Quality of Literature: Linguistic Studies in Literary Evaluation, edited by Peer, Willie van, 191–207. Amsterdam: John Benjamins Publishing Company.CrossRefGoogle Scholar
Friedman, Susan Stanford. 2013. “World Modernism, World Literature, and Comparaticity.” In The Oxford Handbook of Global Modernisms, edited by Wollaeger, Mark and Eatough, Matt, 499–525. Oxford: Oxford University Press.Google Scholar
Gardner, John. 1983. The Art of Fiction. New York: Random House.Google Scholar
Goldstone, Andrew. 2013. Fictions of Autonomy: Modernism from Wilde to de Man. New York: Oxford University Press.CrossRefGoogle Scholar
Goldstone, Andrew. 2015. “Distant Reading: More Work to Be Done.” August 8 (blog). https://andrewgoldstone.com/blog/2015/08/08/distant/ (accessed December 6, 2020).Google Scholar
Günther, Hans, ed. 1990. The Culture of the Stalin Period. New York: Palgrave Macmillan.CrossRefGoogle Scholar
Jawahar, Ganesh, Sagot, Benoît, and Seddah, Djamé. 2019. “What Does BERT Learn about the Structure of Language?” In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, edited by Preslav Nakov and Alexis Palmer, 3651–57. Stroudsburg, PA: Association for Computational Linguistics.CrossRefGoogle Scholar
Herrnstein Smith, Barbara. 1983. “Contingencies of Value.” Critical Inquiry 10 (1): 1–35.CrossRefGoogle Scholar
Jakobson, Roman. 1960. “Linguistics and Poetics.” In Style in Language, edited by Sebeok, Thomas, 351–77. Cambridge, Mass.: MIT Press.Google Scholar
Jockers, Matthew, and Mimno, David. 2013. “Significant Themes in 19th-Century Literature.” Poetics 41:750–69.CrossRefGoogle Scholar
Jurafsky, Dan, and Martin, James H.. 2019. “Speech and Language Processing.” https://web.stanford.edu/~jurafsky/slp3/ (accessed September 21, 2020).Google Scholar
Chinho, Kang and Yu, Imha. 2009. Ch'ongsŏ ‘pulmyŏl ŭi yŏksa’ yŏn'gu 총서 ‘불멸의 력사’ 연구 [Research on the collection “Immortal History”]. Seoul: Somyŏng Ch'ulp’ansa.Google Scholar
Sangho, Kang. 1993. “Namkigo sip’ŭn iyagi tŭl—naega ch'irŭn pukhan sukch’ŏng” 남기고 싶은 이야기—내가 치른 북한 숙청 [Stories I want to leave behind—The purge I underwent in North Korea]. Chungang Ilbo, January 11–October 12.Google Scholar
Yongt'aek, Kang. 2003. Kim Sowŏl kwa Cho Kich’ŏn ŭi siŏ sayong yangsang pigyo yŏn'gu 김소월과 조기천의 시어 사용 양상 비교 연구 [Comparative study of the use of poetic words by Kim Sowŏl and Cho Kich’ŏn]. Seoul: Yŏngnak.Google Scholar
Kim, Chaeung. 2012. “Pukhan ŭi kyegŭp chŏngch'aek kwa kyegŭp wigye kujo ŭi hyŏngsŏng (1945–1950)” 북한의 계급정책과 계급위계구조의 형성(1945–1950) [Class policies and formation of hierarchical class structure in North Korea (1945–1950)]. Yŏksawa hyŏnsil 85:393–433.Google Scholar
Kim, Chaeyong. 1994. Pukhan munhak ŭi yŏksajŏk ihae 북한 문학의 역사적 이해 [A historical understanding of North Korean literature]. Seoul: Munhak kwa chisŏngsa.Google Scholar
Kim, Chinhwan. 2016. “Pukhan sahoe kyegŭp yŏn'gu” 북한사회 계급 연구 [Research on classes in North Korean society]. In 2016-nyŏn t'ongil pukhan yŏn'gu ch'oejong pogosŏ 2016 년 통일⋅북한 연구 최종보고서 [Final research report on North Korea and Unification], 235–77. Seoul: National Institute for Unification Education.Google ScholarPubMed
Kim, Immanuel. 2012. “North Korean Literature: Margins of Writing Memory, Gender, and Sexuality.” PhD diss., University of California, Riverside.Google Scholar
Kim, Immanuel. 2016. “Problems with Institutionalizing the April 15 Literary Production Unit.” Korea Journal 56 (1): 140–64.Google Scholar
Kim, Jong Il. [1967a] 1987. “Munhak yesul pumun esŏ dang ŭi yuil sasang ch'egye rŭl t’ŭnt’ŭnhi seulde taehayŏ” 문학예술부문에서 당의 유일사상체계를 튼튼히 세울데 대하여 [On strongly building the party's monolithic ideology in the domain of arts and letters]. In Chuch'e hyŏngmyŏng wiŏp ŭi wansŏng ŭl wihayŏ 주체 혁명 위업 의 완성 을 위하여 [For the completion of the great task of the Juch'e revolution], 38–52. Pyongyang: Chosŏn rodongdang ch'ulp’ansa.Google Scholar
Kim, Jong Il. [1967b] 1994. “4.15 munhak ch'angjaktan ŭl naeolte taehayŏ” 4.15 문학창작단을 내올데 대하여 [On unveiling the 4.15 literary production unit]. In Kim Jong Il sŏnjip 김정일선집 [Selected works of Kim Jong Il], vol. 1, 242–84. Pyongyang: Chosŏn rodongdang ch'ulp’ansa.Google Scholar
Kim, Jong Il. 1992a. Chuch'e munhangnon 주체문학론 [Theory of Juche literature]. Pyongyang: Chosŏn rodongdang ch'ulp’ansa.Google Scholar
Kim, Jong Il. 1992b. On Juche Literature. Pyongyang: Foreign Languages Publishing House.Google Scholar
Kim, Jong Il. 2007. The Songun-Based Revolutionary Line Is a Great Revolutionary Line of Our Era and an Ever Victorious Banner of Our Revolution. Pyongyang: Foreign Language Publishing House.Google Scholar
Lankov, Andrei. 2002. “Kim Takes Control: The ‘Great Purge’ in North Korea, 1956–1960.” Korean Studies 26 (1): 87–119.CrossRefGoogle Scholar
Lankov, Andrei. 2014. The Real North Korea: Life and Politics in the Failed Stalinist Utopia. Oxford: Oxford University Press.Google Scholar
Latynina, Alla. 1996. “The Stalin Prizes in Literature as the Quintessence of Socialist Realism” In In the Party Spirit: Socialist Realism and Literary Practice in the Soviet Union, East Germany and China, edited by Chung, Hillary, 106–28. Amsterdam: Rodopi.Google Scholar
Ch’ŏnggu, Lee. 1950. “Ssobet’ŭ si munhak kwa uri siin tŭl” 쏘베트 시문학과 우리 시인들 [Soviet poetry and our poets]. Munhak yesul 2:55–66.Google Scholar
Kwangsu, Lee. 1916. “Munhak iran hao” 文學이란 何오 [What is literature?]. Maeil Shinbo, November 10–23.Google Scholar
Sangsuk, Lee. 2013. “Kim Jong Un sidae ŭi ch'ulbal kwa pukhan si ŭi ch'ui” 김정은 시대의 출발과 북한시의 추이 [Study on North Korean poetry in the era of Kim Jeongun]. Han'guk Sihak Yŏn'gu 38:181–212.Google Scholar
Louwerse, Max, Benesh, Nick, and Zhang, Bin. 2008. “Computationally Discriminating Literary from Non-Literary Texts.” In Directions in Empirical Literary Studies, edited by Zyngier, S. et al. , 175–91. Amsterdam: John Benjamins Publishing Company.CrossRefGoogle Scholar
Lumley, Thomas, Diehr, Paula, Emerson, Scott, and Chen, Lu. 2002. “The Importance of the Normality Assumption in Large Public Health Data Sets.” Annual Review of Public Health 23:151–69.CrossRefGoogle ScholarPubMed
Merriman, Ben. 2015. “A Science of Literature.” Boston Review, June 23. https://bostonreview.net/articles/ben-merriman-moretti-jockers-digital-humanities/ (accessed January 14, 2022).Google Scholar
Milner, Andrew. 2005. Literature, Culture and Society. New York: Routledge.Google Scholar
Pyŏnggyun, Min. [1949] 1993. “Ssobet’ŭ si munhak ŭl sŏpch'wi ham e isŏsŏ kyŏngŏm kwa kyohun” 쏘배트 시문학을 섭취함에 있어서 경험과 교훈 [Experiences and lessons in the assimilation of Soviet poetry]. Rodong Sinmun, April 29. In Hyŏndae Munhak pip'yŏng charyojip. Ibukp'yŏn 현대문학 비평 자료집 [Anthology of contemporary literary criticism: Northern volume], edited by Chaeyong, Kim et al. , 383–91. Seoul: Taehaksa.Google Scholar
Moretti, Franco. 2013. Distant Reading. London: Verso.Google Scholar
Moretti, Franco. 2015. “A Response.” PMLA 132 (3): 686–89.Google Scholar
Mun, Yŏngho. 1993. Chosŏnŏ pindo susajŏn 조선어빈도수사전 [Dictionary of frequencies in Korean]. Pyongyang: Kwahak paekkwa sajŏn chonghap ch'ulp’ansa.Google Scholar
Myers, Brian. 1994. Han Sŏrya and North Korean Literature. Ithaca, N.Y.: Cornell University Press.Google Scholar
Neumann, Birgit, and Rippl, Gabriele. 2017. “Anglophone World Literatures: Introduction.” Anglia 135 (1): 1–20.CrossRefGoogle Scholar
Sŏnggŭn, Paek. 2017. “Hyŏngmyŏng sosŏl ch'angjak ŭi midŭmjik'an kiji” [The trustworthy base of the creation of revolutionary novels]. Rodong Sinmun, June 18.Google Scholar
Sangsŏ, Pan. 2013. “Hŭimang ŭi ŏndŏk” 희망의 언덕 [Hills of hope]. Ch’ ŏ ngnyŏn Munhak 5 (8): 53–62.Google Scholar
Namsu, Park. 1999. Chŏkch'i 6yŏn ŭi pukhan mundan 적치 6 년의 북한문단 [Red Flag: Six years of the North Korean literary world]. Seoul: Pogosa.Google Scholar
Park, Sunyoung. 2015. The Proletarian Wave: Literature and Leftist Culture in Colonial Korea, 1910–1945. Cambridge, Mass.: Harvard University Press.Google Scholar
Pearl, Judea, and Mackenzie, Dana. 2018. The Book of Why. New York: Basic Books.Google Scholar
Person, James F. 2013. “The 1967 Purge of the Gapsan Faction and Establishment of the Monolithic Ideological System.” North Korea International Documentation Project, Wilson Center. https://www.wilsoncenter.org/publication/the-1967-purge-the-gapsan-faction-and-establishment-the-monolithic-ideological-system (accessed December 6, 2020).Google Scholar
Pihl, Marshall. 1977. “Engineers of the Human Soul: North Korean Literature Today.” Korean Studies 1:63–110.CrossRefGoogle Scholar
Quine, Willard V. 1953. “On What There Is.” In From a Logical Point of View, 1–19. Cambridge, Mass.: Harvard University Press.Google Scholar
Dongwŏn, Ri. 1996. Chuch'e munhak riron 주체문학리론 [Theory of Juche literature]. Pyongyang: Kim Il Sung Chonghap Taehakkyo Ch'ulp’ansa.Google Scholar
Ch'anyŏl, Ryu. 1998. “90-yŏndae ŭi pukhansi” 90 년대의 북한 시 [North Korean poetry of the 1990s]. In Pukhan munhak ŭi inyŏm kwa silch'e 북한문학의 이념과 실체 [Actual state and concept of North Korean literature], edited by Lee, Myŏngjae, 228–47. Seoul: Kuk'hak charyowŏn.Google Scholar
Salazar, Julian, Liang, Davis, Nguyen, Toan Q., and Kirchhof, Katrin. 2020. “Masked Language Model Scoring.” In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, edited by Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetrault. Stroudsburg, PA: Association for Computational Linguistics.Google Scholar
Schmid, Andre. 2018. “Historicizing North Korea: State Socialism, Population Mobility, and Cold War Historiography.” American Historical Review 123 (2): 439–62.CrossRefGoogle Scholar
Hyŏnggi, Shin. 1988. Haebang chikhu ŭi munhak undongnon 해방직후의 문학운동론 [Theory of literary movements in the immediate post-Liberation period]. Seoul: Hwada.Google Scholar
Hyŏnggi, Shin and Oh, Sŏngho. 2000. Pukhanmunhaksa 북한문학사 [North Korean literary history]. Seoul: P'yŏngminsa.Google Scholar
Smith, Hazel. 2015. North Korea: Markets and Military Rule. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
So, Richard Jean. 2017. “All Models Are Wrong.” PMLA 132 (3): 668–73.Google Scholar
So, Richard Jean. 2020. Redlining Culture: A Data History of Racial Inequality and Postwar Fiction. New York: Columbia University Press.Google Scholar
Spivak, Gayatri. 2003. Death of a Discipline. New York: Columbia University Press.Google Scholar
Suleiman, Susan Rubin. 1983. Authoritarian Fictions: The Ideological Novel as a Literary Genre. New York: Columbia University Press.Google Scholar
Underwood, Ted. 2017. “A Genealogy of Distant Reading,” Digital Humanities Quarterly 11 (2). http://www.digitalhumanities.org/dhq/vol/11/2/000317/000317.html (accessed August 31, 2020).Google Scholar
Underwood, Ted. 2020. “How Predictable Is Fiction?” July 5 (blog). https://tedunderwood.com/2020/07/05/how-predictable-is-fiction/ (accessed September 20, 2020).Google Scholar
Underwood, Ted, and Sellers, Jordan. 2016. “The Longue Durée of Literary Prestige.” Modern Language Quarterly 77 (3): 321–44.CrossRefGoogle Scholar
Van Cranenburgh, Andreas, van Dalen-Oskam, Karina, and van Zundert, Joris. 2019. “Vector Space Explorations of Literary Language.” Language Resources and Evaluation 53:625–50.CrossRefGoogle Scholar
Van Rooden, Aukje. 2015. “Reconsidering Literary Autonomy: From an Individual towards a Relational Paradigm.” Journal of the History of Ideas 76 (2): 167–90.CrossRefGoogle Scholar
Wilmot, David, and Keller, Frank. 2020. “Modelling Suspense in Short Stories as Uncertainty Reduction over Neural Representation.” In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, edited by Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetrault, 1763–88. Stroudsburg, PA: Association for Computational Linguistics.CrossRefGoogle Scholar
Wittgenstein, Ludwig. 1953. Philosophical Investigations. Oxford: Basil Blackwell.Google Scholar