Hostname: page-component-669899f699-7tmb6 Total loading time: 0 Render date: 2025-05-01T23:24:37.932Z Has data issue: false hasContentIssue false
  • English
  • Français

Context Matters: Understanding Student Usage, Skills, and Attitudes Toward AI to Inform Classroom Policies

Published online by Cambridge University Press:  23 May 2024

Christine Cahill Open the ORCID record for Christine Cahill [Opens in a new window]  and
Katherine McCabe Open the ORCID record for Katherine McCabe [Opens in a new window]
Christine Cahill
Affiliation:
Rutgers University, USA
Katherine McCabe*
Affiliation:
Rutgers University, USA
*
[email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

With the growing prevalence of AI tools, such as ChatGPT, political science instructors are navigating how to manage the use and misuse of AI in the classroom. This study underscores the prevalence of AI in academic settings and suggests pedagogical practices to integrate AI in the classroom in ways that are informed by students’ current interactions with and attitudes toward AI. Using a survey of undergraduate students in political science courses, the study finds that ChatGPT usage is widespread at the university level and that students are not confident in their skills for using AI appropriately to improve their writing or prepare for exams. These findings point to key areas where instructors can intervene and integrate AI in ways that enhance student learning, reduce potential achievement gaps that may emerge due to differences in AI usage across student backgrounds, and help students develop critical AI literacy skills to prepare for careers that increasingly are affected by AI.

Type
Article
Information
PS: Political Science & Politics , Volume 57 , Issue 4 , October 2024 , pp. 594 - 601
Copyright
© The Author(s), 2024. Published by Cambridge University Press on behalf of American Political Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

REFERENCES

Bender, Emily M., Gebru, Timnit, McMillan-Major, Angelina, and Shmitchell, Shmargaret. 2021. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? ” In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610–23. FAccT ’21. New York: Association for Computing Machinery. https://doi.org/10.1145/3442188.3445922.CrossRefGoogle Scholar
Brown, Mitchell, Nordyke, Shane, and Thies, Cameron G.. 2022. Teaching Undergraduate Political Methodolgy. Cheltenham, UK: Edward Elgar Publishing.Google Scholar
Cahill, Christine, and McCabe, Katherine. 2024. “Replication Data for ‘Context Matters: Understanding Student Usage, Skills, and Attitudes Toward AI to Inform Classroom Policies.’” PS: Political Science & Politics. DOI:10.7910/DVN/1QA5PC.CrossRefGoogle Scholar
Choi, Jonathan H., Hickman, Kristin E., Monahan, Amy, and Schwarcz, Daniel. 2023. “ChatGPT Goes to Law School.” Rochester, NY: Social Science Research Network Scholarly Paper. https://doi.org/10.2139/ssrn.4335905.CrossRefGoogle Scholar
Dell’Acqua, Fabrizio, McFowland, Edward, Mollick, Ethan R., Lifshitz-Assaf, Hila, Kellogg, Katherine, Rajendran, Saran, Krayer, Lisa, Candelon, François, and Lakhani, Karim R.. 2023. “Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality.” Rochester, NY: Social Science Research Network Scholarly Paper. https://doi.org/10.2139/ssrn.4573321.CrossRefGoogle Scholar
Faverio, Michelle, and Tyson, Alec. 2023. “What the Data Says about Americans’ Views of Artificial Intelligence.” Pew Research Center (blog). Accessed March 21, 2024. www.pewresearch.org/short-reads/2023/11/21/what-the-data-says-about-americans-views-of-artificial-intelligence.Google Scholar
Ford, Heather, and Wajcman, Judy. 2017. “‘Anyone Can Edit,’ Not Everyone Does: Wikipedia’s Infrastructure and the Gender Gap.” Social Studies of Science 47 (4): 511–27. https://doi.org/10.1177/0306312717692172.CrossRefGoogle ScholarPubMed
Fowler, Geoffrey A. 2023a. “Perspective: ChatGPT Can Ace Logic Tests Now. But Don’t Ask It to Be Creative.” Washington Post, April 14. www.washingtonpost.com/technology/2023/03/18/gpt4-review.Google Scholar
Fowler, Geoffrey A. 2023b. “Analysis: Detecting AI May Be Impossible. That’s a Big Problem for Teachers.” Washington Post, June 2. www.washingtonpost.com/technology/2023/06/02/turnitin-ai-cheating-detector-accuracy.Google Scholar
Franklin, Daniel, Weinberg, Joseph, and Reifler, Jason. 2014. “Teaching Writing and Critical Thinking in Large Political Science Classes.” Journal of Political Science Education 10 (2): 155–65. https://doi.org/10.1080/15512169.2014.892431.CrossRefGoogle Scholar
Gilson, Aidan, Safranek, Conrad W., Huang, Thomas, Socrates, Vimig, Chi, Ling, Taylor, Richard Andrew, and Chartash, David. 2023. “How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment.” JMIR Medical Education 9 (February): e45312. https://doi.org/10.2196/45312.CrossRefGoogle ScholarPubMed
Hargittai, Eszter, and Shaw, Aaron. 2015. “Mind the Skills Gap: The Role of Internet Know-How and Gender in Differentiated Contributions to Wikipedia.” Information, Communication & Society 18 (4): 424–42. https://doi.org/10.1080/1369118X.2014.957711.CrossRefGoogle Scholar
Kalaf-Hughes, Nicole, and Cravens, Royal G.. 2021. “Does a Wikipedia-Based Assignment Increase Self-Efficacy among Female Students? A Qualified Maybe.” Journal of Political Science Education 17 (Sup1): 862–79. https://doi.org/10.1080/15512169.2021.1921586.CrossRefGoogle Scholar
Klein, Alyson. 2023. “Students and Parents Are Bullish on AI’s Potential for Education.” Education Week, May 15. www.edweek.org/technology/students-and-parents-are-bullish-on-ais-potential-for-education/2023/05.Google Scholar
Liang, Weixin, Yuksekgonul, Mert, Mao, Yining, Wu, Eric, and Zou, James. 2023. “GPT Detectors Are Biased Against Non-Native English Writers.” Patterns 4 (7): 100779. https://doi.org/10.1016/j.patter.2023.100779.CrossRefGoogle ScholarPubMed
Mollick, Ethan. 2023a. “How to Use AI to Do Stuff: An Opinionated Guide.” One Useful Thing, September 16. www.oneusefulthing.org/p/how-to-use-ai-to-do-stuff-an-opinionated.Google Scholar
Mollick, Ethan. 2023b. “On-Boarding Your AI Intern.” One Useful Thing, September 16. www.oneusefulthing.org/p/on-boarding-your-ai-intern.Google Scholar
Noy, Shakked, and Zhang, Whitney. 2023. “Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence.” Science 381 (6654): 187–92. https://doi.org/10.1126/science.adh2586.CrossRefGoogle ScholarPubMed
Oldmixon, Elizabeth A. 2018. “‘It Was My Understanding That There Would Be No Math’: Using Thematic Cases to Teach Undergraduate Research Methods.” Journal of Political Science Education 14 (2): 249–59. https://doi.org/10.1080/15512169.2017.1405356.CrossRefGoogle Scholar
Park, Eugenie, and Gelles-Watnick, Risa. 2023. “Most Americans Haven’t Used ChatGPT; Few Think It Will Have a Major Impact on Their Job.” Pew Research Center (blog). Accessed March 22, 2024. www.pewresearch.org/short-reads/2023/08/28/most-americans-havent-used-chatgpt-few-think-it-will-have-a-major-impact-on-their-job.Google Scholar
Peng, Sida, Kalliamvakou, Eirini, Cihon, Peter, and Demirer, Mert. 2023. “The Impact of AI on Developer Productivity: Evidence from GitHub Copilot.” arXiv. http://arxiv.org/abs/2302.06590.Google Scholar
Pennock, Andrew. 2011. “The Case for Using Policy Writing in Undergraduate Political Science Courses.” PS: Political Science & Politics 44 (1): 141–46.Google Scholar
Shaw, Aaron, and Hargittai, Eszter. 2018. “The Pipeline of Online Participation Inequalities: The Case of Wikipedia Editing.” Journal of Communication 68 (1): 143–68. https://doi.org/10.1093/joc/jqx003.CrossRefGoogle Scholar
Stebleton, Michael J., and Soria, Krista M.. 2012. “Breaking Down Barriers: Academic Obstacles of First-Generation Students at Research Universities.” Learning Assistance Review 17 (2): 7–20.Google Scholar
Trucano, Michael. 2023. “AI and the Next Digital Divide in Education.” Brookings, July 10. Accessed March 22, 2024. www.brookings.edu/articles/ai-and-the-next-digital-divide-in-education.Google Scholar
Villasenor, John. 2023. “How AI Will Revolutionize the Practice of Law.” Brookings, March 20. Accessed March 22, 2024. www.brookings.edu/articles/how-ai-will-revolutionize-the-practice-of-law.Google Scholar
Young, Erin, Wajcman, Judy, and Sprejer, Laila. 2023. “Mind the Gender Gap: Inequalities in the Emergent Professions of Artificial Intelligence (AI) and Data Science.” New Technology, Work and Employment 38 (3): 391–414. https://doi.org/10.1111/ntwe.12278.CrossRefGoogle Scholar
Supplementary material: File

Cahill and McCabe supplementary material

Cahill and McCabe supplementary material
Download Cahill and McCabe supplementary material(File)
File 1.2 MB