USA Since its launch in November 2022, OpenAI’s large language model–based chatbot ChatGPT has impacted people, businesses and industries around the world and challenged some fundamental ideas about how we create, communicate and learn. The question on everyone’s mind is what role will this generative AI tool play in our future, particularly in the education sector. Teachers, parents and academics are questioning whether utilising ChatGPT as a learning resource can benefit their students, but so much is still yet unknown about its capabilities, and more importantly, its accuracy. Earlier this year law professors at the University of Minnesota decided to put this theory to the test by using ChatGPT to generate answers to exams in four courses and then grading them blindly alongside actual students' tests. The chatbot passed despite ‘mediocre’ grades ranging from a B to C-, but what does this mean for the future of AI and the study of law?
ChatGPT As A Law Student
Professors Jonathan Choi, Kristin Hickman, Amy Monahan and Daniel Schwarcz at Minnesota University Law School had ChatGPT take exams in torts, employee benefits, taxation, and aspects of constitutional law. The tests included a total of 95 multiple-choice questions and 12 essay questions. The chatbot generally did better on the essays than the multiple-choice questions, scoring in the 17th percentile of all students and the 7th percentile, respectively. But its essay performance was inconsistent.
Overall the AI student earned an average C+ performance, below the humans' B+ average. If applied to the full curriculum, these results would be enough for ChatGPT to earn a degree but would be placed on academic probation if studying at the University of Minnesota.
"Alone, ChatGPT would be a pretty mediocre law student," said lead study author Jonathan Choi.
“But we expect that collaborating with humans, language models like ChatGPT would be very useful to law students taking exams and to practising lawyers.”
The sentiment was echoed by the study’s co-author, Law Professor Daniel Schwarcz.
“It is becoming increasingly likely that in the near future, many lawyers will need to collaborate with AIs, like ChatGPT, both to save time and money and to improve the quality of their work product,” he said.
Is ChatGPT Better As A Business Student?
Early this year, researchers at the Wharton School of the University of Pennsylvania wanted to see if the popular bot could earn a Master of Business Administration (MBA). According to Christian Terwiesch, a Wharton business professor and author of the study, ChatGPT scored between a B- and B in the final exam for the school's MBA program.
“The bot did an amazing job at basic operations management and process analysis questions including those that are based on case studies,” said Terwiesch.
However, the results also indicated limitations in the bot’s knowledge.
“Chat GPT3 at times makes surprising mistakes in relatively simple
calculations at the level of 6th grade Math. These mistakes can be massive in magnitude,” noted Terwiesch.
While ChatGPT now uses the GPT-3.5 model that includes a fine-tuning process for its algorithm as well as a paid ChatGPT Plus upgrade that uses GPT-4 with a faster response time and internet plugins, Terwiesch’s concerns about the accuracy and overall capabilities of this tool are still ongoing.
Technology & The Study of Law
Over the last 40 years, technological innovations have changed and shaped the education sector from primary all the way through tertiary and the study of law is no exception. Blackboards in college auditoriums turned into overhead projectors, scouring the library shelves for the right textbook gave way to online academic databases like JSTOR, and in more recent years, studying law at University meant completing a Master of Laws online instead of on campus. So why can’t AI platforms like ChatGPT be used to help students with analysing cases and statutes, legal research, document generation or even generating practice questions and answers for exam preparation?
Well as of this moment, the accuracy of answers generated by ChatGPT is not always reliable due to the limitations of how it was “taught”. ChatGPT interacts with users via a single-line text entry field (similar to Google) and provides results based on the context and intent behind the user’s questions. But while Google can access an enormous database of information that’s constantly growing and updating thanks to its crawl bots, ChatGPT’s results are based only on digitally-accessible text-based information available prior to 2021 when its generative training model (the architecture that drives AI) was finalised.
As a result, ChatGPT cannot pull from new legislative acts, judicial opinions, or scholarly articles post-2021, making it a limited asset for law students. Additionally, the nuances and complexities of the law often require seasoned human judgement, which AI platforms can't yet fully emulate. So it’s fair to say ChatGPT is a tool, not a replacement. For now, the human touch remains vital in the realm of legal studies.
University Policy Hasn’t Yet Caught Up
Leaving aside the accuracy of ChatGPT responses to legal questions and its effectiveness as a study aid, many students also have ethical concerns regarding this platform. According to a recent BestColleges survey of 1,000 current undergraduate and graduate students in the US, 51% believe that using AI tools like ChatGPT to complete assignments and exams is cheating. Two in ten (20%) disagreed, and the remainder were neutral. While using AI to complete coursework is different to looking at a classmate's answers or sneaking notes into an exam, it’s easy to understand why using ChatGPT could be viewed as a form of cheating. After all, assignments are supposed to be a demonstration of applied learning, and using AI means students haven’t done this to completion. They can potentially reap the rewards of study and hard work without necessarily doing the study and hard work. It raises the question about whether or not this counts as academic misconduct at the very least.
What’s more, university policies are yet to catch up regarding the use of AI and cheating. The Los Angeles Unified School District, Baltimore County Public Schools, Seattle Public Schools and many more have already banned the use of ChatGPT in K-12 learning, but as of now, no US colleges have announced an official ban. Washington University in St. Louis hasn’t banned ChatGPT but did update its academic integrity policies to include generative AI under the definition of plagiarism and The University of Vermont in Burlington is looking to do the same.
In this digital age, ChatGPT's achievements bring a fresh set of challenges and questions for educators. Its success in both law and business exams is noteworthy, but it's clear that it's not a perfect solution. As technology keeps pushing boundaries, we must reassess how we define learning and academic honesty. Schools and colleges need to find a middle ground. While tools like ChatGPT can be helpful study buddies, real learning is about human curiosity, deep understanding, and personal growth. As we shape the education policies of the future, it's essential to remember that while AI can inform and assist, the heart of education will always be human.
Source: Story.KISSPR.com
Release ID: 780461