Grad Students Fooled by AI Teaching Assistant

AI has a growing place in our society and a number of them are becoming harder and harder to distinguish from humans while performing their tasks. This has been further shown off by the Georgia Institute of Technology’s Jill Watson, a teaching assistant that the university had been employing for a full semester and also an artificial intelligence.

Jill was created to assist Ashok Goel, a Georgia Tech professor who teaches a class entitled Knowledge-Based Artificial Intelligence. This class was a mandatory core module as part of Georgia Tech’s online master’s program in computer science, which saw Goel and his eight teaching assistants snowed under in questions, with the 300 students enrolled on the course posting in excess of 10,000 questions on the course forums over a single semester. In order to help the team keep on top of the questions, Goel had to find an extra assistant, Jill.

As her name implies, Jill is based upon IBM’s Watson platform and trained by Goel and his assistants using the 40,000 questions that had been made about the course since it started running in 2014. It wasn’t all smooth sailing for Jill, as when she first started work in January, she was found to often post odd or irrelevant answers, but in the safety of a subforum invisible to students. One such answer was in response to a student attempting to organize a meet-up to go over video lessons with others, which she answered by referring to the textbook that could be used to supplement the video lessons, which, while related, was in the wrong context. After some further adjustments, the team was happy that Jill was capable of answering questions with as much as 97% certainty, with the human TAs initially posting Jill’s responses to the forums manually. By March, Jill was deemed capable of working alone and was able to post her answers directly to the students if she was 97% sure the answer was correct.

Through all of this, Jill’s true nature was kept secret from the students until the big reveal in late April, when Goel let his AI students in on the fact that the TA that had been answering their questions all semester was actually an AI itself. The response to this was incredibly positive, with one student reporting that her mind was “blown” with some setting up an alumni forum in order to keep up to date with news on Jill’s development and others launching an open-source effort to replicate her. For now, Jill will return to helping the students of the Knowledge-Based Artificial Intelligence next semester with the target of answering 40% of all class questions by the end of the year.