When AI is Not AI

Abstract

Empirical research into these higher education chatbots has largely focused on how chatbots are used in two domains: 1) helping students through the enrollment management (ex: admissions, financial aid, housing) process and 2) providing instructor support to enhance student learning in a curricular context. This study reports on structured human-computer interactions with higher education chatbots to explore whether these chatbots were programmed to provide financial counseling to college students. During our structured interactions with chatbots, we learned that many systems marked as AI chatbots fell short of adaptive, generative capabilities which are the essence of AI systems (Regulations (EU) 2024/1689, Art. 3(1)). Specifically, we identified five points along the capability spectrum of generating responses: 1) chatbots were often “offline” during business hours, rendering them non-functional; 2) chatbots were fillable forms, thus functioning like emails; 3) chatbots were decision trees and not large language models, thus functioning like search engines and lacking adaptive response generation; 4) chatbots were actually chat applications staffed by human agents without disclosure of their non-automation; and 5) artificial intelligence chatbots were not overly intelligent in its responses to our queries. We address the implications of these findings for both research and communication ethics, sustainability, and institutional efficiency.