A collaborative human-AI case study in advanced physics problems

Abstract

Generative AI (GenAI), such as ChatGPT, has emerged as a powerful collaborator in numerical problem-solving across scientific domains. However, its reliability in addressing novel physics problems—those beyond the standard undergraduate curriculum—remains largely unvetted. We present a case study investigating the electric potential of uniformly charged elliptical geometries using Python simulations generated by ChatGPT and other AI models, demonstrating how GenAI can assist researchers in deriving numerical solutions and extracting physical insights from complex systems. While GenAI is a reliable tool for generating functional Python code for numerical calculations and data visualization, some internal inconsistencies have been observed in its outputs, and responses may vary across different AI models. Furthermore, even valid AI-generated solutions do not always align with human intuition, particularly in special limiting cases, e.g. extreme aspect ratios in elliptical coordinate systems. By employing careful coordinate transformations, we resolve these subtle mathematical ambiguities in a more intuitive framework and expose the profound connection between the Coulomb potential and elliptical geometry, revealing power-law scaling behavior at the singularity. To validate the versatility of this human-AI collaborative framework, we extend the methodology to a classical mechanics problem: the moments of inertia for various elliptical shapes. Our findings underscore the critical necessity of expert oversight and rigorous cross-examination when GenAI is deployed for unconventional scientific challenges. Ultimately, this work highlights a dual reality in modern research: while AI is a transformative tool for accelerating scientific discovery, its optimal performance remains deeply dependent upon iterative, interactive feedback from a human domain expert.