A Novel Multi-modal Graph Neural Network MAGNet for Detecting AI-Generated Scientific Contents

Abstract

The accelerated development of sophisticated large language models (LLMs) poses a serious and increasingly critical risk to academic integrity as it effortlessly generates high-quality scientific text that it is often barely differentiable to work by humans. Current detection systems, often based on statistical analysis or single-feature analysis, have severe shortcomings in generalizability between AI models and are not available to the broader academic community. In response to these issues, this study introduced MAGNet, a new Multimodal Graph Neural Network architecture that can be used to identify AI-made scientific text with high robustness and flexibility. Our model goes beyond conventional and unites and combines several feature modalities such as syntactic patterns, semantic relationships, and graph-based structural features into a single deep learning model. With such design, MAGNet is capable of finding weak, subtle artifacts and trends that are typical of AI generation that simpler statistical methods fail to detect. Trained on 5,000 scientific texts, a highly rigorously curated dataset, to guarantee a direct and fair comparison with modern studies, our model has an accuracy of 91.0% state-of-the-art, with a precision of 89.5% and a recall of 93.0%. Most importantly, MAGNet has shown a high level of performance in difficult situations, achieving the accuracies of 87.1% in cross-model generalization test and 84.5% in deliberately paraphrased documents, thus mentioning its high resilience to evasion behaviors. We have created and implemented a publicly accessible web application, to connect the gap between research and practice to provide the academic community with a powerful and user friendly interface to use this new advanced detection capability. Not only does our work set a new standard in performance in AI-text detection, but it also provides a highly valuable practical resource of educators, researchers, and publishers interested in protecting the integrity of scientific communication. The study concluded that introduced a new multimodal, graph-based neural network called MAGNet that determines AI-written scientific text. They showed that a combination of semantic, statistical, and structural analysis in the same deep learning system shows the highest performance, and the accuracy of the system is 91.0% on a large-scale, curated dataset.