Ant Group Awarded Shanghai Science and Technology Progress Prize for Advanced AI Security Technology

Shanghai Jiao Tong University and Ant Group, among others, snagged a top prize at the 2024 Shanghai Science and Technology Awards. Their collaborative project, titled “Key Technologies and Applications for Network Media Content Security Detection in Complex Adversarial Scenarios,” has earned them the prestigious Shanghai Science and Technology Progress Award, First Prize.

During the rigorous evaluation of the project’s core technologies, a panel of experts, led by Academician Wang Yaonan of the Chinese Academy of Engineering, concluded that “the technology as a whole has reached an internationally advanced level, with technologies such as AIGC detection reaching an international leading level.”

(Caption: Jiang Xinghao of Shanghai Jiao Tong University and Wang Weiqiang of Ant Group, key contributors to the project, at the Shanghai Science and Technology Awards Ceremony)

The rise of generative AI and large language models has brought immense potential but also introduced complex challenges. Highly intelligent content generation technologies can, unfortunately, propagate content that clashes with mainstream societal values, ethics, and ideological norms. This poses a significant threat to the healthy dissemination of information and the effective management of harmful online content. To combat this, Shanghai Jiao Tong University, Ant Group, and other organizations joined forces to develop key technologies in multi-modal understanding, AI authentication, anomaly reasoning, and multi-modal document parsing for complex adversarial scenarios in the age of large models.

“By improving the network governance system and promoting a positive online ecosystem, the research teams from the various participating organizations worked together to solve the problem of protecting information content security in the age of artificial intelligence,” stated Jiang Xinghao, Vice President of Shanghai Jiao Tong University and the project’s primary contributor, “actively leading a new model of content security governance in the era of generative AI.”

The project achieved four key technological innovations, addressing critical issues such as the generalization challenges of traditional detection models, the challenge of collecting abnormal samples, the detection of subtle content, and constantly evolving adversarial attacks.

First, it proposes a chain-of-thought-based multi-modal large model hallucination mitigation technology and develops deep learning models to solve model generalization and intrinsic security problems, providing a technical basis for content generation, detection, and traceability. Second, it develops a controllable multi-modal data intelligent generation method to meet the needs of model training for abnormal samples. Third, it develops an integrated solution for network media content security monitoring, overcoming the challenges of detecting generative subtle harmful content and AIGC content, and obtaining a 5-star rating from the China Academy of Information and Communications Technology. Fourth, it proposes a model security protection technology based on confrontation and traceability, and constructs an artificial intelligence model attack-defense algorithm library to improve the ability to prevent data abuse and model theft.

These key technological innovations not only significantly improve the prevention and control effects and efficiency under complex attacks in the field of content security, bringing breakthrough progress in the governance of harmful information on the Internet, but also provide strong technical support for the application of AI technology in a wider range of fields.

The project has been granted 41 invention patents, 1 national standard and 1 local standard, 10 software copyrights, and 118 papers. The project has been applied to content security business prevention and control in multiple platforms, greatly improving the effectiveness of new risk prevention and control, and achieving breakthroughs in instruction response time efficiency from days to minutes. Among them, AIGC security detection and other technologies serve the content-related businesses of Alipay, realizing the efficient identification and interception of harmful content and ensuring the health and security of platform content.

Wang Weiqiang, Chief Scientist of Ant Security Lab and a key contributor to the project, commented, “Under the guidance of the national cybersecurity strategy, Ant Group has been collaborating with academic institutions and research organizations to develop a smart governance system for Internet information security, incorporating core technologies such as implicit reasoning and AIGC recognition. This not only provides a safe and trustworthy creation environment for a vast number of creators but also strengthens the risk barrier with continuously iterating model capabilities. Ant Group will continue to invest in security technology, deeply cultivate reliable AI technology research and development, and promote more technological breakthroughs in the intelligent process of network security.”