Online platforms are tightening oversight of AI-generated content and automated accounts as regulators and companies move to safeguard authentic user interactions and curb emerging risks tied to AI-powered tools.

Social media and content platform RedNote, also known as Xiaohongshu, announced new governance measures recently targeting accounts automatically operated through AI tools, saying violations could lead to account bans in the most serious cases.

The platform stressed that using AI to impersonate real users, create non-authentic content or conduct fake interactions will be strictly prohibited.

Under the new rules, accounts that are directly registered, operated and interacted with through AI hosting tools, or those whose posts are entirely generated and published by such tools, will face account bans, according to the statement.

The move comes as automated AI tools proliferate the market. With the recent wave of OpenClaw, an open-source AI agent, a large number of AI agent tools capable of automatically executing tasks have emerged.

Driven by commercial interests, such AI hosting can lead to a gray industry chain of "account creation, account nurturing and resale or monetization", according to a report by Securities Daily. Operators use AI to generate artificial popularity at scale before selling the accounts for profit or converting them into channels for product promotion or online courses.

"As AI execution capabilities grow exponentially, the 'cat-and-mouse game' between content platforms and AI production tools is likely to intensify," said a recent Citi report, adding that Xiaohongshu's move aims to build a firewall for the community to prevent the voices of real users from being drowned out by algorithm-generated content.

"It is every genuine record and every sincere exchange that together create the warmth of the community," Xiaohongshu said in its announcement, calling on users to use AI tools responsibly during the creative process and ensure that posts continue to reflect real experiences and personal insights.

Xiaohongshu's move is not an isolated case, as major Chinese online platforms have been stepping up efforts to curb AI-generated content.

ByteDance's Douyin announced last week that it had handled 42,000 pieces of vulgar or rule-violating AI-generated content since the beginning of the year. Around 14,000 accounts received penalties including traffic restrictions, temporary bans or permanent suspensions.

Weibo has also intensified efforts to this end, removing large amounts of content and suspending related accounts recently. The platform has also upgraded its AI detection technologies and called on users to label AI-generated content.

These moves align with broader regulatory efforts to strengthen governance frameworks for AI agents and AI-generated content. The Ministry of Industry and Information Technology issued a set of recommendations last week to help prevent security risks linked to the OpenClaw open-source AI agent framework, including potential cyberattacks and data leaks.

lijiaying@chinadaily.com.cn