Twitter and its new owner, Elon Musk, have recently vowed to crack down on bots. But the flood of spam for Chinese users in recent weeks has underscored the challenges the company faces in policing fraudulent and inauthentic activity, especially in foreign languages that have traditionally been more loosely policed by large American social media platforms.
For many Chinese who turned to the platform as demonstrations against Covid-19 restrictions had grown political and widespread — more so than any protests there in decades — the experience of using the app appeared to capture a different reality.
Much of the spam is linked to commercial bot networks that have operated on Chinese-language Twitter since before the protests began on Nov. 25, according to an analysis by The New York Times of thousands of tweets, interviews with people behind some of the bot accounts and a report by a researcher at Stanford University.
The Times searched on Twitter for 10 Chinese cities, like “北京” (“Beijing”), and reviewed the results for spam. Bots were active throughout, the analysis found, even for cities where protests were not held. When contacted, two businesses that appeared in spam tweets said that they had purchased the tweets using advertising services.
The Times also searched for six city names outside of China using English and simplified Chinese. Only the English searches were free from spam bots.
The Times analysis matched findings published on Monday by David Thiel, the chief technologist at the Stanford Internet Observatory. Mr. Thiel reviewed millions of tweets by searching for 30 Chinese cities and found that bots were active before the protests began and continued after they had ebbed. Twitter suspended many of the accounts, but the spam continued as other bots kept tweeting, he wrote in the report.
Mr. Musk, who took over Twitter a month before the protests began, has said that controlling bots is an “absolute top priority.” But in the weeks since acquiring the company, he cut Twitter’s work force by about half. He also dissolved the company’s Trust and Safety Council, an advisory group formed in 2016 to address hate speech and other problems on Twitter.
In an internal email about the decision to end the Trust and Safety Council, the company said it would be “moving faster and more aggressively” to make Twitter safer, but provided no details on its plans. The company and Mr. Musk did not respond to a request for comment.
“I almost feel like it’s the Wild West,” Eirliani Rahman, a former member of the Trust and Safety Council, said in an interview. In a letter posted to Twitter announcing her resignation this month, she wrote that the “safety and well-being” of Twitter users was in decline, adding that “shooting from the hip is not how you do content moderation.”
In a recent live chat, Mr. Musk suggested that the company was being too aggressive in banning bots, suspending some legitimate accounts in the process. Yet among Chinese-language content on Twitter, the bots appear to maintain a significant presence.
Note: Contact information for the account has been obscured.
Some internet watchdogs have suggested that the Chinese government was behind the spam bots. The New York Times and Mr. Thiel at Stanford University found no evidence backing the idea.
The ad campaigns do share some resemblance with Chinese government propaganda campaigns, which tend to use swarms of bots to overwhelm authentic posts in search results. But those campaigns typically use political hashtags and echo state-backed messages.
The bots posting content during the protest did not focus on related hashtags; instead they included broader terms like the names of Chinese cities alongside adult content.
In China, prostitution and pornographic content is illegal, and powerful censors wage an unending battle to cut back advertising from those services. So those businesses often flock to platforms like Twitter and Instagram, which have far fewer resources dedicated to monitoring and maintaining content shared in Chinese.
Estimates about the number of bots on Twitter vary. The company has said in public disclosures that bots account for fewer than 5 percent of its user base. In 2017, researchers at Indiana University and the University of Southern California estimated that number was closer to 15 percent.
Yoel Roth, the former head of safety and integrity at Twitter who resigned last month, has criticized efforts to scrutinize bot activity on Twitter, writing in 2020 that automated accounts could be a “powerful tool” for customer service and other similar cases. He added that the company had instead focused its attention on limiting the “malicious use of automation” that aims to manipulate conversations.
As Twitter and social media platforms have prioritized eliminating fraudulent activity, misinformation experts say that the companies have largely failed in policing posts that are not in English. The Times analysis found that the bot activity for searches about #Beijing appeared far worse in Chinese-language searches compared with English ones.
Searching for “北京”
Searching for “Beijing”
A search on Twitter for “Beijing” in simplified Chinese
brought up tweet after tweet of spam …
… but searching for “Beijing” in English showed
no bot or spam activity among the top tweets.
Searching for “北京”
Searching for “Beijing”
A search on Twitter for “Beijing” in
simplified Chinese brought up tweet after
tweet of spam …
… but searching for “Beijing” in English
showed no bot or spam activity among
the top tweets.
Searching for “北京”
A search on Twitter for “Beijing” in
simplified Chinese brought up tweet after
tweet of spam …
Searching for “Beijing”
… but searching for “Beijing” in English
showed no bot or spam activity among
the top tweets.
Mr. Thiel, the researcher from Stanford, collected millions of tweets over several weeks to track the bot campaigns. He found hundreds of thousands of accounts with clear markers of bot activity, such as repeatedly tweeting the same content with a different combination of hashtags and photos. It often took Twitter days to suspend inauthentic accounts, which is typical for the social network, Mr. Thiel said. That allowed their content to overrun search results for days until the company acted — just in time for new accounts to be created and continue the drumbeat of spam.
“There’s always new spam campaigns being brought up, new evasion methods, new accounts getting created,” Mr. Thiel said in an interview.
Many accounts analyzed by The Times provide contact information in their profile or tweets. An employee from one business listed in a spam tweet said in an online chat that it provides escort services and had no connection to the government. The employee, who declined to give a name, said the company also runs ad campaigns on Facebook and Instagram.
Bot advertising on Twitter
A company listed on some spam tweets confirmed it ran an advertising business using bots. This rate guide describes prices and the services they offered.
Twitter ads business introduction
The price guide shows the number of bot accounts that can be purchased along with the daily price, monthly discount and monthly price.
• We provide all the accounts used for posting ads
• Every account tweets at least 150 times a day (one post every 5-10 min)
• We can let clients check the names of the accounts posting ads
• We provide free image and video design services
• The minimum length is 3 days, and minimum number of accounts is 50
Twitter ads business introduction
The price guide shows the number of bot accounts that can be purchased along with the daily price, monthly discount and monthly price.
• We provide all the accounts used for posting ads
• Every account tweets at least 150 times a day (one post every 5-10 min)
• We can let clients check the names of the accounts posting ads
• We provide free image and video design services
• The minimum length is 3 days, and minimum number of accounts is 50
Twitter ads business introduction
The price guide shows the number of bot accounts that can be purchased along with the daily price, monthly discount and monthly price.
• We provide all the accounts used for posting ads
• Every account tweets at least 150 times a day (one post every 5-10 min)
• We can let clients check the names of the accounts posting ads
• We provide free image and video design services
• The minimum length is 3 days, and minimum number of accounts is 50
In another online chat, a provider of advertising for escort and gambling services said that the business was responsible for some of the spam tweets. The person provided a rate sheet for online advertising through bots, which charges 9,600 renminbi, about $1,400, a month for an ad campaign on Twitter involving 200 bot accounts that will tweet at least 150 times a day. The company also offers photo and video design.
Spam accounts can also be purchased and repurposed by governments from bot marketplaces, making it hard to tell whether a campaign is commercial or political, according to one former Twitter employee who worked in its safety and integrity group and has analyzed state-backed influence operations. The former employee spoke on the condition of anonymity because of the terms of a severance agreement with the company.
“Given how effective it was at drowning out other people, I wouldn’t be surprised if a government of some kind tries to use this technique in the future,” Mr. Thiel said.