China’s social credit system raises questions, concerns and potential

china social credit header image

Credit systems are used for everything from bank loans to credit card approval. But China’s social credit system project (SCS/SCSP) takes the idea further, using “credits” to determine a person’s or business’ trustworthiness, incentivizing good behavior and penalizing those who practice non-compliance.

The SCSP, piloted in 2010 and implemented in 2014, can use data about citizens and businesses to decide what flights you can take, if your kids can go to private school, if you can join a dating website or get a business license, when you’re granted parole, and more.

Data is aggregated from government records and financial institutions, and purportedly also sources like social media activities and health records. Police use security cameras and facial recognition. But Xin Dai, professor and associate dean at Ocean University of China Law School, doesn’t believe data generated from those sensors go directly into the social credit database. Rather, if a person or business violates a legal or regulatory requirement, then the record of that violation is entered into the credit database.

Once a profile is created, a social credit score is assigned. And it can fluctuate: donate to charity or volunteer and it might go up. Get caught speeding, robbing a bank, or defrauding customers? It will go down.

Curbing bad behavior and corruption

shanghai skyscrapers by the water

China wants to make the SCS fully operational by 2020, with data on all 1.4 billion citizens available publicly. Last year, any company operating a Chinese business license had to participate. And to date, more than 40 local government-run systems are ranking citizens, businesses and social organizations across the country, along with commercial pilots from technology companies like Alibaba and Tencent.

“Keeping trust is glorious and breaking trust is disgraceful,” reads a government document. The purpose of the system is to improve governance and restore market order. China is hoping these incentives (and punishments) will encourage citizens and businesses to behave appropriately and honestly.

According to a survey by Genia Kostka, Professor of Chinese Politics at Freie Universitat in Berlin, 80 percent of Chinese citizens view the SCSP favourably, and not as an instrument of surveillance, believing it can help protect them from fraud, food safety and quality issues, environmental pollutions, government corruption and bad business overall.

They also value the potential benefits for high scorers, like quicker travel applications, booking hotels without a deposit and bypassing lines. Are the positive responses influenced by living in an authoritarian society? That’s unclear. But a large number of value-neutral responses would also suggest that there are people who hold doubts.

A “Black Mirror” episode come to life? Not exactly

It’s easy to see how this system raises as many concerns as it does questions. It’s reminiscent of the fictional society portrayed in Black Mirror’s “Nosedive” episode, whereby social points determine everything from what parties you’re invited to, to what neighborhoods you can live in. The result? People forced to live disingenuous, repressed lives in an effort to keep their scores up.

While that’s an exaggerated example, China’s social credit system has already impacted those who aren’t upholding social conventions. Nine million people were not able to buy a domestic flight ticket due to their credit scores, while dog owners in one town reportedly lost points for infractions, like not walking the dog on a leash or failing to pick up the dog poop.

Once they lose a certain number of points, citizens must take a test on dog ownership regulations before getting their pet back. It might sound shocking, but it’s not unlike our own demerit point system for drivers.

asian women crouches next to her baby and dog

U.S. vice president Mike Pence calls the SCSP “Orwellian,” saying it is “premised on controlling virtually every facet of human life.” But Dai cautions against believing some of the embellished reports that suggest things like people being unable to enroll in school after refusing military service. Military service in China, by the way, is not mandatory.

While the very idea of an SCS is something most Westerners can’t fathom, it might not be as frightening as many media sources make it out to be. China’s SCS isn’t an entirely foreign concept:

  • Electricity companies incentivize customers with reduced rates when they use smart thermostats.
  • Ratings on Airbnb or Yelp impact customer decisions.
  • Uber drivers can be penalized if their rating drops below a certain level. As a passenger, you might not get picked up if your rating is low too.
  • In New York, Los Angeles and Toronto, restaurants must display health inspector ratings on windows, and are shut down if they fail.

But the SCS needs to be fair and transparent and encourage a desire for citizens and businesses to practice good behavior versus fear of living in a dystopian society. Many Chinese citizens already believe the government can access their information anyway, so an SCS works in their favor.

Privacy concerns and possible abuse

crowds in china visiting temple

While the SCS can dissuade companies from engaging in bad behavior, there are privacy issues to consider, too. Is it right to publicly shame people for their behavior? Or prevent them from visiting family abroad because they didn’t pay their phone bills on time? Can situations be taken out of context and ruin someone’s reputation, even their chances of getting a job?

Other risks include anti-competitive behaviors, the whitewashing of data, biased or untruthful reporting, officials or businesses abusing the system to target specific persons or competitors, and having the same standards not applying to people in power. There’s still a lot of work to be done if the SCS is to function effectively.

Technological infrastructure needs to be straightened out, says Dai. “There’s a long way to go for system interoperability among different regions.” There also needs to be clear-cut rules for credit evaluation. Currently, it’s unknown how the algorithms actually calculate scores.

“The government may, in the future, increasingly try to bring in private companies to assist with these efforts,” Dai adds. “But the two parties’ incentives may not necessarily always align well.”

Additionally, there’s no way of knowing yet whether the program is actually effective and justifiable in relation to the cost of implementation. “The supreme court each year will simply report that millions of deadbeats choose to repay their debt under the pressure of the SCSP,” says Dai. “But there is no way to validate claims for such causal effect.”

It does have the potential to change law and government as we know them in China, if institutional arrangements and technological progress align.

A work in progress

Dai says they’re making progress towards setting up regulations over government processing of personal and corporate data. But if China’s social credit system is to be more than just a vehicle for expanding the reach of the government and the law, Dai believes there needs to be “a proper… relationship between a new layer of government regulatory power, which is to define good and bad ‘credit’ on individuals and entities and subject the same to credit-based incentives/disincentives, and the pre-existing government administrative and regulatory apparatus. This,” he says, “calls for something akin to constitutional design, as the state needs to figure out where to locate its different types of powers.”

If China can find a healthy balance, the SCS could curb growing issues in the country, like fraud, economic crime and enforcing court decisions.

“It does have the potential to change law and government as we know them in China,” says Dai, “if institutional arrangements and technological progress align.”

About Futurithmic

It is our mission to explore the implications of emerging technologies, seeking answers to next-level questions about how they will affect society, business, politics and the environment of tomorrow.

We aim to inform and inspire through thoughtful research, responsible reporting, and clear, unbiased writing, and to create a platform for a diverse group of innovators to bring multiple perspectives.

Futurithmic is building the media that connects the conversation.

You might also enjoy
cybersecurity header image
An inside job: Why every company must hack its own cybersecurity