On February 5, 2020, the District Court of The Hague delivered its judgment in NJCM and others v. the Netherlands (ECLI:NL:RBDHA:2020:1878), a case brought by civil-society organizations against the Dutch state. At issue was SyRI (Systeem Risico Indicatie), a government system that linked data across agencies and ran it through a secret risk model to flag people as possible welfare, tax, or benefits fraudsters. SyRI had been deployed mainly in low-income neighborhoods.
The court ruled that the legislation governing SyRI did not strike a fair balance between the state’s interest in detecting fraud and the individual right to private life protected by Article 8 of the European Convention on Human Rights. Because the system was opaque, people had no way to know how they had been profiled or to challenge the result, and the court found the interference with privacy was not necessary and proportionate. It declared the SyRI legislation unlawful and ordered that the system not be used. The government did not appeal, so the ruling became final.
The decision was one of the first anywhere in which a court halted a government automated decision system on human rights grounds, and it was widely cited by privacy advocates and a UN special rapporteur as a precedent for the digital welfare state.
For organizations of any kind, the takeaway is that an automated scoring system that people cannot see into or contest is a legal and ethical liability, not just a technical one. Transparency and the ability to appeal are not optional extras when software decides who gets investigated.