The AI Act should use humans to monitor AI only when effective

 

The EU’s proposed Artificial Intelligence Act (AI Act) wants to use humans to oversee AI-generated decisions, but recent evidence suggests one should carefully test it when it becomes possible.

Johannes Walter is a researcher at the Leibniz-Centre for European Economic Research and a PhD Candidate at KIT in Technical Economics.

Nijeer Parks was arrested for a crime he didn’t commit. He was arrested due to an incorrect match produced by a facial recognition algorithm. Being already the third known case of a Black man being wrongfully arrested in the US, these false arrests illustrate the lower accuracy of facial recognition algorithms for Black faces. 

But what failed Mr Parks was not just technology. Had police officers double-checked the matching images, they would have noticed the suspect, and Mr Parks did not look alike. The suspect in the photo even wore earrings — whereas Mr. Parks had no piercings. The human police officers should have overseen the algorithmically generated arrest recommendation but failed to do so.  (...)