POST OFFICE HORIZON SCANDAL Post Office Compensation offers have been made to more than 2,000 current and former sub-postmasters and postmistresses “It reached the point where it was going to get embarrassing if they admitted that there were problems, because they’d been prosecuting on a really aggressive basis for several years,” Christie adds. The computer is always right However, political pressure eventually forced the Post Office’s hand and in 2012 the organization hired the forensic accounting firm Second Sight to conduct a third-party audit of the Horizon software. Second Sight had its contract with the Post Office terminated after it produced two reports revealing problems with the Horizon system, which the Post Office refuted. In addition to the unfair burden placed on them through their contract with the Post Office, the sub-postmasters were burdened by the undue credibility given to computer evidence under UK law, according Dr Sam De Silva, chair of the law specialist group at BCS – the UK Chartered Institute for IT, and a technology partner at the law firm CMS. Under existing UK law, says De Silva, “Evidence produced by computers is treated as reliable unless other evidence suggests otherwise.” He adds, “This way of handling evidence is known as a ‘rebuttable presumption’. A court will treat a computer as if it is working perfectly unless someone can show that is not the case. Unfortunately, this presumption poses a challenge to those who dispute evidence produced by a computer system. Frequently the challenge is insurmountable, particularly where a major institution or corporation operates the system.” A COURT WILL TREAT A COMPUTER AS IF IT IS WORKING PERFECTLY UNLESS SOMEONE CAN SHOW THAT IS NOT THE CASE” Dr Sam De Silva, BCS Christie agrees, explaining, “I’ve worked as a software test monitor and so I’ve got experience of delving into the deep technical detail of systems to find out whether they’re flawed. That’s difficult technical work even for somebody who’s got the right technical experience and unrestricted access to the system. The idea that an attorney or a defendant in court would be able to do that, it’s just laughable. This presumption of computer liability in effect flipped the burden of proof Five important lessons to [onto the postmasters].” learn from the Post Office BCS is among those calling for a Horizon scandal change in the UK computer evidence law in the wake of the Post Office scandal. 1) When problems are reported, This is particularly important given the carry out an independent rise of AI, which could make IT systems assessment of the software – don’t even harder to verify. just rely on the manufacturer’s word. De Silva comments, “The information and procedures embedded through 2) Ensure the people installing the machine learning in AI systems differ software know how it will be used from conventional rule-based and understand the functionality procedural steps seen in ordinary required. If possible, make sure computer software. Instead, they are a technical expert is involved in encoded in the weights of features and procurement and rollout. the architecture of various components, 3) Explore contractual issues adopting a probabilistic approach. and who is at fault if something “Consequently, traditional software goes wrong. Is it a fair contract testing methods for identifying errors for both parties? and bugs may prove inadequate. This is particularly true if AI systems contain 4) Question whether the law’s latent errors that only manifest in stance on the computer always exceptional circumstances.” being right unless proven otherwise is fair. Make sure you know how Cultural beliefs to properly test a system to find But even so, the intricacies of UK any glitches, especially with the computer law are only a small part of rise of AI. the story. The much bigger question is 5) Check your company culture. how the situation was allowed to get Try to foster a culture where your so far out of hand that innocent people staff are believed first and foremost, were dragged through the courts in the and work with them to uncover first place. To answer this requires and understand problems with examining the culture within the Post software systems. Office itself at the time, Christie believes. “I think part of it was maybe a complacency,” he says. “The people working there absorbed the Post Office culture and they just deeply, instinctively believed that they were the good guys, and couldn’t conceive that they’d do anything as terrible as this.” According to Durgan Cooper, the big take-away for other postal companies is the importance of fostering a culture in which staff are believed. “I think that on the whole you have to trust people,” argues Cooper. “And when they are raising problems, they need to be listened to and it be taken on board professionally so that the risks can be identified and investigated.” KEY TAKEAWAYS 58 www.ParcelandPostalTechnologyInternational.com March 2024