I spy with my artificial eye...
Welcome to the second issue of my monthly newsletter
I'll be sharing analysis and short stories about digital transformation, practical recommendations, or recommended reading on this platform.
This month, some thoughts about what it means if we not only generate more and more information automatically but also analyze this information automatically. A short story about the potential downsides of automating information analysis.
Please enjoy!
He’s had it. This was the last time that they would disregard his analysis. For years he had warned the board of the Federal Reserve that there were significant flaws in the models they used to decide on the interest rates and that their communication via press statement was outdated and highly problematic. For years, hedge funds and fintech start-ups had specialized in analyzing every statement of the Fed – be it on social media a press release or even using facial recognition and sentiment analysis during conferences or media appearances – to gain an edge and anticipate decisions. He had warned his superiors time and time again about these developments and the need to rethink how the institution and other market-relevant institutions communicated in this new age. But he was ignored. Well, after tomorrow, no-one could ignore him again.
He had just re-run a final test on tomorrow’s press release. Everything worked as intended. Reading through it, he was still astonished at what he had accomplished. But only tomorrow would show, whether his revenge would bear fruit.
“It is chaos out here!” *click* “No one really knows what’s going on!” *click* “The markets are in free fall around the world this morning after the press release by the Federal Reserve on its decision to lower interest rates, leaving economists everywhere baffled.” Staring at the news on every channel, the Chairman of the Federal Reserve was taken out of his shock when his secretary reminded him: “Sir, I have the President on the line for you.” How could this have happened? Yes, the economy was in a tough spot but that was precisely why they lowered the interest rates from their previous high level. And significantly too. The wording of the statement couldn’t have been clearer! Did somebody say something confusing at a media appearance, was there a mix up in the statements?
Over 200 miles to the north in the center of Manhattan a distressed hedge fund manager had gathered his quantitative analysts to the board room, his head almost as red from anger and screaming as the performance numbers flashing on the screens. “What is going on?!? Talk to me, guys!” “Our algorithmic trading has issued massive sell orders to our stock portfolio based on the automated analysis of the Fed’s press release.” “WHY WOULD IT DO THAT!? Why would we sell when we should be buying?” “We are working on that.” “Ah, good to hear! My team of alleged geniuses is working on it…well can I expect anything useful before all our liquidity goes out the window?!” After a moment of silence that felt like an eternity, a young analyst raised her voice: “It could be data poisoning.” “What?” “It could be a case of data poisoning. We as humans read the text and arrive at one conclusion, but a machine might process the text and arrive at a completely different conclusion.” “How is that possible? I mean I have the stupid statement right here and it says black on white: we lower the interest rate. Why is the market behaving like it said the opposite!” “There might be little changes, strange punctuation, a word or two out of place that doesn’t necessarily appear to us as humans but completely throws off the algorithm. I mean to be honest, not even we really understand what those algos are doing most of the time.” While that frank statement gathered angry looks from the other analysts, the manager had suddenly become calm. “I need to make a phone call…”
By 10:30, trading on the markets had been halted, trillions had vanished, and the dust was settling. What was called a “technical glitch” vis-à-vis the public had been an unfortunate combination of previously experienced flash crashes, driven by automated high-speed trading which was in turn triggered by faulty automated information analysis, all fooled by a disgruntled employee that had messed with the contents of the press release. When he was apprehended at his desk, throwing away a life-long career in public service, he was still smiling, having witnessed his successful revenge all morning on the news.
Background:
Although this story is of course fiction, it is far from unrealistic. What a disgruntled employee did in this short story could be done with intent by any malicious actor either for chaos or for profit. It could also simply happen by accident and negligence. Already today, we see direct effects if regulatory information is misinterpreted or wrong information is spread by believable sources, e.g. when an false announcements regarding Bitcoin ETFs from a hacked social media account of the SEC had real consequences on prices. It is also well-known in AI research, that changes can be made to the data that lead to completely different interpretations for machines, but those changes are not visible to humans. This not only affects images but of course also text-based documents which can have serious implications for international affairs, see e.g. this paper. As we move in a direction where not only more information is produced automatically but we also increasingly rely on automation to analyze and make sense of all this information, we need to be aware of the potential vulnerabilities of such systems.
Subscribe to this newsletter and follow me on social media for more publications and thoughts:
LinkedIn
Twitter
Bluesky
Mastodon