AI and Big Tech should not have control over health

Big-data health-care solutions are being rushed to market in without meaningful regulation, transparency, standardisation, accountability, or robust validation practices



By Leeza Osipenko


Published: Sat 25 Jul 2020, 12:48 PM

Last updated: Sat 25 Jul 2020, 2:56 PM

In an interview with the Wall Street Journal earlier this year, David Feinberg, the head of Google Health and a self-professed astrology buff, enthused that, "If you believe me that all we are doing is organising information to make it easier for your doctor, I'm going to get a little paternalistic here: I'm never going to let that get opted out." In other words, patients will soon have no choice but to receive personalised clinical horoscopes based on their own medical histories and inferences drawn from a growing pool of patient records. But even if we want such a world, we should take a hard look at what today's health-tech proponents are really selling.
In recent years, most of America's Big Tech firms - along with many startups, the Big Pharma companies, and others - have entered the health-tech sector. With big-data analytics, artificial intelligence, and other novel methods, they promise to cut costs for struggling health-care systems, revolutionise how doctors make medical decisions, and save us from ourselves. What could possibly go wrong?
Quite a lot, it turns out. In Weapons of Math Destruction, data scientist Cathy O'Neil lists many examples of how algorithms and data can fail us in unsuspecting ways. When transparent data-feedback algorithms were applied to baseball, they worked better than expected; but when similar models are used in finance, insurance, law enforcement, and education, they can be highly discriminatory and destructive.
Health care is no exception. Individuals' medical data are susceptible to subjective clinical decision-making, medical errors, and evolving practices, and the quality of larger data sets is often diminished by missing records, measurement errors, and a lack of structure and standardisation. Nonetheless, the big-data revolution in health care is being sold as if these troubling limitations did not exist. Worse, many medical decision-makers are falling for the hype.
One could argue that as long as new solutions offer some benefits, they are worth it. But we cannot really know whether data analytics and AI actually do improve on the status quo without large, well-designed empirical studies. Not only is such evidence lacking; there is no infrastructure or regulatory framework in place to generate it. Big-data applications are simply being introduced into health-care settings as if they were harmless or unquestionably beneficial.
Consider Project Nightingale, a private data-sharing arrangement between Google Health and Ascension, a massive non-profit health system in the US. When the Wall Street Journal first reported on this secret relationship last November, it triggered a scandal over concerns about patient data and privacy. Worse, as Feinberg admitted just two months later, "We didn't know what we were doing."
Given that the Big Tech companies have no experience in healthcare, such admissions should come as no surprise, despite the attempts to reassure us otherwise. Worse, at a time when individual privacy is becoming more of a luxury than a right, the algorithms that are increasingly ruling our lives are becoming inaccessible black boxes, shielded from public or regulatory scrutiny to protect corporate interests. And in the case of healthcare, algorithmic diagnostic and decision models sometimes return results that doctors themselves do not understand.
Although many of those pouring into the health-tech arena are well-intentioned, the industry's current approach is fundamentally unethical and poorly informed. No one objects to improving health care with technology. But before rushing into partnerships with tech companies, health-care executives and providers need to improve their understanding of the health-tech field.
For starters, it is critical to remember that big-data inferences are gleaned through statistics and mathematics, which demand their own form of literacy. Another critical area is AI, which requires both its own architecture - that is, the rules and basic logic that determine how the system operates - and access to massive amounts of potentially sensitive data. The goal is to position the system so that it can 'teach' itself how to deliver optimal solutions to stated problems. But, here, one must remember that the creators of the architecture - the people writing the rules and articulating the problems - are as biased as anyone else, whether they mean to be or not. Moreover, AI systems are guided by data from the current health-care system, making them prone to replicating its own failures and successes.
At the end of the day, improving health care through big data and AI will likely take much more trial and error than techno-optimists realise. If conducted transparently and publicly, big-data projects can teach us how to create high-quality data sets prospectively, thereby increasing algorithmic solutions' chances of success. 
Above all, health-care providers and governments should remove their rose-tinted glasses and think critically about the implications of largely untested new applications in health care. Having been massively overhyped, big-data health-care solutions are being rushed to market in without meaningful regulation, transparency, standardisation, accountability, or robust validation practices. Patients deserve health systems and providers that will protect them, rather than using them as mere sources of data for profit-driven experiments. 
Leeza Osipenko is Senior Lecturer in Practice in the Department of Health Policy at the London School of Economics and Political Science. -Project Syndicate 
 


More news from OPINION
Unjabbed Djokovic is humbled Down Under

Opinion

Unjabbed Djokovic is humbled Down Under

Real champions put spectators first in the pursuit of glory. Novak, however, has emerged the Djoker of the pack by riding slipshod over the rules. He almost got away with his antics until good sense prevailed and the Australian government and legal system intervened to show him the door.

Opinion1 week ago

India is the market for the next decade

Opinion

India is the market for the next decade

Corporate earnings appear to be on the cusp of revival. The earnings growth is expected to be more than 50 per cent between FY20 and FY22; earnings growth momentum is likely to continue at more than 25 per cent annually over the next couple of years.

Opinion1 week ago

An assault, a trial and a road to nowhere

Opinion

An assault, a trial and a road to nowhere

A day after a leading south Indian actress opened up on social media about her life as victim and survivor of a sexual assault in 2017, a groundswell of support is forming in the film industry of her home state, Kerala.

Opinion1 week ago

Can US elections be made safe from Capitol-type mob violence?

Opinion

Can US elections be made safe from Capitol-type mob violence?

It’s understandable that Biden may be reluctant to prosecute his 2020 election opponent and potential opponent in 2024. Even if neither man runs in 2024, prosecution of a former president by his successor would be a huge distraction from Biden’s efforts to govern, would divide the country even further, and perhaps lead to even worse violence than a year ago.

Opinion1 week ago

Digitisation set to revolutionise healthcare

Opinion

Digitisation set to revolutionise healthcare

The future of healthcare is shaping up in front of us. You will see humanoid robots in many hospitals — many will be unseen, sitting inside computers and providing diagnosis and treatment protocols to help the doctors. Wearables and trackers with remote monitoring has already become quite popular and will increase significantly.

Opinion1 week ago