AI can be transformative for all areas of healthcare, but often these systems are built off biased datasets that don’t reflect the true diversity of the general public – leading to approaches that only work for specific populations.
It might be tempting to think that because AI tools are machines, they are free from the subconscious biases and diversity issues that have long been problematic for clinical research. But in truth, these technologies are usually built off datasets that reflect human bias, which can, in turn, affect how an AI system operates.
Because of this, there are increasing calls for ‘AI Equity’ to be promoted across the industry.
AI Equity refers to how effective AI is in a broad range of scenarios with a variety of population groups and demographics. This requires taking into account the biases that often are automatically present in data.
The bias in the data is often exposed when companies run tests using models created from the data. Often AI is simply uncovering what was already there – it’s not necessarily creating a new problem, but it can enhance a problem that already exists.
These tests might involve identifying the groups of people who will be most affected by an AI system, then finding the right data sources with which to train the algorithm.
The data itself can often be biased, so it’s important to match it up with the demographics who are ultimately going to use the system that you are creating.
Hospitals or other systems that generate this data have patient populations who are typically wealthy, more connected, and with more access to digital systems. In other words, they are skewed towards populations who already have access – which leads to systems that are skewed in that way as well.
For example, Imagine a new AI-generated COVID test that is 99% accurate with a white Caucasian population and only accurate 40% of the time with southeast Asian populations. Millions could potentially become very ill. If the datasets don't have enough and similar information about all the scenarios being evaluated, results may turn severely flawed.
One example of an approach that seeks to overcome such variances in data comes from researchers at MIT, who have developed a deep learning system that can predict a patient’s breast cancer risk leveraging only the person’s mammograms, with the hope that this means it will work equally effectively regardless of race or ethnicity.
Nonetheless, these kinds of AI inequities are still a widespread problem.
There are thousands of pilot projects ongoing all over the world, and the more pilot systems that are created, the more widespread the problem becomes.
Stringent regulations in healthcare mean that the sector often fares better in data bias compared to others – but there is also more potential for harm if healthcare gets this wrong.
During the COVID-19 pandemic, models created in one specific environment or country were often used to create policy in another country (often due to lack of resources) without consideration for how populations might provide different data.
If a drug is developed using genetic data from a certain group of people, and it does not work with the same level of efficacy for other populations, that would be very problematic and potentially even harmful for those other groups.
An additional effect would be reduced trust in both AI and healthcare systems in general.
If a model is proven to be wrong, it’s going to be hard to get people to trust the next model that comes along.
Making AI equitable from the ground up
To implement AI in an equitable way, companies need to have the right experts on their teams.
Often what happens, especially in pilot projects, is that companies have experts who know a lot about the technology but not enough about the demographics of the end-users and their needs. It’s important to have domain experts involved in the project from the beginning.
Another challenge is having access to the right data in the first place.
There’s lots of data out there, but just having large volumes of data doesn’t mean that the issue of bias is already taken care of.
Likewise, it can be tempting to use certain algorithms simply because they are easy to implement – but, again, it’s important to look at a variety of options and the pros and cons of each one to find the right AI algorithm - the validation of those algorithms need to be done with the right sets of test cases.
From there, it’s important to think about who will have access to the project pilot.
It’s easy to release to a pilot population who already have access to such technology and are primed in other ways to benefit from the technology – but companies need to make sure that the pilot group also represents the variety within that population who will ultimately use it.
All of this can be more time- and resource-intensive than companies might have expected, which can be a barrier – but if companies don’t start an AI project in the right way, they will end up with built-in issues in the system.
They need to focus on the ultimate benefit rather than the immediate return on investment (ROI). Being impatient with ROI can lead to suboptimal systems in the long term.
Benefits for humanity
But despite these teething issues, the feedback about how AI is shaping the future of pharma and healthcare has been positive.
Electronic health record (EHR) systems have started to use AI-based platforms that can recommend the right treatment pathway for a patient depending on their profile. Meanwhile, AI is aiding drug discovery by screening data to predict the properties of a potential compound, generate ideas for novel compounds, and save time for researchers.
Most successful tech companies, e.g., Google and Facebook, are really AI companies – and that this is a shift pharma is likely to see too.
Ultimately, AI equity is important because AI is here to stay.
It really is going to become embedded in every facet of our lives. The transformative value it can create will be comparable to the difference between doing research pre-Google and post-Google, or communication pre-internet and post-internet. We’ll see massive benefits of AI for humanity as it becomes easier to create treatments for specific populations based on their needs, their behaviours, and their genetic makeup.