In the first blog of this series, we explored how an organisation’s biggest barrier to AI adoption is not the model itself, but inconsistent and unreliable data. The question has now shifted: how can AI be trusted by regulators, leadership and users alike? Answering this requires a focus on ethics, explainability and governance.
Adopting AI successfully is not just about building something that works, but building something people trust. AI can create real value, but it also introduces risks. Bias, unclear decision-making and misuse of data can all weaken confidence if mishandled. That is why fairness, transparency, accountability and strong data governance must be built in from the start, not added later as a fix.
Anchoring AI in purpose
It is impossible to build a trustworthy AI system without first understanding its purpose. An organisation must be clear on what the system is designed to do, who it serves and what outcomes it is intended to deliver. This clarity shapes everything: what data is collected, how it is labelled, which models are used and how outputs are evaluated.
For example, if you are trying to understand causes of prostate cancer, there is little value in having a 50/50 split universe of males and females, other factors will be more important to be represented in the dataset. Or, if you want to use demographic data to predict insurance pricing premiums, you need to have a greater focus on the history of the data you are using and what historic biases may be in play. Being clear on the purpose will dictate how you behave ethically for the audience you are supporting.
Ethics in AI
Regulation is becoming a bigger part of the picture. Different regions are setting their own rules, with the EU AI Act being one of the most developed examples, using a risk-based approach to govern how AI is used. Regulation alone cannot tell you how to behave or how to build your system, however, and organisations working across countries must keep up with these changes. Getting it wrong is not just a compliance issue: it can affect reputation, finances and customer trust.
In every scenario, ‘risk’ is in the eye of the beholder. You must apply ethics when determining how and where harm could occur, even when the law is not explicit. This comes back to purpose, because what works for one audience may not for another. For example, some may view using browsing history to better understand product preferences as completely acceptable. Using the same data to predict political allegiances and feed the algorithm accordingly, however, is a very different, more sensitive application.
The lens of ‘harm’ to place boundaries around AI system capabilities (where can harm be caused and to which audiences) must be considered:
- Misinformation
- Privacy breaches
- Discrimination and dignity infringement
- Exclusion from participation.
These impact health and wellbeing, human creativity or originality and create a power imbalance between the system user and the recipient.
Ethics in AI ensures systems behave in ways that apply the ethical core values. When principles like fairness, transparency and accountability are grounded in what your organisation already stands for, such as integrity, inclusivity and responsibility, they stop feeling new or separate. They become an extension of “Doing the Right Thing”, as we say at CACI.
This connection matters. People are more likely to trust AI when it behaves in ways that feel consistent with what they already expect from an organisation. If there is a mismatch, such as if your organisation promotes equality but its AI produces biased results, that gap can quickly undermine confidence.
Grounding AI in existing values makes it easier to manage and scale. It creates consistency between technology and culture, which in turn supports better governance and more responsible innovation.
These ethical considerations become most tangible when applied to fairness in AI systems.
Fairness and bias
Fairness sounds straightforward, but in practice, it is one of the hardest parts of designing AI systems. The difficulty comes from the fact that fairness can be defined in many ways.
One approach focuses on individuals; similar people should be treated in similar ways based on relevant factors. Another looks at groups, where outcomes should be balanced across different demographics. The challenge is that these approaches often conflict.
Take a lending system as an example. If decisions are based purely on financial risk, approval rates might differ between groups because of patterns in the data. That could satisfy individual fairness but still produce unequal outcomes. Trying to correct for those differences might improve group fairness, but it can mean treating similar individuals differently.
Bias adds another layer of complexity. If historical data contains bias, which it often does, AI systems can learn and repeat those patterns. This can reinforce existing inequalities rather than reduce them. Once there is clarity around an AI system’s purpose and its intended outcomes, however, these issues can be tested and monitored.
There is no single correct solution. Instead, organisations must make informed trade-offs, such as adjusting thresholds or introducing constraints to reduce disparities without losing too much accuracy. Evaluating these trade-offs against the system’s intended purpose and real-world outcomes will be key, as opposed to gauging a model’s fairness based on performance alone.
eBook
The AI playbook: How to design, deploy & deliver AI value
This playbook brings a practical, real-world perspective on how to move beyond experimentation and make AI deliver for your organisation’s people and ways of working.
Transparency and explainability
For AI to be trusted, people must understand how it works, at least to a reasonable degree.
Transparency is about the overall system: how it is built, what data it uses and how decisions are made. Explainability clarifies why a particular decision was reached.
When both are in place, decisions feel less like a black box. Users and stakeholders can see what drives outcomes, which makes it easier to trust them. It also makes it easier to spot problems: if something does not look right, there is a way to investigate and understand why.
These ideas are also closely linked to fairness. When decisions are visible and explainable, it is easier to identify patterns that might be unfair or biased. Without that visibility, issues can go unnoticed.
They also support continuous improvement. If decisions can be reviewed and challenged, systems can be refined over time rather than remaining static.
However, understanding AI decisions is not enough on its own: you must also take responsibility for them.
Accountability and governance
Understanding how a system behaves is only part of the picture. Accountability is equally critical. It means clearly defining ownership, responsibility and processes for addressing issues when they arise.
If an AI system produces a harmful or unfair result, you should have a clear way to investigate and mitigate it. This relies on good governance, including audit trails, monitoring and clearly assigned roles.
It also connects directly to regulation and security. Laws such as GDPR require organisations to be transparent about how they use personal data and, in some cases, to explain automated decisions. Additionally, strong cybersecurity is essential to protect data and ensure systems are not compromised.
When governance, accountability and security are treated as part of the same foundation, AI systems are far more likely to be reliable and compliant.
Data governance and risk
Good data is the foundation of any effective AI system. If data is poor quality, incomplete or biased, the outcomes will reflect that. Managing data properly is just as important as building the model itself.
The level of control needed depends on the level of risk. Using general, low-impact data requires less oversight than using sensitive or personal information. When data involves things like personal identity or behaviour, the stakes are higher. Misuse can lead to privacy issues, harm to individuals and loss of trust.
Risk-based approaches like the EU AI Act are helpful. Systems are grouped based on their potential impact and higher-risk use cases come with stricter requirements around governance, documentation and oversight. While this can add complexity, it helps ensure that more sensitive applications are handled with the care they require.
It is also important to think beyond the initial deployment. AI systems evolve over time and changes can introduce new risks. Maintaining control over how systems develop and making sure they stay focused on their original purpose helps avoid unnecessary data use and prevents scope from drifting in ways that increase risk.
How CACI can help you build trusted AI
Building trust in AI is not a single step, it is the result of strong data foundations, clear purpose and consistent governance applied over time.
Trust is earned by ensuring data is reliable, decisions are explainable and systems behave in line with both regulatory expectations and organisational values. This is particularly critical in complex, regulated environments, where the consequences of getting it wrong extend beyond compliance to reputation and customer confidence.
At CACI, we recognise that trust cannot be retrofitted. It must be designed into the data, the models and the processes that surround them. Contact us to find out how we can help organisations like yours embed ethics, explainability and governance into your data foundations from the outset to enable AI that is not only effective, but trusted at scale.
In the next blog, we will explore how data cataloguing and labelling build on these foundations to further strengthen trust and shape AI-driven outcomes.
