AI adoption continues to accelerate across health and social care. The Care Quality Commission (CQC) supports the use of innovative technologies, including AI, that improves outcomes.
However, the use of AI presents both significant opportunities and risks. For the CQC, ensuring AI contributes to safe, equitable and person-centred care is central to its role as a regulator. As AI adoption grows, care providers must be able to demonstrate that any AI use aligns with existing guidelines and future regulatory requirements within the CQC’s next framework.
At CACI, we recognise both the benefits and risks that AI brings to social care. That is why our innovative care management software, Certa, was built to support providers in delivering high-quality care while maintaining essential human oversight. The goal is improving outcomes for people using services while ensuring care remains person-centred, with AI playing a supporting role.
What is the CQC’s position on AI?
The CQC encourages innovation, but providers must remain accountable for ensuring AI is used to deliver “safe, equitable and person-centred care”.
Among the benefits outlined by the CQC are:
- Clinical efficiency through faster medical image diagnoses and optimised workflows
- Reduced administrative tasks
- Enhanced resource management
- Earlier identification and monitoring of higher-risk, vulnerable patients.
As AI adoption grows, care providers should not only focus on operational efficiencies, but on demonstrating how these technologies improve outcomes and support equitable access to care.
The CQC also highlighted several anticipated risks, including:
- Increased health inequalities resulting from access and poorer outcomes
- Privacy, data protection and confidentiality
- Safety concerns arising from misunderstanding or over-reliance on AI-generated outputs
- AI hallucinations presenting untrue information as fact
- Staff becoming de-skilled
- Reduced trust, as AI can make mistakes that lead to misdiagnoses
- De-personalised health and social care, with staff lacking trust, confidence or competence in AI use
- Unclear accountability and liability
- Environmental concerns associated with the growing carbon footprint of AI.
Care providers must remain vigilant, as different AI technologies present different risks depending on how and where they are used, with some risks being harder to identify and quantify until they have already affected service users.
The eleven principles care providers need to understand
The CQC has outlined 11 principles to help care providers adopt AI in a way that promotes outcomes while adhering to regulations. These principles include:
- Support, not replace, professional judgement, with AI intended to enhance human decision-making, not replace it
- Human oversight, continuously monitoring and evaluating AI outputs and processes
- Transparency and choice, where service users are equipped with the necessary information to make informed decision-making when it comes to their care
- Safety and reliability, with providers ensuring AI leads to safe care and equitable outcomes for users
- Data security, ensuring technology is resilient to cyber-attacks and information storage and processing is secure and GDPR compliant
- Fairness and bias mitigation, where AI should accelerate equity of care in terms of access and outcomes and mitigate bias
- Sufficient staff AI-readiness training to instill confidence in using the technology and assess its viability for integration into existing pathways
- Governance and risk management, including establishing effective mechanisms and systems such as risk assessments and reporting processes
- Data Protection Impact Assessment (DPIA) to understand, document and mitigate risks
- Accountability, where clear mechanisms are established to rectify any issues caused by AI
- Procurement, with AI tools procured in line with relevant regulatory standards.
What does this mean day-to-day?
In practice, care providers should focus on where AI can support existing processes instead of replacing them, from helping staff find information more quickly and reducing administrative burdens to improving monitoring and reporting and strengthening evidence gathering for inspection readiness.
Any efficiencies gained through AI should be balanced with appropriate human oversight, governance and evaluation. Care providers remain responsible for understanding how AI is used, monitoring its outputs and ensuring it delivers positive outcomes for users.
Applying the CQC’s principles with Certa
With Certa, you will spend less time on administrative tasks and more time delivering quality care.
Human oversight remains central. Through Certa’s digital care records, care plans, scheduling tools and real-time operational visibility, schedulers and administrators can securely and effectively update records and match clients and care workers while reacting to exceptions and alerts in an informed, proactive manner. Assessments, care notes and medication records are easily recorded and updated in Certa, allowing all interactions to be personal and informed; not having to rely on paper records.
Certa’s upcoming AI Form Assist feature is designed to help reduce administrative burdens while keeping human oversight at the centre of care delivery. By securely listening to and transcribing conversations, this feature will help assessors and support workers by pre-filling electronic forms, generating care notes and care plan suggestions based on the information captured. This will enable assessors to focus fully on client conversations during assessments and ensure support workers can record interactions more efficiently while maintaining their attention on the person receiving care.
AI Form Assist is being developed with governance and clinical safety in mind., with the system only providing suggestions around For clinically sensitive questions or areas requiring professional judgement, the system will only provide suggestions rather than automatically populating responses from the conversation transcript. This will keep care professionals responsible for reviewing, validating and completing information rather than relying on AI to make decisions that could impact the safety of a vulnerable adult or child.
Certa considers clinical safety and governance, instilling confidence in having what you need for the CQC with an SAF-aligned, real-time reporting suite that meets quality statements.
AI readiness checklist for care providers
- What problem are we solving?
- Who is accountable?
- How will staff be trained?
- How is AI explained to service users?
- What data is processed?
- Have we completed a DPIA?
- How is bias managed?
- How are errors monitored?
- Can governance be evidenced to CQC?
Ensuring AI in social care delivers optimal outcomes
AI should be adopted responsibly, transparently and with robust oversight. Organisations that demonstrate governance and positive outcomes will be best placed to benefit. By working with CACI, you can become one of them.
With over 25 years of experience in the sector, our service desk and web support system can help you successfully adopt AI and remain secure and compliant in your usage. Our latest AI capabilities within Certa are designed specifically for the realities of community care, interpreting operational evidence already held in Certa– care records, medication data, staffing, incidents– and surfacing a continuously updated picture of where a service is performing well and where risks are emerging.
The developments are running on a secure, managed infrastructure with appropriate safeguards, meaning AI assists your team’s judgement rather than replacing it.
Book a Certa demo to find out more.
