AI Regulatory Readiness: What to Do Now and What to Prove in 2027
A support assistant goes live with permission to read product manuals. Six months later it can search customer tickets, draft refunds and hand work to an agent with write access. The model name on the purchase order may be unchanged. The system the organisation is responsible for has changed considerably.
That is the practical problem behind AI regulatory readiness. A policy can set the boundaries. The harder work is showing what is deployed, which data and permissions it actually uses, what was tested and who can intervene when it goes wrong. The evidence has to survive changes of model, supplier and workflow.
Microsoft Security’s Regulatory Readiness for Trusted and Secure AI Adoption makes the case for one operating model across governance, data security, resilience and supplier risk. This article takes that premise into day-to-day decisions across commercial applications, managed model APIs, self-hosted systems and agents. It follows BlackTree’s October review of the AI Act, DORA and the CRA, which sets out the regulatory timetable in more detail.
The dates now shaping AI deployment
The EU’s AI Omnibus entered into force on 27 July 2026. The Commission’s implementation timetable reflects the enacted changes. Start with the system’s intended use and the organisation’s role. Those two decisions tell a team which parts of the timetable it needs to act on.
| Milestone | What teams need to check |
|---|---|
| 2 August 2025 | General-purpose AI model obligations began applying, with a separate transition for models already on the market. |
| 2 August 2026 | Article 50 transparency rules began applying. Map the applicable disclosure or marking duty to the provider or deployer. |
| 2 December 2026 | New prohibitions on systems generating specified non-consensual intimate material and child sexual abuse material begin. The Article 50(2) marking and detectability transition ends for qualifying providers whose synthetic-content systems were already on the market before 2 August 2026. |
| 2 August 2027 | The transition ends for general-purpose AI models placed on the market before 2 August 2025. |
| 2 December 2027 | The deferred high-risk requirements begin for specified Annex III uses, including certain employment and essential-services decisions. |
| 2 August 2028 | The deferred high-risk requirements begin for qualifying Article 6(1) systems associated with regulated products in Annex I. |
The Article 50 distinction is already consequential. A provider of a system intended to interact directly with people generally has to design it so people are informed they are interacting with AI, subject to the provision’s exceptions. Other parts address machine-readable marking of synthetic output and deployer disclosures for specified uses. A customer support chatbot, a system generating an image and a publisher disclosing a deepfake present different questions. The December transition concerns a particular provider obligation. Check the relevant paragraph of Article 50 against the service people actually encounter.
Sector duties run on their own clocks. Where the GDPR applies, map the purposes, lawful bases, retention and transfers for personal data. Assess whether a data protection impact assessment is required and whether provisions governing solely automated decisions with legal or similarly significant effects are engaged. A nominal human reviewer is a poor operational safeguard if they cannot understand an output, challenge it or change the result. The GDPR and the EDPB’s opinion on AI models are starting points, including for the case-specific question of whether a model can be regarded as anonymous.
Under DORA, a financial entity in scope remains responsible for its obligations when it uses an ICT third party. Whether a service supports a critical or important function affects the work on contracts, controls and exit. For entities covered by national laws implementing NIS2, supply-chain security, business continuity and incident handling may already apply to the business service in which AI is embedded. Where software is placed on the EU market, assess the Cyber Resilience Act separately. Its manufacturer reporting obligations began in September 2026, ahead of the main December 2027 requirements. A single incident can require different determinations and notifications under different regimes.
Keep a record of how the high-risk decision was made
Imagine a recruitment tool that extracts qualifications and then ranks applicants. The procurement brief calls it an administrative assistant. The hiring team uses the ranking to decide whose applications to open. A classification based on the brief alone would miss the part that can shape a recruitment outcome. Capture what the software does, how staff use it and what happens to people when its recommendation is wrong or ignored. Record the intended purpose with the provider and examine whether the deployment has moved outside it.
Article 6 sets out the conditions under which a system referred to in Annex III can fall outside the high-risk classification. The system must pose no significant risk to health, safety or fundamental rights and meet one of the specified conditions, such as performing a narrow procedural task. An Annex III system that profiles natural persons remains high-risk. Providers relying on the exception need to document the assessment before putting the system into service or placing it on the market, and register under Article 49(2). For a flagged workflow, record who reviewed the scope and which actual behaviour supported the conclusion.
Build the evidence for the deployer role as well. For in-scope high-risk systems, Article 26 covers matters such as using the system in line with instructions, assigning human oversight and monitoring its operation. Article 27 requires a fundamental rights impact assessment for specified deployers and uses. Its scope includes certain public bodies and public service providers as well as specified creditworthiness and life or health insurance systems. Check the exceptions and exact uses. The GDPR impact assessment has a separate trigger. The two reviews can reuse evidence about the same people, data and decisions.
Inventory the workflow that people actually use
Give each production workflow an owner and a record. Include internal copilots, AI enabled by default inside existing SaaS products, model APIs called by applications and locally run models. Include conventional prediction and ranking systems where their output matters to a decision. A list of chat subscriptions misses a fraud model that holds payments, a résumé screener embedded in recruitment software and an agent using a service account to write to a ticketing system.
Record the intended purpose, users, affected people and markets. Then draw the data flow from source to model and back. A complete record names the model or service version where available, the application binding it to the business process, the retrieval indexes, the integrations and the output destinations. Include who can grant access and how operators will identify the version that served a request. Where a vendor controls version changes, record what notice or other evidence is actually available.
Define the potential effect of failure before assigning a risk tier. An incorrect draft for a staff newsletter and an incorrect instruction in a clinical setting require different quality thresholds. A refund agent can move money even if its prose is accurate. An internal assistant can disclose another department’s files even if the supplier processes data exactly as contracted. Review impact on people, confidentiality, business continuity and the ability to undo an action.
Find systems through the records they leave
An annual staff survey will tell the team which assistants people remember using. Cross-check it with applications authorised through single sign-on, software renewals, API gateway traffic where available and the service identities registered in the integration platform. Ask owners of major SaaS products which AI features have become available under an existing subscription. An application can acquire new access to customer data without a separate invoice or a new account.
Start with evidence the organisation is allowed to collect. A gateway record may show that a team is calling a service. It cannot establish what happened to a prompt after the service accepted it. Use that signal to find the owner and verify the service configuration and contract. If infrastructure logs show unexpected external traffic, investigate the route and data categories before bringing raw prompts into a central review. A discovery programme needs a clear purpose, controlled access and its own retention decisions.
Track exceptions with an end date. A development team may need a short test of a new model using synthetic records. Capture the permitted users, dataset, network destinations and deletion date. If the team later wants production records or wider access, give that change an explicit production review. This gives teams a practical way to try a new option while keeping the record of approved production systems reliable.
Track who is supplying the model and who is supplying the system
A model provider can supply a general-purpose component. Another company can integrate it into a product. A customer can deploy that product in a particular setting. The same organisation may hold more than one role. Where a high-risk system is rebranded, substantially modified, or repurposed so it becomes high-risk, Article 25 can put provider obligations on another party under the conditions set out there. Capture that assessment when an integration changes where the output is used, particularly if it starts informing decisions about people.
Open weights change who can inspect and operate the model. Their availability alone tells a buyer little about the licence, training provenance, suitability or retained obligations. The Commission’s general-purpose AI guidance explains the conditions and limits of exemptions for qualifying free and open-source releases. The people operating a self-hosted deployment still need to assess what they have built and keep track of who updates and secures it.
Put the controls where the permissions sit
Hosted products make some changes on the customer’s behalf. Self-hosting moves patching and operational responsibility inside the organisation. Connecting a tool introduces a new route to data or action either way. The table translates those differences into evidence that engineering, procurement and risk teams can review together.
| Deployment | Evidence to retain |
|---|---|
| Hosted AI application | Tenant settings, approved data classes, connector scopes, retention and training-use terms, notice of material changes and the latest export or deletion test. |
| Managed model API | Model identifier or alias, configured region, routing and fallback rules, request-logging settings, quota behaviour and evaluation results for each approved route. |
| Self-hosted or open-weight model | Model and dependency provenance, licence, artifact hashes, runtime and network configuration, patch responsibility, tested version and rollback procedure. |
| AI agent with tools | Service identity, separate read and write permissions, allowed destinations, approval thresholds, action logs and proof that operators can revoke access. |
| Prediction or ranking system | Intended population, data and model versions, performance and drift measures, review or appeal process, and evidence that overrides reach the underlying decision. |
Test retrieval in the identity of the person asking
An assistant that searches company knowledge can expose access problems faster than a conventional search page. If it retrieves documents through a broad service account or an index that has lost document-level permissions, the model can repeat content the requester could never open directly. Check access at retrieval time, keep user or tenant boundaries intact, and test what happens after a staff move, a revoked grant or a deleted source document. Review copies held in embeddings, exports, caches and conversation histories alongside the original repository.
Make those tests specific. Seed a synthetic contract in a restricted folder and query for it from an unauthorised account. Confirm that the assistant cannot expose text, metadata or a usable download link. Repeat the test after the index refreshes, and check that authorised users still receive the expected result. Keep the pass and fail conditions with the workflow record.
Keep untrusted content away from authority to act
A prompt injection can arrive in a web page an agent reads, an attachment it summarises or a ticket it is asked to process. Design as though some of that content will attempt to direct the system. Put identity, destination and transaction checks in the application or tool gateway. Give an agent a bounded identity, validate tool arguments and require an explicit approval for consequential actions such as sending sensitive data outside the organisation, changing permissions or moving money.
Test the approval step against the real tool. A convincing screen is weak evidence if a retry, alternative connector or queued job can skip the check. Log attempted and completed actions separately so investigators can reconstruct the sequence. The UK National Cyber Security Centre’s secure AI development guidance puts these decisions into the wider lifecycle of design, development, deployment and operation.
Translate supplier promises into checks at the deployment boundary
Ask suppliers to walk one representative request through the actual service tier and region. Follow it through the public endpoint, orchestration service, model, moderation and telemetry. Find out what happens during support investigations, which subprocessors may receive data and what is kept for which purpose. For an API routed through a broker, include the broker and any fallback providers. Dig into the difference between the region that receives traffic and the locations where data may be processed or retained.
Put repeatable tests behind the answers. When a supplier says retention can be disabled, record the configuration and the terms applying to that setting. Distinguish request logs from account telemetry, persistent files, abuse-monitoring records and backups. When a vendor promises deletion on exit, run an export and deletion exercise on disposable data and ask what the confirmation covers. If the service exposes limited evidence, record the gap and decide what type of data the business can reasonably send.
For self-hosted deployments, review the path by which a model enters production. A model file is only one component. The inference server, tokenizer, drivers, containers and any loader that executes code can introduce vulnerabilities or operational dependencies. Establish where the components come from and which versions were tested together. Put the approved files in a controlled registry, keep hashes for the installed build and exercise rollback with the same interfaces the application uses. If the team cannot maintain that stack, its ability to bring processing in-house offers limited reassurance.
Make every material change reviewable
A supplier can update a hosted model overnight. A team can swap in a cheaper fallback, expand a retrieval index or install a connector at lunchtime. Each change can alter the behaviour people rely on. Define review triggers for model and prompt changes that affect output, new data classes, expanded permissions, new jurisdictions and new roles in decisions. A monthly meeting may catch slow changes. A release gate should catch the important ones before production expands.
Keep the test evidence with the release
Test representative tasks, failure cases and affected groups where relevant. Measure both the usefulness of outputs and the cost of failure. For a classifier, that could include false declines across relevant populations and changes in performance over time. For generative AI, record unsupported assertions, leakage attempts, and how the service handles contradictory evidence or declines to answer. For agents, measure unauthorised actions and test how retry, cancellation and timeouts behave.
Capture the version, configuration, test set, date, threshold, outcome and approver. Store enough to reproduce or interpret the result while limiting access to sensitive examples. Review an inconclusive result with the business owner before broadening the rollout. The NIST generative AI risk profile gives teams a common vocabulary for risks and evaluation. It is voluntary guidance. Local obligations and approved thresholds still need their own decisions.
Exercise human oversight on a busy day
Run evaluations under the conditions operators are likely to face. For a ranking system, look at which cases people actually review when a queue grows. For an assistant, give staff source material containing realistic contradictions and measure whether they can find and correct the wrong claims. For an agent, check that a reviewer sees the proposed target, action and material parameters before approving it. If a single screen asks someone to clear dozens of actions, observe how much of the underlying context they can still check.
Give reviewers a practical route to override a result. The action may require returning a case to the queue, notifying the affected person or correcting a record in another system. Test the full sequence with the people on duty and confirm the correction propagates. When a reviewer intervenes, record the reason in a form that lets the team distinguish poor model performance from unclear instructions or a broken integration.
Decide where to sample after release. Capture risk signals such as repeated refusals on legitimate tasks, unusual affected-group error rates, sharp changes in override rates or a jump in attempted tool actions. Give each alert a response owner and a route to limit or suspend the workflow. Teams can organise this work around the broader NIST AI Risk Management Framework across model types and draw on its generative AI profile where relevant.
Rehearse the day the AI service fails
Decide what staff will do if a hosted model becomes unavailable, produces unreliable results or changes its terms. Try the manual route or an approved alternative with the team that will actually use it. A fallback model needs its own review of data routing, retention and quality. An untested fallback can turn an availability incident into a confidentiality incident.
Practise revoking a connector, isolating an index and suspending a write-capable identity. Establish how staff will recover outstanding work and distinguish harmless retries from repeated transactions. Check that logs, alerts and supplier assistance let incident responders establish which data left, what action was taken, when the problem was detected and which people or customers are affected. Give legal, security and the service owner a shared incident record so each can make the required notification decision against the right trigger.
How the same service creates different responsibilities
Consider a bank that wants an assistant to help staff handle payment disputes. The bank buys a hosted model API, runs retrieval against its own case records and gives an agent access to the dispute platform. The supplier controls the model service. The bank controls the workflow, the data it sends and the actions available to the agent. If an integrator runs the connector, it introduces another operational dependency. The legal roles depend on the facts and the applicable rules, but the operating records can already show who can change each part.
Start with the boundary between drafting and execution. Staff can review a proposed message against the underlying case. Closing a dispute or issuing a refund presents a different risk. Put that action behind the same entitlement and approval controls used by staff, with a clear record of the decision and a way to reverse it where possible. Have privacy and AI governance teams reassess the use if the assistant starts scoring disputes or its recommendations determine the outcome in practice. Have the resilience team check how the bank would continue if the model API or connector stopped working.
A second business might run the same model weights on its own infrastructure to review publicly available engineering documents. It can impose tighter network and version controls, but also inherits the work of patching the runtime, evaluating updates and restoring service. If a later release adds unpublished design files, revisit the access model and the retention of copied or embedded content. The similarity of the underlying models says little about whether these two workflows should share an approval.
Walk through an incident with a supplier in the loop
Suppose a customer uploads a document that persuades the disputes agent to send a case summary to an unauthorised address. The bank stops outbound actions and rotates the agent’s credentials. Its next problem is to determine the extent of the disclosure. It needs the original case-event references, the recorded outbound destinations and the supplier’s account of how its service processed the request. In the exercise, ask the platform team to produce each record and time the supplier escalation through the channel named in the contract.
The same timeline supports separate decisions. The security team needs to check whether credentials were used elsewhere. Privacy needs to assess the personal data and who may be affected. The resilience team needs to establish what function was disrupted, what workaround was available and how the service was restored. The person coordinating the incident should capture when each team had enough information to make its determination. Each applicable reporting regime may measure timing from a different event.
Close the exercise by changing one condition and running it again. If separate outbound permissions would have stopped the transfer, demonstrate the block. If the provider took too long to supply logs, see whether an independent audit trail can answer the first questions. Record the revised configuration, the resulting test and the operational owner who can deploy or roll it back. That gives a later reviewer a test result they can trace to the improvement.
Use shared evidence across markets
Keep a core set of records for everyone, then map local obligations to it. A release test or permission export can support several reviews. The applicability decision should still record the entity, jurisdiction, role, legal basis and responsible owner. For example, the EU rules above may be directly relevant to one placement or deployment, while a US team also needs to check its sector and state rules. In Singapore, the PDPC advisory guidelines on AI recommendation and decision systems explain how existing personal data obligations can apply in the uses they cover. Neither a US risk framework nor regional guidance resolves a question of another market’s law.
In the US, New York’s financial regulator explains how covered entities should consider AI-related cyber exposures under existing Part 500 requirements. In the UK, the government’s AI Risk Management Toolkit offers operational guidance that teams can use alongside the rules applicable to their sector and deployment. Both can help a team ask better questions of a supplier, while the legal basis for each check belongs in the local assessment.
China’s interim measures for generative AI services distinguish services offered to the public in China from internal development and use where no such public service is provided. Teams serving that market should also assess the separate labelling measures for AI-generated and synthetic content. A product may be able to reuse parts of its labelling and testing implementation across markets. It still needs to demonstrate that the required disclosures and technical markings survive the actual publishing and download routes for each applicable rule.
Capture evidence where people already work. Procurement can attach the signed data terms and change-notification clauses. Engineering can link evaluation results to a release. Security can keep proof that access restrictions and monitoring fired. Privacy teams can link the assessment and retention decisions to the processing record. A common identifier for the workload connects those records so a reviewer can see which ones describe the configuration in use today.
Log enough to investigate and audit. Minimise what gets retained. A useful action log may contain an event ID, system and tool version, actor, destination, approval reference and outcome without preserving every prompt. Where richer content is needed, document retention, access and deletion, and periodically test those controls. An evidence repository can itself become a concentrated store of sensitive information. Map any applicable retention duty before choosing what to delete. For example, Article 26(6) ordinarily gives deployers a minimum of six months for automatically generated high-risk system logs under their control, subject to applicable EU or national law.
Build a short evidence trail that a second team can follow
Take one control, such as keeping restricted source documents out of answers to unauthorised users. The record needs to identify which workflow it governs, where access is enforced and who maintains it. Link the test run for the specific release and the approval to proceed. Record any exception with its owner, expiry and permitted users. Someone joining the company next month should be able to find the current evidence, understand the result and recognise when a change has invalidated it.
For a model migration, record which services will swap, which evaluations will be rerun and who will compare the results. An old acceptance decision can remain useful as a record of the risk tolerance and the original scope. A changed provider, model version, connector or dataset may require new evidence. Keeping those dependencies visible allows the team to retest the controls the migration actually touches. It also helps procurement estimate the time and data access needed to leave a supplier.
Use the same record in a board review, a customer assurance request and a regulatory examination where each has a legitimate need. A director may need trends and the decision on a critical exception. An auditor may need the test inputs, results and approval chain. A customer may need a narrower account of how its data is protected. Link to the same underlying release and evidence and share only the detail each recipient is entitled to receive.
What to finish before the next rollout
Choose one existing workflow with a meaningful impact on customers, staff or operations. Ask the service owner to show its current configuration and name the decisions it can influence. Ask security to demonstrate an access control and a revocation. Ask procurement or the platform team to show where the data goes and how supplier changes become visible. Ask risk and privacy to identify which requirements actually apply and where review is still open.
Then exercise a failure. Use a restricted test document or a harmless attempted outbound action and show that the application prevents it. Record what happened, who saw the alert, which record proves the result and how quickly an operator could intervene. Repeat the exercise when a materially different model, data source or permission changes that result.
Report progress in terms the board can act on. How many high-impact workflows have an owner and current tests? How long have critical exceptions been open? Which systems can still take consequential actions without an exercised approval or rollback? These measures make it easier to distinguish a growing deployment programme from one accumulating poorly understood exposure.
When a regulator, customer or board member asks how AI is governed, start with one live workflow. Show why it was approved, what it can access, which release was tested and who can stop it. If that account can be reconstructed from current records, the organisation has something it can defend and improve as the technology changes.


