Security of Critical Infrastructure Act 2018: Measures 5 and 17

Summary of recommendations

1. Make clear, in the Act or in guidance that tricking an AI-enabled system with hidden instructions placed in content it reads (prompt injection) is manipulation of the system, even though no password or firewall is breached. (Measure 5, Questions 23 and 24)
2. Define when a system is "materially uncontrolled": when it uses or tries to use, capabilities beyond those the responsible entity has documented it may exercise or when the human oversight and shut-off mechanisms specified for it fail or are bypassed. Guidance should frame the documentation as treating each AI system as a role in the entity's operating model: a defined delegation with boundaries and guardrails and a named person accountable for what goes into the system, what comes out of it and switching it off. (Measure 5, Question 24)
3. Publish worked examples of what is and is not, a cyber security incident for automated and AI systems and adopt two-tier reporting: unusual AI behaviour with no apparent impact triggers a documented internal triage window; a Part 2B report is required, within the existing timeframes, once manipulation is reasonably suspected or the Part 2B thresholds are met, without waiting for the investigation to finish. Guidance should also make explicit that an entity's risk management program is expected to record where and how AI is used within the critical function. (Measure 5, Question 25)
4. Define “material practical control” by what a provider is able to do, not what accounts it holds, so it covers control exercised through software: granting and revoking access, holding encryption keys, changing the software through updates, controlling identity and log-in systems, administering AI systems within the function and holding the only authoritative record of how the function's processes and delegations work. Make clear that this concept complements the influence or control test in section 8A rather than restating it, capturing operational and software-mediated control held by a party with no interest in the asset, and that any one of the listed capabilities is sufficient to constitute it. (Measure 17)
5. Require a relevant operator to notify the responsible entity when an AI or automated system it runs goes out of control and when a foreign government direction, legal process or licence condition threatens its ability to keep providing the service, so far as it can lawfully say so. (Measure 17)
6. Require operator registration to record who ultimately owns and controls the operator, where that owner is incorporated, which foreign laws could compel the operator to hand over data or assist a foreign government and where administrator-level access is exercised from. (Measure 17)

Conclusion

Kitbag supports the enactment of Measures 5 and 17 substantially as proposed, subject to the six recommendations set out in this submission, each of which can be implemented in the amendments themselves or at the guidance level. The recommendations share a single design principle. In an environment of automated execution and concentrated third-party operation, the control of a critical function rests on whether the responsible entity itself holds, in a governed form it owns and can audit, a current map of where AI operates within the function and on what information, the definition of what each system is authorised to do, a named human accountable for each automated role and able to interrupt it and the operating logic by which the function runs, held separately from the platforms and models that execute it. Entities that hold that layer can classify incidents, complete triage, satisfy Part 2B, replace an AI system that is compromised, withdrawn or no longer fit for purpose and survive the failure or direction of any single operator. Entities that have surrendered it to the platforms they depend on can do none of these things and no notification duty can substitute. Each recommendation gives that principle a concrete statutory expression.