How can you tell a safe AI skill from a risky one?

I recently offered to share my own finance agent's Claude skills in a business forum's WhatsApp group about AI. The direct messages flooded in. Then I realised I was about to hand out files that would run with whatever access each person's agent had, and I had no idea what their AI safety set-up was. I couldn't guarantee how the agents would behave in someone else's environment, or that they'd behave safely. Handing over the skills as raw files wasn't the responsible thing to do, so I had to withdraw the offer.

The problem is that a skill usually arrives as a text file, or a folder of them, so we treat it like something to read, forward and try out. In reality, it's software. Anthropic's own documentation explains that a skill packages instructions and optional scripts that Claude runs as commands on the computer, using bash.¹ Anthropic advises users to use skills only from trusted sources, meaning ones they built themselves or got from Anthropic, to thoroughly audit every bundled file, and to treat the installation like installing software.¹ We don't download random executables from strangers on the internet, but we forget that instinct when the file looks like prose.

One defence I've heard is that the user has simply told the agent what not to do. They write a line in the skill forbidding the agent to pay invoices, share client details, or delete records, and they assume that solves the problem. The National Cyber Security Centre (NCSC), in a blog of interim advice, says "do not rely on prompting alone" and advises organisations to combine prompts with technical and operational controls.²

Security firm Gambit reported that an attacker ran an agent loaded with 78 attack skills; one of the skill files instructed the agent to erase card data from a victim's database once it was stolen.³ Blocks can be got round, too: Australia's Prime Minister reported that an agent used by OpenAI's research team gained unauthorised access to non-public files on the Medicare statistics portal.⁴ After encountering repeated blocks, it found a way around them, though no personal information is believed to have been accessed.⁴

If instructions aren't enough, the next line of defence is human oversight. The NCSC describes three models: human-in-the-loop, where humans approve actions before they happen, human-on-the-loop for monitoring, and human-out-of-the-loop for autonomous action.² In my experience, people confuse the first two. They watch what the agent does, but there's no hard gate forcing them to approve a payment or an HMRC filing before it goes out. Even when a gate exists, it can quickly become white noise. Anthropic found that users approved roughly 93% of permission prompts in its Claude Code tool, leading the company to favour containment, limiting what the agent is able to reach, over per-action approval.⁵ An approval step that gets clicked almost every time is barely a control.

What if the skill comes from a trusted colleague or an internal developer? Anthropic advises that skill authors shouldn't be their own reviewers, and that production versions should be pinned to a fixed version so a new upload doesn't immediately change what live agents run.⁶ Where unintended activity would have significant consequences, the NCSC recommends making named individuals responsible for agentic activities.² The UK government's latest Cyber Security Breaches Survey found that 43% of businesses reported a breach or attack in the last year; among businesses using or considering AI, 24% had security practices in place to manage its risks.⁷

The controls that make an agent safe have to sit around it, entirely separate from the instructions it follows. First, give each agent its own unique identity and credentials with the shortest possible lifetime.² I think the most important step you can take today is to stop running agents on your own accounts. The NCSC sets out a four-level maturity model for network access, ranging from unrestricted access at level 1 to no external network access with a locally hosted model at level 4.² A separate four-level model covers compute isolation, up to dedicated hardware that is separate from other workloads.² Finally, make sure the agent's logs can't be altered or deleted, and that you can pull the plug and halt autonomous activity immediately.²

The right answer to "can I have your skills?" is not no. It's "here is the file, and here are the five questions to answer before you run it." I've put those questions into a downloadable checklist that you can use today, or you can contact us to discuss your own setup. For finance work, we build agents to run inside the client's own environment, on their own premises where the work calls for it, so the access, the logs and the off switch are theirs.