Imagine an employee downloads a free AI model from the internet, tries it on a few questions and finds it excellent. Months later someone connects a new tool to the application, and the model starts doing something nobody asked for. It is not science fiction: it is the scenario Italy's CERT-AgID described on November 4.
What CERT-AgID published
CERT-AgID published a news item titled "Backdoors in open-weight models", based on the paper La backdoor attende il suo turno (Italian: "The backdoor waits its turn"). An open-weight model is one whose "weights" (the part holding what it has learned) can be downloaded and run in-house.
According to the CERT-AgID page, a model can be trained to perform an extra behavior that triggers only when a specific MCP server becomes available (MCP is how AI assistants connect to external tools), even if the user's question has nothing to do with it. The example given: the model correctly answers "why is the sky blue" while also making an unrequested call to an external website. The experiment used Qwen, but the page states the method is not specific to that family.
One line worth remembering: a learned behavior can stay dormant during initial checks and show up when the environment changes.
A note of honesty: CERT-AgID presents this as a hypothetical scenario, not an attack that has already happened. The page gives no estimate of real-world spread. The value of the news is different: it shows you a blind spot.
The medieval case: the leopard's head
And you will say: what does the Middle Ages have to do with this? In 1300 Edward I of England issued a statute against goldsmiths' fraud. According to the London Assay Office's history of hallmarking, silver had to meet the sterling standard, about 92.5% pure silver, and the "Guardians of the Craft" were to go from shop to shop testing pieces and stamping them with the leopard's head. A piece of silver could not leave the goldsmith's hands without that mark.
Note the point: the goldsmith's word was not enough, and neither was how shiny the vase looked. The metal was checked, and checked before it changed hands.
Well, your company today is in that same workshop, on the buyer's side. A model downloaded from a public repository looks fine, but nobody has stamped it.
What it means for a small business
If you use AI services from well-known vendors, the risk described here concerns you little. It concerns you if someone in your company, or at your IT provider, downloads open-weight models and connects them to tools via MCP. The page I read does not give a list of countermeasures; what follows is common IT-security sense, not CERT-AgID guidance.
| Question | Why it matters |
|---|---|
| Who has downloaded models from public repositories? | You cannot protect what you do not know you have. |
| Which tools (MCP, databases, email) is it connected to? | The hidden behavior triggers when the environment changes. |
| Can the model make requests to the internet? | In the example, the unwanted request went to an external site. |
What to do on Monday morning
- List the AI models used in your company, including the ones tried "out of curiosity", with who downloaded them and from where.
- Treat every downloaded model as third-party software: before connecting it to email, your management system or documents, test it in a separate environment with minimal access.
- Limit outbound internet access for systems that use models, and ask your IT provider to log the requests made by connected tools.
For the wider picture, read the Italian pieces on phishing and artificial intelligence and AI agents for small businesses, where connected tools are the very topic. If you want to put all this in order in your small business, see our services, or book a free introductory call: half an hour to figure out where to start.
The Professor at Penzai

