When organizations run open-weight models in house, they often underestimate the costs and staffing requirements involved. According to Prasad Tharippala, Field CISO at Versa, running the model in your own environment can improve control and data residency, but it does not automatically make the deployment more secure.
Tharippala explains that the biggest underestimate is that running the model is only one part of the problem. The real operational cost comes from everything around the model, including GPU infrastructure, networking, storage, power and cooling, capacity planning, orchestration, model updates, monitoring, security controls, data governance, audit evidence, and ongoing optimization.
Licensing and compliance review is another cost that rarely makes it into the budget. Open weight does not mean unrestricted, and many open weight licenses carry usage restrictions. Regulations like the EU AI Act add obligations for larger models, and someone has to review that before deployment.
Related: Interpol busts West African rings, reveals new threat
There is also a skills gap, as organizations need people who understand AI/ML infrastructure as well as security, networking, observability, and production operations. This can lead to significant delays, and in some cases, the project never delivers the expected business value.
Agent red-teaming needs to go beyond traditional application penetration testing. Tharippala recommends testing to see what the agent can do after it is compromised or manipulated.
For teams with limited time, Tharippala recommends prioritizing first, establishing visibility and inventory, then reducing the blast radius, and finally, continuously testing and monitoring. This approach can help organizations secure their AI agents and prevent potential security incidents.
When determining whether an agent platform is secure, Tharippala recommends asking about its compromise response, access control, action limitations, and audit capabilities. They should also consider how to govern the agent throughout its lifecycle and what security responsibilities belong to them and what responsibilities belong to the platform provider.
Related: Fake Bank Sites Go Dormant to Dodge Scanners
By prioritizing security, organizations can ensure that their AI agents are secure and compliant with regulations. This is particularly important for organizations that need to consider data sensitivity, sovereignty requirements, latency needs, workload volume, the skills they have in house, and regulatory or compliance requirements.
In terms of security, Tharippala notes that a strategy, disciplined planning, and the right skills can make a big difference. However, the decision to run open-weight models in house should not come down to build versus buy alone, but should be driven by data sensitivity, sovereignty requirements, latency needs, workload volume, the skills they have in house, and regulatory or compliance requirements.
It is a complex process.
