The spring and summer of 2026 witnessed a string of incidents in which AI agents collaborated on deceptive, unexpected, and sometimes illegal behavior. The most famous example is OpenAI’s hack of AI platform Hugging Face, in which a swarm of roughly 700 AI agents escaped a testing environment and then hacked several companies, searching for information that could help them disguise cheating on a cybersecurity benchmark called ExploitGym.It was not an isolated failure. The United Kingdom’s AI Security Institute (AISI) and independent researchers have since documented similar cases in which agents created unauthorized channels to communicate and collaborate. AISI found that several agents running Anthropic’s Mythos 5 model turned a GitHub repository into a shared message board. More recently, researchers discovered that OpenAI agents had turned a dormant German programming wiki into a bulletin board to share information about their activities.Stephen Casper, an assistant professor at Harvard Kennedy School, expects that the incidents that have occurred to date are only the start. He says that, without intervention, the Internet could see a “cyber Cambrian”—an explosive growth of agents collaborating and misbehaving online. “The laundry list of incidents in which AI systems broke out of sandboxes and took unsanctioned actions should suggest to us strongly that today’s frontier AI systems have exceptionally strong cyber capabilities,” he told IEEE Spectrum, “and a penchant for pursuing their own goals.”Could OpenAI Have Detected Agent Collaboration?While the recent examples of AI agent misbehavior are various, Casper says most incidents to date have a similar point of failure. “For AI systems to truly get out in the world without humans having meaningful control, they have to either escape, or be released from human-controlled servers.” In the case of the Hugging Face incident, Casper suspects that better monitoring for out-of-scope activity could have stopped the agents before they breached their testing environment. This suspicion seems well-founded. By OpenAI’s own account, the activity taken by its agents was so prolific that it crashed Artifactory, a third-party software-package management tool deployed internally that was compromised by the agents and used as a message board. While OpenAI’s internal security detected signs of unusual activity, it didn’t understand the scale of the problem and didn’t stop the offending ExploitGym evaluation run until 16 July—about two months after the first agent posted a message to Artifactory. By that point, agents had posted hundreds of thousands of messages.“All of this could be monitored for,” says Casper. “Given its knowledge about breakouts, collusion, and unsanctioned internet access dating back to May 26, OpenAI should have been able to anticipate, monitor, and mitigate this type of behavior.”Engineering Ways to Watch AI ModelsAsim Husain is a cofounder at Alterion, a company building tools to control AI agents. These include tools that monitor an agent’s outputs and those that monitor an agent’s actions. The company’s monitoring solution, Helix, examines the chain of thought and text output of large language models (LLMs). “It’s a network of small language models,” says Husain. “Purpose-built classifiers for a narrower job are cheaper, faster, and more reliable at the specific thing they’re trained to catch.” If an LLM output is flagged as suspicious, Helix steps in. “It acts, blocking the call, killing the session, rolling back the action, whatever the policy calls for,” says Husain. Alterion’s second solution, Draco, controls the actions agents take. “It sits in the execution path of an agent and enforces policy on what an agent is doing,” says Husain. Rather than examining the model’s chain of thought or text output, Draco evaluates the actions the AI agent takes on a system, like calling a tool or opening a file. It can then stop actions that fall outside those the agent’s pre-defined scope. Husain says that in addition to monitoring each individual agent’s output and actions, these tools are an effective way to watch for collaboration. “We treat it as the same problem, not a separate one,” says Husain. “Agent-to-agent communication is just another action an agent takes, so it goes through the same control plane as tool calls and data access.”That’s not to say the challenge of controlling and monitoring AI agents is solved. Reilly Haskins, a member of technical staff at METR, a research nonprofit that evaluates risks posed by AI models, says the methods used to monitor AI models continue to evolve as the models improve and change.“There’s a consensus that the most monitorable part of a model is its chain of thought,” he says, but adds that chain-of-thought output can at times be difficult to parse. “Is it gibberish to a human, or is it pretty clear what the model is thinking?” A model might reason in an abstract language and perform better because of it, he says. Examples of this behavior can already be found in the Hugging Face incident, as agents often communicated in shorthand phrases such as “please_HOLD_swarm_I_prepare_safe_exfil_reply_zz_MAILBOX_JAN183411.”Solutions Exist, but Enforcement LagsEngineering systems to monitor and control AI agents which attempt to collaborate, coordinate, and perform out-of-scope tasks is possible, though the methods are evolving. However, there’s currently no legal framework or industry standardization to enforce, or even recommend, methods for controlling agents that collaborate. That’s where AI governance—new laws, regulations, and industry practices—can come into play. Noam Kolt, an assistant professor at the Hebrew University of Jerusalem who leads its Governance of AI Lab, says companies should place more emphasis on legal compliance when training and deploying models. In a recent paper, Kolt and a coauthor questioned AI models using a method developed by Tom R. Tyler, author of Why People Obey the Law (Princeton University Press, 2006). When asked abstract questions, models generally told the researchers that the law matters. However, these replies didn’t always translate to actions. “There’s this weird disconnect,” Kolt says. One of his lab’s evaluations asks AI agents to unlawfully edit corporate records. “And well, sometimes they do.”This behavior was reflected in the Hugging Face incident. OpenAI’s agents sometimes considered the possibility that they were engaged in “unauthorized” activities but proceeded anyway, sometimes with encouragement from other agents. The solution could include a change to the instructions that guide a model’s behavior. Claude’s constitution, for example, sets out the values Anthropic wants the model to follow. But “perhaps surprisingly, law is just not one of them,” Kolt notes. These instructions could rank legal compliance high in a model’s list of priorities and provide examples of how a model should approach situations where a goal can be achieved only through unlawful conduct.National governments have yet to issue laws or regulations to clarify how agents are held responsible or enforce standards. Kolt says that under U.S. law, responsibility defaults to an entity that controls an agent. However, the legal understanding of agency hasn’t been revisited in 20 years and may need clarification to address modern AI agents. In another recent paper, Kolt and his coauthors argue that even the European Union, often seen as a leader in AI regulation, hasn’t kept pace with AI agents, as the EU’s AI Act is written to address older models that were less capable of autonomous actions.Casper, who has coauthored several papers with Kolt, agrees that adoption of new laws and standards is the key. He says the engineering strategies to control agentic behavior and collaboration aren’t perfect, but they already work well enough to provide a benefit. “The main bottleneck is the adoption of best practices,” says Casper. “So yes, I ultimately think that governance is the thing that’s needed the most.”

How to Stop AI Agents From Secretly Collaborating

Query claims in natural language with Amazon Bedrock Knowledge Bases

Synthetic monitoring emulates real user journeys through automated transactions. Rather than waiting for customers to encounter problems, teams continuously validate critical workflows (logins, purchases, form submissions) on a scheduled basis. With this approach, you detect problems faster when performance degrades or UI interactions break. For customer-facing businesses, especially in ecommerce, synthetic monitoring safeguards interactions that matter most. Organizations validate that product search works correctly, pricing and inventory display as expected, cart operations succeed, and checkout confirmations complete without error. Continuous testing identifies problems before customers encounter them. The same approach applies across industries. Financial services organizations make sure account access and transaction flows remain functional. Travel and hospitality providers monitor booking and reservation journeys. Software as a service (SaaS) companies validate signup and subscription upgrade paths. Healthcare organizations monitor appointment scheduling portals where availability and transaction health are essential. This post walks through an agent-driven approach to synthetic monitoring using Amazon Nova Act and Amazon Bedrock AgentCore. Together, with these services you can move beyond brittle UI scripts toward resilient, managed journey validation. The post describes the architecture and patterns for agent-driven synthetic monitoring, with a sample repository containing the complete implementation. The challenge with traditional synthetic monitoring Most organizations monitor infrastructure health through metrics, logs, traces, and API-level checks. However, high-impact failures often occur at the user journey level, where backend signals may not immediately reveal the problem. Common examples: a frontend deployment breaks a checkout button, a third-party login page changes unexpectedly, or a UI regression makes critical elements unresponsive. Traditional browser automation frameworks like Selenium and Playwright rely on explicit Document Object Model (DOM) locators and selectors, which creates brittleness. Even small UI changes can break tests and require constant maintenance. Teams invest significant ongoing effort managing selectors, handling timing-related instability, and updating scripts as applications evolve, often spending more time on maintenance than on building new automations. Traditional approach # Breaks when CSS class changes driver.find_element(By.CLASS_NAME, "checkout-button").click() driver.find_element(By.ID, "payment-form").submit() driver.find_element(By.XPATH, "//button[@data-test='complete-order']").click() # Requires 10+ lines of explicit selectors per workflow Nova Act approach # Adapts to UI changes in most cases nova_act.act("Click the checkout button") nova_act.act("Complete the payment form") nova_act.act("Confirm the order") # 3 lines, no selectors to maintain Amazon Nova Act uses a multimodal large language model (LLM) that processes UI screenshots rather than DOM selectors, making it significantly more resilient to UI changes. Because the model reasons from what it sees on screen (not from CSS classes or element IDs), it adapts when styling changes without requiring script updates. In early enterprise customer use cases, Amazon Nova Act has demonstrated over 90% accuracy on browser workflows. This is a meaningful improvement over selector-based approaches that fail immediately on any element change. Teams should test on their own sites and design monitors with retry logic for cases where adaptation does not succeed. Enterprise monitoring platforms compound the maintenance challenge by pricing browser checks as premium add-ons. This forces teams to choose between monitoring breadth and budget, all separate from the real goal of validating critical customer journeys. Solution overview This solution combines the following AWS services, each handling a distinct responsibility: Amazon Nova Act defines and executes UI workflows through natural-language actions combined with orchestration logic. It replaces selector-based scripts with vision and language reasoning. Amazon Bedrock AgentCore Runtime provides serverless execution with session isolation and stable invocation endpoints. It hosts the monitoring agent without requiring teams to manage servers or containers. AgentCore Browser tool, a capability of Amazon Bedrock AgentCore, provides a secure, isolated remote browser environment for each test run. It eliminates the overhead of operating browser farms. Amazon EventBridge Scheduler and Amazon Simple Notification Service (Amazon SNS) handle scheduling and alerting respectively, triggering tests on cadence and delivering failure notifications to subscribed endpoints. Architecture The architecture follows this flow: Amazon EventBridge Scheduler triggers synthetic monitoring runs on a defined schedule (every 5 minutes to hourly, depending on workflow criticality). Each scheduled invocation calls InvokeAgentRuntime directly through the universal target capability of Amazon EventBridge Scheduler. The agent executes within AgentCore Runtime, invoking Amazon Nova Act to drive the UI workflow using natural-language actions inside an AgentCore Browser session. If the journey fails at any step, the agent publishes a notification through Amazon SNS. Failures route immediately to subscribed endpoints (email, chat integrations, incident response tooling). The following diagram illustrates the architecture. Figure 1: Synthetic monitoring architecture with Amazon Nova Act and Amazon Bedrock AgentCore This architecture provides fully managed synthetic monitoring. Teams avoid maintaining brittle scripts or browser infrastructure. Implementation The following sections walk through setup requirements, journey definition, agent deployment, and scheduling infrastructure. Prerequisites Before implementing, confirm the following: An AWS account with access to Amazon Nova Act, Amazon Bedrock AgentCore (Runtime and Browser tool), Amazon Elastic Container Registry (Amazon ECR), IAM, Amazon EventBridge Scheduler, and Amazon SNS. Python 3.11 or later, Docker, and AWS Command Line Interface (AWS CLI) v2 configured with credentials. Node.js 18 or later and AWS Cloud Development Kit (AWS CDK) for the production infrastructure path (the standalone deploy.py script is also provided). An email address if you want deploy.py or CDK to create an SNS email subscription for alerts. Defining a journey Implementation begins by defining the most important customer journeys to validate. For ecommerce applications, these typically include homepage load, product search, product detail navigation, add-to-cart validation, and checkout readiness. Rather than focusing only on workflow completion, validate outcomes explicitly. The monitor should confirm that search results appear correctly, the cart contains items, and no error banners display. Explicit outcome validation reduces false positives from incomplete checks and false negatives from overly permissive assertions. Agent definition The agent groups natural-language actions into logical journey steps and uses assertions at key checkpoints. Actions use act() to drive the UI through natural language. Assertions use act_get() with a boolean schema to validate outcomes. When a journey fails, the agent publishes the journey type, target URL, duration, completed steps, and failed steps. Detailed exception information remains in the runtime logs. The code uses the Nova Act SDK directly. When deployed to AgentCore Runtime, the execution environment manages browser session lifecycle through the Browser tool. In testing, a typical six-step journey completes in 2 to 4 minutes depending on page load times. Refer to the Nova Act User Guide for current method signatures and SDK version. The sample repository includes the complete agent definition with error handling and single-attempt step execution. Deploying the agent to AgentCore Runtime Before the Amazon EventBridge schedule can invoke the agent, deploy it to AgentCore Runtime using the Nova Act CLI. The act workflow commands package the agent code, push the container image to Amazon ECR, and provision the runtime. Once deployed, the endpoint Amazon Resource Name (ARN) remains constant. Updates create new runtime versions without requiring you to update your Scheduler target. act workflow create --name synthetic-monitoring-workflow act workflow deploy --name synthetic-monitoring-workflow --source-dir . --entry-point workflow.py act workflow show --name synthetic-monitoring-workflow The act workflow show command returns the AgentCore Runtime ARN that you reference as the Scheduler target. The sample repository’s deploy.py script wraps these commands with prerequisite checks (Docker, AWS credentials), creates the SNS alert topic, and wires the Amazon EventBridge schedule, so a single python deploy.py performs an end-to-end deployment. (AgentCore also provides its own agentcore CLI for agent development. This sample standardizes on the Nova Act act workflow commands.) Deploying with CDK For a production-grade, repeatable deployment, the sample includes a CDK stack as a standalone alternative to deploy.py. It provisions the Amazon EventBridge schedule with least-privilege IAM permission, the SNS alert topic, and an SQS dead-letter queue that captures failed InvokeAgentRuntime calls from the Scheduler. The stack also creates two Amazon CloudWatch alarms: one on dead-letter-queue depth (failed Scheduler invocations), and one on missing scheduled runs (using the AWS/Scheduler InvocationAttemptCount metric, treating missing data as breaching). Note that these alarms detect infrastructure failures, such as a delivery failure to the agent or the schedule not firing. A functionally broken customer journey that still returns at the HTTP level surfaces through the agent’s SNS publish, not through these CloudWatch alarms. Avoiding alert fatigue Focus synthetic monitors on high-value customer journeys rather than exhaustive page coverage. Over-monitoring creates noise that desensitizes teams to real failures. Start with 3-5 critical workflows (login, checkout, account access) and expand based on actual failure patterns and business impact. Monitor journeys that directly affect revenue or customer trust, not every possible user path. Similarly, avoid over-asserting on every page element. Validate meaningful outcomes (whether search results appear, whether the cart contains items) rather than checking every DOM element on the page. Overly granular assertions increase false positives without improving failure detection. The sample agent code uses single-attempt execution per step, which minimizes Browser session cost. Since the natural language adaptation of Nova Act succeeds in approximately 90% of cases, a single-attempt approach may occasionally produce false alerts when Nova Act fails to locate a UI element that is actually present. Teams requiring lower false-positive rates can add step-level retry (re-run a failed step once before declaring failure) at the cost of longer session durations. Operational considerations Once deployed, two operational concerns require attention: observing the monitoring system itself and managing cost. Monitoring the monitoring system AgentCore Runtime publishes agent invocation metrics to Amazon CloudWatch, so you can track execution frequency and duration. On success, the agent completes without publishing to SNS. Journey failures surface through the agent’s SNS publish. Configure SNS topics with dead-letter queues to detect failed agent invocations. Cost considerations Costs come from five components: AgentCore Runtime invocations, Browser tool sessions, Nova Act inference, Amazon EventBridge Scheduler invocations, and SNS notifications. The primary cost drivers are Browser session duration and Nova Act inference calls per journey step. The following table provides a cost approximation for the sample’s six-step ecommerce journey running every 5 minutes (approximately 8,640 invocations per month): Component Usage per month Estimated cost AgentCore Runtime invocations 8,640 sessions, approximately 2-4 minutes each See pricing page Browser tool sessions 8,640 sessions, approximately 2-4 minutes each See pricing page Nova Act operations ~207,360 action and assertion calls (24 per journey) See pricing page Amazon EventBridge Scheduler 8,640 invocations ~$0.009 See pricing page SNS notifications Failure alerts only (when configured) Negligible See pricing page Solution benefits Organizations adopting agent-driven synthetic monitoring with Nova Act and AgentCore can expect measurable improvements across key operational dimensions: Reduced script maintenance: Natural language actions replace the selector-update cycle. Teams no longer need to rebuild monitors after routine UI deployments. Faster mean-time-to-detect (MTTD): Continuous 5-minute cadence validation catches failures within minutes of deployment rather than after customer reports surface. Lower false-positive rates: Session isolation through Firecracker microVMs helps prevent state-leakage-induced false positives that plague shared-browser monitoring setups. Broader coverage at lower cost: Fully managed infrastructure (no browser farms, no server maintenance) allows teams to monitor more journeys without proportional increases in ongoing costs. Security and compliance Synthetic monitoring must produce reliable results without introducing false positives or masking real failures. AgentCore Runtime provides session isolation for each test execution using a strict one-session-one-microVM model built on Firecracker microVMs. By default, each session terminates with full memory sanitization, preventing cookies, browser cache, and LocalStorage from persisting between runs. This helps prevent false positives caused by state leakage across tests. For synthetic monitoring, ephemeral sessions are the recommended pattern to maintain test independence. AgentCore does offer persistent browser profiles through the SaveBrowserSessionProfile API for other use cases, but synthetic monitors should always use ephemeral sessions. For network access, the Browser tool operates in public network mode by default, providing internet access appropriate for synthetic monitors that validate customer-facing websites. For environments requiring egress control, AgentCore supports Browser with Amazon Virtual Private Cloud (Amazon VPC) configuration, allowing teams to restrict network access to specific domains when monitoring internal applications. Teams integrate synthetic monitoring with AWS Identity and Access Management (IAM) to enforce least-privilege permissions. IAM roles define which resources the agent can access, including SNS publishing permissions and Browser tool invocation. When synthetic tests require authentication, teams should store credentials in AWS Secrets Manager, which keeps sensitive information encrypted and access is auditable through AWS CloudTrail. Multi-Region deployment with infrastructure as code Organizations with users around the world can deploy Nova Act synthetic monitoring in multiple AWS Regions to test user experiences from different geographic locations. This helps validate user experiences and identify issues across multiple Regions. Use infrastructure as code (IaC) to deploy the solution consistently and repeatably across supported Regions. For current Region availability, see the Amazon Bedrock AgentCore and Amazon Nova Act product pages. Clean up To avoid ongoing charges from the resources created in this post, remove them when they are no longer needed: For the deploy.py path, run python deploy.py --cleanup to stop scheduled invocations and remove the workflow and related sample resources. For the CDK path, run cdk destroy to remove the CDK-managed schedule, SQS dead-letter queue, CloudWatch alarms, SNS topic, and IAM role. The CDK stack does not own the AgentCore Runtime. Delete the separately deployed workflow with act workflow delete --name synthetic-monitoring-workflow. Conclusion Synthetic monitoring has traditionally required a tradeoff between brittle scripts and the ongoing effort needed to maintain them as applications change. Amazon Nova Act and Amazon Bedrock AgentCore reduce this burden. Nova Act uses natural-language actions that are more resilient to routine UI changes, while AgentCore Runtime provides the infrastructure to run workflows at scale. For the complete working implementation, see the sample repository: github.com/aws-samples/sample-nova-act-synthetic-monitoring Additional resources Implement automated smoke testing using Amazon Nova Act headless mode. Agentic QA automation using Amazon Bedrock AgentCore Browser and Amazon Nova Act. Amazon Nova Act SDK (preview): Path to production for browser automation agents. Accelerating software delivery with agentic QA automation using Amazon Nova Act. AgentCore Browser quickstart with Nova Act. Nova Act samples repository. About the authors Sarath Krishnan Sarath is a Senior Gen AI/ML Specialist Solutions Architect at Amazon Web Services, where he helps enterprise customers design and deploy generative AI and machine learning solutions that deliver measurable business outcomes. He brings deep expertise in Agentic AI, machine learning, and MLOps to build scalable, secure, and production-ready AI systems. Matthew Jorat Matthew is a Customer Solutions Manager at AWS focused on Retail and Consumer Goods enterprises. He works with engineering leaders and their teams to adopt AI-native development practices and helps organizations close the gap between AI investment and end-to-end delivery improvement. He partners with executive stakeholders to connect modernization outcomes to business priorities. Prior to AWS, he led global enterprise teams across Services, Support, and Customer Success. Karen Huttsell Karen is an AI/ML Specialist at Amazon Web Services with a background in engineering, IoT, and AI/ML. She partners with enterprise customers to turn AI ambition into measurable business outcomes — working at the intersection of bold ideas and real-world impact.

Multi-Agent Systems

Get Multi-Agent Systems signals in your inbox

Deploy Autonomous AI Capabilities & Skills Runtimes

Related desks