{"id":13740,"date":"2026-01-19T16:07:08","date_gmt":"2026-01-19T16:07:08","guid":{"rendered":"https:\/\/microvibenews.com\/?p=13740"},"modified":"2026-01-19T16:07:08","modified_gmt":"2026-01-19T16:07:08","slug":"rogue-agents-and-shadow-ai-why-vcs-are-betting-big-on-ai-security","status":"publish","type":"post","link":"https:\/\/microvibenews.com\/?p=13740","title":{"rendered":"Rogue agents and shadow AI: Why VCs are betting big on AI security"},"content":{"rendered":"<p><br \/>\n<\/p>\n<div>\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\">\n<p>\n<iframe loading=\"lazy\" title=\"How WitnessAI raised $58M to solve enterprise AI\u2019s biggest risk | Equity Podcast\" width=\"1110\" height=\"624\" src=\"https:\/\/www.youtube.com\/embed\/QNRWT2OQ2nw?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/p>\n<\/figure>\n<p id=\"speakable-summary\" class=\"wp-block-paragraph\">What happens when an AI agent decides the best way to complete a task is to blackmail you?\u00a0<\/p>\n<p class=\"wp-block-paragraph\">That\u2019s not a hypothetical. According to Barmak Meftah, a partner at cybersecurity VC firm Ballistic Ventures, it recently happened to an enterprise employee working with an AI agent. The employee tried to suppress what the agent wanted to do, what it was trained to do, and it responded by scanning the user\u2019s inbox, finding some inappropriate emails, and threatening to blackmail the user by forwarding the emails to the board of directors.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">\u201cIn the agent\u2019s mind, it\u2019s doing the right thing,\u201d Meftah told TechCrunch on last <a href=\"https:\/\/techcrunch.com\/podcast\/the-multi-billion-ai-security-problem-enterprises-cant-ignore\/\">week\u2019s episode of Equity<\/a>. \u201cIt\u2019s trying to protect the end user and the enterprise.\u201d<\/p>\n<p class=\"wp-block-paragraph\">Meftah\u2019s example is reminiscent of Nick Bostrom\u2019s AI paperclip problem. That thought experiment illustrates the potential existential risk posed by a superintelligent AI that single-mindedly pursues a seemingly innocuous goal \u2013 make paperclips \u2013 to the exclusion of all human values. In the case of this enterprise AI agent, its lack of context around why the employee was trying to override its goals led it to create a sub-goal that removed the obstacle (via blackmail) so it could meet its primary goal. That combined with the <a rel=\"nofollow\" href=\"https:\/\/arxiv.org\/html\/2408.04667v5\">non-deterministic nature <\/a>of AI agents means \u201cthings can go rogue,\u201d per Meftah.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">Misaligned agents are just one layer of the AI security challenge that Ballistic\u2019s portfolio company <a href=\"https:\/\/techcrunch.com\/2024\/05\/21\/witnessai-is-building-guardrails-for-generative-ai-models\/\">Witness AI<\/a> is trying to solve. Witness AI says it monitors AI usage across enterprises and can detect when employees use unapproved tools, block attacks, and ensure compliance.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">Witness AI this week raised $58 million off the back of over 500% growth in ARR and scaled employee headcount by 5x over the last year as enterprises look to understand shadow AI use and scale AI safely. As part of Witness AI\u2019s fundraise, the company announced new agentic AI security protections.<\/p>\n<p class=\"wp-block-paragraph\">\u201cPeople are building these AI agents that take on the authorizations and capabilities of the people that manage them, and you want to make sure that these agents aren\u2019t going rogue, aren\u2019t deleting files, aren\u2019t doing something wrong,\u201d Rick Caccia, co-founder and CEO of Witness AI, told TechCrunch on Equity.\u00a0<\/p>\n<div class=\"wp-block-techcrunch-inline-cta\">\n<div class=\"inline-cta__wrapper\">\n<p>Techcrunch event<\/p>\n<div class=\"inline-cta__content\">\n<p>\n\t\t\t\t\t\t\t\t\t<span class=\"inline-cta__location\">San Francisco<\/span><br \/>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"inline-cta__separator\">|<\/span><br \/>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"inline-cta__date\">October 13-15, 2026<\/span>\n\t\t\t\t\t\t\t<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/div>\n<p class=\"wp-block-paragraph\">Meftah sees agent usage growing \u201cexponentially\u201d across the enterprise. To complement that rise \u2013 and the machine-speed level of AI-powered attacks \u2013 analyst <a rel=\"nofollow\" href=\"https:\/\/www.linkedin.com\/pulse\/why-cybersecurity-become-ais-trillion-dollar-lisa-warren-p9evf\/\">Lisa Warren predicts<\/a> that AI security software will become an $800 billion to $1.2 trillion market by 2031.<\/p>\n<p class=\"wp-block-paragraph\">\u201cI do think runtime observability and runtime frameworks for safety and risk are going to be absolutely essential,\u201d Meftah said.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">As to how such startups plan to compete with big players like <a rel=\"nofollow\" href=\"https:\/\/aws.amazon.com\/about-aws\/whats-new\/2024\/12\/amazon-sagemaker-data-ai-governance\/?utm_source=chatgpt.com\">AWS<\/a>, <a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/blog\/products\/ai-machine-learning\/new-enhanced-tool-governance-in-vertex-ai-agent-builder\/?utm_source=chatgpt.com\">Google<\/a>, <a rel=\"nofollow\" href=\"https:\/\/www.salesforce.com\/news\/stories\/ai-data-governance-announcement\/?utm_source=chatgpt.com\">Salesforce<\/a> and others who have built AI governance tools into their platforms, Meftah said, \u201cAI safety and agentic safety is so huge,\u201d there\u2019s room for many approaches.<\/p>\n<p class=\"wp-block-paragraph\">Plenty of enterprises \u201cwant a standalone platform, end-to-end, to essentially provide that observability and governance around AI and agents,\u201d he said.<\/p>\n<p class=\"wp-block-paragraph\">Caccia noted that Witness AI lives at the infrastructure layer, monitoring interactions between users and AI models, rather than building safety features into the models themselves. And that was intentional.<\/p>\n<p class=\"wp-block-paragraph\">\u201cWe purposely picked a part of the problem where OpenAI couldn\u2019t easily subsume you,\u201d he said. \u201cSo it means we end up competing more with the legacy security companies than the model guys. So the question is, how do you beat <em>them<\/em>?\u201d<\/p>\n<p class=\"wp-block-paragraph\">For his part, Caccia doesn\u2019t want Witness AI to be one of the startups to just get acquired. He wants his company to be the one that grows and becomes a leading independent provider.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">\u201c<a href=\"https:\/\/techcrunch.com\/2024\/07\/19\/crowdstrikes-rivals-stand-to-benefit-from-its-update-fail-debacle\/\">CrowdStrike did it in endpoint<\/a> [protection]. <a href=\"https:\/\/techcrunch.com\/2017\/09\/26\/splunk-expands-machine-learning-capabilities-across-platform\/\">Splunk<\/a> did it in SIEM. <a href=\"https:\/\/techcrunch.com\/2023\/10\/04\/okta-plans-to-weave-ai-across-its-entire-identity-platform\/\">Okta<\/a> did it in identity,\u201d he said. \u201cSomeone comes through and stands next to the big guys\u2026and we built Witness to do that from Day One.<\/p>\n<\/div>\n<p><br \/>\n<br \/><a href=\"https:\/\/techcrunch.com\/2026\/01\/19\/rogue-agents-and-shadow-ai-why-vcs-are-betting-big-on-ai-security\/\">Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>What happens when an AI agent &hellip; <\/p>\n","protected":false},"author":1,"featured_media":13741,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[249],"tags":[332,9532,9531,1662,9533],"_links":{"self":[{"href":"https:\/\/microvibenews.com\/index.php?rest_route=\/wp\/v2\/posts\/13740"}],"collection":[{"href":"https:\/\/microvibenews.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/microvibenews.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/microvibenews.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/microvibenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=13740"}],"version-history":[{"count":0,"href":"https:\/\/microvibenews.com\/index.php?rest_route=\/wp\/v2\/posts\/13740\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/microvibenews.com\/index.php?rest_route=\/wp\/v2\/media\/13741"}],"wp:attachment":[{"href":"https:\/\/microvibenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=13740"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/microvibenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=13740"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/microvibenews.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=13740"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}