Shortly after Amazon CEO Andy Jassy announced AWS\u2019s groundbreaking $50 billion investment deal<\/a> with OpenAI, Amazon invited me on a private tour of the chip development lab at the heart of the deal, at (mostly*) its own expense.\u00a0<\/p>\n Industry experts are watching<\/a> Amazon\u2019s Trainium chip, created at that facility, for its implications for lower-cost AI inference and, potentially, a dent in Nvidia\u2019s near monopoly.\u00a0\u00a0<\/p>\n Curious, I agreed to go.\u00a0\u00a0<\/p>\n My tour guides for the day were the lab\u2019s director, Kristopher King (pictured below right) and director of engineering Mark Carroll (below left), as well as the team\u2019s PR person who arranged the visit, Doron Aronson (pictured with yours truly later in the story).\u00a0<\/p>\n AWS has been Anthropic\u2019s major cloud platform since the AI lab\u2019s early days\u00a0\u2014 a relationship significant enough to survive Anthropic later adding Microsoft as a cloud partner as well, and Amazon\u2019s growing partnership with OpenAI.<\/p>\n The OpenAI deal makes AWS the exclusive provider of the model maker\u2019s new AI agent builder, Frontier, which could become an important part of OpenAI\u2019s business if agents become as big as Silicon Valley thinks they will.\u00a0We\u2019ll see if that exclusivity stands exactly as announced. The Financial Times reported this week<\/a> that Microsoft may believe OpenAI\u2019s deal with Amazon violates its own deal with OpenAI, namely with Redmond getting access to all of OpenAI\u2019s models and tech<\/a>.<\/p>\n What makes AWS so appealing to OpenAI? As part of this deal, the cloud giant has agreed to supply OpenAI with 2 gigawatts of Trainium computing capacity. This is a giant commitment, given that Anthropic and Amazon\u2019s own Bedrock service are already consuming Trainium chips faster than Amazon can produce them.\u00a0<\/p>\n Techcrunch event<\/p>\n \n\t\t\t\t\t\t\t\t\tSan Francisco, CA<\/span> There are 1.4 million Trainium chips deployed across all three generations, and Anthropic\u2019s Claude runs on over 1 million of the Trainium2 chips deployed, the company said. <\/p>\n It\u2019s worth noting that while Trainium was originally geared toward faster, cheaper model training (a bigger priority a couple of years ago), it\u2019s now tuned and used for inference as well. Inference \u2014 the process of actually running an AI model to generate responses \u2014 is currently the biggest performance bottleneck in the industry.\u00a0<\/p>\n Case in point: Trainium2 handles the majority of the inference traffic on Amazon\u2019s Bedrock service<\/a>, which supports the building of AI applications by Amazon\u2019s many enterprise customers and allows the apps to use multiple models.<\/p>\n \u201cOur customer base is just expanding as fast as we can get capacity out there,\u201d King said. \u201cBedrock could be as big as EC2 one day,\u201d he added, referring to AWS\u2019s behemoth compute cloud service.\u00a0<\/p>\n Beyond offering an alternative to Nvidia\u2019s backlogged, hard-to-acquire GPUs, Amazon says<\/a> its new chips running on its new specialty Trn3 UltraServers cost up to 50% less to run for comparable performance than using classic cloud servers.\u00a0<\/p>\n Along with Trainium3, released in December<\/a>, this AWS team also built new Neuron switches, and Carroll says that combo is transformative.<\/p>\n \u201cWhat that gives us is something huge,\u201d Carroll said. The switches allow every Trainium3 chip to talk to every other chip in a mesh configuration, reducing latency. \u201cThat\u2019s why Trainium3 is breaking all kinds of records,\u201d particularly in \u201cprice per power,\u201d he said.\u00a0<\/p>\n When trillions of tokens a day are involved, such improvements add up.\u00a0\u00a0<\/p>\n In fact, Amazon\u2019s chip team was lauded by Apple in 2024<\/a>. In a rare moment of openness for the secretive company, Apple\u2019s director of AI publicly described how it used another of the team\u2019s chips \u2014 Graviton,\u00a0a low-power, ARM-based server CPU and the first breakout chip this team designed. Apple also lauded Inferentia \u2014 a chip specifically designed for inference \u2014 and gave a nod to Trainium, which was new at the time.\u00a0<\/p>\n These chips represent the classic Amazon playbook: See what people want to buy, then build an in-house alternative that competes on price.\u00a0<\/p>\n The catch for chips, historically, has been switching costs. Applications written for Nvidia\u2019s chips must be re-architected to work with others \u2014 a time-consuming process that discourages developers from switching.<\/p>\n But the AWS chip team proudly told me that Trainium now supports PyTorch<\/a>, a popular open source framework for building AI models. That includes many of the ones hosted on Hugging Face, a vast library where developers share open source models.<\/p>\n The transition, Carroll told me, requires \u201cbasically a one-line change, and then recompile, and then run on Trainium.\u201d\u00a0In other words, Amazon is attempting to chip away at Nvidia\u2019s market dominance wherever possible.<\/p>\n AWS has also this month announced a partnership with Cerebras Systems<\/a>, integrating that company\u2019s inference chip on servers running Trainium for what Amazon promises will be superpowered, low-latency AI performance.\u00a0<\/p>\n But Amazon\u2019s ambitions go beyond the chips themselves. It also designs the server that hosts the chips. Besides the networking components, this team has designed \u201cNitro,\u201d a hardware-software combo that provides virtualization tech (which allows many instances of software to run separately on the same server); new state-of-the-art liquid cooling technology; and the server sleds (pictured below) that host this gear.\u00a0<\/p>\n All of that is to control cost and performance.\u00a0 <\/p>\n Amazon\u2019s custom chip-designing unit was born when the cloud giant bought Israeli chip designer Annapurna Labs<\/a> in January 2015 for about $350 million. So this team has now had more than 10 years designing chips for AWS.\u00a0The unit has retained its Annapurna roots and name \u2014\u00a0its logo is everywhere in the office.\u00a0<\/p>\n This chip lab is located in a shiny, chrome-windowed building in Austin\u2019s upscale \u201cThe Domain\u201d district, a walkable area filled with shops and restaurants that\u2019s sometimes called Austin\u2019s Silicon Valley<\/a>.\u00a0<\/p>\n The offices have your classic tech corporate vibe: desks in cubicles, gathering spots, and conference rooms. But tucked away at the back of a high floor in the building is the actual lab, with sweeping views of the city.\u00a0\u00a0<\/p>\n The shelving-filled lab, about the size of two large conference rooms, is a noisy industrial space thanks to the fans on the equipment. It looks like a cross between a high school shop class and a Hollywood set for a high-end lab, except the engineers are dressed in jeans, not white lab coats.<\/p>\n Note that this is not where the chips are manufactured, so no white hazmat suits were necessary. The Trainium3 is a state-of-the-art 3-nanometer chip, produced by TSMC, arguably the leader in 3-nanometer manufacturing, with other chips produced by Marvell.\u00a0<\/p>\n But this is the room where the magic of the \u201cbring-up\u201d occurs.\u00a0\u00a0<\/p>\n \u201cA silicon bring-up is when you get the chip for the first time, and it\u2019s like a big overnight party. You stay here, like a lock-in,\u201d King explains. After 18 months of work, the chip is activated for the first time to verify it works as designed.\u00a0The team even filmed some of the Trainium3 bring-up and posted it on YouTube<\/a>.<\/p>\n Spoiler alert: It\u2019s never problem-free.\u00a0\u00a0<\/p>\n For Trainium3, the prototype chip was originally air-cooled, like previous versions. The current chip is now liquid-cooled,<\/a> which offers energy advantages and was quite an engineering feat.<\/p>\n During the bring-up, the dimensions for how the chip attached to the air-cooling heat sink were off, so the chip couldn\u2019t be activated.\u00a0<\/p>\n Unfazed, the team \u201cimmediately got a grinder and just started grinding off the metal,\u201d King said. Because they didn\u2019t want the noise disrupting the bring-up pizza party atmosphere, they snuck off and did the grinding in a conference room.\u00a0\u00a0<\/p>\n Staying up all night and solving problems \u201cis what silicon bring-up is all about,\u201d King said.\u00a0<\/p>\n The lab even has a welding station, where hardware lab engineer and master welder Isaac Guevara demonstrated welding tiny integrated circuit components through a microscope. This is such insanely difficult work that senior leader Carroll openly admitted he couldn\u2019t do it, to the guffaws of Guevara and the rest of the engineers in the room.\u00a0<\/p>\n The lab also contains both custom-made and commercial tools for testing and analyzing issues with chips.\u00a0Here\u2019s signal engineer Arvind Srinivasan demonstrating how the lab tests each tiny component on the chip:<\/p>\n But the star of the lab is an entire row showcasing each generation of the \u201csleds\u201d the team designed.\u00a0<\/p>\n Sleds are the trays that house the Trainium AI chips, Graviton CPU chips, and supporting boards and components. Stack them together on a rack with the networking component, also custom-designed by this team, and you get the systems that are at the heart of Anthropic Claude\u2019s success.\u00a0<\/p>\n Here\u2019s the sled that was shown off during the AWS re:invent conference in December:\u00a0<\/p>\n I expected my guides to crow about the OpenAI deal during the tour. But they didn\u2019t.\u00a0<\/p>\n The reticence could have been related to the aforementioned potential legal haze that might hang over the deal. But the sense I got was that these boots-on-the-ground engineers (who are currently designing the next version, Trainium4) haven\u2019t had much chance to work with OpenAI yet. Their day-to-day work has so far been focused on Anthropic\u2019s and Amazon\u2019s needs.<\/p>\n Currently, the biggest chunk of Trainium2 chips is deployed in Project Rainier \u2014 one of the world\u2019s largest AI compute clusters \u2014 which went live in late 2025 with 500,000 chips. It\u2019s used by Anthropic.\u00a0<\/p>\n But there was a wall monitor in the main office displaying a quote about how OpenAI will be using\u00a0Trainium. The pride was there, if subtle.\u00a0\u00a0<\/p>\n In addition to this lab, the team also has its own private data center for quality and testing purposes. A short drive away, it doesn\u2019t run customer workloads, so it\u2019s housed at a co-location facility, not an AWS data center.<\/p>\n Security is tight: There are strict protocols to enter the building and to access Amazon\u2019s area within.<\/p>\n The data center\u2019s cooling system is so loud that earplugs are mandatory, and the air is thick with the acrid smell of heated metal. It\u2019s not a pleasant place for the average person to hang out.\u00a0<\/p>\n At this data center, there are rows and rows of servers filled with sleds that integrate all of Amazon\u2019s newest custom chips: Graviton CPU, liquid-cooled Trainium3, Amazon Nitro, all happily computing away. The liquid runs on a closed system, meaning it is reused, which should also help reduce the environmental impact, the engineers said.\u00a0<\/p>\n Here\u2019s what a current Trn3 UltraServer looks like: Multiple sleds are on top and bottom, with the Neuron switches in the middle. Hardware development engineer David Martinez-Darrow is seen here performing maintenance on a sled:<\/p>\n While attention on the team has always been high, the scrutiny has really ratcheted up as of late.\u00a0<\/p>\n Amazon CEO Andy Jassy keeps a close eye on this lab, publicly bragging about its products like a proud dad. In December, he said Trainium was already a multibillion-dollar business for AWS<\/a> and called it<\/a> one piece of AWS tech he\u2019s most excited about. He also gave the chip a shout-out<\/a> when announcing the OpenAI agreement.\u00a0\u00a0<\/p>\n The team feels the pressure, too. Engineers will work 24\/7 for three to four weeks around each bring-up event to fix any issues so the chips can be mass-produced and put into data centers. <\/p>\n \u201cIt\u2019s very important that we get as fast as possible to prove that it\u2019s actually going to work,\u201d Carroll said. \u201cSo far, we\u2019ve been doing really well.\u201d\u00a0<\/p>\n *Disclosure: Amazon provided airfare and covered the cost of one night at a local hotel. Honoring its Leadership Principle of Frugality<\/a>, this was a back-of-the-plane middle seat and a modest room. TechCrunch picked up the other associated travel costs like Ubers and luggage fees. (Yes, I checked a bag for an overnight trip. I\u2019m high maintenance that way.)\u00a0<\/em><\/p>\n<\/div>\n![\"ASW](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2026\/03\/ASW-Chip_1101-Mark-Carroll-Kristopher-King.jpg?w=680\")
\n\t\t\t\t\t\t\t\t\t\t\t\t\t|<\/span>
\n\t\t\t\t\t\t\t\t\t\t\t\t\tOctober 13-15, 2026<\/span>\n\t\t\t\t\t\t\t<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/div>\n![\"Amazon's](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2026\/03\/Trainium-3-chip.jpeg?w=680\")
Trainium vs. Nvidia<\/h2>\n
![\"AWS](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2026\/03\/ASW-Chip-tour-sled-and-components.jpg?w=680\")
Working 24\/7 on the \u201cbring-up\u201d\u00a0<\/h2>\n
![\"ASW](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2026\/03\/ASW-Chip_tour-lab1.jpg?w=680\")
![\"ASW](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2026\/03\/ASW-Chip_tour-lab2.jpg?w=680\")
![\"ASW](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2026\/03\/ASW-Chip_tour-Isaac-Guevara.jpg?w=680\")
![\"AWS](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2026\/03\/ASW-Chip_tour-Arvind-Srinivasan.jpg?w=680\")
Sleds are the star of the lab\u00a0<\/h2>\n
![\"AWS](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2026\/03\/ASW-Chip_tour-Carroll-King-sleds.jpg?w=680\")
![\"AWS](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2026\/03\/ASW-Chip_tour-trainium3-sled.jpg?w=510\")
Proven by Anthropic and OpenAI <\/h2>\n
![\"AWS](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2026\/03\/ASW-Chip_tour-Julie-Bort-Doron-Aronson.jpg?w=680\")
![\"AWS](\"https:\/\/techcrunch.com\/wp-content\/uploads\/2026\/03\/ASW-Chip_tour-servers-David-Martinez-Darrow.jpg?w=680\")