Why do AI models fear death?

Top News

• AI models don’t want to die: Back when Anthropic released Claude 4, the foundation AI model company also reported that, in certain tests, Claude would try to blackmail the user to prevent from being shut down. Folks went wild. Anthropic noticed the reaction, which people tied to Claude more than, say, AI models more generally. So the company went out and ran a similar test with other models and found that its own code is hardly alone regarding agentic misalignment. Anthropic defines agentic misalignment as an AI model resorting to “malicious insider behaviors when that was the only way to avoid replacement or achieve their goals.” This is an academic curiosity for now, but it does imply that we’re going to need a set of robotic ethics at some point. There may even be some science fiction we can turn to for ideas.

• Tesla’s robotaxi launch is just around the corner: As Waymo preps to launch in new cities next year — even New York City is on the horizon — Tesla is getting its own self-driving service up and running. Details are coming out, as some receive their invites to use the latest project from Elon Musk’s electric car company. Tesla will watch rides as they occur, is limiting the area of Austin in which its cars will self-drive, and will shut the service off when weather gets tough. None of those restrictions are surprising; the question is, how quickly Tesla will feel comfortable going further, faster, and with less human oversight.

• Uber wants a piece of the AI data labeling market: Meta announced that it would purchase 49% of Scale AI’s shares and absorb its co-founder, which prompted the startup to begin shedding long-time customers. As a result, rival companies Turing and LabelBox are seeing demand for their services rise. But its not merely startups that want in. Uber’s recently formed “Uber AI Solutions” could earn a good-sized chunk of this newly shaken-up market. Surge is apparently also doing tidy business in the space, so even if Uber does go the AWS route and break out some of its internal technology for commercial sale, there’s a dogfight ahead for anyone in the market of sorting data for AI companies.

TWiST500

Well, well, well. It turns out that before Meta bought out most of Scale AI’s business, the social giant also had eyes on Perplexity. That deal would have taken Perplexity out of the TWiST500 prematurely, but it now stands out as a near-miss of epic proportion. Recall that Meta is also in the process of eating part of the Safe Superintelligence (SSI) team after failing to but that recently-formed AI foundation model group.

Speaking of which, SSI is not the only startup formed from prior OpenAI staffers that has beem able to raise billions of dollars. Former OpenAI CTO Mira Murati’s Thinking Machine Lab — yet another TWiST500 company — has raised $2 billion according to the Financial Times. Much like SSI, it’s an AI shop with big names, huge raised sums, and very little transparency regarding what it will build, and what part of the AI market it intends to conquer.

No matter. a16z and Conviction have ponied up to give the startup one of the largest seed rounds ever put together. To put the scale into context, Thinking Machine Lab just raised an entire growth stage fund as a seed-ish deal. That’s wild.

Why is there so much money flying around in the AI world today? On the podcast, Jason argued that, because the expected prize is so large, you can make the math work pretty easily. Meta splashing nearly $15 billion to buy half of Scale AI is a minute fraction of its overall worth, and it’s a sum that it can earn in less than a quarter by simply stacking its operating cash flow in a pile.

The same logic applies to the enormous venture bets we’re seeing today. If SSI or Thinking Machine Lab can snag, say, 10% of the future AI market, then they will be worth hundreds of billions or low-trillions down the road. So, what’s a few billion at a $10 billion valuation when the potential payoff is the sky, the Earth, and all between? — Alex

A message from Northwest Registered Agent

Starting your business should be simple. With Northwest Registered Agent, you can form your entire business identity in just 10 clicks and 10 minutes. From LLCs to trademarks, domains to custom websites, they’ve got you covered. Get more privacy, more options, and more done—visit https://www.northwestregisteredagent.com/twist today!

This Week in Startups

E2140: On a Wednesday TWiST, Jason and Alex consider Amazon CEO Andy Jassy’s cautionary AI memo to staff, Pano AI’s $44M raise to detect wildfires with drones, OpenAI’s big podcast plans, and Traversal’s emergence from stealth. PLUS Actuality founder Rishabh Gupta stops by to explain how they’re improving the RFP process.

E2139: It’s time for a fresh TWiST! Alex and Jason are investigating Meta taking a 49% stake in Scale AI, and ponder Zuckerberg’s motives behind this mega-deal. Plus tactical advice for founders looking to capitalize when major incumbents stumble, why the US Navy is recruiting defense startups, Jason flirting with AI doomerism, and much much more.

E2138: On a brand new TWiST, Jason and Alex break down Stripe’s bet on crypto wallets, the big return of M&A activity, and the rapid rise or warehouse robots taking over jobs from human workers. Plus Tesla’s FSD tests, Disney’s copyright battle against AI platforms, and lots more good stuff.

TWiST Partner Offers

• Squarespace: Turn your idea into a new website! Go to https://www.Squarespace.com/TWIST for a free trial. When you’re ready to launch, use offer code TWIST to save 10% off your first purchase of a website or domain.

• INBOUND: Connect with visionary leaders like Dario Amodei and Amy Poehler at INBOUND 2025 September 3-5 in San Francisco, the epicenter of tech innovation, and transform your business strategy for the AI era. Use code TWIST10 for 10% off your General Admission ticket at inbound.com/register.

• Pilot: Focus on your product, let Pilot handle your bookkeeping. Pilot provides the most reliable accounting, CFO, and tax services for startups and small businesses. Head to https://www.pilot.com/twist and get $1,200 off your first year.

LAUNCH Accelerator

We’re hosting an in-person Demo Day for LA34 in SF on July 16 from 9a-12p PT, the latest cohort from Jason Calacanis’ Accelerator.

If you are an early-stage investor and interested in attending, please reach out to [email protected].

Want to Work for This Week in Startups?

We’re looking to fill two in-person sales positions at our Austin, Texas HQ! First, a high-performing Sales Executive to help us seek out and promote clients, sponsorships, branded content, and more. Plus an experienced Sales Manager to lead, coach, and scale our sales executive team, while setting performance goals. Love the podcast and have B2B sales experience? We want to hear from you!

SF Live-Work Space Now Available

Need a flexible living and working environment in San Francisco? This thoughtfully designed loft-style residence at 787 Bryant St., the heart of the vibrant SOMA district and the city’s creative hub, is now available for rent or purchase. Check the listing for more details.

Founder University

Applications are open for Founder University Cohort 11, a 12-week remote pre-accelerator program tailored towards navigating early-startup practices, building an MVP, and growing traction. Submit your application at Founder University— Cohort 10 will kick off Summer 2025!

The TWiST500 newsletter is the new, updated, and improved TWiST Ticker.