Contrary Research Rundown #93
How much energy AI will require, plus new memos on BetterUp, Instabase, and more
Research Rundown
Achieving carbon neutral status has been a growing priority for a number of companies. From June 2022 to October 2023, the number of publicly listed companies with carbon neutral goals rose 40% from 702 to ~1K. But at the same time, the siren call of AI investment has gotten louder and louder, making that carbon neutral goal much less accessible.
Google is the textbook example. In the last five years, Google’s energy consumption grew 50% in large part due the energy required for its AI efforts. In addition, the company is no longer claiming to be carbon neutral and is making significant changes to how it thinks about things like carbon offsets.
To unpack this topic, Anna-Sofia Lesiv, the author of Foundations & Frontiers, has started a new series called Insights to tackle short, issue-based topics. In her first entry in this series, Anna address the question: “how much energy will it take to power AI?” Here’s an excerpt from the essay:
There is a lot of speculation about how much energy the growing demand for AI will require. Some estimates suggest that by 2026, electricity demand for data centers will double — globally using up as much electricity as all of Japan. These estimates match projections from analysts at Goldman Sachs who believe that by 2030, data center energy demands will grow by 160% and comprise nearly half of all new demand in that period.
Training foundational AI models can be quite energy-intensive. GPT-3, OpenAI’s 175 billion parameter model, reportedly used 1,287 MWh to train, while DeepMind’s 280 billion parameter model used 1,066 MWh. This is about 100 times the energy used by the average US household in a year. However, training a model is a one-time fixed energy cost, which can be amortized over the lifetime of the model.
Where energy costs can start to add up even further is in the use of the model over its lifetime. Running a query through an AI model is called inference, where the query is passed through the model’s parameters to deliver an output. Depending on how long the model is in use, which can be years, and how many queries it is fielding per day, the vast bulk of an AI model’s energy usage will come from inference.
Estimating exactly how much energy is devoted to inference in some of the largest and most popular models, like ChatGPT, is a highly speculative exercise as it depends on so many variables including the size of the average query, daily active user counts, the efficiency of the model, nature of the hardware stack, and more. In most cases, this is private information. However, some back-of-the-envelope estimates can help us understand the scale of energy that will be needed.
Check out the full essay to read Anna’s napkin math on AI costs.
BetterUp is a “human transformation platform” that, through virtual coaching, seeks to help individuals, organizations, and teams achieve their highest potential of performance. To learn more, read our full memo here and check out some open roles below:
Senior Full-Stack Engineer, Core Experiences - Amsterdam, Netherlands
Staff Full-Stack Engineer - Remote (US)
Instabase provides AI-powered document processing and workflow automation solutions. To learn more, read our full memo here and check out some open roles below:
Senior Software Engineer, Product Backend - San Francisco, CA
Senior Product Manager, Infrastructure - San Francisco, CA
Check out some standout roles from this week.
Carda Health | Remote (US) - Full Stack Software Engineer
Warp | Remote (US) - Product Manager, Software Engineer
Power | San Francisco, CA - Product Designer
Meta is set to release Llama 3 405B on July 23, just about a year after Llama 2 was announced.
OpenAI laid out what it sees as the five steps towards AGI: (1) Chatbots, Al with conversational language, (2) Reasoners, human-level problem solving, (3) Agents, systems that can take actions, (4) Innovators, Al that can aid in invention, and (5) Organizations, Al that can do the work of an organization. OpenAI says we’re approaching Level 2.
As regulatory scrutiny over AI intensifies, Apple and Microsoft have stated they won’t take board seats with OpenAI, even after large deals with the company, including implementing ChatGPT on Apple devices.
As part of an ongoing lawsuit between OpenAI and The New York Times, OpenAI has pushed the newspaper for source material for its articles to investigate whether OpenAI’s models are leveraging NYT articles or if they’re both pulling from similar sources.
Perplexity is, once again, coming under fire as the result of an investigation that indicated the chat platform was increasingly citing AI-generated sources, including “an AI-generated blog that contains contradictory and inaccurate information and was published by a medical clinic whose ownership remains unclear.”
Kevin Scott, Microsoft’s CTO, shared in a podcast episode that he believes there will be a shift across the compute ecosystem from training to inference as the frontier models continue to improve, serving wider and more reliable use cases.
Specialized LLMs are becoming increasingly popular. For example, EvolutionaryScale, a former Meta AI project to build LLMs for biology, has raised a $142 million seed round.
SpaceX’s most recent Falcon 9 launch had a rare hiccup, failing to complete its second burn. This resulted in a series of Starlink satellites to fail to reach high enough orbit, and are likely to slowly fail as they lose altitude.
Tomasz Tunguz shared some analysis on AI pricing strategies indicating that products like Google, Gitlab, and Microsoft that offer copilots end up adding a 70% premium for the functionality.
After launching Notion Calendar and Notion Sites, Notion’s CTO outlined where the productivity platform would go from here to justify its $10 billion valuation.
Runway, a generative AI platform for video, is reportedly in talks to raise a new round of funding at a $4 billion valuation with a reported $25 million in ARR.
Several drone companies, such as Zipline, DroneUp, and Alphabet’s Wing, have established relationships with Walmart to conduct drone deliveries. Unfortunately, some people have taken to shooting down those drones.
At Contrary Research, our vision is to become the best starting place to understand private tech companies. We can't do it alone, nor would we want to. We focus on bringing together a variety of different perspectives.
That's why we're opening applications for our Research Fellowship. In the past, we've worked with software engineers, product managers, investors, and more. If you're interested in researching and writing about tech companies, apply here!