Contrary Research Rundown #100
Dangerous inefficiencies in the energy consumption of AI, plus new memos on Telegram, Strava, and more
Announcements
Contrary Research is growing our team, hiring a Research Analyst. Come help us build the best starting place to understand any private tech company. To apply, email research@contrary.com.
Research Rundown
As artificial intelligence revolutionizes industries worldwide, it brings with it a surge in energy consumption. We first covered this trend in a Foundations and Frontiers article on the energy needed to power AI. This growing demand for power, particularly in data centers that form the backbone of AI operations, raises critical questions about the sustainability of AI's rapid expansion and the future of energy production required to support it.
AI models, especially large language models (LLMs) like GPT-3 and GPT-4, are notoriously energy-intensive. The training phase alone for GPT-3 consumed an estimated 1,287 MWh—equivalent to the annual energy usage of 100 U.S. households. However, the energy requirements extend beyond training. The inference phase, where models process real-time queries, accumulates even greater energy consumption over time.
The energy disparity between AI-powered and traditional search queries is stark. A single ChatGPT query consumes approximately ten times more energy than a Google search. If Google's daily search volume were to be powered by ChatGPT, or a similar AI, it would require about 9 terawatt-hours annually—roughly 4% of the total energy consumption of all U.S. data centers.
In a recent appearance on Patrick O'Shaughnessy's "Invest Like the Best" podcast, Gavin Baker — managing partner and CIO of Atreides Management — discussed a wide range of topics that are shaping the future of AI. He mentions the escalating energy consumption of AI systems and the pressing need for improved efficiency in data center infrastructure due do the the mismatch between GPU capabilities and supporting technologies.
While GPUs — the workhorses of AI computations — have seen dramatic speed improvements, other crucial components like networking, storage, and memory lag behind. This disparity results in inefficient GPU utilization, with powerful processors often idling while waiting for data.
Baker emphasizes the need for advanced networking to ensure faster data transfer to GPUs, enhanced storage solutions for quicker data retrieval and processing, and improved memory technologies to keep pace with GPU processing speeds. Additionally, optimizing Power Usage Effectiveness (PUE) is crucial for managing electricity costs in data centers. Baker warns that without significant advancements in these areas, the energy demands of AI could potentially outstrip available power resources, leading to a scenario where data centers consume an unsustainable portion of global electricity.
Tech giants are exploring innovative power solutions to meet AI's growing energy needs. Google's partnership with Fervo to use geothermal energy for its Nevada data centers exemplifies the industry's push towards sustainable energy sources. The development of more energy-efficient AI models, such as OpenAI's GPT-4 Turbo, represents another approach to mitigating AI's energy impact. These models aim to deliver equivalent performance at lower energy costs.
Meeting the energy demands of AI will require a multi-faceted approach. Continued innovation in energy-efficient technologies, both in AI models and data center infrastructure, is essential. Expanding and diversifying sustainable power generation sources will play a crucial role. Optimizing data center design with a focus on efficiency and sustainable practices is equally important. Policy and industry collaboration will also be necessary to ensure responsible AI development and deployment.
The future of AI hinges not just on computational power, but on the ability to power it sustainably. This challenge presents an opportunity to revolutionize our approach to energy production and consumption, potentially leading to breakthroughs that benefit not only the tech industry but society as a whole.
Telegram is a messaging app that focuses on “speed and security” and describes itself as “like SMS and email combined”. To learn more, read our full memo here and check out some open roles below:
C/C++ Software Engineer - Remote
Site Reliability Engineer - Remote
Strava is a platform for athletes and fitness enthusiasts. It provides a fitness tracking tool that doubles as a sports and fitness-focused social network, empowering users to record and share their workouts, connect with others, and engage in friendly competition. To learn more, read our full memo here and check out some open roles below:
Cloud Platform Engineer - San Francisco, CA
Engineering Manager (Data Platform) - New York, NY
Check out some standout roles from this week.
Skild AI | Pittsburgh, San Francisco or Bengaluru - Embedded Security Engineer, Embedded Systems Engineer, Robotics Software Engineer, Software Engineer (AI Training and Infrastructure)
Abridge | New York, San Francisco or Remote - Backend Engineer (Temporal Staff/Senior), Full Stack Engineer (Security Staff/Senior), Integration Engineer (Senior/Staff)
Inworld | Mountain View, CA - Staff Cloud DevOps/Site Reliability Engineer (SRE), Staff/Principal Machine Learning Engineer (Speech), Senior Unreal Engine Developer
Chainguard | Remote (US) - Associate Software Engineer (DevOps), Associate Technical Program Manager
Bolt's $450 million fundraise at a $14 billion valuation has been marred by confusion, with one of the lead investors, Silverbear Capital, claiming they were never involved in the deal and were mistakenly named in the financial documents.
Mark Zuckerberg (CEO of Meta) and Daniel Ek (CEO of Spotify) argue that Europe risks falling behind in AI development due to "incoherent and complex regulation", suggesting the continent should embrace open-source AI to stay competitive.
ChatGPT and Meta AI both report ~400 million monthly active users.
Noland Arbaugh, Neuralink's first patient, has named his brain implant "Eve" and is using it to learn French and Japanese, in addition to relearning math in preparation for potentially going back to school.
AI models can "collapse" and produce lower-quality, less diverse output when trained repeatedly on their own synthetic data, posing a threat to the long-term progress of A.I. technology.
The battle over AI training data is playing out in the obscure robots.txt file, which governs how web crawlers access websites. "I find it fascinating that one of the most consequential technologies of our era is being developed, and the battle for its training data is playing out on this really obscure text file, in public for us all to see”.
Nvidia reported its fourth-straight quarter of triple-digit revenue growth on Wednesday and the company announced its $50 billion buyback program. However, the stock dropped 7% in extended trading.
The risk of underinvesting in AI is far greater than the risk of over-investing, according to tech leaders like Mark Zuckerberg and Sundar Pichai. Major tech companies are willing to err on the side of overbuilding AI infrastructure, as they believe the potential downside of falling behind in this critical technology is much more severe.
California legislature approved a bill to regulate AI, which could set a national standard for AI safety if signed into law.
Cerebras Systems — a platform for fast and effortless AI training and inference — set a record for AI inference performance, serving LLama 3.1 8B at 1,850 output tokens per second. Look out for our upcoming memo in the coming weeks!
Supabase, a company that enables developers to build applications, announced a partnership and first-party integration with Vercel, a software development workflow and deployment platform. This partnership will enable managing Supabase databases directly from the Vercel dashboard.
Meta and Instagram are developing a new social music-sharing feature, which could signal a move into the music industry.
OpenSea has received a Wells notice from the SEC threatening to sue them because they believe NFTs on our platform are securities.
OpenAI demonstrated its mysterious "Strawberry" AI technology to U.S. national security officials this summer, indicating the technology could have important applications beyond just conversational AI.
OpenAI is in talks to raise several billion dollars in a new funding round that would value the company above $100 billion, up from the $86 billion mark it was valued at last year when employees sold existing shares.
Google disputes Amazon's claims about its AI capabilities, emphasizing its superior AI infrastructure and wider array of models.
Uber and Coco announced a partnership to launch Coco's sidewalk robots on Uber Eats, starting in Los Angeles this week and expanding to more cities in the months to come.
Speaking of Uber, CEO Dara Khosrowshahi has been an unlikely cheerleader for Instacart, praising its "very strong advertising product" and "very high quality" suburban customer base, despite being a competitor in the grocery delivery space.
Fireworks AI is one of the fastest growing AI startup in the industry. Over the past six months, Fireworks AI has experienced a 100x increase in traffic. In July, the company closed a Series B funding round led by Sequoia, bringing its valuation to $552 million.
Sarah Tavel, a partner at Benchmark, argues AGI potentially represents a multi-trillion dollar opportunity and its economic value is constrained only by our imaginations.
The merger between struggling startups Truepill and LetsGetChecked is driven by the fact that they share major shareholders, such as the venture capital arm of UnitedHealth Group, the largest U.S. health insurance firm. This suggests that the merger is being facilitated by the companies' common investors, rather than solely driven by strategic business reasons.
The USA doesn't have a digital-only bank at the 100m+ user scale which can be attributed to regulation, competition, and fragmentation in the U.S. market, which make it much harder to replicate Nubank's success in Brazil.
Podcasts are becoming more ad-heavy, moving away from their ad-light past. Podcasters are now including 8 ads per hour-long episode, up from just 1 ad at the beginning and 1 in the middle previously, as they race to monetize the growing podcast audience.
Forerunner Ventures, a consumer-focused VC firm, believes Gen AI can finally deliver on the promise of personalization by surfacing insights and outputs with more comprehensive context than any one person could have themselves.
The potential of applying generative AI to business data is significant, the key challenge lies in ensuring the data itself is well-defined, labeled, and understood. This "semantic layer" is essential for LLMs and SLMs to provide accurate and context-aware responses, but can also be a major obstacle that data teams struggle with.
Lockheed Martin and General Dynamics have formed a partnership to ramp up production of solid rocket motors, which is critical for advancing the "21st Century Security" vision and strengthening the domestic defense industrial base.
A limited number of the Foundations & Frontiers Zine Vol. 2 are now available for order. This issue on space tech features essays on satellites, rocketry, and Mars colonization. Claim your copy here.
At Contrary Research, we’re building the best starting place to understand private tech companies. We can't do it alone, nor would we want to. We focus on bringing together a variety of different perspectives.
That's why we're opening applications for our Research Fellowship. In the past, we've worked with software engineers, product managers, investors, and more. If you're interested in researching and writing about tech companies, apply here!