This Week in AI: Copilot agents, GPU shortage, AI memorandum

Getting your Trinity Audio player ready...

Microsoft (NASDAQ: MSFT) has announced that it will begin rolling out its autonomous agents via its Copilot Studio next month.

These autonomous agents are task-specific artificial intelligence (AI) models designed to handle work tasks from start to finish. Similar to AI chatbots, the agents are accessible to all users as long as they enter a sufficient prompt; all a user needs to do is enter a clear set of rules or guidelines that the agent must follow to complete the task at hand, along with any reference documents that the AI can refer to when executing the task.

Microsoft has also allowed room for the AI agent’s owners or operators to step in and check the agent’s work along the way or for the agent to notify its owner/operator if it hits a roadblock and isn’t sure what to do next.

“Agents draw on the context of your work data in Microsoft 365 Graph, systems of record, Dataverse, and Fabric, and can support everything from your IT help desk to employee onboarding and act as a personal concierge for sales and service,” read a Microsoft announcement.

Several major companies, such as McKinsey & Company and Thomson Reuters, have already tested these agents and reported that lead times could be reduced by 90%, and administrative work slashed by 30%, thanks to the autonomous agents. On its website, Microsoft highlights a sales qualification agent, a supplier communications agent and a customer knowledge management agent, but users will be able to create agents for practically any of their mundane, formulaic workflows.

These autonomous agents will undoubtedly be a positive for anyone who can find a way to automate part of their workflow or anyone who finds themselves doing a repetitive task that can easily be automated. But beyond clear use cases for businesses, Microsoft likely sees these autonomous agents as a way to generate revenue from its AI offerings. As we’ve covered in the past weeks, it’s no secret that AI companies and the AI branches of big tech firms are operating at a loss.

They are spending billions on computing infrastructure and data to train and run their models while not generating enough revenue from those same models to cover their costs. Putting a tool as useful as the autonomous agents behind the $200 per month Copilot Studio subscription may incentivize more businesses, especially corporates, to become paying customers of Microsoft’s AI products, inching the company closer to the profitability its investors are eagerly waiting for in its AI operations.

Biden’s AI National Security Memorandum

On Monday, President Joe Biden issued the first-ever National Security Memorandum (NSM) on AI. The NSM’s overarching goal is to ensure the United States stays ahead in AI by securing systems and setting the tone for global governance around the technology.

The NSM’s core objectives are to enhance the security of AI systems, protect U.S. innovations from espionage and ensure AI development aligns with democratic (American) values. While these goals sound great on paper, I am always skeptical about the real impact of these memorandums, especially given that this wave of AI innovation is largely driven by consumer-facing AI. I get that all government entities want to appear informed about AI since it’s one of the most popular pieces of technological infrastructure right now, but beyond the lip service, there’s rarely much done in terms of execution.

However, I will admit that the NSM could be different because it involves national security, which generally gets more attention and action than other areas. I’d even bet that some of these goals are already in motion. Still, I don’t expect the average person to feel the effects of this memo, although there may be some trickle-down benefits for consumers if AI systems become more robust and secure, as the NSM intends. This could benefit a broad range of players in the industry, and those benefits would likely be passed on to consumers in the form of the apps and services they use.

A16z tackles GPU shortage for its AI startups

a16z’s Oxygen program has set out to address a problem plaguing AI companies: the GPU shortage. In a recent podcast, General Partner Anjney Midha explained that the venture capital firm secured Nvidia (NASDAQ: NVDA) H100 GPUs for its portfolio companies, giving them the computing power to train and run their models in an increasingly competitive market. Some estimate that a16z’s GPU cluster could be as large as 20,000 Nvidia H100s.

The boom in consumer-facing AI applications spiked demand for these GPUs, which is why there currently isn’t enough supply to meet that newly increased demand. Larger companies that are willing to pay more and lock in longer contracts monopolized the market, leaving emerging AI startups at a significant disadvantage, effectively without access to the computing infrastructure they needed to succeed in the market.

“The hyperscale cloud providers have very sensitive margins tied to the occupancy rates or utilization rates for their clusters, so they were basically prioritizing long-term contracts over short-term contracts,” said Midha.

“From about late 2020 to mid-2023, the market rate for short-term GPU capacity had increased by 3-4x. Startups were being asked to commit more capital than they had raised—or even planned to raise in the next year—to get access to those rates,” Midha added.

a16z responded by assembling its own GPU cluster and giving its portfolio companies access to these GPUs. This allows the startups to focus on more critical aspects of their businesses, like growth and customer acquisition, without worrying about how they would get the GPUs they need or how much it would cost them. While this setup is clearly beneficial for the portfolio companies, it also puts A16z in a position of strength when it comes to investing.

The firm is able to leverage GPU access in funding negotiations and has found that it can invest more at a lower valuation because its startups don’t need to raise as much capital when their computing needs are already covered.

It’s widely discussed that compute is the biggest expense for many AI service providers, and it’s one of the main reasons OpenAI is not expected to turn a profit until 2029. What’s partially responsible for this inflated expense is the fact that there is a chip shortage, which chip providers are aware of and are claiming to be working hard to fix. However, this shortage may just create a new opportunity for investors who have the capital to secure the needed computing power, which they can turn around and use as a bargaining chip when negotiating terms during fundraising rounds.

In order for artificial intelligence (AI) to work right within the law and thrive in the face of growing challenges, it needs to integrate an enterprise blockchain system that ensures data input quality and ownership—allowing it to keep data safe while also guaranteeing the immutability of data. Check out CoinGeek’s coverage on this emerging tech to learn more why Enterprise blockchain will be the backbone of AI.

Watch: Demonstrating the potential of blockchain’s fusion with AI