Amazon CEO Andy Jassy described demand for artificial intelligence as a "barbell," with AI laboratories at one end and cost-cutting enterprises at the other, while the potential in between remains largely untapped because of high inference costs.

Inference — the compute-intensive process of running trained models to generate responses — is a recurring cost incurred with every query a model serves. Jassy's remarks underscore the challenge Amazon faces in making AI technology economically viable for broader enterprise use. They come as the company promotes its custom AI accelerators — specialized chips for AI computations — such as Trainium, which reportedly offer better price-performance compared to competitors. Trainium is developed by Amazon's Annapurna Labs unit and powers AI workloads across the company's Amazon Web Services (AWS) cloud platform.

The comments carry implications for the broader AI industry, including for companies like Anthropic, the developer of the Claude AI assistant. Amazon has invested billions of dollars in Anthropic, and the startup's models are made available to customers through AWS.

Market pricing referenced alongside Jassy's comments highlights the cost challenges in AI adoption and weighs on the valuation outlook for companies such as Anthropic. Current pricing appears consistent with scenarios in which Anthropic may need to address investment requirements stemming from high inference costs. Market indicators nonetheless suggest that, despite the challenges, there is moderate optimism that Anthropic's valuation could reach certain targets by the end of the year.

Attention is now on developments from Amazon regarding strategies to lower AI inference costs, which could influence broader market perceptions. Any announcements from Anthropic concerning funding rounds or partnerships could also shift market pricing. Observers will be watching closely for strategic initiatives or technological advances that could alter the current cost dynamics of AI deployment.