Sam Altman’s RAM Comments Explained: Why AI Data Centres Want More Memory

Sam Altman has sparked debate about the future of personal computing after suggesting that allocating large amounts of the world's limited memory supply to individual computers is inefficient compared with using it in cloud data centres.

Speaking on the 29 October 2026 episode of the Cheeky Pint podcast, hosted by Stripe co-founder John Collison, the OpenAI CEO questioned the assumption that running artificial intelligence (AI) models locally on personal computers would reduce demand for large-scale computing infrastructure.

Altman argued that memory installed in personal computers often sits unused for much of the time, whereas cloud infrastructure can distribute workloads across shared resources and keep memory more consistently occupied.

He speculated that personal computer memory was being used only around five per cent of the time, thus suggesting that it would be more efficiently allocated to shared infrastructure. His comments have sparked critique from technology enthusiasts, especially in a time when consumers increasingly face concerns about rising memory prices and hardware availability.

The underlying question is worth examining: is cloud infrastructure genuinely a more efficient way to use memory, and does that make local AI less practical?

What Did Sam Altman Mean by His RAM Comments?

Altman's comments come amid a broader conversation about the global compute shortage and the challenges of scaling infrastructure to meet growing AI demand.

There was a discussion around the idea of open-source AI models allowing more people and companies to run inference on their own machines and therefore be less reliant on big cloud-based systems.

Inference is the process of taking a trained AI model and using it to make an answer, prediction or other output. It can take place locally on a device or remotely within a data centre. Altman challenged the idea that local inference would automatically reduce demand for centralised computing.

His argument was that millions of individual computers could each hold substantial amounts of DRAM while using relatively little of their available capacity for AI workloads. Cloud providers, meanwhile, can share computing resources across multiple customers.

For example, a personal computer with 32GB of RAM might spend much of its time performing relatively light tasks, while a cloud provider can allocate resources across customers with different usage patterns.

There is a reasonable efficiency argument here, but Altman's five per cent estimate should be treated as a conversational illustration rather than an independently established measurement of PC memory utilisation.

It is also important to distinguish between memory sitting unused and memory serving its intended purpose. A computer doesn’t need to process data continuously for its RAM to be useful.

Why Do AI Data Centres Need So Much Memory?

AI models require a lot of memory to store the parameters and information needed to process queries. The values that a model learns during training and that it uses to generate outputs are called parameters. Usually, the larger the model, the more memory it requires. Also, the more users and requests you are handling at the same time, the more memory you will need.

For example, AI systems have to deal with individual user requests, conversation histories and intermediate processing data. This relates to the key-value (KV) cache, which stores information about prior tokens to make it easier for models to handle later parts of a conversation.

This means that longer conversations, larger context windows and more concurrent users can increase memory demand.

Two types of memory are especially relevant to AI infrastructure:

• Dynamic random-access memory (DRAM): Standard working memory found in personal computers, servers and other computing devices.
• High-bandwidth memory (HBM): A kind of DRAM that’s designed for quickly moving huge amounts of data between memory and processors, especially AI accelerators.

HBM provides more bandwidth because of the stacked memory dies (vertical stack), which is very important for demanding AI workloads.

But HBM and standard DRAM aren’t to be used interchangeably. They serve different purposes and require different manufacturing and packaging.

This distinction is important when talking about the memory shortage. Increasing HBM production doesn’t automatically resolve constraints affecting conventional DRAM used in personal computers and enterprise servers.

For more on the wider hardware implications, see EM360Tech's coverage of advanced chip packaging for AI hardware.

Is Sam Altman Right About How Computers Use Memory?

Altman’s point is technically true: Cloud infrastructure can increase resource usage by pooling computing capacity instead of each organisation having to maintain its own hardware.

To share capacity among customers, cloud providers use virtualisation, workload scheduling and resource allocation. This is especially effective when workloads have different peak times.

Think of it like the bed of nails demonstration. A person's weight is spread over a much larger surface by hundreds of nails, rather than being concentrated in one spot. The same principle is applied to computing resources in cloud infrastructure. Instead of putting the whole workload on individual systems, workloads are distributed across available capacity.

Rather than each organisation purchasing infrastructure sufficient for its peak computing needs, a cloud provider can share resources between customers with differing usage patterns. This is called statistical multiplexing. But memory utilisation is more involved than simply measuring how busy it is.

Idle memory isn’t necessarily a waste.

RAM on a personal computer supports operating systems, applications and background processes. Enough memory also helps devices not rely too much on slower storage when switching between tasks.

Likewise, cloud data centres cannot always re-allocate memory when a workload becomes idle. You might have to have the AI models loaded and ready to respond to requests. Memory can also be provisioned for certain workloads or customers to meet performance and service level objectives.

Data centres need spare capacity to handle unexpected demand and to keep spare. Providers that are working at maximum utilisation all the time may not be able to respond effectively to sudden increases in activity.

So the better distinction is between shared resource capacity and always occupied memory. Cloud infrastructure can improve utilisation by pooling resources but this doesn’t mean that every gigabyte of memory in a data centre is productively used all the time.

Cloud and local computing serve different purposes

Altman’s argument also doesn’t take into account the different reasons that organisations might use local computing.

A business laptop isn’t meant to operate as a small cloud server. It supports applications, communications, data processing and other day-to-day tasks that don't necessarily have to be moved to a data centre.

AI is the same. Cloud infrastructure offers scalability and access to powerful models, but local inference can provide benefits in privacy, latency, resilience and operational control.

For organisations that manage sensitive information, processing data locally can help reduce the need to transmit data to external services.

Industrial environments are also a good example. An AI system that monitors equipment on a production line might have to identify faults instantly, even if the internet is unavailable. Running inference locally can help keep things going without relying on a remote data center.

There are tradeoffs however. Local hardware places limits on the size and complexity of models that can be deployed. Organisations also need to factor in the cost of equipment, maintenance, electricity and upgrades.

Cloud inference allows you to access larger models with lower upfront hardware investment . But it comes with ongoing service costs and dependence on external infrastructure .

No single method is always more efficient. The right choice depends on the workload and the operational requirements of the organisation.

Why Hybrid AI Infrastructure May Be the Better Approach

For businesses using both local and cloud inference together often makes more sense than picking just one option. Smaller AI models can take care of tasks right on the device. At the time more intensive AI jobs can be sent to larger models running in the cloud.

For instance, a company might use a model on a device to sort documents, summarise internal messages or handle regular service requests. When the job gets more complex it can be passed over to a cloud-based model, for analysis.

This approach helps cut down on cloud use while still letting organisations tap into stronger computing power. It also gives them room to adapt as AI models grow and hardware improves.

Instead, companies should think of local and cloud AI as two pieces that need to mesh together and work in concert within a single computing environment. The real difficulty lies in figuring out where each task should go while balancing speed, cost, security and memory needs.

What OpenAI's Memory Agreements Tell Us About AI Demand

Altman’s comments also need to be understood in the context of OpenAI’s infrastructure requirements.

The company has been working with semiconductor manufacturers to secure the memory capacity needed to support its expanding AI operations. In October 2025 OpenAI announced partnerships with Samsung Electronics and SK hynix as part of its Stargate initiative.

According to OpenAI’s announcement, the companies were targeting production to 900,000 DRAM wafer starts per month through an accelerated capacity rollout.

A wafer start refers to a semiconductor wafer entering the manufacturing process. It doesn’t represent a memory module or a completed chip ready for installation in a data centre.

The target also should not be interpreted as 900,000 memory products being delivered exclusively to OpenAI every month.

Instead the announcement demonstrates the scale of manufacturing expansion and supply chain cooperation being pursued to support AI infrastructure.

For memory manufacturers long-term agreements provide visibility into future demand and can help justify investment in additional production capacity. However, for technology buyers, the increasing competition for memory can create procurement challenges.

The implications extend beyond AI companies. Personal computers, smartphones, enterprise servers and other devices all depend on the broader semiconductor ecosystem.

Enterprises could face higher hardware costs and more uncertainty in planning upgrades as manufacturers respond to the increasing demand for AI memory.

What Does Sam Altman's RAM Comments Mean for Enterprise Infrastructure?

What Altman is talking about is a real problem for the tech industry: How do you share increasingly valuable computing resources now that AI is taking off?

Cloud infrastructure offers advantages in resource pooling, scalability and utilisation. These characteristics make it important for organisations running large models or serving substantial numbers of users.

However, maximising cloud utilisation doesn’t automatically make local computing unnecessary.

Personal computers serve a much wider range of purposes than AI inference, while local models offer advantages in privacy, latency, resilience and control.

For enterprise technology leaders, the more important consideration is how to distribute workloads across available infrastructure.

This means evaluating which tasks require large cloud-based models, which can be handled locally and where hybrid deployments could improve performance or reduce costs.

It also means recognising that memory availability is becoming a strategic procurement consideration.

Ultimately, the future of enterprise AI is unlikely to be defined by local or cloud computing alone. It will depend on how effectively organisations combine both approaches to meet their operational requirements.

Altman's comments may have started as a debate about personal computer memory, but they raise a much bigger question: who should control the computing resources that increasingly underpin modern business?

For more insights into the technologies shaping enterprise AI, explore EM360Tech's AI coverage for the latest news, trends and expert perspectives.