Deploying large language models outside of cloud infrastructure has long been constrained by memory limitations. Even when compute resources are available, moving large parameter matrices between CPU and GPU memory introduces latency and inefficiency. These constraints have shaped how models are developed, tested, and deployed in practice.
Unified memory architecture changes that constraint. By placing the processors on one physical pool of memory, it lets a single machine hold models that previously required specialized hardware. The examples here use Apple silicon, where the CPU, GPU, and Neural Engine all address the same memory. That shift matters most in workflows where privacy, latency, and cost control decide what runs where.
What unified memory changes
Traditional systems separate memory into distinct pools. CPUs access system RAM, while GPUs rely on their own dedicated video memory, or VRAM. When a model runs across both, data must be copied from one region to the other across the link that joins them. The copy costs time, and the GPU can only work on what fits inside its own VRAM.
Unified memory architecture consolidates these resources into a single shared memory space. Both processors read the same physical bytes, so the model does not need a second copy to be usable by the GPU. The result is a system where:
- Data movement is minimized, because the CPU and GPU address the same memory.
- Model capacity is bounded by the whole memory pool rather than by a separate VRAM budget.
- A large model can load once into a single addressable space.
For large language models this matters because weights often run to tens of gigabytes. Removing the separate VRAM budget can decide whether a model fits on one device at all. Figure 1 contrasts the two layouts and shows which step the shared pool removes.
One qualification is worth making early. Sharing a pool removes the copy that a discrete GPU forces on you, but it does not make memory free or infinitely fast. Apple describes a unified memory model in which the CPU and GPU share system memory, while still exposing storage modes that govern how each processor reaches it. Capacity, bandwidth, and compute remain three separate limits, and the rest of this article treats them separately.
Figure 1 · Memory architecture
Two ways to give a model its memory
A discrete GPU keeps its own VRAM, so the model is copied across the Peripheral Component Interconnect Express (PCIe) link before it can run. A unified memory system keeps one physical pool that the CPU and GPU both address, which removes that copy and the separate VRAM ceiling. Capacity, bandwidth, and compute limits still apply.
Scroll or swipe the diagram sideways to see all of it.
Discrete GPU view: the model weights are copied from system RAM across a narrow PCIe link into the GPU’s separate VRAM. With a 70B model the weights exceed a typical 24 GB VRAM budget, so the copy cannot complete without offloading or additional GPUs.
What unified memory removes
- The host-to-device copy across PCIe, because weights already sit in shared memory.
- The separate VRAM ceiling, so model size is bounded by one larger pool instead of a small VRAM budget.
What stays the same
- Total capacity is finite, so very large models can still overflow the pool.
- Bandwidth and compute set a ceiling on speed, and access characteristics still differ by processor.
Two different things called unified memory
The phrase describes more than one architecture, and the difference changes what can be expected. On Apple silicon it is a physical property of the chip. The CPU and GPU sit in one package beside a single pool of memory, and both read the same bytes in place.
NVIDIA uses the same words for a different mechanism. CUDA Unified Memory presents one virtual address space across host and device, while the physical memories stay separate. Pages migrate between them when a processor touches data that currently lives elsewhere, which is convenient to program against but still moves bytes. Newer coherent designs narrow the gap, so the label alone does not tell you whether a copy happens.
This article describes the Apple silicon case throughout. Where it says unified memory, it means one physical pool shared by the processors on the same chip.
Hardware design in the Apple M-series
Modern implementations of unified memory are most visible in systems built on Apple M-series chips. These system-on-chip designs integrate CPU cores, GPU cores, and specialized accelerators into a single package connected to a shared memory pool.
Key characteristics include:
- Bandwidth that depends on the tier. Apple's published figures for the M4 generation range from 120 GB/s on the base M4 to 273 GB/s on the M4 Pro, and up to 546 GB/s on the M4 Max used in the example below. Bandwidth belongs to the individual chip.
- Shared access. Every compute unit reads the same pool, though access characteristics still differ by unit.
- Dynamic allocation. Memory is assigned according to workload demand rather than split in advance between processors.
That range is the point. Unified memory does not confer bandwidth on its own, because each chip supplies whatever its own memory system supports. The top M4 Max reaches 546 GB/s while the base M4 sits near a fifth of that, and both are unified memory designs.
This architecture lets a large model load once and be read by every compute unit without a separate VRAM copy. It also removes the ceiling that limited GPU memory imposes on discrete setups. Weights are only part of the total, though. The interactive estimate in Figure 2 separates the model weights from the memory that the runtime, the operating system, and the KV cache also claim from the same pool. That cache holds the attention keys and values for every token already in the conversation, so it grows as the context does.
Figure 2 · Memory budget
What fills unified memory during inference
Model weights are the largest item, but the KV cache, runtime buffers, and the memory that the operating system and other apps use also count against the same pool. Adjust the settings and watch the budget fill.
Estimated total 84.6 GB of 128 GB unified memory
Estimates use decimal GB (1 GB = 1,000,000,000 bytes). Memory sold as GB is often binary GiB, which is a few percent larger. Weights = parameters × bits ÷ 8, which is a floor. Quantized formats also store a scale for every block of weights, so a real 8-bit file runs a few percent larger and a real 4-bit file 12 to 20 percent larger. Since the estimates in this article use the floor, read them as a lower bound on memory and an upper bound on speed. KV cache is computed from each model's layers and grouped-query attention. Runtime buffers and the OS reserve are approximate and vary by framework and workload.
Why this matters for LLM deployment
Unified memory addresses one of the hardest problems in local deployment, which is fitting the model into memory. Whether performance stays usable is a separate question, and Figure 3 shows why a model that fits can still generate text slowly. In practical terms, a shared pool enables:
- Single-device inference for large models.
- Reduced reliance on cloud infrastructure.
- No sharding or layer offloading, because the model is not split across separate memory regions.
- Simpler deployment pipelines, with no memory partitioning strategy to maintain.
These properties matter most where data cannot easily leave the local environment. Healthcare and biomedical research are the clearest examples, and the same reasoning applies to finance, legal analytics, and enterprise systems handling sensitive records.
Figure 3 · Capacity, bandwidth, compute
Three limits that decide local inference
Pick a machine and a model. Each panel answers one question. The highlighted panel shows the limit that binds first, which is why a model that fits can still generate text slowly.
Does the model fit?
The model needs about 76.6 GB and this machine offers about 120 GB after a generic OS reserve, leaving room to load on a single device. macOS also caps how much of the pool the GPU may wire, about three quarters of it by default.
How fast can weights stream?
Each token streams about 70.6 GB of weights. At 546 GB/s that caps single-stream speed near 7.73 tokens per second.
Is compute the limit?
Generating one token does only a couple of math operations per byte of weights read, so memory bandwidth is reached before the processor is. Serving many requests at once raises the compute share.
Estimates for single-stream generation in decimal GB and GB/s. Token ceiling = memory bandwidth ÷ weight bytes, the upper bound when weights dominate reads at a moderate context length. Real systems land below this ceiling. The run described in this article reached about 81 percent of it. Mixture-of-experts models stream only their active experts, so they can run faster than their total size suggests.
Example: Deploying a 70B model locally
UnifiedMemory
How unified memory lets one Apple M-series Mac run a 70B language model locally. The model is quantized, loaded once into a shared pool, and served entirely on device for privacy, lower cost, and energy efficiency.
Unified Memory. How unified memory lets one Apple M-series Mac run a 70B language model locally. The model is quantized, loaded once into a shared pool, and served entirely on device for privacy, lower cost, and energy efficiency. Key topics covered: Unified memory architecture, Separate memory pools, Memory in use with the model loaded, 8-bit quantized 70B, Memory bandwidth, Setup and monitoring, Compute utilization, Inference throughput.
A concrete run makes the trade-offs visible. An 8-bit quantized version of Llama 3.3 70B was deployed on a MacBook Pro with an M4 Max chip and 128 GB of unified memory. Quantizing to 8 bits stores each weight in a single byte rather than the two a 16-bit model uses, which roughly halves what the weights occupy. The figures below describe a single observed run on one machine, and they are reported as such.
Measurement context
- Machine
- MacBook Pro, Apple M4 Max, 16-core CPU (4 efficiency, 12 performance), 128 GB unified memory
- Model
- Llama 3.3 70B, 8-bit quantized
- Runtime
- LM Studio 0.3.16, monitored with mactop
- Prompt
- A single request, "Write a 500 word story", producing 640 tokens and stopping on the end-of-sequence token
- Result
- 6.26 tokens per second, 1.27 seconds to the first token, 91.03 GB of 128 GB resident
Not recorded: the macOS build, the batch size, and any repeat runs. This is one observation of one prompt on one machine, so read it as a well-documented data point and not as a reproducible average. Every measured figure quoted in this section is legible in Figures 4 to 6.
Setup and monitoring
System performance was monitored using mactop, a lightweight terminal monitor. Baseline resource usage was recorded before loading the model, and Figure 4 shows that starting point. The 19.04 GB already in use is the operating system and everything else that was open, which is memory the model never gets.
Memory behavior
After loading the model:
- Unified memory usage increased to roughly 90 GB.
- This represented about 70 percent of total available memory.
- The full model loaded into the shared pool in one piece.
Because memory is shared, there was no need to partition the model across devices or offload layers dynamically. The gap between the roughly 70 GB the weights occupy and the roughly 90 GB actually resident is the rest of the categories from Figure 2, though not at that figure’s default sizes. On this machine the operating system and open apps alone accounted for 19.04 GB, against the 8 GB the figure assumes as a generic reserve. The screenshots make the arithmetic legible. LM Studio reports 71.28 GB for itself in Figure 6, the machine was already using 19.04 GB before the model loaded in Figure 4, and the two together land within a gigabyte of the 91.03 GB that Figure 5 records for the whole system.
Compute utilization
During inference:
- GPU utilization reached 100 percent, which reports the share of time the GPU was busy and says nothing about how much of its arithmetic throughput was in use.
- CPU usage remained relatively low.
- All model layers ran from the shared pool, with no host-to-device copy across PCIe, though every weight still streams from that pool once per token.
Figure 5 records the GPU at 100 percent against a CPU below 4 percent, so the GPU clearly does nearly all of the work. That reading is easy to misinterpret. Utilization of this kind measures the share of time the processor was busy, which is a much looser thing than the share of its arithmetic throughput in use. A GPU that is waiting on memory still counts as busy. During single-stream generation most of that busy time goes on waiting for weights to arrive, so the number is consistent with a memory-bound workload and is no evidence of a compute-bound one. Figure 3 separates the two readings.
Performance
The system achieved an inference speed of approximately 6.26 tokens per second, shown in Figure 6. That is modest next to a distributed cluster, and notable for a portable machine running a 70B model at all. The same panel separates the two phases of the request. The prompt took 1.27 seconds to process before the first token appeared, and generation then proceeded at an average of 6.26 tokens per second. Reading a prompt is parallel work over many tokens at once, while producing the answer is a sequential loop that repeats for every token.
The number also lands about where the hardware says it should. Generating one token requires reading every weight once, so an 8-bit 70B model moves roughly 70 GB of weights per token. At 546 GB/s that traffic alone caps single-stream output near 7.7 tokens per second, before any other cost. The measured 6.26 tokens per second is close to 81 percent of that ceiling, which is the signature of a workload limited by memory bandwidth rather than by arithmetic. Adding compute would not move this number much. Adding bandwidth would.
It also shows what the architecture is buying here. A high-end consumer graphics card has roughly twice the memory bandwidth of this machine and would generate faster on any model small enough to fit its 24 GB of video memory. A 70B model at 8-bit is not one of them. The advantage on offer is capacity rather than speed, and capacity is what decides whether the model runs on the device in front of you at all.
Practical implications for data science
Unified memory simplifies several aspects of working with large models:
Model prototyping
Large models can be tested locally without complex sharding or offloading strategies. This shortens iteration cycles and reduces engineering overhead.
Privacy-sensitive workflows
Keeping inference local avoids transmitting sensitive data to external services. This is particularly relevant for regulated domains.
Cost management
Running models locally removes ongoing inference costs associated with cloud APIs or GPU instances. The trade-off shifts toward upfront hardware investment.
Energy efficiency
Moving data costs energy, so removing a copy step removes one source of it. The monitor does report power, rising from 0.35 W at rest in Figure 4 to about 30 W during generation in Figure 5, almost all of it the GPU. What the run does not include is the same model on a discrete GPU for comparison, so those numbers describe this machine rather than a saving attributable to unified memory. Treat the efficiency argument as directional until such a comparison exists. It would matter most in sustained workloads and institutional deployments, where small per-token differences accumulate.
Trade-offs and where it fits
Unified memory changes which models fit, and several constraints continue to shape the result.
- Memory is still finite. Extremely large models may still exceed available capacity.
- Capacity is not speed. A large pool decides which models load. The chip's bandwidth decides how quickly they run. For single-stream generation, bandwidth is usually the limit that binds first.
- Bandwidth is lower than a high-end discrete GPU's. The pool is far larger, and each byte in it streams more slowly than it would from dedicated video memory.
- Throughput is limited by device scale. Single-device setups cannot match distributed systems.
- Hardware lock-in. Optimizations are often tied to specific architectures.
These factors decide where a workload belongs. Local inference on unified memory is strongest when the model must stay on the machine and a few tokens per second is fast enough. Cloud infrastructure still wins on raw throughput and on models too large for any single device.
Key Takeaways
128 GB of unified memory holds an 8-bit Llama 3.3 70B, which a 24 GB discrete GPU cannot load whatever its speed. The advantage on offer is the set of models that run at all on the device in front of you.
Generating one token reads every weight once, so roughly 70 GB of 8-bit weights streaming at 546 GB/s caps single-stream output near 7.7 tokens per second. The run measured 6.26, about 81 percent of that ceiling.
Apple silicon shares one physical pool between processors in the same package. CUDA Unified Memory shares a virtual address space while the physical memories stay separate, so pages still move when a processor touches data held elsewhere.
The monitor reported the GPU at 100 percent with the CPU near 4 percent. That reading counts the time the processor was busy, and a processor waiting on memory still counts as busy.
The runtime reported 71.28 GB for itself, the machine already held 19.04 GB before the model loaded, and the system reached 91.03 GB. The KV cache, runtime buffers, and the operating system claim the same pool.
The 6.26 tokens per second and 1.27 seconds to the first token come from a single short prompt that produced 640 tokens on one machine, so the energy argument is directional, and repeat runs with a discrete-GPU comparison would sharpen it.
Data & License
No third-party dataset. The benchmarks and measurements shown are the author’s own, recorded on the hardware and software described in this article.
© 2025 Philip Sarajlic. All rights reserved for the article’s original text and figures.
Code examples in this article are licensed under the Common Public Attribution License Version 1.0 (CPAL-1.0), an OSI-approved copyleft license based on the Mozilla Public License 1.1. Initial Developer: Philip Sarajlic.
Attribution required by CPAL Exhibit B: © 2025 Philip Sarajlic · “Based on code by Philip Sarajlic” · philipsarajlic.com · no graphic image. This attribution must be displayed in Larger Works.
Modifications must be released in Source Code form under CPAL-1.0. Making the code usable by anyone other than you over a network is External Deployment under the license and is treated as distribution, so the Source Code must be made available to those users.
Full text: opensource.org/license/cpal-1-0 (SPDX identifier CPAL-1.0)
