The “2x” in NVMe memory tiering is neither a magic multiplier nor just a marketing promise. Broadcom published that doubling across several benchmarks at a 1:1 DRAM:NVMe ratio, with a cost ranging from none in one VDI test to 5–10% under HammerDB. The useful question is not “does it work?” but “do my working set, NVMe devices, and operating constraints resemble the documented conditions?”
At Nordwind Logistics, cl-fret-dens-01 is a 4-host ESA cluster considered for consolidation. Project PRJ-2026-DENS-01 shares one budget across DRAM, storage NVMe, and tiering NVMe. None of those three lines can be counted twice.
These figures are not Nordwind measurements
No local lab was run. The cited results were published by Broadcom with their benchmarks. They make the use case credible, but eligibility and application cost must be verified on a test platform or during a controlled pilot.
TL;DR
- Memory tiering has been GA since ESX 9.0; its Technical Preview phase belonged to vSphere 8.0 U3.
- At the recommended 1:1 ratio,
tier_size_pct=100, an NVMe device equal to DRAM provides 2x total memory capacity. - Broadcom publishes 2x density with workload-dependent cost; active memory ≤ 50% of DRAM and no overcommit are the first admission gates.
- The tiering NVMe must remain local, compatible, and unclaimed by vSAN.
An ESX feature, not a vSAN feature
Memory Tiering over NVMe adds a second memory class to an ESX host:
- Tier 0: DRAM, for primary memory;
- Tier 1: local NVMe, for VM memory pages placed in the secondary tier.
NVMe pages are used only for VM memory allocations. The mechanism is not a datastore, vSAN swap, or a replacement for overcommit or ballooning. It is deactivated by default.
The timeline prevents a common misunderstanding. vSphere 8.0 U3 offered a Technical Preview that was not intended for production and was not fully tuned. vSphere 9.0 documentation establishes GA. VCF 9.1 adds configuration without reboot, removes limitations on some VM profiles, and adds software mirroring to a second NVMe device.
DOCUMENTÉ — the starting ratio
Broadcom recommends a 1:1 DRAM:NVMe ratio. With tier_size_pct=100, the NVMe tier equals DRAM and total memory capacity becomes 2x. The parameter accepts 1 to 400, but no density result was found for another ratio.
The tier partition cannot exceed 4 TB. At the recommended ratio, NVMe must be at least as large as system DRAM. A 9.1 announcement says up to 4x more memory can be available per host, but no reviewed source confirms that this figure maps exactly to tier_size_pct=400. That connection remains a RECONSTITUTION, not a sizing rule.
The published 2x, with its cost
Every result below comes from a 2025 Broadcom study and uses the 1:1 ratio. CPU and NVMe models and network configuration did not appear in the loaded page, so applying the results directly to Nordwind hardware would be incorrect.
| Broadcom benchmark | DRAM only | Memory Tiering 1:1 | Published cost |
|---|---|---|---|
| Login Enterprise, 3-host vSAN cluster | 300 sessions | 600 sessions | no loss announced versus the all-DRAM equivalent |
| VMmark 3.1 | 3 tiles | 6 tiles | 5% in the 1 TB versus 1 TB comparison |
| DVD Store / Oracle, 1 ESX host | 4 VMs | 8 VMs | less than 5% in the 1 TB versus 1 TB comparison |
| HammerDB, SQL and MySQL | reference density | 2x | 5–10% |
This table does not say that every application accepts 2x as many VMs. It shows that Broadcom observed the result across four benchmark families at 1:1, with different sensitivity. The 5–10% HammerDB loss matters as much as the doubling: it is a decision order, not a production database promise.
Broadcom also recommends NVMe read latency below 200 µs for best performance. That is a published target, not a local measurement.
Active memory is the real criterion
TechDocs says most workloads typically maintain 10 to 30% active memory. For planning, the requirement is stricter: a host’s active memory must never have exceeded 50% of DRAM before activation.
Cold pages start moving to NVMe when consumption reaches about 80% of DRAM. This explains the model: secondary capacity absorbs pages that do not need immediate DRAM latency while the active working set retains priority in Tier 0.
À_VALIDER_EN_LAB — workload eligibility
Average active memory is not enough. Check peaks, application latency, and batch periods. A workload that exceeds 50% active DRAM, or whose working set changes sharply, must not be declared eligible from a Broadcom benchmark alone.
A lightweight qualification works even without a permanent lab: extract active-memory history, select representative VMs, establish a DRAM-only baseline, then repeat the scenario on a temporary test platform. Compare density, application latency percentile, CPU, active memory, and NVMe read latency.
RÉSULTAT_ATTENDU: at 1:1, the cluster provides 2x total memory capacity and stays within the pilot’s performance budget. Until that pilot exists, the record must say “verify on a test platform” rather than carry an estimated value.
One drive can have only one role
The link with vSAN ESA is physical, not functional. An NVMe device already claimed by a datastore, including vSAN, cannot serve memory tiering. The drive must remain unclaimed, local, and compatible. Broadcom documents three paths: reserve it before vSAN claims storage, exclude it from claiming, or remove a device from the storage pool after migrating data.
For cl-fret-dens-01, the checklist becomes:
- inventory slots and certified devices;
- reserve a tiering NVMe distinct from storage-pool NVMe devices;
- verify that its capacity covers DRAM at 1:1 without exceeding the 4 TB partition;
- verify active memory ≤ 50% of DRAM and prohibit overcommit;
- decide whether a second NVMe is reserved for software mirroring;
- qualify read latency and endurance from vendor data, then by test.
In 9.1, the second mirrored device must have capacity equal to or greater than the first. Broadcom says mirroring has no performance impact and prevents tier failure regardless of NVMe lifetime. That product assertion also deserves verification on the selected hardware.
Exclusions that change operations
Memory tiering is incompatible with hot-plug, prepare to remove, suspend-to-memory, Quick Boot, Intel Optane PMem, NVDIMM-N, and NVMe over fabric or Ethernet. Tiering must not be described as an overcommit lever either: TechDocs explicitly says not to use memory overcommit when it is active.
HYPOTHÈSE_DE_DESIGN — Nordwind guardrail
Nordwind starts only at 1:1, on hosts that stayed below 50% active memory, without overcommit, and with separate devices for vSAN. Any other ratio becomes a separate experiment.
This discipline extends VCF 9.1 infrastructure efficiency and TCO: density helps only when DRAM and slot savings remain greater than qualification, NVMe, and operating costs.
Conclusion
Start at 1:1
The published 2x belongs to the recommended ratio, not an extrapolation from 1 to 400.
The working set decides
Active memory and application latency determine real eligibility.
One slot, one role
A tiering NVMe no longer contributes to the ESA storage pool.
The 2x result is strong enough to open a project, not close validation. Once memory devices are reserved, recalculate what remains for storage: ESA versus OSA capacity planning, what really changes.
Primary sources: vSphere 9.1 Memory Tiering, best practices, activation with vSAN, and Broadcom performance study.
Get the next one by email
New articles and series, sent when they are published. No other mail.
