Headroom

GPUs & accelerators

93 data center, workstation, consumer and Apple devices with vendor datasheet figures. Memory bandwidth sets how fast tokens come out; memory size sets what fits.

93 GPUs
Apple M3 Ultra (80-core GPU, up to 512GB)Apple · Apple · 2025512 GB819 GB/s28––≈ 372B––
Apple M5 Ultra (80-core GPU, up to 512GB)Apple · Apple · 2026512 GB1,229 GB/s128––≈ 372B––
AMD Instinct MI355X 288GBAMD · Data center · 2025288 GB8,000 GB/s2,5175,03310,066≈ 209B1,400 W$8.32
AMD Instinct MI350X 288GBAMD · Data center · 2025288 GB8,000 GB/s2,3074,6149,228≈ 209B1,000 W$4.13
NVIDIA GB300 (Blackwell Ultra, per GPU in GB300 NVL72)NVIDIA · Data center · 2025279 GB8,000 GB/s2,5005,00015,000≈ 202B1,400 W$18.00
NVIDIA B300 SXM (HGX B300)NVIDIA · Data center · 2025270 GB8,000 GB/s2,2504,50013,500≈ 195B1,100 W$8.99
AMD Instinct MI325X 256GBAMD · Data center · 2024256 GB6,000 GB/s1,3072,615–≈ 185B1,000 W$3.11
AMD Instinct MI300X 192GBAMD · Data center · 2023192 GB5,300 GB/s1,3072,615–≈ 138B750 W$2.99
Apple M2 Ultra (76-core GPU, up to 192GB)Apple · Apple · 2023192 GB800 GB/s27––≈ 138B––
Google TPU7x (Ironwood)Google · Accelerator · 2025192 GB7,380 GB/s2,3074,614–≈ 138B–$12.00
NVIDIA GB200 (Blackwell, per GPU in GB200 NVL72)NVIDIA · Data center · 2024186 GB8,000 GB/s2,5005,00010,000≈ 134B1,200 W$13.25
NVIDIA B200 SXM (HGX B200)NVIDIA · Data center · 2024180 GB8,000 GB/s2,2504,5009,000≈ 130B1,000 W$7.99
NVIDIA GH200 Grace Hopper 144GB HBM3e (GPU part)NVIDIA · Data center · 2024144 GB4,900 GB/s9891,979–≈ 103B1,000 W–
NVIDIA H200 SXM 141GBNVIDIA · Data center · 2024141 GB4,800 GB/s9891,979–≈ 101B700 W$5.29
NVIDIA H200 NVL 141GBNVIDIA · Data center · 2024141 GB4,800 GB/s8361,671–≈ 101B600 W–
NVIDIA DGX Spark (GB10, 128GB unified)NVIDIA · Workstation · 2025128 GB273 GB/s213214500≈ 91.7B140 W–
NVIDIA Jetson AGX Thor T5000 128GBNVIDIA · Edge · 2025128 GB273 GB/s2595171,035≈ 91.7B130 W–
AMD Instinct MI300A APU 128GB (unified HBM)AMD · Data center · 2023128 GB5,300 GB/s9811,961–≈ 91.7B550 W–
AMD Instinct MI250X 128GBAMD · Data center · 2021128 GB3,277 GB/s383383–≈ 91.7B500 W–
AMD Ryzen AI Max+ 395 (Strix Halo, Radeon 8060S, 128GB unified)AMD · Consumer · 2025128 GB256 GB/s5959–≈ 91.7B120 W–
Intel Gaudi 3 128GB (HL-325L OAM)Intel · Accelerator · 2024128 GB3,700 GB/s1,6781,678–≈ 91.7B900 W–
Intel Data Center GPU Max 1550 128GBIntel · Data center · 2023128 GB3,277 GB/s8391,678–≈ 91.7B600 W–
Apple M3 Max (40-core GPU, up to 128GB)Apple · Apple · 2023128 GB400 GB/s14––≈ 91.7B––
Apple M4 Max (40-core GPU, up to 128GB)Apple · Apple · 2024128 GB546 GB/s18––≈ 91.7B––
Apple M5 Max (40-core GPU, up to 128GB)Apple · Apple · 2026128 GB614 GB/s74––≈ 91.7B––
NVIDIA H20 96GB (China export variant)NVIDIA · Data center · 202496 GB4,000 GB/s148296–≈ 68.4B400 W–
NVIDIA GH200 Grace Hopper 96GB HBM3 (GPU part)NVIDIA · Data center · 202396 GB4,000 GB/s9891,979–≈ 68.4B1,000 W$5.85
NVIDIA RTX PRO 6000 Blackwell Server Edition 96GBNVIDIA · Data center · 202596 GB1,597 GB/s5001,0002,000≈ 68.4B600 W$2.49
NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GBNVIDIA · Workstation · 202596 GB1,792 GB/s5001,0002,000≈ 68.4B600 W–
NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition 96GBNVIDIA · Workstation · 202596 GB1,792 GB/s4398781,756≈ 68.4B300 W–
Intel Gaudi 2 96GB (HL-225 OAM)Intel · Accelerator · 202296 GB2,460 GB/s432865–≈ 68.4B600 W$1.10
Google TPU v5pGoogle · Accelerator · 202395 GB2,765 GB/s459459–≈ 67.6B–$4.20
NVIDIA H100 NVL 94GBNVIDIA · Data center · 202394 GB3,900 GB/s8361,671–≈ 66.9B400 W$3.19
NVIDIA H100 SXM 80GBNVIDIA · Data center · 202280 GB3,350 GB/s9891,979–≈ 56.7B700 W$3.99
NVIDIA H100 PCIe 80GBNVIDIA · Data center · 202280 GB2,000 GB/s7571,513–≈ 56.7B350 W$2.89
NVIDIA H800 SXM 80GB (China export variant)NVIDIA · Data center · 202380 GB3,350 GB/s9891,979–≈ 56.7B700 W–
NVIDIA A100 SXM 80GBNVIDIA · Data center · 202080 GB2,039 GB/s312624–≈ 56.7B400 W$2.27
NVIDIA A100 PCIe 80GBNVIDIA · Data center · 202180 GB1,935 GB/s312624–≈ 56.7B300 W$1.82
NVIDIA A800 SXM 80GB (China export variant)NVIDIA · Data center · 202280 GB2,039 GB/s312624–≈ 56.7B400 W$1.18
NVIDIA Jetson AGX Orin 64GBNVIDIA · Edge · 202264 GB205 GB/s4385–≈ 45.0B60 W–
AMD Instinct MI210 64GBAMD · Data center · 202264 GB1,638 GB/s181181–≈ 45.0B300 W$0.93
Apple M4 Pro (20-core GPU, up to 64GB)Apple · Apple · 202464 GB273 GB/s9––≈ 45.0B––
Apple M5 Pro (20-core GPU, up to 64GB)Apple · Apple · 202664 GB307 GB/s37––≈ 45.0B––
NVIDIA L40S 48GBNVIDIA · Data center · 202348 GB864 GB/s362733–≈ 33.3B350 W$1.60
NVIDIA L40 48GBNVIDIA · Data center · 202248 GB864 GB/s181362–≈ 33.3B300 W$1.06
NVIDIA L20 48GB (China export variant)NVIDIA · Data center · 202448 GB864 GB/s120239–≈ 33.3B275 W–
NVIDIA A40 48GBNVIDIA · Data center · 202048 GB696 GB/s150299–≈ 33.3B300 W$0.72
NVIDIA RTX PRO 5000 Blackwell 48GBNVIDIA · Workstation · 202548 GB1,344 GB/s2585161,032≈ 33.3B300 W$0.82
NVIDIA RTX 6000 Ada Generation 48GBNVIDIA · Workstation · 202248 GB960 GB/s364729–≈ 33.3B300 W$1.07
NVIDIA RTX A6000 48GBNVIDIA · Workstation · 202048 GB768 GB/s155310–≈ 33.3B300 W$0.59
AMD Radeon PRO W7900 48GBAMD · Workstation · 202348 GB864 GB/s123123–≈ 33.3B295 W–
NVIDIA A100 SXM 40GBNVIDIA · Data center · 202040 GB1,555 GB/s312624–≈ 27.5B400 W$1.92
NVIDIA V100 SXM2 32GBNVIDIA · Data center · 201832 GB900 GB/s125––≈ 21.6B300 W$0.89
NVIDIA RTX PRO 4500 Blackwell 32GBNVIDIA · Workstation · 202532 GB896 GB/s202404809≈ 21.6B200 W$0.92
NVIDIA RTX 5000 Ada Generation 32GBNVIDIA · Workstation · 202332 GB576 GB/s261522–≈ 21.6B250 W$0.49
NVIDIA GeForce RTX 5090 32GBNVIDIA · Consumer · 202532 GB1,792 GB/s2104191,676≈ 21.6B575 W$0.95
AMD Radeon PRO W7800 32GBAMD · Workstation · 202332 GB576 GB/s9090–≈ 21.6B260 W–
AMD Radeon AI PRO R9700 32GBAMD · Workstation · 202532 GB640 GB/s191383–≈ 21.6B300 W–
Apple M5 (10-core GPU, up to 32GB)Apple · Apple · 202532 GB153 GB/s18––≈ 21.6B––
Google TPU v6e (Trillium)Google · Accelerator · 202432 GB1,638 GB/s9181,836–≈ 21.6B–$2.70
NVIDIA L4 24GBNVIDIA · Data center · 202324 GB300 GB/s121243–≈ 15.8B72 W$0.72
NVIDIA A30 24GBNVIDIA · Data center · 202124 GB933 GB/s165330–≈ 15.8B165 W$0.68
NVIDIA A10 24GBNVIDIA · Data center · 202124 GB600 GB/s125250–≈ 15.8B150 W$1.55
NVIDIA A10G 24GB (AWS)NVIDIA · Data center · 202124 GB600 GB/s70––≈ 15.8B300 W$1.60
NVIDIA RTX PRO 4000 Blackwell 24GBNVIDIA · Workstation · 202524 GB672 GB/s161323645≈ 15.8B145 W$0.57
NVIDIA RTX 4500 Ada Generation 24GBNVIDIA · Workstation · 202324 GB432 GB/s159317–≈ 15.8B210 W–
NVIDIA RTX A5000 24GBNVIDIA · Workstation · 202124 GB768 GB/s111222–≈ 15.8B230 W$0.83
NVIDIA GeForce RTX 4090 24GBNVIDIA · Consumer · 202224 GB1,008 GB/s165330–≈ 15.8B450 W$0.67
NVIDIA GeForce RTX 3090 Ti 24GBNVIDIA · Consumer · 202224 GB1,008 GB/s80320–≈ 15.8B450 W$0.19
NVIDIA GeForce RTX 3090 24GBNVIDIA · Consumer · 202024 GB936 GB/s71284–≈ 15.8B350 W$0.39
NVIDIA TITAN RTX 24GBNVIDIA · Consumer · 201824 GB672 GB/s130261–≈ 15.8B280 W–
AMD Radeon RX 7900 XTX 24GBAMD · Consumer · 202224 GB960 GB/s123123–≈ 15.8B355 W–
Intel Arc Pro B60 24GBIntel · Workstation · 202524 GB456 GB/s99197–≈ 15.8B200 W–
NVIDIA RTX 4000 Ada Generation 20GBNVIDIA · Workstation · 202320 GB360 GB/s107214–≈ 12.9B130 W$0.75
AMD Radeon RX 7900 XT 20GBAMD · Consumer · 202220 GB800 GB/s103103–≈ 12.9B315 W–
NVIDIA A16 (per GPU; 4x 16GB GPUs per board)NVIDIA · Data center · 202116 GB200 GB/s1836–≈ 9.96B63 W–
NVIDIA T4 16GBNVIDIA · Data center · 201816 GB320 GB/s65130–≈ 9.96B70 W$0.69
NVIDIA RTX A4000 16GBNVIDIA · Workstation · 202116 GB448 GB/s77153–≈ 9.96B140 W$0.82
NVIDIA GeForce RTX 5080 16GBNVIDIA · Consumer · 202516 GB960 GB/s113225900≈ 9.96B360 W$0.28
NVIDIA GeForce RTX 5070 Ti 16GBNVIDIA · Consumer · 202516 GB896 GB/s88176703≈ 9.96B300 W$0.24
NVIDIA GeForce RTX 5060 Ti 16GBNVIDIA · Consumer · 202516 GB448 GB/s4795380≈ 9.96B180 W$0.11
NVIDIA GeForce RTX 4080 SUPER 16GBNVIDIA · Consumer · 202416 GB736 GB/s105209–≈ 9.96B320 W–
NVIDIA GeForce RTX 4080 16GBNVIDIA · Consumer · 202216 GB717 GB/s98195–≈ 9.96B320 W$0.24
NVIDIA GeForce RTX 4070 Ti SUPER 16GBNVIDIA · Consumer · 202416 GB672 GB/s88177–≈ 9.96B285 W$0.35
NVIDIA GeForce RTX 4060 Ti 16GBNVIDIA · Consumer · 202316 GB288 GB/s4488–≈ 9.96B165 W$0.13
AMD Radeon RX 9070 XT 16GBAMD · Consumer · 202516 GB640 GB/s195389–≈ 9.96B304 W–
Google TPU v5eGoogle · Accelerator · 202316 GB819 GB/s197393–≈ 9.96B–$1.20
NVIDIA GeForce RTX 5070 12GBNVIDIA · Consumer · 202512 GB672 GB/s62124494≈ 7.04B250 W$0.19
NVIDIA GeForce RTX 4070 SUPER 12GBNVIDIA · Consumer · 202412 GB504 GB/s71142–≈ 7.04B220 W–
NVIDIA GeForce RTX 3080 Ti 12GBNVIDIA · Consumer · 202112 GB912 GB/s68273–≈ 7.04B350 W$0.29
NVIDIA GeForce RTX 3080 12GBNVIDIA · Consumer · 202212 GB912 GB/s61245–≈ 7.04B350 W–
NVIDIA GeForce RTX 3060 12GBNVIDIA · Consumer · 202112 GB360 GB/s26102–≈ 7.04B170 W$0.09
Intel Arc B580 12GBIntel · Consumer · 202412 GB456 GB/s117233–≈ 7.04B190 W–

“Fits at 8-bit” is a rough single-device estimate (weights at 8 bits, ~20% of usable memory kept for the KV cache). Dense TFLOPS, no sparsity. Prices are typical cloud on-demand rates. Pick a row to plan on that GPU.