GPUs & accelerators
93 data center, workstation, consumer and Apple devices with vendor datasheet figures. Memory bandwidth sets how fast tokens come out; memory size sets what fits.
93 GPUs
| Apple M3 Ultra (80-core GPU, up to 512GB)Apple · Apple · 2025 | 512 GB | 819 GB/s | 28 | – | – | ≈ 372B | – | – |
| Apple M5 Ultra (80-core GPU, up to 512GB)Apple · Apple · 2026 | 512 GB | 1,229 GB/s | 128 | – | – | ≈ 372B | – | – |
| AMD Instinct MI355X 288GBAMD · Data center · 2025 | 288 GB | 8,000 GB/s | 2,517 | 5,033 | 10,066 | ≈ 209B | 1,400 W | $8.32 |
| AMD Instinct MI350X 288GBAMD · Data center · 2025 | 288 GB | 8,000 GB/s | 2,307 | 4,614 | 9,228 | ≈ 209B | 1,000 W | $4.13 |
| NVIDIA GB300 (Blackwell Ultra, per GPU in GB300 NVL72)NVIDIA · Data center · 2025 | 279 GB | 8,000 GB/s | 2,500 | 5,000 | 15,000 | ≈ 202B | 1,400 W | $18.00 |
| NVIDIA B300 SXM (HGX B300)NVIDIA · Data center · 2025 | 270 GB | 8,000 GB/s | 2,250 | 4,500 | 13,500 | ≈ 195B | 1,100 W | $8.99 |
| AMD Instinct MI325X 256GBAMD · Data center · 2024 | 256 GB | 6,000 GB/s | 1,307 | 2,615 | – | ≈ 185B | 1,000 W | $3.11 |
| AMD Instinct MI300X 192GBAMD · Data center · 2023 | 192 GB | 5,300 GB/s | 1,307 | 2,615 | – | ≈ 138B | 750 W | $2.99 |
| Apple M2 Ultra (76-core GPU, up to 192GB)Apple · Apple · 2023 | 192 GB | 800 GB/s | 27 | – | – | ≈ 138B | – | – |
| Google TPU7x (Ironwood)Google · Accelerator · 2025 | 192 GB | 7,380 GB/s | 2,307 | 4,614 | – | ≈ 138B | – | $12.00 |
| NVIDIA GB200 (Blackwell, per GPU in GB200 NVL72)NVIDIA · Data center · 2024 | 186 GB | 8,000 GB/s | 2,500 | 5,000 | 10,000 | ≈ 134B | 1,200 W | $13.25 |
| NVIDIA B200 SXM (HGX B200)NVIDIA · Data center · 2024 | 180 GB | 8,000 GB/s | 2,250 | 4,500 | 9,000 | ≈ 130B | 1,000 W | $7.99 |
| NVIDIA GH200 Grace Hopper 144GB HBM3e (GPU part)NVIDIA · Data center · 2024 | 144 GB | 4,900 GB/s | 989 | 1,979 | – | ≈ 103B | 1,000 W | – |
| NVIDIA H200 SXM 141GBNVIDIA · Data center · 2024 | 141 GB | 4,800 GB/s | 989 | 1,979 | – | ≈ 101B | 700 W | $5.29 |
| NVIDIA H200 NVL 141GBNVIDIA · Data center · 2024 | 141 GB | 4,800 GB/s | 836 | 1,671 | – | ≈ 101B | 600 W | – |
| NVIDIA DGX Spark (GB10, 128GB unified)NVIDIA · Workstation · 2025 | 128 GB | 273 GB/s | 213 | 214 | 500 | ≈ 91.7B | 140 W | – |
| NVIDIA Jetson AGX Thor T5000 128GBNVIDIA · Edge · 2025 | 128 GB | 273 GB/s | 259 | 517 | 1,035 | ≈ 91.7B | 130 W | – |
| AMD Instinct MI300A APU 128GB (unified HBM)AMD · Data center · 2023 | 128 GB | 5,300 GB/s | 981 | 1,961 | – | ≈ 91.7B | 550 W | – |
| AMD Instinct MI250X 128GBAMD · Data center · 2021 | 128 GB | 3,277 GB/s | 383 | 383 | – | ≈ 91.7B | 500 W | – |
| AMD Ryzen AI Max+ 395 (Strix Halo, Radeon 8060S, 128GB unified)AMD · Consumer · 2025 | 128 GB | 256 GB/s | 59 | 59 | – | ≈ 91.7B | 120 W | – |
| Intel Gaudi 3 128GB (HL-325L OAM)Intel · Accelerator · 2024 | 128 GB | 3,700 GB/s | 1,678 | 1,678 | – | ≈ 91.7B | 900 W | – |
| Intel Data Center GPU Max 1550 128GBIntel · Data center · 2023 | 128 GB | 3,277 GB/s | 839 | 1,678 | – | ≈ 91.7B | 600 W | – |
| Apple M3 Max (40-core GPU, up to 128GB)Apple · Apple · 2023 | 128 GB | 400 GB/s | 14 | – | – | ≈ 91.7B | – | – |
| Apple M4 Max (40-core GPU, up to 128GB)Apple · Apple · 2024 | 128 GB | 546 GB/s | 18 | – | – | ≈ 91.7B | – | – |
| Apple M5 Max (40-core GPU, up to 128GB)Apple · Apple · 2026 | 128 GB | 614 GB/s | 74 | – | – | ≈ 91.7B | – | – |
| NVIDIA H20 96GB (China export variant)NVIDIA · Data center · 2024 | 96 GB | 4,000 GB/s | 148 | 296 | – | ≈ 68.4B | 400 W | – |
| NVIDIA GH200 Grace Hopper 96GB HBM3 (GPU part)NVIDIA · Data center · 2023 | 96 GB | 4,000 GB/s | 989 | 1,979 | – | ≈ 68.4B | 1,000 W | $5.85 |
| NVIDIA RTX PRO 6000 Blackwell Server Edition 96GBNVIDIA · Data center · 2025 | 96 GB | 1,597 GB/s | 500 | 1,000 | 2,000 | ≈ 68.4B | 600 W | $2.49 |
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition 96GBNVIDIA · Workstation · 2025 | 96 GB | 1,792 GB/s | 500 | 1,000 | 2,000 | ≈ 68.4B | 600 W | – |
| NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition 96GBNVIDIA · Workstation · 2025 | 96 GB | 1,792 GB/s | 439 | 878 | 1,756 | ≈ 68.4B | 300 W | – |
| Intel Gaudi 2 96GB (HL-225 OAM)Intel · Accelerator · 2022 | 96 GB | 2,460 GB/s | 432 | 865 | – | ≈ 68.4B | 600 W | $1.10 |
| Google TPU v5pGoogle · Accelerator · 2023 | 95 GB | 2,765 GB/s | 459 | 459 | – | ≈ 67.6B | – | $4.20 |
| NVIDIA H100 NVL 94GBNVIDIA · Data center · 2023 | 94 GB | 3,900 GB/s | 836 | 1,671 | – | ≈ 66.9B | 400 W | $3.19 |
| NVIDIA H100 SXM 80GBNVIDIA · Data center · 2022 | 80 GB | 3,350 GB/s | 989 | 1,979 | – | ≈ 56.7B | 700 W | $3.99 |
| NVIDIA H100 PCIe 80GBNVIDIA · Data center · 2022 | 80 GB | 2,000 GB/s | 757 | 1,513 | – | ≈ 56.7B | 350 W | $2.89 |
| NVIDIA H800 SXM 80GB (China export variant)NVIDIA · Data center · 2023 | 80 GB | 3,350 GB/s | 989 | 1,979 | – | ≈ 56.7B | 700 W | – |
| NVIDIA A100 SXM 80GBNVIDIA · Data center · 2020 | 80 GB | 2,039 GB/s | 312 | 624 | – | ≈ 56.7B | 400 W | $2.27 |
| NVIDIA A100 PCIe 80GBNVIDIA · Data center · 2021 | 80 GB | 1,935 GB/s | 312 | 624 | – | ≈ 56.7B | 300 W | $1.82 |
| NVIDIA A800 SXM 80GB (China export variant)NVIDIA · Data center · 2022 | 80 GB | 2,039 GB/s | 312 | 624 | – | ≈ 56.7B | 400 W | $1.18 |
| NVIDIA Jetson AGX Orin 64GBNVIDIA · Edge · 2022 | 64 GB | 205 GB/s | 43 | 85 | – | ≈ 45.0B | 60 W | – |
| AMD Instinct MI210 64GBAMD · Data center · 2022 | 64 GB | 1,638 GB/s | 181 | 181 | – | ≈ 45.0B | 300 W | $0.93 |
| Apple M4 Pro (20-core GPU, up to 64GB)Apple · Apple · 2024 | 64 GB | 273 GB/s | 9 | – | – | ≈ 45.0B | – | – |
| Apple M5 Pro (20-core GPU, up to 64GB)Apple · Apple · 2026 | 64 GB | 307 GB/s | 37 | – | – | ≈ 45.0B | – | – |
| NVIDIA L40S 48GBNVIDIA · Data center · 2023 | 48 GB | 864 GB/s | 362 | 733 | – | ≈ 33.3B | 350 W | $1.60 |
| NVIDIA L40 48GBNVIDIA · Data center · 2022 | 48 GB | 864 GB/s | 181 | 362 | – | ≈ 33.3B | 300 W | $1.06 |
| NVIDIA L20 48GB (China export variant)NVIDIA · Data center · 2024 | 48 GB | 864 GB/s | 120 | 239 | – | ≈ 33.3B | 275 W | – |
| NVIDIA A40 48GBNVIDIA · Data center · 2020 | 48 GB | 696 GB/s | 150 | 299 | – | ≈ 33.3B | 300 W | $0.72 |
| NVIDIA RTX PRO 5000 Blackwell 48GBNVIDIA · Workstation · 2025 | 48 GB | 1,344 GB/s | 258 | 516 | 1,032 | ≈ 33.3B | 300 W | $0.82 |
| NVIDIA RTX 6000 Ada Generation 48GBNVIDIA · Workstation · 2022 | 48 GB | 960 GB/s | 364 | 729 | – | ≈ 33.3B | 300 W | $1.07 |
| NVIDIA RTX A6000 48GBNVIDIA · Workstation · 2020 | 48 GB | 768 GB/s | 155 | 310 | – | ≈ 33.3B | 300 W | $0.59 |
| AMD Radeon PRO W7900 48GBAMD · Workstation · 2023 | 48 GB | 864 GB/s | 123 | 123 | – | ≈ 33.3B | 295 W | – |
| NVIDIA A100 SXM 40GBNVIDIA · Data center · 2020 | 40 GB | 1,555 GB/s | 312 | 624 | – | ≈ 27.5B | 400 W | $1.92 |
| NVIDIA V100 SXM2 32GBNVIDIA · Data center · 2018 | 32 GB | 900 GB/s | 125 | – | – | ≈ 21.6B | 300 W | $0.89 |
| NVIDIA RTX PRO 4500 Blackwell 32GBNVIDIA · Workstation · 2025 | 32 GB | 896 GB/s | 202 | 404 | 809 | ≈ 21.6B | 200 W | $0.92 |
| NVIDIA RTX 5000 Ada Generation 32GBNVIDIA · Workstation · 2023 | 32 GB | 576 GB/s | 261 | 522 | – | ≈ 21.6B | 250 W | $0.49 |
| NVIDIA GeForce RTX 5090 32GBNVIDIA · Consumer · 2025 | 32 GB | 1,792 GB/s | 210 | 419 | 1,676 | ≈ 21.6B | 575 W | $0.95 |
| AMD Radeon PRO W7800 32GBAMD · Workstation · 2023 | 32 GB | 576 GB/s | 90 | 90 | – | ≈ 21.6B | 260 W | – |
| AMD Radeon AI PRO R9700 32GBAMD · Workstation · 2025 | 32 GB | 640 GB/s | 191 | 383 | – | ≈ 21.6B | 300 W | – |
| Apple M5 (10-core GPU, up to 32GB)Apple · Apple · 2025 | 32 GB | 153 GB/s | 18 | – | – | ≈ 21.6B | – | – |
| Google TPU v6e (Trillium)Google · Accelerator · 2024 | 32 GB | 1,638 GB/s | 918 | 1,836 | – | ≈ 21.6B | – | $2.70 |
| NVIDIA L4 24GBNVIDIA · Data center · 2023 | 24 GB | 300 GB/s | 121 | 243 | – | ≈ 15.8B | 72 W | $0.72 |
| NVIDIA A30 24GBNVIDIA · Data center · 2021 | 24 GB | 933 GB/s | 165 | 330 | – | ≈ 15.8B | 165 W | $0.68 |
| NVIDIA A10 24GBNVIDIA · Data center · 2021 | 24 GB | 600 GB/s | 125 | 250 | – | ≈ 15.8B | 150 W | $1.55 |
| NVIDIA A10G 24GB (AWS)NVIDIA · Data center · 2021 | 24 GB | 600 GB/s | 70 | – | – | ≈ 15.8B | 300 W | $1.60 |
| NVIDIA RTX PRO 4000 Blackwell 24GBNVIDIA · Workstation · 2025 | 24 GB | 672 GB/s | 161 | 323 | 645 | ≈ 15.8B | 145 W | $0.57 |
| NVIDIA RTX 4500 Ada Generation 24GBNVIDIA · Workstation · 2023 | 24 GB | 432 GB/s | 159 | 317 | – | ≈ 15.8B | 210 W | – |
| NVIDIA RTX A5000 24GBNVIDIA · Workstation · 2021 | 24 GB | 768 GB/s | 111 | 222 | – | ≈ 15.8B | 230 W | $0.83 |
| NVIDIA GeForce RTX 4090 24GBNVIDIA · Consumer · 2022 | 24 GB | 1,008 GB/s | 165 | 330 | – | ≈ 15.8B | 450 W | $0.67 |
| NVIDIA GeForce RTX 3090 Ti 24GBNVIDIA · Consumer · 2022 | 24 GB | 1,008 GB/s | 80 | 320 | – | ≈ 15.8B | 450 W | $0.19 |
| NVIDIA GeForce RTX 3090 24GBNVIDIA · Consumer · 2020 | 24 GB | 936 GB/s | 71 | 284 | – | ≈ 15.8B | 350 W | $0.39 |
| NVIDIA TITAN RTX 24GBNVIDIA · Consumer · 2018 | 24 GB | 672 GB/s | 130 | 261 | – | ≈ 15.8B | 280 W | – |
| AMD Radeon RX 7900 XTX 24GBAMD · Consumer · 2022 | 24 GB | 960 GB/s | 123 | 123 | – | ≈ 15.8B | 355 W | – |
| Intel Arc Pro B60 24GBIntel · Workstation · 2025 | 24 GB | 456 GB/s | 99 | 197 | – | ≈ 15.8B | 200 W | – |
| NVIDIA RTX 4000 Ada Generation 20GBNVIDIA · Workstation · 2023 | 20 GB | 360 GB/s | 107 | 214 | – | ≈ 12.9B | 130 W | $0.75 |
| AMD Radeon RX 7900 XT 20GBAMD · Consumer · 2022 | 20 GB | 800 GB/s | 103 | 103 | – | ≈ 12.9B | 315 W | – |
| NVIDIA A16 (per GPU; 4x 16GB GPUs per board)NVIDIA · Data center · 2021 | 16 GB | 200 GB/s | 18 | 36 | – | ≈ 9.96B | 63 W | – |
| NVIDIA T4 16GBNVIDIA · Data center · 2018 | 16 GB | 320 GB/s | 65 | 130 | – | ≈ 9.96B | 70 W | $0.69 |
| NVIDIA RTX A4000 16GBNVIDIA · Workstation · 2021 | 16 GB | 448 GB/s | 77 | 153 | – | ≈ 9.96B | 140 W | $0.82 |
| NVIDIA GeForce RTX 5080 16GBNVIDIA · Consumer · 2025 | 16 GB | 960 GB/s | 113 | 225 | 900 | ≈ 9.96B | 360 W | $0.28 |
| NVIDIA GeForce RTX 5070 Ti 16GBNVIDIA · Consumer · 2025 | 16 GB | 896 GB/s | 88 | 176 | 703 | ≈ 9.96B | 300 W | $0.24 |
| NVIDIA GeForce RTX 5060 Ti 16GBNVIDIA · Consumer · 2025 | 16 GB | 448 GB/s | 47 | 95 | 380 | ≈ 9.96B | 180 W | $0.11 |
| NVIDIA GeForce RTX 4080 SUPER 16GBNVIDIA · Consumer · 2024 | 16 GB | 736 GB/s | 105 | 209 | – | ≈ 9.96B | 320 W | – |
| NVIDIA GeForce RTX 4080 16GBNVIDIA · Consumer · 2022 | 16 GB | 717 GB/s | 98 | 195 | – | ≈ 9.96B | 320 W | $0.24 |
| NVIDIA GeForce RTX 4070 Ti SUPER 16GBNVIDIA · Consumer · 2024 | 16 GB | 672 GB/s | 88 | 177 | – | ≈ 9.96B | 285 W | $0.35 |
| NVIDIA GeForce RTX 4060 Ti 16GBNVIDIA · Consumer · 2023 | 16 GB | 288 GB/s | 44 | 88 | – | ≈ 9.96B | 165 W | $0.13 |
| AMD Radeon RX 9070 XT 16GBAMD · Consumer · 2025 | 16 GB | 640 GB/s | 195 | 389 | – | ≈ 9.96B | 304 W | – |
| Google TPU v5eGoogle · Accelerator · 2023 | 16 GB | 819 GB/s | 197 | 393 | – | ≈ 9.96B | – | $1.20 |
| NVIDIA GeForce RTX 5070 12GBNVIDIA · Consumer · 2025 | 12 GB | 672 GB/s | 62 | 124 | 494 | ≈ 7.04B | 250 W | $0.19 |
| NVIDIA GeForce RTX 4070 SUPER 12GBNVIDIA · Consumer · 2024 | 12 GB | 504 GB/s | 71 | 142 | – | ≈ 7.04B | 220 W | – |
| NVIDIA GeForce RTX 3080 Ti 12GBNVIDIA · Consumer · 2021 | 12 GB | 912 GB/s | 68 | 273 | – | ≈ 7.04B | 350 W | $0.29 |
| NVIDIA GeForce RTX 3080 12GBNVIDIA · Consumer · 2022 | 12 GB | 912 GB/s | 61 | 245 | – | ≈ 7.04B | 350 W | – |
| NVIDIA GeForce RTX 3060 12GBNVIDIA · Consumer · 2021 | 12 GB | 360 GB/s | 26 | 102 | – | ≈ 7.04B | 170 W | $0.09 |
| Intel Arc B580 12GBIntel · Consumer · 2024 | 12 GB | 456 GB/s | 117 | 233 | – | ≈ 7.04B | 190 W | – |
“Fits at 8-bit” is a rough single-device estimate (weights at 8 bits, ~20% of usable memory kept for the KV cache). Dense TFLOPS, no sparsity. Prices are typical cloud on-demand rates. Pick a row to plan on that GPU.