vllm/docs/source/features/quantization/supported_hardware.md

(quantization-supported-hardware)=

# Supported Hardware

The table below shows the compatibility of various quantization implementations with different hardware platforms in vLLM:

```{list-table}
:header-rows: 1
:widths: 20 8 8 8 8 8 8 8 8 8 8

* - Implementation
  - Volta
  - Turing
  - Ampere
  - Ada
  - Hopper
  - AMD GPU
  - Intel GPU
  - x86 CPU
  - AWS Inferentia
  - Google TPU
* - AWQ
  - ✗
  - ✅︎
  - ✅︎
  - ✅︎
  - ✅︎
  - ✗
  - ✅︎
  - ✅︎
  - ✗
  - ✗
* - GPTQ
  - ✅︎
  - ✅︎
  - ✅︎
  - ✅︎
  - ✅︎
  - ✗
  - ✅︎
  - ✅︎
  - ✗
  - ✗
* - Marlin (GPTQ/AWQ/FP8)
  - ✗
  - ✗
  - ✅︎
  - ✅︎
  - ✅︎
  - ✗
  - ✗
  - ✗
  - ✗
  - ✗
* - INT8 (W8A8)
  - ✗
  - ✅︎
  - ✅︎
  - ✅︎
  - ✅︎
  - ✗
  - ✗
  - ✅︎
  - ✗
  - ✗
* - FP8 (W8A8)
  - ✗
  - ✗
  - ✗
  - ✅︎
  - ✅︎
  - ✅︎
  - ✗
  - ✗
  - ✗
  - ✗
* - AQLM
  - ✅︎
  - ✅︎
  - ✅︎
  - ✅︎
  - ✅︎
  - ✗
  - ✗
  - ✗
  - ✗
  - ✗
* - bitsandbytes
  - ✅︎
  - ✅︎
  - ✅︎
  - ✅︎
  - ✅︎
  - ✗
  - ✗
  - ✗
  - ✗
  - ✗
* - DeepSpeedFP
  - ✅︎
  - ✅︎
  - ✅︎
  - ✅︎
  - ✅︎
  - ✗
  - ✗
  - ✗
  - ✗
  - ✗
* - GGUF
  - ✅︎
  - ✅︎
  - ✅︎
  - ✅︎
  - ✅︎
  - ✅︎
  - ✗
  - ✗
  - ✗
  - ✗
```

- Volta refers to SM 7.0, Turing to SM 7.5, Ampere to SM 8.0/8.6, Ada to SM 8.9, and Hopper to SM 9.0.
- "✅︎" indicates that the quantization method is supported on the specified hardware.
- "✗" indicates that the quantization method is not supported on the specified hardware.

```{note}
This compatibility chart is subject to change as vLLM continues to evolve and expand its support for different hardware platforms and quantization methods.

For the most up-to-date information on hardware support and quantization methods, please refer to <gh-dir:vllm/model_executor/layers/quantization> or consult with the vLLM development team.
```