# GPU

vLLM is a Python library that supports the following GPU variants. Select your GPU type to see vendor specific instructions:

::::{tab-set}
:sync-group: device

:::{tab-item} CUDA
:sync: cuda

```{include} cuda.inc.md
:start-after: "# Installation"
:end-before: "## Requirements"
```

:::

:::{tab-item} ROCm
:sync: rocm

```{include} rocm.inc.md
:start-after: "# Installation"
:end-before: "## Requirements"
```

:::

:::{tab-item} XPU
:sync: xpu

```{include} xpu.inc.md
:start-after: "# Installation"
:end-before: "## Requirements"
```

:::

::::

## Requirements

- OS: Linux
- Python: 3.9 -- 3.12

::::{tab-set}
:sync-group: device

:::{tab-item} CUDA
:sync: cuda

```{include} cuda.inc.md
:start-after: "## Requirements"
:end-before: "## Set up using Python"
```

:::

:::{tab-item} ROCm
:sync: rocm

```{include} rocm.inc.md
:start-after: "## Requirements"
:end-before: "## Set up using Python"
```

:::

:::{tab-item} XPU
:sync: xpu

```{include} xpu.inc.md
:start-after: "## Requirements"
:end-before: "## Set up using Python"
```

:::

::::

## Set up using Python

### Create a new Python environment

```{include} ../python_env_setup.inc.md
```

::::{tab-set}
:sync-group: device

:::{tab-item} CUDA
:sync: cuda

```{include} cuda.inc.md
:start-after: "## Create a new Python environment"
:end-before: "### Pre-built wheels"
```

:::

:::{tab-item} ROCm
:sync: rocm

There is no extra information on creating a new Python environment for this device.

:::

:::{tab-item} XPU
:sync: xpu

There is no extra information on creating a new Python environment for this device.

:::

::::

### Pre-built wheels

::::{tab-set}
:sync-group: device

:::{tab-item} CUDA
:sync: cuda

```{include} cuda.inc.md
:start-after: "### Pre-built wheels"
:end-before: "### Build wheel from source"
```

:::

:::{tab-item} ROCm
:sync: rocm

```{include} rocm.inc.md
:start-after: "### Pre-built wheels"
:end-before: "### Build wheel from source"
```

:::

:::{tab-item} XPU
:sync: xpu

```{include} xpu.inc.md
:start-after: "### Pre-built wheels"
:end-before: "### Build wheel from source"
```

:::

::::

(build-from-source)=

### Build wheel from source

::::{tab-set}
:sync-group: device

:::{tab-item} CUDA
:sync: cuda

```{include} cuda.inc.md
:start-after: "### Build wheel from source"
:end-before: "## Set up using Docker"
```

:::

:::{tab-item} ROCm
:sync: rocm

```{include} rocm.inc.md
:start-after: "### Build wheel from source"
:end-before: "## Set up using Docker"
```

:::

:::{tab-item} XPU
:sync: xpu

```{include} xpu.inc.md
:start-after: "### Build wheel from source"
:end-before: "## Set up using Docker"
```

:::

::::

## Set up using Docker

### Pre-built images

::::{tab-set}
:sync-group: device

:::{tab-item} CUDA
:sync: cuda

```{include} cuda.inc.md
:start-after: "### Pre-built images"
:end-before: "### Build image from source"
```

:::

:::{tab-item} ROCm
:sync: rocm

```{include} rocm.inc.md
:start-after: "### Pre-built images"
:end-before: "### Build image from source"
```

:::

:::{tab-item} XPU
:sync: xpu

```{include} xpu.inc.md
:start-after: "### Pre-built images"
:end-before: "### Build image from source"
```

:::

::::

### Build image from source

::::{tab-set}
:sync-group: device

:::{tab-item} CUDA
:sync: cuda

```{include} cuda.inc.md
:start-after: "### Build image from source"
:end-before: "## Supported features"
```

:::

:::{tab-item} ROCm
:sync: rocm

```{include} rocm.inc.md
:start-after: "### Build image from source"
:end-before: "## Supported features"
```

:::

:::{tab-item} XPU
:sync: xpu

```{include} xpu.inc.md
:start-after: "### Build image from source"
:end-before: "## Supported features"
```

:::

::::

## Supported features

::::{tab-set}
:sync-group: device

:::{tab-item} CUDA
:sync: cuda

```{include} cuda.inc.md
:start-after: "## Supported features"
```

:::

:::{tab-item} ROCm
:sync: rocm

```{include} rocm.inc.md
:start-after: "## Supported features"
```

:::

:::{tab-item} XPU
:sync: xpu

```{include} xpu.inc.md
:start-after: "## Supported features"
```

:::

::::