vllm/docs/source/getting_started/installation/cpu/s390x.inc.md

Installation

vLLM has experimental support for s390x architecture on IBM Z platform. For now, users shall build from the vLLM source to natively run on IBM Z platform.

Currently the CPU implementation for s390x architecture supports FP32 datatype only.

:::{attention} There are no pre-built wheels or images for this device, so you must build vLLM from source. :::

Requirements

• OS: Linux
• SDK: gcc/g++ >= 12.3.0 or later with Command Line Tools
• Instruction Set Architecture (ISA): VXE support is required. Works with Z14 and above.
• Build install python packages: pyarrow, torch and torchvision

Set up using Python

Pre-built wheels

Build wheel from source

Install the following packages from the package manager before building the vLLM. For example on RHEL 9.4:

dnf install -y \
    which procps findutils tar vim git gcc g++ make patch make cython zlib-devel \
    libjpeg-turbo-devel libtiff-devel libpng-devel libwebp-devel freetype-devel harfbuzz-devel \
    openssl-devel openblas openblas-devel wget autoconf automake libtool cmake numactl-devel

Install rust>=1.80 which is needed for outlines-core and uvloop python packages installation.

curl https://sh.rustup.rs -sSf | sh -s -- -y && \
    . "$HOME/.cargo/env"

Execute the following commands to build and install vLLM from the source.

::::{tip} Please build the following dependencies, torchvision, pyarrow from the source before building vLLM. ::::

    sed -i '/^torch/d' requirements-build.txt    # remove torch from requirements-build.txt since we use nightly builds
    pip install -v \
        --extra-index-url https://download.pytorch.org/whl/nightly/cpu \
        -r requirements-build.txt \
        -r requirements-cpu.txt \
    VLLM_TARGET_DEVICE=cpu python setup.py bdist_wheel && \
    pip install dist/*.whl

Set up using Docker

Pre-built images

Build image from source

Extra information