20231088/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
vllm/docs/source/getting_started/installation/cpu-apple.md
Wallas Henrique cfd3219f58
[Hardware][Apple] Native support for macOS Apple Silicon (#11696) 
Signed-off-by: Wallas Santos <wallashss@ibm.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
2025-01-08 16:35:49 +08:00

1.6 KiB
Raw Blame History

(installation-apple)=

Installation for macOS

vLLM has experimental support for macOS with Apple Silicon. For now, users shall build from the source vLLM to natively run on macOS. For more details, like running on vLLM in a docker container, see ARM CPU Documentation

Currently the CPU implementation for macOS supports FP32 and FP16 datatypes.

Requirements

  • Operating System: macOS Sonoma or later
  • SDK XCode 15.4 or later with Command Line Tools
  • Compilers: Apple Clang >= 15.0.0

Build and installation

After installation of XCode and the Command Line Tools, which include Apple Clang, execute the following commands to build and install vLLM from the source.

$ git clone https://github.com/vllm-project/vllm.git
$ cd vllm
$ pip install -r requirements-cpu.txt
$ pip install -e . 

On macOS the `VLLM_TARGET_DEVICE` is automatically set to `cpu`, which currently is the only supported device.

Troubleshooting

If the build has error like the following snippet where standard C++ headers cannot be found, try to remove and reinstall your Command Line Tools for Xcode.

[...] fatal error: 'map' file not found
          1 | #include <map>
            |          ^~~~~
      1 error generated.
      [2/8] Building CXX object CMakeFiles/_C.dir/csrc/cpu/pos_encoding.cpp.o

[...] fatal error: 'cstddef' file not found
         10 | #include <cstddef>
            |          ^~~~~~~~~
      1 error generated.