Quickstart¶
Currently the recommended way to start vLLM-MetaX is via docker.
You could get the docker image at MetaX develop community.
Note
After v0.11.2, vllm-metax moved its _C and _moe_C kernels into a separate package named mcoplib.
mcoplib is open-sourced at MetaX-mcoplib and would maintain its own release cycle. Please always install the corresponding version of mcoplib when using vLLM-MetaX.
Though the csrc folder is still kept in this repo for development convenience, and there is no guarantee that the code is always in sync with mcoplib. Not only the performance but also the correctness may differ from mcoplib.
If you want build the latest vllm-metax, please refer to installation to build from source.
Please always use mcoplib for production usage.
Releases¶
Below is version mapping to released plugin and mcoplib with maca:
| plugin version | maca version | mcoplib version | docker image url |
|---|---|---|---|
| v0.8.5 | maca2.33.1.13 | N/A | vllm:0.8.5 |
| v0.9.1 | maca3.0.0.5 | N/A | vllm:0.9.1 |
| v0.10.2 | maca3.2.1.7 | N/A | vllm-metax:0.10.2 |
| v0.11.0 | maca3.3.0.x | 0.1.1 | vllm-metax:0.11.0 |
| v0.11.2 | maca3.3.0.x | 0.2.0 | vllm-metax:0.11.2 |
| v0.12.0 | maca3.3.0.x | 0.3.0 | vllm-metax:0.12.0 |
| v0.13.0 | maca3.3.0.x | 0.3.1 | vllm-metax:0.13.0 |
| v0.14.0 | maca3.5.3.x | 0.4.0 | vllm-metax:0.14.0 |
| v0.15.0 | maca3.5.3.x | 0.4.1 | vllm-metax:0.15.0 |
| v0.16.0 | N/A | N/A | Skipped |
| v0.17.0 | maca3.5.3.x | 0.4.2 | vllm-metax:0.17.0 |
| v0.18.0 | maca3.5.3.x | 0.4.3 | vllm-metax:0.18.0 |
| v0.19.0 | maca3.5.3.x | 0.4.4 | vllm-metax:0.19.0 |
| v0.20.0 | maca3.7.0.x | 0.4.5 | vllm-metax:0.20.0 |
Usage Warning
vLLM-MetaX is out of box via docker images provided above.
All the vllm tests are based on the related maca version. Using incorresponding version of maca for vllm may cause unexpected bugs or errors. This is not guaranteed.