Преглед изворни кода

[Feat] Add genai-vllm-server Dockerfile and build script (#4680)

* Add genai-vllm-server Dockerfile and build script

* Add lower bound for paddlex version

* Set paddleocr lower version to 3.3.5
Lin Manhui пре 2 недеља
родитељ
комит
1da53a1aa4
2 измењених фајлова са 27 додато и 0 уклоњено
  1. 18 0
      deploy/genai_vllm_server_docker/Dockerfile
  2. 9 0
      deploy/genai_vllm_server_docker/build.sh

+ 18 - 0
deploy/genai_vllm_server_docker/Dockerfile

@@ -0,0 +1,18 @@
+FROM python:3.10
+
+RUN apt-get update \
+    && apt-get install -y libgl1 \
+    && rm -rf /var/lib/apt/lists/*
+
+ENV PIP_NO_CACHE_DIR=0
+ENV PYTHONUNBUFFERED=1
+ENV PYTHONDONTWRITEBYTECODE=1
+
+RUN python -m pip install 'paddlex>=3.3.5,<3.4'
+
+RUN python -m pip install https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/download/v0.3.14/flash_attn-2.8.2+cu128torch2.8-cp310-cp310-linux_x86_64.whl \
+    && paddlex --install genai-vllm-server
+
+EXPOSE 8080
+
+CMD ["paddlex_genai_server", "--model_name", "PaddleOCR-VL-0.9B", "--host", "0.0.0.0", "--port", "8080", "--backend", "vllm"]

+ 9 - 0
deploy/genai_vllm_server_docker/build.sh

@@ -0,0 +1,9 @@
+#!/usr/bin/env bash
+
+docker build \
+    -t "ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddlex-genai-vllm-server:${1:latest}" \
+    --build-arg http_proxy="${http_proxy}" \
+    --build-arg https_proxy="${https_proxy}" \
+    --build-arg no_proxy="${no_proxy}" \
+    --build-arg PIP_INDEX_URL="${PIP_INDEX_URL}" \
+    .