|
@@ -0,0 +1,178 @@
|
|
|
+# POC部署说明
|
|
|
+## 文件目录说明
|
|
|
+```
|
|
|
+├── config
|
|
|
+│ └── application.yaml
|
|
|
+├── docker-compose-base-gpu.yml
|
|
|
+├── docker-compose.yml
|
|
|
+├── docker.tar.gz
|
|
|
+├── huggingface
|
|
|
+│ ├── hub
|
|
|
+│ └── modules
|
|
|
+├── images
|
|
|
+│ ├── elasticsearch-8.11.3.tgz
|
|
|
+│ ├── gotenberg-8.9.1.tgz
|
|
|
+│ ├── infinity-0.0.55.tgz
|
|
|
+│ ├── llm-app-55.tgz
|
|
|
+│ ├── llm-doc-parser.tgz
|
|
|
+│ ├── llm-rag-35.tgz
|
|
|
+│ ├── minio-RELEASE.2024-09-22T00-33-43Z.tgz
|
|
|
+│ ├── mysql-5.7.44.tgz
|
|
|
+│ ├── redis-7.2.4.tgz
|
|
|
+│ └── unstructured-api-0.0.80.tgz
|
|
|
+├── init_extra.sql
|
|
|
+├── init.sql
|
|
|
+├── install_docker.sh
|
|
|
+├── install.sh
|
|
|
+├── mysql.yaml
|
|
|
+├── README.md
|
|
|
+└── update_db.sh
|
|
|
+```
|
|
|
+## 部署架构图
|
|
|
+```mermaid
|
|
|
+flowchart TD
|
|
|
+ subgraph External
|
|
|
+ Client((Client))
|
|
|
+ end
|
|
|
+
|
|
|
+ subgraph Application["Application Layer"]
|
|
|
+ llm-app["llm-app
|
|
|
+ (RAG前后端应用层)"]
|
|
|
+ end
|
|
|
+
|
|
|
+ subgraph Storage["数据库中间件"]
|
|
|
+ MySQL[(MySQL)]
|
|
|
+ Redis[(Redis)]
|
|
|
+ MinIO[(MinIO)]
|
|
|
+ ES[(Elasticsearch)]
|
|
|
+ end
|
|
|
+
|
|
|
+ subgraph Services["Service Layer"]
|
|
|
+ gotenberg["Gotenberg
|
|
|
+ (PDF转换服务)"]
|
|
|
+ llm-rag["llm-rag
|
|
|
+ (RAG基础服务)"]
|
|
|
+ unstructured["Unstructured API
|
|
|
+ (Unstructured文档解析)"]
|
|
|
+ llm-infinity["LLM Infinity
|
|
|
+ (Embedding和Rerank
|
|
|
+ 模型推理)"]
|
|
|
+ doc-parser["doc-parser
|
|
|
+ (文档解析聚合服务)"]
|
|
|
+ end
|
|
|
+
|
|
|
+ %% External connections
|
|
|
+ Client --> llm-app
|
|
|
+
|
|
|
+ %% Application layer connections
|
|
|
+ llm-app --> MySQL
|
|
|
+ llm-app --> Redis
|
|
|
+ llm-app --> MinIO
|
|
|
+ llm-app --> llm-rag
|
|
|
+ llm-app --> gotenberg
|
|
|
+
|
|
|
+ %% llm-rag connections
|
|
|
+ llm-rag --> ES
|
|
|
+ llm-rag --> unstructured
|
|
|
+ llm-rag --> llm-infinity
|
|
|
+ llm-rag --> doc-parser
|
|
|
+
|
|
|
+ %% Styling
|
|
|
+ classDef storage fill:#f9f,stroke:#333,stroke-width:2px
|
|
|
+ classDef service fill:#bbf,stroke:#333,stroke-width:2px
|
|
|
+ classDef application fill:#bfb,stroke:#333,stroke-width:2px
|
|
|
+ classDef client fill:#fff,stroke:#333,stroke-width:2px
|
|
|
+
|
|
|
+ class MySQL,Redis,MinIO,ES storage
|
|
|
+ class gotenberg,unstructured,llm-infinity,doc-parser service
|
|
|
+ class llm-app,llm-rag application
|
|
|
+ class Client client
|
|
|
+```
|
|
|
+## 服务配置
|
|
|
+
|
|
|
+### 后端服务配置
|
|
|
+[config/application.yaml](./config/application.yaml) 智能问答后端服务配置
|
|
|
+### Docker 环境变量
|
|
|
+查看 [.env](./.env)文件,内容如下
|
|
|
+```yaml
|
|
|
+# Version of Elastic products
|
|
|
+STACK_VERSION=8.11.3
|
|
|
+
|
|
|
+# Port to expose Elasticsearch HTTP API to the host
|
|
|
+ES_PORT=9201
|
|
|
+
|
|
|
+# Set the Elasticsearch password
|
|
|
+ELASTIC_PASSWORD=8IpWQfcZXXmRsIkX7+yy
|
|
|
+
|
|
|
+
|
|
|
+# Increase or decrease based on the available host memory (in bytes)
|
|
|
+
|
|
|
+MEM_LIMIT=8073741824
|
|
|
+
|
|
|
+
|
|
|
+MYSQL_PASSWORD=llmllmllm
|
|
|
+MYSQL_PORT=3336
|
|
|
+
|
|
|
+# Port to expose minio to the host
|
|
|
+MINIO_CONSOLE_PORT=9001
|
|
|
+MINIO_PORT=9000
|
|
|
+
|
|
|
+MINIO_USER=root
|
|
|
+MINIO_PASSWORD=llmllmllm
|
|
|
+
|
|
|
+REDIS_PORT=6389
|
|
|
+REDIS_PASSWORD=llmllmllm
|
|
|
+
|
|
|
+# 智能问答服务访问入口
|
|
|
+SVR_HTTP_PORT=8080
|
|
|
+SVR_HTTP_HOST='llm-app'
|
|
|
+
|
|
|
+TIMEZONE='Asia/Shanghai'
|
|
|
+```
|
|
|
+
|
|
|
+## 数据库脚本
|
|
|
+### 数据库初始化脚本
|
|
|
+[init.sql](init.sql),第一次启动MySQL时会自动执行,如果执行错误,需要手动再次执行
|
|
|
+
|
|
|
+### 额外的数据库初始化脚本
|
|
|
+由于MySQL安装初始化脚本有大小限制,目前是有个背景图片需要额外导入
|
|
|
+* 脚本 [init_extra.sql](init_extra.sql)
|
|
|
+* 在数据库完全启动之后手动通过脚本执行:
|
|
|
+ `bash update_db.sh`
|
|
|
+
|
|
|
+## 服务部署
|
|
|
+
|
|
|
+### 安装docker和docker compose
|
|
|
+1. 下载Docker二进制安装文件[docker.tar.gz](http://110.16.193.170:40002/share/UtURYif0)到当前目录
|
|
|
+2. 执行安装脚本 `./install_docker.sh`
|
|
|
+
|
|
|
+### docker-compose文件说明
|
|
|
+* docker-compose.yml 安装所有服务的入口,文件中定义app和rag服务,引用`docker-compose-base-gpu.yaml`
|
|
|
+ * llm-app 前后端服务
|
|
|
+ * llm-rag Python后端
|
|
|
+* docker-compose-base-gpu.yml 除了app和rag的所有服务的安装定义
|
|
|
+ * es: 向量数据库存储
|
|
|
+ * mysql:关系数据库
|
|
|
+ * redis:缓存数据库
|
|
|
+ * minio:对象存储
|
|
|
+ * gotenberg: 转pdf服务
|
|
|
+ * unstructured-api: unstructured文档解析
|
|
|
+ * llm-infinity:embedding模型和rerank模型推理服务
|
|
|
+ * doc-parser 文件解析聚合服务,包括deepdoc和marker,并且依赖unstructured文档解析
|
|
|
+ * chat模型推理(待更新)
|
|
|
+ * vllm(优先)qwen2.5-72b + qwen2.5-7b + qwen2-vl-7b
|
|
|
+ * ollama
|
|
|
+
|
|
|
+### 镜像文件
|
|
|
+下载镜像文件到[images](http://110.16.193.170:40002/share/eUvp3_tn)
|
|
|
+
|
|
|
+### 模型文件
|
|
|
+下载模型文件到[huggingface](http://110.16.193.170:40002/share/Nj4zYupf)
|
|
|
+> 注意:模型文件不包含chat模型,只有embedding、rerank和文档解析相关模型
|
|
|
+
|
|
|
+### 服务部署
|
|
|
+* 一键部署,前提是已经完成以上docker和dockercompose的安装
|
|
|
+```bash
|
|
|
+./install.sh
|
|
|
+```
|
|
|
+* 手动部署参考脚本`install.sh`
|