The face recognition pipeline includes a face detection module and a face feature module, with several models in each module. Which models to use can be selected based on the benchmark data below. If you prioritize model accuracy, choose models with higher accuracy; if you prioritize inference speed, choose models with faster inference; if you prioritize model size, choose models with smaller storage requirements.
> The inference time only includes the model inference time and does not include the time for pre- or post-processing.
Face Detection Module:
| Model | Model Download Link | AP (%) Easy/Medium/Hard |
GPU Inference Time (ms) [Normal Mode / High-Performance Mode] |
CPU Inference Time (ms) [Normal Mode / High-Performance Mode] |
Model Storage Size (MB) | Description |
|---|---|---|---|---|---|---|
| BlazeFace | Inference Model/Training Model | 77.7/73.4/49.5 | 50.90 / 45.74 | 71.92 / 71.92 | 0.447 | A lightweight and efficient face detection model |
| BlazeFace-FPN-SSH | Inference Model/Training Model | 83.2/80.5/60.5 | 58.99 / 51.75 | 87.39 / 87.39 | 0.606 | Improved BlazeFace with FPN and SSH structures |
| PicoDet_LCNet_x2_5_face | Inference Model/Training Model | 93.7/90.7/68.1 | 33.91 / 26.53 | 153.56 / 79.21 | 28.9 | Face detection model based on PicoDet_LCNet_x2_5 |
| PP-YOLOE_plus-S_face | Inference Model/Training Model | 93.9/91.8/79.8 | 21.28 / 11.09 | 137.26 / 72.09 | 26.5 | Face detection model based on PP-YOLOE_plus-S |
Face Recognition Module:
| Model | Model Download Link | Output Feature Dimension | Acc (%) AgeDB-30/CFP-FP/LFW |
GPU Inference Time (ms) [Normal Mode / High-Performance Mode] |
CPU Inference Time (ms) [Normal Mode / High-Performance Mode] |
Model Storage Size (MB) | Description |
|---|---|---|---|---|---|---|---|
| MobileFaceNet | Inference Model/Training Model | 128 | 96.28/96.71/99.58 | 3.31 / 0.73 | 5.93 / 1.30 | 4.1 | Face recognition model trained on MS1Mv3 based on MobileFaceNet |
| ResNet50_face | Inference Model/Training Model | 512 | 98.12/98.56/99.77 | 6.12 / 3.11 | 15.85 / 9.44 | 87.2 | Face recognition model trained on MS1Mv3 based on ResNet50 |
| Mode | GPU Configuration | CPU Configuration | Acceleration Technology Combination |
|---|---|---|---|
| Normal Mode | FP32 Precision / No TRT Acceleration | FP32 Precision / 8 Threads | PaddleInference |
| High-Performance Mode | Optimal combination of pre-selected precision types and acceleration strategies | FP32 Precision / 8 Threads | Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.) |
| Parameter | Description | Type | Default Value | |
|---|---|---|---|---|
pipeline |
The name of the pipeline or the path to the pipeline configuration file. If it is the name of the pipeline, it must be a pipeline supported by PaddleX. | str |
None | |
config |
Specific configuration information for the pipeline (if set simultaneously with the pipeline, it takes precedence over the pipeline, and the pipeline name must match the pipeline).
|
dict[str, Any] |
None |
|
device |
The inference device for the pipeline. Supports specifying the specific card number of the GPU, such as "gpu:0", the specific card number of other hardware, such as "npu:0", and CPU such as "cpu". | str |
gpu:0 |
|
use_hpip |
Whether to enable the high-performance inference plugin. If set to None, the setting from the configuration file or config will be used. |
bool |
None | None |
hpi_config |
High-performance inference configuration | dict | None |
None | None |
| Parameter | Description | Type | Options | Default Value |
|---|---|---|---|---|
gallery_imgs |
The base library images to be added, required parameter | str|list |
|
None |
gallery_label |
The annotation information of the base library images, required parameter | str|list |
|
None |
metric_type |
Feature measurement method, optional parameter | str |
|
"IP" |
index_type |
Index type, optional parameter | str |
|
"HNSW32" |
| Parameter | Description | Type | Default Value |
|---|---|---|---|
save_path |
The save directory of the feature library file, such as drink_index. |
str |
None |
| Parameter | Description | Type | Options | Default |
|---|---|---|---|---|
input |
Data to be predicted, supports multiple input types (required parameter) | Python Var|str|list |
|
None |
index |
The feature library used for pipeline inference prediction (optional parameter). If this parameter is not provided, the index library specified in the pipeline configuration file will be used by default. | str|paddlex.inference.components.retrieval.faiss.IndexData|None |
|
None |
| Method | Description | Parameter | Type | Description | Default |
|---|---|---|---|---|---|
print() |
Print the result to the terminal | format_json |
bool |
Whether to format the output content using JSON indentation |
True |
indent |
int |
Specify the indentation level to beautify the output JSON data, making it more readable. Effective only when format_json is True |
4 | ||
ensure_ascii |
bool |
Control whether to escape non-ASCII characters to Unicode. When set to True, all non-ASCII characters will be escaped; False will retain the original characters. Effective only when format_json is True |
False |
||
save_to_json() |
Save the result as a JSON file | save_path |
str |
Path to save the file. When it is a directory, the saved file name will be consistent with the input file type | None |
indent |
int |
Specify the indentation level to beautify the output JSON data, making it more readable. Effective only when format_json is True |
4 | ||
ensure_ascii |
bool |
Control whether to escape non-ASCII characters to Unicode. When set to True, all non-ASCII characters will be escaped; False will retain the original characters. Effective only when format_json is True |
False |
||
save_to_img() |
Save the result as an image file | save_path |
str |
Path to save the file, supports directory or file path | None |
* Additionally, it also supports obtaining the visualized image with results and prediction results through attributes, as follows:
| Attribute | Description |
|---|---|
json |
Get the prediction result in json format. |
img |
Get the visualized image in dict format. |
| Parameter | Description | Type | Options | Default Value |
|---|---|---|---|---|
gallery_imgs |
Gallery images to be added, required parameter | str|list |
|
None |
gallery_label |
Labels for gallery images, required parameter | str|list |
|
None |
metric_type |
Feature measurement method, optional parameter | str |
|
"IP" |
index_type |
Type of index, optional parameter | str |
|
"HNSW32" |
remove_ids |
Indices to be removed | str|list |
|
None |
index |
Feature library used for pipeline inference | str|paddlex.inference.components.retrieval.faiss.IndexData |
|
None |
HNSW32 has compatibility issues on the Windows platform, which may prevent the index library from being built or loaded.
### 2.3 Data Organization for Building the Feature Library
The face recognition pipeline example of PaddleX requires a pre-built feature library for face feature retrieval. If you wish to build a face feature library with your private data, you need to organize the data as follows:
```bash
data_root # Root directory of the dataset, the directory name can be changed
├── images # Directory for storing images, the directory name can be changed
│ ├── ID0 # Identity ID name, preferably a meaningful name, such as a person's name
│ │ ├── xxx.jpg # Image, nested levels are supported here
│ │ ├── xxx.jpg # Image, nested levels are supported here
│ │ ...
│ ├── ID1 # Identity ID name, preferably a meaningful name, such as a person's name
│ │ ├── xxx.jpg # Image, nested levels are supported here
│ │ ├── xxx.jpg # Image, nested levels are supported here
│ │ ...
│ ...
└── gallery.txt # Annotation file for the feature library dataset, the file name can be changed. Each line provides the path and label of the face image to be retrieved, separated by a space. Example content: images/Chandler/Chandler00037.jpg Chandler
```
## 3. Development Integration/Deployment
If the face recognition pipeline meets your requirements for inference speed and accuracy, you can proceed directly with development integration/deployment.
If you need to apply the face recognition pipeline directly in your Python project, you can refer to the example code in [2.2.2 Python Script Integration](#222-python-script-integration).
Additionally, PaddleX provides three other deployment methods, detailed as follows:
🚀 High-Performance Inference: In actual production environments, many applications have stringent standards for the performance metrics of deployment strategies (especially response speed) to ensure efficient system operation and smooth user experience. To this end, PaddleX offers a high-performance inference plugin aimed at deeply optimizing the performance of model inference and pre/post-processing, significantly speeding up the end-to-end process. For detailed high-performance inference processes, please refer to [PaddleX High-Performance Inference Guide](../../../pipeline_deploy/high_performance_inference.en.md).
☁️ Service Deployment: Service deployment is a common form of deployment in actual production environments. By encapsulating the inference function as a service, clients can access these services via network requests to obtain inference results. PaddleX supports multiple pipeline service deployment schemes. For detailed pipeline service deployment processes, please refer to [PaddleX Service Deployment Guide](../../../pipeline_deploy/serving.en.md).
Below is the API reference for basic service deployment and multi-language service call examples:
For the main operations provided by the service:
200, and the properties of the response body are as follows:| Name | Type | Meaning |
|---|---|---|
logId |
string |
The UUID of the request. |
errorCode |
integer |
Error code. Fixed at 0. |
errorMsg |
string |
Error description. Fixed at "Success". |
result |
object |
The result of the operation. |
| Name | Type | Meaning |
|---|---|---|
logId |
string |
The UUID of the request. |
errorCode |
integer |
Error code. Same as the response status code. |
errorMsg |
string |
Error description. |
The main operations provided by the service are as follows:
buildIndexBuild feature vector index.
POST /face-recognition-index-build
| Name | Type | Meaning | Required |
|---|---|---|---|
imageLabelPairs |
array |
Image-label pairs used to build the index. | Yes |
Each element in imageLabelPairs is an object with the following properties:
| Name | Type | Meaning |
|---|---|---|
image |
string |
The URL of the image file accessible by the server or the Base64-encoded result of the image file content. |
label |
string |
Label. |
result in the response body has the following properties:| Name | Type | Meaning |
|---|---|---|
indexKey |
string |
The key corresponding to the index, used to identify the created index. It can be used as input for other operations. |
imageCount |
integer |
The number of images indexed. |
addImagesToIndexAdd images (corresponding feature vectors) to the index.
POST /face-recognition-index-add
| Name | Type | Description | Required |
|---|---|---|---|
imageLabelPairs |
array |
Image-label pairs used to build the index. | Yes |
indexKey |
string |
The key corresponding to the index. Provided by the buildIndex operation. |
Yes |
Each element in imageLabelPairs is an object with the following properties:
| Name | Type | Description |
|---|---|---|
image |
string |
The URL of an image file accessible by the server or the Base64-encoded content of the image file. |
label |
string |
The label. |
result in the response body has the following properties:| Name | Type | Description |
|---|---|---|
imageCount |
integer |
The number of images indexed. |
removeImagesFromIndexRemove images (corresponding feature vectors) from the index.
POST /face-recognition-index-remove
| Name | Type | Description | Required |
|---|---|---|---|
ids |
array |
The IDs of the vectors to be removed from the index. | Yes |
indexKey |
string |
The key corresponding to the index. Provided by the buildIndex operation. |
Yes |
result in the response body has the following properties:| Name | Type | Description |
|---|---|---|
imageCount |
integer |
The number of images indexed. |
inferPerform image recognition.
POST /face-recognition-infer
| Name | Type | Description | Required |
|---|---|---|---|
image |
string |
The URL of an image file accessible by the server or the Base64-encoded content of the image file. | Yes |
indexKey |
string |
The key corresponding to the index. Provided by the buildIndex operation. |
No |
detThreshold |
number | null |
Please refer to description of the det_threshold parameter of the pipeline object's predict method. |
No |
recThreshold |
number | null |
Please refer to description of the rec_threshold parameter of the pipeline object's predict method. |
No |
hammingRadius |
number | null |
Please refer to the description of the hamming_radius parameter of the pipeline object's predict method. |
No |
topk |
integer | null |
Please refer to description of the topk parameter of the pipeline object's predict method. |
No |
visualize |
boolean | null |
Whether to return the final visualization image and intermediate images during the processing.
For example, adding the following setting to the pipeline config file: visualize parameter in the request.If neither the request body nor the configuration file is set (If visualize is set to null in the request and not defined in the configuration file), the image is returned by default.
|
No |
result in the response body has the following properties:| Name | Type | Description |
|---|---|---|
faces |
array |
Information about detected faces. |
image |
string | null |
The recognition result image. The image is in JPEG format and is Base64-encoded. |
Each element in faces is an object with the following properties:
| Name | Type | Description |
|---|---|---|
bbox |
array |
The location of the face target. The elements of the array are the x-coordinate of the top-left corner, the y-coordinate of the top-left corner, the x-coordinate of the bottom-right corner, and the y-coordinate of the bottom-right corner. |
recResults |
array |
Recognition results. |
score |
number |
The detection score. |
Each element in recResults is an object with the following properties:
| Name | Type | Description |
|---|---|---|
label |
string |
The label. |
score |
number |
The recognition score. |
import base64
import pprint
import sys
import requests
API_BASE_URL = "http://127.0.0.1:8080"
base_image_label_pairs = [
{"image": "./demo0.jpg", "label": "ID0"},
{"image": "./demo1.jpg", "label": "ID1"},
{"image": "./demo2.jpg", "label": "ID2"},
]
image_label_pairs_to_add = [
{"image": "./demo3.jpg", "label": "ID2"},
]
ids_to_remove = [1]
infer_image_path = "./demo4.jpg"
output_image_path = "./out.jpg"
for pair in base_image_label_pairs:
with open(pair["image"], "rb") as file:
image_bytes = file.read()
image_data = base64.b64encode(image_bytes).decode("ascii")
pair["image"] = image_data
payload = {"imageLabelPairs": base_image_label_pairs}
resp_index_build = requests.post(f"{API_BASE_URL}/face-recognition-index-build", json=payload)
if resp_index_build.status_code != 200:
print(f"Request to face-recognition-index-build failed with status code {resp_index_build}.")
pprint.pp(resp_index_build.json())
sys.exit(1)
result_index_build = resp_index_build.json()["result"]
print(f"Number of images indexed: {result_index_build['imageCount']}")
for pair in image_label_pairs_to_add:
with open(pair["image"], "rb") as file:
image_bytes = file.read()
image_data = base64.b64encode(image_bytes).decode("ascii")
pair["image"] = image_data
payload = {"imageLabelPairs": image_label_pairs_to_add, "indexKey": result_index_build["indexKey"]}
resp_index_add = requests.post(f"{API_BASE_URL}/face-recognition-index-add", json=payload)
if resp_index_add.status_code != 200:
print(f"Request to face-recognition-index-add failed with status code {resp_index_add}.")
pprint.pp(resp_index_add.json())
sys.exit(1)
result_index_add = resp_index_add.json()["result"]
print(f"Number of images indexed: {result_index_add['imageCount']}")
payload = {"ids": ids_to_remove, "indexKey": result_index_build["indexKey"]}
resp_index_remove = requests.post(f"{API_BASE_URL}/face-recognition-index-remove", json=payload)
if resp_index_remove.status_code != 200:
print(f"Request to face-recognition-index-remove failed with status code {resp_index_remove}.")
pprint.pp(resp_index_remove.json())
sys.exit(1)
result_index_remove = resp_index_remove.json()["result"]
print(f"Number of images indexed: {result_index_remove['imageCount']}")
with open(infer_image_path, "rb") as file:
image_bytes = file.read()
image_data = base64.b64encode(image_bytes).decode("ascii")
payload = {"image": image_data, "indexKey": result_index_build["indexKey"]}
resp_infer = requests.post(f"{API_BASE_URL}/face-recognition-infer", json=payload)
if resp_infer.status_code != 200:
print(f"Request to face-recogntion-infer failed with status code {resp_infer}.")
pprint.pp(resp_infer.json())
sys.exit(1)
result_infer = resp_infer.json()["result"]
with open(output_image_path, "wb") as file:
file.write(base64.b64decode(result_infer["image"]))
print(f"Output image saved at {output_image_path}")
print("\nDetected faces:")
pprint.pp(result_infer["faces"])