1 line
8.4 KiB
JSON
1 line
8.4 KiB
JSON
{"repo": "ccl-123/rk3588-npu", "files": {"README_en.md": {"49cc54b2b1472a0ab410cd8214ae4b220844f8cf": {"t": 1785596444.4761653, "n": 8090, "h": "a58bc847da1b9473", "c": "<div align=\"center\">\n\n[English](./README_en.md) | [简体中文](./README.md)\n\n# RK3588 NPU Face Recognition System\n\n**High-Performance Embedded Face Recognition & Attendance Solution based on Rockchip NPU**\n\n[](https://en.cppreference.com/w/cpp/17)\n[](http://www.rock-chips.com/)\n[](https://www.qt.io/)\n[](LICENSE)\n[](https://github.com/airockchip/rknn-toolkit2)\n\n[Core Features](#core-features) • [Tech Stack](#tech-stack) • [Quick Start](#quick-start) • [Documentation](#documentation)\n\n</div>\n\n---\n\n## Project Introduction\n\nThis project is a high-performance face recognition attendance system designed specifically for the Rockchip RK3588 platform. It not only achieves extreme performance through deep integration of RKNN hardware acceleration inference, RGA graphics acceleration engine, and V4L2 zero-copy capture technology, but also **pioneers a \"Dual-Engine\" intelligent analysis capability**.\n\nThe system no longer just records clock-ins; it integrates **On-Device Offline Large Language Model (Local LLM)** with **Cloud LLM API**, enabling deep analysis of attendance patterns, diagnosis of abnormal behaviors, and automatic report generation through natural language dialogue. It upgrades traditional hardware terminals into intelligent management Agents with logical thinking capabilities, fully supporting **offline closed-loop operation** to ensuring data privacy and security.\n\n---\n\n## Core Features\n\n### 🤖 Dual-Engine AI Agent\n\n<div align=\"center\">\n<img src=\"C++/face_recognition_cap/docs/文档image/rkllm_image.png\" alt=\"RKLLM Model Conversion Pipeline\" width=\"80%\">\n<p><em>Complete Pipeline for Converting HuggingFace LLM to RKLLM Format</em></p>\n</div>\n\n- **On-Device LLM (Local Mode)**: Deep integration with **RKLLM**, running 2B/7B class LLMs (e.g., Qwen-2B) locally on the RK3588 NPU.\n - **Offline Operation**: No internet required, data never leaves the device, ensuring high privacy.\n - **NPU Acceleration**: Fully utilizes NPU computing power, with fast inference (~10 token/s) and < 200ms first-token latency.\n- **Cloud LLM (Cloud Mode)**: Seamless integration with standard Cloud LLM APIs for complex general knowledge queries.\n- **OpenAI-Compatible Remote LLM**: Remote mode calls an OpenAI-compatible `/v1/chat/completions` backend configured by `LLAMA_CPP_SERVER_URL`.\n- **One-Click Switch**: Unique NPU resource scheduling mechanism allows smooth switching between \"Face Recognition Mode\" and \"LLM Analysis Mode\".\n\n### 📊 Full Data Intelligent Diagnosis\n- **Multi-dimensional Analysis**: Supports deep diagnosis on \"Today / Last 7 Days / Last 30 Days\" full attendance data.\n- **Intelligent Insight**: Instantly identifies potential attendance anomaly patterns (e.g., long-term lateness, early departure trends) and provides targeted management suggestions.\n- **Streaming Reports**: Smooth typewriter-style streaming response for both local and cloud modes.\n\n### 🚀 Extreme Performance Optimization\n- **Dynamic NPU Resource Scheduling**: Implements a **mutual exclusion resource management strategy** for RKNN (Vision) and RKLLM (Language), resolving NPU contention deadlocks and ensuring both run at full speed in exclusive mode.\n- **NPU Hardware Acceleration**: Integrated with RKNN Runtime, implementing full-process NPU offloading for YOLOv8-face detection and FaceNet recognition.\n- **RGA Graphics Acceleration**: Utilizes the Rockchip RGA 2D hardware engine for image scaling, flipping, and format conversion (YUV -> RGB).\n- **Multi-Threaded Pipeline**: Adopts a 5-stage pipeline design to maximize parallel processing capabilities.\n\n### 🛡️ Comprehensive Business Functions\n- **Flexible Interaction Modes**: Provides a modern touch-enabled GUI interface based on Qt5.\n- **High-Performance Database**: Built-in SQLite3, supporting sub-second queries and writes for millions of records.\n- **High Robustness Design**: Includes automatic recovery mechanism for camera hot-plugging, real-time synchronization, and dynamic loading of face feature libraries.\n\n---\n\n## Tech Stack\n\n| Module | Technology | Description |\n| :--- | :--- | :--- |\n| **Language** | **C++17** | Core logic development, fully utilizing new standard library features |\n| **Local LLM** | **RKLLM Runtime** | **Core Highlight: RK3588 NPU Offline LLM Inference (Qwen-2B)** |\n| **Cloud LLM** | Cloud LLM API | Backup Online LLM Engine (SSE Streaming Protocol) |\n| **UI Framework** | Qt 5.15+ | Modern graphical user interface, supporting dynamic theme switching |\n| **Deep Learning** | RKNN Toolkit2 | NPU Model Inference (YOLOv8, FaceNet) |\n| **Computer Vision** | OpenCV 4.5+ | Image processing and algorithm assistance |\n| **Hardware Accel** | Rockchip RGA | 2D Hardware Acceleration Engine |\n| **Data Capture** | V4L2 | Linux Video Driver Interface (mmap zero-copy mode) |\n| **Database** | SQLite3 | Embedded local storage, DAO architecture design |\n| **Concurrency** | C++ Thread + Qt Signal | Multi-threaded pipeline and asynchronous event-driven |\n\n---\n\n## Quick Start\n\n### 1. Requirements\nEnsure the hardware is an RK3588 series development board (e.g., Orange Pi 5, Rock 5B), and the system has basic build tools and a C++17 compatible compiler installed:\n\n```bash\nsudo apt update\nsudo apt install cmake build-essential libopencv-dev qt5-default libsqlite3-dev\n```\n\n### 2. Build & Run\n```bash\ncd C++/face_recognition_cap\n./device_build.sh\n\n# Start GUI Mode\n./build/build_linux_aarch64/face_recognition_cap\n```\n\n### 3. Configure LLM Backends\n\n#### Local RKLLM\n\n```bash\nexport LOCAL_LLM_MODEL_PATH=/path/to/your_model.rkllm\n```\n\n#### OpenAI-Compatible Remote API\n\n```bash\nexport LLAMA_CPP_SERVER_URL=http://127.0.0.1:8080\nexport LLAMA_CPP_SERVER_MODEL=gpt-4o-mini\nexport LLAMA_CPP_SERVER_API_KEY=sk-your-key\n```\n\nNotes:\n\n- `LLAMA_CPP_SERVER_URL` should be the base URL only; the code appends `/v1/chat/completions`\n- The variable name keeps the historical `LLAMA_CPP_*` prefix, but the backend only needs to be OpenAI-compatible, not necessarily llama.cpp\n- `LLAMA_CPP_SERVER_URL` is required for remote Agent traffic\n\n---\n\n## Documentation\n\n* [Quick Start Guide](C++/face_recognition_cap/docs/用户文档/快速开始.md) - Environment setup and detailed compilation steps.\n* [Architecture Design](C++/face_recognition_cap/docs/开发文档/README.md) - System architecture, pipeline design, and core module explanation.\n* [API Documentation](C++/face_recognition_cap/docs/开发文档/API文档.md) - Secondary development and integration interfaces.\n* [User Manual](C++/face_recognition_cap/docs/用户文档/用户使用手册.md) - GUI function operation guide.\n* [Agent Module Guide](C++/face_recognition_cap/docs/开发文档/Agent模块说明.md) - Current Agent architecture, toolset, and OpenAI-compatible remote integration.\n* [OpenAI-Compatible Integration Guide](C++/face_recognition_cap/docs/开发文档/OpenAI兼容接口接入说明.md) - Environment variables and remote custom model access.\n\n---\n\n## License & Disclaimer\n\n### 1. Terms of Use\nThis project and its source code are **for learning, research, and personal exchange only**. You are free to obtain and modify the code subject to the following conditions:\n- **No Commercial Use**: You may not use this project or any modified version for any form of commercial product, paid service, or profit-making activity.\n- **Preserve Notices**: You must retain the original copyright notice and this license when distributing or propagating the code.\n\n### 2. Disclaimer\nThis program is provided \"AS IS\", without warranty of any kind, express or implied. The author does not guarantee the stability or safety of the program and is not responsible for any losses caused by the use of this program.\n\n---\n\n## Copyright\n\nCopyright © 2025 **Edge2-NPU Project**. All Rights Reserved.\n"}}}} |