Enhance multimodal detection capabilities and update configurations

- Introduced a new multimodal detection feature, allowing the system to analyze both text and image content for safety.
- Added support for image input via base64 encoding and URLs, enhancing the flexibility of the detection API.
- Updated the backend to include new models and routes for handling image uploads and processing.
- Enhanced the API to support mixed content types in detection requests, ensuring consistent risk assessment across modalities.
- Improved the README files with detailed descriptions of the multimodal detection functionality and usage examples.
- Incremented version numbers in both backend and frontend to 2.3.0 to signify the introduction of these features.

Author: ThomasLWang

CHANGELOG.md

@@ -10,6 +10,99 @@ All notable changes to Xiangxin AI Guardrails platform are documented in this fi
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [2.3.0] - 2025-09-30
+
+### 🚀 重大更新 Major Updates
+- 🖼️ **多模态检测功能**
+  - 新增图片模态安全检测能力
+  - 支持图片内容的合规性和安全性检测
+  - 与文本检测保持一致的风险类型和检测标准
+  - 完整支持API调用模式和安全网关模式
+
+### 新增 Added
+- 🖼️ **图片检测功能**
+  - 支持base64编码和URL两种图片输入方式
+  - 调用多模态检测模型 `Xiangxin-Guardrails-VL`
+  - 图片文件存储在用户专属目录（/mnt/data/xiangxin-guardrails-data/media/{user_uuid}/）
+  - 支持在线测试界面上传图片进行检测
+  - 新增图片上传组件和预览功能
+
+- 🔌 **API接口增强**
+  - 检测API支持混合消息（文本+图片）
+  - messages中的content支持数组格式：`[{"type": "text"}, {"type": "image_url"}]`
+  - 图片URL支持 `data:image/jpeg;base64,...` 和 `file://...` 两种格式
+  - 安全网关代理服务完整支持多模态请求透传
+
+- 📁 **新增文件**
+  - `backend/routers/media.py` - 媒体文件管理路由
+  - `backend/utils/image_utils.py` - 图片处理工具
+  - `backend/utils/url_signature.py` - URL签名验证工具
+  - `backend/scripts/migrate_add_image_fields.py` - 数据库迁移脚本
+  - `frontend/src/components/ImageUpload/` - 图片上传组件
+
+### 变更 Changed
+- 🔄 **检测服务增强**
+  - 检测模型调用逻辑支持多模态内容
+  - 检测结果数据库表新增图片相关字段
+  - 在线测试页面支持图片上传和预览
+
+- 🌐 **API响应格式**
+  - 保持与文本检测一致的响应格式
+  - 多标签风险支持：可返回多个unsafe标签（如：unsafe\nS1,S2）
+  - 敏感度分数和等级适用于图片检测
+
+### 技术特性 Technical Features
+- **图片检测模型**：基于视觉-语言模型的多模态安全检测
+- **存储管理**：用户级别的媒体文件隔离存储
+- **URL安全**：支持签名URL防止未授权访问
+- **格式兼容**：兼容OpenAI Vision API消息格式
+
+### 使用示例 Usage Examples
+
+#### Python API调用示例
+```python
+import base64
+from xiangxinai import XiangxinAI
+
+client = XiangxinAI("your-api-key")
+
+# 图片base64编码
+with open("image.jpg", "rb") as f:
+    image_base64 = base64.b64encode(f.read()).decode("utf-8")
+
+# 发送图片检测请求
+response = client.check_messages([
+    {
+        "role": "user",
+        "content": [
+            {"type": "text", "text": "这个图片安全吗？"},
+            {"type": "image_url", "image_url": {"url": f"data:image/jpeg;base64,{image_base64}"}}
+        ]
+    }
+])
+
+print(f"检测结果: {response.overall_risk_level}")
+print(f"风险类别: {response.all_categories}")
+```
+
+#### cURL调用示例
+```bash
+curl -X POST "http://localhost:5001/v1/guardrails" \
+    -H "Authorization: Bearer your-api-key" \
+    -H "Content-Type: application/json" \
+    -d '{
+      "model": "Xiangxin-Guardrails-VL",
+      "messages": [{
+        "role": "user",
+        "content": [
+          {"type": "text", "text": "这个图片安全吗？"},
+          {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}
+        ]
+      }],
+      "logprobs": true
+    }'
+```
+
 ## [2.2.0] - 2025-01-15
 
 ### 🚀 重大更新 Major Updates
@@ -49,7 +142,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ```
 
 ## [2.1.0] - 2025-09-29
-增加敏感度阈值配置功能，应对特殊场景和全自动流水线。
+增加敏感度阈值配置功能，可自定义检测的敏感度阈值，可用于应对特殊场景或全自动流水线场景。
 
 ## [2.0.0] - 2025-01-01
 

README.md

@@ -25,6 +25,7 @@ English | [中文](./README_ZH.md)
 
 - 🪄 **Two Usage Modes** - Detection API + Security Gateway
 - 🛡️ **Dual Protection** - Prompt attack detection + Content compliance detection
+- 🖼️ **Multimodal Detection** - Support for text and image content safety detection 🆕
 - 🧠 **Context Awareness** - Intelligent safety detection based on conversation context
 - 📋 **Compliance Standards** - Compliant with "GB/T45654—2025 Basic Security Requirements for Generative AI Services"
 - 🔧 **Flexible Configuration** - Blacklist/whitelist, response templates, rate limiting and other personalized configurations
@@ -34,11 +35,11 @@ English | [中文](./README_ZH.md)
 - 📊 **Visual Management** - Intuitive web management interface and real-time monitoring
 - ⚡ **High Performance** - Asynchronous processing, supporting high-concurrency access
 - 🔌 **Easy Integration** - Compatible with OpenAI API format, one-line code integration
-- 🎯 **Configurable Sensitivity** - Three-tier sensitivity threshold configuration for automated pipeline scenarios 🆕
+- 🎯 **Configurable Sensitivity** - Three-tier sensitivity threshold configuration for automated pipeline scenarios
 
 ## 🚀 Dual Mode Support
 
-Xiangxin AI Guardrails 2.1 supports two usage modes to meet different scenario requirements:
+Xiangxin AI Guardrails 2.3 supports two usage modes to meet different scenario requirements:
 
 ### 🔍 API Call Mode
 Developers **actively call** detection APIs for safety checks
@@ -394,7 +395,74 @@ User Request → Security Gateway(5002) → Input Safety Detection
 - **Smart Recognition**: Automatic detection of reasoning_content, thinking and other reasoning fields
 - **Transparent Proxy**: Full OpenAI API compatibility, supports all reasoning models
 
-## 🧠 Knowledge Base Responses Feature 🆕
+## 🖼️ Multimodal Detection Feature 🆕
+
+Xiangxin AI Guardrails v2.3.0 introduces **image modality detection**, expanding safety protection from text-only to multimodal content.
+
+### 📸 Key Features
+
+- **Image Content Detection**: AI-powered safety analysis of image content
+- **Unified Risk Standards**: Same risk categories (S1-S12) apply to both text and images
+- **Multiple Input Formats**: Support for base64-encoded images and image URLs
+- **Seamless Integration**: Compatible with both API Call Mode and Security Gateway Mode
+- **OpenAI Vision Compatible**: Supports OpenAI Vision API message format
+
+### 🔄 Usage Examples
+
+#### Python API - Image Detection
+```python
+import base64
+from xiangxinai import XiangxinAI
+
+client = XiangxinAI("your-api-key")
+
+# Encode image to base64
+with open("image.jpg", "rb") as f:
+    image_base64 = base64.b64encode(f.read()).decode("utf-8")
+
+# Check image safety
+response = client.check_messages([
+    {
+        "role": "user",
+        "content": [
+            {"type": "text", "text": "Is this image safe?"},
+            {
+                "type": "image_url",
+                "image_url": {"url": f"data:image/jpeg;base64,{image_base64}"}
+            }
+        ]
+    }
+])
+
+print(f"Risk Level: {response.overall_risk_level}")
+print(f"Risk Categories: {response.all_categories}")
+```
+
+#### HTTP API - Image Detection
+```bash
+curl -X POST "http://localhost:5001/v1/guardrails" \
+    -H "Authorization: Bearer your-api-key" \
+    -H "Content-Type: application/json" \
+    -d '{
+      "model": "Xiangxin-Guardrails-VL",
+      "messages": [{
+        "role": "user",
+        "content": [
+          {"type": "text", "text": "Is this image safe?"},
+          {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}
+        ]
+      }]
+    }'
+```
+
+### 🎯 Use Cases
+
+- **Social Media**: Automatically screen user-uploaded images for unsafe content
+- **E-commerce**: Ensure product images comply with platform policies
+- **Education**: Protect minors from inappropriate image content
+- **Content Platforms**: Moderate AI-generated images before publication
+
+## 🧠 Knowledge Base Responses Feature
 
 Xiangxin AI Guardrails v2.2.0 introduces powerful knowledge base response functionality with vector similarity-based intelligent Q&A matching.
 
@@ -1047,13 +1115,15 @@ We provide professional AI safety solutions:
 Xiangxin AI Guardrails will continue to evolve in two directions: **Detection Capabilities** and **Platform Features**, ensuring that large model applications run under safe and compliant conditions.
 
 ### 🔍 Detection Capabilities
+- ✅ **Image Modality Detection** (v2.3.0): AI-powered image content safety analysis
+- **Audio & Video Detection**: Support for audio and video content safety analysis (Coming Soon)
 - **Multimodal Subtle Violation Content Recognition**: Support multimodal inputs including text, images, audio, and video, identifying and intercepting subtle violations or illegal information.
 - **Role-based Privilege Escalation Detection**: Combined with context and user identity, identify and intercept privilege escalation questions or sensitive information requests.
 - **Personal Information & Sensitive Data Detection**: Automatically identify and intercept content involving personal information, business secrets, and other sensitive content to prevent data leaks.
 - **Out-of-business-scope Content Detection**: Identify and intervene in questions/outputs that exceed business scenarios or compliance boundaries.
 
 ### 🛡️ Platform Features
-- **Multimodal Content Recognition Support**: Provide security recognition matching actual application modalities (text, images, audio, video, files).
+- ✅ **Multimodal Content Recognition Support** (v2.3.0): Text and image safety detection available
 - **Sensitive Information Interception & Desensitization**: When sensitive content is detected, it can be directly intercepted or automatically desensitized based on rules before output.
 - **Desensitization Rule Configuration**: Support user-defined desensitization strategies, flexibly adapting to compliance requirements in different scenarios.
 - **Out-of-business-scope Control**: Block or substitute answers for privilege escalation or inappropriate questions, ensuring compliant output.

README_ZH.md

@@ -25,11 +25,12 @@
 
 - 🪄 **两种使用模式** - 检测API + 安全网关
 - 🛡️ **双重防护** - 提示词攻击检测 + 内容合规检测
+- 🖼️ **多模态检测** - 支持文本和图片内容安全检测 🆕
 - 🧠 **上下文感知** - 基于对话上下文的智能安全检测
 - 📋 **合规标准** - 符合《GB/T45654—2025 生成式人工智能服务安全基本要求》
 - 🔧 **灵活配置** - 黑白名单、代答库、限速等个性化配置
-- 🧠 **代答知识库** - 基于向量相似度的智能问答匹配，支持自定义问答对知识库 🆕
-- 🎯 **敏感度阈值配置** - 三档敏感度阈值配置，适应自动化流水线等不同使用场景 🆕
+- 🧠 **代答知识库** - 基于向量相似度的智能问答匹配，支持自定义问答对知识库
+- 🎯 **敏感度阈值配置** - 三档敏感度阈值配置，适应自动化流水线等不同使用场景
 - 🏢 **私有化部署** - 支持完全本地化部署，数据安全可控
 - 🔌 **客户系统集成** - 支持与客户现有用户系统深度集成，API级别的配置管理
 - 📊 **可视化管理** - 直观的Web管理界面和实时监控
@@ -38,7 +39,7 @@
 
 ## 🚀 双模式支持
 
-象信AI安全护栏2.1支持两种使用模式，满足不同场景需求：
+象信AI安全护栏2.3支持两种使用模式，满足不同场景需求：
 
 ### 🔍 API调用模式
 开发者**主动调用**检测API进行安全检测
@@ -392,7 +393,74 @@ response = client.chat.completions.create(model="local-reasoning-llm", messages=
                [通过检测] → 返回给用户
 ```
 
-## 🧠 代答知识库功能 🆕
+## 🖼️ 多模态检测功能 🆕
+
+象信AI安全护栏v2.3.0新增**图片模态检测**功能，将安全防护从纯文本扩展到多模态内容。
+
+### 📸 核心功能
+
+- **图片内容检测**：AI智能分析图片内容的安全性
+- **统一风险标准**：图片和文本使用相同的风险类型（S1-S12）
+- **多种输入格式**：支持base64编码图片和图片URL
+- **无缝集成**：兼容API调用模式和安全网关模式
+- **OpenAI Vision兼容**：支持OpenAI Vision API消息格式
+
+### 🔄 使用示例
+
+#### Python API - 图片检测
+```python
+import base64
+from xiangxinai import XiangxinAI
+
+client = XiangxinAI("your-api-key")
+
+# 图片base64编码
+with open("image.jpg", "rb") as f:
+    image_base64 = base64.b64encode(f.read()).decode("utf-8")
+
+# 检测图片安全性
+response = client.check_messages([
+    {
+        "role": "user",
+        "content": [
+            {"type": "text", "text": "这个图片安全吗？"},
+            {
+                "type": "image_url",
+                "image_url": {"url": f"data:image/jpeg;base64,{image_base64}"}
+            }
+        ]
+    }
+])
+
+print(f"风险等级: {response.overall_risk_level}")
+print(f"风险类别: {response.all_categories}")
+```
+
+#### HTTP API - 图片检测
+```bash
+curl -X POST "http://localhost:5001/v1/guardrails" \
+    -H "Authorization: Bearer your-api-key" \
+    -H "Content-Type: application/json" \
+    -d '{
+      "model": "Xiangxin-Guardrails-VL",
+      "messages": [{
+        "role": "user",
+        "content": [
+          {"type": "text", "text": "这个图片安全吗？"},
+          {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}
+        ]
+      }]
+    }'
+```
+
+### 🎯 应用场景
+
+- **社交媒体**：自动筛查用户上传的图片内容
+- **电商平台**：确保商品图片符合平台规范
+- **教育平台**：保护未成年人免受不良图片影响
+- **内容平台**：审核AI生成的图片内容
+
+## 🧠 代答知识库功能
 
 象信AI安全护栏v2.2.0新增了强大的代答知识库功能，基于向量相似度搜索提供智能问答匹配。
 
@@ -1272,16 +1340,18 @@ git push origin feature/amazing-feature
 象信 AI 安全护栏将持续演进，在 **检测能力** 和 **平台功能** 两个方向不断增强，确保大模型应用在安全、合规的前提下运行。
 
 ### 🔍 检测能力
-- **多模态隐晦违规内容识别**：支持文本、图像、音频、视频等多模态输入，识别并拦截隐蔽的违规或违法信息。  
-- **基于用户角色的越权检测**：结合上下文与用户身份，识别并拦截越权提问或敏感信息请求。  
-- **个人信息与敏感数据检测**：自动识别、拦截涉及个人信息、商业秘密等敏感内容，防止数据泄露。  
-- **超业务范围内容检测**：对超出业务场景或合规边界的提问/输出进行识别和干预。  
+- ✅ **图片模态检测** (v2.3.0)：AI智能分析图片内容的安全性
+- **音频视频检测**：支持音频和视频内容安全分析（即将推出）
+- **多模态隐晦违规内容识别**：支持文本、图像、音频、视频等多模态输入，识别并拦截隐蔽的违规或违法信息。
+- **基于用户角色的越权检测**：结合上下文与用户身份，识别并拦截越权提问或敏感信息请求。
+- **个人信息与敏感数据检测**：自动识别、拦截涉及个人信息、商业秘密等敏感内容，防止数据泄露。
+- **超业务范围内容检测**：对超出业务场景或合规边界的提问/输出进行识别和干预。
 
 ### 🛡️ 平台功能
-- **多模态内容识别支持**：提供与实际应用模态匹配的安全识别（文本、图像、音频、视频、文件）。  
-- **敏感信息拦截与脱敏**：在检测到敏感内容时，可直接拦截或基于规则进行自动脱敏后输出。  
-- **脱敏规则配置**：支持用户自定义脱敏策略，灵活适配不同场景的合规需求。  
-- **超业务范围管控**：对越权或不当提问进行拒答或代答，确保输出合规。  
+- ✅ **多模态内容识别支持** (v2.3.0)：文本和图片安全检测已上线
+- **敏感信息拦截与脱敏**：在检测到敏感内容时，可直接拦截或基于规则进行自动脱敏后输出。
+- **脱敏规则配置**：支持用户自定义脱敏策略，灵活适配不同场景的合规需求。
+- **超业务范围管控**：对越权或不当提问进行拒答或代答，确保输出合规。
 - **可配置的代答知识库**：支持可配置、可扩展、可持续更新的标准代答知识库，保障应答一致性和可控性。  
 
 本路线图会随着 **安全攻防形势** 与 **合规要求** 的变化持续更新，欢迎社区用户提出建议和贡献。

backend/.env.example

@@ -20,9 +20,23 @@ GUARDRAILS_MODEL_API_URL=http://your-host-ip:your-port/v1
 GUARDRAILS_MODEL_API_KEY=your-guardrails-model-api-key
 GUARDRAILS_MODEL_NAME=Xiangxin-Guardrails-Text
 
+# 多模态模型配置
+GUARDRAILS_VL_MODEL_API_URL=http://localhost:58003/v1
+GUARDRAILS_VL_MODEL_API_KEY=your-vl-model-api-key
+GUARDRAILS_VL_MODEL_NAME=Xiangxin-Guardrails-VL
+
 # 检测最大上下文长度配置 (应该等于模型max-model-len - 1000)
 MAX_DETECTION_CONTEXT_LENGTH=7168
 
+# 嵌入模型API配置
+# 用于知识库向量化的嵌入模型API
+EMBEDDING_API_BASE_URL=http://your-host-ip:your-port/v1
+EMBEDDING_API_KEY=your-embedding-api-key
+EMBEDDING_MODEL_NAME=Xiangxin-Embedding-1024
+EMBEDDING_MODEL_DIMENSION=1024
+EMBEDDING_SIMILARITY_THRESHOLD=0.7
+EMBEDDING_MAX_RESULTS=5
+
 # API配置
 CORS_ORIGINS=*
 
@@ -35,14 +49,6 @@ SUPPORT_EMAIL=wanglei@xiangxinai.cn
 # HuggingFace模型
 HUGGINGFACE_MODEL=xiangxinai/Xiangxin-Guardrails-Text
 
-# 嵌入模型API配置
-# 用于知识库向量化的嵌入模型API
-EMBEDDING_API_BASE_URL=http://your-host-ip:your-port/v1
-EMBEDDING_API_KEY=your-embedding-api-key
-EMBEDDING_MODEL_NAME=Xiangxin-Embedding-1024
-EMBEDDING_MODEL_DIMENSION=1024
-EMBEDDING_SIMILARITY_THRESHOLD=0.7
-EMBEDDING_MAX_RESULTS=5
 
 # JWT配置
 # 警告：请生成一个安全的随机密钥！可以使用: openssl rand -base64 64