Skip to content

Commit 9bc5b11

Browse files
DefTruthjiangjiajun
DefTruth
and
jiangjiajun
authored
[Doc] Add PicoDet & PaddleClas Android demo docs (#412)
* [Backend] Add override flag to lite backend * [Docs] Add Android C++ SDK build docs * [Doc] fix android_build_docs typos * Update CMakeLists.txt * Update android.md * [Doc] Add PicoDet Android demo docs * [Doc] Update PicoDet Andorid demo docs * [Doc] Update PaddleClasModel Android demo docs * [Doc] Update fastdeploy android jni docs * [Doc] Update fastdeploy android jni usage docs Co-authored-by: Jason <jiangjiajun@baidu.com>
1 parent f2619b0 commit 9bc5b11

File tree

9 files changed

+599
-4
lines changed

9 files changed

+599
-4
lines changed

‎docs/cn/build_and_install/android.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,3 +102,4 @@ make install
102102
如何使用FastDeploy Android C++ SDK 请参考使用案例文档：
103103
- [图像分类Android使用文档](../../../examples/vision/classification/paddleclas/android/README.md)
104104
- [目标检测Android使用文档](../../../examples/vision/detection/paddledetection/android/README.md)
105+
- [在 Android 通过 JNI 中使用 FastDeploy C++ SDK](../../../../../docs/cn/faq/use_cpp_sdk_on_android.md)

‎docs/cn/faq/use_cpp_sdk_on_android.md

Lines changed: 231 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,231 @@
1+
# 在 Android 中通过 JNI 使用 FastDeploy C++ SDK
2+
本文档将以PicoDet为例，讲解如何通过JNI，将FastDeploy中的模型封装到Android中进行调用。阅读本文档，您至少需要了解C++、Java、JNI以及Android的基础知识。如果您主要关注如何在Java层如何调用FastDeploy的API，则可以不阅读本文档。
3+
4+
## 目录
5+
- [新建Java类并定义native API](#Java)
6+
- [Android Studio 生成JNI函数定义](#JNI)
7+
- [在C++层实现JNI函数](#CPP)
8+
- [编写CMakeLists.txt及配置build.gradle](#CMakeAndGradle)
9+
- [更多FastDeploy Android 使用案例](#Examples)
10+
11+
## 新建Java类并定义native API
12+
<div id="Java"></div>
13+
14+
```java
15+
public class PicoDet {
16+
protected long mNativeModelContext = 0; // Context from native.
17+
protected boolean mInitialized = false;
18+
// ...
19+
// Bind predictor from native context.
20+
private static native long bindNative(String modelFile,
21+
String paramsFile,
22+
String configFile,
23+
int cpuNumThread,
24+
boolean enableLiteFp16,
25+
int litePowerMode,
26+
String liteOptimizedModelDir,
27+
boolean enableRecordTimeOfRuntime,
28+
String labelFile);
29+
30+
// Call prediction from native context.
31+
private static native long predictNative(long nativeModelContext,
32+
Bitmap ARGB8888Bitmap,
33+
boolean saved,
34+
String savedImagePath,
35+
float scoreThreshold,
36+
boolean rendering);
37+
38+
// Release buffers allocated in native context.
39+
private static native boolean releaseNative(long nativeModelContext);
40+
41+
// Initializes at the beginning.
42+
static {
43+
FastDeployInitializer.init();
44+
}
45+
}
46+
```
47+
这些被标记为native的接口是需要通过JNI的方式实现，并在Java层供PicoDet类调用。完整的PicoDet Java代码请参考 [PicoDet.java](../../../examples/vision/detection/paddledetection/android/app/src/main/java/com/baidu/paddle/fastdeploy/vision/detection/PicoDet.java) 。各个函数说明如下：
48+
- `bindNative`: C++层初始化模型资源，如果成功初始化，则返回指向该模型的指针(long类型)，否则返回0指针
49+
- `predictNative`: 通过已经初始化好的模型指针，在C++层执行预测代码，如果预测成功则返回指向预测结果的指针，否则返回0指针。注意，该结果指针在当次预测使用完之后需要释放，具体操作请参考 [PicoDet.java](../../../examples/vision/detection/paddledetection/android/app/src/main/java/com/baidu/paddle/fastdeploy/vision/detection/PicoDet.java) 中的predict函数。
50+
- `releaseNative`: 根据传入的模型指针，在C++层释放模型资源。
51+
52+
## Android Studio 生成JNI函数定义
53+
<div id="JNI"></div>
54+
55+
Android Studio 生成 JNI 函数定义: 鼠标停留在Java中定义的native函数上，Android Studio 便会提示是否要创建JNI函数定义；这里，我们把JNI函数定义创建在一个事先创建好的c++文件`picodet_jni.cc`上;
56+
57+
- 使用Android Studio创建JNI函数定义：
58+
![](https://user-images.githubusercontent.com/31974251/197341065-cdf8f626-4bb1-4a57-8d7a-80b382fe994e.png)
59+
60+
- 将JNI函数定义创建在picodet_jni.cc上：
61+
![](https://user-images.githubusercontent.com/31974251/197341190-b887dec5-fa75-43c9-9ab3-7ead50c0eb45.png)
62+
63+
- 创建的JNI函数定义如下：
64+
![](https://user-images.githubusercontent.com/31974251/197341274-e9671bac-9e77-4043-a870-9d5db914586b.png)
65+
66+
其他native函数对应的JNI函数定义的创建和此流程一样。
67+
68+
## 在C++层实现JNI函数
69+
<div id="CPP"></div>
70+
71+
以下为PicoDet JNI层实现的示例，相关的辅助函数不在此处赘述，完整的C++代码请参考 [android/app/src/main/cpp](../../../examples/vision/detection/paddledetection/android/app/src/main/cpp/).
72+
```C++
73+
#include <jni.h> // NOLINT
74+
#include "fastdeploy_jni.h" // NOLINT
75+
76+
#ifdef __cplusplus
77+
extern "C" {
78+
#endif
79+
80+
// 绑定C++层的模型
81+
JNIEXPORT jlong JNICALL
82+
Java_com_baidu_paddle_fastdeploy_vision_detection_PicoDet_bindNative(
83+
JNIEnv *env, jclass clazz, jstring model_file, jstring params_file,
84+
jstring config_file, jint cpu_num_thread, jboolean enable_lite_fp16,
85+
jint lite_power_mode, jstring lite_optimized_model_dir,
86+
jboolean enable_record_time_of_runtime, jstring label_file) {
87+
std::string c_model_file = fastdeploy::jni::ConvertTo<std::string>(env, model_file);
88+
std::string c_params_file = fastdeploy::jni::ConvertTo<std::string>(env, params_file);
89+
std::string c_config_file = astdeploy::jni::ConvertTo<std::string>(env, config_file);
90+
std::string c_label_file = fastdeploy::jni::ConvertTo<std::string>(env, label_file);
91+
std::string c_lite_optimized_model_dir = fastdeploy::jni::ConvertTo<std::string>(env, lite_optimized_model_dir);
92+
auto c_cpu_num_thread = static_cast<int>(cpu_num_thread);
93+
auto c_enable_lite_fp16 = static_cast<bool>(enable_lite_fp16);
94+
auto c_lite_power_mode = static_cast<fastdeploy::LitePowerMode>(lite_power_mode);
95+
fastdeploy::RuntimeOption c_option;
96+
c_option.UseCpu();
97+
c_option.UseLiteBackend();
98+
c_option.SetCpuThreadNum(c_cpu_num_thread);
99+
c_option.SetLitePowerMode(c_lite_power_mode);
100+
c_option.SetLiteOptimizedModelDir(c_lite_optimized_model_dir);
101+
if (c_enable_lite_fp16) {
102+
c_option.EnableLiteFP16();
103+
}
104+
// 如果您实现的是其他模型，比如PPYOLOE，请注意修改此处绑定的C++类型
105+
auto c_model_ptr = new fastdeploy::vision::detection::PicoDet(
106+
c_model_file, c_params_file, c_config_file, c_option);
107+
// Enable record Runtime time costs.
108+
if (enable_record_time_of_runtime) {
109+
c_model_ptr->EnableRecordTimeOfRuntime();
110+
}
111+
// Load detection labels if label path is not empty.
112+
if ((!fastdeploy::jni::AssetsLoaderUtils::IsDetectionLabelsLoaded()) &&
113+
(!c_label_file.empty())) {
114+
fastdeploy::jni::AssetsLoaderUtils::LoadDetectionLabels(c_label_file);
115+
}
116+
// WARN: need to release manually in Java !
117+
return reinterpret_cast<jlong>(c_model_ptr); // native model context
118+
}
119+
120+
// 通过传入的模型指针在C++层进行预测
121+
JNIEXPORT jlong JNICALL
122+
Java_com_baidu_paddle_fastdeploy_vision_detection_PicoDet_predictNative(
123+
JNIEnv *env, jclass clazz, jlong native_model_context,
124+
jobject argb8888_bitmap, jboolean saved, jstring saved_image_path,
125+
jfloat score_threshold, jboolean rendering) {
126+
if (native_model_context == 0) {
127+
return 0;
128+
}
129+
cv::Mat c_bgr;
130+
if (!fastdeploy::jni::ARGB888Bitmap2BGR(env, argb8888_bitmap, &c_bgr)) {
131+
return 0;
132+
}
133+
auto c_model_ptr = reinterpret_cast<fastdeploy::vision::detection::PicoDet *>(
134+
native_model_context);
135+
auto c_result_ptr = new fastdeploy::vision::DetectionResult();
136+
t = fastdeploy::jni::GetCurrentTime();
137+
if (!c_model_ptr->Predict(&c_bgr, c_result_ptr)) {
138+
delete c_result_ptr;
139+
return 0;
140+
}
141+
// ...
142+
return reinterpret_cast<jlong>(c_result_ptr); // native result context
143+
}
144+
145+
// 在C++层释放模型资源
146+
JNIEXPORT jboolean JNICALL
147+
Java_com_baidu_paddle_fastdeploy_vision_detection_PicoDet_releaseNative(
148+
JNIEnv *env, jclass clazz, jlong native_model_context) {
149+
if (native_model_context == 0) {
150+
return JNI_FALSE;
151+
}
152+
auto c_model_ptr = reinterpret_cast<fastdeploy::vision::detection::PicoDet *>(
153+
native_model_context);
154+
// ...
155+
delete c_model_ptr;
156+
return JNI_TRUE;
157+
}
158+
159+
#ifdef __cplusplus
160+
}
161+
#endif
162+
```
163+
## 编写CMakeLists.txt及配置build.gradle
164+
<div id="CMakeAndGradle"></div>
165+
166+
实现好的JNI代码，需要被编译成so库，才能被Java调用，为实现该目的，需要在build.gradle中添加JNI项目支持，并编写对应的CMakeLists.txt。
167+
- build.gradle中配置NDK、CMake以及Android ABI
168+
```java
169+
android {
170+
defaultConfig {
171+
// 省略其他配置 ...
172+
externalNativeBuild {
173+
cmake {
174+
arguments '-DANDROID_PLATFORM=android-21', '-DANDROID_STL=c++_shared', "-DANDROID_TOOLCHAIN=clang"
175+
abiFilters 'armeabi-v7a', 'arm64-v8a'
176+
cppFlags "-std=c++11"
177+
}
178+
}
179+
}
180+
// 省略其他配置 ...
181+
externalNativeBuild {
182+
cmake {
183+
path file('src/main/cpp/CMakeLists.txt')
184+
version '3.10.2'
185+
}
186+
}
187+
ndkVersion '20.1.5948944'
188+
}
189+
```
190+
- 编写CMakeLists.txt示例
191+
```cmake
192+
cmake_minimum_required(VERSION 3.10.2)
193+
project("fastdeploy_jni")
194+
195+
set(FastDeploy_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../../../libs/fastdeploy-android-0.4.0-shared")
196+
197+
find_package(FastDeploy REQUIRED)
198+
199+
include_directories(${CMAKE_CURRENT_SOURCE_DIR})
200+
include_directories(${FastDeploy_INCLUDE_DIRS})
201+
202+
add_library(
203+
fastdeploy_jni
204+
SHARED
205+
utils_jni.cc
206+
bitmap_jni.cc
207+
vision/results_jni.cc
208+
vision/visualize_jni.cc
209+
vision/detection/picodet_jni.cc
210+
vision/classification/paddleclas_model_jni.cc)
211+
212+
find_library(log-lib log)
213+
214+
target_link_libraries(
215+
# Specifies the target library.
216+
fastdeploy_jni
217+
jnigraphics
218+
${FASTDEPLOY_LIBS}
219+
GLESv2
220+
EGL
221+
${log-lib}
222+
)
223+
```
224+
完整的工程示例，请参考 [android/app/src/main/cpp/CMakelists.txt](../../../examples/vision/detection/paddledetection/android/app/src/main/cpp/) 以及 [android/app/build.gradle](../../../examples/vision/detection/paddledetection/android/app/build.gradle).
225+
226+
## 更多FastDeploy Android 使用案例
227+
<div id="Examples"></div>
228+
229+
更多FastDeploy Android 使用案例请参考以下文档:
230+
- [图像分类Android使用文档](../../../examples/vision/classification/paddleclas/android/README.md)
231+
- [目标检测Android使用文档](../../../examples/vision/detection/paddledetection/android/README.md)

‎docs/cn/faq/use_java_sdk_on_android.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
## 在 Android 中使用 FastDeploy Java SDK
2+
- TODO

0 commit comments

Comments
 (0)