diff --git a/demos/TTSAndroid/.gitignore b/demos/TTSAndroid/.gitignore new file mode 100644 index 00000000000..2b75303ac58 --- /dev/null +++ b/demos/TTSAndroid/.gitignore @@ -0,0 +1,13 @@ +*.iml +.gradle +/local.properties +/.idea/caches +/.idea/libraries +/.idea/modules.xml +/.idea/workspace.xml +/.idea/navEditor.xml +/.idea/assetWizardSettings.xml +.DS_Store +/build +/captures +.externalNativeBuild diff --git a/demos/TTSAndroid/README.md b/demos/TTSAndroid/README.md new file mode 100644 index 00000000000..d6013562081 --- /dev/null +++ b/demos/TTSAndroid/README.md @@ -0,0 +1,189 @@ +# 语音合成 Java API Demo 使用指南 + +在 Android 上实现语音合成功能，此 Demo 有很好的的易用性和开放性，如在 Demo 中跑自己训练好的模型等。 + +本文主要介绍语音合成 Demo 运行方法。 + +## 如何运行语音合成 Demo + +### 环境准备 + +1. 在本地环境安装好 Android Studio 工具，详细安装方法请见 [Android Stuido 官网](https://developer.android.com/studio)。 +2. 准备一部 Android 手机，并开启 USB 调试模式。开启方法: `手机设置 -> 查找开发者选项 -> 打开开发者选项和 USB 调试模式`。 + +**注意**： +> 如果您的 Android Studio 尚未配置 NDK ，请根据 Android Studio 用户指南中的[安装及配置 NDK 和 CMake ](https://developer.android.com/studio/projects/install-ndk)内容，预先配置好 NDK 。您可以选择最新的 NDK 版本，或者使用 Paddle Lite 预测库版本一样的 NDK。 + +### 部署步骤 + +1. 用 Android Studio 打开 TTSAndroid 工程。 +2. 手机连接电脑，打开 USB 调试和文件传输模式，并在 Android Studio 上连接自己的手机设备（手机需要开启允许从 USB 安装软件权限）。 + +**注意：** +>1. 如果您在导入项目、编译或者运行过程中遇到 NDK 配置错误的提示，请打开 `File > Project Structure > SDK Location`，修改 `Andriod NDK location` 为您本机配置的 NDK 所在路径。 +>2. 如果您是通过 Andriod Studio 的 SDK Tools 下载的 NDK (见本章节"环境准备")，可以直接点击下拉框选择默认路径。 +>3. 还有一种 NDK 配置方法，你可以在 `TTSAndroid/local.properties` 文件中手动添加 NDK 路径配置 `nkd.dir=/root/android-ndk-r20b` +>4. 如果以上步骤仍旧无法解决 NDK 配置错误，请尝试根据 Andriod Studio 官方文档中的[更新 Android Gradle 插件](https://developer.android.com/studio/releases/gradle-plugin?hl=zh-cn#updating-plugin)章节，尝试更新 Android Gradle plugin 版本。 + +3. 点击 Run 按钮，自动编译 APP 并安装到手机。(该过程会自动下载 Paddle Lite 预测库和模型，需要联网) + 成功后效果如下： + - pic 1：APP 安装到手机。 + - pic 2：APP 打开后的效果，在下拉框中选择待合成的文本。 + - pic 3：合成后点击按钮播放音频。 + +

+ +## 更新预测库 + +* Paddle Lite + 项目：[https://github.com/PaddlePaddle/Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite)。 + + +参考 [Paddle Lite 源码编译文档](https://www.paddlepaddle.org.cn/lite/v2.11/source_compile/compile_env.html)，编译 +Android 预测库。 + +* 编译最终产物位于 `build.lite.xxx.xxx.xxx` 下的 `inference_lite_lib.xxx.xxx` +* 替换 java 库 + * jar 包 + 将生成的 `build.lite.android.xxx.gcc/inference_lite_lib.android.xxx/java/jar/PaddlePredictor.jar` + 替换 Demo 中的 `TTSAndroid/app/libs/PaddlePredictor.jar`。 + * Java so + * arm64-v8a + 将生成的 `build.lite.android.armv8.gcc/inference_lite_lib.android.armv8/java/so/libpaddle_lite_jni.so` + 库替换 Demo 中的 `TTSAndroid/app/src/main/jniLibs/arm64-v8a/libpaddle_lite_jni.so`。 + +## Demo 内容介绍 + +先整体介绍下目标检测 Demo 的代码结构，然后介绍 Java 各功能模块的功能。 + +

+image +

+ +### 重点关注内容 + +1. `Predictor.java`： 预测代码。 + +```bash +# 位置： +TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/Predictor.java +``` + +2. `fastspeech2_csmsc_arm.nb` 和 `mb_melgan_csmsc_arm.nb`: 模型文件 (opt 工具转化后 Paddle Lite 模型) + ，分别来自 [fastspeech2_cnndecoder_csmsc_pdlite_1.3.0.zip](https://paddlespeech.bj.bcebos.com/Parakeet/released_models/fastspeech2/fastspeech2_cnndecoder_csmsc_pdlite_1.3.0.zip) + 和 [mb_melgan_csmsc_pdlite_1.3.0.zip](https://paddlespeech.bj.bcebos.com/Parakeet/released_models/mb_melgan/mb_melgan_csmsc_pdlite_1.3.0.zip)。 + +```bash +# 位置： +TTSAndroid/app/src/main/assets/models/cpu/fastspeech2_csmsc_arm.nb +TTSAndroid/app/src/main/assets/models/cpu/mb_melgan_csmsc_arm.nb +``` + +3. `libpaddle_lite_jni.so`、`PaddlePredictor.jar`：Paddle Lite Java 预测库与 jar 包。 + +```bash +# 位置 +TTSAndroid/app/src/main/jniLibs/arm64-v8a/libpaddle_lite_jni.so +TTSAndroid/app/libs/PaddlePredictor.jar +``` + +> 如果要替换动态库 so 和 jar 文件，则将新的动态库 so 更新到 `TTSAndroid/app/src/main/jniLibs/arm64-v8a/` 目录下 新的 jar 文件更新到 `TTSAndroid/app/libs/` 目录下 + +4. `build.gradle` : 定义编译过程的 gradle 脚本。（不用改动，定义了自动下载 Paddle Lite 预测和模型的过程） + +```bash +# 位置 +TTSAndroid/app/build.gradle +``` + +如果需要手动更新模型和预测库，则可将 gradle 脚本中的 `download*` 接口注释即可, 将新的预测库替换至相应目录下 + +### Java 端 + +* 模型存放，将下载好的模型解压存放在 `app/src/assets/models` 目录下。 +* TTSAndroid Java 包在 `app/src/main/java/com/baidu/paddle/lite/demo/tts` 目录下，实现 APP 界面消息事件。 +* MainActivity 实现 APP 的创建、运行、释放功能，重点关注 `onLoadModel` 和 `onRunModel` 函数，实现 APP 界面值传递和推理处理。 + + ```java + public boolean onLoadModel() { + return predictor.init(MainActivity.this, modelPath, AMmodelName, VOCmodelName, cpuThreadNum, + cpuPowerMode); + } + + public boolean onRunModel() { + return predictor.isLoaded() && predictor.runModel(phones); + } + ``` + +* SettingActivity 实现设置界面各个元素的更新与显示如模型地址、线程数、输入 shape 大小等，如果新增/删除界面的某个元素，均在这个类里面实现： + - 参数的默认值可在 `app/src/main/res/values/strings.xml` 查看 + - 每个元素的 ID 和 value 是对应 `app/src/main/res/xml/settings.xml` + 和 `app/src/main/res/values/string.xml` 文件中的值 + - 这部分内容不建议修改，如果有新增属性，可以按照此格式进行添加 + +* Predictor 使用 Java API 实现语音合成模型的预测功能，重点关注 `init`、和 `runModel` 函数，实现 Paddle Lite 端侧推理功能： + ```java + // 初始化函数，完成预测器初始化 + public boolean init(Context appCtx, String modelPath, String AMmodelName, String VOCmodelName, int cpuThreadNum, String cpuPowerMode); + // 模型推理函数 + public boolean runModel(float[] phones); + ``` + +## 代码讲解 （使用 Paddle Lite `Java API` 执行预测） + +Android 示例基于 Java API 开发，调用 Paddle Lite `Java API` 包括以下五步。更详细的 `API` +描述参考：[Paddle Lite Java API ](https://www.paddlepaddle.org.cn/lite/v2.11/api_reference/java_api_doc.html)。 + +## 如何更新模型和输入 + +### 更新模型 + +1. 将优化后的模型存放到目录 `TTSAndroid/app/src/main/assets/models/cpu/` + 下，可任意换成 [released_model.md](https://github.com/PaddlePaddle/PaddleSpeech/blob/develop/docs/source/released_model.md) + 中的 `*_pdlite_*.zip/*_arm.nb` + 格式的声学模型和声码器，注意更换声学模型需要对应修改 `TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/MainActivity.java` + 中的 `sentencesToChoose` 数组。 +2. 如果模型名字跟工程中模型名字一模一样，即均是使用`fastspeech2_csmsc_arm.nb` （假设声学模型的 `phone_id_map.txt` + 也一样）和 `mb_melgan_csmsc_arm.nb` + ，则代码不需更新；否则，需要修改 `TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/MainActivity.java` + 中的 `AMmodelName` 和 `VOCmodelName`： + +

+ +

+ +3. 如果更新模型的输入/输出 Tensor 个数、shape 和 Dtype + 发生更新，需要更新文件 `TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/Predictor.java`。 + +### 更新输入 + +**本 Demo 不包含文本前端模块**，通过下拉框选择预先设置好的文本，在代码中映射成对应的 phone_id，**如需文本前端模块请自行处理**，`phone_id_map.txt` +请参考 [fastspeech2_cnndecoder_csmsc_pdlite_1.3.0.zip](https://paddlespeech.bj.bcebos.com/Parakeet/released_models/fastspeech2/fastspeech2_cnndecoder_csmsc_pdlite_1.3.0.zip)。 + +## 通过 setting 界面更新语音合成的相关参数 + +### setting 界面参数介绍 + +可通过 APP 上的 Settings 按钮，实现语音合成 Demo 中参数的更新，目前支持以下参数的更新： +参数的默认值可在 `app/src/main/res/values/strings.xml` 查看 + +- CPU setting： + - power_mode 默认是 `LITE_POWER_HIGH` + - thread_num 默认是 1 + +### setting 界面参数更新 + +1. 打开 APP，点击右上角的 `:` 符合，选择 `Settings..` 选项，打开 setting 界面； +2. 再将 setting 界面的 Enable custom settings 选中☑️，然后更新部分参数； +3. 假设更新线程数据，将 CPU Thread Num 设置为 4，更新后，返回原界面，APP 将自动重新加载模型，在下拉框中选择文本会进行合成，合成结束后悔打印 4 线程的耗时和结果 + +## 性能优化方法 + +如果你觉得当前性能不符合需求，想进一步提升模型性能，可参考[性能优化文档](https://github.com/PaddlePaddle/Paddle-Lite-Demo#%E6%80%A7%E8%83%BD%E4%BC%98%E5%8C%96)完成性能优化。 + +## Release + +[2022-11-29-app-release.apk](https://paddlespeech.bj.bcebos.com/demos/TTSAndroid/2022-11-29-app-release.apk) + +## More +本 Demo 合并自 [yt605155624/TTSAndroid](https://github.com/yt605155624/TTSAndroid)。 diff --git a/demos/TTSAndroid/app/.gitignore b/demos/TTSAndroid/app/.gitignore new file mode 100644 index 00000000000..796b96d1c40 --- /dev/null +++ b/demos/TTSAndroid/app/.gitignore @@ -0,0 +1 @@ +/build diff --git a/demos/TTSAndroid/app/build.gradle b/demos/TTSAndroid/app/build.gradle new file mode 100644 index 00000000000..40ee5e12344 --- /dev/null +++ b/demos/TTSAndroid/app/build.gradle @@ -0,0 +1,108 @@ +import java.security.MessageDigest + +apply plugin: 'com.android.application' + +android { + compileSdkVersion 28 + defaultConfig { + applicationId "com.baidu.paddle.lite.demo.tts" + minSdkVersion 15 + targetSdkVersion 28 + versionCode 1 + versionName "1.0" + testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner" + } + buildTypes { + release { + minifyEnabled false + proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro' + } + } +} + +dependencies { + implementation fileTree(include: ['*.jar'], dir: 'libs') + implementation 'com.android.support:appcompat-v7:28.0.0' + implementation 'com.android.support.constraint:constraint-layout:1.1.3' + implementation 'com.android.support:design:28.0.0' + testImplementation 'junit:junit:4.12' + androidTestImplementation 'com.android.support.test:runner:1.0.2' + androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.2' + implementation files('libs/PaddlePredictor.jar') +} + +def paddleLiteLibs = 'https://paddlespeech.bj.bcebos.com/demos/TTSAndroid/paddle_lite_libs_68b66fd3.tar.gz' +task downloadAndExtractPaddleLiteLibs(type: DefaultTask) { + doFirst { + println "Downloading and extracting Paddle Lite libs" + } + doLast { + // Prepare cache folder for libs + if (!file("cache").exists()) { + mkdir "cache" + } + // Generate cache name for libs + MessageDigest messageDigest = MessageDigest.getInstance('MD5') + messageDigest.update(paddleLiteLibs.bytes) + String cacheName = new BigInteger(1, messageDigest.digest()).toString(32) + // Download libs + if (!file("cache/${cacheName}.tar.gz").exists()) { + ant.get(src: paddleLiteLibs, dest: file("cache/${cacheName}.tar.gz")) + } + // Unpack libs + if (!file("cache/${cacheName}").exists()) { + copy { + from tarTree("cache/${cacheName}.tar.gz") + into "cache/${cacheName}" + } + } + // Copy PaddlePredictor.jar + if (!file("libs/PaddlePredictor.jar").exists()) { + copy { + from "cache/${cacheName}/java/PaddlePredictor.jar" + into "libs" + } + } + if (!file("src/main/jniLibs/arm64-v8a/libpaddle_lite_jni.so").exists()) { + copy { + from "cache/${cacheName}/java/libs/arm64-v8a/" + into "src/main/jniLibs/arm64-v8a" + } + } + } +} +preBuild.dependsOn downloadAndExtractPaddleLiteLibs + +def paddleLiteModels = [['src' : 'https://paddlespeech.bj.bcebos.com/demos/TTSAndroid/fs2cnn_mbmelgan_cpu_v1.3.0.tar.gz', + 'dest': 'src/main/assets/models'],] +task downloadAndExtractPaddleLiteModels(type: DefaultTask) { + doFirst { + println "Downloading and extracting Paddle Lite models" + } + doLast { + // Prepare cache folder for models + String cachePath = "cache" + if (!file("${cachePath}").exists()) { + mkdir "${cachePath}" + } + paddleLiteModels.eachWithIndex { model, index -> + MessageDigest messageDigest = MessageDigest.getInstance('MD5') + messageDigest.update(model.src.bytes) + String cacheName = new BigInteger(1, messageDigest.digest()).toString(32) + // Download the target model if not exists + boolean copyFiles = !file("${model.dest}").exists() + if (!file("${cachePath}/${cacheName}.tar.gz").exists()) { + ant.get(src: model.src, dest: file("${cachePath}/${cacheName}.tar.gz")) + copyFiles = true // force to copy files from the latest archive files + } + // Copy model file + if (copyFiles) { + copy { + from tarTree("${cachePath}/${cacheName}.tar.gz") + into "${model.dest}" + } + } + } + } +} +preBuild.dependsOn downloadAndExtractPaddleLiteModels diff --git a/demos/TTSAndroid/app/proguard-rules.pro b/demos/TTSAndroid/app/proguard-rules.pro new file mode 100644 index 00000000000..f1b424510da --- /dev/null +++ b/demos/TTSAndroid/app/proguard-rules.pro @@ -0,0 +1,21 @@ +# Add project specific ProGuard rules here. +# You can control the set of applied configuration files using the +# proguardFiles setting in build.gradle. +# +# For more details, see +# http://developer.android.com/guide/developing/tools/proguard.html + +# If your project uses WebView with JS, uncomment the following +# and specify the fully qualified class name to the JavaScript interface +# class: +#-keepclassmembers class fqcn.of.javascript.interface.for.webview { +# public *; +#} + +# Uncomment this to preserve the line number information for +# debugging stack traces. +#-keepattributes SourceFile,LineNumberTable + +# If you keep the line number information, uncomment this to +# hide the original source file name. +#-renamesourcefileattribute SourceFile diff --git a/demos/TTSAndroid/app/src/androidTest/java/com/baidu/paddle/lite/demo/tts/ExampleInstrumentedTest.java b/demos/TTSAndroid/app/src/androidTest/java/com/baidu/paddle/lite/demo/tts/ExampleInstrumentedTest.java new file mode 100644 index 00000000000..d68e8d20959 --- /dev/null +++ b/demos/TTSAndroid/app/src/androidTest/java/com/baidu/paddle/lite/demo/tts/ExampleInstrumentedTest.java @@ -0,0 +1,26 @@ +package com.baidu.paddle.lite.demo.tts; + +import android.content.Context; +import android.support.test.InstrumentationRegistry; +import android.support.test.runner.AndroidJUnit4; + +import org.junit.Test; +import org.junit.runner.RunWith; + +import static org.junit.Assert.*; + +/** + * Instrumented test, which will execute on an Android device. + * + * @see Testing documentation + */ +@RunWith(AndroidJUnit4.class) +public class ExampleInstrumentedTest { + @Test + public void useAppContext() { + // Context of the app under test. + Context appContext = InstrumentationRegistry.getTargetContext(); + + assertEquals("com.baidu.paddle.lite.demo", appContext.getPackageName()); + } +} diff --git a/demos/TTSAndroid/app/src/main/AndroidManifest.xml b/demos/TTSAndroid/app/src/main/AndroidManifest.xml new file mode 100644 index 00000000000..1507c7965b3 --- /dev/null +++ b/demos/TTSAndroid/app/src/main/AndroidManifest.xml @@ -0,0 +1,27 @@ + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/demos/TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/AppCompatPreferenceActivity.java b/demos/TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/AppCompatPreferenceActivity.java new file mode 100644 index 00000000000..5765dbec761 --- /dev/null +++ b/demos/TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/AppCompatPreferenceActivity.java @@ -0,0 +1,122 @@ +/* + * Copyright (C) 2014 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.baidu.paddle.lite.demo.tts; + +import android.content.res.Configuration; +import android.os.Bundle; +import android.preference.PreferenceActivity; +import android.support.annotation.LayoutRes; +import android.support.v7.app.ActionBar; +import android.support.v7.app.AppCompatDelegate; +import android.view.MenuInflater; +import android.view.View; +import android.view.ViewGroup; + +/** + * A {@link android.preference.PreferenceActivity} which implements and proxies the necessary calls + * to be used with AppCompat. + *

+ * This technique can be used with an {@link android.app.Activity} class, not just + * {@link android.preference.PreferenceActivity}. + */ +public abstract class AppCompatPreferenceActivity extends PreferenceActivity { + private AppCompatDelegate mDelegate; + + @Override + protected void onCreate(Bundle savedInstanceState) { + getDelegate().installViewFactory(); + getDelegate().onCreate(savedInstanceState); + super.onCreate(savedInstanceState); + } + + @Override + protected void onPostCreate(Bundle savedInstanceState) { + super.onPostCreate(savedInstanceState); + getDelegate().onPostCreate(savedInstanceState); + } + + public ActionBar getSupportActionBar() { + return getDelegate().getSupportActionBar(); + } + + + @Override + public MenuInflater getMenuInflater() { + return getDelegate().getMenuInflater(); + } + + @Override + public void setContentView(@LayoutRes int layoutResID) { + getDelegate().setContentView(layoutResID); + } + + @Override + public void setContentView(View view) { + getDelegate().setContentView(view); + } + + @Override + public void setContentView(View view, ViewGroup.LayoutParams params) { + getDelegate().setContentView(view, params); + } + + @Override + public void addContentView(View view, ViewGroup.LayoutParams params) { + getDelegate().addContentView(view, params); + } + + @Override + protected void onPostResume() { + super.onPostResume(); + getDelegate().onPostResume(); + } + + @Override + protected void onTitleChanged(CharSequence title, int color) { + super.onTitleChanged(title, color); + getDelegate().setTitle(title); + } + + @Override + public void onConfigurationChanged(Configuration newConfig) { + super.onConfigurationChanged(newConfig); + getDelegate().onConfigurationChanged(newConfig); + } + + @Override + protected void onStop() { + super.onStop(); + getDelegate().onStop(); + } + + @Override + protected void onDestroy() { + super.onDestroy(); + getDelegate().onDestroy(); + } + + public void invalidateOptionsMenu() { + getDelegate().invalidateOptionsMenu(); + } + + private AppCompatDelegate getDelegate() { + if (mDelegate == null) { + mDelegate = AppCompatDelegate.create(this, null); + } + return mDelegate; + } +} diff --git a/demos/TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/MainActivity.java b/demos/TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/MainActivity.java new file mode 100644 index 00000000000..4156c361b2c --- /dev/null +++ b/demos/TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/MainActivity.java @@ -0,0 +1,400 @@ +package com.baidu.paddle.lite.demo.tts; + +import android.Manifest; +import android.app.ProgressDialog; +import android.content.Intent; +import android.content.SharedPreferences; +import android.content.pm.PackageManager; +import android.media.MediaPlayer; +import android.os.Bundle; +import android.os.Environment; +import android.os.Handler; +import android.os.HandlerThread; +import android.os.Message; +import android.preference.PreferenceManager; +import android.support.annotation.NonNull; +import android.support.v4.app.ActivityCompat; +import android.support.v4.content.ContextCompat; +import android.support.v7.app.AppCompatActivity; +import android.text.method.ScrollingMovementMethod; +import android.util.Log; +import android.view.Menu; +import android.view.MenuInflater; +import android.view.MenuItem; +import android.view.View; +import android.widget.AdapterView; +import android.widget.ArrayAdapter; +import android.widget.Button; +import android.widget.Spinner; +import android.widget.TextView; +import android.widget.Toast; + +import java.io.File; +import java.io.IOException; + +public class MainActivity extends AppCompatActivity implements View.OnClickListener, MediaPlayer.OnPreparedListener, MediaPlayer.OnErrorListener, AdapterView.OnItemSelectedListener { + public static final int REQUEST_LOAD_MODEL = 0; + public static final int REQUEST_RUN_MODEL = 1; + public static final int RESPONSE_LOAD_MODEL_SUCCESSED = 0; + public static final int RESPONSE_LOAD_MODEL_FAILED = 1; + public static final int RESPONSE_RUN_MODEL_SUCCESSED = 2; + public static final int RESPONSE_RUN_MODEL_FAILED = 3; + public MediaPlayer mediaPlayer = new MediaPlayer(); + private static final String TAG = Predictor.class.getSimpleName(); + protected ProgressDialog pbLoadModel = null; + protected ProgressDialog pbRunModel = null; + // Receive messages from worker thread + protected Handler receiver = null; + // Send command to worker thread + protected Handler sender = null; + // Worker thread to load&run model + protected HandlerThread worker = null; + // UI components of image classification + protected TextView tvInputSetting; + protected TextView tvInferenceTime; + protected Button btn_play; + protected Button btn_pause; + protected Button btn_stop; + // Model settings of image classification + protected String modelPath = ""; + protected int cpuThreadNum = 1; + protected String cpuPowerMode = ""; + protected Predictor predictor = new Predictor(); + int sampleRate = 24000; + private final String wavName = "tts_output.wav"; + private final String wavFile = Environment.getExternalStorageDirectory() + File.separator + wavName; + private final String AMmodelName = "fastspeech2_csmsc_arm.nb"; + private final String VOCmodelName = "mb_melgan_csmsc_arm.nb"; + private float[] phones = {}; + private final float[][] sentencesToChoose = { + // 009901 昨日，这名“伤者”与医生全部被警方依法刑事拘留。 + {261, 231, 175, 116, 179, 262, 44, 154, 126, 177, 19, 262, 42, 241, 72, 177, 56, 174, 245, 37, 186, 37, 49, 151, 127, 69, 19, 179, 72, 69, 4, 260, 126, 177, 116, 151, 239, 153, 141}, + // 009902 钱伟长想到上海来办学校是经过深思熟虑的。 + {174, 83, 213, 39, 20, 260, 89, 40, 30, 177, 22, 71, 9, 153, 8, 37, 17, 260, 251, 260, 99, 179, 177, 116, 151, 125, 70, 233, 177, 51, 176, 108, 177, 184, 153, 242, 40, 45}, + // 009903 她见我一进门就骂，吃饭时也骂，骂得我抬不起头。 + {182, 2, 151, 85, 232, 73, 151, 123, 154, 52, 151, 143, 154, 5, 179, 39, 113, 69, 17, 177, 114, 105, 154, 5, 179, 154, 5, 40, 45, 232, 182, 8, 37, 186, 174, 74, 182, 168}, + // 009904 李述德在离开之前，只说了一句“柱驼杀父亲了”。 + {153, 74, 177, 186, 40, 42, 261, 10, 153, 73, 152, 7, 262, 113, 174, 83, 179, 262, 115, 177, 230, 153, 45, 73, 151, 242, 180, 262, 186, 182, 231, 177, 2, 69, 186, 174, 124, 153, 45}, + // 009905 这种车票和保险单捆绑出售属于重复性购买。 + {262, 44, 262, 163, 39, 41, 173, 99, 71, 42, 37, 28, 260, 84, 40, 14, 179, 152, 220, 37, 21, 39, 183, 177, 170, 179, 177, 185, 240, 39, 162, 69, 186, 260, 128, 70, 170, 154, 9}, + // 009906 戴佩妮的男友西米露接唱情歌，让她非常开心。 + {40, 10, 173, 49, 155, 72, 40, 45, 155, 15, 142, 260, 72, 154, 74, 153, 186, 179, 151, 103, 39, 22, 174, 126, 70, 41, 179, 175, 22, 182, 2, 69, 46, 39, 20, 152, 7, 260, 120}, + // 009907 观大势、谋大局、出大策始终是该院的办院方针。 + {70, 199, 40, 5, 177, 116, 154, 168, 40, 5, 151, 240, 179, 39, 183, 40, 5, 38, 44, 179, 177, 115, 262, 161, 177, 116, 70, 7, 247, 40, 45, 37, 17, 247, 69, 19, 262, 51}, + // 009908 他们骑着摩托回家，正好为农忙时的父母帮忙。 + {182, 2, 154, 55, 174, 73, 262, 45, 154, 157, 182, 230, 71, 212, 151, 77, 180, 262, 59, 71, 29, 214, 155, 162, 154, 20, 177, 114, 40, 45, 69, 186, 154, 185, 37, 19, 154, 20}, + // 009909 但是因为还没到退休年龄，只能掰着指头捱日子。 + {40, 17, 177, 116, 120, 214, 71, 8, 154, 47, 40, 30, 182, 214, 260, 140, 155, 83, 153, 126, 180, 262, 115, 155, 57, 37, 7, 262, 45, 262, 115, 182, 171, 8, 175, 116, 261, 112}, + // 009910 这几天雨水不断，人们恨不得待在家里不出门。 + {262, 44, 151, 74, 182, 82, 240, 177, 213, 37, 184, 40, 202, 180, 175, 52, 154, 55, 71, 54, 37, 186, 40, 42, 40, 7, 261, 10, 151, 77, 153, 74, 37, 186, 39, 183, 154, 52} + + }; + + @Override + public void onClick(View v) { + switch (v.getId()) { + case R.id.btn_play: + if (!mediaPlayer.isPlaying()) { + mediaPlayer.start(); + } + break; + case R.id.btn_pause: + if (mediaPlayer.isPlaying()) { + mediaPlayer.pause(); + } + break; + case R.id.btn_stop: + if (mediaPlayer.isPlaying()) { + mediaPlayer.reset(); + initMediaPlayer(); + } + break; + default: + break; + } + } + + private void initMediaPlayer() { + try { + File file = new File(wavFile); + // 指定音频文件的路径 + mediaPlayer.setDataSource(file.getPath()); + // 让 MediaPlayer 进入到准备状态 + mediaPlayer.prepare(); + // 该方法使得进入应用时就播放音频 + // mediaPlayer.setOnPreparedListener(this); + // prepare async to not block main thread + mediaPlayer.prepareAsync(); + } catch (Exception e) { + e.printStackTrace(); + } + } + + @Override + public void onPrepared(MediaPlayer player) { + player.start(); + } + + @Override + public boolean onError(MediaPlayer mp, int what, int extra) { + // The MediaPlayer has moved to the Error state, must be reset! + mediaPlayer.reset(); + initMediaPlayer(); + return true; + } + + @Override + protected void onCreate(Bundle savedInstanceState) { + requestAllPermissions(); + super.onCreate(savedInstanceState); + setContentView(R.layout.activity_main); + + // 初始化控件 + Spinner spinner = findViewById(R.id.spinner1); + // 建立数据源 + String[] sentences = getResources().getStringArray(R.array.text); + // 建立 Adapter 并且绑定数据源 + ArrayAdapter adapter = new ArrayAdapter(this, android.R.layout.simple_spinner_dropdown_item, sentences); + // 第一个参数表示在哪个 Activity 上显示，第二个参数是系统下拉框的样式，第三个参数是数组。 + spinner.setAdapter(adapter);//绑定Adapter到控件 + spinner.setOnItemSelectedListener(this); + + btn_play = findViewById(R.id.btn_play); + btn_pause = findViewById(R.id.btn_pause); + btn_stop = findViewById(R.id.btn_stop); + + btn_play.setOnClickListener(this); + btn_pause.setOnClickListener(this); + btn_stop.setOnClickListener(this); + + btn_play.setVisibility(View.INVISIBLE); + btn_pause.setVisibility(View.INVISIBLE); + btn_stop.setVisibility(View.INVISIBLE); + + + // Clear all setting items to avoid app crashing due to the incorrect settings + SharedPreferences sharedPreferences = PreferenceManager.getDefaultSharedPreferences(this); + SharedPreferences.Editor editor = sharedPreferences.edit(); + editor.clear(); + editor.commit(); + + // Prepare the worker thread for mode loading and inference + receiver = new Handler() { + @Override + public void handleMessage(Message msg) { + switch (msg.what) { + case RESPONSE_LOAD_MODEL_SUCCESSED: + pbLoadModel.dismiss(); + onLoadModelSuccessed(); + break; + case RESPONSE_LOAD_MODEL_FAILED: + pbLoadModel.dismiss(); + Toast.makeText(MainActivity.this, "Load model failed!", Toast.LENGTH_SHORT).show(); + onLoadModelFailed(); + break; + case RESPONSE_RUN_MODEL_SUCCESSED: + pbRunModel.dismiss(); + onRunModelSuccessed(); + break; + case RESPONSE_RUN_MODEL_FAILED: + pbRunModel.dismiss(); + Toast.makeText(MainActivity.this, "Run model failed!", Toast.LENGTH_SHORT).show(); + onRunModelFailed(); + break; + default: + break; + } + } + }; + + worker = new HandlerThread("Predictor Worker"); + worker.start(); + sender = new Handler(worker.getLooper()) { + public void handleMessage(Message msg) { + switch (msg.what) { + case REQUEST_LOAD_MODEL: + // Load model and reload test image + if (onLoadModel()) { + receiver.sendEmptyMessage(RESPONSE_LOAD_MODEL_SUCCESSED); + } else { + receiver.sendEmptyMessage(RESPONSE_LOAD_MODEL_FAILED); + } + break; + case REQUEST_RUN_MODEL: + // Run model if model is loaded + if (onRunModel()) { + receiver.sendEmptyMessage(RESPONSE_RUN_MODEL_SUCCESSED); + } else { + receiver.sendEmptyMessage(RESPONSE_RUN_MODEL_FAILED); + } + break; + default: + break; + } + } + }; + + // Setup the UI components + tvInputSetting = findViewById(R.id.tv_input_setting); + tvInferenceTime = findViewById(R.id.tv_inference_time); + tvInputSetting.setMovementMethod(ScrollingMovementMethod.getInstance()); + } + + @Override + protected void onResume() { + super.onResume(); + boolean settingsChanged = false; + SharedPreferences sharedPreferences = PreferenceManager.getDefaultSharedPreferences(this); + String model_path = sharedPreferences.getString(getString(R.string.MODEL_PATH_KEY), + getString(R.string.MODEL_PATH_DEFAULT)); + + settingsChanged |= !model_path.equalsIgnoreCase(modelPath); + + int cpu_thread_num = Integer.parseInt(sharedPreferences.getString(getString(R.string.CPU_THREAD_NUM_KEY), + getString(R.string.CPU_THREAD_NUM_DEFAULT))); + settingsChanged |= cpu_thread_num != cpuThreadNum; + String cpu_power_mode = + sharedPreferences.getString(getString(R.string.CPU_POWER_MODE_KEY), + getString(R.string.CPU_POWER_MODE_DEFAULT)); + settingsChanged |= !cpu_power_mode.equalsIgnoreCase(cpuPowerMode); + + if (settingsChanged) { + modelPath = model_path; + cpuThreadNum = cpu_thread_num; + cpuPowerMode = cpu_power_mode; + // Update UI + tvInputSetting.setText("Model: " + modelPath.substring(modelPath.lastIndexOf("/") + 1) + "\n" + "CPU" + + " Thread Num: " + cpuThreadNum + "\n" + "CPU Power Mode: " + cpuPowerMode + "\n"); + tvInputSetting.scrollTo(0, 0); + // Reload model if configure has been changed + loadModel(); + } + } + + public void loadModel() { + pbLoadModel = ProgressDialog.show(this, "", "Loading model...", false, false); + sender.sendEmptyMessage(REQUEST_LOAD_MODEL); + } + + public void runModel() { + pbRunModel = ProgressDialog.show(this, "", "Running model...", false, false); + sender.sendEmptyMessage(REQUEST_RUN_MODEL); + } + + public boolean onLoadModel() { + return predictor.init(MainActivity.this, modelPath, AMmodelName, VOCmodelName, cpuThreadNum, + cpuPowerMode); + } + + public boolean onRunModel() { + return predictor.isLoaded() && predictor.runModel(phones); + } + + public boolean onLoadModelSuccessed() { + // Load test image from path and run model +// runModel(); + return true; + } + + public void onLoadModelFailed() { + } + + public void onRunModelSuccessed() { + // Obtain results and update UI + btn_play.setVisibility(View.VISIBLE); + btn_pause.setVisibility(View.VISIBLE); + btn_stop.setVisibility(View.VISIBLE); + tvInferenceTime.setText("Inference done！\nInference time: " + predictor.inferenceTime() + " ms" + + "\nRTF: " + predictor.inferenceTime() * sampleRate / (predictor.wav.length * 1000) + "\nAudio saved in " + wavFile); + try { + Utils.rawToWave(wavFile, predictor.wav, sampleRate); + } catch (IOException e) { + e.printStackTrace(); + } + if (ContextCompat.checkSelfPermission(MainActivity.this, + Manifest.permission.WRITE_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED) { + ActivityCompat.requestPermissions(MainActivity.this, new String[]{Manifest.permission.WRITE_EXTERNAL_STORAGE}, 1); + } else { + // 初始化 MediaPlayer + initMediaPlayer(); + } + } + + public void onRunModelFailed() { + } + + + public void onSettingsClicked() { + startActivity(new Intent(MainActivity.this, SettingsActivity.class)); + } + + @Override + public boolean onCreateOptionsMenu(Menu menu) { + MenuInflater inflater = getMenuInflater(); + inflater.inflate(R.menu.menu_action_options, menu); + return true; + } + + @Override + public boolean onOptionsItemSelected(MenuItem item) { + switch (item.getItemId()) { + case android.R.id.home: + finish(); + break; + case R.id.settings: + onSettingsClicked(); + } + return super.onOptionsItemSelected(item); + } + + @Override + public void onRequestPermissionsResult(int requestCode, @NonNull String[] permissions, + @NonNull int[] grantResults) { + + super.onRequestPermissionsResult(requestCode, permissions, grantResults); + if (grantResults[0] != PackageManager.PERMISSION_GRANTED) { + Toast.makeText(this, "Permission Denied", Toast.LENGTH_SHORT).show(); + } + } + + + @Override + protected void onDestroy() { + if (predictor != null) { + predictor.releaseModel(); + } + worker.quit(); + super.onDestroy(); + if (mediaPlayer != null) { + mediaPlayer.stop(); + mediaPlayer.release(); + } + } + + private boolean requestAllPermissions() { + if (ContextCompat.checkSelfPermission(this, Manifest.permission.WRITE_EXTERNAL_STORAGE) + != PackageManager.PERMISSION_GRANTED || ContextCompat.checkSelfPermission(this, + Manifest.permission.CAMERA) + != PackageManager.PERMISSION_GRANTED) { + ActivityCompat.requestPermissions(this, new String[]{Manifest.permission.WRITE_EXTERNAL_STORAGE}, + 0); + return false; + } + return true; + } + + + @Override + public void onItemSelected(AdapterView parent, View view, int position, long id) { + if (position > 0) { + phones = sentencesToChoose[position - 1]; + runModel(); + } + + } + + @Override + public void onNothingSelected(AdapterView parent) { + + } +} diff --git a/demos/TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/Predictor.java b/demos/TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/Predictor.java new file mode 100644 index 00000000000..edaec3d8089 --- /dev/null +++ b/demos/TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/Predictor.java @@ -0,0 +1,149 @@ +package com.baidu.paddle.lite.demo.tts; + +import android.content.Context; +import android.util.Log; + +import com.baidu.paddle.lite.MobileConfig; +import com.baidu.paddle.lite.PaddlePredictor; +import com.baidu.paddle.lite.PowerMode; +import com.baidu.paddle.lite.Tensor; + +import java.io.File; +import java.util.Date; + + +public class Predictor { + private static final String TAG = Predictor.class.getSimpleName(); + public boolean isLoaded = false; + public int cpuThreadNum = 1; + public String cpuPowerMode = "LITE_POWER_HIGH"; + public String modelPath = ""; + protected PaddlePredictor AMPredictor = null; + protected PaddlePredictor VOCPredictor = null; + protected float inferenceTime = 0; + protected float[] wav; + + public boolean init(Context appCtx, String modelPath, String AMmodelName, String VOCmodelName, int cpuThreadNum, String cpuPowerMode) { + // Release model if exists + releaseModel(); + + AMPredictor = loadModel(appCtx, modelPath, AMmodelName, cpuThreadNum, cpuPowerMode); + if (AMPredictor == null) { + return false; + } + VOCPredictor = loadModel(appCtx, modelPath, VOCmodelName, cpuThreadNum, cpuPowerMode); + if (VOCPredictor == null) { + return false; + } + isLoaded = true; + return true; + } + + protected PaddlePredictor loadModel(Context appCtx, String modelPath, String modelName, int cpuThreadNum, String cpuPowerMode) { + // Load model + if (modelPath.isEmpty()) { + return null; + } + String realPath = modelPath; + if (modelPath.charAt(0) != '/') { + // Read model files from custom path if the first character of mode path is '/' + // otherwise copy model to cache from assets + realPath = appCtx.getCacheDir() + "/" + modelPath; + // push model to mobile + Utils.copyDirectoryFromAssets(appCtx, modelPath, realPath); + } + if (realPath.isEmpty()) { + return null; + } + MobileConfig config = new MobileConfig(); + config.setModelFromFile(realPath + File.separator + modelName); + Log.e(TAG, "File:" + realPath + File.separator + modelName); + config.setThreads(cpuThreadNum); + if (cpuPowerMode.equalsIgnoreCase("LITE_POWER_HIGH")) { + config.setPowerMode(PowerMode.LITE_POWER_HIGH); + } else if (cpuPowerMode.equalsIgnoreCase("LITE_POWER_LOW")) { + config.setPowerMode(PowerMode.LITE_POWER_LOW); + } else if (cpuPowerMode.equalsIgnoreCase("LITE_POWER_FULL")) { + config.setPowerMode(PowerMode.LITE_POWER_FULL); + } else if (cpuPowerMode.equalsIgnoreCase("LITE_POWER_NO_BIND")) { + config.setPowerMode(PowerMode.LITE_POWER_NO_BIND); + } else if (cpuPowerMode.equalsIgnoreCase("LITE_POWER_RAND_HIGH")) { + config.setPowerMode(PowerMode.LITE_POWER_RAND_HIGH); + } else if (cpuPowerMode.equalsIgnoreCase("LITE_POWER_RAND_LOW")) { + config.setPowerMode(PowerMode.LITE_POWER_RAND_LOW); + } else { + Log.e(TAG, "Unknown cpu power mode!"); + return null; + } + return PaddlePredictor.createPaddlePredictor(config); + } + + public void releaseModel() { + AMPredictor = null; + VOCPredictor = null; + isLoaded = false; + cpuThreadNum = 1; + cpuPowerMode = "LITE_POWER_HIGH"; + modelPath = ""; + } + + public boolean runModel(float[] phones) { + if (!isLoaded()) { + return false; + } + Date start = new Date(); + Tensor am_output_handle = getAMOutput(phones, AMPredictor); + wav = getVOCOutput(am_output_handle, VOCPredictor); + Date end = new Date(); + inferenceTime = (end.getTime() - start.getTime()); + return true; + } + + public Tensor getAMOutput(float[] phones, PaddlePredictor am_predictor) { + Tensor phones_handle = am_predictor.getInput(0); + long[] dims = {phones.length}; + phones_handle.resize(dims); + phones_handle.setData(phones); + am_predictor.run(); + Tensor am_output_handle = am_predictor.getOutput(0); + // [?, 80] + // long outputShape[] = am_output_handle.shape(); + float[] am_output_data = am_output_handle.getFloatData(); + // [? x 80] + // long[] am_output_data_shape = {am_output_data.length}; + // Log.e(TAG, Arrays.toString(am_output_data)); + // 打印 mel 数组 + // for (int i=0;i preInstalledModelPaths = null; + List preInstalledCPUThreadNums = null; + List preInstalledCPUPowerModes = null; + + + @Override + public void onCreate(Bundle savedInstanceState) { + super.onCreate(savedInstanceState); + addPreferencesFromResource(R.xml.settings); + ActionBar supportActionBar = getSupportActionBar(); + if (supportActionBar != null) { + supportActionBar.setDisplayHomeAsUpEnabled(true); + } + + // Initialized pre-installed models + preInstalledModelPaths = new ArrayList(); + preInstalledCPUThreadNums = new ArrayList(); + preInstalledCPUPowerModes = new ArrayList(); + preInstalledModelPaths.add(getString(R.string.MODEL_PATH_DEFAULT)); + preInstalledCPUThreadNums.add(getString(R.string.CPU_THREAD_NUM_DEFAULT)); + preInstalledCPUPowerModes.add(getString(R.string.CPU_POWER_MODE_DEFAULT)); + + + // Setup UI components + lpChoosePreInstalledModel = (ListPreference) findPreference(getString(R.string.CHOOSE_PRE_INSTALLED_MODEL_KEY)); + String[] preInstalledModelNames = new String[preInstalledModelPaths.size()]; + for (int i = 0; i < preInstalledModelPaths.size(); i++) { + preInstalledModelNames[i] = preInstalledModelPaths.get(i).substring(preInstalledModelPaths.get(i).lastIndexOf("/") + 1); + } + lpChoosePreInstalledModel.setEntries(preInstalledModelNames); + lpChoosePreInstalledModel.setEntryValues(preInstalledModelPaths.toArray(new String[preInstalledModelPaths.size()])); + lpCPUThreadNum = (ListPreference) findPreference(getString(R.string.CPU_THREAD_NUM_KEY)); + lpCPUPowerMode = (ListPreference) findPreference(getString(R.string.CPU_POWER_MODE_KEY)); + cbEnableCustomSettings = (CheckBoxPreference) findPreference(getString(R.string.ENABLE_CUSTOM_SETTINGS_KEY)); + etModelPath = (EditTextPreference) findPreference(getString(R.string.MODEL_PATH_KEY)); + etModelPath.setTitle("Model Path (SDCard: " + Utils.getSDCardDirectory() + ")"); + } + + private void reloadPreferenceAndUpdateUI() { + SharedPreferences sharedPreferences = getPreferenceScreen().getSharedPreferences(); + boolean enableCustomSettings = sharedPreferences.getBoolean(getString(R.string.ENABLE_CUSTOM_SETTINGS_KEY), false); + String modelPath = sharedPreferences.getString(getString(R.string.CHOOSE_PRE_INSTALLED_MODEL_KEY), getString(R.string.MODEL_PATH_DEFAULT)); + int modelIdx = lpChoosePreInstalledModel.findIndexOfValue(modelPath); + if (modelIdx >= 0 && modelIdx < preInstalledModelPaths.size()) { + if (!enableCustomSettings) { + SharedPreferences.Editor editor = sharedPreferences.edit(); + editor.putString(getString(R.string.MODEL_PATH_KEY), preInstalledModelPaths.get(modelIdx)); + editor.putString(getString(R.string.CPU_THREAD_NUM_KEY), preInstalledCPUThreadNums.get(modelIdx)); + editor.putString(getString(R.string.CPU_POWER_MODE_KEY), preInstalledCPUPowerModes.get(modelIdx)); + editor.commit(); + } + lpChoosePreInstalledModel.setSummary(modelPath); + } + cbEnableCustomSettings.setChecked(enableCustomSettings); + etModelPath.setEnabled(enableCustomSettings); + lpCPUThreadNum.setEnabled(enableCustomSettings); + lpCPUPowerMode.setEnabled(enableCustomSettings); + modelPath = sharedPreferences.getString(getString(R.string.MODEL_PATH_KEY), getString(R.string.MODEL_PATH_DEFAULT)); + String cpuThreadNum = sharedPreferences.getString(getString(R.string.CPU_THREAD_NUM_KEY), getString(R.string.CPU_THREAD_NUM_DEFAULT)); + String cpuPowerMode = sharedPreferences.getString(getString(R.string.CPU_POWER_MODE_KEY), getString(R.string.CPU_POWER_MODE_DEFAULT)); + + etModelPath.setSummary(modelPath); + etModelPath.setText(modelPath); + lpCPUThreadNum.setValue(cpuThreadNum); + lpCPUThreadNum.setSummary(cpuThreadNum); + lpCPUPowerMode.setValue(cpuPowerMode); + lpCPUPowerMode.setSummary(cpuPowerMode); + } + + @Override + protected void onResume() { + super.onResume(); + getPreferenceScreen().getSharedPreferences().registerOnSharedPreferenceChangeListener(this); + reloadPreferenceAndUpdateUI(); + } + + @Override + protected void onPause() { + super.onPause(); + getPreferenceScreen().getSharedPreferences().unregisterOnSharedPreferenceChangeListener(this); + } + + @Override + public void onSharedPreferenceChanged(SharedPreferences sharedPreferences, String key) { + if (key.equals(getString(R.string.CHOOSE_PRE_INSTALLED_MODEL_KEY))) { + SharedPreferences.Editor editor = sharedPreferences.edit(); + editor.putBoolean(getString(R.string.ENABLE_CUSTOM_SETTINGS_KEY), false); + editor.commit(); + } + reloadPreferenceAndUpdateUI(); + } +} diff --git a/demos/TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/Utils.java b/demos/TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/Utils.java new file mode 100644 index 00000000000..163d387e24e --- /dev/null +++ b/demos/TTSAndroid/app/src/main/java/com/baidu/paddle/lite/demo/tts/Utils.java @@ -0,0 +1,155 @@ +package com.baidu.paddle.lite.demo.tts; + +import static java.lang.Math.abs; + +import android.content.Context; +import android.os.Environment; + +import java.io.BufferedInputStream; +import java.io.BufferedOutputStream; +import java.io.DataOutputStream; +import java.io.File; +import java.io.FileNotFoundException; +import java.io.FileOutputStream; +import java.io.IOException; +import java.io.InputStream; +import java.io.OutputStream; + +public class Utils { + public static void copyFileFromAssets(Context appCtx, String srcPath, String dstPath) { + if (srcPath.isEmpty() || dstPath.isEmpty()) { + return; + } + InputStream is = null; + OutputStream os = null; + try { + is = new BufferedInputStream(appCtx.getAssets().open(srcPath)); + os = new BufferedOutputStream(new FileOutputStream(new File(dstPath))); + byte[] buffer = new byte[1024]; + int length = 0; + while ((length = is.read(buffer)) != -1) { + os.write(buffer, 0, length); + } + } catch (FileNotFoundException e) { + e.printStackTrace(); + } catch (IOException e) { + e.printStackTrace(); + } finally { + try { + os.close(); + is.close(); + } catch (IOException e) { + e.printStackTrace(); + } + } + } + + public static void copyDirectoryFromAssets(Context appCtx, String srcDir, String dstDir) { + if (srcDir.isEmpty() || dstDir.isEmpty()) { + return; + } + try { + if (!new File(dstDir).exists()) { + new File(dstDir).mkdirs(); + } + for (String fileName : appCtx.getAssets().list(srcDir)) { + String srcSubPath = srcDir + File.separator + fileName; + String dstSubPath = dstDir + File.separator + fileName; + if (new File(srcSubPath).isDirectory()) { + copyDirectoryFromAssets(appCtx, srcSubPath, dstSubPath); + } else { + copyFileFromAssets(appCtx, srcSubPath, dstSubPath); + } + } + } catch (Exception e) { + e.printStackTrace(); + } + } + + + public static String getSDCardDirectory() { + return Environment.getExternalStorageDirectory().getAbsolutePath(); + } + + public static void rawToWave(String file, float[] data, int samplerate) throws IOException { + // creating the empty wav file. + File waveFile = new File(file); + waveFile.createNewFile(); + //following block is converting raw to wav. + DataOutputStream output = null; + try { + output = new DataOutputStream(new FileOutputStream(waveFile)); + // WAVE header + // chunk id + writeString(output, "RIFF"); + // chunk size + writeInt(output, 36 + data.length * 2); + // format + writeString(output, "WAVE"); + // subchunk 1 id + writeString(output, "fmt "); + // subchunk 1 size + writeInt(output, 16); + // audio format (1 = PCM) + writeShort(output, (short) 1); + // number of channels + writeShort(output, (short) 1); + // sample rate + writeInt(output, samplerate); + // byte rate + writeInt(output, samplerate * 2); + // block align + writeShort(output, (short) 2); + // bits per sample + writeShort(output, (short) 16); + // subchunk 2 id + writeString(output, "data"); + // subchunk 2 size + writeInt(output, data.length * 2); + short[] short_data = FloatArray2ShortArray(data); + for (int i = 0; i < short_data.length; i++) { + writeShort(output, short_data[i]); + } + } finally { + if (output != null) { + output.close(); + } + } + } + + private static void writeInt(final DataOutputStream output, final int value) throws IOException { + output.write(value); + output.write(value >> 8); + output.write(value >> 16); + output.write(value >> 24); + } + + private static void writeShort(final DataOutputStream output, final short value) throws IOException { + output.write(value); + output.write(value >> 8); + } + + private static void writeString(final DataOutputStream output, final String value) throws IOException { + for (int i = 0; i < value.length(); i++) { + output.write(value.charAt(i)); + } + } + + public static short[] FloatArray2ShortArray(float[] values) { + float mmax = (float) 0.01; + short[] ret = new short[values.length]; + + for (int i = 0; i < values.length; i++) { + if (abs(values[i]) > mmax) { + mmax = abs(values[i]); + } + } + + for (int i = 0; i < values.length; i++) { + values[i] = values[i] * (32767 / mmax); + ret[i] = (short) (values[i]); + } + return ret; + } + +} diff --git a/demos/TTSAndroid/app/src/main/res/drawable/button_drawable.xml b/demos/TTSAndroid/app/src/main/res/drawable/button_drawable.xml new file mode 100644 index 00000000000..e2f6ba52ca9 --- /dev/null +++ b/demos/TTSAndroid/app/src/main/res/drawable/button_drawable.xml @@ -0,0 +1,20 @@ + + + + + + + + + + + + + + + + + + + + diff --git a/demos/TTSAndroid/app/src/main/res/drawable/logo.jpg b/demos/TTSAndroid/app/src/main/res/drawable/logo.jpg new file mode 100644 index 00000000000..96a03119cbd Binary files /dev/null and b/demos/TTSAndroid/app/src/main/res/drawable/logo.jpg differ diff --git a/demos/TTSAndroid/app/src/main/res/drawable/paddlespeech_logo.png b/demos/TTSAndroid/app/src/main/res/drawable/paddlespeech_logo.png new file mode 100644 index 00000000000..fc3fdd117a1 Binary files /dev/null and b/demos/TTSAndroid/app/src/main/res/drawable/paddlespeech_logo.png differ diff --git a/demos/TTSAndroid/app/src/main/res/layout/activity_main.xml b/demos/TTSAndroid/app/src/main/res/layout/activity_main.xml new file mode 100644 index 00000000000..e14a7e957fc --- /dev/null +++ b/demos/TTSAndroid/app/src/main/res/layout/activity_main.xml @@ -0,0 +1,112 @@ + + + + + + + + + + + + + + + + + +