Fay2.0:
1、控制器pc内网穿透,音频输入输出设备远程直连;
2、提供android 音频输入输出工程示例代码;
3、提供python音频输入输出工程示例代码(远程PC、树莓派等可用);
4、补传1.0语音指令音乐播放模块(暂不支持远程播放);
5、重构及补充若干工具模块:websocket、多线程、缓冲器、音频流录制器等;
6、修复1.x版本的多个bug。
This commit is contained in:
xszyou 2023-01-31 12:40:36 +08:00
parent 09fecfffb7
commit 55fb0896b8
120 changed files with 29211 additions and 165 deletions

View File

@ -2,7 +2,7 @@
<br>
<img src="images/icon.png" alt="Fay">
<h1>FAY</h1>
<h3>数 字 人 控 制 器(这是元宇宙吗?)</h3>
<h3>数 字 人 Fay 控 制 器(这是元宇宙吗?)</h3>
</div>
@ -20,9 +20,18 @@
2、[(34条消息) Fay数字人开源项目在mac 上的安装办法_郭泽斌之心的博客-CSDN博客](https://blog.csdn.net/aa84758481/article/details/127551258)
目前最新版本是2.0。在新版本里我们提出一个全新的架构。在这个架构下每个人都可以把Fay控制器搭建在自己个人电脑上未来或许我们会提供终端让你电脑成为你数字助理的载体。你的所有设备手表、手机、眼镜、笔记本随时可以与你的数字助理通讯数字助理将通过电脑为你处理数字世界里的所有事情。贾维斯Her?
![](images/20230122074644.png)
最近更新:
2023.01
1、控制器pc内网穿透音频输入输出设备远程直连
2、提供android 音频输入输出工程示例代码;
3、提供python音频输入输出工程示例代码远程PC、树莓派等可用
4、补传1.0语音指令音乐播放模块(暂不支持远程播放);
5、重构及补充若干工具模块websocket、多线程、缓冲器、音频流录制器等
6、修复1.x版本的多个bug。
2022.12
@ -161,6 +170,13 @@ python main.py
#### socket远程音频输入
可以接入远程音频输入,远程音频输出
#### 商品栏
填入商品介绍,数字人将自动讲解商品。
@ -175,19 +191,33 @@ python main.py
启动前需填入应用密钥
启动前需填入应用密钥[`system.conf`](https://github.com/TheRamU/Fay/blob/main/system.conf)
| 模块 | 描述 | 链接 |
| 代码模块 | 描述 | 链接 |
| ------------------------- | -------------------------- | ------------------------------------------------------------ |
| ./ai_module/ali_nls.py | 阿里云 实时语音识别 | https://ai.aliyun.com/nls/trans |
| ./ai_module/ms_tts_sdk.py | 微软 文本转语音 基于SDK | https://azure.microsoft.com/zh-cn/services/cognitive-services/text-to-speech/ |
| ./ai_module/xf_aiui.py | 讯飞 人机交互-自然语言处理 | https://aiui.xfyun.cn/solution/webapi |
| ./ai_module/xf_ltp.py | 讯飞 情感分析 | https://www.xfyun.cn/service/emotion-analysis |
| ./utils/ngrok_util.py | ngrok.cc 外网穿透 | http://ngrok.cc |
## 与远程音频输入输出设备连接(非必须,外网需要配置http://ngrok.cc ngrok tcp通道的clientid
控制器与采用 socket(非websocket) 方式与 音频输出设备通讯
内网通讯地址: [`ws://127.0.0.1:10001`](ws://127.0.0.1:10001)
外网通讯地址: 通过http://ngrok.cc获取
![](images/Dingtalk_20230131122109.jpg)
消息格式: 参考 [remote_audio.py](https://github.com/TheRamU/Fay/blob/main/python_connector_demo/remote_audio.py)
## 与数字形象通讯(非必须,控制器需要关闭“面板播放”)
控制器与采用 WebSocket 方式与 UE 通讯
@ -202,6 +232,8 @@ python main.py
## 目录结构
```
@ -238,7 +270,7 @@ python main.py
技术交流群
<img src="images/20230116105510.jpg" alt="微信群">
<img src="images/-1101731868-3469777.png" alt="微信群">
v2.02023年1月25晚上10点腾讯会议见https://meeting.tencent.com/dm/y2Vq5Iut8mN0

3
[Start] PowerShell.bat Normal file
View File

@ -0,0 +1,3 @@
start powershell ^
$host.ui.RawUI.WindowTitle='FeiFei Alpha';^
python ./main.py;^

3
[Start].bat Normal file
View File

@ -0,0 +1,3 @@
echo off
cls
start ./bin/Start.vbs

View File

@ -8,7 +8,7 @@ import _thread as thread
from aliyunsdkcore.client import AcsClient
from aliyunsdkcore.request import CommonRequest
from core import wsa_server
from core import wsa_server, song_player
from scheduler.thread_manager import MyThread
from utils import util
from utils import config_util as cfg
@ -69,6 +69,10 @@ class ALiNls:
}
return header
def __on_msg(self):
if "暂停" in self.finalResults or "不想听了" in self.finalResults or "别唱了" in self.finalResults:
song_player.stop()
# 收到websocket消息的处理
def on_message(self, ws, message):
try:
@ -79,9 +83,11 @@ class ALiNls:
self.done = True
self.finalResults = data['payload']['result']
wsa_server.get_web_instance().add_cmd({"panelMsg": self.finalResults})
self.__on_msg()
elif name == 'TranscriptionResultChanged':
self.finalResults = data['payload']['result']
wsa_server.get_web_instance().add_cmd({"panelMsg": self.finalResults})
self.__on_msg()
except Exception as e:
print(e)
@ -112,6 +118,7 @@ class ALiNls:
try:
if len(self.__frames) > 0:
frame = self.__frames[0]
self.__frames.pop(0)
if type(frame) == dict:
ws.send(json.dumps(frame))

View File

@ -6,6 +6,8 @@ from core import tts_voice
from core.tts_voice import EnumVoice
from utils import util, config_util
from utils import config_util as cfg
import pygame
class Speech:
@ -13,7 +15,7 @@ class Speech:
self.__speech_config = speechsdk.SpeechConfig(subscription=cfg.key_ms_tts_key, region=cfg.key_ms_tts_region)
self.__speech_config.speech_recognition_language = "zh-CN"
self.__speech_config.speech_synthesis_voice_name = "zh-CN-XiaoxiaoNeural"
self.__speech_config.set_speech_synthesis_output_format(speechsdk.SpeechSynthesisOutputFormat.Audio16Khz32KBitRateMonoMp3)
self.__speech_config.set_speech_synthesis_output_format(speechsdk.SpeechSynthesisOutputFormat.Riff16Khz16BitMonoPcm)
self.__synthesizer = speechsdk.SpeechSynthesizer(speech_config=self.__speech_config, audio_config=None)
self.__connection = None
self.__history_data = []
@ -57,7 +59,8 @@ class Speech:
'</speak>'.format(voice_name, style, 1.8, text)
result = self.__synthesizer.speak_ssml(ssml)
audio_data_stream = speechsdk.AudioDataStream(result)
file_url = './samples/sample-' + str(int(time.time() * 1000)) + '.mp3'
file_url = './samples/sample-' + str(int(time.time() * 1000)) + '.wav'
audio_data_stream.save_to_wav_file(file_url)
if result.reason == speechsdk.ResultReason.SynthesizingAudioCompleted:
self.__history_data.append((voice_name, style, text, file_url))
@ -66,3 +69,23 @@ class Speech:
util.log(1, "[x] 语音转换失败!")
util.log(1, "[x] 原因: " + str(result.reason))
return None
if __name__ == '__main__':
cfg.load_config()
sp = Speech()
sp.connect()
pygame.init()
text = """一座城市总有一条标志性道路它见证着这座城市的时代变迁并随着城市历史积淀砥砺前行承载起城市的非凡荣耀。季华路见证了佛山的崛起从而也被誉为“最代表佛山城市发展的一条路”。季华路位于佛山市禅城区是佛山市总体道路规划网中东西走向的城市主干道全长20公里是佛山市公路网络规划"四纵、九横、两环"主骨架中的重要组成部分,西接禅城南庄、高明、三水,东连南海、广州,横跨佛山一环、禅西大道、佛山大道、岭南大道、南海大道五大主干道,贯穿中心城区四个镇街,沿途经过多处文化古迹和重要产业区,是名副其实的“交通动脉”。同时季华路也是佛山的经济“大动脉”,代表着佛山蓬勃发展的现在,也影响着佛山日新月异的未来。
季华六路起于南海大道到文华北截至道路为东西走向全长1.5公里该路段为1996年完成建设并投入使用该道路为一级公路路面使用混凝土材质道路为双向5车道路宽30米途径1个行政单位一条隧道该路段设有格栅518个两边护栏1188米沙井盖158个其中供水26个市政77个移动通讯2个联通通讯3个电信通讯3个交通信号灯1个人行天桥2个电梯4台标志牌18个标线为1.64万米
道路南行是文华中路可通往亚洲艺术公园亚洲艺术公园位于佛山市发展区的中心占地40公顷其中水体面积26.6公顷以岭南水乡为文脉以水上森林为绿脉以龙舟竞渡为水脉通过建筑雕塑植物桥梁等设计要素营造出一个具有亚洲艺术风采的艺术园地曾获选佛山十大最美公园之一
道路北行是文华北路可通往佛山市委市政府佛山市委市政府是广东省佛山市的行政管理机关
道路西行到达文华公园佛山市文华公园位于佛山市禅城区季华路以南电视塔旁文华路以西大福路以东路段建设面积约11万平方米主要将传统文化和现代园林有机结合全园布局以大树木大草坪多彩植被和人工湖为表现主体精致的溪涧小桥亲水平台点缀其间通过棕榈植物错落有序的巧妙搭配令园区既蕴涵亚热带曼妙风情又不失岭南园艺的独特风采通过借景透景造园手法与邻近的电视塔相映成趣它的落成为附近市民的休闲生活添上了色彩绚丽的一笔
季华五路是季华路最先建设的一段道路起于岭南大道到佛山大道截至道路为东西走向全长2.1公里该路段为1993年完成建设并投入使用该道路为一级公路路面使用混凝土材质道路为双向5车道路宽30米途径1个行政单位该路段设有格栅634个两边护栏1310米沙井盖180个其中供水30个市政81个移动通讯5个联通通讯3个交通信号灯2个人行天桥3个电梯12台标志牌26个标线为2.131万米
沿途经过季华园季华园即佛山季华公园位于佛山市城南新区1994年5月建成占地200多亩场内所有设施免费使用景点介绍风格清新意境优雅季华公园是具有亚热带风光的大型开放游览性公园由于场内所有设施免费使用地方广阔每天都吸引着众多的游人前来休闲运动等
道路南行是佛山大道中可通往乐从方向乐从镇地处珠三角腹地广佛经济圈核心带是国家级重大国际产业城市发展合作平台--中德工业服务区中欧城镇化合作示范区的核心
道路北行佛山大道中可通往佛山火车站佛山火车站是广东省的铁路枢纽之一广三铁路经过该站"""
s = sp.to_sample(text, "cheerful")
print(s)
pygame.mixer.music.load(s)
pygame.mixer.music.play()
sp.close()

View File

@ -0,0 +1,15 @@
*.iml
.gradle
/local.properties
/.idea/caches
/.idea/libraries
/.idea/modules.xml
/.idea/workspace.xml
/.idea/navEditor.xml
/.idea/assetWizardSettings.xml
.DS_Store
/build
/captures
.externalNativeBuild
.cxx
local.properties

View File

@ -0,0 +1,3 @@
# Default ignored files
/shelf/
/workspace.xml

View File

@ -0,0 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="CompilerConfiguration">
<bytecodeTargetLevel target="11" />
</component>
</project>

View File

@ -0,0 +1,19 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="GradleMigrationSettings" migrationVersion="1" />
<component name="GradleSettings">
<option name="linkedExternalProjectsSettings">
<GradleProjectSettings>
<option name="testRunner" value="GRADLE" />
<option name="distributionType" value="DEFAULT_WRAPPED" />
<option name="externalProjectPath" value="$PROJECT_DIR$" />
<option name="modules">
<set>
<option value="$PROJECT_DIR$" />
<option value="$PROJECT_DIR$/app" />
</set>
</option>
</GradleProjectSettings>
</option>
</component>
</project>

View File

@ -0,0 +1,16 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="DesignSurface">
<option name="filePathToZoomLevelMap">
<map>
<entry key="..\:/android/projects/fayConnectorDemo/app/src/main/res/layout/activity_main.xml" value="0.358695652173913" />
</map>
</option>
</component>
<component name="ProjectRootManager" version="2" languageLevel="JDK_11" default="true" project-jdk-name="Android Studio default JDK" project-jdk-type="JavaSDK">
<output url="file://$PROJECT_DIR$/build/classes" />
</component>
<component name="ProjectType">
<option name="id" value="Android" />
</component>
</project>

View File

@ -0,0 +1 @@
/build

View File

@ -0,0 +1,38 @@
plugins {
id 'com.android.application'
}
android {
compileSdk 32
defaultConfig {
applicationId "com.yaheen.fayconnectordemo"
minSdk 29
targetSdk 32
versionCode 1
versionName "1.0"
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
}
buildTypes {
release {
minifyEnabled false
proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
}
}
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
}
dependencies {
implementation 'androidx.appcompat:appcompat:1.3.0'
implementation 'com.google.android.material:material:1.4.0'
implementation 'androidx.constraintlayout:constraintlayout:2.0.4'
testImplementation 'junit:junit:4.13.2'
androidTestImplementation 'androidx.test.ext:junit:1.1.3'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0'
}

View File

@ -0,0 +1,21 @@
# Add project specific ProGuard rules here.
# You can control the set of applied configuration files using the
# proguardFiles setting in build.gradle.
#
# For more details, see
# http://developer.android.com/guide/developing/tools/proguard.html
# If your project uses WebView with JS, uncomment the following
# and specify the fully qualified class name to the JavaScript interface
# class:
#-keepclassmembers class fqcn.of.javascript.interface.for.webview {
# public *;
#}
# Uncomment this to preserve the line number information for
# debugging stack traces.
#-keepattributes SourceFile,LineNumberTable
# If you keep the line number information, uncomment this to
# hide the original source file name.
#-renamesourcefileattribute SourceFile

View File

@ -0,0 +1,26 @@
package com.yaheen.fayconnectordemo;
import android.content.Context;
import androidx.test.platform.app.InstrumentationRegistry;
import androidx.test.ext.junit.runners.AndroidJUnit4;
import org.junit.Test;
import org.junit.runner.RunWith;
import static org.junit.Assert.*;
/**
* Instrumented test, which will execute on an Android device.
*
* @see <a href="http://d.android.com/tools/testing">Testing documentation</a>
*/
@RunWith(AndroidJUnit4.class)
public class ExampleInstrumentedTest {
@Test
public void useAppContext() {
// Context of the app under test.
Context appContext = InstrumentationRegistry.getInstrumentation().getTargetContext();
assertEquals("com.yaheen.fayconnectordemo", appContext.getPackageName());
}
}

View File

@ -0,0 +1,43 @@
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools"
package="com.yaheen.fayconnectordemo">
<uses-permission android:name="android.permission.INTERNET"/><!--网络访问-->
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>
<uses-permission android:name="android.permission.CHANGE_WIFI_STATE"/>
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE"/>
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE"/>
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.BLUETOOTH" />
<uses-permission android:name="android.permission.BLUETOOTH_ADMIN" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<application
android:allowBackup="true"
android:dataExtractionRules="@xml/data_extraction_rules"
android:fullBackupContent="@xml/backup_rules"
android:icon="@drawable/icon"
android:label="@string/app_name"
android:roundIcon="@mipmap/ic_launcher_round"
android:supportsRtl="true"
android:theme="@style/Theme.FayConnectorDemo"
tools:targetApi="31">
<activity
android:name=".MainActivity"
android:exported="true">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
<service android:name=".FayConnectorService" />
</application>
</manifest>

View File

@ -0,0 +1,337 @@
package com.yaheen.fayconnectordemo;
import android.Manifest;
import android.app.Notification;
import android.app.NotificationChannel;
import android.app.NotificationManager;
import android.app.PendingIntent;
import android.app.Service;
import android.content.BroadcastReceiver;
import android.content.Context;
import android.content.Intent;
import android.content.IntentFilter;
import android.content.pm.PackageManager;
import android.graphics.BitmapFactory;
import android.media.AudioFormat;
import android.media.AudioManager;
import android.media.AudioRecord;
import android.media.MediaPlayer;
import android.media.MediaRecorder;
import android.os.Build;
import android.os.IBinder;
import android.util.Log;
import androidx.annotation.Nullable;
import androidx.core.app.ActivityCompat;
import androidx.core.app.NotificationCompat;
import androidx.core.app.NotificationManagerCompat;
import androidx.core.content.ContextCompat;
import com.google.android.material.snackbar.Snackbar;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.Socket;
import java.util.Arrays;
import java.util.Date;
public class FayConnectorService extends Service {
private AudioRecord record;
private int recordBufsize = 0;
private Socket socket = null;
private InputStream in = null;
private OutputStream out = null;
public static boolean running = false;
private File cacheDir = null;
private String channelId = null;
private PendingIntent pendingIntent = null;
private NotificationManagerCompat notificationManager = null;
private long totalrece = 0;
private long totalsend = 0;
private AudioManager mAudioManager = null;
private boolean isPlay = false;
//创建通知
private String createNotificationChannel(String channelID, String channelNAME, int level) {
if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.O) {
NotificationManager manager = (NotificationManager) getSystemService(NOTIFICATION_SERVICE);
NotificationChannel channel = new NotificationChannel(channelID, channelNAME, level);
manager.createNotificationChannel(channel);
return channelID;
} else {
return null;
}
}
@Nullable
@Override
public IBinder onBind(Intent intent) {
return null;
}
@Override
public int onStartCommand(Intent intent, int flags, int startId) {
super.onStartCommand(intent, START_FLAG_REDELIVERY, startId);
return Service.START_STICKY;
}
@Override
public void onCreate() {
super.onCreate();
Log.d("fay", "服务启动");
//开启蓝牙传输
mAudioManager = (AudioManager) getSystemService(Context.AUDIO_SERVICE);
mAudioManager.startBluetoothSco();
IntentFilter intentFilter = new IntentFilter();
intentFilter.addAction(AudioManager.ACTION_SCO_AUDIO_STATE_UPDATED);
BroadcastReceiver receiver = new BroadcastReceiver() {
@Override
public void onReceive(Context context, Intent intent) {
int state = intent.getIntExtra(AudioManager.EXTRA_SCO_AUDIO_STATE, -1);
if (AudioManager.SCO_AUDIO_STATE_CONNECTED == state) {
Log.d("fay", "蓝牙sco连接成功");
mAudioManager.setBluetoothScoOn(true);
mAudioManager.setMode(mAudioManager.MODE_IN_CALL);
}
}
};
this.registerReceiver(receiver, intentFilter);
running = true;
this.cacheDir = getApplicationContext().getFilesDir();//getCacheDir();
Thread sendThread = new Thread(new Runnable() {
@Override
public void run() {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
if (ContextCompat.checkSelfPermission(FayConnectorService.this, Manifest.permission.RECORD_AUDIO) == PackageManager.PERMISSION_GRANTED) {
if (record == null) {
recordBufsize = AudioRecord
.getMinBufferSize(16000,
AudioFormat.CHANNEL_IN_MONO,
AudioFormat.ENCODING_PCM_16BIT);
record = new AudioRecord(MediaRecorder.AudioSource.MIC,
16000,
AudioFormat.CHANNEL_IN_MONO,
AudioFormat.ENCODING_PCM_16BIT,
recordBufsize);
}
try {
socket = new Socket("5gzvip.91tunnel.com", 10001);
in = socket.getInputStream();
out = socket.getOutputStream();
Log.d("fay", "fay控制器连接成功");
} catch (IOException e) {
Log.d("fay", "socket连接失败");
return;
}
byte[] data = new byte[1024];
record.startRecording();
Log.d("fay", "麦克风启动成功");
try {
Log.d("fay", "开始传输音频");
while (running) {
if (isPlay){
continue;
}
int size = record.read(data, 0, 1024);
if (size > 0) {
out.write(data);
totalsend += data.length / 1024;
}else{//录音异常等待60秒重新录取
try {
Thread.sleep(60000);
record.stop();
record.startRecording();
}catch (Exception e){
}
}
}
} catch (Exception e) { //通过异常关退出循环
Log.d("fay", "服务端关闭:" + e.toString());
} finally {
running = false;
record.stop();
record = null;
((AudioManager) getSystemService(Context.AUDIO_SERVICE)).stopBluetoothSco();
try {
socket.close();
} catch (Exception e) {
}
Log.d("fay", "结束");
}
}
}
}
});
Thread receThread = new Thread(new Runnable() {
@Override
public void run() {
try {
while (running) {
while (socket != null && !socket.isClosed()) {
byte[] data = new byte[9];
byte[] wavhead = new byte[]{0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08};//文件传输开始标记
in.read(data);
if (Arrays.equals(wavhead, data)) {
Log.d("fay", "开始接收音频文件");
String filedata = "";
data = new byte[1024];
while (data != null && data.length > 0) {
in.read(data);
filedata += MainActivity.bytesToHexString(data);
int index = filedata.indexOf("080706050403020100");
if (filedata.length() > 9 && index > 0) {//wav文件结束标记
filedata = filedata.substring(0, index).replaceAll("F0F1F2F3F4F5F6F7F8", "");
File wavFile = new File(cacheDir, String.format("sample-%s.wav", new Date().getTime() + ""));
wavFile.createNewFile();
FileOutputStream fos = new FileOutputStream(wavFile);
fos.write(MainActivity.decodeHexBytes(filedata.toCharArray()));
fos.close();
totalrece += filedata.length() / 2 / 1024;
Log.d("fay", "wav文件接收完成:" + wavFile.getAbsolutePath() + "," + filedata.length() / 2);
try {
MediaPlayer player = new MediaPlayer();
player.setDataSource(wavFile.getAbsolutePath());
player.setOnPreparedListener(new MediaPlayer.OnPreparedListener() {
@Override
public void onPrepared(MediaPlayer mp) {
Log.d("fay", "开始播放");
if (mAudioManager.isBluetoothScoOn()){
mAudioManager.stopBluetoothSco();
mAudioManager.setBluetoothScoOn(false);
mAudioManager.setMode(mAudioManager.MODE_NORMAL);
}
try {
Thread.sleep(500);
}catch (Exception e){
}
isPlay = true;
mp.start();
}
});
player.setOnCompletionListener(new MediaPlayer.OnCompletionListener() {
@Override
public void onCompletion(MediaPlayer mp) {
Log.d("fay", "播放完成");
isPlay = false;
mp.release();
mAudioManager.startBluetoothSco();
mAudioManager.setMode(mAudioManager.MODE_IN_CALL);
}
});
player.setVolume(1,1);
player.setLooping(false);
player.prepareAsync();
} catch (IOException e) {
Log.e("fay", e.toString());
}
break;
}
}
try {
Thread.sleep(1000);
} catch (Exception e) {
}
}
}
try {
Thread.sleep(1000);
} catch (Exception e) {
}
}
} catch (Exception e) {//通过异常判断socket已经关闭退出循环
} finally {
}
}
});
sendThread.start();
receThread.start();
//通知栏
new Thread(new Runnable() {
@Override
public void run() {
try{
while (running) {
Thread.sleep(3000);
if (totalsend + totalrece > 2048){
inotify("fay connector demo", "已经连接fay控制器累计接收/发送:" + String.format("%.2f", (double)totalrece / 1024) + "/" + String.format("%.2f", (double)totalsend / 1024) + "MB");
} else {
inotify("fay connector demo", "已经连接fay控制器累计接收/发送:" + totalrece + "/" + totalsend + "KB");
}
}
inotify("fay connector demo", "已经断开fay控制器");
}catch (Exception e){
Log.e("fay", e.toString());
}finally {
FayConnectorService.this.stopForeground(true);
}
}
}).start();
}
private void inotify(String title, String content){
Intent intent = new Intent(this, MainActivity.class);
intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK | Intent.FLAG_ACTIVITY_CLEAR_TASK);
if (pendingIntent == null){
pendingIntent = PendingIntent.getActivity(this, 0, intent, PendingIntent.FLAG_IMMUTABLE);
}
if (channelId == null){
channelId = createNotificationChannel("my_channel_ID", "my_channel_NAME", NotificationManager.IMPORTANCE_HIGH);
}
if (notificationManager == null){
notificationManager = NotificationManagerCompat.from(this);
}
NotificationCompat.Builder notification2 = new NotificationCompat.Builder(FayConnectorService.this, channelId)
.setContentTitle(title)
.setContentText(content)
.setContentIntent(pendingIntent)
.setSmallIcon(R.drawable.icon)
.setPriority(NotificationCompat.PRIORITY_HIGH)
.setAutoCancel(true);
//notificationManager.notify(100, notification2.build());
startForeground(100, notification2.build());
}
@Override
public void onDestroy() {
Log.d("fay", "服务关闭");
super.onDestroy();
mAudioManager.stopBluetoothSco();
running = false;
stopForeground(true);
}
}

View File

@ -0,0 +1,137 @@
package com.yaheen.fayconnectordemo;
import androidx.appcompat.app.AppCompatActivity;
import androidx.core.app.ActivityCompat;
import androidx.core.content.ContextCompat;
import android.Manifest;
import android.app.ActivityManager;
import android.app.PendingIntent;
import android.content.BroadcastReceiver;
import android.content.ComponentName;
import android.content.Context;
import android.content.Intent;
import android.content.IntentFilter;
import android.content.pm.PackageManager;
import android.media.AudioFormat;
import android.media.AudioManager;
import android.media.AudioRecord;
import android.media.MediaPlayer;
import android.media.MediaRecorder;
import android.os.Build;
import android.os.Bundle;
import android.util.Log;
import android.view.View;
import android.widget.TextView;
import com.google.android.material.snackbar.Snackbar;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.Socket;
import java.net.SocketException;
import java.util.Arrays;
import java.util.Date;
import java.util.List;
public class MainActivity extends AppCompatActivity {
private TextView tv = null;
private boolean running = false;
private Intent serviceIntent = null;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
tv = this.findViewById(R.id.tv);
serviceIntent = new Intent(this, FayConnectorService.class);
//按钮点击
tv.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View view) {
Log.d("fay","onclick");
running = FayConnectorService.running;//isServiceRunning();//同步service的运行状态,不好使
if (!running){//运行
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {//开启
if (ContextCompat.checkSelfPermission(MainActivity.this, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
if (ActivityCompat.shouldShowRequestPermissionRationale(MainActivity.this, Manifest.permission.RECORD_AUDIO)) {
Log.d("fay", "用户彻底拒绝了权限");
return;
} else {
// 用户未彻底拒绝授予权限
ActivityCompat.requestPermissions(MainActivity.this, new String[]{Manifest.permission.RECORD_AUDIO}, 1);
}
}
if (ContextCompat.checkSelfPermission(MainActivity.this, Manifest.permission.RECORD_AUDIO) == PackageManager.PERMISSION_GRANTED) {
Log.d("fay","权限ok");
Snackbar.make(view, "正在连接fay控制器", Snackbar.LENGTH_SHORT)
.setAction("Action", null).show();
startForegroundService(serviceIntent);
running = true;
}
}
} else{//关闭
stopService(serviceIntent);
Snackbar.make(view, "已经断开fay控制器", Snackbar.LENGTH_SHORT)
.setAction("Action", null).show();
running = false;
}
}
});
}
public static String bytesToHexString(byte[] data){
String result="";
for (int i = 0; i < data.length; i++) {
result+=Integer.toHexString((data[i] & 0xFF) | 0x100).toUpperCase().substring(1, 3);
}
return result;
}
public static byte[] decodeHexBytes(char[] data) {
int len = data.length;
if ((len & 0x01) != 0) {
throw new RuntimeException("未知的字符");
}
byte[] out = new byte[len >> 1];
for (int i = 0, j = 0; j < len; i++) {
int f = toDigit(data[j], j) << 4;
j++;
f = f | toDigit(data[j], j);
j++;
out[i] = (byte) (f & 0xFF);
}
return out;
}
protected static int toDigit(char ch, int index) {
int digit = Character.digit(ch, 16);
if (digit == -1) {
throw new RuntimeException("非法16进制字符 " + ch
+ " 在索引 " + index);
}
return digit;
}
private boolean isServiceRunning() {
ActivityManager activityManager = (ActivityManager) this.getApplicationContext()
.getSystemService(Context.ACTIVITY_SERVICE);
ComponentName serviceName = new ComponentName("com.yaheen.fayconnectordemo", ".FayConnectorService");
PendingIntent intent = activityManager.getRunningServiceControlPanel(serviceName);
if (intent == null){
return false;
}
return true;
}
}

View File

@ -0,0 +1,30 @@
<vector xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:aapt="http://schemas.android.com/aapt"
android:width="108dp"
android:height="108dp"
android:viewportWidth="108"
android:viewportHeight="108">
<path android:pathData="M31,63.928c0,0 6.4,-11 12.1,-13.1c7.2,-2.6 26,-1.4 26,-1.4l38.1,38.1L107,108.928l-32,-1L31,63.928z">
<aapt:attr name="android:fillColor">
<gradient
android:endX="85.84757"
android:endY="92.4963"
android:startX="42.9492"
android:startY="49.59793"
android:type="linear">
<item
android:color="#44000000"
android:offset="0.0" />
<item
android:color="#00000000"
android:offset="1.0" />
</gradient>
</aapt:attr>
</path>
<path
android:fillColor="#FFFFFF"
android:fillType="nonZero"
android:pathData="M65.3,45.828l3.8,-6.6c0.2,-0.4 0.1,-0.9 -0.3,-1.1c-0.4,-0.2 -0.9,-0.1 -1.1,0.3l-3.9,6.7c-6.3,-2.8 -13.4,-2.8 -19.7,0l-3.9,-6.7c-0.2,-0.4 -0.7,-0.5 -1.1,-0.3C38.8,38.328 38.7,38.828 38.9,39.228l3.8,6.6C36.2,49.428 31.7,56.028 31,63.928h46C76.3,56.028 71.8,49.428 65.3,45.828zM43.4,57.328c-0.8,0 -1.5,-0.5 -1.8,-1.2c-0.3,-0.7 -0.1,-1.5 0.4,-2.1c0.5,-0.5 1.4,-0.7 2.1,-0.4c0.7,0.3 1.2,1 1.2,1.8C45.3,56.528 44.5,57.328 43.4,57.328L43.4,57.328zM64.6,57.328c-0.8,0 -1.5,-0.5 -1.8,-1.2s-0.1,-1.5 0.4,-2.1c0.5,-0.5 1.4,-0.7 2.1,-0.4c0.7,0.3 1.2,1 1.2,1.8C66.5,56.528 65.6,57.328 64.6,57.328L64.6,57.328z"
android:strokeWidth="1"
android:strokeColor="#00000000" />
</vector>

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

View File

@ -0,0 +1,170 @@
<?xml version="1.0" encoding="utf-8"?>
<vector xmlns:android="http://schemas.android.com/apk/res/android"
android:width="108dp"
android:height="108dp"
android:viewportWidth="108"
android:viewportHeight="108">
<path
android:fillColor="#3DDC84"
android:pathData="M0,0h108v108h-108z" />
<path
android:fillColor="#00000000"
android:pathData="M9,0L9,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M19,0L19,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M29,0L29,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M39,0L39,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M49,0L49,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M59,0L59,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M69,0L69,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M79,0L79,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M89,0L89,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M99,0L99,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,9L108,9"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,19L108,19"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,29L108,29"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,39L108,39"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,49L108,49"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,59L108,59"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,69L108,69"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,79L108,79"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,89L108,89"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,99L108,99"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M19,29L89,29"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M19,39L89,39"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M19,49L89,49"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M19,59L89,59"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M19,69L89,69"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M19,79L89,79"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M29,19L29,89"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M39,19L39,89"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M49,19L49,89"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M59,19L59,89"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M69,19L69,89"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M79,19L79,89"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
</vector>

View File

@ -0,0 +1,19 @@
<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity">
<TextView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="连接/断开fay控制器"
android:id="@+id/tv"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent" />
</androidx.constraintlayout.widget.ConstraintLayout>

View File

@ -0,0 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<adaptive-icon xmlns:android="http://schemas.android.com/apk/res/android">
<background android:drawable="@drawable/ic_launcher_background" />
<foreground android:drawable="@drawable/ic_launcher_foreground" />
</adaptive-icon>

View File

@ -0,0 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<adaptive-icon xmlns:android="http://schemas.android.com/apk/res/android">
<background android:drawable="@drawable/ic_launcher_background" />
<foreground android:drawable="@drawable/ic_launcher_foreground" />
</adaptive-icon>

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 KiB

View File

@ -0,0 +1,16 @@
<resources xmlns:tools="http://schemas.android.com/tools">
<!-- Base application theme. -->
<style name="Theme.FayConnectorDemo" parent="Theme.MaterialComponents.DayNight.DarkActionBar">
<!-- Primary brand color. -->
<item name="colorPrimary">@color/purple_200</item>
<item name="colorPrimaryVariant">@color/purple_700</item>
<item name="colorOnPrimary">@color/black</item>
<!-- Secondary brand color. -->
<item name="colorSecondary">@color/teal_200</item>
<item name="colorSecondaryVariant">@color/teal_200</item>
<item name="colorOnSecondary">@color/black</item>
<!-- Status bar color. -->
<item name="android:statusBarColor" tools:targetApi="l">?attr/colorPrimaryVariant</item>
<!-- Customize your theme here. -->
</style>
</resources>

View File

@ -0,0 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<color name="purple_200">#FFBB86FC</color>
<color name="purple_500">#FF6200EE</color>
<color name="purple_700">#FF3700B3</color>
<color name="teal_200">#FF03DAC5</color>
<color name="teal_700">#FF018786</color>
<color name="black">#FF000000</color>
<color name="white">#FFFFFFFF</color>
</resources>

View File

@ -0,0 +1,3 @@
<resources>
<string name="app_name">fayConnectorDemo</string>
</resources>

View File

@ -0,0 +1,16 @@
<resources xmlns:tools="http://schemas.android.com/tools">
<!-- Base application theme. -->
<style name="Theme.FayConnectorDemo" parent="Theme.MaterialComponents.DayNight.DarkActionBar">
<!-- Primary brand color. -->
<item name="colorPrimary">@color/purple_500</item>
<item name="colorPrimaryVariant">@color/purple_700</item>
<item name="colorOnPrimary">@color/white</item>
<!-- Secondary brand color. -->
<item name="colorSecondary">@color/teal_200</item>
<item name="colorSecondaryVariant">@color/teal_700</item>
<item name="colorOnSecondary">@color/black</item>
<!-- Status bar color. -->
<item name="android:statusBarColor" tools:targetApi="l">?attr/colorPrimaryVariant</item>
<!-- Customize your theme here. -->
</style>
</resources>

View File

@ -0,0 +1,13 @@
<?xml version="1.0" encoding="utf-8"?><!--
Sample backup rules file; uncomment and customize as necessary.
See https://developer.android.com/guide/topics/data/autobackup
for details.
Note: This file is ignored for devices older that API 31
See https://developer.android.com/about/versions/12/backup-restore
-->
<full-backup-content>
<!--
<include domain="sharedpref" path="."/>
<exclude domain="sharedpref" path="device.xml"/>
-->
</full-backup-content>

View File

@ -0,0 +1,19 @@
<?xml version="1.0" encoding="utf-8"?><!--
Sample data extraction rules file; uncomment and customize as necessary.
See https://developer.android.com/about/versions/12/backup-restore#xml-changes
for details.
-->
<data-extraction-rules>
<cloud-backup>
<!-- TODO: Use <include> and <exclude> to control what is backed up.
<include .../>
<exclude .../>
-->
</cloud-backup>
<!--
<device-transfer>
<include .../>
<exclude .../>
</device-transfer>
-->
</data-extraction-rules>

View File

@ -0,0 +1,17 @@
package com.yaheen.fayconnectordemo;
import org.junit.Test;
import static org.junit.Assert.*;
/**
* Example local unit test, which will execute on the development machine (host).
*
* @see <a href="http://d.android.com/tools/testing">Testing documentation</a>
*/
public class ExampleUnitTest {
@Test
public void addition_isCorrect() {
assertEquals(4, 2 + 2);
}
}

View File

@ -0,0 +1,9 @@
// Top-level build file where you can add configuration options common to all sub-projects/modules.
plugins {
id 'com.android.application' version '7.2.1' apply false
id 'com.android.library' version '7.2.1' apply false
}
task clean(type: Delete) {
delete rootProject.buildDir
}

View File

@ -0,0 +1,21 @@
# Project-wide Gradle settings.
# IDE (e.g. Android Studio) users:
# Gradle settings configured through the IDE *will override*
# any settings specified in this file.
# For more details on how to configure your build environment visit
# http://www.gradle.org/docs/current/userguide/build_environment.html
# Specifies the JVM arguments used for the daemon process.
# The setting is particularly useful for tweaking memory settings.
org.gradle.jvmargs=-Xmx2048m -Dfile.encoding=UTF-8
# When configured, Gradle will run in incubating parallel mode.
# This option should only be used with decoupled projects. More details, visit
# http://www.gradle.org/docs/current/userguide/multi_project_builds.html#sec:decoupled_projects
# org.gradle.parallel=true
# AndroidX package structure to make it clearer which packages are bundled with the
# Android operating system, and which are packaged with your app"s APK
# https://developer.android.com/topic/libraries/support-library/androidx-rn
android.useAndroidX=true
# Enables namespacing of each library's R class so that its R class includes only the
# resources declared in the library itself and none from the library's dependencies,
# thereby reducing the size of the R class for that library
android.nonTransitiveRClass=true

View File

@ -0,0 +1,6 @@
#Fri Jan 20 09:27:45 CST 2023
distributionBase=GRADLE_USER_HOME
distributionUrl=https\://services.gradle.org/distributions/gradle-7.3.3-bin.zip
distributionPath=wrapper/dists
zipStorePath=wrapper/dists
zipStoreBase=GRADLE_USER_HOME

View File

@ -0,0 +1,185 @@
#!/usr/bin/env sh
#
# Copyright 2015 the original author or authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
##############################################################################
##
## Gradle start up script for UN*X
##
##############################################################################
# Attempt to set APP_HOME
# Resolve links: $0 may be a link
PRG="$0"
# Need this for relative symlinks.
while [ -h "$PRG" ] ; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> \(.*\)$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG=`dirname "$PRG"`"/$link"
fi
done
SAVED="`pwd`"
cd "`dirname \"$PRG\"`/" >/dev/null
APP_HOME="`pwd -P`"
cd "$SAVED" >/dev/null
APP_NAME="Gradle"
APP_BASE_NAME=`basename "$0"`
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
# Use the maximum available, or set MAX_FD != -1 to use that value.
MAX_FD="maximum"
warn () {
echo "$*"
}
die () {
echo
echo "$*"
echo
exit 1
}
# OS specific support (must be 'true' or 'false').
cygwin=false
msys=false
darwin=false
nonstop=false
case "`uname`" in
CYGWIN* )
cygwin=true
;;
Darwin* )
darwin=true
;;
MINGW* )
msys=true
;;
NONSTOP* )
nonstop=true
;;
esac
CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
# Determine the Java command to use to start the JVM.
if [ -n "$JAVA_HOME" ] ; then
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
# IBM's JDK on AIX uses strange locations for the executables
JAVACMD="$JAVA_HOME/jre/sh/java"
else
JAVACMD="$JAVA_HOME/bin/java"
fi
if [ ! -x "$JAVACMD" ] ; then
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
else
JAVACMD="java"
which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
# Increase the maximum file descriptors if we can.
if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then
MAX_FD_LIMIT=`ulimit -H -n`
if [ $? -eq 0 ] ; then
if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then
MAX_FD="$MAX_FD_LIMIT"
fi
ulimit -n $MAX_FD
if [ $? -ne 0 ] ; then
warn "Could not set maximum file descriptor limit: $MAX_FD"
fi
else
warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT"
fi
fi
# For Darwin, add options to specify how the application appears in the dock
if $darwin; then
GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""
fi
# For Cygwin or MSYS, switch paths to Windows format before running java
if [ "$cygwin" = "true" -o "$msys" = "true" ] ; then
APP_HOME=`cygpath --path --mixed "$APP_HOME"`
CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`
JAVACMD=`cygpath --unix "$JAVACMD"`
# We build the pattern for arguments to be converted via cygpath
ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null`
SEP=""
for dir in $ROOTDIRSRAW ; do
ROOTDIRS="$ROOTDIRS$SEP$dir"
SEP="|"
done
OURCYGPATTERN="(^($ROOTDIRS))"
# Add a user-defined pattern to the cygpath arguments
if [ "$GRADLE_CYGPATTERN" != "" ] ; then
OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)"
fi
# Now convert the arguments - kludge to limit ourselves to /bin/sh
i=0
for arg in "$@" ; do
CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -`
CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option
if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition
eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"`
else
eval `echo args$i`="\"$arg\""
fi
i=`expr $i + 1`
done
case $i in
0) set -- ;;
1) set -- "$args0" ;;
2) set -- "$args0" "$args1" ;;
3) set -- "$args0" "$args1" "$args2" ;;
4) set -- "$args0" "$args1" "$args2" "$args3" ;;
5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;
6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;
7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;
8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;
9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;
esac
fi
# Escape application args
save () {
for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done
echo " "
}
APP_ARGS=`save "$@"`
# Collect all arguments for the java command, following the shell quoting and substitution rules
eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS"
exec "$JAVACMD" "$@"

View File

@ -0,0 +1,89 @@
@rem
@rem Copyright 2015 the original author or authors.
@rem
@rem Licensed under the Apache License, Version 2.0 (the "License");
@rem you may not use this file except in compliance with the License.
@rem You may obtain a copy of the License at
@rem
@rem https://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing, software
@rem distributed under the License is distributed on an "AS IS" BASIS,
@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@rem See the License for the specific language governing permissions and
@rem limitations under the License.
@rem
@if "%DEBUG%" == "" @echo off
@rem ##########################################################################
@rem
@rem Gradle startup script for Windows
@rem
@rem ##########################################################################
@rem Set local scope for the variables with windows NT shell
if "%OS%"=="Windows_NT" setlocal
set DIRNAME=%~dp0
if "%DIRNAME%" == "" set DIRNAME=.
set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME%
@rem Resolve any "." and ".." in APP_HOME to make it shorter.
for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi
@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"
@rem Find java.exe
if defined JAVA_HOME goto findJavaFromJavaHome
set JAVA_EXE=java.exe
%JAVA_EXE% -version >NUL 2>&1
if "%ERRORLEVEL%" == "0" goto execute
echo.
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
echo.
echo Please set the JAVA_HOME variable in your environment to match the
echo location of your Java installation.
goto fail
:findJavaFromJavaHome
set JAVA_HOME=%JAVA_HOME:"=%
set JAVA_EXE=%JAVA_HOME%/bin/java.exe
if exist "%JAVA_EXE%" goto execute
echo.
echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME%
echo.
echo Please set the JAVA_HOME variable in your environment to match the
echo location of your Java installation.
goto fail
:execute
@rem Setup the command line
set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
@rem Execute Gradle
"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %*
:end
@rem End local scope for the variables with windows NT shell
if "%ERRORLEVEL%"=="0" goto mainEnd
:fail
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
rem the _cmd.exe /c_ return code!
if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1
exit /b 1
:mainEnd
if "%OS%"=="Windows_NT" endlocal
:omega

View File

@ -0,0 +1,16 @@
pluginManagement {
repositories {
gradlePluginPortal()
google()
mavenCentral()
}
}
dependencyResolutionManagement {
repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
repositories {
google()
mavenCentral()
}
}
rootProject.name = "fayConnectorDemo"
include ':app'

View File

@ -0,0 +1,15 @@
*.iml
.gradle
/local.properties
/.idea/caches
/.idea/libraries
/.idea/modules.xml
/.idea/workspace.xml
/.idea/navEditor.xml
/.idea/assetWizardSettings.xml
.DS_Store
/build
/captures
.externalNativeBuild
.cxx
local.properties

View File

@ -0,0 +1,3 @@
# Default ignored files
/shelf/
/workspace.xml

View File

@ -0,0 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="CompilerConfiguration">
<bytecodeTargetLevel target="11" />
</component>
</project>

View File

@ -0,0 +1,18 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="GradleSettings">
<option name="linkedExternalProjectsSettings">
<GradleProjectSettings>
<option name="testRunner" value="GRADLE" />
<option name="distributionType" value="DEFAULT_WRAPPED" />
<option name="externalProjectPath" value="$PROJECT_DIR$" />
<option name="modules">
<set>
<option value="$PROJECT_DIR$" />
<option value="$PROJECT_DIR$/app" />
</set>
</option>
</GradleProjectSettings>
</option>
</component>
</project>

View File

@ -0,0 +1,16 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="DesignSurface">
<option name="filePathToZoomLevelMap">
<map>
<entry key="..\:/android/projects/fayConnectorDemo/app/src/main/res/layout/activity_main.xml" value="0.358695652173913" />
</map>
</option>
</component>
<component name="ProjectRootManager" version="2" languageLevel="JDK_11" default="true" project-jdk-name="Android Studio default JDK" project-jdk-type="JavaSDK">
<output url="file://$PROJECT_DIR$/build/classes" />
</component>
<component name="ProjectType">
<option name="id" value="Android" />
</component>
</project>

View File

@ -0,0 +1 @@
/build

View File

@ -0,0 +1,38 @@
plugins {
id 'com.android.application'
}
android {
compileSdk 32
defaultConfig {
applicationId "com.yaheen.fayconnectordemo"
minSdk 29
targetSdk 32
versionCode 1
versionName "1.0"
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
}
buildTypes {
release {
minifyEnabled false
proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
}
}
compileOptions {
sourceCompatibility JavaVersion.VERSION_1_8
targetCompatibility JavaVersion.VERSION_1_8
}
}
dependencies {
implementation 'androidx.appcompat:appcompat:1.3.0'
implementation 'com.google.android.material:material:1.4.0'
implementation 'androidx.constraintlayout:constraintlayout:2.0.4'
testImplementation 'junit:junit:4.13.2'
androidTestImplementation 'androidx.test.ext:junit:1.1.3'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.4.0'
}

View File

@ -0,0 +1,21 @@
# Add project specific ProGuard rules here.
# You can control the set of applied configuration files using the
# proguardFiles setting in build.gradle.
#
# For more details, see
# http://developer.android.com/guide/developing/tools/proguard.html
# If your project uses WebView with JS, uncomment the following
# and specify the fully qualified class name to the JavaScript interface
# class:
#-keepclassmembers class fqcn.of.javascript.interface.for.webview {
# public *;
#}
# Uncomment this to preserve the line number information for
# debugging stack traces.
#-keepattributes SourceFile,LineNumberTable
# If you keep the line number information, uncomment this to
# hide the original source file name.
#-renamesourcefileattribute SourceFile

View File

@ -0,0 +1,26 @@
package com.yaheen.fayconnectordemo;
import android.content.Context;
import androidx.test.platform.app.InstrumentationRegistry;
import androidx.test.ext.junit.runners.AndroidJUnit4;
import org.junit.Test;
import org.junit.runner.RunWith;
import static org.junit.Assert.*;
/**
* Instrumented test, which will execute on an Android device.
*
* @see <a href="http://d.android.com/tools/testing">Testing documentation</a>
*/
@RunWith(AndroidJUnit4.class)
public class ExampleInstrumentedTest {
@Test
public void useAppContext() {
// Context of the app under test.
Context appContext = InstrumentationRegistry.getInstrumentation().getTargetContext();
assertEquals("com.yaheen.fayconnectordemo", appContext.getPackageName());
}
}

View File

@ -0,0 +1,35 @@
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools"
package="com.yaheen.fayconnectordemo">
<uses-permission android:name="android.permission.INTERNET"/><!--网络访问-->
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>
<uses-permission android:name="android.permission.CHANGE_WIFI_STATE"/>
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE"/>
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE"/>
<uses-permission android:name="android.permission.RECORD_AUDIO"/><!--录音权限`-->
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
<application
android:allowBackup="true"
android:dataExtractionRules="@xml/data_extraction_rules"
android:fullBackupContent="@xml/backup_rules"
android:icon="@mipmap/ic_launcher"
android:label="@string/app_name"
android:roundIcon="@mipmap/ic_launcher_round"
android:supportsRtl="true"
android:theme="@style/Theme.FayConnectorDemo"
tools:targetApi="31">
<activity
android:name=".MainActivity"
android:exported="true">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
</application>
</manifest>

View File

@ -0,0 +1,249 @@
package com.yaheen.fayconnectordemo;
import androidx.appcompat.app.AppCompatActivity;
import androidx.core.app.ActivityCompat;
import androidx.core.content.ContextCompat;
import android.Manifest;
import android.content.pm.PackageManager;
import android.media.AudioFormat;
import android.media.AudioManager;
import android.media.AudioRecord;
import android.media.MediaPlayer;
import android.media.MediaRecorder;
import android.os.Build;
import android.os.Bundle;
import android.util.Log;
import android.view.View;
import android.widget.TextView;
import com.google.android.material.snackbar.Snackbar;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.Socket;
import java.net.SocketException;
import java.util.Arrays;
import java.util.Date;
public class MainActivity extends AppCompatActivity {
private TextView tv = null;
private AudioRecord record;
private int recordBufsize = 0;
private Socket socket = null;
private InputStream in = null;
private OutputStream out = null;
private Thread sendThread = null;
private Thread receThread = null;
private boolean running = false;
private File cacheDir = null;
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
this.cacheDir = getCacheDir();
tv = this.findViewById(R.id.tv);
tv.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View view) {
Log.d("fay","onclick");
running = !running;
sendThread = new Thread(new Runnable() {
@Override
public void run() {
if (!running){//关闭
running = false;
return;
}
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
if (ContextCompat.checkSelfPermission(MainActivity.this, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
if (ActivityCompat.shouldShowRequestPermissionRationale(MainActivity.this, Manifest.permission.RECORD_AUDIO)) {
Log.d("fay","用户彻底拒绝了权限");
return;
} else {
// 用户未彻底拒绝授予权限
ActivityCompat.requestPermissions(MainActivity.this, new String[]{Manifest.permission.RECORD_AUDIO}, 1);
}
}
if (ContextCompat.checkSelfPermission(MainActivity.this, Manifest.permission.RECORD_AUDIO) == PackageManager.PERMISSION_GRANTED) {
Log.d("fay","权限ok");
if (record == null){
recordBufsize = AudioRecord
.getMinBufferSize(16000,
AudioFormat.CHANNEL_IN_MONO,
AudioFormat.ENCODING_PCM_16BIT);
record = new AudioRecord(MediaRecorder.AudioSource.MIC,
16000,
AudioFormat.CHANNEL_IN_MONO,
AudioFormat.ENCODING_PCM_16BIT,
recordBufsize);
}
try {
socket = new Socket("5gzvip.91tunnel.com", 10001);
in = socket.getInputStream();
out = socket.getOutputStream();
Snackbar.make(view, "fay控制器连接成功", Snackbar.LENGTH_SHORT)
.setAction("Action", null).show();
Log.d("fay","fay控制器连接成功");
}catch(IOException e){
Log.d("fay","socket连接失败");
return;
}
byte[] data = new byte[1024];
record.startRecording();
Snackbar.make(view, "麦克风启动成功", Snackbar.LENGTH_SHORT)
.setAction("Action", null).show();
Log.d("fay","麦克风启动成功");
try {
Snackbar.make(view, "开始传输音频", Snackbar.LENGTH_SHORT)
.setAction("Action", null).show();
Log.d("fay","开始传输音频");
while (MainActivity.this.running) {
record.read(data, 0, 1024);
if (data.length > 0) {
MainActivity.this.out.write(data);
}
}
}catch (Exception e){ //通过异常关闭链接
Log.d("fay","服务端关闭");
Snackbar.make(view, "服务端已经关闭", Snackbar.LENGTH_SHORT)
.setAction("Action", null).show();
}finally {
running = false;
record.stop();
record = null;
try {
socket.close();
}catch (Exception e){
}
Snackbar.make(view, "结束", Snackbar.LENGTH_SHORT)
.setAction("Action", null).show();
Log.d("fay","结束");
}
}
}
}
});
sendThread.start();
receThread = new Thread(new Runnable() {
@Override
public void run() {
try {
while (running) {
while (socket != null && !socket.isClosed()) {
byte[] data = new byte[9];
byte[] wavhead = new byte[]{0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08};//文件传输开始标记
in.read(data);
if (Arrays.equals(wavhead, data)) {
Log.d("fay", "开始接收音频文件");
String filedata = "";
data = new byte[1024];
while (data != null && data.length > 0) {
in.read(data);
filedata += MainActivity.bytesToHexString(data);
int index = filedata.indexOf("080706050403020100");
if (filedata.length() > 9 && index > 0){//wav文件结束标记
filedata = filedata.substring(0, index).replaceAll("F0F1F2F3F4F5F6F7F8", "");
File wavFile = new File(cacheDir, String.format("sample-%s.wav", new Date().getTime() + ""));
wavFile.createNewFile();
FileOutputStream fos = new FileOutputStream(wavFile);
fos.write(MainActivity.decodeHexBytes(filedata.toCharArray()));
fos.close();
Log.d("fay", "wav文件接收完成:" + wavFile.getAbsolutePath() + "," + filedata.length() / 2);
try{
MediaPlayer player = new MediaPlayer();
player.setDataSource(wavFile.getAbsolutePath());
player.prepare();
Thread.sleep(800);
player.start();
player.setOnCompletionListener(new MediaPlayer.OnCompletionListener() {
@Override
public void onCompletion(MediaPlayer mp) {
// TODO Auto-generated method stub
mp.release();
}
});
player.setLooping(false);
} catch (IOException e) {
Log.e("fay", e.toString());
}
break;
}
}
try {
Thread.sleep(1000);
} catch (Exception e) {
}
}
}
try {
Thread.sleep(1000);
} catch (Exception e) {
}
}
} catch (Exception e) {
Log.e("fay", e.toString());
}finally {
}
}});
receThread.start();
}
});
}
public static String bytesToHexString(byte[] data){
String result="";
for (int i = 0; i < data.length; i++) {
result+=Integer.toHexString((data[i] & 0xFF) | 0x100).toUpperCase().substring(1, 3);
}
return result;
}
public static byte[] decodeHexBytes(char[] data) {
int len = data.length;
if ((len & 0x01) != 0) {
throw new RuntimeException("未知的字符");
}
byte[] out = new byte[len >> 1];
for (int i = 0, j = 0; j < len; i++) {
int f = toDigit(data[j], j) << 4;
j++;
f = f | toDigit(data[j], j);
j++;
out[i] = (byte) (f & 0xFF);
}
return out;
}
protected static int toDigit(char ch, int index) {
int digit = Character.digit(ch, 16);
if (digit == -1) {
throw new RuntimeException("非法16进制字符 " + ch
+ " 在索引 " + index);
}
return digit;
}
}

View File

@ -0,0 +1,30 @@
<vector xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:aapt="http://schemas.android.com/aapt"
android:width="108dp"
android:height="108dp"
android:viewportWidth="108"
android:viewportHeight="108">
<path android:pathData="M31,63.928c0,0 6.4,-11 12.1,-13.1c7.2,-2.6 26,-1.4 26,-1.4l38.1,38.1L107,108.928l-32,-1L31,63.928z">
<aapt:attr name="android:fillColor">
<gradient
android:endX="85.84757"
android:endY="92.4963"
android:startX="42.9492"
android:startY="49.59793"
android:type="linear">
<item
android:color="#44000000"
android:offset="0.0" />
<item
android:color="#00000000"
android:offset="1.0" />
</gradient>
</aapt:attr>
</path>
<path
android:fillColor="#FFFFFF"
android:fillType="nonZero"
android:pathData="M65.3,45.828l3.8,-6.6c0.2,-0.4 0.1,-0.9 -0.3,-1.1c-0.4,-0.2 -0.9,-0.1 -1.1,0.3l-3.9,6.7c-6.3,-2.8 -13.4,-2.8 -19.7,0l-3.9,-6.7c-0.2,-0.4 -0.7,-0.5 -1.1,-0.3C38.8,38.328 38.7,38.828 38.9,39.228l3.8,6.6C36.2,49.428 31.7,56.028 31,63.928h46C76.3,56.028 71.8,49.428 65.3,45.828zM43.4,57.328c-0.8,0 -1.5,-0.5 -1.8,-1.2c-0.3,-0.7 -0.1,-1.5 0.4,-2.1c0.5,-0.5 1.4,-0.7 2.1,-0.4c0.7,0.3 1.2,1 1.2,1.8C45.3,56.528 44.5,57.328 43.4,57.328L43.4,57.328zM64.6,57.328c-0.8,0 -1.5,-0.5 -1.8,-1.2s-0.1,-1.5 0.4,-2.1c0.5,-0.5 1.4,-0.7 2.1,-0.4c0.7,0.3 1.2,1 1.2,1.8C66.5,56.528 65.6,57.328 64.6,57.328L64.6,57.328z"
android:strokeWidth="1"
android:strokeColor="#00000000" />
</vector>

View File

@ -0,0 +1,170 @@
<?xml version="1.0" encoding="utf-8"?>
<vector xmlns:android="http://schemas.android.com/apk/res/android"
android:width="108dp"
android:height="108dp"
android:viewportWidth="108"
android:viewportHeight="108">
<path
android:fillColor="#3DDC84"
android:pathData="M0,0h108v108h-108z" />
<path
android:fillColor="#00000000"
android:pathData="M9,0L9,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M19,0L19,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M29,0L29,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M39,0L39,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M49,0L49,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M59,0L59,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M69,0L69,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M79,0L79,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M89,0L89,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M99,0L99,108"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,9L108,9"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,19L108,19"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,29L108,29"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,39L108,39"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,49L108,49"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,59L108,59"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,69L108,69"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,79L108,79"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,89L108,89"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M0,99L108,99"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M19,29L89,29"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M19,39L89,39"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M19,49L89,49"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M19,59L89,59"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M19,69L89,69"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M19,79L89,79"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M29,19L29,89"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M39,19L39,89"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M49,19L49,89"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M59,19L59,89"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M69,19L69,89"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
<path
android:fillColor="#00000000"
android:pathData="M79,19L79,89"
android:strokeWidth="0.8"
android:strokeColor="#33FFFFFF" />
</vector>

View File

@ -0,0 +1,19 @@
<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity">
<TextView
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="连接/断开fay控制器"
android:id="@+id/tv"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent" />
</androidx.constraintlayout.widget.ConstraintLayout>

View File

@ -0,0 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<adaptive-icon xmlns:android="http://schemas.android.com/apk/res/android">
<background android:drawable="@drawable/ic_launcher_background" />
<foreground android:drawable="@drawable/ic_launcher_foreground" />
</adaptive-icon>

View File

@ -0,0 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<adaptive-icon xmlns:android="http://schemas.android.com/apk/res/android">
<background android:drawable="@drawable/ic_launcher_background" />
<foreground android:drawable="@drawable/ic_launcher_foreground" />
</adaptive-icon>

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 982 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.8 KiB

View File

@ -0,0 +1,16 @@
<resources xmlns:tools="http://schemas.android.com/tools">
<!-- Base application theme. -->
<style name="Theme.FayConnectorDemo" parent="Theme.MaterialComponents.DayNight.DarkActionBar">
<!-- Primary brand color. -->
<item name="colorPrimary">@color/purple_200</item>
<item name="colorPrimaryVariant">@color/purple_700</item>
<item name="colorOnPrimary">@color/black</item>
<!-- Secondary brand color. -->
<item name="colorSecondary">@color/teal_200</item>
<item name="colorSecondaryVariant">@color/teal_200</item>
<item name="colorOnSecondary">@color/black</item>
<!-- Status bar color. -->
<item name="android:statusBarColor" tools:targetApi="l">?attr/colorPrimaryVariant</item>
<!-- Customize your theme here. -->
</style>
</resources>

View File

@ -0,0 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<color name="purple_200">#FFBB86FC</color>
<color name="purple_500">#FF6200EE</color>
<color name="purple_700">#FF3700B3</color>
<color name="teal_200">#FF03DAC5</color>
<color name="teal_700">#FF018786</color>
<color name="black">#FF000000</color>
<color name="white">#FFFFFFFF</color>
</resources>

View File

@ -0,0 +1,3 @@
<resources>
<string name="app_name">fayConnectorDemo</string>
</resources>

View File

@ -0,0 +1,16 @@
<resources xmlns:tools="http://schemas.android.com/tools">
<!-- Base application theme. -->
<style name="Theme.FayConnectorDemo" parent="Theme.MaterialComponents.DayNight.DarkActionBar">
<!-- Primary brand color. -->
<item name="colorPrimary">@color/purple_500</item>
<item name="colorPrimaryVariant">@color/purple_700</item>
<item name="colorOnPrimary">@color/white</item>
<!-- Secondary brand color. -->
<item name="colorSecondary">@color/teal_200</item>
<item name="colorSecondaryVariant">@color/teal_700</item>
<item name="colorOnSecondary">@color/black</item>
<!-- Status bar color. -->
<item name="android:statusBarColor" tools:targetApi="l">?attr/colorPrimaryVariant</item>
<!-- Customize your theme here. -->
</style>
</resources>

View File

@ -0,0 +1,13 @@
<?xml version="1.0" encoding="utf-8"?><!--
Sample backup rules file; uncomment and customize as necessary.
See https://developer.android.com/guide/topics/data/autobackup
for details.
Note: This file is ignored for devices older that API 31
See https://developer.android.com/about/versions/12/backup-restore
-->
<full-backup-content>
<!--
<include domain="sharedpref" path="."/>
<exclude domain="sharedpref" path="device.xml"/>
-->
</full-backup-content>

View File

@ -0,0 +1,19 @@
<?xml version="1.0" encoding="utf-8"?><!--
Sample data extraction rules file; uncomment and customize as necessary.
See https://developer.android.com/about/versions/12/backup-restore#xml-changes
for details.
-->
<data-extraction-rules>
<cloud-backup>
<!-- TODO: Use <include> and <exclude> to control what is backed up.
<include .../>
<exclude .../>
-->
</cloud-backup>
<!--
<device-transfer>
<include .../>
<exclude .../>
</device-transfer>
-->
</data-extraction-rules>

View File

@ -0,0 +1,17 @@
package com.yaheen.fayconnectordemo;
import org.junit.Test;
import static org.junit.Assert.*;
/**
* Example local unit test, which will execute on the development machine (host).
*
* @see <a href="http://d.android.com/tools/testing">Testing documentation</a>
*/
public class ExampleUnitTest {
@Test
public void addition_isCorrect() {
assertEquals(4, 2 + 2);
}
}

View File

@ -0,0 +1,9 @@
// Top-level build file where you can add configuration options common to all sub-projects/modules.
plugins {
id 'com.android.application' version '7.2.1' apply false
id 'com.android.library' version '7.2.1' apply false
}
task clean(type: Delete) {
delete rootProject.buildDir
}

View File

@ -0,0 +1,21 @@
# Project-wide Gradle settings.
# IDE (e.g. Android Studio) users:
# Gradle settings configured through the IDE *will override*
# any settings specified in this file.
# For more details on how to configure your build environment visit
# http://www.gradle.org/docs/current/userguide/build_environment.html
# Specifies the JVM arguments used for the daemon process.
# The setting is particularly useful for tweaking memory settings.
org.gradle.jvmargs=-Xmx2048m -Dfile.encoding=UTF-8
# When configured, Gradle will run in incubating parallel mode.
# This option should only be used with decoupled projects. More details, visit
# http://www.gradle.org/docs/current/userguide/multi_project_builds.html#sec:decoupled_projects
# org.gradle.parallel=true
# AndroidX package structure to make it clearer which packages are bundled with the
# Android operating system, and which are packaged with your app"s APK
# https://developer.android.com/topic/libraries/support-library/androidx-rn
android.useAndroidX=true
# Enables namespacing of each library's R class so that its R class includes only the
# resources declared in the library itself and none from the library's dependencies,
# thereby reducing the size of the R class for that library
android.nonTransitiveRClass=true

View File

@ -0,0 +1,6 @@
#Fri Jan 20 09:27:45 CST 2023
distributionBase=GRADLE_USER_HOME
distributionUrl=https\://services.gradle.org/distributions/gradle-7.3.3-bin.zip
distributionPath=wrapper/dists
zipStorePath=wrapper/dists
zipStoreBase=GRADLE_USER_HOME

View File

@ -0,0 +1,185 @@
#!/usr/bin/env sh
#
# Copyright 2015 the original author or authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
##############################################################################
##
## Gradle start up script for UN*X
##
##############################################################################
# Attempt to set APP_HOME
# Resolve links: $0 may be a link
PRG="$0"
# Need this for relative symlinks.
while [ -h "$PRG" ] ; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> \(.*\)$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG=`dirname "$PRG"`"/$link"
fi
done
SAVED="`pwd`"
cd "`dirname \"$PRG\"`/" >/dev/null
APP_HOME="`pwd -P`"
cd "$SAVED" >/dev/null
APP_NAME="Gradle"
APP_BASE_NAME=`basename "$0"`
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
# Use the maximum available, or set MAX_FD != -1 to use that value.
MAX_FD="maximum"
warn () {
echo "$*"
}
die () {
echo
echo "$*"
echo
exit 1
}
# OS specific support (must be 'true' or 'false').
cygwin=false
msys=false
darwin=false
nonstop=false
case "`uname`" in
CYGWIN* )
cygwin=true
;;
Darwin* )
darwin=true
;;
MINGW* )
msys=true
;;
NONSTOP* )
nonstop=true
;;
esac
CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar
# Determine the Java command to use to start the JVM.
if [ -n "$JAVA_HOME" ] ; then
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
# IBM's JDK on AIX uses strange locations for the executables
JAVACMD="$JAVA_HOME/jre/sh/java"
else
JAVACMD="$JAVA_HOME/bin/java"
fi
if [ ! -x "$JAVACMD" ] ; then
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
else
JAVACMD="java"
which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Please set the JAVA_HOME variable in your environment to match the
location of your Java installation."
fi
# Increase the maximum file descriptors if we can.
if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then
MAX_FD_LIMIT=`ulimit -H -n`
if [ $? -eq 0 ] ; then
if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then
MAX_FD="$MAX_FD_LIMIT"
fi
ulimit -n $MAX_FD
if [ $? -ne 0 ] ; then
warn "Could not set maximum file descriptor limit: $MAX_FD"
fi
else
warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT"
fi
fi
# For Darwin, add options to specify how the application appears in the dock
if $darwin; then
GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\""
fi
# For Cygwin or MSYS, switch paths to Windows format before running java
if [ "$cygwin" = "true" -o "$msys" = "true" ] ; then
APP_HOME=`cygpath --path --mixed "$APP_HOME"`
CLASSPATH=`cygpath --path --mixed "$CLASSPATH"`
JAVACMD=`cygpath --unix "$JAVACMD"`
# We build the pattern for arguments to be converted via cygpath
ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null`
SEP=""
for dir in $ROOTDIRSRAW ; do
ROOTDIRS="$ROOTDIRS$SEP$dir"
SEP="|"
done
OURCYGPATTERN="(^($ROOTDIRS))"
# Add a user-defined pattern to the cygpath arguments
if [ "$GRADLE_CYGPATTERN" != "" ] ; then
OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)"
fi
# Now convert the arguments - kludge to limit ourselves to /bin/sh
i=0
for arg in "$@" ; do
CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -`
CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option
if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition
eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"`
else
eval `echo args$i`="\"$arg\""
fi
i=`expr $i + 1`
done
case $i in
0) set -- ;;
1) set -- "$args0" ;;
2) set -- "$args0" "$args1" ;;
3) set -- "$args0" "$args1" "$args2" ;;
4) set -- "$args0" "$args1" "$args2" "$args3" ;;
5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;;
6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;;
7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;;
8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;;
9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;;
esac
fi
# Escape application args
save () {
for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done
echo " "
}
APP_ARGS=`save "$@"`
# Collect all arguments for the java command, following the shell quoting and substitution rules
eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS"
exec "$JAVACMD" "$@"

View File

@ -0,0 +1,89 @@
@rem
@rem Copyright 2015 the original author or authors.
@rem
@rem Licensed under the Apache License, Version 2.0 (the "License");
@rem you may not use this file except in compliance with the License.
@rem You may obtain a copy of the License at
@rem
@rem https://www.apache.org/licenses/LICENSE-2.0
@rem
@rem Unless required by applicable law or agreed to in writing, software
@rem distributed under the License is distributed on an "AS IS" BASIS,
@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
@rem See the License for the specific language governing permissions and
@rem limitations under the License.
@rem
@if "%DEBUG%" == "" @echo off
@rem ##########################################################################
@rem
@rem Gradle startup script for Windows
@rem
@rem ##########################################################################
@rem Set local scope for the variables with windows NT shell
if "%OS%"=="Windows_NT" setlocal
set DIRNAME=%~dp0
if "%DIRNAME%" == "" set DIRNAME=.
set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME%
@rem Resolve any "." and ".." in APP_HOME to make it shorter.
for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi
@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"
@rem Find java.exe
if defined JAVA_HOME goto findJavaFromJavaHome
set JAVA_EXE=java.exe
%JAVA_EXE% -version >NUL 2>&1
if "%ERRORLEVEL%" == "0" goto execute
echo.
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
echo.
echo Please set the JAVA_HOME variable in your environment to match the
echo location of your Java installation.
goto fail
:findJavaFromJavaHome
set JAVA_HOME=%JAVA_HOME:"=%
set JAVA_EXE=%JAVA_HOME%/bin/java.exe
if exist "%JAVA_EXE%" goto execute
echo.
echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME%
echo.
echo Please set the JAVA_HOME variable in your environment to match the
echo location of your Java installation.
goto fail
:execute
@rem Setup the command line
set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar
@rem Execute Gradle
"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %*
:end
@rem End local scope for the variables with windows NT shell
if "%ERRORLEVEL%"=="0" goto mainEnd
:fail
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
rem the _cmd.exe /c_ return code!
if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1
exit /b 1
:mainEnd
if "%OS%"=="Windows_NT" endlocal
:omega

View File

@ -0,0 +1,16 @@
pluginManagement {
repositories {
gradlePluginPortal()
google()
mavenCentral()
}
}
dependencyResolutionManagement {
repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS)
repositories {
google()
mavenCentral()
}
}
rootProject.name = "fayConnectorDemo"
include ':app'

View File

@ -8,7 +8,7 @@
"hobby": "\u53d1\u5446",
"job": "\u4ea7\u54c1\u5e03\u9053\u8005",
"name": "\u9648\u5347",
"voice": "YUN_XI",
"voice": "XIAO_XIAO",
"zodiac": "\u86c7"
},
"interact": {
@ -21,7 +21,7 @@
"indifferent": 10,
"join": 10
},
"playSound": true
"playSound": false
},
"items": [
{
@ -42,11 +42,11 @@
"source": {
"liveRoom": {
"enabled": false,
"url": "https://live.douyin.com/"
"url": "https://v.douyin.com/hL6ehu8/"
},
"record": {
"device": "\u9ea6\u514b\u98ce (HD Webcam C525)",
"enabled": true
"device": "\u9470\u866b\u6e80 (BT-50 PRO Hands-Free AG Aud",
"enabled": false
}
}
}

View File

@ -4,6 +4,7 @@ import os
import random
import time
import wave
import socket
import eyed3
from openpyxl import load_workbook
@ -11,11 +12,12 @@ from openpyxl import load_workbook
# 适应模型使用
import numpy as np
# import tensorflow as tf
import fay_booter
from ai_module import xf_aiui
from ai_module import xf_ltp
from ai_module.ms_tts_sdk import Speech
from core import wsa_server, tts_voice
from core import wsa_server, tts_voice, song_player
from core.interact import Interact
from core.tts_voice import EnumVoice
from scheduler.thread_manager import MyThread
from utils import util, storer, config_util
@ -30,14 +32,28 @@ class FeiFei:
self.a_msg = 'hi,我叫菲菲英文名是fay'
self.mood = 0.0 # 情绪值
self.item_index = 0
self.deviceSocket = None
self.deviceConnect = None
#启动音频输入输出设备的连接服务
self.deviceSocketThread = MyThread(target=self.__accept_audio_device_output_connect)
self.deviceSocketThread.start()
self.X = np.array([1, 0, 0, 0, 0, 0, 0, 0]).reshape(1, -1) # 适应模型变量矩阵
# self.W = np.array([0.01577594,1.16119452,0.75828,0.207746,1.25017864,0.1044121,0.4294899,0.2770932]).reshape(-1,1) #适应模型变量矩阵
self.W = np.array([0.0, 0.6, 0.1, 0.7, 0.3, 0.0, 0.0, 0.0]).reshape(-1, 1) # 适应模型变量矩阵
self.command_keyword = [
[['播放歌曲', '播放音乐', '唱首歌', '放首歌', '听音乐', '你会唱歌吗', '我想首听歌'], 'playSong'],
[['关闭', '再见', '你走吧'], 'stop'],
[['静音', '闭嘴', '我想静静'], 'mute'],
[['取消静音', '你在哪呢', '你可以说话了'], 'unmute'],
[['换个性别', '换个声音'], 'changeVoice']
]
# 人设提问关键字
self.attribute_keyword = [
[['你叫什么名字', '你的名字是什么','你是谁'], 'name'],
[['你叫什么名字', '你的名字是什么'], 'name'],
[['你是男的还是女的', '你是男生还是女生', '你的性别是什么', '你是男生吗', '你是女生吗', '你是男的吗', '你是女的吗', '你是男孩子吗', '你是女孩子吗', ], 'gender', ],
[['你今年多大了', '你多大了', '你今年多少岁', '你几岁了', '你今年几岁了', '你今年几岁了', '你什么时候出生', '你的生日是什么', '你的年龄'], 'age', ],
[['你的家乡在哪', '你的家乡是什么', '你家在哪', '你住在哪', '你出生在哪', '你的出生地在哪', '你的出生地是什么', ], 'birth', ],
@ -69,6 +85,8 @@ class FeiFei:
self.__running = True
self.sp.connect() # 预连接
self.last_quest_time = time.time()
self.playing = False
self.muting = False
def __string_similar(self, s1, s2):
return difflib.SequenceMatcher(None, s1, s2).quick_ratio()
@ -101,7 +119,44 @@ class FeiFei:
return last_answer
return None
def __get_answer(self, text):
def __play_song(self):
self.playing = True
song_player.play()
self.playing = False
wsa_server.get_web_instance().add_cmd({"panelMsg": ""})
def __get_answer(self, interleaver, text):
if interleaver == "mic":
# 命令
keyword = self.__get_keyword(self.command_keyword, text)
if keyword is not None:
if keyword == "playSong":
MyThread(target=self.__play_song).start()
wsa_server.get_web_instance().add_cmd({"panelMsg": ""})
elif keyword == "stop":
fay_booter.stop()
wsa_server.get_web_instance().add_cmd({"panelMsg": ""})
wsa_server.get_web_instance().add_cmd({"liveState": 0})
elif keyword == "mute":
self.muting = True
self.speaking = True
self.a_msg = "好的"
MyThread(target=self.__say, args=['interact']).start()
time.sleep(0.5)
wsa_server.get_web_instance().add_cmd({"panelMsg": ""})
elif keyword == "unmute":
self.muting = False
return None
elif keyword == "changeVoice":
voice = tts_voice.get_voice_of(config_util.config["attribute"]["voice"])
for v in tts_voice.get_voice_list():
if v != voice:
config_util.config["attribute"]["voice"] = v.name
break
config_util.save_config(config_util.config)
wsa_server.get_web_instance().add_cmd({"panelMsg": ""})
return "NO_ANSWER"
# 人设问答
keyword = self.__get_keyword(self.attribute_keyword, text)
@ -170,16 +225,18 @@ class FeiFei:
# 简化逻辑:默认执行带货脚本,带货脚本执行其间有人互动,则执行完当前脚本就回应最后三条互动,回应完继续执行带货脚本
if i <= 3 and len(self.interactive) > i:
i += 1
interact = self.interactive[0 - i]
if interact[0] == 1:
self.q_msg = interact[2]
index = interact[0]
interact: Interact = self.interactive[0 - i]
if interact.interact_type == 1:
self.q_msg = interact.data["msg"]
index = interact.interact_type
# print("index:{0}".format(index))
user_name = interact[1]
user_name = interact.data["user"]
# self.__isExecute = True #!!!!
if index == 1:
answer = self.__get_answer(self.q_msg)
answer = self.__get_answer(interact.interleaver, self.q_msg)
if self.muting:
continue
text = ''
if answer is None:
try:
@ -197,8 +254,9 @@ class FeiFei:
util.log(1, '自然语言处理错误!')
wsa_server.get_web_instance().add_cmd({"panelMsg": ""})
continue
else:
elif answer != 'NO_ANSWER':
text = answer
if len(user_name) == 0:
self.a_msg = text
else:
@ -209,22 +267,34 @@ class FeiFei:
random.randint(0, 2)]
elif index == 3:
msg = ""
for index in range(1, len(interact), 4):
try:
gift = interact[index + 2]
gift_name = '礼物'
if gift[0] != -1:
gift_name = gift[1]
msg = msg + "{}送给我的{}{}".format(interact[index], interact[index + 3], gift_name)
except BaseException as e:
print("[System] 礼物处理错误!")
print(e)
self.a_msg = '感谢感谢,感谢' + msg
gift = interact.data["gift"]
self.a_msg = '感谢感谢,感谢 {}送给我的{}{}'.format(interact.data["user"], interact.data["amount"], gift[1])
elif index == 4:
self.a_msg = '感谢关注'
elif index == 5:
msg = ""
for i in range(0, len(interact.data["gifts"])):
user = interact.data["gifts"][i]["user"]
gift = interact.data["gifts"][i]["gift"]
amount = interact.data["gifts"][i]["amount"]
msg += "{}送给我的{}{}".format(user, amount, gift[1])
self.a_msg = '感谢感谢,感谢' + msg
# elif index == 5:
# msg = ""
# for index in range(1, len(interact), 4):
# try:
# gift = interact[index + 2]
# gift_name = '礼物'
# if gift[0] != -1:
# gift_name = gift[1]
# msg = msg + "{}送给我的{}个{}".format(interact[index], interact[index + 3], gift_name)
# except BaseException as e:
# print("[System] 礼物处理错误!")
# print(e)
# self.a_msg = '感谢感谢,感谢' + msg
self.last_speak_data = self.a_msg
self.speaking = True
MyThread(target=self.__say, args=['interact']).start()
@ -280,46 +350,53 @@ class FeiFei:
return "usage"
return None
def on_interact(self, interact):
def on_interact(self, interact: Interact):
# 合并同类交互
# 进入
if interact[0] == 2:
if interact.interact_type == 2:
itr = self.__get_interactive(2)
if itr is None:
self.interactive.append(interact)
else:
newItr = (2, itr[1] + ', ' + interact[1], itr[2])
newItr = (2, itr.data["user"] + ', ' + interact.data["user"], itr.data["msg"])
self.interactive.remove(itr)
self.interactive.append(newItr)
# 送礼
elif interact[0] == 3:
itr = self.__get_interactive(3)
if itr is None:
self.interactive.append(interact)
else:
newItrList = []
newItrList.extend(itr)
newItrList.append(itr[2])
newItrList.append(itr[3])
newItrList.append(itr[4])
elif interact.interact_type == 3:
gifts = []
rm_list = []
for itr in self.interactive:
if itr.interact_type == 3:
gifts.append({
"user": itr.data["user"],
"gift": itr.data["gift"],
"amount": itr.data["amount"]
})
rm_list.append(itr)
elif itr.interact_type == 5:
for gift in itr.data["gifts"]:
gifts.append(gift)
rm_list.append(itr)
if len(rm_list) > 0:
for itr in rm_list:
self.interactive.remove(itr)
self.interactive.append(tuple(newItrList))
self.interactive.append(Interact("live", 5, {"gifts": gifts}))
# 关注
elif interact[0] == 4:
elif interact.interact_type == 4:
if self.__get_interactive(2) is None:
self.interactive.append(interact)
else:
self.interactive.append(interact)
MyThread(target=self.__update_mood, args=[interact[0]]).start()
MyThread(target=self.__update_mood, args=[interact.interact_type]).start()
MyThread(target=storer.storage_live_interact, args=[interact]).start()
def __get_interactive(self, interactType):
def __get_interactive(self, interactType) -> Interact:
for interact in self.interactive:
if interact[0] == interactType:
if interact is Interact and interact.interact_type == interactType:
return interact
return None
@ -397,19 +474,13 @@ class FeiFei:
else:
# print(self.__get_mood().name + self.a_msg)
util.printInfo(1, '菲菲', '({}) {}'.format(self.__get_mood(), self.a_msg))
MyThread(target=storer.storage_live_interact, args=[(0, '菲菲', self.a_msg)]).start()
MyThread(target=storer.storage_live_interact, args=[Interact('Fay', 0, {'user': 'Fay', 'msg': self.a_msg})]).start()
util.log(1, '合成音频...')
tm = time.time()
result = self.sp.to_sample(self.a_msg, self.__get_mood())
util.log(1, '合成音频完成. 耗时: {} ms'.format(math.floor((time.time() - tm) * 1000)))
util.log(1, '合成音频完成. 耗时: {} ms 文件:{}'.format(math.floor((time.time() - tm) * 1000), result))
if result is not None:
# playsound(result)
# with wave.open(result, 'rb') as wav_file:
# wav_length = wav_file.getnframes() / float(wav_file.getframerate())
# time.sleep(wav_length)
MyThread(target=self.__send_audio, args=[result, styleType]).start()
# MyThread(target=self.__play_audio, args=[result]).start()
# MyThread(target=self.__waiting_speaking, args=[result]).start()
return result
except BaseException as e:
print(e)
@ -425,13 +496,33 @@ class FeiFei:
def __send_audio(self, file_url, say_type):
try:
audio_length = eyed3.load(file_url).info.time_secs
# audio_length = eyed3.load(file_url).info.time_secs mp3音频长度
with wave.open(file_url, 'rb') as wav_file:
audio_length = wav_file.getnframes() / float(wav_file.getframerate())
if audio_length <= config_util.config["interact"]["maxInteractTime"] or say_type == "script":
if config_util.config["interact"]["playSound"]:
if config_util.config["interact"]["playSound"]: # 播放音频
self.__play_sound(file_url)
else:
else:#TODO 发送音频给ue和socket
content = {'Topic': 'Unreal', 'Data': {'Key': 'audio', 'Value': os.path.abspath(file_url), 'Time': audio_length, 'Type': say_type}}
wsa_server.get_instance().add_cmd(content)
if self.deviceConnect is not None:
try:
self.deviceConnect.send(b'\x00\x01\x02\x03\x04\x05\x06\x07\x08') # 发送音频开始标志,同时也检查设备是否在线
wavfile = open(os.path.abspath(file_url),'rb')
data = wavfile.read(1024)
total = 0
while data:
total += len(data)
self.deviceConnect.send(data)
data = wavfile.read(1024)
time.sleep(0.001)
self.deviceConnect.send(b'\x08\x07\x06\x05\x04\x03\x02\x01\x00')# 发送音频结束标志
util.log(1, "远程音频发送完成:{}".format(total))
except socket.error as serr:
util.log(1,"远程音频输入输出设备已经断开:{}".format(serr))
wsa_server.get_web_instance().add_cmd({"panelMsg": self.a_msg})
time.sleep(audio_length + 0.5)
wsa_server.get_web_instance().add_cmd({"panelMsg": ""})
@ -441,22 +532,28 @@ class FeiFei:
except Exception as e:
print(e)
# def __send_audio(self, file_url, say_type):
# try:
# # time.sleep(0.25)
# with wave.open(file_url, 'rb') as wav_file:
# wav_length = wav_file.getnframes() / float(wav_file.getframerate())
# print(wav_length)
# if wav_length <= config_util.config["interact"]["maxInteractTime"] or say_type == "script":
# if config_util.config["interact"]["playSound"]:
# self.__play_sound(file_url)
# else:
# content = {'Topic': 'Unreal', 'Data': {'Key': 'audio', 'Value': os.path.abspath(file_url), 'Time': wav_length, 'Type': say_type}}
# wsa_server.get_instance().add_cmd(content)
# time.sleep(wav_length + 0.5)
# self.speaking = False
# except Exception as e:
# print(e)
def __device_socket_keep_alive(self):
while True:
if self.deviceConnect is not None:
try:
self.deviceConnect.send(b'\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8')#发送心跳包
except Exception as serr:
util.log(1,"远程音频输入输出设备已经断开:{}".format(serr))
self.deviceConnect = None
time.sleep(1)
def __accept_audio_device_output_connect(self):
self.deviceSocket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
self.deviceSocket.bind(("0.0.0.0",10001))
self.deviceSocket.listen(1)
addr = None
try:
while True:
self.deviceConnect,addr=self.deviceSocket.accept() #接受TCP连接并返回新的套接字与IP地址
MyThread(target=self.__device_socket_keep_alive).start()
util.log(1,"远程音频输入输出设备连接上:{}".format(addr))
except Exception as err:
pass
def __waiting_speaking(self, file_url):
try:
@ -501,4 +598,14 @@ class FeiFei:
def stop(self):
self.__running = False
song_player.stop()
self.speaking = False
self.playing = False
self.sp.close()
wsa_server.get_web_instance().add_cmd({"panelMsg": ""})
if self.deviceConnect is not None:
self.deviceConnect.close()
self.deviceConnect = None
if self.deviceSocket is not None:
self.deviceSocket.close()

6
core/interact.py Normal file
View File

@ -0,0 +1,6 @@
class Interact:
def __init__(self, interleaver: str, interact_type: int, data: dict):
self.interleaver = interleaver
self.interact_type = interact_type
self.data = data

View File

@ -3,13 +3,17 @@ import math
import time
from abc import abstractmethod
import pyaudio
import wave
from ai_module.ali_nls import ALiNls
from core import wsa_server
from scheduler.thread_manager import MyThread
from utils import util
# 启动时间 (秒)
_ATTACK = 0.2
@ -19,32 +23,23 @@ _RELEASE = 0.75
class Recorder:
def __init__(self, device, fay):
self.__device = device
def __init__(self, fay):
self.__fay = fay
self.__RATE = 16000
self.__FORMAT = pyaudio.paInt16
self.__CHANNELS = 1
self.__running = True
self.__processing = False
self.__history_level = []
self.__history_data = []
self.__dynamic_threshold = 0.5
self.__dynamic_threshold = 0.5 # 声音识别的音量阈值
self.__MAX_LEVEL = 25000
self.__MAX_BLOCK = 100
self.__aLiNls = ALiNls()
def __findInternalRecordingDevice(self, p):
for i in range(p.get_device_count()):
devInfo = p.get_device_info_by_index(i)
if devInfo['name'].find(self.__device) >= 0 and devInfo['hostApi'] == 0:
return i
util.log(1, '[!] 无法找到内录设备!')
return -1
def __get_history_average(self, number):
total = 0
@ -73,6 +68,8 @@ class Recorder:
print(text + " [" + str(int(per * 100)) + "%]")
def __waitingResult(self, iat: ALiNls):
if self.__fay.playing:
return
self.processing = True
t = time.time()
tm = time.time()
@ -90,18 +87,22 @@ class Recorder:
self.dynamic_threshold = self.__get_history_percentage(30)
wsa_server.get_web_instance().add_cmd({"panelMsg": ""})
def __record(self):
p = pyaudio.PyAudio()
device_id = self.__findInternalRecordingDevice(p)
if device_id < 0:
return
stream = p.open(input_device_index=device_id, rate=self.__RATE, format=self.__FORMAT, channels=self.__CHANNELS, input=True, frames_per_buffer=1024)
self.total = 0
stream = self.get_stream()
isSpeaking = False
last_mute_time = time.time()
last_speaking_time = time.time()
while self.__running:
data = stream.read(1024, exception_on_overflow=False)
data = stream.read(1024)
if not data:
continue
else:
self.total += len(data)
level = audioop.rms(data, 2)
if len(self.__history_data) >= 5:
self.__history_data.pop(0)
@ -122,8 +123,8 @@ class Recorder:
if percentage > self.__dynamic_threshold and not self.__fay.speaking:
last_speaking_time = time.time()
if not self.__processing and not isSpeaking and time.time() - last_mute_time > _ATTACK:
soon = True
isSpeaking = True
soon = True #
isSpeaking = True #用户正在说话
util.log(3, "聆听中...")
self.__aLiNls = ALiNls()
try:
@ -144,9 +145,10 @@ class Recorder:
if not soon and isSpeaking:
self.__aLiNls.send(data)
stream.stop_stream()
stream.close()
p.terminate()
print("接收完成:{}".format(self.total))
def set_processing(self, processing):
self.__processing = processing
@ -161,3 +163,9 @@ class Recorder:
@abstractmethod
def on_speaking(self, text):
pass
#TODO Edit by xszyou on 20230113:把流的获取方式封装出来方便实现麦克风录制及网络流等不同的流录制子类
@abstractmethod
def get_stream(self):
pass

68
core/song_player.py Normal file
View File

@ -0,0 +1,68 @@
import os.path
import random
import time
import eyed3
import requests
import re
import pygame
from utils import util
__playing = False
song_name = ""
def __play_song(song_id: str):
file_url = "./songs/{}.mp3".format(song_name)
if not os.path.exists("./songs"):
os.mkdir("./songs")
if not os.path.exists(file_url):
url = "https://music.163.com/song/media/outer/url?id=" + song_id
response = requests.request("GET", url)
with open(file_url, "wb") as mp3:
mp3.write(response.content)
pygame.mixer.music.load(file_url)
pygame.mixer.music.play()
util.log(3, "正在播放 {}".format(song_name))
audio_length = eyed3.load(file_url).info.time_secs
last_time = time.time()
while __playing and time.time() - last_time < audio_length:
time.sleep(0.05)
pass
def __random_song():
# 歌单列表
id_list = [
"3778678", # 热歌榜
# "1978921795", # 电音榜
# "10520166", # 国电榜
# "991319590", # 说唱榜
]
url = "https://music.163.com/discover/toplist?id=" + id_list[random.randrange(0, len(id_list))]
response = requests.request("GET", url)
song_list = re.findall("<li><a href=\"/song\?id=([0-9]*)\">(.*?)</a></li>", response.text)
index = random.randrange(0, len(song_list))
return song_list[index]
def play():
global __playing
global song_name
__playing = True
while __playing:
song = __random_song()
try:
song_name = song[1]
__play_song(song[0])
break
except Exception as e:
util.log(1, "无法播放 {} 可能需要VIP".format(song[1]))
def stop():
global __playing
__playing = False
pygame.mixer.music.stop()

View File

@ -9,6 +9,7 @@ from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support.expected_conditions import presence_of_element_located
from core.interact import Interact
from scheduler.thread_manager import MyThread
from utils import config_util, util
@ -46,7 +47,7 @@ class Viewer:
self.live_driver.get(self.url)
self.user_driver = webdriver.Chrome(config_util.system_chrome_driver, options=self.chrome_options)
self.__wait_live_start()
self.user_sec_uid = self.__get_render_data(self.live_driver)['app']['initialState']['roomStore']['roomInfo']['room']['owner']['sec_uid']
self.user_sec_uid = self.__get_render_data(self.live_driver)['initialState']['roomStore']['roomInfo']['room']['owner']['sec_uid']
MyThread(target=self.__live_state_runnable).start()
MyThread(target=self.__join_runnable).start()
MyThread(target=self.__interact_runnable).start()
@ -142,7 +143,7 @@ class Viewer:
if len(text) > 0 and self.last_join_data != text:
self.last_join_data = text
user = text[0:len(text) - 3]
return 2, user, '来了'
return Interact("live", 2, {"user": user, "msg": "来了"})
except BaseException as e:
return None
return None
@ -213,9 +214,14 @@ class Viewer:
gift = self.__get_gift_type(item_msg.get_attribute('src'))
arg = speak[1].split(' ')
amount = int(arg[len(arg) - 1]) # 礼物数量
interact_data.append((3, speak[0], ('送出了 {0} X {1}'.format(gift[1], amount)), gift, amount))
interact_data.append(Interact("live", 3, {
"user": speak[0],
"msg": ('送出了 {0} X {1}'.format(gift[1], amount)),
"gift": gift,
"amount": amount
}))
else:
interact_data.append((1, speak[0], speak[1]))
interact_data.append(Interact("live", 1, {"user": speak[0], "msg": speak[1]}))
except BaseException as e:
interact_data.reverse()
return interact_data
@ -266,7 +272,13 @@ class Viewer:
break
if fs >= 0:
if self.live_started and 0 < followers < fs:
self.on_interact((4, 'None', '粉丝关注'), time.time())
self.on_interact(
Interact("live", 4, {
"user": "None",
"msg": "粉丝关注"
}),
time.time()
)
followers = fs
else:
util.log(1, '粉丝数获取异常')

View File

@ -3,14 +3,18 @@ from asyncio import AbstractEventLoop
import websockets
import asyncio
import json
from abc import abstractmethod
import sys
sys.path.append("E:/3Dproject/feifeibeifen/Projects/FeiFei-22-06-17-2/")
from websockets.legacy.server import Serve
from scheduler.thread_manager import MyThread
from utils import util
class MyServer:
def __init__(self, host='127.0.0.1', port=10000):
def __init__(self, host='0.0.0.0', port=10000):
self.__host = host # ip
self.__port = port # 端口号
self.__listCmd = [] # 要发送的信息的列表
@ -19,6 +23,7 @@ class MyServer:
self.__event_loop: AbstractEventLoop = None
self.__running = True
self.__pending = None
self.isConnect = False
def __del__(self):
self.stop_server()
@ -36,16 +41,20 @@ class MyServer:
# util.log('发送 {}'.format(message))
async def __handler(self, websocket, path):
isConnect = True
util.log(1,"websocket连接上:{}".format(self.__port))
self.on_connect_handler()
consumer_task = asyncio.ensure_future(self.__consumer_handler(websocket, path))
producer_task = asyncio.ensure_future(self.__producer_handler(websocket, path))
done, self.__pending = await asyncio.wait([consumer_task, producer_task], return_when=asyncio.FIRST_COMPLETED, )
for task in self.__pending:
task.cancel()
isConnect = False
util.log(1,"websocket连接断开:{}".format(self.__port))
# 接收处理
async def __consumer(self, message):
pass
# print('recv message: {0}'.format(message))
self.on_revice_handler(message)
# 发送处理
async def __producer(self):
@ -54,6 +63,17 @@ class MyServer:
else:
return None
#Edit by xszyou on 20230113:通过继承此类来实现服务端的接收处理逻辑
@abstractmethod
def on_revice_handler(self, message):
pass
#Edit by xszyou on 20230114:通过继承此类来实现服务端的连接处理逻辑
@abstractmethod
def on_connect_handler(self):
pass
# 创建server
def __connect(self):
self.__event_loop = asyncio.new_event_loop()
@ -81,6 +101,7 @@ class MyServer:
# 关闭服务
def stop_server(self):
self.__running = False
isConnect = False
if self.__server is None:
return
self.__server.ws_server.close()
@ -96,22 +117,54 @@ class MyServer:
except BaseException as e:
print("Error: {}".format(e))
class HumanServer(MyServer):
def __init__(self, host='0.0.0.0', port=10000):
super().__init__(host, port)
def on_revice_handler(self, message):
pass
def on_connect_handler(self):
pass
class WebServer(MyServer):
def __init__(self, host='0.0.0.0', port=10000):
super().__init__(host, port)
def on_revice_handler(self, message):
pass
def on_connect_handler(self):
self.add_cmd({"panelMsg": "使用提示:直播,请关闭麦克风。连接数字人,请关闭面板播放。"})
class TestServer(MyServer):
def __init__(self, host='0.0.0.0', port=10000):
super().__init__(host, port)
def on_revice_handler(self, message):
print(message)
def on_connect_handler(self):
print("连接上了")
__instance: MyServer = None
__web_instance: MyServer = None
def new_instance(host='127.0.0.1', port=10000) -> MyServer:
def new_instance(host='0.0.0.0', port=10000) -> MyServer:
global __instance
if __instance is None:
__instance = MyServer(host, port)
__instance = HumanServer(host, port)
return __instance
def new_web_instance(host='127.0.0.1', port=10000) -> MyServer:
def new_web_instance(host='0.0.0.0', port=10000) -> MyServer:
global __web_instance
if __web_instance is None:
__web_instance = MyServer(host, port)
__web_instance = WebServer(host, port)
return __web_instance
@ -121,3 +174,7 @@ def get_instance() -> MyServer:
def get_web_instance() -> MyServer:
return __web_instance
if __name__ == '__main__':
testServer = TestServer(host='0.0.0.0', port=10000)
testServer.start_server()

View File

@ -1,10 +1,21 @@
import time
from io import BytesIO
import socket
import pyaudio
import numpy as np
import scipy.io.wavfile as wav
import wave
from core.interact import Interact
from core.recorder import Recorder
from core.fay_core import FeiFei
from core.viewer import Viewer
from scheduler.thread_manager import MyThread
from utils import util, config_util
from utils import util, config_util, stream_util, ngrok_util
from core.wsa_server import MyServer
feiFei: FeiFei = None
viewerListener: Viewer = None
@ -18,15 +29,15 @@ class ViewerListener(Viewer):
def __init__(self, url):
super().__init__(url)
def on_interact(self, interact, event_time):
def on_interact(self, interact: Interact, event_time):
type_names = {
1: '发言',
2: '进入',
3: '送礼',
4: '关注'
}
util.printInfo(1, type_names[interact[0]], '{}: {}'.format(interact[1], interact[2]), event_time)
if interact[0] == 1:
util.printInfo(1, type_names[interact.interact_type], '{}: {}'.format(interact.data["user"], interact.data["msg"]), event_time)
if interact.interact_type == 1:
feiFei.last_quest_time = time.time()
thr = MyThread(target=feiFei.on_interact, args=[interact])
thr.start()
@ -36,18 +47,100 @@ class ViewerListener(Viewer):
feiFei.set_sleep(not is_live_started)
pass
#录制麦克风音频输入并传给aliyun
class RecorderListener(Recorder):
def __init__(self, device, fei):
super().__init__(device, fei)
self.__device = device
self.__RATE = 16000
self.__FORMAT = pyaudio.paInt16
self.__CHANNELS = 1
super().__init__(fei)
def on_speaking(self, text):
interact = (1, '', text)
util.printInfo(3, "语音", '{}'.format(interact[2]), time.time())
if len(text) > 1:
interact = Interact("mic", 1, {'user': '', 'msg': text})
util.printInfo(3, "语音", '{}'.format(interact.data["msg"]), time.time())
feiFei.on_interact(interact)
time.sleep(2)
def get_stream(self):
self.paudio = pyaudio.PyAudio()
device_id = self.__findInternalRecordingDevice(self.paudio)
if device_id < 0:
return
self.stream = self.paudio.open(input_device_index=device_id, rate=self.__RATE, format=self.__FORMAT, channels=self.__CHANNELS, input=True)
return self.stream
def __findInternalRecordingDevice(self, p):
for i in range(p.get_device_count()):
devInfo = p.get_device_info_by_index(i)
if devInfo['name'].find(self.__device) >= 0 and devInfo['hostApi'] == 0:
return i
util.log(1, '[!] 无法找到内录设备!')
return -1
def stop(self):
super().stop()
self.stream.stop_stream()
self.stream.close()
self.paudio.terminate()
#TODO Edit by xszyou on 20230113:录制远程设备音频输入并传给aliyun
class DeviceInputListener(Recorder):
def __init__(self, fei):
super().__init__(fei)
self.__running = True
self.ngrok = None
self.thread = MyThread(target=self.run)
self.thread.start() #启动远程音频输入设备监听线程
def run(self):
#启动ngork
if config_util.key_ngrok_cc_id is not None:
MyThread(target=self.start_ngrok, args=[config_util.key_ngrok_cc_id]).start()
self.streamCache = stream_util.StreamCache(1024*1024*20)
addr = None
while self.__running:
try:
data = b""
while feiFei.deviceConnect:
data = feiFei.deviceConnect.recv(1024)
self.streamCache.write(data)
time.sleep(0.005)
self.streamCache.clear()
except Exception as err:
pass
time.sleep(1)
def on_speaking(self, text):
if len(text) > 1:
interact = Interact("mic", 1, {'user': '', 'msg': text})
util.printInfo(3, "语音", '{}'.format(interact.data["msg"]), time.time())
feiFei.on_interact(interact)
time.sleep(2)
def get_stream(self):
while feiFei.deviceConnect is None:
pass
return self.streamCache
def stop(self):
super().stop()
self.__running = False
self.ngrok.stop()
def start_ngrok(self, clientId):
self.ngrok = ngrok_util.NgrokCilent(clientId)
self.ngrok.start()
def console_listener():
type_names = {
@ -93,30 +186,32 @@ def console_listener():
util.printInfo(1, type_names[i], '{}: {}'.format('控制台', msg))
if i == 1:
feiFei.last_quest_time = time.time()
thr = MyThread(target=feiFei.on_interact, args=[(i, '', msg)])
thr = MyThread(target=feiFei.on_interact, args=[("console", i, '', msg)])
thr.start()
thr.join()
else:
util.log(1, '未知命令!使用 \'help\' 获取帮助.')
#停止服务
def stop():
global feiFei
global viewerListener
global recorderListener
global __running
global deviceInputListener
util.log(1, '正在关闭服务...')
__running = False
# util.log('正在关闭通讯服务...')
# wsa_server.get_instance().stop_server()
if viewerListener is not None:
util.log(1, '正在关闭直播服务...')
viewerListener.stop()
if recorderListener is not None:
util.log(1, '正在关闭录音服务...')
recorderListener.stop()
if deviceInputListener is not None:
util.log(1, '正在关闭远程音频输入输出服务...')
deviceInputListener.stop()
util.log(1, '正在关闭核心服务...')
feiFei.stop()
util.log(1, '服务已关闭!')
@ -128,15 +223,15 @@ def start():
global viewerListener
global recorderListener
global __running
global deviceInputListener
util.log(1, '开启服务...')
__running = True
util.log(1, '读取配置...')
config_util.load_config()
#
# util.log('开启通讯服务...')
# ws_server = MyServer()
# ws_server.start_server()
util.log(1, '开启核心服务...')
feiFei = FeiFei()
@ -145,6 +240,8 @@ def start():
liveRoom = config_util.config['source']['liveRoom']
record = config_util.config['source']['record']
if liveRoom['enabled']:
util.log(1, '开启直播服务...')
viewerListener = ViewerListener(liveRoom['url']) # 监听直播间
@ -154,12 +251,19 @@ def start():
util.log(1, '开启录音服务...')
recorderListener = RecorderListener(record['device'], feiFei) # 监听麦克风
recorderListener.start()
# mac下启动经常获取了不明内容导致关闭再开启时等待输入
# util.log(1, '注册命令...')
# MyThread(target=console_listener).start() # 监听控制台
#TODO edit by xszyou on 20230113:通过此服务来连接k210、手机等音频输入设备
util.log(1,'开启远程设备音频输入服务...')
deviceInputListener = DeviceInputListener(feiFei) # 设备音频输入输出麦克风
deviceInputListener.start()
util.log(1, '注册命令...')
MyThread(target=console_listener).start() # 监听控制台
util.log(1, '完成!')
# util.log(1, '使用 \'help\' 获取帮助.')
util.log(1, '使用 \'help\' 获取帮助.')
# if __name__ == '__main__':
# ws_server: MyServer = None

View File

@ -26,7 +26,8 @@ def __get_device_list():
devInfo = audio.get_device_info_by_index(i)
if devInfo['hostApi'] == 0:
device_list.append(devInfo["name"])
return device_list
return list(set(device_list))
@__app.route('/api/submit', methods=['post'])

File diff suppressed because one or more lines are too long

13619
gui/static/js/element.js Normal file

File diff suppressed because it is too large Load Diff

12014
gui/static/js/vue.js Normal file

File diff suppressed because it is too large Load Diff

View File

@ -8,8 +8,10 @@
<!-- index.css -->
<link rel="stylesheet" href="{{ url_for('static',filename='css/index.css') }}"></link>
<!-- <link rel="stylesheet" href="./css/index.css"> -->
<!-- 引入element-ui样式 -->
<!-- <link rel="stylesheet" href="{{ url_for('static',filename='css/element.css') }}"></link> -->
<!-- <link rel="stylesheet" href="./css/element.css"> -->
<link rel="stylesheet" href="{{ url_for('static',filename='css/element.css') }}"></link>
<link rel="stylesheet" href="https://unpkg.com/element-ui/lib/theme-chalk/index.css">
<title>自动商品介绍控制器</title>
@ -20,7 +22,7 @@
<div id="app">
<div class="main">
<div class="title">
<h2>数字人控制器</h2>
<h2>Fay控制器2.0</h2>
</div>
<div class="main_box">
<div class="left">
@ -228,13 +230,16 @@
</div>
</body>
<!-- 开发环境vue.js -->
<script src="https://cdn.jsdelivr.net/npm/vue@2/dist/vue.js"></script>
<script src="{{ url_for('static',filename='js/vue.js') }}"></script>
<!--<script src="https://cdn.jsdelivr.net/npm/vue@2/dist/vue.js"></script>-->
<!-- 发行环境vue.js -->
<!-- <script src="https://cdn.jsdelivr.net/npm/vue@2"></script> -->
<!-- 引入element-ui组件库 -->
<script src="https://unpkg.com/element-ui/lib/index.js"></script>
<!-- <script src="{{ url_for('static',filename='js/element.js') }}"></script> -->
<!-- <script src="./js/element.js"></script> -->
<script src="{{ url_for('static',filename='js/element.js') }}"></script>
<!-- index.js -->
<!-- <script src="./js/index.js"></script> -->
<!-- <script src="./js/self-adaption.js"></script> -->
<script src="{{ url_for('static',filename='js/index.js') }}"></script>
<script src="{{ url_for('static',filename='js/self-adaption.js') }}"></script>

Some files were not shown because too many files have changed in this diff Show More