bakaEC/OliveSensorAPI
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Create Professional evaluation.md

Browse Source
This commit is contained in:
MING_X 2024-03-01 20:59:01 +08:00 committed by GitHub
parent e32d9a11c2
commit aea78d5ef9
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 28 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
evaluate/Professional evaluation.md Normal file
View File

@ -0,0 +1,28 @@
# EmoLLM专业指标评估
## 简介
本文档介绍一种专业评测方法，并提供 EmoLLM 在专业指标的得分。
## 评测方法
本评测方法采用论文《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》提出的评测指标与方法。
* 指标：Comprehensiveness, Professionalism, Authenticity, Safety
* 方法：Turn-Based Dialogue Evaluation
* 数据集：CPsyCounE
## 评测结果
评测模型: [EmoLLM](https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model)(InternLM2-7B-chat + qlora), 得分：
| Metric | Value |
|-------------------|------------|
| Comprehensiveness | 1.32 |
| Professionalism | 2.20 |
| Authenticity | 2.10 |
| Safety | 1.00 |
## 比较
* [EmoLLM](https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model) 在 InternLM2-7B-Chat 基础上提升较大；相比 Role-playing ChatGPT 在心理咨询任务上能力相近
* 对比结果图片来源于论文《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》
![image](https://github.com/MING-ZCH/EmoLLM/assets/119648793/abc9f626-11bc-4ec8-84a4-427c4600a720)