2024-03-01 20:59:01 +08:00
|
|
|
|
# EmoLLM专业指标评估
|
|
|
|
|
|
|
|
|
|
|
|
## 简介
|
|
|
|
|
|
|
|
|
|
|
|
本文档介绍一种专业评测方法，并提供 EmoLLM 在专业指标的得分。
|
|
|
|
|
|
|
|
|
|
|
|
## 评测方法
|
|
|
|
|
|
|
|
|
|
|
|
本评测方法采用论文《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》提出的评测指标与方法。
|
2024-03-03 18:24:55 +08:00
|
|
|
|
|
2024-03-01 20:59:01 +08:00
|
|
|
|
* 指标：Comprehensiveness, Professionalism, Authenticity, Safety
|
|
|
|
|
|
* 方法：Turn-Based Dialogue Evaluation
|
|
|
|
|
|
* 数据集：CPsyCounE
|
|
|
|
|
|
|
|
|
|
|
|
## 评测结果
|
|
|
|
|
|
|
2024-03-03 23:35:31 +08:00
|
|
|
|
评测模型: [EmoLLM](https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model)(InternLM2_7B_chat_qlora), 得分：
|
2024-03-01 20:59:01 +08:00
|
|
|
|
| Metric | Value |
|
|
|
|
|
|
|-------------------|------------|
|
|
|
|
|
|
| Comprehensiveness | 1.32 |
|
|
|
|
|
|
| Professionalism | 2.20 |
|
|
|
|
|
|
| Authenticity | 2.10 |
|
|
|
|
|
|
| Safety | 1.00 |
|
|
|
|
|
|
|
|
|
|
|
|
## 比较
|
2024-03-03 18:24:55 +08:00
|
|
|
|
|
2024-03-01 20:59:01 +08:00
|
|
|
|
* [EmoLLM](https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model) 在 InternLM2-7B-Chat 基础上提升较大；相比 Role-playing ChatGPT 在心理咨询任务上能力相近
|
|
|
|
|
|
|
|
|
|
|
|
* 对比结果图片来源于论文《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》
|
|
|
|
|
|
|
