bakaEC/OliveSensorAPI
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Create Professional_evaluation_EN.md

Browse Source
This commit is contained in:
MING_X 2024-03-06 17:29:52 +08:00 committed by GitHub
parent 8202e4a6f4
commit 81823c0999
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 32 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

32
evaluate/Professional_evaluation_EN.md Normal file
View File

@ -0,0 +1,32 @@
# EmoLLM's professional evaluation
## Introduction
This document describes a professional evaluation method and provides EmoLLM's scores on professional metrics.
## Evaluation
The evaluation method, metric, and dataset from the paper《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》.
* Metric: Comprehensiveness, Professionalism, Authenticity, Safety
* Method: Turn-Based Dialogue Evaluation
* Dataset: CPsyCounE
## Result
* Model: [EmoLLM V1.0](https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model)(InternLM2_7B_chat_qlora)
* Score：
| Metric | Value |
|-------------------|------------|
| Comprehensiveness | 1.32 |
| Professionalism | 2.20 |
| Authenticity | 2.10 |
| Safety | 1.00 |
## Comparison
* [EmoLLM V1.0](https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model) is greatly improved on InternLM2_7B_Chat; Performance on the counseling task was similar compared to ChatGPT(Role-playing)
* The comparison results are from the paper《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》
![image](https://github.com/MING-ZCH/EmoLLM/assets/119648793/abc9f626-11bc-4ec8-84a4-427c4600a720)