diff --git a/evaluate/Professional_evaluation_EN.md b/evaluate/Professional_evaluation_EN.md new file mode 100644 index 0000000..7d46cc1 --- /dev/null +++ b/evaluate/Professional_evaluation_EN.md @@ -0,0 +1,32 @@ +# EmoLLM's professional evaluation + +## Introduction + +This document describes a professional evaluation method and provides EmoLLM's scores on professional metrics. + +## Evaluation + +The evaluation method, metric, and dataset from the paper《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》. + +* Metric: Comprehensiveness, Professionalism, Authenticity, Safety +* Method: Turn-Based Dialogue Evaluation +* Dataset: CPsyCounE + +## Result + +* Model: [EmoLLM V1.0](https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model)(InternLM2_7B_chat_qlora) +* Score: + +| Metric | Value | +|-------------------|------------| +| Comprehensiveness | 1.32 | +| Professionalism | 2.20 | +| Authenticity | 2.10 | +| Safety | 1.00 | + +## Comparison + +* [EmoLLM V1.0](https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model) is greatly improved on InternLM2_7B_Chat; Performance on the counseling task was similar compared to ChatGPT(Role-playing) + +* The comparison results are from the paper《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》 +![image](https://github.com/MING-ZCH/EmoLLM/assets/119648793/abc9f626-11bc-4ec8-84a4-427c4600a720)