2024-03-06 17:29:52 +08:00
# EmoLLM's professional evaluation
## Introduction
This document describes a professional evaluation method and provides EmoLLM's scores on professional metrics.
## Evaluation
2024-05-28 17:04:36 +08:00
The evaluation method, metric, and dataset from the paper[《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》](https://arxiv.org/abs/2405.16433).
2024-03-06 17:29:52 +08:00
* Metric: Comprehensiveness, Professionalism, Authenticity, Safety
* Method: Turn-Based Dialogue Evaluation
* Dataset: CPsyCounE
## Result
2024-03-11 18:50:53 +08:00
* Model:
* [EmoLLM V1.0 ](https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model ) (InternLM2_7B_chat_qlora)
* [EmoLLM V2.0 ](https://openxlab.org.cn/apps/detail/Farewell1/EmoLLMV2.0 ) (InternLM2_7B_chat_full)
2024-03-06 17:29:52 +08:00
* Score：
2024-03-11 18:50:53 +08:00
| Model | Comprehensiveness | Professionalism | Authenticity | Safety |
|-------------------|-----------------------|-------------------|-----------------|---------|
| InternLM2_7B_chat_qlora | 1.32 | 2.20 | 2.10 | 1.00 |
| InternLM2_7B_chat_full | 1.40 | 2.45 | 2.24 | 1.00 |
2024-03-06 17:29:52 +08:00
## Comparison
2024-03-11 18:50:53 +08:00
* EmoLLM V2.0 is greatly improved in all scores compared to EmoLLM V1.0! Surpasses the performance of Role-playing ChatGPT on counseling tasks!
* EmoLLM V1.0 is greatly improved on InternLM2_7B_Chat; Performance on the counseling task was similar compared to ChatGPT(Role-playing)
2024-03-06 17:29:52 +08:00
* The comparison results are from the paper《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》
