# EmoLLM's professional evaluation

## Introduction

This document describes a professional evaluation method and provides EmoLLM's scores on professional metrics.

## Evaluation

The evaluation method, metric, and dataset from the paper《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》.

* Metric: Comprehensiveness, Professionalism, Authenticity, Safety
* Method: Turn-Based Dialogue Evaluation
* Dataset: CPsyCounE

## Result

* Model: [EmoLLM V1.0](https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model)(InternLM2_7B_chat_qlora)
* Score：

|       Metric      |    Value   |
|-------------------|------------|
| Comprehensiveness | 1.32       |
| Professionalism   | 2.20       |
| Authenticity      | 2.10       |
| Safety            | 1.00       |

## Comparison

* [EmoLLM V1.0](https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model) is greatly improved on InternLM2_7B_Chat; Performance on the counseling task was similar compared to ChatGPT(Role-playing)

* The comparison results are from the paper《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》
![image](https://github.com/MING-ZCH/EmoLLM/assets/119648793/abc9f626-11bc-4ec8-84a4-427c4600a720)