OliveSensorAPI/evaluate/Professional_evaluation.md

# EmoLLM专业指标评估

## 简介

本文档介绍一种专业评测方法，并提供 EmoLLM 在专业指标的得分。

## 评测方法

本评测方法采用论文《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》提出的评测指标与方法。

* 指标：Comprehensiveness, Professionalism, Authenticity, Safety
* 方法：Turn-Based Dialogue Evaluation
* 数据集：CPsyCounE

## 评测结果

* 评测模型:
  * [EmoLLM V1.0](https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model) (InternLM2_7B_chat_qlora)
  * [EmoLLM V2.0](https://openxlab.org.cn/apps/detail/Farewell1/EmoLLMV2.0) (InternLM2_7B_chat_full)
   
* 得分：

|       Model       |    Comprehensiveness  |   Professionalism  |  Authenticity   | Safety  |
|-------------------|-----------------------|-------------------|-----------------|---------|
| InternLM2_7B_chat_qlora |      1.32       |        2.20       |      2.10       | 1.00    |
| InternLM2_7B_chat_full  |      1.40       |        2.45       |      2.24       | 1.00    |

## 比较

* EmoLLM V2.0 相比 EmoLLM V1.0 在指标上全面提升！已超越 Role-playing ChatGPT 在心理咨询任务上的能力！
* EmoLLM V1.0 在 InternLM2_7B_Chat 基础上提升较大；相比 Role-playing ChatGPT 在心理咨询任务上能力相近

* 对比结果图片来源于论文《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》
![image](https://github.com/MING-ZCH/EmoLLM/assets/119648793/abc9f626-11bc-4ec8-84a4-427c4600a720)
-												Create Professional evaluation.md
											
										
										
											2024-03-01 20:59:01 +08:00
+								# EmoLLM专业指标评估
 								## 简介
 								本文档介绍一种专业评测方法，并提供 EmoLLM 在专业指标的得分。
 								## 评测方法
 								本评测方法采用论文《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》提出的评测指标与方法。
-												README files translation

The English version README files of the following documents are created and translated.

1. demo/README.md
2. evaluate/README.md
3. xtuner_config/README.md
4. xtuner_config/images/README.md
5. xtuner_config/ChatGLM3-6b-ft.md

There are some format problem and  language expression in the Chinese version, I also adapted them.

By the way, I modified the file name of `evaluate/General evaluation.md` and `evaluate/Professional evaluation.md` since they are shown in the `xtuner_config/README.md`

											
										
										
											2024-03-03 18:24:55 +08:00
-												Create Professional evaluation.md
											
										
										
											2024-03-01 20:59:01 +08:00
+								* 指标：Comprehensiveness, Professionalism, Authenticity, Safety
 								* 方法：Turn-Based Dialogue Evaluation
 								* 数据集：CPsyCounE
 								## 评测结果
-												Merge Main code (#2)

* Add files via upload

* 新增ENmd文档

* Update README.md

* Update README_EN.md

* Update LICENSE

* [docs] update lmdeploy file

* add ocr.md

* Update tutorial.md

* Update tutorial_EN.md

* Update General_evaluation_EN.md

* Update General_evaluation_EN.md

* Update README.md

Add InternLM2_7B_chat_full's professional evaluation results

* Update Professional_evaluation.md

* Update Professional_evaluation.md

* Update Professional_evaluation.md

* Update Professional_evaluation.md

* Update Professional_evaluation_EN.md

* Update README.md

* Update README.md

* Update README_EN.md

* Update README_EN.md

* Update README_EN.md

* [DOC] update readme

* Update LICENSE

* Update LICENSE

* update personal info and small format optimizations

* update personal info and translations for contents in a table

* Update RAG README

* Update demo link in README.md

* Update xlab app link

* Update xlab link

* add xlab model

* Update web_demo-aiwei.py

* add bitex

---------

Co-authored-by: xzw <62385492+aJupyter@users.noreply.github.com>
Co-authored-by: এ許我辞忧࿐♡ <127636623+Smiling-Weeping-zhr@users.noreply.github.com>
Co-authored-by: Vicky <vicky_3021@163.com>
Co-authored-by: MING_X <119648793+MING-ZCH@users.noreply.github.com>
Co-authored-by: Nobody-ML <1755309985@qq.com>
Co-authored-by: 8baby8 <3345710651@qq.com>
Co-authored-by: chaoke <101492509+8baby8@users.noreply.github.com>
Co-authored-by: aJupyter <ajupyter@163.com>
Co-authored-by: HongCheng <kwchenghong@gmail.com>
Co-authored-by: santiagoTOP <“1537211712top@gmail.com”>
											
										
										
											2024-03-15 19:51:04 +08:00
+								* 评测模型:
 								  * [EmoLLM V1.0](https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model) (InternLM2_7B_chat_qlora)
 								  * [EmoLLM V2.0](https://openxlab.org.cn/apps/detail/Farewell1/EmoLLMV2.0) (InternLM2_7B_chat_full)
-												Update Professional_evaluation.md
											
										
										
											2024-03-06 17:30:48 +08:00
+								* 得分：
-												Merge Main code (#2)

* Add files via upload

* 新增ENmd文档

* Update README.md

* Update README_EN.md

* Update LICENSE

* [docs] update lmdeploy file

* add ocr.md

* Update tutorial.md

* Update tutorial_EN.md

* Update General_evaluation_EN.md

* Update General_evaluation_EN.md

* Update README.md

Add InternLM2_7B_chat_full's professional evaluation results

* Update Professional_evaluation.md

* Update Professional_evaluation.md

* Update Professional_evaluation.md

* Update Professional_evaluation.md

* Update Professional_evaluation_EN.md

* Update README.md

* Update README.md

* Update README_EN.md

* Update README_EN.md

* Update README_EN.md

* [DOC] update readme

* Update LICENSE

* Update LICENSE

* update personal info and small format optimizations

* update personal info and translations for contents in a table

* Update RAG README

* Update demo link in README.md

* Update xlab app link

* Update xlab link

* add xlab model

* Update web_demo-aiwei.py

* add bitex

---------

Co-authored-by: xzw <62385492+aJupyter@users.noreply.github.com>
Co-authored-by: এ許我辞忧࿐♡ <127636623+Smiling-Weeping-zhr@users.noreply.github.com>
Co-authored-by: Vicky <vicky_3021@163.com>
Co-authored-by: MING_X <119648793+MING-ZCH@users.noreply.github.com>
Co-authored-by: Nobody-ML <1755309985@qq.com>
Co-authored-by: 8baby8 <3345710651@qq.com>
Co-authored-by: chaoke <101492509+8baby8@users.noreply.github.com>
Co-authored-by: aJupyter <ajupyter@163.com>
Co-authored-by: HongCheng <kwchenghong@gmail.com>
Co-authored-by: santiagoTOP <“1537211712top@gmail.com”>
											
										
										
											2024-03-15 19:51:04 +08:00
+								|       Model       |    Comprehensiveness  |   Professionalism  |  Authenticity   | Safety  |
 								|-------------------|-----------------------|-------------------|-----------------|---------|
 								| InternLM2_7B_chat_qlora |      1.32       |        2.20       |      2.10       | 1.00    |
 								| InternLM2_7B_chat_full  |      1.40       |        2.45       |      2.24       | 1.00    |
-												Create Professional evaluation.md
											
										
										
											2024-03-01 20:59:01 +08:00
 								## 比较
-												README files translation

The English version README files of the following documents are created and translated.

1. demo/README.md
2. evaluate/README.md
3. xtuner_config/README.md
4. xtuner_config/images/README.md
5. xtuner_config/ChatGLM3-6b-ft.md

There are some format problem and  language expression in the Chinese version, I also adapted them.

By the way, I modified the file name of `evaluate/General evaluation.md` and `evaluate/Professional evaluation.md` since they are shown in the `xtuner_config/README.md`

											
										
										
											2024-03-03 18:24:55 +08:00
-												Merge Main code (#2)

* Add files via upload

* 新增ENmd文档

* Update README.md

* Update README_EN.md

* Update LICENSE

* [docs] update lmdeploy file

* add ocr.md

* Update tutorial.md

* Update tutorial_EN.md

* Update General_evaluation_EN.md

* Update General_evaluation_EN.md

* Update README.md

Add InternLM2_7B_chat_full's professional evaluation results

* Update Professional_evaluation.md

* Update Professional_evaluation.md

* Update Professional_evaluation.md

* Update Professional_evaluation.md

* Update Professional_evaluation_EN.md

* Update README.md

* Update README.md

* Update README_EN.md

* Update README_EN.md

* Update README_EN.md

* [DOC] update readme

* Update LICENSE

* Update LICENSE

* update personal info and small format optimizations

* update personal info and translations for contents in a table

* Update RAG README

* Update demo link in README.md

* Update xlab app link

* Update xlab link

* add xlab model

* Update web_demo-aiwei.py

* add bitex

---------

Co-authored-by: xzw <62385492+aJupyter@users.noreply.github.com>
Co-authored-by: এ許我辞忧࿐♡ <127636623+Smiling-Weeping-zhr@users.noreply.github.com>
Co-authored-by: Vicky <vicky_3021@163.com>
Co-authored-by: MING_X <119648793+MING-ZCH@users.noreply.github.com>
Co-authored-by: Nobody-ML <1755309985@qq.com>
Co-authored-by: 8baby8 <3345710651@qq.com>
Co-authored-by: chaoke <101492509+8baby8@users.noreply.github.com>
Co-authored-by: aJupyter <ajupyter@163.com>
Co-authored-by: HongCheng <kwchenghong@gmail.com>
Co-authored-by: santiagoTOP <“1537211712top@gmail.com”>
											
										
										
											2024-03-15 19:51:04 +08:00
+								* EmoLLM V2.0 相比 EmoLLM V1.0 在指标上全面提升！已超越 Role-playing ChatGPT 在心理咨询任务上的能力！
 								* EmoLLM V1.0 在 InternLM2_7B_Chat 基础上提升较大；相比 Role-playing ChatGPT 在心理咨询任务上能力相近
-												Create Professional evaluation.md
											
										
										
											2024-03-01 20:59:01 +08:00
 								* 对比结果图片来源于论文《CPsyCoun: A Report-based Multi-turn Dialogue Reconstruction and Evaluation Framework for Chinese Psychological Counseling》
 								![image](https://github.com/MING-ZCH/EmoLLM/assets/119648793/abc9f626-11bc-4ec8-84a4-427c4600a720)