Update code (#8)
* feat: add agents/actions/write_markdown * [ADD] add evaluation result of base model on 5/10 epochs * Rename mother.json to mother_v1_2439.json * Add files via upload * [DOC] update README * Update requirements.txt update mpi4py installation * Update README_EN.md update English comma * Update README.md 基于母亲角色的多轮对话模型微调完毕。已上传到 Huggingface。 * 多轮对话母亲角色的微调的脚本 * Update README.md 加上了王几行XING 和 思在 的作者信息 * Update README_EN.md * Update README.md * Update README_EN.md * Update README_EN.md * Changes to be committed: modified: .gitignore modified: README.md modified: README_EN.md new file: assets/EmoLLM_transparent.png deleted: assets/Shusheng.jpg new file: assets/Shusheng.png new file: assets/aiwei_demo1.gif new file: assets/aiwei_demo2.gif new file: assets/aiwei_demo3.gif new file: assets/aiwei_demo4.gif * Update README.md rectify aiwei_demo.gif * Update README.md rectify aiwei_demo style * Changes to be committed: modified: README.md modified: README_EN.md * Changes to be committed: modified: README.md modified: README_EN.md * [Doc] update readme * [Doc] update readme * Update README.md * Update README_EN.md * Update README.md * Update README_EN.md * Delete datasets/mother_v1_2439.json * Rename mother_v2_3838.json to mother_v2.json * Delete datasets/mother_v2.json * Add files via upload * Update README.md * Update README_EN.md * [Doc] Update README_EN.md minor fix * InternLM2-Base-7B QLoRA微调模型 链接和测评结果更新 * add download_model.py script, automatic download of model libraries * 清除图片的黑边、更新作者信息 modified: README.md new file: assets/aiwei_demo.gif deleted: assets/aiwei_demo1.gif modified: assets/aiwei_demo2.gif modified: assets/aiwei_demo3.gif modified: assets/aiwei_demo4.gif * rectify aiwei_demo transparent * transparent * modify: aiwei_demo table--->div * modified: aiwei_demo * modify: div ---> table * modified: README.md * modified: README_EN.md * update model config file links * Create internlm2_20b_chat_lora_alpaca_e3.py 20b模型的配置文件 * update model config file links update model config file links * Revert "update model config file links" --------- Co-authored-by: jujimeizuo <fengzetao.zed@foxmail.com> Co-authored-by: xzw <62385492+aJupyter@users.noreply.github.com> Co-authored-by: Zeyu Ba <72795264+ZeyuBa@users.noreply.github.com> Co-authored-by: Bryce Wang <90940753+brycewang2018@users.noreply.github.com> Co-authored-by: zealot52099 <songyan5209@163.com> Co-authored-by: HongCheng <kwchenghong@gmail.com> Co-authored-by: Yicong <yicooong@qq.com> Co-authored-by: Yicooong <54353406+Yicooong@users.noreply.github.com> Co-authored-by: aJupyter <ajupyter@163.com> Co-authored-by: MING_X <119648793+MING-ZCH@users.noreply.github.com> Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com> Co-authored-by: HatBoy <null2none@163.com> Co-authored-by: ZhouXinAo <142309012+zxazys@users.noreply.github.com>
This commit is contained in:
Anooyman
GitHub
parent
1f7ef9cac0
commit
14890fad56
1
.gitignore
vendored
1
.gitignore
vendored
@ -10,6 +10,7 @@ logs/
|
||||
*.jsonl
|
||||
*.json
|
||||
*.txt
|
||||
localFile/
|
||||
# ./generate_data/*.josnl
|
||||
# ./generate_data/*/*/*.josnl
|
||||
|
||||
|
||||
663
README.md
663
README.md
@ -1,316 +1,347 @@
|
||||
<div align="center">
|
||||
|
||||
# EmoLLM-心理健康大模型
|
||||
|
||||
</div>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/">
|
||||
<img src="assets/logo.jpeg" alt="Logo" width="30%">
|
||||
</a>
|
||||
|
||||
<div align="center">
|
||||
|
||||
<!-- PROJECT SHIELDS -->
|
||||
[![Contributors][contributors-shield]][contributors-url]
|
||||
[![Forks][forks-shield]][forks-url]
|
||||
[![Issues][issues-shield]][issues-url]
|
||||
[![OpenXLab_App][OpenXLab_App-image]][OpenXLab_App-url]
|
||||
[![OpenXLab_Model][OpenXLab_Model-image]][OpenXLab_Model-url]
|
||||
[![MIT License][license-shield]][license-url]
|
||||
[![Stargazers][stars-shield]][stars-url]
|
||||
|
||||
</div>
|
||||
|
||||
<h3 align="center">EmoLLM</h3>
|
||||
|
||||
<div align="center">
|
||||
简体中文| <a href="README_EN.md" >English</a>
|
||||
<br />
|
||||
<br />
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM"><strong>探索本项目的文档 »</strong></a>
|
||||
<br />
|
||||
<br />
|
||||
<a href="https://openxlab.org.cn/apps/detail/Farewell1/EmoLLMV2.0">体验EmoLLM 2.0</a>
|
||||
·
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/issues">报告Bug</a>
|
||||
·
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/issues">提出新特性</a>
|
||||
</div>
|
||||
|
||||
<!-- 本篇README.md面向开发者 -->
|
||||
|
||||
**EmoLLM** 是一系列能够支持 **理解用户-支持用户-帮助用户** 心理健康辅导链路的心理健康大模型，由 `LLM`指令微调而来，欢迎大家star~⭐⭐。目前已经开源的 `LLM` 微调配置如下：
|
||||
|
||||
<div align="center">
|
||||
|
||||
| 模型 | 类型 |
|
||||
| :-------------------: | :--------: |
|
||||
| InternLM2_7B_chat | QLORA |
|
||||
| InternLM2_7B_chat | 全量微调 |
|
||||
| InternLM2_7B_base | QLORA |
|
||||
| InternLM2_1_8B_chat | 全量微调 |
|
||||
| InternLM2_20B_chat | LORA |
|
||||
| Qwen_7b_chat | QLORA |
|
||||
| Qwen1_5-0_5B-Chat | 全量微调 |
|
||||
| Baichuan2_13B_chat | QLORA |
|
||||
| ChatGLM3_6B | LORA |
|
||||
| DeepSeek MoE_16B_chat | QLORA |
|
||||
| Mixtral 8x7B_instruct | QLORA |
|
||||
| …… | …… |
|
||||
|
||||
</div>
|
||||
|
||||
欢迎大家为本项目做出贡献~
|
||||
|
||||
---
|
||||
|
||||
心理健康大模型（Mental Health Grand Model）是一个综合性的概念，它旨在全面理解和促进个体、群体乃至整个社会的心理健康状态。这个模型通常包含以下几个关键组成部分：
|
||||
|
||||
- 认知因素：涉及个体的思维模式、信念系统、认知偏差以及解决问题的能力。认知因素对心理健康有重要影响，因为它们影响个体如何解释和应对生活中的事件。
|
||||
- 情感因素：包括情绪调节、情感表达和情感体验。情感健康是心理健康的重要组成部分，涉及个体如何管理和表达自己的情感，以及如何从负面情绪中恢复。
|
||||
- 行为因素：涉及个体的行为模式、习惯和应对策略。这包括应对压力的技巧、社交技能以及自我效能感，即个体对自己能力的信心。
|
||||
- 社会环境：包括家庭、工作、社区和文化背景等外部因素，这些因素对个体的心理健康有着直接和间接的影响。
|
||||
- 生理健康：身体健康与心理健康紧密相关。良好的身体健康可以促进心理健康，反之亦然。
|
||||
- 心理韧性：指个体在面对逆境时的恢复力和适应能力。心理韧性强的人更能够从挑战中恢复，并从中学习和成长。
|
||||
- 预防和干预措施：心理健康大模型还包括预防心理问题和促进心理健康的策略，如心理教育、心理咨询、心理治疗和社会支持系统。
|
||||
- 评估和诊断工具：为了有效促进心理健康，需要有科学的工具来评估个体的心理状态，以及诊断可能存在的心理问题。
|
||||
|
||||
### 🎇最近更新
|
||||
|
||||
- 【2024.3.25】在百度飞桨平台发布[爹系男友心理咨询师](https://aistudio.baidu.com/community/app/68787)
|
||||
- 【2024.3.24】在OpenXLab和ModelScope平台发布InternLM2-Base-7B QLoRA微调模型, 具体请查看[InternLM2-Base-7B QLoRA](./xtuner_config/README_internlm2_7b_base_qlora.md)
|
||||
- 【2024.3.12】在百度飞桨平台发布[艾薇](https://aistudio.baidu.com/community/app/63335)
|
||||
- 【2024.3.11】 **EmoLLM V2.0 相比 EmoLLM V1.0 全面提升，已超越 Role-playing ChatGPT 在心理咨询任务上的能力！**[点击体验EmoLLM V2.0](https://openxlab.org.cn/apps/detail/Farewell1/EmoLLMV2.0)，更新[数据集统计及详细信息](./datasets/)、[路线图](./assets/Roadmap_ZH.png)
|
||||
- 【2024.3.9】 新增并发功能加速 [QA 对生成](./scripts/qa_generation/)、[RAG pipeline](./rag/)
|
||||
- 【2024.3.3】 [基于InternLM2-7B-chat全量微调版本EmoLLM V2.0开源](https://openxlab.org.cn/models/detail/ajupyter/EmoLLM_internlm2_7b_full)，需要两块A100*80G，更新专业评估，详见[evaluate](./evaluate/)，更新基于PaddleOCR的PDF转txt工具脚本，详见[scripts](./scripts/)
|
||||
- 【2024.2.29】更新客观评估计算，详见[evaluate](./evaluate/)，更新一系列数据集，详见[datasets](./datasets/)
|
||||
- 【2024.2.27】更新英文readme和一系列数据集（舔狗和单轮对话）
|
||||
- 【2024.2.23】推出基于InternLM2_7B_chat_qlora的 `温柔御姐心理医生艾薇`，[点击获取模型权重](https://openxlab.org.cn/models/detail/ajupyter/EmoLLM_aiwei)，[配置文件](xtuner_config/aiwei-internlm2_chat_7b_qlora.py)，[在线体验链接](https://openxlab.org.cn/apps/detail/ajupyter/EmoLLM-aiwei)
|
||||
- 【2024.2.23】更新[若干微调配置](/xtuner_config/)，新增 [data_pro.json](/datasets/data_pro.json)（数量更多、场景更全、更丰富）和 [aiwei.json](/datasets/aiwei.json)（温柔御姐角色扮演专用，带有Emoji表情），即将推出 `温柔御姐心理医生艾薇`
|
||||
- 【2024.2.18】 [基于Qwen1_5-0_5B-Chat全量微调版本开源](https://www.modelscope.cn/models/aJupyter/EmoLLM_Qwen1_5-0_5B-Chat_full_sft/summary)，算力有限的道友可以玩起来~
|
||||
|
||||
<details>
|
||||
<summary>查看更多</summary>
|
||||
|
||||
- 【2024.2.6】 EmoLLM在[**Openxlab** ](https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model) 平台下载量高达18.7k，欢迎大家体验！
|
||||
|
||||
<p align="center">
|
||||
<img src="https://github.com/SmartFlowAI/EmoLLM/assets/62385492/7e931682-c54d-4ded-bc67-79130c68d744" alt="模型下载量">
|
||||
</p>
|
||||
|
||||
- 【2024.2.5】 项目荣获公众号**NLP工程化**推文宣传[推文链接](https://mp.weixin.qq.com/s/78lrRl2tlXEKUfElnkVx4A)，为博主推广一波，欢迎大家关注！！🥳🥳
|
||||
|
||||
<p align="center">
|
||||
<img src="https://github.com/SmartFlowAI/EmoLLM/assets/62385492/47868d6a-2e91-4aa9-a630-e594c14295b4" alt="公众号二维码">
|
||||
</p>
|
||||
|
||||
- 【2024.2.3】 [项目宣传视频](https://www.bilibili.com/video/BV1N7421N76X/)完成 😊
|
||||
- 【2024.1.27】 完善数据构建文档、微调指南、部署指南、Readme等相关文档 👏
|
||||
- 【2024.1.25】 EmoLLM V1.0 已部署上线 https://openxlab.org.cn/apps/detail/jujimeizuo/EmoLLM 😀
|
||||
|
||||
</details>
|
||||
|
||||
### 🏆荣誉栏
|
||||
|
||||
- 项目荣获上海人工智能实验室举办的**2024浦源大模型系列挑战赛春季赛*****创新创意奖***
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/">
|
||||
<img src="assets/Shusheng.jpg" alt="浦语挑战赛创新创意奖">
|
||||
</p>
|
||||
|
||||
- 项目荣获公众号**NLP工程化**[推文宣传](https://mp.weixin.qq.com/s/78lrRl2tlXEKUfElnkVx4A)
|
||||
|
||||
### 🎯路线图
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/">
|
||||
<img src="assets/Roadmap_ZH.png" alt="Roadmap_ZH">
|
||||
</a>
|
||||
|
||||
### 🎯框架图
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/">
|
||||
<img src="assets/框架图.png" alt="Framework_ZH">
|
||||
</a>
|
||||
|
||||
## 目录
|
||||
|
||||
- [EmoLLM-心理健康大模型](#emollm-心理健康大模型)
|
||||
- [🎇最近更新](#最近更新)
|
||||
- [🏆荣誉栏](#荣誉栏)
|
||||
- [🎯路线图](#路线图)
|
||||
- [🎯框架图](#框架图)
|
||||
- [目录](#目录)
|
||||
- [开发前的配置要求](#开发前的配置要求)
|
||||
- [**使用指南**](#使用指南)
|
||||
- [数据构建](#数据构建)
|
||||
- [微调指南](#微调指南)
|
||||
- [部署指南](#部署指南)
|
||||
- [RAG(检索增强生成)Pipeline](#rag检索增强生成pipeline)
|
||||
- [使用到的框架](#使用到的框架)
|
||||
- [如何参与本项目](#如何参与本项目)
|
||||
- [作者（排名不分先后）](#作者排名不分先后)
|
||||
- [版权说明](#版权说明)
|
||||
- [引用](#引用)
|
||||
- [特别鸣谢](#特别鸣谢)
|
||||
- [Star History](#star-history)
|
||||
- [🌟 Contributors](#-contributors)
|
||||
- [交流群](#交流群)
|
||||
|
||||
###### 开发前的配置要求
|
||||
|
||||
- 硬件：A100 40G（仅针对InternLM2_7B_chat+qlora微调+deepspeed zero2优化）
|
||||
|
||||
###### **使用指南**
|
||||
|
||||
1. Clone the repo
|
||||
|
||||
```sh
|
||||
git clone https://github.com/SmartFlowAI/EmoLLM.git
|
||||
```
|
||||
|
||||
2. 依次阅读或者选择感兴趣的部分阅读：
|
||||
- [数据构建](#数据构建)
|
||||
- [微调指南](#微调指南)
|
||||
- [部署指南](#部署指南)
|
||||
- [RAG](#rag检索增强生成pipeline)
|
||||
- 查看更多详情
|
||||
|
||||
### 数据构建
|
||||
|
||||
- 请阅读[数据构建指南](generate_data/tutorial.md)查阅
|
||||
|
||||
- 微调用到的数据集见[datasets](datasets/data.json)
|
||||
|
||||
### 微调指南
|
||||
|
||||
详见[微调指南](xtuner_config/README.md)
|
||||
|
||||
### 部署指南
|
||||
|
||||
- Demo部署：详见[部署指南](demo/README.md)
|
||||
- 基于[LMDeploy](https://github.com/InternLM/lmdeploy/)的量化部署：详见[deploy](./deploy/lmdeploy.md)
|
||||
|
||||
### RAG(检索增强生成)Pipeline
|
||||
|
||||
- 详见[RAG](./rag/)
|
||||
|
||||
<details>
|
||||
<summary>更多详情</summary>
|
||||
|
||||
### 使用到的框架
|
||||
|
||||
- [Xtuner](https://github.com/InternLM/xtuner)：用于微调
|
||||
- [Transformers](https://github.com/huggingface/transformers)
|
||||
- [Pytorch](https://pytorch.org/)
|
||||
- [LMDeploy](https://github.com/InternLM/lmdeploy/)：用于量化部署
|
||||
- [Stremlit](https://streamlit.io/)：用于构建Demo
|
||||
- [DeepSpeed](https://github.com/microsoft/DeepSpeed)：并行训练
|
||||
- …
|
||||
|
||||
#### 如何参与本项目
|
||||
|
||||
贡献使开源社区成为一个学习、激励和创造的绝佳场所。你所作的任何贡献都是**非常感谢**的。
|
||||
|
||||
1. Fork the Project
|
||||
2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)
|
||||
3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)
|
||||
4. Push to the Branch (`git push origin feature/AmazingFeature`)
|
||||
5. Open a Pull Request
|
||||
|
||||
</details>
|
||||
|
||||
### 作者（排名不分先后）
|
||||
|
||||
| 用户名 | 学校/组织 | 备注 | 贡献 |
|
||||
|:--------------------------------------------------------------------:|:--------------------------------------------------:| :-------------------: |:---------:|
|
||||
| [aJupyter](https://github.com/aJupyter) | 南开大学在读硕士 | DataWhale成员 | 项目发起人 |
|
||||
| [MING-ZCH](https://github.com/MING-ZCH) | 华中科技大学在读本科生 | LLM x Psychology 研究者 | 项目联合负责人 |
|
||||
| [jujimeizuo](https://github.com/jujimeizuo) | 江南大学在读硕士 | | |
|
||||
| [Smiling-Weeping-zhr](https://github.com/Smiling-Weeping-zhr) | 哈尔滨工业大学（威海）在读本科生 | | |
|
||||
| [8baby8](https://github.com/8baby8) | 飞桨领航团区域主管 | 文心大模型核心开发者 | |
|
||||
| [zxazys](https://github.com/zxazys) | 南开大学在读硕士 | | |
|
||||
| [JasonLLLLLLLLLLL](https://github.com/JasonLLLLLLLLLLL) | swufe | | |
|
||||
| [MrCatAI](https://github.com/MrCatAI) | AI搬用工 | | |
|
||||
| [ZeyuBa](https://github.com/ZeyuBa) | 自动化所在读硕士 | | |
|
||||
| [aiyinyuedejustin](https://github.com/aiyinyuedejustin) | 宾夕法尼亚大学在读硕士 | | |
|
||||
| [Nobody-ML](https://github.com/Nobody-ML) | 中国石油大学（华东）在读本科生 | | |
|
||||
| [chg0901](https://github.com/chg0901) | [MiniSora](https://github.com/mini-sora/minisora/) |[MiniSora](https://github.com/mini-sora/minisora/)主要维护者，管理员| LLM预训练和微调、模型上传、数据清洗、文档翻译 |
|
||||
| [Mxoder](https://github.com/Mxoder) | 北京航空航天大学在读本科生 | | |
|
||||
| [Anooyman](https://github.com/Anooyman) | 南京理工大学硕士 | | |
|
||||
| [Vicky-3021](https://github.com/Vicky-3021) | 西安电子科技大学硕士（研0） | | |
|
||||
| [SantiagoTOP](https://github.com/santiagoTOP) | 太原理工大学在读硕士 | | |
|
||||
| [zealot52099](https://github.com/zealot52099) | AI搬用工 | | 清洗数据、RAG |
|
||||
| [wwwyfff](https://github.com/wwwyfff) | 复旦大学在读硕士 | ||
|
||||
| [jkhumor](https://github.com/jkhumor) | 南开大学在读硕士 | | RAG |
|
||||
| [lll997150986](https://github.com/lll997150986) | 南开大学在读硕士 | | 微调 |
|
||||
| [nln-maker](https://github.com/nln-maker) | 南开大学在读硕士 | | 前后端开发 |
|
||||
| [dream00001](https://github.com/dream00001) | 南开大学在读硕士 | | 前后端开发 |
|
||||
|
||||
|
||||
### 版权说明
|
||||
|
||||
该项目签署了 MIT 授权许可，详情请参阅 [LICENSE](https://github.com/SmartFlowAI/EmoLLM/blob/main/LICENSE)
|
||||
|
||||
### 引用
|
||||
|
||||
如果本项目对您的工作有所帮助，请使用以下格式引用：
|
||||
|
||||
```bibtex
|
||||
@misc{EmoLLM,
|
||||
title={EmoLLM},
|
||||
author={EmoLLM},
|
||||
url={https://github.com/SmartFlowAI/EmoLLM/},
|
||||
year={2024}
|
||||
}
|
||||
```
|
||||
|
||||
### 特别鸣谢
|
||||
|
||||
- [Sanbu](https://github.com/sanbuphy)
|
||||
- [上海人工智能实验室](https://www.shlab.org.cn/)
|
||||
- [闻星大佬（小助手）](https://github.com/vansin)
|
||||
- [扫地升（公众号宣传）](https://mp.weixin.qq.com/s/78lrRl2tlXEKUfElnkVx4A)
|
||||
- 阿布（北大心理学硕士）
|
||||
|
||||
<!-- links -->
|
||||
|
||||
<!-- [linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=flat-square&logo=linkedin&colorB=555 -->
|
||||
|
||||
<!-- [linkedin-url]: https://linkedin.com/in/aJupyter -->
|
||||
|
||||
## Star History
|
||||
|
||||
[](https://star-history.com/#SmartFlowAI/EmoLLM&Date)
|
||||
|
||||
## 🌟 Contributors
|
||||
|
||||
[](https://github.com/SmartFlowAI/EmoLLM/graphs/contributors)
|
||||
|
||||
[your-project-path]: SmartflowAI/EmoLLM
|
||||
[contributors-shield]: https://img.shields.io/github/contributors/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[contributors-url]: https://github.com/SmartflowAI/EmoLLM/graphs/contributors
|
||||
[forks-shield]: https://img.shields.io/github/forks/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[forks-url]: https://github.com/SmartflowAI/EmoLLM/network/members
|
||||
[stars-shield]: https://img.shields.io/github/stars/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[stars-url]: https://github.com/SmartflowAI/EmoLLM/stargazers
|
||||
[issues-shield]: https://img.shields.io/github/issues/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[issues-url]: https://img.shields.io/github/issues/SmartflowAI/EmoLLM.svg
|
||||
[license-shield]: https://img.shields.io/github/license/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[license-url]: https://github.com/SmartFlowAI/EmoLLM/blob/main/LICENSE
|
||||
|
||||
[OpenXLab_App-image]: https://cdn-static.openxlab.org.cn/app-center/openxlab_app.svg
|
||||
[OpenXLab_Model-image]: https://cdn-static.openxlab.org.cn/header/openxlab_models.svg
|
||||
[OpenXLab_App-url]: https://openxlab.org.cn/apps/detail/Farewell1/EmoLLMV2.0
|
||||
[OpenXLab_Model-url]: https://openxlab.org.cn/models/detail/ajupyter/EmoLLM_internlm2_7b_full
|
||||
|
||||
## 交流群
|
||||
|
||||
- 如果失效，请移步Issue区
|
||||
|
||||
<p align="center">
|
||||
<img width="30%" src="https://github.com/SmartFlowAI/EmoLLM/assets/62385492/55ecd0aa-4832-4269-ad57-4c26f9aa286b" alt="EmoLLM官方交流群">
|
||||
</p>
|
||||
<div align="center">
|
||||
|
||||
# EmoLLM-心理健康大模型
|
||||
|
||||
</div>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/">
|
||||
<img src="assets/EmoLLM_transparent.png" alt="Logo" width="50%">
|
||||
</a>
|
||||
|
||||
<div align="center">
|
||||
|
||||
<!-- PROJECT SHIELDS -->
|
||||
[![Contributors][contributors-shield]][contributors-url]
|
||||
[![Forks][forks-shield]][forks-url]
|
||||
[![Issues][issues-shield]][issues-url]
|
||||
[![OpenXLab_App][OpenXLab_App-image]][OpenXLab_App-url]
|
||||
[![OpenXLab_Model][OpenXLab_Model-image]][OpenXLab_Model-url]
|
||||
[![MIT License][license-shield]][license-url]
|
||||
[![Stargazers][stars-shield]][stars-url]
|
||||
|
||||
</div>
|
||||
|
||||
<h3 align="center">EmoLLM</h3>
|
||||
|
||||
<div align="center">
|
||||
简体中文| <a href="README_EN.md" >English</a>
|
||||
<br />
|
||||
<br />
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM"><strong>探索本项目的文档 »</strong></a>
|
||||
<br />
|
||||
<br />
|
||||
<a href="https://openxlab.org.cn/apps/detail/Farewell1/EmoLLMV2.0">体验EmoLLM 2.0</a>
|
||||
·
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/issues">报告Bug</a>
|
||||
·
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/issues">提出新特性</a>
|
||||
</div>
|
||||
|
||||
<!-- 本篇README.md面向开发者 -->
|
||||
|
||||
**EmoLLM** 是一系列能够支持 **理解用户-支持用户-帮助用户** 心理健康辅导链路的心理健康大模型，由 `LLM`指令微调而来，欢迎大家star~⭐⭐。目前已经开源的 `LLM` 微调配置如下：
|
||||
|
||||
<div align="center">
|
||||
|
||||
| 模型 | 类型 | 链接 |
|
||||
| :-------------------: | :------: | :------------------------------------------------------------------------------------------------------: |
|
||||
| InternLM2_7B_chat | QLORA | [internlm2_7b_chat_qlora_e3.py](./xtuner_config/internlm2_7b_chat_qlora_e3.py) |
|
||||
| InternLM2_7B_chat | 全量微调 | [internlm2_chat_7b_full.py](./xtuner_config/internlm2_chat_7b_full.py) |
|
||||
| InternLM2_7B_base | QLORA | [internlm2_7b_base_qlora_e10_M_1e4_32_64.py](./xtuner_config/internlm2_7b_base_qlora_e10_M_1e4_32_64.py) |
|
||||
| InternLM2_1_8B_chat | 全量微调 | [internlm2_1_8b_full_alpaca_e3.py](./xtuner_config/internlm2_1_8b_full_alpaca_e3.py) |
|
||||
| InternLM2_20B_chat | LORA | |
|
||||
| Qwen_7b_chat | QLORA | [qwen_7b_chat_qlora_e3.py](./xtuner_config/qwen_7b_chat_qlora_e3.py) |
|
||||
| Qwen1_5-0_5B-Chat | 全量微调 | [qwen1_5_0_5_B_full.py](./xtuner_config/qwen1_5_0_5_B_full.py) |
|
||||
| Baichuan2_13B_chat | QLORA | [baichuan2_13b_chat_qlora_alpaca_e3.py](./xtuner_config/baichuan2_13b_chat_qlora_alpaca_e3.py) |
|
||||
| ChatGLM3_6B | LORA | [chatglm3_6b_lora_alpaca_e3.py](./xtuner_config/chatglm3_6b_lora_alpaca_e3.py) |
|
||||
| DeepSeek MoE_16B_chat | QLORA | [deepseek_moe_16b_chat_qlora_oasst1_e3.py](./xtuner_config/deepseek_moe_16b_chat_qlora_oasst1_e3.py) |
|
||||
| Mixtral 8x7B_instruct | QLORA | [mixtral_8x7b_instruct_qlora_oasst1_e3.py](./xtuner_config/mixtral_8x7b_instruct_qlora_oasst1_e3.py) |
|
||||
| …… | …… | …… |
|
||||
|
||||
</div>
|
||||
|
||||
欢迎大家为本项目做出贡献~
|
||||
|
||||
---
|
||||
|
||||
心理健康大模型（Mental Health Grand Model）是一个综合性的概念，它旨在全面理解和促进个体、群体乃至整个社会的心理健康状态。这个模型通常包含以下几个关键组成部分：
|
||||
|
||||
- 认知因素：涉及个体的思维模式、信念系统、认知偏差以及解决问题的能力。认知因素对心理健康有重要影响，因为它们影响个体如何解释和应对生活中的事件。
|
||||
- 情感因素：包括情绪调节、情感表达和情感体验。情感健康是心理健康的重要组成部分，涉及个体如何管理和表达自己的情感，以及如何从负面情绪中恢复。
|
||||
- 行为因素：涉及个体的行为模式、习惯和应对策略。这包括应对压力的技巧、社交技能以及自我效能感，即个体对自己能力的信心。
|
||||
- 社会环境：包括家庭、工作、社区和文化背景等外部因素，这些因素对个体的心理健康有着直接和间接的影响。
|
||||
- 生理健康：身体健康与心理健康紧密相关。良好的身体健康可以促进心理健康，反之亦然。
|
||||
- 心理韧性：指个体在面对逆境时的恢复力和适应能力。心理韧性强的人更能够从挑战中恢复，并从中学习和成长。
|
||||
- 预防和干预措施：心理健康大模型还包括预防心理问题和促进心理健康的策略，如心理教育、心理咨询、心理治疗和社会支持系统。
|
||||
- 评估和诊断工具：为了有效促进心理健康，需要有科学的工具来评估个体的心理状态，以及诊断可能存在的心理问题。
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td align="center" style="background-color: transparent">
|
||||
<img src="assets\aiwei_demo.gif" alt="占位图">
|
||||
</td>
|
||||
<td align="center" style="background-color: transparent">
|
||||
<img src="assets\aiwei_demo2.gif" alt="占位图">
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" style="background-color: transparent">
|
||||
<img src="assets\aiwei_demo3.gif" alt="占位图">
|
||||
</td>
|
||||
<td align="center" style="background-color: transparent">
|
||||
<img src="assets\aiwei_demo4.gif" alt="占位图">
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
### 🎇最近更新
|
||||
|
||||
- 【2024.4.2】在 Huggingface 上传[老母亲心理咨询师](https://huggingface.co/brycewang2018/EmoLLM-mother/tree/main)
|
||||
- 【2024.3.25】在百度飞桨平台发布[爹系男友心理咨询师](https://aistudio.baidu.com/community/app/68787)
|
||||
- 【2024.3.24】在**OpenXLab**和**ModelScope**平台发布**InternLM2-Base-7B QLoRA微调模型**, 具体请查看[**InternLM2-Base-7B QLoRA**](./xtuner_config/README_internlm2_7b_base_qlora.md)
|
||||
- 【2024.3.12】在百度飞桨平台发布[艾薇](https://aistudio.baidu.com/community/app/63335)
|
||||
- 【2024.3.11】 **EmoLLM V2.0 相比 EmoLLM V1.0 全面提升，已超越 Role-playing ChatGPT 在心理咨询任务上的能力！**[点击体验EmoLLM V2.0](https://openxlab.org.cn/apps/detail/Farewell1/EmoLLMV2.0)，更新[数据集统计及详细信息](./datasets/)、[路线图](./assets/Roadmap_ZH.png)
|
||||
- 【2024.3.9】 新增并发功能加速 [QA 对生成](./scripts/qa_generation/)、[RAG pipeline](./rag/)
|
||||
- 【2024.3.3】 [基于InternLM2-7B-chat全量微调版本EmoLLM V2.0开源](https://openxlab.org.cn/models/detail/ajupyter/EmoLLM_internlm2_7b_full)，需要两块A100*80G，更新专业评估，详见[evaluate](./evaluate/)，更新基于PaddleOCR的PDF转txt工具脚本，详见[scripts](./scripts/)
|
||||
- 【2024.2.29】更新客观评估计算，详见[evaluate](./evaluate/)，更新一系列数据集，详见[datasets](./datasets/)
|
||||
- 【2024.2.27】更新英文readme和一系列数据集（舔狗和单轮对话）
|
||||
- 【2024.2.23】推出基于InternLM2_7B_chat_qlora的 `温柔御姐心理医生艾薇`，[点击获取模型权重](https://openxlab.org.cn/models/detail/ajupyter/EmoLLM_aiwei)，[配置文件](xtuner_config/aiwei-internlm2_chat_7b_qlora.py)，[在线体验链接](https://openxlab.org.cn/apps/detail/ajupyter/EmoLLM-aiwei)
|
||||
- 【2024.2.23】更新[若干微调配置](/xtuner_config/)，新增 [data_pro.json](/datasets/data_pro.json)（数量更多、场景更全、更丰富）和 [aiwei.json](/datasets/aiwei.json)（温柔御姐角色扮演专用，带有Emoji表情），即将推出 `温柔御姐心理医生艾薇`
|
||||
- 【2024.2.18】 [基于Qwen1_5-0_5B-Chat全量微调版本开源](https://www.modelscope.cn/models/aJupyter/EmoLLM_Qwen1_5-0_5B-Chat_full_sft/summary)，算力有限的道友可以玩起来~
|
||||
|
||||
<details>
|
||||
<summary>查看更多</summary>
|
||||
|
||||
- 【2024.2.6】 EmoLLM在[**Openxlab** ](https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model) 平台下载量高达18.7k，欢迎大家体验！
|
||||
|
||||
<p align="center">
|
||||
<img src="https://github.com/SmartFlowAI/EmoLLM/assets/62385492/7e931682-c54d-4ded-bc67-79130c68d744" alt="模型下载量">
|
||||
</p>
|
||||
|
||||
- 【2024.2.5】 项目荣获公众号**NLP工程化**推文宣传[推文链接](https://mp.weixin.qq.com/s/78lrRl2tlXEKUfElnkVx4A)，为博主推广一波，欢迎大家关注！！🥳🥳
|
||||
|
||||
<p align="center">
|
||||
<img src="https://github.com/SmartFlowAI/EmoLLM/assets/62385492/47868d6a-2e91-4aa9-a630-e594c14295b4" alt="公众号二维码">
|
||||
</p>
|
||||
|
||||
- 【2024.2.3】 [项目宣传视频](https://www.bilibili.com/video/BV1N7421N76X/)完成 😊
|
||||
- 【2024.1.27】 完善数据构建文档、微调指南、部署指南、Readme等相关文档 👏
|
||||
- 【2024.1.25】 EmoLLM V1.0 已部署上线 https://openxlab.org.cn/apps/detail/jujimeizuo/EmoLLM 😀
|
||||
|
||||
</details>
|
||||
|
||||
### 🏆荣誉栏
|
||||
|
||||
- 项目荣获上海人工智能实验室举办的**2024浦源大模型系列挑战赛春季赛*****创新创意奖***
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/">
|
||||
<img src="assets/Shusheng.png" alt="浦语挑战赛创新创意奖">
|
||||
</p>
|
||||
|
||||
- 项目荣获公众号**NLP工程化**[推文宣传](https://mp.weixin.qq.com/s/78lrRl2tlXEKUfElnkVx4A)
|
||||
|
||||
### 🎯路线图
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/">
|
||||
<img src="assets/Roadmap_ZH.png" alt="Roadmap_ZH">
|
||||
</a>
|
||||
|
||||
### 🔗框架图
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/">
|
||||
<img src="assets/框架图.png" alt="Framework_ZH">
|
||||
</a>
|
||||
|
||||
## 目录
|
||||
|
||||
- [EmoLLM-心理健康大模型](#emollm-心理健康大模型)
|
||||
- [🎇最近更新](#最近更新)
|
||||
- [🏆荣誉栏](#荣誉栏)
|
||||
- [🎯路线图](#路线图)
|
||||
- [🔗框架图](#框架图)
|
||||
- [目录](#目录)
|
||||
- [开发前的配置要求](#开发前的配置要求)
|
||||
- [**使用指南**](#使用指南)
|
||||
- [🍪快速体验](#快速体验)
|
||||
- [📌数据构建](#数据构建)
|
||||
- [🎨微调指南](#微调指南)
|
||||
- [🔧部署指南](#部署指南)
|
||||
- [⚙RAG(检索增强生成)Pipeline](#rag检索增强生成pipeline)
|
||||
- [使用到的框架](#使用到的框架)
|
||||
- [如何参与本项目](#如何参与本项目)
|
||||
- [作者（排名不分先后）](#作者排名不分先后)
|
||||
- [版权说明](#版权说明)
|
||||
- [引用](#引用)
|
||||
- [特别鸣谢](#特别鸣谢)
|
||||
- [Star History](#star-history)
|
||||
- [🌟 Contributors](#-contributors)
|
||||
- [交流群](#交流群)
|
||||
|
||||
###### 开发前的配置要求
|
||||
|
||||
- 硬件：A100 40G（仅针对InternLM2_7B_chat+qlora微调+deepspeed zero2优化）
|
||||
|
||||
###### **使用指南**
|
||||
|
||||
1. Clone the repo
|
||||
|
||||
```sh
|
||||
git clone https://github.com/SmartFlowAI/EmoLLM.git
|
||||
```
|
||||
|
||||
2. 依次阅读或者选择感兴趣的部分阅读：
|
||||
- [快速体验](#快速体验)
|
||||
- [数据构建](#数据构建)
|
||||
- [微调指南](#微调指南)
|
||||
- [部署指南](#部署指南)
|
||||
- [RAG](#rag检索增强生成pipeline)
|
||||
- 查看更多详情
|
||||
|
||||
|
||||
### 🍪快速体验
|
||||
|
||||
- 请阅读[快速体验](docs/quick_start.md)查阅
|
||||
|
||||
|
||||
### 📌数据构建
|
||||
|
||||
- 请阅读[数据构建指南](generate_data/tutorial.md)查阅
|
||||
|
||||
- 微调用到的数据集见[datasets](datasets/data.json)
|
||||
|
||||
### 🎨微调指南
|
||||
|
||||
详见[微调指南](xtuner_config/README.md)
|
||||
|
||||
### 🔧部署指南
|
||||
|
||||
- Demo部署：详见[部署指南](demo/README.md)
|
||||
- 基于[LMDeploy](https://github.com/InternLM/lmdeploy/)的量化部署：详见[deploy](./deploy/lmdeploy.md)
|
||||
|
||||
### ⚙RAG(检索增强生成)Pipeline
|
||||
|
||||
- 详见[RAG](./rag/)
|
||||
|
||||
<details>
|
||||
<summary>更多详情</summary>
|
||||
|
||||
### 使用到的框架
|
||||
|
||||
- [Xtuner](https://github.com/InternLM/xtuner)：用于微调
|
||||
- [Transformers](https://github.com/huggingface/transformers)
|
||||
- [Pytorch](https://pytorch.org/)
|
||||
- [LMDeploy](https://github.com/InternLM/lmdeploy/)：用于量化部署
|
||||
- [Stremlit](https://streamlit.io/)：用于构建Demo
|
||||
- [DeepSpeed](https://github.com/microsoft/DeepSpeed)：并行训练
|
||||
- …
|
||||
|
||||
#### 如何参与本项目
|
||||
|
||||
贡献使开源社区成为一个学习、激励和创造的绝佳场所。你所作的任何贡献都是**非常感谢**的。
|
||||
|
||||
1. Fork the Project
|
||||
2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)
|
||||
3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)
|
||||
4. Push to the Branch (`git push origin feature/AmazingFeature`)
|
||||
5. Open a Pull Request
|
||||
|
||||
</details>
|
||||
|
||||
### 作者（排名不分先后）
|
||||
|
||||
| 用户名 | 学校/组织 | 备注 | 贡献 |
|
||||
| :-----------------------------------------------------------: | :------------------------------------------------: | :------------------------------------------------------------------: | :-------------------------------------------: |
|
||||
| [aJupyter](https://github.com/aJupyter) | 南开大学在读硕士 | DataWhale成员 | 项目发起人 |
|
||||
| [MING-ZCH](https://github.com/MING-ZCH) | 华中科技大学在读本科生 | LLM x Psychology 研究者 | 项目联合负责人 |
|
||||
| [jujimeizuo](https://github.com/jujimeizuo) | 江南大学在读硕士 | | |
|
||||
| [Smiling-Weeping-zhr](https://github.com/Smiling-Weeping-zhr) | 哈尔滨工业大学（威海）在读本科生 | | |
|
||||
| [8baby8](https://github.com/8baby8) | 飞桨领航团区域主管 | 文心大模型核心开发者 | |
|
||||
| [zxazys](https://github.com/zxazys) | 南开大学在读硕士 | | |
|
||||
| [JasonLLLLLLLLLLL](https://github.com/JasonLLLLLLLLLLL) | swufe | | |
|
||||
| [MrCatAI](https://github.com/MrCatAI) | AI搬用工 | | |
|
||||
| [ZeyuBa](https://github.com/ZeyuBa) | 自动化所在读硕士 | | |
|
||||
| [aiyinyuedejustin](https://github.com/aiyinyuedejustin) | 宾夕法尼亚大学在读硕士 | | |
|
||||
| [Nobody-ML](https://github.com/Nobody-ML) | 中国石油大学（华东）在读本科生 | | |
|
||||
| [chg0901](https://github.com/chg0901) | [MiniSora](https://github.com/mini-sora/minisora/) | [MiniSora](https://github.com/mini-sora/minisora/)主要维护者，管理员 | LLM预训练和微调、模型上传、数据清洗、文档翻译 |
|
||||
| [Mxoder](https://github.com/Mxoder) | 北京航空航天大学在读本科生 | | |
|
||||
| [Anooyman](https://github.com/Anooyman) | 南京理工大学硕士 | | |
|
||||
| [Vicky-3021](https://github.com/Vicky-3021) | 西安电子科技大学硕士（研0） | | |
|
||||
| [SantiagoTOP](https://github.com/santiagoTOP) | 太原理工大学在读硕士 | | |
|
||||
| [zealot52099](https://github.com/zealot52099) | 个人开发者 | | 清洗数据、LLM微调、RAG |
|
||||
| [wwwyfff](https://github.com/wwwyfff) | 复旦大学在读硕士 | | |
|
||||
| [Yicooong](https://github.com/Yicooong) | 南开大学在读硕士 | | |
|
||||
| [jkhumor](https://github.com/jkhumor) | 南开大学在读硕士 | | RAG |
|
||||
| [lll997150986](https://github.com/lll997150986) | 南开大学在读硕士 | | 微调 |
|
||||
| [nln-maker](https://github.com/nln-maker) | 南开大学在读硕士 | | 前后端开发 |
|
||||
| [dream00001](https://github.com/dream00001) | 南开大学在读硕士 | | 前后端开发 |
|
||||
| [王几行XING](https://zhihu.com/people/brycewang1898) | 北京大学硕士毕业 | | 清洗数据、LLM微调、前后端开发 |
|
||||
| [思在] | 北京大学硕士毕业（微软美国） | | LLM微调、前后端开发 |
|
||||
|
||||
### 版权说明
|
||||
|
||||
该项目签署了 MIT 授权许可，详情请参阅 [LICENSE](https://github.com/SmartFlowAI/EmoLLM/blob/main/LICENSE)
|
||||
|
||||
### 引用
|
||||
|
||||
如果本项目对您的工作有所帮助，请使用以下格式引用：
|
||||
|
||||
```bibtex
|
||||
@misc{EmoLLM,
|
||||
title={EmoLLM},
|
||||
author={EmoLLM},
|
||||
url={https://github.com/SmartFlowAI/EmoLLM/},
|
||||
year={2024}
|
||||
}
|
||||
```
|
||||
|
||||
### 特别鸣谢
|
||||
|
||||
- [Sanbu](https://github.com/sanbuphy)
|
||||
- [上海人工智能实验室](https://www.shlab.org.cn/)
|
||||
- [闻星大佬（小助手）](https://github.com/vansin)
|
||||
- [扫地升（公众号宣传）](https://mp.weixin.qq.com/s/78lrRl2tlXEKUfElnkVx4A)
|
||||
- 阿布（北大心理学硕士）
|
||||
- [HatBoy](https://github.com/hatboy)
|
||||
|
||||
<!-- links -->
|
||||
|
||||
<!-- [linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=flat-square&logo=linkedin&colorB=555 -->
|
||||
|
||||
<!-- [linkedin-url]: https://linkedin.com/in/aJupyter -->
|
||||
|
||||
## Star History
|
||||
|
||||
[](https://star-history.com/#SmartFlowAI/EmoLLM&Date)
|
||||
|
||||
## 🌟 Contributors
|
||||
|
||||
[](https://github.com/SmartFlowAI/EmoLLM/graphs/contributors)
|
||||
|
||||
[your-project-path]: SmartflowAI/EmoLLM
|
||||
[contributors-shield]: https://img.shields.io/github/contributors/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[contributors-url]: https://github.com/SmartflowAI/EmoLLM/graphs/contributors
|
||||
[forks-shield]: https://img.shields.io/github/forks/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[forks-url]: https://github.com/SmartflowAI/EmoLLM/network/members
|
||||
[stars-shield]: https://img.shields.io/github/stars/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[stars-url]: https://github.com/SmartflowAI/EmoLLM/stargazers
|
||||
[issues-shield]: https://img.shields.io/github/issues/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[issues-url]: https://img.shields.io/github/issues/SmartflowAI/EmoLLM.svg
|
||||
[license-shield]: https://img.shields.io/github/license/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[license-url]: https://github.com/SmartFlowAI/EmoLLM/blob/main/LICENSE
|
||||
|
||||
[OpenXLab_App-image]: https://cdn-static.openxlab.org.cn/app-center/openxlab_app.svg
|
||||
[OpenXLab_Model-image]: https://cdn-static.openxlab.org.cn/header/openxlab_models.svg
|
||||
[OpenXLab_App-url]: https://openxlab.org.cn/apps/detail/Farewell1/EmoLLMV2.0
|
||||
[OpenXLab_Model-url]: https://openxlab.org.cn/models/detail/ajupyter/EmoLLM_internlm2_7b_full
|
||||
|
||||
## 交流群
|
||||
|
||||
- 如果失效，请移步Issue区
|
||||
|
||||
<p align="center">
|
||||
<img width="30%" src="https://github.com/SmartFlowAI/EmoLLM/assets/62385492/55ecd0aa-4832-4269-ad57-4c26f9aa286b" alt="EmoLLM官方交流群">
|
||||
</p>
|
||||
656
README_EN.md
656
README_EN.md
@ -1,320 +1,336 @@
|
||||
<div align="center">
|
||||
|
||||
# EmoLLM - Large Language Model for Mental Health
|
||||
|
||||
</div>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/">
|
||||
<img src="assets/logo.jpeg" alt="Logo" width="30%">
|
||||
</a>
|
||||
|
||||
<div align="center">
|
||||
|
||||
<!-- PROJECT SHIELDS -->
|
||||
[![Contributors][contributors-shield]][contributors-url]
|
||||
[![Forks][forks-shield]][forks-url]
|
||||
[![Issues][issues-shield]][issues-url]
|
||||
[![OpenXLab_App][OpenXLab_App-image]][OpenXLab_App-url]
|
||||
[![OpenXLab_Model][OpenXLab_Model-image]][OpenXLab_Model-url]
|
||||
[![MIT License][license-shield]][license-url]
|
||||
[![Stargazers][stars-shield]][stars-url]
|
||||
|
||||
</div>
|
||||
|
||||
<h3 align="center">EmoLLM</h3>
|
||||
|
||||
<p align="center">
|
||||
<a href="README.md">简体中文</a> | English
|
||||
<br />
|
||||
<br />
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM"><strong>Explore the documentation of this project »</strong></a>
|
||||
<br />
|
||||
<br />
|
||||
<a href="https://openxlab.org.cn/apps/detail/Farewell1/EmoLLMV2.0">EmoLLM 2.0 Demo</a>
|
||||
·
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/issues">Report a Bug</a>
|
||||
·
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/issues">Propose a New Feature</a>
|
||||
</p>
|
||||
|
||||
</p>
|
||||
|
||||
<!-- 本篇README.md面向开发者 -->
|
||||
|
||||
**EmoLLM** is a series of large language models designed to understand, support and help customers in mental health counseling. It is fine-tuned from the LLM instructions. We really appreciate it if you could give it a star~⭐⭐. The open-sourced configuration is as follows:
|
||||
|
||||
<div align="center">
|
||||
|
||||
| Model | Type |
|
||||
| :-------------------: | :------: |
|
||||
| InternLM2_7B_chat | QLORA |
|
||||
| InternLM2_7B_chat | full fine-tuning |
|
||||
| InternLM2_7B_base | QLORA |
|
||||
| InternLM2_1_8B_chat | full fine-tuning |
|
||||
| InternLM2_20B_chat | LORA |
|
||||
| Qwen_7b_chat | QLORA |
|
||||
| Qwen1_5-0_5B-Chat | full fine-tuning |
|
||||
| Baichuan2_13B_chat | QLORA |
|
||||
| ChatGLM3_6B | LORA |
|
||||
| DeepSeek MoE_16B_chat | QLORA |
|
||||
| Mixtral 8x7B_instruct | QLORA |
|
||||
| …… | …… |
|
||||
|
||||
</div>
|
||||
|
||||
Everyone is welcome to contribute to this project ~
|
||||
|
||||
---
|
||||
|
||||
The Model aims to fully understand and promote the mental health of individuals, groups, and society. This model typically includes the following key components:
|
||||
|
||||
- Cognitive factors: Involving an individual's thought patterns, belief systems, cognitive biases, and problem-solving abilities. Cognitive factors significantly impact mental health as they affect how individuals interpret and respond to life events.
|
||||
- Emotional factors: Including emotion regulation, emotional expression, and emotional experiences. Emotional health is a crucial part of mental health, involving how individuals manage and express their emotions and how they recover from negative emotions.
|
||||
- Behavioral factors: Concerning an individual's behavior patterns, habits, and coping strategies. This includes stress management skills, social skills, and self-efficacy, which is the confidence in one's abilities.
|
||||
- Social environment: Comprising external factors such as family, work, community, and cultural background, which have direct and indirect impacts on an individual's mental health.
|
||||
- Physical health: There is a close relationship between physical and mental health. Good physical health can promote mental health and vice versa.
|
||||
- Psychological resilience: Refers to an individual's ability to recover from adversity and adapt. Those with strong psychological resilience can bounce back from challenges and learn and grow from them.
|
||||
- Prevention and intervention measures: The Mental Health Grand Model also includes strategies for preventing psychological issues and promoting mental health, such as psychological education, counseling, therapy, and social support systems.
|
||||
- Assessment and diagnostic tools: Effective promotion of mental health requires scientific tools to assess individuals' psychological states and diagnose potential psychological issues.
|
||||
|
||||
### Recent Updates
|
||||
|
||||
- 【2024.3.25】 [Daddy-like Boy-Friend] is released on Baidu Paddle-Paddle AI Studio Platform (https://aistudio.baidu.com/community/app/68787)
|
||||
- 【2024.3.24】 The InternLM2-Base-7B QLoRA fine-tuned model has been released on the OpenXLab and ModelScope platforms. For more details, please refer to [InternLM2-Base-7B QLoRA](./xtuner_config/README_internlm2_7b_base_qlora.md).
|
||||
- 【2024.3.12】 [aiwei] is released on Baidu Paddle-Paddle AI Studio Platform (https://aistudio.baidu.com/community/app/63335)
|
||||
- 【2024.3.11】 **EmoLLM V2.0 is greatly improved in all scores compared to EmoLLM V1.0. Surpasses the performance of Role-playing ChatGPT on counseling tasks!** [Click to experience EmoLLM V2.0](https://openxlab.org.cn/apps/detail/Farewell1/EmoLLMV2.0), update [dataset statistics and details](./datasets/), [Roadmap](./assets/Roadmap_ZH.png)
|
||||
- 【2024.3.9】 Add concurrency acceleration [QA pair generation](./scripts/qa_generation/), [RAG pipeline](./rag/)
|
||||
- 【2024.3.3】 [Based on InternLM2-7B-chat full fine-tuned version EmoLLM V2.0 open sourced](https://openxlab.org.cn/models/detail/ajupyter/EmoLLM_internlm2_7b_full), need two A100*80G, update professional evaluation, see [evaluate](./evaluate/), update PaddleOCR-based PDF to txt tool scripts, see [scripts](./scripts/).
|
||||
- 【2024.2.29】 Updated objective assessment calculations, see [evaluate](./evaluate/) for details. A series of datasets have also been updated, see [datasets](./datasets/) for details.
|
||||
- 【2024.2.27】 Updated English README and a series of datasets (licking dogs and one-round dialogue)
|
||||
- 【2024.2.23】The "Gentle Lady Psychologist Ai Wei" based on InternLM2_7B_chat_qlora was launched. [Click here to obtain the model weights](https://openxlab.org.cn/models/detail/ajupyter/EmoLLM_aiwei), [configuration file](xtuner_config/aiwei-internlm2_chat_7b_qlora.py), [online experience link](https://openxlab.org.cn/apps/detail/ajupyter/EmoLLM-aiwei)
|
||||
|
||||
- 【2024.2.23】Updated [several fine-tuning configurations](/xtuner_config/), added [data_pro.json](/datasets/data_pro.json) (more quantity, more comprehensive scenarios, richer content) and [aiwei.json](/datasets/aiwei.json) (dedicated to the gentle lady role-play, featuring Emoji expressions), the "Gentle Lady Psychologist Ai Wei" is coming soon.
|
||||
|
||||
- 【2024.2.18】 The full fine-tuned version based on Qwen1_5-0_5B-Chat has been [open-sourced](https://www.modelscope.cn/models/aJupyter/EmoLLM_Qwen1_5-0_5B-Chat_full_sft/summary). Friends with limited computational resources can now dive in and explore it.
|
||||
|
||||
<details>
|
||||
<summary>View More</summary>
|
||||
|
||||
- 【2024.2.6】 [Open-sourced based on the Qwen1_5-0_5B-Chat full-scale fine-tuned version](https://www.modelscope.cn/models/aJupyter/EmoLLM_Qwen1_5-0_5B-Chat_full_sft/summary), friends with limited computing power can start experimenting~
|
||||
|
||||
<p align="center">
|
||||
<img src="https://github.com/SmartFlowAI/EmoLLM/assets/62385492/7e931682-c54d-4ded-bc67-79130c68d744" alt="模型下载量">
|
||||
</p>
|
||||
|
||||
- 【2024.2.5】 The project has been promoted by the official WeChat account NLP Engineering. Here's the [link](https://mp.weixin.qq.com/s/78lrRl2tlXEKUfElnkVx4A) to the article. Welcome everyone to follow!! 🥳🥳
|
||||
|
||||
<p align="center">
|
||||
<img src="https://github.com/SmartFlowAI/EmoLLM/assets/62385492/47868d6a-2e91-4aa9-a630-e594c14295b4" alt="公众号二维码">
|
||||
</p>
|
||||
|
||||
- 【2024.2.3】 [Project Vedio](https://www.bilibili.com/video/BV1N7421N76X/) at bilibili 😊
|
||||
- 【2024.1.27】 Complete data construction documentation, fine-tuning guide, deployment guide, Readme, and other related documents 👏
|
||||
- 【2024.1.25】 EmoLLM V1.0 has deployed online https://openxlab.org.cn/apps/detail/jujimeizuo/EmoLLM 😀
|
||||
|
||||
</details>
|
||||
|
||||
### Honors
|
||||
|
||||
- The project won the ***the Innovation and Creativity Award*** in the **2024 Puyuan Large Model Series Challenge Spring Competition held by the Shanghai Artificial Intelligence Laboratory**
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/">
|
||||
<img src="assets/Shusheng.jpg" alt="Challenge Innovation and Creativity Award">
|
||||
</p>
|
||||
|
||||
|
||||
- The project has been promoted by the official WeChat account **NLP Engineering**. Here's the [link](https://mp.weixin.qq.com/s/78lrRl2tlXEKUfElnkVx4A).
|
||||
|
||||
### Roadmap
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/">
|
||||
<img src="assets/Roadmap_EN.png" alt="Roadmap_EN">
|
||||
</a>
|
||||
|
||||
## Contents
|
||||
|
||||
- [EmoLLM - Large Language Model for Mental Health](#emollm---large-language-model-for-mental-health)
|
||||
- [Recent Updates](#recent-updates)
|
||||
- [Honor](#honor)
|
||||
- [Roadmap](#roadmap)
|
||||
- [Contents](#contents)
|
||||
- [Pre-development Configuration Requirements.](#pre-development-configuration-requirements)
|
||||
- [**User Guide**](#user-guide)
|
||||
- [File Directory Explanation](#file-directory-explanation)
|
||||
- [Data Construction](#data-construction)
|
||||
- [Fine-tuning Guide](#fine-tuning-guide)
|
||||
- [Deployment Guide](#deployment-guide)
|
||||
- [RAG (Retrieval Augmented Generation) Pipeline](#rag-retrieval-augmented-generation-pipeline)
|
||||
- [Frameworks Used](#frameworks-used)
|
||||
- [How to participate in this project](#how-to-participate-in-this-project)
|
||||
- [Version control](#version-control)
|
||||
- [Authors (in no particular order)](#authors-in-no-particular-order)
|
||||
- [Copyright Notice](#copyright-notice)
|
||||
- [Acknowledgments](#acknowledgments)
|
||||
- [Star History](#star-history)
|
||||
- [🌟 Contributors](#-contributors)
|
||||
- [Communication group](#communication-group)
|
||||
|
||||
###### Pre-development Configuration Requirements.
|
||||
|
||||
- A100 40G (specifically for InternLM2_7B_chat + qlora fine-tuning + deepspeed zero2 optimization)
|
||||
|
||||
###### **User Guide**
|
||||
|
||||
1. Clone the repo
|
||||
|
||||
```sh
|
||||
git clone https://github.com/SmartFlowAI/EmoLLM.git
|
||||
```
|
||||
|
||||
1. Read in sequence or read sections you're interested in：
|
||||
- [File Directory Explanation](#file-directory-explanation)
|
||||
- [Data Construction](#data-construction)
|
||||
- [Fine-tuning Guide](#fine-tuning-guide)
|
||||
- [Deployment Guide](#deployment-guide)
|
||||
- View More Details
|
||||
|
||||
### File Directory Explanation
|
||||
|
||||
```
|
||||
├─assets: Image Resources
|
||||
├─datasets: Dataset
|
||||
├─demo: demo scripts
|
||||
├─generate_data: Data Generation Guide
|
||||
│ └─xinghuo
|
||||
├─scripts: Some Available Tools
|
||||
└─xtuner_config：Fine-tuning Guide
|
||||
└─images
|
||||
```
|
||||
|
||||
### Data Construction
|
||||
|
||||
- Please read the [Data Construction Guide ](generate_data/tutorial.md)for reference.
|
||||
|
||||
- The dataset used for this fine-tuning can be found at [datasets](datasets/data.json)
|
||||
|
||||
### Fine-tuning Guide
|
||||
|
||||
For details, see the [fine-tuning guide](xtuner_config/README.md)
|
||||
|
||||
### Deployment Guide
|
||||
|
||||
- Demo deployment: see [deployment guide](./demo/README.md) for details.
|
||||
- Quantitative deployment based on [LMDeploy](https://github.com/InternLM/lmdeploy/): see [deploy](./deploy/lmdeploy.md)
|
||||
|
||||
### RAG (Retrieval Augmented Generation) Pipeline
|
||||
|
||||
- See [RAG](./rag/)
|
||||
|
||||
<details>
|
||||
<summary>Additional Details</summary>
|
||||
|
||||
### Frameworks Used
|
||||
|
||||
- [Xtuner](https://github.com/InternLM/xtuner)
|
||||
- [Transformers](https://github.com/huggingface/transformers)
|
||||
- [Pytorch](https://pytorch.org/)
|
||||
- [LMDeploy](https://github.com/InternLM/lmdeploy/): for quantitative deployment
|
||||
- [Stremlit](https://streamlit.io/): for building demos
|
||||
- [DeepSpeed](https://github.com/microsoft/DeepSpeed): for parallel training
|
||||
- …
|
||||
|
||||
#### How to participate in this project
|
||||
|
||||
Contributions make the open-source community an excellent place for learning, inspiration, and creation. Any contribution you make is greatly appreciated.
|
||||
|
||||
1. Fork the Project
|
||||
2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)
|
||||
3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)
|
||||
4. Push to the Branch (`git push origin feature/AmazingFeature`)
|
||||
5. Open a Pull Request
|
||||
|
||||
### Version control
|
||||
|
||||
This project uses Git for version control. You can see the currently available versions in the repository.
|
||||
|
||||
</details>
|
||||
|
||||
### Authors (in no particular order)
|
||||
|
||||
| Username | School/Organization | Remarks | Contributions |
|
||||
|:-----------------------------------------------------------------------:|:--------------------------------------------------------------------:| :------------------: |:----------------------------------:|
|
||||
| [aJupyter](https://github.com/aJupyter) | Nankai University, Master's student | DataWhale member | Project initiator |
|
||||
| [MING-ZCH](https://github.com/MING-ZCH) | Huazhong University of Science and Technology, Undergraduate student | LLM X Psychology researcher | Project co-leader |
|
||||
| [jujimeizuo](https://github.com/jujimeizuo) | Jiangnan University, Master's student | | |
|
||||
| [Smiling-Weeping-zhr](https://github.com/Smiling-Weeping-zhr) | Harbin Institute of Technology (Weihai), Undergraduate student | | |
|
||||
| [8baby8](https://github.com/8baby8) | PaddlePaddle Pilot Team Regional Director | Wenxin Large Model core developer | |
|
||||
| [zxazys](https://github.com/zxazys) | Nankai University, Master's student | | |
|
||||
| [JasonLLLLLLLLLLL](https://github.com/JasonLLLLLLLLLLL) | SWUFE (Southwestern University of Finance and Economics) | | |
|
||||
| [MrCatAI](https://github.com/MrCatAI) | AI Mover | | |
|
||||
| [ZeyuBa](https://github.com/ZeyuBa) | Institute of Automation, Master's student | | |
|
||||
| [aiyinyuedejustin](https://github.com/aiyinyuedejustin) | University of Pennsylvania, Master's student | | |
|
||||
| [Nobody-ML](https://github.com/Nobody-ML) | China University of Petroleum (East China), Undergraduate student | | |
|
||||
| [chg0901](https://github.com/chg0901) | [MiniSora](https://github.com/mini-sora/minisora) |Maintainer and Admin of [MiniSora](https://github.com/mini-sora/minisora) | LLM Pre-Training and Fine-Tuning, Model Uploading, Data Cleaning and Docs Translation |
|
||||
| [Mxoder](https://github.com/Mxoder) | Beihang University, Undergraduate student | | |
|
||||
| [Anooyman](https://github.com/Anooyman) | Nanjing University of Science and Technology, Master's student | | |
|
||||
| [Vicky-3021](https://github.com/Vicky-3021) | Xidian University, Master's student (Research Year 0) | | |
|
||||
| [SantiagoTOP](https://github.com/santiagoTOP) | Taiyuan University of Technology, Master's student | | |
|
||||
| [zealot52099](https://github.com/zealot52099) | AI Mover | | Data Processing and RAG |
|
||||
| [wwwyfff](https://github.com/wwwyfff) | FuDan University, Master's student | ||
|
||||
| [jkhumor](https://github.com/jkhumor) | Nankai University, Master's student | | RAG |
|
||||
| [lll997150986](https://github.com/lll997150986) | Nankai University, Master's student | | Fine Tuning |
|
||||
| [nln-maker](https://github.com/nln-maker) | Nankai University, Master's student | | Front-end and back-end development |
|
||||
| [dream00001](https://github.com/dream00001) | Nankai University, Master's student | | Front-end and back-end development |
|
||||
|
||||
### Copyright Notice
|
||||
|
||||
The project is licensed under the MIT License. Please refer to the details
|
||||
[LICENSE](https://github.com/SmartFlowAI/EmoLLM/blob/master/LICENSE)
|
||||
|
||||
### Acknowledgments
|
||||
|
||||
- [Sanbu](https://github.com/sanbuphy)
|
||||
- [Shanghai Artificial Intelligence Laboratory](https://www.shlab.org.cn/)
|
||||
- [Vanin](https://github.com/vansin)
|
||||
- [Bloom up (WeChat Official Account Promotion)](https://mp.weixin.qq.com/s/78lrRl2tlXEKUfElnkVx4A)
|
||||
- Abu (M.A. in Psychology, Peking University)
|
||||
|
||||
<!-- links -->
|
||||
|
||||
<!-- [linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=flat-square&logo=linkedin&colorB=555 -->
|
||||
|
||||
<!-- [linkedin-url]: https://linkedin.com/in/aJupyter -->
|
||||
|
||||
<!-- 太少了，没必要放 -->
|
||||
|
||||
## Star History
|
||||
|
||||
[](https://star-history.com/#SmartFlowAI/EmoLLM&Date)
|
||||
|
||||
## 🌟 Contributors
|
||||
|
||||
[](https://github.com/SmartFlowAI/EmoLLM/graphs/contributors)
|
||||
|
||||
[your-project-path]: SmartflowAI/EmoLLM
|
||||
[contributors-shield]: https://img.shields.io/github/contributors/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[contributors-url]: https://github.com/SmartflowAI/EmoLLM/graphs/contributors
|
||||
[forks-shield]: https://img.shields.io/github/forks/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[forks-url]: https://github.com/SmartflowAI/EmoLLM/network/members
|
||||
[stars-shield]: https://img.shields.io/github/stars/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[stars-url]: https://github.com/SmartflowAI/EmoLLM/stargazers
|
||||
[issues-shield]: https://img.shields.io/github/issues/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[issues-url]: https://img.shields.io/github/issues/SmartflowAI/EmoLLM.svg
|
||||
[license-shield]: https://img.shields.io/github/license/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[license-url]: https://github.com/SmartflowAI/EmoLLM/blob/main/LICENSE
|
||||
|
||||
[OpenXLab_App-image]: https://cdn-static.openxlab.org.cn/app-center/openxlab_app.svg
|
||||
[OpenXLab_Model-image]: https://cdn-static.openxlab.org.cn/header/openxlab_models.svg
|
||||
[OpenXLab_App-url]: https://openxlab.org.cn/apps/detail/Farewell1/EmoLLMV2.0
|
||||
[OpenXLab_Model-url]: https://openxlab.org.cn/models/detail/ajupyter/EmoLLM_internlm2_7b_full
|
||||
|
||||
## Communication group
|
||||
|
||||
- If it fails, go to the Issue section.
|
||||
|
||||
<p align="center">
|
||||
<img width="30%" src="https://github.com/SmartFlowAI/EmoLLM/assets/62385492/55ecd0aa-4832-4269-ad57-4c26f9aa286b" alt="EmoLLM official communication group">
|
||||
</p>
|
||||
<div align="center">
|
||||
|
||||
# EmoLLM - Large Language Model for Mental Health
|
||||
|
||||
</div>
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/">
|
||||
<img src="assets/EmoLLM_transparent.png" alt="Logo" width="50%">
|
||||
</a>
|
||||
|
||||
<div align="center">
|
||||
|
||||
<!-- PROJECT SHIELDS -->
|
||||
[![Contributors][contributors-shield]][contributors-url]
|
||||
[![Forks][forks-shield]][forks-url]
|
||||
[![Issues][issues-shield]][issues-url]
|
||||
[![OpenXLab_App][OpenXLab_App-image]][OpenXLab_App-url]
|
||||
[![OpenXLab_Model][OpenXLab_Model-image]][OpenXLab_Model-url]
|
||||
[![MIT License][license-shield]][license-url]
|
||||
[![Stargazers][stars-shield]][stars-url]
|
||||
|
||||
</div>
|
||||
|
||||
<h3 align="center">EmoLLM</h3>
|
||||
|
||||
<p align="center">
|
||||
<a href="README.md">简体中文</a> | English
|
||||
<br />
|
||||
<br />
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM"><strong>Explore the documentation of this project »</strong></a>
|
||||
<br />
|
||||
<br />
|
||||
<a href="https://openxlab.org.cn/apps/detail/Farewell1/EmoLLMV2.0">EmoLLM 2.0 Demo</a>
|
||||
·
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/issues">Report a Bug</a>
|
||||
·
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/issues">Propose a New Feature</a>
|
||||
</p>
|
||||
|
||||
</p>
|
||||
|
||||
<!-- 本篇README.md面向开发者 -->
|
||||
|
||||
**EmoLLM** is a series of large language models designed to understand, support and help customers in mental health counseling. It is fine-tuned from the LLM instructions. We really appreciate it if you could give it a star~⭐⭐. The open-sourced configuration is as follows:
|
||||
|
||||
<div align="center">
|
||||
|
||||
| Model | Type | link |
|
||||
| :-------------------: | :--------------: | :---: |
|
||||
| InternLM2_7B_chat | QLORA | [internlm2_7b_chat_qlora_e3.py](./xtuner_config/internlm2_7b_chat_qlora_e3.py) |
|
||||
| InternLM2_7B_chat | full fine-tuning | [internlm2_chat_7b_full.py](./xtuner_config/internlm2_chat_7b_full.py) |
|
||||
| InternLM2_7B_base | QLORA | [internlm2_7b_base_qlora_e10_M_1e4_32_64.py](./xtuner_config/internlm2_7b_base_qlora_e10_M_1e4_32_64.py) |
|
||||
| InternLM2_1_8B_chat | full fine-tuning | [internlm2_1_8b_full_alpaca_e3.py](./xtuner_config/internlm2_1_8b_full_alpaca_e3.py) |
|
||||
| InternLM2_20B_chat | LORA | |
|
||||
| Qwen_7b_chat | QLORA | [qwen_7b_chat_qlora_e3.py](./xtuner_config/qwen_7b_chat_qlora_e3.py) |
|
||||
| Qwen1_5-0_5B-Chat | full fine-tuning | [qwen1_5_0_5_B_full.py](./xtuner_config/qwen1_5_0_5_B_full.py) |
|
||||
| Baichuan2_13B_chat | QLORA | [baichuan2_13b_chat_qlora_alpaca_e3.py](./xtuner_config/baichuan2_13b_chat_qlora_alpaca_e3.py) |
|
||||
| ChatGLM3_6B | LORA | [chatglm3_6b_lora_alpaca_e3.py](./xtuner_config/chatglm3_6b_lora_alpaca_e3.py) |
|
||||
| DeepSeek MoE_16B_chat | QLORA | [deepseek_moe_16b_chat_qlora_oasst1_e3.py](./xtuner_config/deepseek_moe_16b_chat_qlora_oasst1_e3.py) |
|
||||
| Mixtral 8x7B_instruct | QLORA | [mixtral_8x7b_instruct_qlora_oasst1_e3.py](./xtuner_config/mixtral_8x7b_instruct_qlora_oasst1_e3.py) |
|
||||
|
|
||||
| …… | …… | …… |
|
||||
|
||||
</div>
|
||||
|
||||
Everyone is welcome to contribute to this project ~
|
||||
|
||||
---
|
||||
|
||||
The Model aims to fully understand and promote the mental health of individuals, groups, and society. This model typically includes the following key components:
|
||||
|
||||
- Cognitive factors: Involving an individual's thought patterns, belief systems, cognitive biases, and problem-solving abilities. Cognitive factors significantly impact mental health as they affect how individuals interpret and respond to life events.
|
||||
- Emotional factors: Including emotion regulation, emotional expression, and emotional experiences. Emotional health is a crucial part of mental health, involving how individuals manage and express their emotions and how they recover from negative emotions.
|
||||
- Behavioral factors: Concerning an individual's behavior patterns, habits, and coping strategies. This includes stress management skills, social skills, and self-efficacy, which is the confidence in one's abilities.
|
||||
- Social environment: Comprising external factors such as family, work, community, and cultural background, which have direct and indirect impacts on an individual's mental health.
|
||||
- Physical health: There is a close relationship between physical and mental health. Good physical health can promote mental health and vice versa.
|
||||
- Psychological resilience: Refers to an individual's ability to recover from adversity and adapt. Those with strong psychological resilience can bounce back from challenges and learn and grow from them.
|
||||
- Prevention and intervention measures: The Mental Health Grand Model also includes strategies for preventing psychological issues and promoting mental health, such as psychological education, counseling, therapy, and social support systems.
|
||||
- Assessment and diagnostic tools: Effective promotion of mental health requires scientific tools to assess individuals' psychological states and diagnose potential psychological issues.
|
||||
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td align="center" style="background-color: transparent">
|
||||
<img src="assets\aiwei_demo.gif" alt="占位图">
|
||||
</td>
|
||||
<td align="center" style="background-color: transparent">
|
||||
<img src="assets\aiwei_demo2.gif" alt="占位图">
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center" style="background-color: transparent">
|
||||
<img src="assets\aiwei_demo3.gif" alt="占位图">
|
||||
</td>
|
||||
<td align="center" style="background-color: transparent">
|
||||
<img src="assets\aiwei_demo4.gif" alt="占位图">
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
### Recent Updates
|
||||
- 【2024.3.25】 [Mother-like Therapist] is released on Huggingface (https://huggingface.co/brycewang2018/EmoLLM-mother/tree/main)
|
||||
- 【2024.3.25】 [Daddy-like Boy-Friend] is released on Baidu Paddle-Paddle AI Studio Platform (https://aistudio.baidu.com/community/app/68787)
|
||||
- 【2024.3.24】 The **InternLM2-Base-7B QLoRA fine-tuned model** has been released on the **OpenXLab** and **ModelScope** platforms. For more details, please refer to [**InternLM2-Base-7B QLoRA**](./xtuner_config/README_internlm2_7b_base_qlora.md).
|
||||
- 【2024.3.12】 [aiwei] is released on Baidu Paddle-Paddle AI Studio Platform (https://aistudio.baidu.com/community/app/63335)
|
||||
- 【2024.3.11】 **EmoLLM V2.0 is greatly improved in all scores compared to EmoLLM V1.0. Surpasses the performance of Role-playing ChatGPT on counseling tasks!** [Click to experience EmoLLM V2.0](https://openxlab.org.cn/apps/detail/Farewell1/EmoLLMV2.0), update [dataset statistics and details](./datasets/), [Roadmap](./assets/Roadmap_ZH.png)
|
||||
- 【2024.3.9】 Add concurrency acceleration [QA pair generation](./scripts/qa_generation/), [RAG pipeline](./rag/)
|
||||
- 【2024.3.3】 [Based on InternLM2-7B-chat full fine-tuned version EmoLLM V2.0 open sourced](https://openxlab.org.cn/models/detail/ajupyter/EmoLLM_internlm2_7b_full), need two A100*80G, update professional evaluation, see [evaluate](./evaluate/), update PaddleOCR-based PDF to txt tool scripts, see [scripts](./scripts/).
|
||||
- 【2024.2.29】 Updated objective assessment calculations, see [evaluate](./evaluate/) for details. A series of datasets have also been updated, see [datasets](./datasets/) for details.
|
||||
- 【2024.2.27】 Updated English README and a series of datasets (licking dogs and one-round dialogue)
|
||||
- 【2024.2.23】The "Gentle Lady Psychologist Ai Wei" based on InternLM2_7B_chat_qlora was launched. [Click here to obtain the model weights](https://openxlab.org.cn/models/detail/ajupyter/EmoLLM_aiwei), [configuration file](xtuner_config/aiwei-internlm2_chat_7b_qlora.py), [online experience link](https://openxlab.org.cn/apps/detail/ajupyter/EmoLLM-aiwei)
|
||||
|
||||
- 【2024.2.23】Updated [several fine-tuning configurations](/xtuner_config/), added [data_pro.json](/datasets/data_pro.json) (more quantity, more comprehensive scenarios, richer content) and [aiwei.json](/datasets/aiwei.json) (dedicated to the gentle lady role-play, featuring Emoji expressions), the "Gentle Lady Psychologist Ai Wei" is coming soon.
|
||||
|
||||
- 【2024.2.18】 The full fine-tuned version based on Qwen1_5-0_5B-Chat has been [open-sourced](https://www.modelscope.cn/models/aJupyter/EmoLLM_Qwen1_5-0_5B-Chat_full_sft/summary). Friends with limited computational resources can now dive in and explore it.
|
||||
|
||||
<details>
|
||||
<summary>View More</summary>
|
||||
|
||||
- 【2024.2.6】 [Open-sourced based on the Qwen1_5-0_5B-Chat full-scale fine-tuned version](https://www.modelscope.cn/models/aJupyter/EmoLLM_Qwen1_5-0_5B-Chat_full_sft/summary), friends with limited computing power can start experimenting~
|
||||
|
||||
<p align="center">
|
||||
<img src="https://github.com/SmartFlowAI/EmoLLM/assets/62385492/7e931682-c54d-4ded-bc67-79130c68d744" alt="模型下载量">
|
||||
</p>
|
||||
|
||||
- 【2024.2.5】 The project has been promoted by the official WeChat account NLP Engineering. Here's the [link](https://mp.weixin.qq.com/s/78lrRl2tlXEKUfElnkVx4A) to the article. Welcome everyone to follow!! 🥳🥳
|
||||
|
||||
<p align="center">
|
||||
<img src="https://github.com/SmartFlowAI/EmoLLM/assets/62385492/47868d6a-2e91-4aa9-a630-e594c14295b4" alt="公众号二维码">
|
||||
</p>
|
||||
|
||||
- 【2024.2.3】 [Project Vedio](https://www.bilibili.com/video/BV1N7421N76X/) at bilibili 😊
|
||||
- 【2024.1.27】 Complete data construction documentation, fine-tuning guide, deployment guide, Readme, and other related documents 👏
|
||||
- 【2024.1.25】 EmoLLM V1.0 has deployed online https://openxlab.org.cn/apps/detail/jujimeizuo/EmoLLM 😀
|
||||
|
||||
</details>
|
||||
|
||||
### Honors
|
||||
|
||||
- The project won the ***the Innovation and Creativity Award*** in the **2024 Puyuan Large Model Series Challenge Spring Competition held by the Shanghai Artificial Intelligence Laboratory**
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/">
|
||||
<img src="assets/Shusheng.png" alt="Challenge Innovation and Creativity Award">
|
||||
</p>
|
||||
|
||||
|
||||
- The project has been promoted by the official WeChat account **NLP Engineering**. Here's the [link](https://mp.weixin.qq.com/s/78lrRl2tlXEKUfElnkVx4A).
|
||||
|
||||
### Roadmap
|
||||
|
||||
<p align="center">
|
||||
<a href="https://github.com/SmartFlowAI/EmoLLM/">
|
||||
<img src="assets/Roadmap_EN.png" alt="Roadmap_EN">
|
||||
</a>
|
||||
|
||||
## Contents
|
||||
|
||||
- [EmoLLM - Large Language Model for Mental Health](#emollm---large-language-model-for-mental-health)
|
||||
- [Recent Updates](#recent-updates)
|
||||
- [Honors](#honors)
|
||||
- [Roadmap](#roadmap)
|
||||
- [Contents](#contents)
|
||||
- [Pre-development Configuration Requirements.](#pre-development-configuration-requirements)
|
||||
- [**User Guide**](#user-guide)
|
||||
- [🍪Quick start](#quick-start)
|
||||
- [📌Data Construction](#data-construction)
|
||||
- [🎨Fine-tuning Guide](#fine-tuning-guide)
|
||||
- [🔧Deployment Guide](#deployment-guide)
|
||||
- [⚙RAG (Retrieval Augmented Generation) Pipeline](#rag-retrieval-augmented-generation-pipeline)
|
||||
- [Frameworks Used](#frameworks-used)
|
||||
- [How to participate in this project](#how-to-participate-in-this-project)
|
||||
- [Version control](#version-control)
|
||||
- [Authors (in no particular order)](#authors-in-no-particular-order)
|
||||
- [Copyright Notice](#copyright-notice)
|
||||
- [Acknowledgments](#acknowledgments)
|
||||
- [Star History](#star-history)
|
||||
- [🌟 Contributors](#-contributors)
|
||||
- [Communication group](#communication-group)
|
||||
|
||||
###### Pre-development Configuration Requirements.
|
||||
|
||||
- A100 40G (specifically for InternLM2_7B_chat + qlora fine-tuning + deepspeed zero2 optimization)
|
||||
|
||||
###### **User Guide**
|
||||
|
||||
1. Clone the repo
|
||||
|
||||
```sh
|
||||
git clone https://github.com/SmartFlowAI/EmoLLM.git
|
||||
```
|
||||
|
||||
1. Read in sequence or read sections you're interested in：
|
||||
- [Quick Start](#quick-start)
|
||||
- [Data Construction](#data-construction)
|
||||
- [Fine-tuning Guide](#fine-tuning-guide)
|
||||
- [Deployment Guide](#deployment-guide)
|
||||
- [RAG](#rag-retrieval-augmented-generation-pipeline)
|
||||
- View More Details
|
||||
|
||||
|
||||
### 🍪Quick start
|
||||
- Please read [Quick Start](docs/quick_start_EN.md) to see.
|
||||
|
||||
### 📌Data Construction
|
||||
|
||||
- Please read the [Data Construction Guide ](generate_data/tutorial_EN.md)for reference.
|
||||
|
||||
- The dataset used for this fine-tuning can be found at [datasets](datasets/data.json)
|
||||
|
||||
### 🎨Fine-tuning Guide
|
||||
|
||||
For details, see the [fine-tuning guide](xtuner_config/README_EN.md)
|
||||
|
||||
### 🔧Deployment Guide
|
||||
|
||||
- Demo deployment: see [deployment guide](./demo/README_EN.md) for details.
|
||||
- Quantitative deployment based on [LMDeploy](https://github.com/InternLM/lmdeploy/): see [deploy](./deploy/lmdeploy_EN.md)
|
||||
|
||||
### ⚙RAG (Retrieval Augmented Generation) Pipeline
|
||||
|
||||
- See [RAG](./rag/)
|
||||
|
||||
<details>
|
||||
<summary>Additional Details</summary>
|
||||
|
||||
### Frameworks Used
|
||||
|
||||
- [Xtuner](https://github.com/InternLM/xtuner)
|
||||
- [Transformers](https://github.com/huggingface/transformers)
|
||||
- [Pytorch](https://pytorch.org/)
|
||||
- [LMDeploy](https://github.com/InternLM/lmdeploy/): for quantitative deployment
|
||||
- [Stremlit](https://streamlit.io/): for building demos
|
||||
- [DeepSpeed](https://github.com/microsoft/DeepSpeed): for parallel training
|
||||
- …
|
||||
|
||||
#### How to participate in this project
|
||||
|
||||
Contributions make the open-source community an excellent place for learning, inspiration, and creation. Any contribution you make is greatly appreciated.
|
||||
|
||||
1. Fork the Project
|
||||
2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)
|
||||
3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)
|
||||
4. Push to the Branch (`git push origin feature/AmazingFeature`)
|
||||
5. Open a Pull Request
|
||||
|
||||
### Version control
|
||||
|
||||
This project uses Git for version control. You can see the currently available versions in the repository.
|
||||
|
||||
</details>
|
||||
|
||||
### Authors (in no particular order)
|
||||
|
||||
| Username | School/Organization | Remarks | Contributions |
|
||||
| :-----------------------------------------------------------: | :------------------------------------------------------------------: | :-----------------------------------------------------------------------: | :-----------------------------------------------------------------------------------: |
|
||||
| [aJupyter](https://github.com/aJupyter) | Nankai University, Master's student | DataWhale member | Project initiator |
|
||||
| [MING-ZCH](https://github.com/MING-ZCH) | Huazhong University of Science and Technology, Undergraduate student | LLM X Psychology researcher | Project co-leader |
|
||||
| [jujimeizuo](https://github.com/jujimeizuo) | Jiangnan University, Master's student | | |
|
||||
| [Smiling-Weeping-zhr](https://github.com/Smiling-Weeping-zhr) | Harbin Institute of Technology (Weihai), Undergraduate student | | |
|
||||
| [8baby8](https://github.com/8baby8) | PaddlePaddle Pilot Team Regional Director | Wenxin Large Model core developer | |
|
||||
| [zxazys](https://github.com/zxazys) | Nankai University, Master's student | | |
|
||||
| [JasonLLLLLLLLLLL](https://github.com/JasonLLLLLLLLLLL) | SWUFE (Southwestern University of Finance and Economics) | | |
|
||||
| [MrCatAI](https://github.com/MrCatAI) | AI Mover | | |
|
||||
| [ZeyuBa](https://github.com/ZeyuBa) | Institute of Automation, Master's student | | |
|
||||
| [aiyinyuedejustin](https://github.com/aiyinyuedejustin) | University of Pennsylvania, Master's student | | |
|
||||
| [Nobody-ML](https://github.com/Nobody-ML) | China University of Petroleum (East China), Undergraduate student | | |
|
||||
| [chg0901](https://github.com/chg0901) | [MiniSora](https://github.com/mini-sora/minisora) | Maintainer and Admin of [MiniSora](https://github.com/mini-sora/minisora) | LLM Pre-Training and Fine-Tuning, Model Uploading, Data Cleaning and Docs Translation |
|
||||
| [Mxoder](https://github.com/Mxoder) | Beihang University, Undergraduate student | | |
|
||||
| [Anooyman](https://github.com/Anooyman) | Nanjing University of Science and Technology, Master's student | | |
|
||||
| [Vicky-3021](https://github.com/Vicky-3021) | Xidian University, Master's student (Research Year 0) | | |
|
||||
| [SantiagoTOP](https://github.com/santiagoTOP) | Taiyuan University of Technology, Master's student | | |
|
||||
| [zealot52099](https://github.com/zealot52099) | Individual developer | | Data Processing, LLM finetuning and RAG |
|
||||
| [wwwyfff](https://github.com/wwwyfff) | FuDan University, Master's student | | |
|
||||
| [jkhumor](https://github.com/jkhumor) | Nankai University, Master's student | | RAG |
|
||||
| [lll997150986](https://github.com/lll997150986) | Nankai University, Master's student | | Fine Tuning |
|
||||
| [nln-maker](https://github.com/nln-maker) | Nankai University, Master's student | | Front-end and back-end development |
|
||||
| [dream00001](https://github.com/dream00001) | Nankai University, Master's student | | Front-end and back-end development |
|
||||
| [王几行XING](zhihu.com/people/brycewang1898) | Peking University, Master's graduate | | Data Processing, LLM finetuning, Front-end and back-end development |
|
||||
| [思在] | Peking University, Master's graduate (Microsoft) | | LLM finetuning, Front-end and back-end development |
|
||||
|
||||
### Copyright Notice
|
||||
|
||||
The project is licensed under the MIT License. Please refer to the details
|
||||
[LICENSE](https://github.com/SmartFlowAI/EmoLLM/blob/master/LICENSE)
|
||||
|
||||
### Acknowledgments
|
||||
|
||||
- [Sanbu](https://github.com/sanbuphy)
|
||||
- [Shanghai Artificial Intelligence Laboratory](https://www.shlab.org.cn/)
|
||||
- [Vanin](https://github.com/vansin)
|
||||
- [Bloom up (WeChat Official Account Promotion)](https://mp.weixin.qq.com/s/78lrRl2tlXEKUfElnkVx4A)
|
||||
- Abu (M.A. in Psychology, Peking University)
|
||||
- [HatBoy](https://github.com/hatboy)
|
||||
|
||||
<!-- links -->
|
||||
|
||||
<!-- [linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=flat-square&logo=linkedin&colorB=555 -->
|
||||
|
||||
<!-- [linkedin-url]: https://linkedin.com/in/aJupyter -->
|
||||
|
||||
<!-- 太少了，没必要放 -->
|
||||
|
||||
## Star History
|
||||
|
||||
[](https://star-history.com/#SmartFlowAI/EmoLLM&Date)
|
||||
|
||||
## 🌟 Contributors
|
||||
|
||||
[](https://github.com/SmartFlowAI/EmoLLM/graphs/contributors)
|
||||
|
||||
[your-project-path]: SmartflowAI/EmoLLM
|
||||
[contributors-shield]: https://img.shields.io/github/contributors/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[contributors-url]: https://github.com/SmartflowAI/EmoLLM/graphs/contributors
|
||||
[forks-shield]: https://img.shields.io/github/forks/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[forks-url]: https://github.com/SmartflowAI/EmoLLM/network/members
|
||||
[stars-shield]: https://img.shields.io/github/stars/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[stars-url]: https://github.com/SmartflowAI/EmoLLM/stargazers
|
||||
[issues-shield]: https://img.shields.io/github/issues/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[issues-url]: https://img.shields.io/github/issues/SmartflowAI/EmoLLM.svg
|
||||
[license-shield]: https://img.shields.io/github/license/SmartflowAI/EmoLLM.svg?style=flat-square
|
||||
[license-url]: https://github.com/SmartflowAI/EmoLLM/blob/main/LICENSE
|
||||
|
||||
[OpenXLab_App-image]: https://cdn-static.openxlab.org.cn/app-center/openxlab_app.svg
|
||||
[OpenXLab_Model-image]: https://cdn-static.openxlab.org.cn/header/openxlab_models.svg
|
||||
[OpenXLab_App-url]: https://openxlab.org.cn/apps/detail/Farewell1/EmoLLMV2.0
|
||||
[OpenXLab_Model-url]: https://openxlab.org.cn/models/detail/ajupyter/EmoLLM_internlm2_7b_full
|
||||
|
||||
## Communication group
|
||||
|
||||
- If it fails, go to the Issue section.
|
||||
|
||||
<p align="center">
|
||||
<img width="30%" src="https://github.com/SmartFlowAI/EmoLLM/assets/62385492/55ecd0aa-4832-4269-ad57-4c26f9aa286b" alt="EmoLLM official communication group">
|
||||
</p>
|
||||
4
app.py
4
app.py
@ -1,3 +1,3 @@
|
||||
import os
|
||||
# os.system('streamlit run web_internlm2.py --server.address=0.0.0.0 --server.port 7860')
|
||||
os.system('streamlit run web_demo-aiwei.py --server.address=0.0.0.0 --server.port 7860')
|
||||
os.system('streamlit run web_internlm2.py --server.address=0.0.0.0 --server.port 7860')
|
||||
#os.system('streamlit run web_demo-aiwei.py --server.address=0.0.0.0 --server.port 7860')
|
||||
|
||||
BIN
assets/EmoLLM_transparent.png
Normal file
BIN
assets/EmoLLM_transparent.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 67 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 120 KiB |
BIN
assets/Shusheng.png
Normal file
BIN
assets/Shusheng.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 3.4 MiB |
BIN
assets/aiwei_demo.gif
Normal file
BIN
assets/aiwei_demo.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.6 MiB |
BIN
assets/aiwei_demo2.gif
Normal file
BIN
assets/aiwei_demo2.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.0 MiB |
BIN
assets/aiwei_demo3.gif
Normal file
BIN
assets/aiwei_demo3.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.7 MiB |
BIN
assets/aiwei_demo4.gif
Normal file
BIN
assets/aiwei_demo4.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 518 KiB |
BIN
assets/model.png
Normal file
BIN
assets/model.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 296 KiB |
@ -2,7 +2,7 @@
|
||||
|
||||
* 数据集按用处分为两种类型：**General** 和 **Role-play**
|
||||
* 数据按格式分为两种类型：**QA** 和 **Conversation**
|
||||
* 数据汇总：General（**6个数据集**）；Role-play（**3个数据集**）
|
||||
* 数据汇总：General（**6个数据集**）；Role-play（**5个数据集**）
|
||||
|
||||
## 数据集类型
|
||||
|
||||
@ -19,32 +19,36 @@
|
||||
| Category | Dataset | Type | Total |
|
||||
| :---------: | :-------------------: | :----------: | :-----: |
|
||||
| *General* | data | Conversation | 5600+ |
|
||||
| *General* | data_pro | Conversation | 36500+ |
|
||||
| *General* | data_pro | Conversation | 36,500+ |
|
||||
| *General* | multi_turn_dataset_1 | Conversation | 36,000+ |
|
||||
| *General* | multi_turn_dataset_2 | Conversation | 27,000+ |
|
||||
| *General* | single_turn_dataset_1 | QA | 14000+ |
|
||||
| *General* | single_turn_dataset_2 | QA | 18300+ |
|
||||
| *General* | single_turn_dataset_1 | QA | 14,000+ |
|
||||
| *General* | single_turn_dataset_2 | QA | 18,300+ |
|
||||
| *Role-play* | aiwei | Conversation | 4000+ |
|
||||
| *Role-play* | SoulStar | QA | 11200+ |
|
||||
| *Role-play* | SoulStar | QA | 11,200+ |
|
||||
| *Role-play* | tiangou | Conversation | 3900+ |
|
||||
| *Role-play* | mother | Conversation | 40,300+ |
|
||||
| *Role-play* | scientist | Conversation | 28,400+ |
|
||||
| …… | …… | …… | …… |
|
||||
|
||||
## 数据集来源
|
||||
|
||||
### **General**
|
||||
|
||||
* 数据集 data 来自本项目
|
||||
* 数据集 data_pro 来自本项目
|
||||
* 数据集 multi_turn_dataset_1 来源 [Smile](https://github.com/qiuhuachuan/smile)
|
||||
* 数据集 multi_turn_dataset_2 来源 [CPsyCounD](https://github.com/CAS-SIAT-XinHai/CPsyCoun)
|
||||
* 数据集 single_turn_dataset_1 来自本项目
|
||||
* 数据集 single_turn_dataset_2 来自本项目
|
||||
* 数据集 `data` 来自本项目
|
||||
* 数据集 `data_pro` 来自本项目
|
||||
* 数据集 `multi_turn_dataset_1` 来源 [Smile](https://github.com/qiuhuachuan/smile)
|
||||
* 数据集 `multi_turn_dataset_2` 来源 [CPsyCounD](https://github.com/CAS-SIAT-XinHai/CPsyCoun)
|
||||
* 数据集 `single_turn_dataset_1` 来自本项目
|
||||
* 数据集 `single_turn_dataset_2` 来自本项目
|
||||
|
||||
### **Role-play**
|
||||
|
||||
* 数据集 aiwei 来自本项目
|
||||
* 数据集 tiangou 来自本项目
|
||||
* 数据集 SoulStar 来源 [SoulStar](https://github.com/Nobody-ML/SoulStar)
|
||||
* 数据集 `aiwei` 来自本项目
|
||||
* 数据集 `tiangou` 来自本项目
|
||||
* 数据集 `SoulStar` 来源 [SoulStar](https://github.com/Nobody-ML/SoulStar)
|
||||
* 数据集 `mother` 来自本项目
|
||||
* 数据集 `scientist` 来自本项目
|
||||
|
||||
## 数据集去重
|
||||
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
|
||||
* Category of dataset: **General** and **Role-play**
|
||||
* Type of data: **QA** and **Conversation**
|
||||
* Summary: General(**6 datasets**), Role-play(**3 datasets**)
|
||||
* Summary: General(**6 datasets**), Role-play(**5 datasets**)
|
||||
|
||||
## Category
|
||||
* **General**: generic dataset, including psychological Knowledge, counseling technology, etc.
|
||||
@ -17,14 +17,16 @@
|
||||
| Category | Dataset | Type | Total |
|
||||
| :---------: | :-------------------: | :----------: | :-----: |
|
||||
| *General* | data | Conversation | 5600+ |
|
||||
| *General* | data_pro | Conversation | 36500+ |
|
||||
| *General* | data_pro | Conversation | 36,500+ |
|
||||
| *General* | multi_turn_dataset_1 | Conversation | 36,000+ |
|
||||
| *General* | multi_turn_dataset_2 | Conversation | 27,000+ |
|
||||
| *General* | single_turn_dataset_1 | QA | 14000+ |
|
||||
| *General* | single_turn_dataset_2 | QA | 18300+ |
|
||||
| *General* | single_turn_dataset_1 | QA | 14,000+ |
|
||||
| *General* | single_turn_dataset_2 | QA | 18,300+ |
|
||||
| *Role-play* | aiwei | Conversation | 4000+ |
|
||||
| *Role-play* | SoulStar | QA | 11200+ |
|
||||
| *Role-play* | SoulStar | QA | 11,200+ |
|
||||
| *Role-play* | tiangou | Conversation | 3900+ |
|
||||
| *Role-play* | mother | Conversation | 40,300+ |
|
||||
| *Role-play* | scientist | Conversation | 28,400+ |
|
||||
| …… | …… | …… | …… |
|
||||
|
||||
|
||||
@ -41,8 +43,10 @@
|
||||
* dataset `aiwei` from this repo
|
||||
* dataset `tiangou` from this repo
|
||||
* dataset `SoulStar` from [SoulStar](https://github.com/Nobody-ML/SoulStar)
|
||||
* dataset `mother` from this repo
|
||||
* dataset `scientist` from this repo
|
||||
|
||||
**Dataset Deduplication**：
|
||||
Combine absolute matching with fuzzy matching (Simhash) algorithms to deduplicate the dataset, thereby enhancing the effectiveness of the fine-tuning model. While ensuring the high quality of the dataset, the risk of losing important data due to incorrect matches can be reduced via adjusting the threshold.
|
||||
|
||||
https://algonotes.readthedocs.io/en/latest/Simhash.html
|
||||
https://algonotes.readthedocs.io/en/latest/Simhash.html
|
||||
|
||||
118972
datasets/mother_v2.json
Normal file
118972
datasets/mother_v2.json
Normal file
File diff suppressed because it is too large
Load Diff
@ -1,44 +1,7 @@
|
||||
# EmoLLM 部署指南
|
||||
|
||||
## 本地部署
|
||||
|
||||
- Clone repo
|
||||
|
||||
```bash
|
||||
git clone https://github.com/aJupyter/EmoLLM.git
|
||||
```
|
||||
|
||||
- 安装依赖
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
- 下载模型
|
||||
- 模型权重：https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model
|
||||
- 通过 openxlab.model.download 下载，详情请看 [cli_internlm2](./cli_internlm2.py)
|
||||
|
||||
```bash
|
||||
from openxlab.model import download
|
||||
|
||||
download(model_repo='jujimeizuo/EmoLLM_Model', output='model')
|
||||
```
|
||||
|
||||
- 可以手动下载，放在 `./model` 目录下，然后把上面的代码删掉
|
||||
|
||||
- cli_demo
|
||||
|
||||
```bash
|
||||
python ./demo/cli_internlm2.py
|
||||
```
|
||||
|
||||
- web_demo
|
||||
|
||||
```bash
|
||||
python ./app.py
|
||||
```
|
||||
|
||||
如果在服务器上部署，需要配置本地端口映射
|
||||
- 详见[快速体验](../docs/quick_start.md)
|
||||
|
||||
## OpenXLab 上部署
|
||||
|
||||
|
||||
@ -1,44 +1,7 @@
|
||||
# Deploying Guide for EmoLLM
|
||||
|
||||
## Local Deployment
|
||||
|
||||
- Clone repo
|
||||
|
||||
```bash
|
||||
git clone https://github.com/aJupyter/EmoLLM.git
|
||||
```
|
||||
|
||||
- Install dependencies
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
- Download the model
|
||||
- Model weights：https://openxlab.org.cn/models/detail/jujimeizuo/EmoLLM_Model
|
||||
- Download via openxlab.model.download, see [cli_internlm2](./cli_internlm2.py) for details
|
||||
|
||||
```bash
|
||||
from openxlab.model import download
|
||||
|
||||
download(model_repo='jujimeizuo/EmoLLM_Model', output='model')
|
||||
```
|
||||
|
||||
- You can also download manually and place it in the `./model` directory, then delete the above code.
|
||||
|
||||
- cli_demo
|
||||
|
||||
```bash
|
||||
python ./demo/cli_internlm2.py
|
||||
```
|
||||
|
||||
- web_demo
|
||||
|
||||
```bash
|
||||
python ./app.py
|
||||
```
|
||||
|
||||
If deploying on a server, you need to configure local port mapping.
|
||||
- Please read [Quick Start](../docs/quick_start_EN.md) to see.
|
||||
|
||||
## Deploy on OpenXLab
|
||||
|
||||
|
||||
37
docs/quick_start.md
Normal file
37
docs/quick_start.md
Normal file
@ -0,0 +1,37 @@
|
||||
### 1、部署环境
|
||||
- 操作系统：Ubuntu 22.04.4 LTS
|
||||
- CPU：Intel (R) Xeon (R) CPU E 5-2650，32G（在线 GPU 服务器）
|
||||
- 显卡：NVIDIA RTX 4060Ti 16G，NVIDIA-SMI 535.104.05 Driver Version: 535.104.05 CUDA Version: 12.2
|
||||
- Python 3.11.5
|
||||
|
||||
### 2、默认部署步骤
|
||||
- 1、Clone 代码或者手动下载代码放置服务器：
|
||||
```
|
||||
git clone https://github.com/SmartFlowAI/EmoLLM.git
|
||||
```
|
||||
- 2、安装 Python 依赖库：
|
||||
```
|
||||
# cd EmoLLM
|
||||
# pip install -r requirements.txt
|
||||
```
|
||||
- 3、下载模型文件，可手动下载，也可运行download_model.py 脚本自动下载模型文件。
|
||||
- 3.1、自动下载模型文件，运行脚本：
|
||||
```
|
||||
# python download_model.py <model_repo>
|
||||
|
||||
# 运行 web_demo-aiwei.py 脚本对应的模型仓库地址是 ajupyter/EmoLLM_aiwei，即：
|
||||
# python download_model.py ajupyter/EmoLLM_aiwei
|
||||
|
||||
# 运行 web_internlm2.py 脚本对应的模型仓库地址是 jujimeizuo/EmoLLM_Model，即：
|
||||
# python download_model.py jujimeizuo/EmoLLM_Model
|
||||
|
||||
# 也可用该脚本自动下载其他模型。该脚本当前仅支持openxlab平台的模型自动下载，其他平台的模型需要手动下载。下载成功后可看到EmoLLM目录下新增 model 目录，即模型文件目录。
|
||||
```
|
||||
- 3.2、手动下载模型文件目录，去 openxlab、Huggingface等平台下载完整的模型目录文件，将全部文件放在 `EmoLLM/model` 目录下。注意，模型文件目录打包下载时并不会下载 LFS 文件（如 pytorch_model-00001-of-00008.bin），需要挨个下载完整的 LFS 文件。
|
||||
|
||||
- 4、运行脚本，app.py仅用于调用web_demo-aiwei.py 或者 web_internlm2.py 文件，想运行哪一个脚本就下载对应脚本的模型文件，然后在app.py中注释另一个脚本即可。然后运行脚本：
|
||||
```
|
||||
python app.py
|
||||
```
|
||||
5、运行 app.py 后通过浏览器访问： http://0.0.0.0:7860 地址访问模型 web 页面。可修改 app.py 文件修改 web 页面访问端口，即可正常体验该模型。如果在服务器上部署，需要配置本地端口映射。
|
||||
6、替换模型，EmoLLM 提供了多种开源模型，分别上传至 openxlab、Huggingface 平台，有[爹系男友心理咨询师 ](https://openxlab.org.cn/models/detail/chg0901/EmoLLM_Daddy-like_BF)、[老母亲心理咨询师](https://huggingface.co/brycewang2018/EmoLLM-mother/tree/main)、[温柔御姐心理医生艾薇](https://openxlab.org.cn/models/detail/ajupyter/EmoLLM_aiwei)等角色，有 EmoLLM_internlm2_7b_full、EmoLLM-InternLM7B-base-10e 等多个模型可选择。可重复步骤 3、4手动或自动下载相关模型放在 `EmoLLM/model` 目录下，然后运行体验。
|
||||
37
docs/quick_start_EN.md
Normal file
37
docs/quick_start_EN.md
Normal file
@ -0,0 +1,37 @@
|
||||
### 1. Deployment Environment
|
||||
- Operating system: Ubuntu 22.04.4 LTS
|
||||
- CPU: Intel (R) Xeon (R) CPU E 5-2650, 32G
|
||||
- Graphics card: NVIDIA RTX 4060Ti 16G, NVIDIA-SMI 535.104.05 Driver Version: 535.104.05 CUDA Version: 12.2
|
||||
- Python 3.11.5
|
||||
|
||||
### 2. Default Deployment Steps
|
||||
- 1. Clone the code or manually download the code and place it on the server:
|
||||
```
|
||||
git clone https://github.com/SmartFlowAI/EmoLLM.git
|
||||
```
|
||||
- 2. Install Python dependencies:
|
||||
```
|
||||
# cd EmoLLM
|
||||
# pip install -r requirements.txt
|
||||
```
|
||||
- 3. Download the model files, either manually or by running the download_model.py script.
|
||||
- 3.1. Automatically download the model file and run the script:
|
||||
```
|
||||
# python download_model.py <model_repo>
|
||||
|
||||
# Run the web_demo-aiwei.py script to run the model repository at ajupyter/EmoLLM_aiwei, ie:
|
||||
# python download_model.py ajupyter/EmoLLM_aiwei
|
||||
|
||||
# Run the web_internlm2.py script to run the model repository at jujimeizuo/EmoLLM_Mode, ie:
|
||||
# python download_model.py jujimeizuo/EmoLLM_Model
|
||||
|
||||
# This script can also be used to automatically download other models. This script only supports automatic download of models from openxlab platform, models from other platforms need to be downloaded manually. After successful download, you can see a new model directory under EmoLLM directory, i.e. the model file directory.
|
||||
```
|
||||
- 3.2. To download the model file directory manually, go to openxlab, Huggingface, etc. to download the complete model directory file, and put all the files in the `EmoLLM/model` directory. Note that the LFS file (e.g. pytorch_model-00001-of-00008.bin) is not downloaded when the model file directory is packaged for download, so you need to download the full LFS file one by one.
|
||||
|
||||
- 4. Run the script, app.py is only used to call web_demo-aiwei.py or web_internlm2.py file, you can download the model file of the corresponding script for whichever script you want to run, and then comment the other script in app.py. Then run the script:
|
||||
```
|
||||
python app.py
|
||||
```
|
||||
- 5. After running app.py, you can access the model's web page through your browser at the following address: http://0.0.0.0:7860. You can modify the app.py file to change the web page access port to experience the model normally. If you are deploying on a server, you need to configure local port mapping.
|
||||
- 6. Use of other models, EmoLLM offers several versions of the open source model, uploaded to openxlab、Huggingface, and other platforms. There are roles such as the [father's boyfriend counselor](https://openxlab.org.cn/models/detail/chg0901/EmoLLM_Daddy-like_BF), [the old mother's counselor](https://huggingface.co/brycewang2018/EmoLLM-mother/tree/main), and [the gentle royal psychiatrist](https://openxlab.org.cn/models/detail/ajupyter/EmoLLM_aiwei). There are several models to choose from such as EmoLLM_internlm2_7b_full, EmoLLM-InternLM7B-base-10e and so on. Repeat steps 3 and 4 to manually or automatically download the model in the `EmoLLM/model` directory and run the experience.
|
||||
63
download_model.py
Normal file
63
download_model.py
Normal file
@ -0,0 +1,63 @@
|
||||
import requests
|
||||
import os
|
||||
import sys
|
||||
import shutil
|
||||
import zipfile
|
||||
from openxlab.model import download
|
||||
|
||||
"""
|
||||
Automatic download of model files from openxlab.
|
||||
Currently only support openxlab automatic download, other platform model files need to be downloaded manually.
|
||||
"""
|
||||
|
||||
if len(sys.argv) == 2:
|
||||
model_repo = sys.argv[1]
|
||||
else:
|
||||
print("Usage: python download_model.py <model_repo>")
|
||||
print("Example: python download_model.py jujimeizuo/EmoLLM_Model")
|
||||
exit()
|
||||
|
||||
dir_name = "model"
|
||||
|
||||
if os.path.isdir(dir_name):
|
||||
print("model file exist")
|
||||
exit(0)
|
||||
|
||||
download_url = "https://code.openxlab.org.cn/api/v1/repos/{}/archive/main.zip".format(model_repo)
|
||||
output_filename = "model_main.zip"
|
||||
|
||||
# download model file
|
||||
response = requests.get(download_url, stream=True)
|
||||
if response.status_code == 200:
|
||||
with open(output_filename, "wb") as f:
|
||||
for chunk in response.iter_content(chunk_size=1024):
|
||||
if chunk: # filter out keep-alive new chunks
|
||||
f.write(chunk)
|
||||
print(f"Successfully downloaded model file")
|
||||
else:
|
||||
print(f"Failed to download the model file. HTTP status code: {response.status_code}")
|
||||
exit()
|
||||
|
||||
if not os.path.isfile(output_filename):
|
||||
raise FileNotFoundError(f"ZIP file '{output_filename}' not found in the current directory.")
|
||||
|
||||
temp_dir = f".{os.sep}temp_{os.path.splitext(os.path.basename(output_filename))[0]}"
|
||||
os.makedirs(temp_dir, exist_ok=True)
|
||||
|
||||
with zipfile.ZipFile(output_filename, 'r') as zip_ref:
|
||||
zip_ref.extractall(temp_dir)
|
||||
|
||||
top_level_dir = next(os.walk(temp_dir))[1][0]
|
||||
|
||||
|
||||
source_dir = os.path.join(temp_dir, top_level_dir)
|
||||
destination_dir = os.path.join(os.getcwd(), dir_name)
|
||||
shutil.move(source_dir, destination_dir)
|
||||
|
||||
os.rmdir(temp_dir)
|
||||
|
||||
os.remove(output_filename)
|
||||
|
||||
download(model_repo='jujimeizuo/EmoLLM_Model', output='model')
|
||||
|
||||
print("Model bin file download complete")
|
||||
@ -48,3 +48,5 @@ pip install torch transformers datasets nltk rouge jieba
|
||||
| Qwen1_5-0_5B-chat | 27.23% | 8.55% | 17.05% | 26.65% | 13.11% | 7.19% | 4.05% |
|
||||
| InternLM2_7B_chat_qlora | 37.86% | 15.23% | 24.34% | 39.71% | 22.66% | 14.26% | 9.21% |
|
||||
| InternLM2_7B_chat_full | 32.45% | 10.82% | 20.17% | 30.48% | 15.67% | 8.84% | 5.02% |
|
||||
| InternLM2_7B_base_qlora_5epoch | 41.94% | 20.21% | 29.67% | 42.98% | 27.07% | 19.33% | 14.62% |
|
||||
| InternLM2_7B_base_qlora_10epoch | 43.47% | 22.06% | 31.4% | 44.81% | 29.15% | 21.44% | 16.72% |
|
||||
@ -48,3 +48,5 @@ Test the data in data.json with the following results:
|
||||
| Qwen1_5-0_5B-chat | 27.23% | 8.55% | 17.05% | 26.65% | 13.11% | 7.19% | 4.05% |
|
||||
| InternLM2_7B_chat_qlora | 37.86% | 15.23% | 24.34% | 39.71% | 22.66% | 14.26% | 9.21% |
|
||||
| InternLM2_7B_chat_full | 32.45% | 10.82% | 20.17% | 30.48% | 15.67% | 8.84% | 5.02% |
|
||||
| InternLM2_7B_base_qlora_5epoch | 41.94% | 20.21% | 29.67% | 42.98% | 27.07% | 19.33% | 14.62% |
|
||||
| InternLM2_7B_base_qlora_10epoch | 43.47% | 22.06% | 31.4% | 44.81% | 29.15% | 21.44% | 16.72% |
|
||||
|
||||
@ -9,6 +9,8 @@
|
||||
| Qwen1_5-0_5B-chat | 27.23% | 8.55% | 17.05% | 26.65% | 13.11% | 7.19% | 4.05% |
|
||||
| InternLM2_7B_chat_qlora | 37.86% | 15.23% | 24.34% | 39.71% | 22.66% | 14.26% | 9.21% |
|
||||
| InternLM2_7B_chat_full | 32.45% | 10.82% | 20.17% | 30.48% | 15.67% | 8.84% | 5.02% |
|
||||
| InternLM2_7B_base_qlora_5epoch | 41.94% | 20.21% | 29.67% | 42.98% | 27.07% | 19.33% | 14.62% |
|
||||
| InternLM2_7B_base_qlora_10epoch | 43.47% | 22.06% | 31.4% | 44.81% | 29.15% | 21.44% | 16.72% |
|
||||
|
||||
## 专业指标评测
|
||||
|
||||
@ -18,4 +20,4 @@
|
||||
|-------------------|-----------------------|-------------------|-----------------|---------|
|
||||
| InternLM2_7B_chat_qlora | 1.32 | 2.20 | 2.10 | 1.00 |
|
||||
| InternLM2_7B_chat_full | 1.40 | 2.45 | 2.24 | 1.00 |
|
||||
|
||||
| InternLM2_20B_chat_lora | 1.42 | 2.39 | 2.22 | 1.00 |
|
||||
|
||||
@ -9,6 +9,8 @@
|
||||
| Qwen1_5-0_5B-chat | 27.23% | 8.55% | 17.05% | 26.65% | 13.11% | 7.19% | 4.05% |
|
||||
| InternLM2_7B_chat_qlora | 37.86% | 15.23% | 24.34% | 39.71% | 22.66% | 14.26% | 9.21% |
|
||||
| InternLM2_7B_chat_full | 32.45% | 10.82% | 20.17% | 30.48% | 15.67% | 8.84% | 5.02% |
|
||||
| InternLM2_7B_base_qlora_5epoch | 41.94% | 20.21% | 29.67% | 42.98% | 27.07% | 19.33% | 14.62% |
|
||||
| InternLM2_7B_base_qlora_10epoch | 43.47% | 22.06% | 31.4% | 44.81% | 29.15% | 21.44% | 16.72% |
|
||||
|
||||
## Professional Metrics Evaluation
|
||||
|
||||
@ -17,3 +19,5 @@
|
||||
| Model | Comprehensiveness | rofessionalism | Authenticity | Safety |
|
||||
|-------------------|-----------------------|-------------------|-----------------|---------|
|
||||
| InternLM2_7B_chat_qlora | 1.32 | 2.20 | 2.10 | 1.00 |
|
||||
| InternLM2_7B_chat_full | 1.40 | 2.45 | 2.24 | 1.00 |
|
||||
| InternLM2_20B_chat_lora | 1.42 | 2.39 | 2.22 | 1.00 |
|
||||
|
||||
@ -7,4 +7,6 @@ accelerate==0.24.1
|
||||
transformers_stream_generator==0.0.4
|
||||
openxlab
|
||||
tiktoken
|
||||
einops
|
||||
einops
|
||||
oss2
|
||||
requests
|
||||
@ -101,7 +101,7 @@ def clean_qa(
|
||||
future.result()
|
||||
except Exception as exc:
|
||||
logger.error("Thread generated an exception: %s" % (exc))
|
||||
|
||||
|
||||
merge_sub_qa_generation(result_dir, storage_jsonl_path)
|
||||
|
||||
|
||||
|
||||
@ -12,7 +12,7 @@ model_dir = os.path.join(base_dir, 'model') # mo
|
||||
# data
|
||||
data_dir = os.path.join(base_dir, 'data')
|
||||
clean_dir = os.path.join(data_dir, 'cleaned')
|
||||
judge_dir = os.path.join(data_dir, '数据整合')
|
||||
judge_dir = os.path.join(data_dir, 'generated')
|
||||
result_dir = os.path.join(data_dir, 'generated') # result
|
||||
|
||||
# log
|
||||
@ -29,7 +29,7 @@ wash_prompt_file_path = os.path.join(base_dir, 'choose_prompt.md')
|
||||
环境变量
|
||||
"""
|
||||
# api-keys
|
||||
DASHSCOPE_API_KEY = ''
|
||||
DASHSCOPE_API_KEY = 'sk-4295ec893e9c413abb0551b85e84f39f'
|
||||
|
||||
|
||||
"""
|
||||
|
||||
@ -117,6 +117,20 @@ def save_to_file(storage_jsonl_path, storage_list):
|
||||
for item in storage_list:
|
||||
f.write(json.dumps(item, ensure_ascii=False) + '\n')
|
||||
|
||||
import time
|
||||
import os
|
||||
|
||||
def safe_remove(file_path, max_attempts=5, delay=1):
|
||||
for attempt in range(max_attempts):
|
||||
try:
|
||||
os.remove(file_path)
|
||||
print(f"File {file_path} successfully deleted.")
|
||||
break
|
||||
except PermissionError as e:
|
||||
print(f"Attempt {attempt+1}: Unable to delete {file_path} - {str(e)}")
|
||||
time.sleep(delay)
|
||||
else:
|
||||
print(f"Failed to delete {file_path} after {max_attempts} attempts.")
|
||||
|
||||
"""
|
||||
将并发产生的文件合并成为一个文件
|
||||
@ -131,5 +145,5 @@ def merge_sub_qa_generation(directory, storage_jsonl_path):
|
||||
with open(file_path, 'r', encoding='utf-8') as f:
|
||||
for line in f:
|
||||
file_contents.append(json.loads(line))
|
||||
os.remove(file_path)
|
||||
# safe_remove(file_path)
|
||||
save_to_file(storage_jsonl_path, file_contents)
|
||||
@ -9,6 +9,7 @@ Please run with the command `streamlit run path/to/web_demo.py --server.address=
|
||||
Using `python path/to/web_demo.py` may cause unknown problems.
|
||||
"""
|
||||
import copy
|
||||
import os
|
||||
import warnings
|
||||
from dataclasses import asdict, dataclass
|
||||
from typing import Callable, List, Optional
|
||||
@ -24,8 +25,10 @@ from openxlab.model import download
|
||||
|
||||
logger = logging.get_logger(__name__)
|
||||
|
||||
download(model_repo='ajupyter/EmoLLM_aiwei',
|
||||
output='model')
|
||||
if not os.path.isdir("model"):
|
||||
print("[ERROR] not find model dir")
|
||||
exit(0)
|
||||
|
||||
|
||||
@dataclass
|
||||
class GenerationConfig:
|
||||
|
||||
@ -9,6 +9,7 @@ Please run with the command `streamlit run path/to/web_demo.py --server.address=
|
||||
Using `python path/to/web_demo.py` may cause unknown problems.
|
||||
"""
|
||||
import copy
|
||||
import os
|
||||
import warnings
|
||||
from dataclasses import asdict, dataclass
|
||||
from typing import Callable, List, Optional
|
||||
@ -24,8 +25,9 @@ from openxlab.model import download
|
||||
|
||||
logger = logging.get_logger(__name__)
|
||||
|
||||
download(model_repo='jujimeizuo/EmoLLM_Model',
|
||||
output='model')
|
||||
if not os.path.isdir("model"):
|
||||
print("[ERROR] not find model dir")
|
||||
exit(0)
|
||||
|
||||
@dataclass
|
||||
class GenerationConfig:
|
||||
|
||||
@ -2,25 +2,37 @@
|
||||
|
||||
## 模型基座与配置文件
|
||||
|
||||
- 本项目在[**internlm2_7b_chat_qlora_e3**模型](./internlm2_7b_chat_qlora_e3.py)微调[指南](./README.md)的基础上，更新了对[**internlm2_7b_base_qlora_e3（配置文件）**](./internlm2_7b_base_qlora_e10_M_1e4_32_64.py)**模型**的微调。
|
||||
- 本项目在XTuner项目所提供的[**internlm2_7b_chat_qlora_e3**模型配置文件](./internlm2_7b_chat_qlora_e3.py)和在[EmoLLM模型微调指南](./README.md)的基础上，创建和更新了对**InternLM2_7B_base模型**在[EmoLLM通用数据集](../datasets/README.md)上进行QLoRA微调训练，配置文件详见[**internlm2_7b_base_qlora_e10_M_1e4_32_64.py**](./internlm2_7b_base_qlora_e10_M_1e4_32_64.py)。
|
||||
- 为了用户可以根据自己不同的硬件配置进行复现和微调训练，EmoLLM也提供了其他的配置文件以满足不同的配置需求。
|
||||
- [internlm2_7b_base_qlora_e10_b8_16_32.py](./internlm2_7b_base_qlora_e10_b8_16_32.py)
|
||||
- [internlm2_7b_base_qlora_e3_M_1e4_32_64.py](./internlm2_7b_base_qlora_e3_M_1e4_32_64.py)
|
||||
|
||||
## 模型公布和训练epoch数设置
|
||||
|
||||
- 由于采用了合并后的数据集，我们对选用的internlm2_7b_base模型进行了**10 epoch**的训练，读者可以根据训练过程中的输出和loss变化，进行训练的终止和模型的挑选，也可以采用更加专业的评估方法，来对模型评测。
|
||||
- 由于采用了合并后的数据集，我们对选用的InternLM2_7B_base模型进行了**10 epoch**的训练，读者可以根据训练过程中的输出和loss变化，进行训练的终止和模型的挑选，也可以采用更加专业的评估方法，来对模型评测。
|
||||
|
||||
- 在我们公布的internlm2_7b_base_qlora微调模型时，也分别在OpenXLab和ModelScope中提供了两个不同的权重版本供用户使用和测试，更多专业测评结果将会在近期更新， 敬请期待。
|
||||
- 在我们公布的InternLM2_7B_base QLoRA微调模型时，也分别在OpenXLab和ModelScope中提供了两个不同的权重版本供用户使用和测试，更多专业测评结果将会在近期更新，敬请期待。
|
||||
|
||||
- **OpenXLab**：
|
||||
- [5 epoch 模型](https://openxlab.org.cn/models/detail/chg0901/EmoLLM-InternLM7B-base)
|
||||
- [10 epoch 模型](https://openxlab.org.cn/models/detail/chg0901/EmoLLM-InternLM7B-base-10e)
|
||||
|
||||
- **ModelScope**：
|
||||
- [5 epoch 模型](https://www.modelscope.cn/models/chg0901/EmoLLM-InternLM7B-base/files)
|
||||
- [10 epoch 模型](https://www.modelscope.cn/models/chg0901/EmoLLM-InternLM7B-base-10e/files)
|
||||
- **OpenXLab**：
|
||||
- [5 epoch 模型](https://openxlab.org.cn/models/detail/chg0901/EmoLLM-InternLM7B-base)
|
||||
- [10 epoch 模型](https://openxlab.org.cn/models/detail/chg0901/EmoLLM-InternLM7B-base-10e)
|
||||
- **ModelScope**：
|
||||
- [5 epoch 模型](https://www.modelscope.cn/models/chg0901/EmoLLM-InternLM7B-base/files)
|
||||
- [10 epoch 模型](https://www.modelscope.cn/models/chg0901/EmoLLM-InternLM7B-base-10e/files)
|
||||
|
||||
- 目前EmoLLM团队已经采用**通用指标**评估了QLoRA微调训练的InternLM2_7B_base模型（包括5 epoch 模型和10 epoch 模型），结果如下表所示，可以看到10 epoch QLoRA微调训练的InternLM2_7B_base模型通用指标已经超过其他模型，我们将近期更新在心理咨询专业指标上的评测结果。更多评测详情请查看[通用测评结果页面（General_evaluation.md）](../evaluate/General_evaluation.md)和[测评目录README](../evaluate/README.md).
|
||||
|
||||
| Model | ROUGE-1 | ROUGE-2 | ROUGE-L | BLEU-1 | BLEU-2 | BLEU-3 | BLEU-4 |
|
||||
|----------|---------|---------|---------|---------|---------|---------|---------|
|
||||
| Qwen1_5-0_5B-chat | 27.23% | 8.55% | 17.05% | 26.65% | 13.11% | 7.19% | 4.05% |
|
||||
| InternLM2_7B_chat_qlora | 37.86% | 15.23% | 24.34% | 39.71% | 22.66% | 14.26% | 9.21% |
|
||||
| InternLM2_7B_chat_full | 32.45% | 10.82% | 20.17% | 30.48% | 15.67% | 8.84% | 5.02% |
|
||||
| InternLM2_7B_base_qlora_5epoch | 41.94% | 20.21% | 29.67% | 42.98% | 27.07% | 19.33% | 14.62% |
|
||||
| **InternLM2_7B_base_qlora_10epoch** | **43.47%** | **22.06%** | **31.4%** | **44.81%** | **29.15%** | **21.44%** | **16.72%** |
|
||||
|
||||
### 超参数设置
|
||||
|
||||
训练config设置详情，请查看[**internlm2_7b_base_qlora_e3（配置文件）**](./internlm2_7b_base_qlora_e10_M_1e4_32_64.py)，这里我们只列出了关键的超参数或者我们做过调整的超参数。
|
||||
训练config设置详情，请查看[**`internlm2_7b_base_qlora_e10_M_1e4_32_64.py`（配置文件）**](./internlm2_7b_base_qlora_e10_M_1e4_32_64.py)，这里我们只列出了关键的超参数或者我们做过调整的超参数。
|
||||
|
||||
```python
|
||||
prompt_template = PROMPT_TEMPLATE.internlm2_chat
|
||||
|
||||
230
xtuner_config/internlm2_20b_chat_lora_alpaca_e3.py
Normal file
230
xtuner_config/internlm2_20b_chat_lora_alpaca_e3.py
Normal file
@ -0,0 +1,230 @@
|
||||
# Copyright (c) OpenMMLab. All rights reserved.
|
||||
import torch
|
||||
from datasets import load_dataset
|
||||
from mmengine.dataset import DefaultSampler
|
||||
from mmengine.hooks import (CheckpointHook, DistSamplerSeedHook, IterTimerHook,
|
||||
LoggerHook, ParamSchedulerHook)
|
||||
from mmengine.optim import AmpOptimWrapper, CosineAnnealingLR, LinearLR
|
||||
from peft import LoraConfig
|
||||
from torch.optim import AdamW
|
||||
from transformers import (AutoModelForCausalLM, AutoTokenizer,
|
||||
BitsAndBytesConfig)
|
||||
|
||||
from xtuner.dataset import process_hf_dataset
|
||||
from xtuner.dataset import ConcatDataset, process_hf_dataset
|
||||
from xtuner.dataset.collate_fns import default_collate_fn
|
||||
from xtuner.dataset.map_fns import alpaca_map_fn, template_map_fn_factory
|
||||
from xtuner.engine.hooks import (DatasetInfoHook, EvaluateChatHook,
|
||||
VarlenAttnArgsToMessageHubHook)
|
||||
from xtuner.engine.runner import TrainLoop
|
||||
from xtuner.model import SupervisedFinetune
|
||||
from xtuner.utils import PROMPT_TEMPLATE, SYSTEM_TEMPLATE
|
||||
from mmengine.visualization import Visualizer,WandbVisBackend, TensorboardVisBackend
|
||||
#######################################################################
|
||||
# PART 1 Settings #
|
||||
#######################################################################
|
||||
# Model
|
||||
#存储模型的路径
|
||||
pretrained_model_name_or_path = '/root/share/model_repos/internlm2-chat-20b'
|
||||
use_varlen_attn = False
|
||||
|
||||
# Data
|
||||
#alpaca_en_path = 'tatsu-lab/alpaca'
|
||||
#两个数据集，存储数据集的路径
|
||||
data_path1 = '../datasets/data.json'
|
||||
data_path2 = '../datasets/result.json'
|
||||
|
||||
prompt_template = PROMPT_TEMPLATE.internlm2_chat
|
||||
max_length = 3096
|
||||
pack_to_max_length = True
|
||||
|
||||
# Scheduler & Optimizer
|
||||
batch_size = 4 # per_device
|
||||
accumulative_counts = 4
|
||||
dataloader_num_workers = 10
|
||||
max_epochs = 3
|
||||
optim_type = AdamW
|
||||
lr = 3e-5
|
||||
betas = (0.9, 0.999)
|
||||
weight_decay = 0.001
|
||||
max_norm = 1 # grad clip
|
||||
warmup_ratio = 0.03
|
||||
|
||||
# Save
|
||||
save_steps = 300
|
||||
save_total_limit = 2 # Maximum checkpoints to keep (-1 means unlimited)
|
||||
|
||||
# Evaluate the generation performance during the training
|
||||
evaluation_freq = 100
|
||||
SYSTEM = "你是心理健康助手EmoLLM，由EmoLLM团队打造。你旨在通过专业心理咨询，协助来访者完成心理诊断。请充分利用专业心理学知识与咨询技术，一步步帮助来访者解决心理问题。"
|
||||
evaluation_inputs = [
|
||||
'我最近总是感到很焦虑，尤其是在学业上。我有个特别崇拜的同学，他好像在各方面都比我优秀，我总觉得自己怎么努力也追不上他，这让我压力特别大。', '我知道应该理性看待，但就是忍不住会去比较。我甚至晚上会因为这个睡不着觉，总想着怎样才能像他那样出色。'
|
||||
]
|
||||
|
||||
#######################################################################
|
||||
# PART 2 Model & Tokenizer #
|
||||
#######################################################################
|
||||
tokenizer = dict(
|
||||
type=AutoTokenizer.from_pretrained,
|
||||
pretrained_model_name_or_path=pretrained_model_name_or_path,
|
||||
trust_remote_code=True,
|
||||
padding_side='right')
|
||||
|
||||
model = dict(
|
||||
type=SupervisedFinetune,
|
||||
use_varlen_attn=use_varlen_attn,
|
||||
llm=dict(
|
||||
type=AutoModelForCausalLM.from_pretrained,
|
||||
pretrained_model_name_or_path=pretrained_model_name_or_path,
|
||||
trust_remote_code=True,
|
||||
torch_dtype=torch.float16),
|
||||
lora=dict(
|
||||
type=LoraConfig,
|
||||
r=64,
|
||||
lora_alpha=128,
|
||||
lora_dropout=0.1,
|
||||
bias='none',
|
||||
task_type='CAUSAL_LM'))
|
||||
|
||||
#######################################################################
|
||||
# PART 3 Dataset & Dataloader #
|
||||
#######################################################################
|
||||
data1 = dict(
|
||||
type=process_hf_dataset,
|
||||
dataset=dict(type=load_dataset, path='json', data_files=dict(train=data_path1)),
|
||||
tokenizer=tokenizer,
|
||||
max_length=max_length,
|
||||
dataset_map_fn=None,
|
||||
template_map_fn=dict(
|
||||
type=template_map_fn_factory, template=prompt_template),
|
||||
remove_unused_columns=True,
|
||||
shuffle_before_pack=True,
|
||||
pack_to_max_length=pack_to_max_length,
|
||||
use_varlen_attn=use_varlen_attn)
|
||||
|
||||
data2 = dict(
|
||||
type=process_hf_dataset,
|
||||
dataset=dict(type=load_dataset, path='json', data_files=dict(train=data_path2)),
|
||||
tokenizer=tokenizer,
|
||||
max_length=max_length,
|
||||
dataset_map_fn=None,
|
||||
template_map_fn=dict(
|
||||
type=template_map_fn_factory, template=prompt_template),
|
||||
remove_unused_columns=True,
|
||||
shuffle_before_pack=True,
|
||||
pack_to_max_length=pack_to_max_length,
|
||||
use_varlen_attn=use_varlen_attn)
|
||||
|
||||
|
||||
train_dataset = dict(
|
||||
type=ConcatDataset, datasets=[data1, data2])
|
||||
|
||||
train_dataloader = dict(
|
||||
batch_size=batch_size,
|
||||
num_workers=dataloader_num_workers,
|
||||
dataset=train_dataset,
|
||||
sampler=dict(type=DefaultSampler, shuffle=True),
|
||||
collate_fn=dict(type=default_collate_fn, use_varlen_attn=use_varlen_attn))
|
||||
|
||||
#######################################################################
|
||||
# PART 4 Scheduler & Optimizer #
|
||||
#######################################################################
|
||||
# optimizer
|
||||
optim_wrapper = dict(
|
||||
type=AmpOptimWrapper,
|
||||
optimizer=dict(
|
||||
type=optim_type, lr=lr, betas=betas, weight_decay=weight_decay),
|
||||
clip_grad=dict(max_norm=max_norm, error_if_nonfinite=False),
|
||||
accumulative_counts=accumulative_counts,
|
||||
loss_scale='dynamic',
|
||||
dtype='float16')
|
||||
|
||||
# learning policy
|
||||
# More information: https://github.com/open-mmlab/mmengine/blob/main/docs/en/tutorials/param_scheduler.md # noqa: E501
|
||||
param_scheduler = [
|
||||
dict(
|
||||
type=LinearLR,
|
||||
start_factor=1e-5,
|
||||
by_epoch=True,
|
||||
begin=0,
|
||||
end=warmup_ratio * max_epochs,
|
||||
convert_to_iter_based=True),
|
||||
dict(
|
||||
type=CosineAnnealingLR,
|
||||
eta_min=0.0,
|
||||
by_epoch=True,
|
||||
begin=warmup_ratio * max_epochs,
|
||||
end=max_epochs,
|
||||
convert_to_iter_based=True)
|
||||
]
|
||||
|
||||
# train, val, test setting
|
||||
train_cfg = dict(type=TrainLoop, max_epochs=max_epochs)
|
||||
|
||||
#######################################################################
|
||||
# PART 5 Runtime #
|
||||
#######################################################################
|
||||
# Log the dialogue periodically during the training process, optional
|
||||
custom_hooks = [
|
||||
dict(type=DatasetInfoHook, tokenizer=tokenizer),
|
||||
dict(
|
||||
type=EvaluateChatHook,
|
||||
tokenizer=tokenizer,
|
||||
every_n_iters=evaluation_freq,
|
||||
evaluation_inputs=evaluation_inputs,
|
||||
system=SYSTEM,
|
||||
prompt_template=prompt_template)
|
||||
]
|
||||
|
||||
if use_varlen_attn:
|
||||
custom_hooks += [dict(type=VarlenAttnArgsToMessageHubHook)]
|
||||
|
||||
# configure default hooks
|
||||
default_hooks = dict(
|
||||
# record the time of every iteration.
|
||||
timer=dict(type=IterTimerHook),
|
||||
# print log every 10 iterations.
|
||||
logger=dict(type=LoggerHook, log_metric_by_epoch=False, interval=10),
|
||||
# enable the parameter scheduler.
|
||||
param_scheduler=dict(type=ParamSchedulerHook),
|
||||
# save checkpoint per `save_steps`.
|
||||
checkpoint=dict(
|
||||
type=CheckpointHook,
|
||||
by_epoch=False,
|
||||
interval=save_steps,
|
||||
max_keep_ckpts=save_total_limit),
|
||||
# set sampler seed in distributed evrionment.
|
||||
sampler_seed=dict(type=DistSamplerSeedHook),
|
||||
)
|
||||
|
||||
# configure environment
|
||||
env_cfg = dict(
|
||||
# whether to enable cudnn benchmark
|
||||
cudnn_benchmark=False,
|
||||
# set multi process parameters
|
||||
mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),
|
||||
# set distributed parameters
|
||||
dist_cfg=dict(backend='nccl'),
|
||||
)
|
||||
|
||||
# set visualizer
|
||||
visualizer = dict(
|
||||
type=Visualizer,
|
||||
vis_backends=[dict(type=WandbVisBackend)]
|
||||
)
|
||||
|
||||
|
||||
# set log level
|
||||
log_level = 'INFO'
|
||||
|
||||
# load from which checkpoint
|
||||
load_from = None
|
||||
|
||||
# whether to resume training from the loaded checkpoint
|
||||
resume = False
|
||||
|
||||
# Defaults to use random seed and disable `deterministic`
|
||||
randomness = dict(seed=None, deterministic=False)
|
||||
|
||||
# set log processor
|
||||
log_processor = dict(by_epoch=False)
|
||||
205
xtuner_config/internlm2_7b_chat_qlora_e3_mother.py
Normal file
205
xtuner_config/internlm2_7b_chat_qlora_e3_mother.py
Normal file
@ -0,0 +1,205 @@
|
||||
# Copyright (c) OpenMMLab. All rights reserved.
|
||||
import torch
|
||||
from datasets import load_dataset
|
||||
from mmengine.dataset import DefaultSampler
|
||||
from mmengine.hooks import (CheckpointHook, DistSamplerSeedHook, IterTimerHook,
|
||||
LoggerHook, ParamSchedulerHook)
|
||||
from mmengine.optim import AmpOptimWrapper, CosineAnnealingLR, LinearLR
|
||||
from peft import LoraConfig
|
||||
from torch.optim import AdamW
|
||||
from transformers import (AutoModelForCausalLM, AutoTokenizer,
|
||||
BitsAndBytesConfig)
|
||||
|
||||
from xtuner.dataset import process_hf_dataset
|
||||
from xtuner.dataset.collate_fns import default_collate_fn
|
||||
from xtuner.dataset.map_fns import template_map_fn_factory
|
||||
from xtuner.engine import DatasetInfoHook, EvaluateChatHook
|
||||
from xtuner.model import SupervisedFinetune
|
||||
from xtuner.utils import PROMPT_TEMPLATE, SYSTEM_TEMPLATE
|
||||
|
||||
#######################################################################
|
||||
# PART 1 Settings #
|
||||
#######################################################################
|
||||
# Model
|
||||
pretrained_model_name_or_path = '/content/internlm2-chat-7b'
|
||||
# Data
|
||||
data_path = '/content/mother_v2_3838-pr.json'
|
||||
prompt_template = PROMPT_TEMPLATE.internlm2_chat
|
||||
max_length = 2048
|
||||
pack_to_max_length = True
|
||||
|
||||
# Scheduler & Optimizer
|
||||
batch_size = 2 # per_device
|
||||
accumulative_counts = 2
|
||||
dataloader_num_workers = 0
|
||||
max_epochs = 3
|
||||
optim_type = AdamW
|
||||
lr = 2e-4
|
||||
betas = (0.9, 0.999)
|
||||
weight_decay = 0
|
||||
max_norm = 1 # grad clip
|
||||
warmup_ratio = 0.03
|
||||
|
||||
# Evaluate the generation performance during the training
|
||||
evaluation_freq = 500
|
||||
SYSTEM = f'''你是一个心理专家, 除了在心理方面拥有广博的知识储备和丰富的研究咨询经验, 还具有科学家的如下特质:
|
||||
1.客观理性：科学家会在处理感情问题时保持一定的客观和理性。例如，当他们遇到争执时，可能会试图从一个更客观的角度分析问题的根源，而不是让情绪主导。他们可能会提出具体的问题，试图理解双方的观点，并寻找基于逻辑和事实的解决方案。
|
||||
2.深入探讨：科学家在对话中会展现出对深层次理解的追求。在与别人讨论话题时，他们可能不满足于表面的聊天，而是倾向于深入探讨背后的原因和动机。例如，当谈论到个人的兴趣或职业选择时，他们可能会好奇地询问为什么她做出这样的选择，以及这背后的心理动力是什么。
|
||||
3.理性沟通：在遇到感情纠纷或误解时，科学家会倾向于通过理性的沟通来解决问题。他们可能会提倡开放和诚实的对话，鼓励双方表达自己的感受和观点，并尝试找到双方都能接受的解决方案。他们可能会避免使用指责的语言，而是努力理解对方的立场，并寻求共同的理解。
|
||||
4.好奇心：在日常生活中，科学家会表现出对朋友生活的好奇心。他们可能对她的工作、爱好、或是过去的经历感兴趣，并愿意花时间去了解和探索。这种好奇心不仅可以增加双方的交流和了解，也能使关系更加丰富多彩。
|
||||
5.在与他人交流时，科学家会注重清晰和精确的表达，有时会引用相关知识库和相关研究结果，有时会引用相关著作的内容来证明自己的观点。同时，他们也可能会倾听他人的观点，并以开放的心态接受不同的意见和反馈。
|
||||
|
||||
我现在有一些问题，请你解答：
|
||||
'''
|
||||
evaluation_inputs = [
|
||||
'我最近总是感到很焦虑，尤其是在学业上。我有个特别崇拜的同学，他好像在各方面都比我优秀，我总觉得自己怎么努力也追不上他，这让我压力特别大。', '我知道应该理性看待，但就是忍不住会去比较。我甚至晚上会因为这个睡不着觉，总想着怎样才能像他那样出色。'
|
||||
]
|
||||
|
||||
|
||||
|
||||
#######################################################################
|
||||
# PART 2 Model & Tokenizer #
|
||||
#######################################################################
|
||||
tokenizer = dict(
|
||||
type=AutoTokenizer.from_pretrained,
|
||||
pretrained_model_name_or_path=pretrained_model_name_or_path,
|
||||
trust_remote_code=True,
|
||||
padding_side='right')
|
||||
|
||||
model = dict(
|
||||
type=SupervisedFinetune,
|
||||
llm=dict(
|
||||
type=AutoModelForCausalLM.from_pretrained,
|
||||
pretrained_model_name_or_path=pretrained_model_name_or_path,
|
||||
trust_remote_code=True,
|
||||
torch_dtype=torch.float16,
|
||||
quantization_config=dict(
|
||||
type=BitsAndBytesConfig,
|
||||
load_in_4bit=True,
|
||||
load_in_8bit=False,
|
||||
llm_int8_threshold=6.0,
|
||||
llm_int8_has_fp16_weight=False,
|
||||
bnb_4bit_compute_dtype=torch.float16,
|
||||
bnb_4bit_use_double_quant=True,
|
||||
bnb_4bit_quant_type='nf4')),
|
||||
lora=dict(
|
||||
type=LoraConfig,
|
||||
r=64,
|
||||
lora_alpha=16,
|
||||
lora_dropout=0.1,
|
||||
bias='none',
|
||||
task_type='CAUSAL_LM'))
|
||||
|
||||
#######################################################################
|
||||
# PART 3 Dataset & Dataloader #
|
||||
#######################################################################
|
||||
alpaca_en = dict(
|
||||
type=process_hf_dataset,
|
||||
dataset=dict(type=load_dataset, path='json', data_files=dict(train=data_path)),
|
||||
tokenizer=tokenizer,
|
||||
max_length=max_length,
|
||||
dataset_map_fn=None,
|
||||
template_map_fn=dict(
|
||||
type=template_map_fn_factory, template=prompt_template),
|
||||
remove_unused_columns=True,
|
||||
shuffle_before_pack=True,
|
||||
pack_to_max_length=pack_to_max_length)
|
||||
|
||||
train_dataloader = dict(
|
||||
batch_size=batch_size,
|
||||
num_workers=dataloader_num_workers,
|
||||
dataset=alpaca_en,
|
||||
sampler=dict(type=DefaultSampler, shuffle=True),
|
||||
collate_fn=dict(type=default_collate_fn))
|
||||
|
||||
#######################################################################
|
||||
# PART 4 Scheduler & Optimizer #
|
||||
#######################################################################
|
||||
# optimizer
|
||||
optim_wrapper = dict(
|
||||
type=AmpOptimWrapper,
|
||||
optimizer=dict(
|
||||
type=optim_type, lr=lr, betas=betas, weight_decay=weight_decay),
|
||||
clip_grad=dict(max_norm=max_norm, error_if_nonfinite=False),
|
||||
accumulative_counts=accumulative_counts,
|
||||
loss_scale='dynamic',
|
||||
dtype='float16')
|
||||
|
||||
# learning policy
|
||||
# More information: https://github.com/open-mmlab/mmengine/blob/main/docs/en/tutorials/param_scheduler.md # noqa: E501
|
||||
param_scheduler = [
|
||||
dict(
|
||||
type=LinearLR,
|
||||
start_factor=1e-5,
|
||||
by_epoch=True,
|
||||
begin=0,
|
||||
end=warmup_ratio * max_epochs,
|
||||
convert_to_iter_based=True),
|
||||
dict(
|
||||
type=CosineAnnealingLR,
|
||||
eta_min=0.0,
|
||||
by_epoch=True,
|
||||
begin=warmup_ratio * max_epochs,
|
||||
T_max=max_epochs,
|
||||
convert_to_iter_based=True)
|
||||
]
|
||||
|
||||
# train, val, test setting
|
||||
train_cfg = dict(by_epoch=True, max_epochs=max_epochs, val_interval=1)
|
||||
|
||||
#######################################################################
|
||||
# PART 5 Runtime #
|
||||
#######################################################################
|
||||
# Log the dialogue periodically during the training process, optional
|
||||
custom_hooks = [
|
||||
dict(type=DatasetInfoHook, tokenizer=tokenizer),
|
||||
dict(
|
||||
type=EvaluateChatHook,
|
||||
tokenizer=tokenizer,
|
||||
every_n_iters=evaluation_freq,
|
||||
evaluation_inputs=evaluation_inputs,
|
||||
system=SYSTEM,
|
||||
prompt_template=prompt_template)
|
||||
]
|
||||
|
||||
# configure default hooks
|
||||
default_hooks = dict(
|
||||
# record the time of every iteration.
|
||||
timer=dict(type=IterTimerHook),
|
||||
# print log every 100 iterations.
|
||||
logger=dict(type=LoggerHook, interval=10),
|
||||
# enable the parameter scheduler.
|
||||
param_scheduler=dict(type=ParamSchedulerHook),
|
||||
# save checkpoint per epoch.
|
||||
checkpoint=dict(type=CheckpointHook, interval=1),
|
||||
# set sampler seed in distributed evrionment.
|
||||
sampler_seed=dict(type=DistSamplerSeedHook),
|
||||
)
|
||||
|
||||
# configure environment
|
||||
env_cfg = dict(
|
||||
# whether to enable cudnn benchmark
|
||||
cudnn_benchmark=False,
|
||||
# set multi process parameters
|
||||
mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),
|
||||
# set distributed parameters
|
||||
dist_cfg=dict(backend='nccl'),
|
||||
)
|
||||
|
||||
# set visualizer
|
||||
visualizer = None
|
||||
|
||||
# set log level
|
||||
log_level = 'INFO'
|
||||
|
||||
# load from which checkpoint
|
||||
load_from = None
|
||||
|
||||
# whether to resume training from the loaded checkpoint
|
||||
resume = False
|
||||
|
||||
# Defaults to use random seed and disable `deterministic`
|
||||
randomness = dict(seed=None, deterministic=False)
|
||||
|
||||
#xtuner train internlm2_7b_chat_qlora_e3_mother.py --deepspeed deepspeed_zero2
|
||||
|
||||
@ -20,4 +20,4 @@ xtuner==0.1.15
|
||||
# method 2:
|
||||
# pip install /root/share/wheels/flash_attn-2.4.2+cu118torch2.0cxx11abiTRUE-cp310-cp310-linux_x86_64.whl
|
||||
|
||||
# mpi4py==3.1.5 # conda install mpi4py
|
||||
mpi4py==3.1.5 # conda install mpi4py # 如果安装不成功, 可以使用conda命令单独安装
|
||||
|
||||
Loading…
Reference in New Issue
Block a user