Commit Graph

341 Commits

Author SHA1 Message Date
HongCheng
275f249709 small update 2024-03-18 23:39:49 +09:00
HongCheng
c16761e289 update three merge_json*.py files and corresponding tutorial in CN and EN
update three merge_json*.py files and corresponding tutorial in CN and EN
2024-03-18 23:35:21 +09:00
HongCheng
3cadeadf09 Merge branch 'main' of https://github.com/chg0901/EmoLLM 2024-03-18 22:24:26 +09:00
HongCheng
72a7746d8b
Merge pull request #1 from SmartFlowAI/main
同步
2024-03-18 22:23:25 +09:00
HongCheng
042146af56 Revert "modified merge_jsonl and merge_jsonl_r"
This reverts commit a38ef60058.
2024-03-18 22:13:35 +09:00
xzw
12eeaf8cc1
Merge pull request #89 from chg0901/main
optimizing the merge_json.py + moving files +Update zhipuai_gen_data.py 添加glm-4 response获取异常处理
2024-03-18 21:02:57 +08:00
xzw
95dd6616fb
Merge pull request #95 from chg0901/patch-1
Update tutorial.md update part 5 and 6
2024-03-18 20:58:56 +08:00
HongCheng
bf174f6e02
Update tutorial.md update part 5 and 6 2024-03-18 21:45:04 +09:00
HongCheng
a833832908
Delete generate_data/upload_openxlab.py 2024-03-18 20:45:48 +09:00
HongCheng
19edf026eb
Delete generate_data/trans_process.py 2024-03-18 20:45:18 +09:00
HongCheng
b894b87a9d Revert "moving files in scripts related to generate data to generate_data folder"
This reverts commit 05ff5e8407.
2024-03-18 20:43:39 +09:00
HongCheng
a38ef60058 modified merge_jsonl and merge_jsonl_r
merge_jsonl is for merge jsonl files in a folder
merge_jsonl_r if for merge jsonl files in one folder's subfolders

uasge:
python merge_jsonl_r.py > qwen2.txt
python merge_jsonl_r.py > zhipuai.txt

python merge_jsonl.py > curr.txt

│   学业_merge.json
│   家人_merge.json
│   就业_merge.json
│   工作_merge.json
│   恋爱_merge.json
│   朋友_merge.json
│   环境_merge.json
│   生活_merge.json
│   社交_merge.json
│   责任_merge.json
│   身体_merge.json
│   隐私_merge.json
│
├───学业
│       兴奋.jsonl
│       冷静.jsonl
│       厌倦.jsonl
│       厌恶.jsonl
│       同情.jsonl
│       困惑.jsonl
│       娱乐.jsonl
│       嫉妒.jsonl
│       尴尬.jsonl
│       崇拜.jsonl
│       快乐.jsonl
│       怀旧.jsonl
│       性欲.jsonl
│       恐惧.jsonl
│       悲伤.jsonl
│       敬畏.jsonl
│       有趣.jsonl
│       欣赏.jsonl
│       浪漫.jsonl
│       渴望.jsonl
│       满意.jsonl
│       满足.jsonl
│       焦虑.jsonl
│       痛恨.jsonl
│       痛苦.jsonl
│       着迷.jsonl
│       钦佩.jsonl
│
├───家人
│       兴奋.jsonl
│       冷静.jsonl
│       厌倦.jsonl
│       厌恶.jsonl
│       同情.jsonl
│       困惑.jsonl
│       娱乐.jsonl
│       嫉妒.jsonl
2024-03-18 20:16:39 +09:00
xzw
8a36a3bd9a
Merge pull request #94 from santiagoTOP/main
Update RAG README
2024-03-17 20:49:06 +08:00
santiagoTOP
f693d5a7df Merge branch 'dev' 2024-03-17 20:38:28 +08:00
santiagoTOP
88218bfd4b Update RAG README 2024-03-17 20:37:26 +08:00
xzw
47ce05fee5
Merge pull request #93 from zealot52099/main
update README.md and README_EN.md
2024-03-17 17:48:22 +08:00
zealot52099
428b24f7a1
updata README.md and README_EN.md 2024-03-17 17:44:33 +08:00
zealot52099
758b4d259c
update README.md and README_EN.md 2024-03-17 17:39:17 +08:00
xzw
4473c924f7
Merge pull request #92 from SmartFlowAI/dev
Dev
2024-03-17 13:16:37 +08:00
xzw
1c2f697d72
Merge pull request #90 from chg0901/patch-9
Update qwen_gen_data_NoBash.py 添加对qwen-max的异常处理
2024-03-17 13:16:01 +08:00
xzw
1e93a3dbd1
Merge pull request #91 from Anooyman/main
RAG module update
2024-03-17 13:15:13 +08:00
edward_ke
b050fe8122 Update README_EN.md 2024-03-17 10:40:26 +08:00
edward_ke
50a5129c77 Update basic RAG pipeline
只加了基本的 pipeline,还未进行测试,等具体接口确定之后进行调试
2024-03-17 10:31:11 +08:00
HongCheng
0553c3877b Merge branch 'main' of https://github.com/chg0901/EmoLLM 2024-03-17 10:57:57 +09:00
HongCheng
441d06463a update gitignore (add *.jsonl) 2024-03-17 10:57:51 +09:00
HongCheng
4bcacd8b67
Delete generate_data/curr_merge.json
This is a merged file
2024-03-17 10:19:14 +09:00
HongCheng
baba0d611d move scripts/upload_openxlab.py and scripts/trans_process.py 2024-03-17 10:13:18 +09:00
HongCheng
05ff5e8407 moving files in scripts related to generate data to generate_data folder
also optimizing the merge_json.py
2024-03-17 10:05:50 +09:00
HongCheng
24a16455ab
Update qwen_gen_data_NoBash.py 添加对qwen-max的异常处理 2024-03-17 08:53:50 +09:00
HongCheng
b47a102994
Update zhipuai_gen_data.py 添加glm-4 response获取异常处理
# 解决方法
``` python
    try:
        response = client.chat.completions.create(
            model='glm-4',
            messages=messages,
            top_p=top_p,
        )
    except:
        response = client.chat.completions.create(
            model='glm-4',
            messages=messages,
            top_p=top_p,
        )

``` 
# 报错
``` python
    # Error code: 400, with error text {"error":{"code":"1301","message":"系统检测到输入或生成内容可能包含不安全或敏感内容,请您避免输入易产生敏感内容的提示语,感谢您的配合。"}}
```
2024-03-17 08:45:33 +09:00
xzw
025061b0ab
Merge pull request #88 from chg0901/patch-8
Update qwen_gen_data_NoBash.py 修改生成数量和保存间隔
2024-03-16 23:20:17 +08:00
xzw
f89b73e10b
Merge pull request #87 from chg0901/patch-7
Update zhipuai_gen_data.py 修改生成的数量
2024-03-16 23:20:01 +08:00
HongCheng
8b3c439717
Update qwen_gen_data_NoBash.py 修改生成数量和保存间隔 2024-03-17 00:18:24 +09:00
HongCheng
9bcb5acab7
Update zhipuai_gen_data.py 修改生成的数量 2024-03-17 00:17:03 +09:00
xzw
60fe587f06
Merge pull request #86 from SmartFlowAI/dev
merge clean_qa
2024-03-16 23:10:14 +08:00
aJupyter
24f9ad7cb4 deleted .idea 2024-03-16 23:08:43 +08:00
aJupyter
4ceb8bd587 Merge branch 'wwwyfff-clean-QA' into dev 2024-03-16 23:08:03 +08:00
aJupyter
ebe031af1a add gitignore 2024-03-16 23:06:42 +08:00
aJupyter
93c810f31a Merge branch 'clean-QA' of github.com:wwwyfff/EmoLLM into wwwyfff-clean-QA 2024-03-16 23:03:42 +08:00
xzw
3c0a12543a
Merge pull request #85 from SmartFlowAI/dev
Dev
2024-03-16 22:51:17 +08:00
xzw
7658803a32
Merge pull request #82 from chg0901/patch-5
Update qwen_gen_data_NoBash.py 修改遍历顺序, 先遍历领域area再遍历心情emotions
2024-03-16 22:48:44 +08:00
xzw
de3f6658b6
Merge pull request #83 from chg0901/patch-6
Update zhipuai_gen_data.py 修改遍历顺序, 先遍历领域area再遍历心情emotions
2024-03-16 22:46:30 +08:00
HongCheng
9851eca641
Update zhipuai_gen_data.py 2024-03-16 23:38:51 +09:00
HongCheng
0eee2198df
Update qwen_gen_data_NoBash.py 2024-03-16 23:38:07 +09:00
MING_X
71c46bcdbf
Merge pull request #80 from chg0901/main
update qwen, zhipuai gen_data and readme
2024-03-16 21:26:07 +08:00
HongCheng
a9721ea582
Update config.yml 2024-03-16 22:02:11 +09:00
HongCheng
33e11caf97
Update requirements.txt 2024-03-16 22:01:35 +09:00
HongCheng
93067cfd64
Update config.yml 2024-03-16 22:00:51 +09:00
HongCheng
a82ae114ea
Update config.yml in generate_data 2024-03-16 21:57:29 +09:00
HongCheng
d51c894d3f
Update requirements.txt 2024-03-16 21:56:19 +09:00