HongCheng a38ef60058 modified merge_jsonl and merge_jsonl_r ... merge_jsonl is for merge jsonl files in a folder merge_jsonl_r if for merge jsonl files in one folder's subfolders uasge: python merge_jsonl_r.py > qwen2.txt python merge_jsonl_r.py > zhipuai.txt python merge_jsonl.py > curr.txt │ 学业_merge.json │ 家人_merge.json │ 就业_merge.json │ 工作_merge.json │ 恋爱_merge.json │ 朋友_merge.json │ 环境_merge.json │ 生活_merge.json │ 社交_merge.json │ 责任_merge.json │ 身体_merge.json │ 隐私_merge.json │ ├───学业 │ 兴奋.jsonl │ 冷静.jsonl │ 厌倦.jsonl │ 厌恶.jsonl │ 同情.jsonl │ 困惑.jsonl │ 娱乐.jsonl │ 嫉妒.jsonl │ 尴尬.jsonl │ 崇拜.jsonl │ 快乐.jsonl │ 怀旧.jsonl │ 性欲.jsonl │ 恐惧.jsonl │ 悲伤.jsonl │ 敬畏.jsonl │ 有趣.jsonl │ 欣赏.jsonl │ 浪漫.jsonl │ 渴望.jsonl │ 满意.jsonl │ 满足.jsonl │ 焦虑.jsonl │ 痛恨.jsonl │ 痛苦.jsonl │ 着迷.jsonl │ 钦佩.jsonl │ ├───家人 │ 兴奋.jsonl │ 冷静.jsonl │ 厌倦.jsonl │ 厌恶.jsonl │ 同情.jsonl │ 困惑.jsonl │ 娱乐.jsonl │ 嫉妒.jsonl		2024-03-18 20:16:39 +09:00
..
final_data	modified merge_jsonl and merge_jsonl_r	2024-03-18 20:16:39 +09:00
qa_generation	moving files in scripts related to generate data to generate_data folder	2024-03-17 10:05:50 +09:00
xinghuo	新增ENmd文档	2024-03-10 15:52:18 +08:00
check.py	add data_process	2024-01-23 21:25:12 +08:00
config.yml	Update config.yml	2024-03-16 22:02:11 +09:00
ernie_gen_data.py	add data_process	2024-01-23 21:25:12 +08:00
gen_metafile.py	moving files in scripts related to generate data to generate_data folder	2024-03-17 10:05:50 +09:00
main.ipynb	add data_process	2024-01-23 21:25:12 +08:00
merge_json.py	moving files in scripts related to generate data to generate_data folder	2024-03-17 10:05:50 +09:00
OCR.md	add ocr.md	2024-03-11 13:15:22 +08:00
pdf2txt.py	moving files in scripts related to generate data to generate_data folder	2024-03-17 10:05:50 +09:00
process.py	moving files in scripts related to generate data to generate_data folder	2024-03-17 10:05:50 +09:00
qwen_gen_data_NoBash.py	update qwen, zhipuai gen_data and readme	2024-03-16 21:46:02 +09:00
qwen_gen_data.py	add data_process	2024-01-23 21:25:12 +08:00
requirements.txt	Update requirements.txt	2024-03-16 22:01:35 +09:00
run_qwen.bash	add data_process	2024-01-23 21:25:12 +08:00
trans_process.py	move scripts/upload_openxlab.py and scripts/trans_process.py	2024-03-17 10:13:18 +09:00
tutorial_EN.md	update qwen, zhipuai gen_data and readme	2024-03-16 21:46:02 +09:00
tutorial.md	update qwen, zhipuai gen_data and readme	2024-03-16 21:46:02 +09:00
upload_openxlab.py	move scripts/upload_openxlab.py and scripts/trans_process.py	2024-03-17 10:13:18 +09:00
zhipuai_gen_data.py	Update zhipuai_gen_data.py 添加glm-4 response获取异常处理	2024-03-17 08:45:33 +09:00