bakaEC/OliveSensorAPI
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add InternLM2_7B_chat_full eval

Browse Source
This commit is contained in:
ZeyuBa 2024-03-03 22:50:54 +08:00
parent 79a135a790
commit b977dc1644
4 changed files with 9 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
evaluate/Genera_evaluation.md
View File

@ -47,3 +47,4 @@ pip install torch transformers datasets nltk rouge jieba
|----------|---------|---------|---------|---------|---------|---------|---------| |----------|---------|---------|---------|---------|---------|---------|---------|
| Qwen1_5-0_5B-Chat | 27.23% | 8.55% | 17.05% | 26.65% | 13.11% | 7.19% | 4.05% | | Qwen1_5-0_5B-Chat | 27.23% | 8.55% | 17.05% | 26.65% | 13.11% | 7.19% | 4.05% |
| InternLM2_7B_chat | 37.86% | 15.23% | 24.34% | 39.71% | 22.66% | 14.26% | 9.21% | | InternLM2_7B_chat | 37.86% | 15.23% | 24.34% | 39.71% | 22.66% | 14.26% | 9.21% |
| InternLM2_7B_chat_full | 32.45% | 10.82% | 20.17% | 30.48% | 15.67% | 8.84% | 5.02% |

2
evaluate/InternLM2_7B_chat_eval.py
View File

@ -102,7 +102,7 @@ for batch in dataloader:
verbose=False, verbose=False,
errors='replace' errors='replace'
).replace("医生：","") for i in range(batch_size)] ).replace("医生：","") for i in range(batch_size)]
hypotheses.extend([r.replace(stop_word," ").split()[0] for r in batch_response if stop_word in r]) hypotheses.extend([r.replace(stop_word," ").split()[0] if stop_word in r else r for r in batch_response])
# Load metric # Load metric

4
evaluate/README.md
View File

@ -7,8 +7,8 @@
| Model | ROUGE-1 | ROUGE-2 | ROUGE-L | BLEU-1 | BLEU-2 | BLEU-3 | BLEU-4 | | Model | ROUGE-1 | ROUGE-2 | ROUGE-L | BLEU-1 | BLEU-2 | BLEU-3 | BLEU-4 |
|----------|---------|---------|---------|---------|---------|---------|---------| |----------|---------|---------|---------|---------|---------|---------|---------|
| Qwen1_5-0_5B-Chat | 27.23% | 8.55% | 17.05% | 26.65% | 13.11% | 7.19% | 4.05% | | Qwen1_5-0_5B-Chat | 27.23% | 8.55% | 17.05% | 26.65% | 13.11% | 7.19% | 4.05% |
| InternLM2_7B_chat | 37.86% | 15.23% | 24.34% | 39.71% | 22.66% | 14.26% | 9.21% | | InternLM2_7B_chat_qlora | 37.86% | 15.23% | 24.34% | 39.71% | 22.66% | 14.26% | 9.21% |
| InternLM2_7B_chat_full | 32.45% | 10.82% | 20.17% | 30.48% | 15.67% | 8.84% | 5.02% |
## 专业指标评测 ## 专业指标评测
* 具体指标、方法见 [Professional_evaluation.md](./Professional_evaluation.md) * 具体指标、方法见 [Professional_evaluation.md](./Professional_evaluation.md)

14
evaluate/README_EN.md
View File

@ -4,15 +4,11 @@
* For specific metrics and methods, see [General_evaluation.md](./General_evaluation_EN.md) * For specific metrics and methods, see [General_evaluation.md](./General_evaluation_EN.md)
| Metric | Value | | Model | ROUGE-1 | ROUGE-2 | ROUGE-L | BLEU-1 | BLEU-2 | BLEU-3 | BLEU-4 |
|---------|----------------------| |----------|---------|---------|---------|---------|---------|---------|---------|
| ROUGE-1 | 27.23% | | Qwen1_5-0_5B-Chat | 27.23% | 8.55% | 17.05% | 26.65% | 13.11% | 7.19% | 4.05% |
| ROUGE-2 | 8.55% | | InternLM2_7B_chat | 37.86% | 15.23% | 24.34% | 39.71% | 22.66% | 14.26% | 9.21% |
| ROUGE-L | 17.05% | | InternLM2_7B_chat_full | 32.45% | 10.82% | 20.17% | 30.48% | 15.67% | 8.84% | 5.02% |
| BLEU-1 | 26.65% |
| BLEU-2 | 13.11% |
| BLEU-3 | 7.19% |
| BLEU-4 | 4.05% |
## Professional Metrics Evaluation ## Professional Metrics Evaluation