bakaEC/kopa
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #25 from zjukg/master

Browse Source 
update README
This commit is contained in:
Yichi Zhang 2023-12-26 20:44:29 +08:00 committed by GitHub
commit 9e5c0c5bb8
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 18 additions and 17 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
README.md
View File

@ -29,23 +29,6 @@ Due to the size of the data, you need to download and unzip the data file data.z
- run KoPA tuning
```shell
export WANDB_DISABLED=true
wandb offline
# For CoDeX-S dataset
CUDA_VISIBLE_DEVICES=0 nohup python finetune_kopa.py \
--base_model 'YOUR LLM PATH' \
--data_path 'data/CoDeX-S-train.json' \
--output_dir 'YOUR SAVE PATH' \
--num_epochs 3 \
--lora_r 64 \
--learning_rate 3e-4 \
--batch_size 12 \
--micro_batch_size 12 \
--num_prefix 1 \
--kge_model 'data/CoDeX-S-rotate.pth' \
--lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' > log.txt &
# For CoDeX-S dataset
export WANDB_DISABLED=true
wandb offline
@ -61,6 +44,24 @@ CUDA_VISIBLE_DEVICES=0 nohup python finetune_kopa.py \
--num_prefix 1 \
--kge_model 'data/CoDeX-S-rotate.pth' \
--lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' > log.txt &
# For FB15K-237N dataset
export WANDB_DISABLED=true
wandb offline
CUDA_VISIBLE_DEVICES=0 nohup python finetune_kopa.py \
--base_model 'YOUR LLM PATH' \
--data_path 'data/FB15K-237N-train.json' \
--output_dir 'YOUR SAVE PATH' \
--num_epochs 3 \
--lora_r 64 \
--learning_rate 3e-4 \
--batch_size 12 \
--micro_batch_size 12 \
--num_prefix 1 \
--kge_model 'data/FB15K-237N-rotate.pth' \
--lora_target_modules='[q_proj,k_proj,v_proj,o_proj]' > log.txt &
```
You may need to fill the LLM path and save path before running. The hyper-parameters can be tuned by yourself.