Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversion to GGUF format #4

Open
Allgot opened this issue Dec 17, 2024 · 1 comment
Open

Conversion to GGUF format #4

Allgot opened this issue Dec 17, 2024 · 1 comment

Comments

@Allgot
Copy link

Allgot commented Dec 17, 2024

Hello there!
I was really exited to see the introduction of a newer, larger model that natively supports Korean.

I'm currently fine-tuning the model for my own task, but I've run into some trouble while trying to convert it into GGUF format.
I noticed that you successfully converted it to GGUF format, so I was wondering if you could share the script or method you used for the conversion. I keep encountering a ValueError(f"Can not map tensor {name!r}") during the process.

Once again, I really appreciate your hard work on advancing Korean LLMs. I look forward to seeing more of your future work (and hopefully a response to this issue as well😅)

@Allgot
Copy link
Author

Allgot commented Dec 17, 2024

Update: although I managed to successfully converted it by skipping all the problematic layers, I'm still curious how you handle this :)

(FYR, these are the skipped layers: attention.k_proj.weight, attention.out_proj.weight, attention.q_proj.weight, attention.v_proj.weight, ln_1.weight, ln_2.weight, mlp.c_fc_0.weight, mlp.c_fc_1.weight, mlp.c_proj.weight.)

Update2: things are getting weird.. I noticed that the map_tensor_name() function in convert_hf_to_gguf.py suddenly fails after successfully mapping a few layers with similar names; e.g., it successfully maps transformer.h.2.attn.attention.k_proj.weight, but then fails to map transformer.h.33.attn.attention.k_proj.weight. It seems like a subtle bug or something—myb it's just specific to me?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant