This repository has been archived by the owner on Jun 24, 2024. It is now read-only.

Use `RMSNorm` for normalization #80

Closed

Tracked by

philpax opened this issue Mar 26, 2023 · 0 comments

Closed

Tracked by

Use `RMSNorm` for normalization #80

philpax opened this issue Mar 26, 2023 · 0 comments

Assignees

Labels

issue:enhancement

Collaborator

philpax commented Mar 26, 2023

Upstream has switched to using RMSNorm for normalization, which is more accurate to the original implementation: ggerganov/llama.cpp#173

We didn't do this at first because there seemed to be some issues, but those seem to have been resolved.

philpax added the issue:enhancement label

This was referenced Mar 26, 2023

Good ideas from llama.cpp #15

Closed

Prepare for release #81

Merged

philpax self-assigned this

philpax closed this as completed in

cdb630d

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.