Skip to content

sync : ggml (backend v2) #3912

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Nov 13, 2023
Merged

sync : ggml (backend v2) #3912

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
aa7a2c4
sync : ggml (backend v2) (wip)
ggerganov Nov 2, 2023
e819070
sync : migrate examples and llama.cpp to dynamic graphs (wip)
ggerganov Nov 2, 2023
4fe646f
sync : update tests + fix max op params to 64
ggerganov Nov 2, 2023
83c96d5
sync : ggml-cuda
ggerganov Nov 2, 2023
8401e3e
llama : fix save/load state context size
ggerganov Nov 2, 2023
815f44e
sync : try to fix build on tvOS
ggerganov Nov 2, 2023
16e819d
sync : pass custom graph sizes in training examples
ggerganov Nov 2, 2023
e2349ec
sync : update graph copies to new ggml API
ggerganov Nov 2, 2023
f3fb45b
Merge branch 'master' into sync
ggerganov Nov 2, 2023
7f8e2a5
sync : update sync-ggml.sh with new files
ggerganov Nov 2, 2023
075ee61
Merge branch 'master' into sync
ggerganov Nov 3, 2023
dc22db7
scripts : fix header in sync script
ggerganov Nov 3, 2023
b1592ea
train : fix context size calculations
ggerganov Nov 3, 2023
e50ab5a
llama : increase inference graph size up to 4096 nodes
ggerganov Nov 3, 2023
081a86d
Merge branch 'master' into sync
ggerganov Nov 6, 2023
aa1f36c
Merge branch 'master' into sync
ggerganov Nov 7, 2023
a4de804
train : allocate grads for backward graphs
ggerganov Nov 7, 2023
548ec46
train : allocate grads for gb_tmp
ggerganov Nov 7, 2023
9efc6b9
Merge branch 'master' into sync
ggerganov Nov 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions examples/finetune/finetune.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1769,7 +1769,7 @@ int main(int argc, char ** argv) {
alloc = ggml_allocr_new_measure(tensor_alignment);
gf = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, true);
gf->order = (enum ggml_cgraph_eval_order) order;
gb = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false);
gb = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, true);
gb_tmp = params.common.use_checkpointing
? ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think gb_tmp also needs the grads=true argument.

: NULL;
Expand Down Expand Up @@ -1802,7 +1802,7 @@ int main(int argc, char ** argv) {
alloc = ggml_allocr_new(mem_compute_data.data(), mem_compute_data.size(), tensor_alignment);
gf = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, true);
gf->order = best_order;
gb = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false);
gb = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, true);
gb_tmp = params.common.use_checkpointing
? ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false)
: NULL;
Expand Down
4 changes: 2 additions & 2 deletions examples/train-text-from-scratch/train-text-from-scratch.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1136,7 +1136,7 @@ int main(int argc, char ** argv) {
alloc = ggml_allocr_new_measure(tensor_alignment);
gf = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, true);
gf->order = (enum ggml_cgraph_eval_order) order;
gb = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false);
gb = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, true);
gb_tmp = params.common.use_checkpointing
? ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false)
: NULL;
Expand Down Expand Up @@ -1169,7 +1169,7 @@ int main(int argc, char ** argv) {
alloc = ggml_allocr_new(mem_compute_data.data(), mem_compute_data.size(), tensor_alignment);
gf = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, true);
gf->order = best_order;
gb = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false);
gb = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, true);
gb_tmp = params.common.use_checkpointing
? ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false)
: NULL;
Expand Down