Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Exo fork update #11

Merged
merged 107 commits into from
Sep 19, 2024
Merged

Exo fork update #11

merged 107 commits into from
Sep 19, 2024

Conversation

risingsunomi
Copy link
Owner

No description provided.

AlexCheema and others added 30 commits August 21, 2024 15:59
@AlexCheema
logs for file filtering, grpc_discovery -> udp_discovery
65e0488
@AlexCheema
update examples: remove old llama3_distributed, add chatgpt_api
5a9f4ba
@AlexCheema
Merge branch 'main' into better_networking
2341aa1
@AlexCheema
fix merge
d4a932e
@AlexCheema
generalise UDPDiscovery to any kind of PeerHandle that accepts an add…
f93f811 
…ress. test it
@AlexCheema
integration test for udp discovery with grpc server
12609cb
@AlexCheema
use NousResearch/Meta-Llama-3.1-70B-Instruct as tinygrad llama-3.1-70…
dc3b2bd 
…b model, previously using non-instruct model
@risingsunomi
rewrite of sharded model using new split testing of huggingface models
032c9b1
@risingsunomi
building out new hf.py class, testing qwen and llama3 8b
626b223
@AlexCheema
todo for speculative model
5721504
@AlexCheema
Merge branch 'main' into better_networking
8baaad7
@risingsunomi
trying to load in weights but transformers/pytorch doesnt allow that …
f983e93 
…unless wanting to rebuild the whole model
@AlexCheema
cleaner discovery
baf6efd
@AlexCheema
test for reconnect
15b5043
@AlexCheema
fix UDPDiscovery params, create a new transport every time we broadcast
3dd81a1
@AlexCheema
test kill pids
dcb3ac7
@AlexCheema
add back listen and cleanup tasks
8114a79
@AlexCheema
more robust discovery / peer handling. now we track if the same node …
355c579 
…id changes address, then we immediately conenct to it
@AlexCheema
update visited with self.id, timeout on collecting topology from a pe…
80c48b9 
…er 5s
@AlexCheema
add id to set
c97da54
@AlexCheema
better logs around peer connecting / disconnecting
8cb678e
@AlexCheema
shorten cli name --chatgpt-api-response-timeout
a0d9c90
@AlexCheema
get rid of -secs suffix
f342cdc
@AlexCheema
consistent remove _secs / -secs suffix
56c1bf9
@AlexCheema
circleci use tee to output logs in realtime as well as capture them
4537d61
@AlexCheema
Merge pull request exo-explore#194 from exo-explore/better_networking
35aba75 
Better networking
@AlexCheema
less aggressive logs for opaque status / download progress. too much …
41dd700 
…spam
@AlexCheema
fix Mistral-Large special case when we pass in a path
01cc6a4
@AlexCheema
DEBUG>=8 for SendOpaqueStatus logs
41f0a22
@AlexCheema
experiment with tinygrad on its own thread, so it doesnt block event …
2950373 
…loop
risingsunomi and others added 28 commits September 17, 2024 01:17
@risingsunomi
fixing scalar issue, reversing passing hidden_states
36d5cde
@risingsunomi
inference bug fix, grpc testing
6917f30
@risingsunomi
inference bug fix, grpc testing
adab336
@risingsunomi
fixing hf model for hidden_states
73146dd
@risingsunomi
fixing hf model for hidden_states
929386d
@risingsunomi
fixing hf model for hidden_states
32b8f67
@risingsunomi
fixing hf model for hidden_states
c86facb
@risingsunomi
fixing hf model for hidden_states
d15b20d
@risingsunomi
fixing hf model for hidden_states
5e41bc4
@risingsunomi
fixing hf model for hidden_states
b29c5f8
@risingsunomi
fixing kvcache issue
ddaa79c
@risingsunomi
fixing kvcache issue
3164d38
@risingsunomi
fixing kvcache issue
e8532bc
@risingsunomi
fixing kvcache issue
6a5b8db
@risingsunomi
working on passing past input_ids between infers and nodes
515687d
@AlexCheema
add support for qwen2.5, initially adding mlx-community/Qwen2.5-14B-I…
3597fba 
…nstruct-4bit
@AlexCheema
add more qwen2.5 models: mlx-community/Qwen2.5-7B-Instruct-4bit mlx-c…
dee83e4 
…ommunity/Qwen2.5-Math-7B-Instruct-4bit mlx-community/Qwen2.5-72B-Instruct-4bit mlx-community/Qwen2.5-Math-72B-Instruct-4bit
@AlexCheema
ignore Qwen models in tokenizers test until bos issue is fixed
68028cc
@AlexCheema
Merge pull request exo-explore#221 from exo-explore/qwen2.5
8ad19a5 
Qwen2.5
@AlexCheema
update twitter handle exolabs_ -> exolabs
311c819
@risingsunomi
implemented infer caching and passing cache information via inference…
92ebdd5 
…_state
@risingsunomi
removing dynamic cache passing in inference_state as model does its o…
f0795bd 
…wn, added cleaning out cached_iids when process is finished
@risingsunomi
removed clearning cache on infer prompt and only on finished infer te…
b8f15a0 
…nsor
@risingsunomi
hidden state dropping between nodes issue
d0f3cb7
@risingsunomi
hidden state dropping between nodes issue
fa6f263
@risingsunomi
hidden state dropping between nodes issue
2b0e7b5
@risingsunomi
Merge branch 'main' of github.com:exo-explore/exo into exo-fork-update
f793c00
@risingsunomi
Merge branch 'exo-fork-update' of github.com:risingsunomi/exo-nvidia …
131c158 
…into exo-fork-update
@risingsunomi risingsunomi merged commit 43a1f61 into main Sep 19, 2024
@risingsunomi risingsunomi deleted the exo-fork-update branch September 19, 2024 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@risingsunomi @AlexCheema @GaetanLepage