Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add disk space requirements to README.md #201

Closed
wants to merge 1 commit into from
Closed

Conversation

G3zz
Copy link

@G3zz G3zz commented Mar 16, 2023

Add disk space requirements from https://cocktailpeanut.github.io/dalai/#/?id=_7b, as suggested in #195.

@G3zz G3zz changed the title Update README.md Add disk space requirements to README.md Mar 16, 2023
@cyloa
Copy link

cyloa commented Mar 16, 2023

The size for the 65B Model is not quite right.
Its more like:

8x15,2 GiB = 121,6 GiB Original Model
8x15.2 GiB = 121,6 GiB GGML             
8x4.75 GiB = 38 GiB Quantized        
281,1 GiB Total          

The other sizes are also not quite right, because the full size is already including the quantized size.

7B:

12,5 GiB Original Model
12,5 GiB GGML             
3,92 GiB Quantized        
28,92 GiB Total          

I don't have the data for 13B.

30B:

4x15,2 GiB = 60,8 GiB Original Model
4x15.2 GiB = 60,8 GiB GGML             
4x4.75 GiB = 19 GiB Quantized        
140,6 GiB Total          

@gjmulder
Copy link
Collaborator

gjmulder commented Mar 16, 2023

$ find . -type d -exec du -hs {} \; | sort -h
30G	./7B
57G	./13B
141G	./30B
282G	./65B
507G	.
$ find . -type f -exec ls -sh {} \; | grep G | sort -rk 2,2
4.0G ./7B/ggml-model-q4_0.bin
13G ./7B/ggml-model-f16.bin
13G ./7B/consolidated.00.pth
4.8G ./65B/ggml-model-q4_0.bin.7
4.8G ./65B/ggml-model-q4_0.bin.6
4.8G ./65B/ggml-model-q4_0.bin.5
4.8G ./65B/ggml-model-q4_0.bin.4
4.8G ./65B/ggml-model-q4_0.bin.3
4.8G ./65B/ggml-model-q4_0.bin.2
4.8G ./65B/ggml-model-q4_0.bin.1
4.8G ./65B/ggml-model-q4_0.bin
16G ./65B/ggml-model-f16.bin.7
16G ./65B/ggml-model-f16.bin.6
16G ./65B/ggml-model-f16.bin.5
16G ./65B/ggml-model-f16.bin.4
16G ./65B/ggml-model-f16.bin.3
16G ./65B/ggml-model-f16.bin.2
16G ./65B/ggml-model-f16.bin.1
16G ./65B/ggml-model-f16.bin
16G ./65B/consolidated.07.pth
16G ./65B/consolidated.06.pth
16G ./65B/consolidated.05.pth
16G ./65B/consolidated.04.pth
16G ./65B/consolidated.03.pth
16G ./65B/consolidated.02.pth
16G ./65B/consolidated.01.pth
16G ./65B/consolidated.00.pth
4.8G ./30B/ggml-model-q4_0.bin.3
4.8G ./30B/ggml-model-q4_0.bin.2
4.8G ./30B/ggml-model-q4_0.bin.1
4.8G ./30B/ggml-model-q4_0.bin
16G ./30B/ggml-model-f16.bin.3
16G ./30B/ggml-model-f16.bin.2
16G ./30B/ggml-model-f16.bin.1
16G ./30B/ggml-model-f16.bin
16G ./30B/consolidated.03.pth
16G ./30B/consolidated.02.pth
16G ./30B/consolidated.01.pth
16G ./30B/consolidated.00.pth
3.8G ./13B/ggml-model-q4_0.bin.1
3.8G ./13B/ggml-model-q4_0.bin
13G ./13B/ggml-model-f16.bin.1
13G ./13B/ggml-model-f16.bin
13G ./13B/consolidated.01.pth
13G ./13B/consolidated.00.pth

@gjmulder gjmulder added the documentation Improvements or additions to documentation label Mar 17, 2023
@prusnak
Copy link
Collaborator

prusnak commented Mar 18, 2023

Requirements added in #269

Closing this one since it had wrong sizes anyway.

@prusnak prusnak closed this Mar 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants