How to change batch size

[JRMeyer]

>>> brian2
[April 29, 2020, 2:05pm]

I have a GPU with 6GB memory, and received out of memory error. I
reduced the batch size in the config file by

import json
from utils.generic_utils import load_config
CONFIG = load_config('config.json')
CONFIG['datasets'][0]['path'] = '../LJSpeech-1.1/'
CONFIG['output_path'] = '../'
CONFIG['batch_size'] = 8
CONFIG['eval_batch_size'] = 8
with open('config.json', 'w') as fp:
json.dump(CONFIG, fp)

but still, I get the memory error

RuntimeError: CUDA out of memory. Tried to allocate 106.00 MiB (GPU 0; 5.93 GiB total capacity; 3.20 GiB already allocated; 25.44 MiB free; 3.47 GiB reserved in total by PyTorch)

[This is an archived TTS discussion thread from discourse.mozilla.org/t/how-to-change-batch-size]

[JRMeyer]

>>> mrthorstenm
[April 30, 2020, 6:38pm]

Seems you've overwritten value in code instead of using configfile
value. slash

github.com

#### mozilla/TTS/blob/2e2221f146f1ca301a3b2b547d8f26b9009676de/config.json slash #L53

<br/> <br/> // DISTRIBUTED TRAINING<br/> <br/> 'distributed':{<br/> <br/> 'backend': 'nccl',<br/> <br/> 'url': 'tcp: *slash* / *slash* /localhost:54321'<br/> <br/> },<br/> <br/> 'reinit_layers': [], // give a list of layer names to restore from the given checkpoint. If not defined, it reloads all heuristically matching layers.<br/> <br/> // TRAINING<br/> <br/> 'batch_size': 32, // Batch size for training. Lower values than 32 might cause hard to learn attention. It is overwritten by 'gradual_training'.<br/> <br/> 'eval_batch_size':16, <br/> <br/> 'r': 7, // Number of decoder frames to predict per iteration. Set the initial values if gradual training is enabled. <br/> <br/> 'gradual_training': [[0, 7, 64], [1, 5, 64], [50000, 3, 32], [130000, 2, 32], [290000, 1, 32]], //set gradual training steps [first_step, r, batch_size]. If it is null, gradual training is disabled. For Tacotron, you might need to reduce the 'batch_size' as you proceeed.<br/> <br/> 'loss_masking': true, // enable / disable loss masking against the sequence padding.<br/> <br/> // VALIDATION<br/> <br/> 'run_eval': true,<br/> <br/> 'test_delay_epochs': 10, //Until attention is aligned, testing only wastes computation time.<br/> <br/> 'test_sentences_file': null, // set a file to load sentences to be used for testing. If it is null then we use default english sentences.<br/> <br/>

I'm in doubt that reaching alignment with a value of 8 is possible.

[Archived Post]

[JRMeyer]

>>> brian2
[April 30, 2020, 7:51pm]

here says
about 9.5GB GPU memory is needed for standard training (batch_size 32),
but it is possible to train with 4GB GPU memory with a batch_size 8-12.

I have 6GB GPU memory, and I think I have enough memory for batch_size
16.

What is the standard way to change the training batch_size?

[Archived Post]

[JRMeyer]

>>> mrthorstenm
[April 30, 2020, 8:16pm]

config.json in repo root directory is the right place. I'm just confused
on the code snipplet on your first post which includes code not config.

[Archived Post]

[JRMeyer]

>>> brian2
[April 30, 2020, 8:45pm]

I am following the instruction given
here.

The config file is re-created, and my code attempts to set new
batch_size. I can verify that the batch_size in the generated config
file is changed, but I still get the memory errors as mentioned above.

[Archived Post]

[JRMeyer]

>>> mrthorstenm
[April 30, 2020, 9:33pm]

Juypter Notebook is not needed. slash
Try starting with readme on mozilla tts repo at
https://github.com/mozilla/tts

[Archived Post]