[TTS]Diffsinger opencpop baseline #2834
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yt605155624
changed the title
lym0302
force-pushed
the
diffsinger
branch
2 times, most recently
from
9888edc to
4d5a6b4
Compare
yt605155624
reviewed
Feb 1, 2023
yt605155624
reviewed
Feb 1, 2023
| @@ -0,0 +1,12 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
这个其实可以直接软连接 csmsc/tts3/local/train.sh (除了 ngpu 不一样)，可能等后期再改
yt605155624
reviewed
Feb 2, 2023
yt605155624
reviewed
Feb 6, 2023
| return outs[0], d_outs[0], p_outs[0], e_outs[0] | ||
|
|
||
|
|
||
| class FastSpeech2MIDILoss(nn.Layer): |
There was a problem hiding this comment.
可以继承自 FastSpeech2Loss, 直接使用父类的 __init__
yt605155624
reviewed
Feb 6, 2023
| self.fs2 = FastSpeech2MIDI( | ||
| idim=idim, | ||
| odim=odim, | ||
| fastspeech2_config=fastspeech2_params, |
There was a problem hiding this comment.
一个是 fastspeech2_config 一个是 fastspeech2_params 此处是否保持一致
| optimizers: Dict[str, Optimizer], | ||
| criterions: Dict[str, Layer], | ||
| dataloader: DataLoader, | ||
| fs2_train_start_steps: int=0, |
There was a problem hiding this comment.
这个参数感觉不需要，fs2_train_start_steps 存在不是 0 的情况吗？
| spk_id = paddle.cast(spk_id, 'int64') | ||
| # forward propagation | ||
| before_outs, after_outs, d_outs, p_outs, e_outs, spk_logits = self._forward( | ||
| xs, |
| es = e.unsqueeze(0) if e is not None else None | ||
|
|
||
| # (1, L, odim) | ||
| _, outs, d_outs, p_outs, e_outs, _ = self._forward( |
| is_inference=True) | ||
| else: | ||
| # (1, L, odim) | ||
| _, outs, d_outs, p_outs, e_outs, _ = self._forward( |
|
|
||
| # (1, L, odim) | ||
| # use *_ to avoid bug in dygraph to static graph | ||
| hs, h_masks = self._forward( |
|
|
||
| # (1, L, odim) | ||
| # use *_ to avoid bug in dygraph to static graph | ||
| hs, _ = self._forward( |
|
|
||
| report("train/loss_ds", float(loss_ds)) | ||
| report("train/l1_loss_ds", float(l1_loss_ds)) | ||
| losses_dict["l1_loss_ds"] = float(l1_loss_ds) |
There was a problem hiding this comment.
这两个 loss 一样的话是否需要 report 两遍？
| self.normalizer = normalizer | ||
| self.acoustic_model = model | ||
|
|
||
| def forward(self, text, note, note_dur, is_slur, get_mel_fs2: bool=False): |
| from paddlespeech.t2s.training.trainer import Trainer | ||
| from paddlespeech.t2s.utils import str2bool | ||
|
|
||
| # from paddlespeech.t2s.models.fastspeech2 import FastSpeech2Loss |
yt605155624
reviewed
Feb 6, 2023
| dataset (str): dataset name | ||
| Returns: | ||
| Dict: the information of sentence, include [phone id (int)], [the frame of phone (int)], [note id (int)], [note duration (float)], [is slur (int)], text(str), speaker name (str) | ||
| tunple: speaker name |
yt605155624
reviewed
Feb 7, 2023
| print("========Config========") | ||
| print(config) | ||
| print( | ||
| f"master see the word size: {dist.get_world_size()}, from pid: {os.getpid()}" |
Closed
yt605155624
reviewed
Feb 9, 2023
| mel_fs2 = mel_fs2.unsqueeze(0).transpose((0, 2, 1)) | ||
| cond_fs2 = self.fs2.encoder_infer(text, note, note_dur, is_slur) | ||
| cond_fs2 = cond_fs2.transpose((0, 2, 1)) | ||
| mel, _ = self.diffusion(mel_fs2, cond_fs2) |
There was a problem hiding this comment.
此处是否应该调用 self.diffusion.inference() , 如果是的话 应该加一个 num_inference_steps 参数控制下步数，默认的 1000 太大了
|
This pull request is now in conflict :( |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
new feature
PR changes
add Diffsinger opencpop baseline （fft training）
Describe
add Diffsinger opencpop baseline （fft training）
fix #2821