Toolbox not working with python3.8

[brcisna]

Hello All,

Debian 11 Bullseye.
Started with python3.7.x and python3.8.x and recently updated to python3.8.xx along with deprecation python3.7. Tried installing python3.7 from source but for some reason python3.7 is never seen as an alternative and did install with the altinstall switch or whatever that is. From a LOT of reading python3.8 will only accept tensorflow2.x This is the major downfall for python3.8 and realtimevoicecloning. Had the realtimevoicecloning working great in virtualenv until debian suggested & done an autoremove of pyton3.7. ,, 'no longer needed'.
Have spent quite a bit of time trying to figure this out,and still no joy.

Thank You

[ghost]

@brcisna As you note, the code will not be compatible with python 3.8 until we can use tensorflow 2.x. There is an issue for this: #370 .

If you want to hack at it, you are welcome to pull my tensorflow 2.x branch and try to figure out the error messages. Start here for more info. #370 (comment)

Edit: If you're able to make my tensorflow2 fork run on Python 3.8 (that is, get through demo_cli.py without an error, regardless of output quality), I will take on the task of converting the pretrained model and getting it merged upstream with the latest updates.

[brcisna]

Hi blue-fish,

Thank You for the information. Am showing my stupidity. I see the #370 comment but don't know how to install the fork of your tensorflow_v2 to my existing virtualenv? I see several commits,but have never dealt with this type of files,,i guess. Would like to hack around on this if i can figure out how to patch your files in?
Just an fyi. What happens is in a fresh python3.8 virtualenv the demo_toolbox.py ends up with the common 'tensorflow.contrib not found' ,,which of course does not exist in tensorflow_v2

Thanks again

[ghost]

No worries @brcisna ! Happy you are taking an interest to this. You need another copy of the Real-Time-Voice-Cloning toolbox, but it will work with your existing python3.8 + tensorflow2 virtualenv. Here are two ways to do it:

1. Quick (harder to share code changes)

This clones the branch from my fork but you will not be able to push anything back without doing a bit of setup.

git clone --depth 1 https://github.com/blue-fish/Real-Time-Voice-Cloning -b 370_tf2_compat

2. Fork method (very easy to share code changes)

If you are willing to spend just a little more time on the setup, I suggest you make a fork so we can easily collaborate on this:

1. In your web browser, go to my fork: https://github.com/blue-fish/Real-Time-Voice-Cloning
2. Click the fork button at top-right
3. In a terminal, clone your fork: git clone https://github.com/brcisna/Real-Time-Voice-Cloning
4. Go into the folder: cd Real-Time-Voice-Cloning
5. Switch to the branch: git checkout 370_tf2_compat

As you make fixes, push your changes back to your fork. Then I will be able to pull them into mine, and push them upstream when everything is working.

Post-setup (after either method)

6. Copy a set of pretrained models to the usual locations
7. At this point you can activate your python3.8 virtualenv and run demo_cli.py or demo_toolbox.py. It will error because there are still some tensorflow2 compatibility issues to work through. But you will be a few hours ahead compared to starting from scratch!

[brcisna]

Hi blue-fish,

Done the #1 routine here just to see what happens. I believe i am correct ,,cloning your branch simply dumps your patched files into my existing Real-Time-Voice-Cloning dir? I am still ending up with identical ' tensorflow.contrib not found'. I did also deactivate and activate the virtualenv one time also after cloning your branch to my dir. Thought i may end up with a slightly different error after cloning your branch?

line 3, in
from tensorflow.contrib.seq2seq import Helper
ModuleNotFoundError: No module named 'tensorflow.contrib'

Adding:
tensorboard 2.2.2
tensorboard-plugin-wit 1.7.0
tensorflow-addons 0.10.0
tensorflow-cpu 2.2.0
tensorflow-estimator 2.2.0

Thanks

[ghost]

Cloning makes a copy of the files with version control enabled. I think the way you ran it placed another Real-Time-Voice-Cloning directory inside the existing Real-Time-Voice-Cloning dir. Then when you tried it out, it was still running the old code.

To help keep things straight, you should either remove the existing copy before cloning, or perform the clone in a different location entirely.

[brcisna]

OK,

I apologize. I was thinking your dir branch patched my existing directory,
So,,now,, i cloned your dir to fresh directory,
Now i am ending up with the following error when running both demo_cli.py & demo_toolbox.py

from tensor2tensor.utils.hparam import HParams
ModuleNotFoundError: No module named 'tensor2tensor'

EDIT: Opps I did not run the requirements.txt,,,, Let me do this and report back.

Thanks

[ghost]

Let's start on demo_cli.py, I find it easier to troubleshoot. After updating the requirements do you get the same error message as me?

ValueError: Shape must be rank 1 but is rank 0 for '{{node Tacotron_model/inference/decoder/concat}} = ConcatV2[N=2, T=DT_INT32, Tidx=DT_INT32](Tacotron_model/inference/decoder/concat/values_0, Tacotron_model/inference/decoder/concat/values_1, Tacotron_model/inference/decoder/concat/axis)' with input shapes: [1], [], [].

I recall working around it by changing this line to return tf.TensorShape([]).with_rank(1) but I'm not sure if it is proper. It might be masking a different error that needs to be fixed.

[brcisna]

blue-fish,
This is error i get after getting all the requirements installed.. I think when i initally configured pytorch on their configuartion wizard i selected a CUDA version rather than NONE? have tried uninstalling and resintalling but still end up with Your PyTorch is unconfigured":

Arguments:
enc_model_fpath: encoder/saved_models/pretrained.pt
syn_model_dir: synthesizer/saved_models/logs-pretrained
voc_model_fpath: vocoder/saved_models/pretrained/pretrained.pt
low_mem: False
no_sound: False
cpu: False

Running a test of your configuration...

Your PyTorch installation is not configured. If you have a GPU ready for deep learning, ensure that the drivers are properly installed, and that your CUDA version matches your PyTorch installation.

If you're trying to use a cpu, please use the option --cpu.

I do NOT have a CUDA capable GPU ,,Just want to use cpu if possible with this setup.

Goes without saying,,I am very green at this.

Thanks

[ghost]

Try enabling CPU support: python demo_cli.py --cpu

[brcisna]

Thank You,,I wasnt sure how to enable --cpu even after googling,,,
after getting saved models added here and there i am ending up with the error you have mentioned above

Getting closer,,,This is running demo_cli.py

Thanks

[brcisna]

OK I tried changing the line you suggested to this: return tf.TensorShape([]).with_rank(1)
I get a syntax error,,I am not a coder so this is probably not understood by me correctly,,,

Thanks.

[ghost]

I should have mentioned at the start that the most direct way for you to get the toolbox working is to reinstall python3.7 (or run it inside a virtual machine). While I would greatly appreciate your help figuring out tensorflow_v2 compatibility, getting the toolbox to run on python3.8 is likely much harder than getting python3.7 back. I'll understand if you want to try restoring python3.7 instead.

[brcisna]

I did spend quite a bit of time trying to get python3.7 installed back onto here. python3.7 is simply not in the debian bullseye repositories now,,,nada . tried setting up backports and all sorts of stuff,, i would guess there was a cutoff date and the maintainers are full on python3.8 for their new dsitro release. I thought i got python3.7 installed through source with the --altinstall switch but cant see it anywhere,,?i never found any decent howtos on installing for a local user in an virtualenv either?.If i can get 3.7 back on,it will work as it did before,, i never realized when i done the autoremove of python3.7 as suggested a couple weeks ago it would kill this virtualenv setup,,never thought about it at the time until i tried using the toolbox again.
Sorry im sorta lost on this stuff.

[ghost]

Can you run a virtual machine with Ubuntu 18.04 and install python3.7 on that?

[brcisna]

That may be what ill have to do. i was really hoping to try and get this setup back to 3.7 Thanks for helping out too.

[brcisna]

blue-fish,

finally with much wrangling used deb. snapshots repositories to get python3.7 reinstalled on this system and appears to be back to were it should be,,,,but,, when i use this command on my original (was working) virtualenv
'python3.7 -m virtualenv MyEnv ' says " virtualenv module not found'
BUT ,,, my newly created virtualenv ( a day ago) with my default python3.8 the demo_cli.py fails with the error you listed above. I tried doing the one line change you suggested above but get syntax errors.

The UPSIDE is the toolbox.py opens fine and i can do recordings! Not sure if the synthesizer or other stuff works. Kinda burned out right now,,Would really like to get this back to working as i would like to use this again for some race history voices ( of now deceased people to narrate) of some videos,i do as a hobby.

Thank You

[ghost]

when i use this command on my original (was working) virtualenv
'python3.7 -m virtualenv MyEnv ' says " virtualenv module not found'

Try installing virtualenv for your python3.7 and then try the above command again.

python3.7 -m pip install virtualenv

[ghost]

@brcisna Based on #404 can I conclude that the toolbox is working for you now? If so can you please go ahead and close this issue?

(Resolution: Use python 3.7)

[brcisna]

will do thanks

…

On Mon, Jul 6, 2020 at 8:04 PM blue-fish ***@***.***> wrote: @brcisna <https://github.com/brcisna> Based on #404 <#404> can I conclude that the toolbox is working for you now? If so can you please go ahead and close this issue? (Resolution: Use python 3.7) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#401 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ALJUC6UQMIFCKMORLBZL6V3R2JYABANCNFSM4OQ6HYKQ> .