Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Anyone willing to pick this up? #332

Closed
nmcbride opened this issue Apr 29, 2020 · 27 comments
Closed

Anyone willing to pick this up? #332

nmcbride opened this issue Apr 29, 2020 · 27 comments

Comments

@nmcbride
Copy link

It's always sad when a really cool open source project gets abandoned to go commercial. Is there anyone else who is willing to pick this up and keep it going?
@castdrian
Copy link

I believe this can be closed as @pusalieth seems to be working on their fork

@Dont-Copy-That-Floppy
Copy link

I'm going to be throwing some pretty heavy time into this, so up to you guys. Hopefully in the end it'll all get merged upstream.

@castdrian
Copy link

castdrian commented Apr 30, 2020 via email

@jardayn
Copy link

jardayn commented Apr 30, 2020

@pusalieth gonna leave this here:
Dont-Copy-That-Floppy@b333e73

mp3's work just fine on Linux, without that change.

@Dont-Copy-That-Floppy
Copy link

@jardayn
Which commands are you using to run the program? If you can get versions, that would be icing on the cake.

@jardayn
Copy link

jardayn commented Apr 30, 2020

@pusalieth
python demo_cli.py

Versions (tell me if i missed anything)

Ubuntu 18.04
Python 3.6
Latest Nvidia CUDA
Nvidia 440 drivers
All the versions from requirements.txt (it's missing torch)
Torch - latest one IntelliJ installed.

@nmcbride
Copy link
Author

@pusalieth I don't have a full understanding of all this stuff yet but I'm willing to help where I can. If you take this and work on it, we can just work out of your fork. There are already good solutions for video that are maintained, we need something in the audio world that is also maintained.

@Dont-Copy-That-Floppy
Copy link

Dont-Copy-That-Floppy commented Apr 30, 2020 •
edited
Loading

@jardayn
You might be using a version where I already fixed the issue with path. The demo_cli.py is the script I used/tested with.

@jardayn
Copy link

jardayn commented May 1, 2020

@pusalieth I was using the original version with none of your changes present.

@Dont-Copy-That-Floppy
Copy link

@jardayn What command did you run? What OS are you using? And what is the path of the file you input?

@jardayn
Copy link

jardayn commented May 2, 2020

@pusalieth
python demo_cli.py

Versions (tell me if i missed anything)

Ubuntu 18.04
Python 3.6
Latest Nvidia CUDA
Nvidia 440 drivers
All the versions from requirements.txt (it's missing torch)
Torch - latest one IntelliJ installed. (latest I guess)

mp3's were in the root directory

@CorentinJ
Copy link
Owner

I'm sure this makes no difference to you, but I want to make a note that this project was my thesis and nothing more. Making it open-source was one of the goals, but beyond a working prototype there were no real plans of maintaining it long-term.

While I cannot share all the differences and improvements from our implementation at Resemble.AI and this one, I can definitely shed light on what is worth rewriting for this project:

  • I highly suggest you entirely get rid of tensorflow. Yes, this is a big piece of work. Among open source implementations of Tacotron, I recommend either Fatchord's or Mozilla's.
  • If the step above doesn't cover that, get rid of forward attention. Plenty new attention mechanism specific to TTS came out in the last years, which all perform better than forward attention and are usually much simpler.
  • Likewise, you might be able to find a better vocoder. I haven't made a search for open-source available ones recently, but you might find some new ones that are faster or/and more robust to artifacts.
  • I would consider using a unified hyperparameters paradigm among the 3 models.

@Dont-Copy-That-Floppy
Copy link

Dont-Copy-That-Floppy commented May 3, 2020 •
edited
Loading

@jardayn
Don't know for sure, but I would guess because the file was in your root dir. Could be many other things though. Either way, it's works now for sure.

I can tell you I'd only want to install pytorch from conda. There's so many dependencies, and conda already has the work done.

@Dont-Copy-That-Floppy
Copy link

@CorentinJ
Thanks for your input. People may not have noticed on your readme that you didn't plan on maintaining it pass Sep 2019. I've only gotten mixed up into ML for like 2 weeks now. Eventually I'll probably go for a Master's in it, who knows. I've got such little experience in it though, your advice is way out of my scope. I'm just coding to use the foundation you built, and make it work to it's maximal capacity.

Just out of curiosity, It's looks like you worked on it for a year or so past your thesis. Did you get hired, or co-found?

@jardayn
Copy link

jardayn commented May 3, 2020

@pusalieth also why Conda? Lots of people are using normal python venvs.

@CorentinJ thanks for the info. Yeah, there are loads of way of improving this.

@CorentinJ
Copy link
Owner

I finished my thesis around June last year and got offers as soon as I made the repo public. I started working immediately after that. I did expect to maintain the project a little more than that, but it was without accounting for the fact that I would work in a very similar vein, and thus having to keep the advancements for myself.

@castdrian
Copy link

castdrian commented May 3, 2020 via email

@jardayn
Copy link

jardayn commented May 3, 2020

@adrifcastr Got a link to the bot?

Also Open Source API's hurt the income of people. Not surprised that the best stuff is commercial.

@castdrian
Copy link

@adrifcastr Got a link to the bot?

Also Open Source API's hurt the income of people. Not surprised that the best stuff is commercial.

here you go, and well, I'm also not making money of my API, I just provide it.

@jardayn
Copy link

jardayn commented May 3, 2020

But for what Jemine is working, there are commercial applications, so... yeah.

I mean, if you want Voice Gen, there's Mozilla TTS

@castdrian
Copy link

castdrian commented May 3, 2020 via email

@Dont-Copy-That-Floppy
Copy link

Dont-Copy-That-Floppy commented May 3, 2020 •
edited
Loading

@jardayn

adrifcastr Got a link to the bot?

Also Open Source API's hurt the income of people. Not surprised that the best stuff is commercial.

I don't want to get into the tit for tat, which is where I think this thread seems to be going. If people want to release open source, it's their choice. Or they can monetize it, and if it's valuable enough to people who can't do it themselves, they'll buy it and that's literally the definition of commerce. Doesn't matter either way to me, but I do prefer open source, and here's why.

Linux, Apache, SQL, and php were/are the backbone of the internet, and all 4 are open source. All major corporation servers are open source. SSL is open source. Google runs 99% open source, including their products, like Android, YouTube, etc. Facebook almost completely runs on open source. Literally the richest software companies in the world run on open source. So, wealth generation and source type are not inextricably linked. There's a few exceptions, but the majority of wealthy corporations run the majority of their software using open source.

Open source benefits the maximal amount of people with the least of amount of money. That's why I would choose open source over closed 80% of the time. The only trouble is certain pieces of software, it's extremely hard to monetize, so they use walled gardens instead. When it comes to AI, I'm in full support of OpenAI objectives. This is the prime time to make everything open source, and sell the models, or usage. That's just my opinion.

@CorentinJ
Copy link
Owner

CorentinJ commented May 4, 2020 •
edited
Loading

Lads if you want to ask Resemble.AI to make the project open source, you go ahead and do it. If you expect a student who just finished university to work full-time on his own and for free on an open source project, you've probably never put a foot in the real world. The closest to what you're asking is Mozilla's repo.

well TTS and voice cloning aren't exactly the same, I don't need a TTS service, I need to replicate a voice, that's all

Yeah they are the same, and you'll get all the features in this repo from Mozzila's repo. Last I checked, erogol had a lot of features from different papers implemented, including sv2tts.

@jardayn
Copy link

jardayn commented May 5, 2020

@pusalieth I should've rephrased that the Open Source comments were in relation to AI.

essentially what @CorentinJ said:

If you expect a student who just finished university to work full-time on his own and for free on an open source project, you've probably never put a foot in the real world

@erogol
Copy link

erogol commented Jun 9, 2020

Lads if you want to ask Resemble.AI to make the project open source, you go ahead and do it. If you expect a student who just finished university to work full-time on his own and for free on an open source project, you've probably never put a foot in the real world. The closest to what you're asking is Mozilla's repo.

well TTS and voice cloning aren't exactly the same, I don't need a TTS service, I need to replicate a voice, that's all

Yeah they are the same, and you'll get all the features in this repo from Mozzila's repo. Last I checked, erogol had a lot of features from different papers implemented, including sv2tts. In fact he's even copied some code from my repo.

Thanks for refering to Mozilla TTS. However, I should emphasize that I did not "copy" anything from here. And yet I' I'd be happy to cite your code as you implemented it before.

@CorentinJ
Copy link
Owner

Yeah I should have worded it better. Sorry about that, I will correct it.

@ghost
Copy link

ghost commented Jul 4, 2020

I believe this can be closed as @CorentinJ has outlined a vision for continued development, and is allowing the community to provide contributions to the repo. See #364

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@nmcbride @erogol @Dont-Copy-That-Floppy @jardayn @CorentinJ @castdrian