Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add LevDoom and COOM benchmarks to the tools/projects list #585

Merged
merged 3 commits into from
Feb 11, 2024

Conversation

TTomilin
Copy link
Contributor

Hi!

I would like to propose adding two new ViZDoom benchmarks to the tools/projects list in the README.

LevDoom is a generalization benchmark with progressively complex visual environments published at CoG 2022 and COOM is a Continual Learning benchmark for task-incremental learning published at the NeurIPS 2023 Datasets and Benchmarks Track.

I believe highlighting these projects can be beneficial to the ViZDoom community. If not for the plentiful new hand-crafted environments they contain, then for drawing inspiration for future research in applying ViZDoom in novel ways.

@mwydmuch
Copy link
Member

Hi @TTomilin, thank you for the PR and for sharing these projects; they look very cool. I'm happy to accept this PR.

I would be happy to have these also listed on ViZDoom documentation (https://vizdoom.farama.org/) under the 3rd-party environments/libraries subpage, but for that it would be nice to have:

  • Documentation for available environments. Right now, it is not clear enough from the README.md what is really available and what kind of options are available.
  • I see that both implement Gym/Gymansium environments. Have you thought about publishing it on PyPI for easier installation?
  • It would also be nice to have LevDoom updated to use the Gymnasium instead of the Gym that is no longer supported.

@TTomilin
Copy link
Contributor Author

Hi @mwydmuch! Thank you very much for the suggestions! I have updated both projects to use the latest dependency versions, published them on PyPi, improved the README.md files, and simplified their usage.

In which section of ViZDoom documentation do you propose they be added and in what form exactly? I could not find an existing 3rd-party environments/libraries subpage.

@mwydmuch
Copy link
Member

That's great! The README.md files are now much clearer for me!

Regarding the ViZDoom documentation, we don't have such a subpage at the moment, but I've planned to add one for some time now. You can create a new .md file under docs/environments directory and provide some description of COOM and LevDoom with essential URLs. Then it needs to be added to docs/index.md. Don't worry about formatting; I will improve it later if needed, but it would be great to have it written by you - the author :)

@mwydmuch
Copy link
Member

I will also fix this pre-commit check, so don't worry about it.

@mwydmuch
Copy link
Member

I've just spent a moment testing both projects, and they install and run perfectly :) I have one more tip: even if you don't plan to work on these projects further, it would be good to make releases on Github to have tags matching the codes that were pushed to the PyPI index. It will make it look more professional.

@TTomilin
Copy link
Contributor Author

TTomilin commented Feb 5, 2024

I created the documentation page. As I'm not sure what your vision is in terms of how long each entry should be or what it provides, feel free to adjust anything. I currently made them more introductory and visual. And thanks for the tip!

@mwydmuch mwydmuch merged commit a8efe37 into Farama-Foundation:master Feb 11, 2024
1 check failed
@mwydmuch
Copy link
Member

@TTomilin thank you very much for the documentation page! I shortened your descriptions a bit, as I believe they shouldn't contain too many details because they may change in the feature, and users should check for them on projects' repos/websites. Don't hesitate to submit another PR if I made some mistakes in these.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants