Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

We should dynamically configure some of the steps depending of the experiment size. #340

Open
3 tasks
ypriverol opened this issue Jan 12, 2024 · 2 comments
Open
3 tasks
Assignees
Labels
bug Something isn't working enhancement New feature or request high-priority

Comments

@ypriverol
Copy link
Member

Description of feature

@daichengxin, @jpfeuffer:

When running multiple datasets, I have found that it would be good to have some logic to configure memory and CPU depending on the number of files. We have done that approach for the proteomicsLFQ step, however other processes could also benefit from it. Here is a list of process to review:

@ypriverol ypriverol added the enhancement New feature or request label Jan 12, 2024
@ypriverol ypriverol changed the title We should dynamically configure some of the steps depeding of the experiment size. We should dynamically configure some of the steps depending of the experiment size. Jan 12, 2024
@ypriverol ypriverol added bug Something isn't working high-priority labels Jan 12, 2024
@daichengxin
Copy link
Collaborator

daichengxin commented Jan 13, 2024

It should be possible to implement this in nextflow syntax. The logic can be divided two ways. What do you think?

  1. number * unit usage.
  2. range (24 G when number < 100, 128G when number<1000, 528G when number<10000, 1024G when number >10000)?
  3. other?

@ypriverol
Copy link
Member Author

Looks like a good strategy.

@ypriverol ypriverol linked a pull request Jan 13, 2024 that will close this issue
11 tasks
@ypriverol ypriverol removed a link to a pull request Jan 13, 2024
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request high-priority
Projects
None yet
Development

No branches or pull requests

2 participants