Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Passing arguments betwen modules #4

Open
simeonem opened this issue Dec 12, 2016 · 0 comments
Open

Passing arguments betwen modules #4

simeonem opened this issue Dec 12, 2016 · 0 comments

Comments

@simeonem
Copy link
Member

I have inputs arguments that needs to be passed through 3 identical modules. The only difference between the modules it the job iterations/labels. Since the limit of jobs submission on Cranium is 3000 and we want to run 9000 jobs per experiment, each module runs 3000 jobs, with labels 1-3000, 3001-6000 and 6001-9000 for each module. I want the module 1-3000 to pass its input arguments (i.e., specs, R script, dataset to process, workspace directory,...) as inputs arguments for the 3001-6000 module, and so on.
Ultimately, I want to pass these specs across different experiments.
The final workflow will have a unique module for initialization, where we specify: i) the R script to use, ii) the dataset to process, iii) the arguments as specs for each experiment, iv) the workspace directory, v) the jobs labels to run, and vi) flags for the shell script. The only module-specific parameter will be the experiment label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant