Cluster snakefiles #12

reid-a · 2023-12-20T16:12:16Z

There's no real point in keeping the cluster_snakefile stuff separate, it should be merged. We can still edit the snakefiles as we move forward, this isn't an attempt to declare the set to be finalized.

The "Snakefile_cluster_iteration" file manages the list of widths, and can iterate over jobs on the cluster and then do a (trivial) aggregation step. The queuing_config.yaml file is a short cluster config that removes the need to type the cluster stuff into the command line. It conventionally goes in $HOME/.config/snakemake/queuing/config.yaml, and is accessed by "snakemake <other-args> --profile=queuing". The Snakefile has mpi stuff for the trivial aggregation rule because the cluster-ness of it applies to all the rules. This is clumsy, but not wrong.

No cluster shenanigans in this one, just straight-up iteration, maybe a useful intermediate step.

github-actions · 2023-12-20T16:12:32Z

Thank you!

Thank you for your pull request 😃

🤖 This automated message can help you check the rendered files in your submission for clarity. If you have any questions, please feel free to open an issue in {sandpaper}.

If you have files that automatically render output (e.g. R Markdown), then you should check for the following:

🎯 correct output
🖼️ correct figures
❓ new warnings
‼️ new errors

Rendered Changes

🔍 Inspect the changes: https://github.com/carpentries-incubator/hpc-workflows/compare/md-outputs..md-outputs-PR-12

The following changes were observed in the rendered markdown documents:

 files/Snakefile_cluster_iteration (new) | 24 ++++++++++++++++++++++++
 files/Snakefile_iterative (new)         | 13 +++++++++++++
 files/queuing_config.yaml (new)         |  6 ++++++
 3 files changed, 43 insertions(+)

What does this mean?

If you have source files that require output and figures to be generated (e.g. R Markdown), then it is important to make sure the generated figures and output are reproducible.

This output provides a way for you to inspect the output in a diff-friendly manner so that it's easy to see the changes that occur due to new software versions or randomisation.

⏱️ Updated at 2023-12-20 16:13:39 +0000

tkphd

LGTM. Thanks @reid-a!

Auto-generated via {sandpaper} Source : a4d7b63 Branch : main Author : Trevor Keller <[email protected]> Time : 2023-12-21 12:13:20 +0000 Message : Merge pull request #12 from carpentries-incubator/cluster_snakefiles Cluster snakefiles

Auto-generated via {sandpaper} Source : 3d286cb Branch : md-outputs Author : GitHub Actions <[email protected]> Time : 2023-12-21 12:14:34 +0000 Message : markdown source builds Auto-generated via {sandpaper} Source : a4d7b63 Branch : main Author : Trevor Keller <[email protected]> Time : 2023-12-21 12:13:20 +0000 Message : Merge pull request #12 from carpentries-incubator/cluster_snakefiles Cluster snakefiles

reid-a added 2 commits October 23, 2023 10:38

Adding an iterative snakefile.

3cb2dfb

No cluster shenanigans in this one, just straight-up iteration, maybe a useful intermediate step.

github-actions bot pushed a commit that referenced this pull request Dec 20, 2023

differences for PR #12

2d51a0d

tkphd approved these changes Dec 21, 2023

View reviewed changes

tkphd merged commit a4d7b63 into main Dec 21, 2023
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cluster snakefiles #12

Cluster snakefiles #12

reid-a commented Dec 20, 2023

github-actions bot commented Dec 20, 2023 •

edited

Loading

tkphd left a comment

Cluster snakefiles #12

Cluster snakefiles #12

Conversation

reid-a commented Dec 20, 2023

github-actions bot commented Dec 20, 2023 • edited Loading

Thank you!

Rendered Changes

tkphd left a comment

Choose a reason for hiding this comment

github-actions bot commented Dec 20, 2023 •

edited

Loading