Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes test data #11

Merged
merged 15 commits into from
Nov 20, 2023
Merged

Changes test data #11

merged 15 commits into from
Nov 20, 2023

Conversation

l-mansouri
Copy link
Contributor

@l-mansouri l-mansouri commented Nov 20, 2023

Overview

This PR aims to standardize the pipeline to better conform to the template, according to the jira ticket and the discussion on SLACK on how to handle mock data.

. There are workarounds for reference data when used in ci tests if reference data is too large for :github-actions: : https://github.com/lifebit-ai/finemapping/blob/dev/conf/data/data.config#L17 & https://github.com/lifebit-ai/finemapping/blob/dev/main.nf#L83 & https://github.com/lifebit-ai/finemapping/blob/dev/conf/tests/ci/test_ci_polyfun.config#L13

Purpose

This PR aims to standardize the pipeline to better conform to the template.

  • parametrisation of the paths of the reference data for the pipeline to better accommodate for different AWS regions using the params.reference_data_bucket parameter
  • the collection of the reference/pipeline specific data in conf/data/data.conf

Acceptance Criteria/Jira Ticket

DEL-16132

Changes

  • Refactors the container names
  • Refactors nextflow.config accordingly
  • Refactors conf/data/data.config to accommodate for the new parametrisation of the path
  • Refactors the documentation to reflect changes

Internal CloudOS Tests

Not necessary if run in Internal CloudOS CI

Client CloudOS Tests

PR Checklist

Author to check:

  • The PR title is informative and begins with appropriate [Fix:|Feat:|Docs:] based on PR type
  • This PR is raised to dev branch for DSL1 or DSL2-dev for DSL2 translations
  • The description contains the acceptance criteria or link to the associated Jira ticket
  • The description contains links to CloudOS tests covering all test cases in client environment

Reviewer to check:

  • A new test profile covering new functions for acceptance criteria is created if it does not already exist
  • Nextflow configs have associated profiles named to match the config basename
  • The Jenkinsfile has been updated with current Nextflow profiles
  • The internal_lifebit_cloudos_ci.yml has been updated to run all current Nextflow profiles
  • Redundant tests have been removed
  • The README.md has been updated with parameter changes and current results/ and conforms to the README.md guidelines
  • The code is logical, readable, reasonably optimal and meets the acceptance criteria
  • The containers folder has been updated with the necessary containers to run this pipeline if needed

Copy link

@monikrzyz monikrzyz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@l-mansouri l-mansouri merged commit b7f9702 into dev Nov 20, 2023
0 of 2 checks passed
@l-mansouri l-mansouri deleted the changes_test_data branch November 20, 2023 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants