Skip to content

Commit

Permalink
Merge pull request #254 from ilaflott/update-fre-cmor-doc
Browse files Browse the repository at this point in the history
some basic doc updates
  • Loading branch information
ilaflott authored Nov 6, 2024
2 parents 20e943a + d2c51a9 commit 3288f22
Show file tree
Hide file tree
Showing 10 changed files with 262 additions and 206 deletions.
13 changes: 7 additions & 6 deletions .github/workflows/create_test_conda_env.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ jobs:
run: |
# $CONDA is an env var pointing to root of miniconda dir
echo $CONDA/bin >> $GITHUB_PATH
- name: Create fre-cli environment
run: |
# create environment containing all dependencies
Expand All @@ -25,11 +26,11 @@ jobs:
# add conda env's executables to github's PATH equiv.
echo $CONDA/envs/fre-cli/bin >> $GITHUB_PATH
# use *conda environment's pip* to install fre-cli
# called w/ full path to conda's python for explicitness
# called as a module (-m pip) for explicitness
$CONDA/envs/fre-cli/bin/python -m pip install --prefix $CONDA/envs/fre-cli .
## use *conda environment's pip* to install fre-cli
## called w/ full path to conda's python for explicitness
## called as a module (-m pip) for explicitness
$CONDA/envs/fre-cli/bin/python -m pip install --prefix $CONDA/envs/fre-cli .
- name: Run pytest in fre-cli environment
run: |
Expand Down Expand Up @@ -61,7 +62,7 @@ jobs:
# run pylint, ignored modules avoid warnings arising from code internal to those modules
pylint --max-args 6 -ry --ignored-modules netCDF4,cmor fre/ || echo "pylint returned non-zero exit code. preventing workflow from dying with this echo."
- name: Install Sphinx and Build Documentation
run: |
pip install sphinx renku-sphinx-theme sphinx-rtd-theme
Expand Down
2 changes: 0 additions & 2 deletions docs/FAQ.rst

This file was deleted.

1 change: 1 addition & 0 deletions docs/badges.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
======
Badges
======

Expand Down
44 changes: 44 additions & 0 deletions docs/contributing_to_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
===========================
Documentation-Documentation
===========================

Welcome to ``fre-cli``'s Documentation-documentation- where we document how the documentation is
documented

How to Contribute to ``fre-cli``'s documentation
================================================



1. fork and poke at the settings

* Fork ``fre-cli`` on github
* On github, navigate to your ``fre-cli`` fork, and click “settings”
* In “settings”, click “pages”
* In “pages”, under “build and deployment”, make sure “source” is set to “Deploy from a branch”
* Under that, find “Branch”, make sure the branch selected is ``gh-pages``.
* The branch ``gh-pages`` is “automagic”- i.e. do not change anything about it nor create a new one,
nor interact with anything in that branch directly.


2. enable workflows for your fork

* note: this step may depend on user-specific settings!
* Back on top where you found “settings”, find and click “actions” to the left
* Enable running the workflow actions assoc with the ``fre-cli`` repo under ``.github/workflows``


3. run your forks first workflow

* The documentation builds as the last steps to ``create_test_conda_env.yml`` when theres a push to ``main``.
To get your first workflow run on your fork, comment out the ``github.ref == ‘refs/heads/main’`` bit
so that it runs when you push to any branch, and make a small, trivial, commit somewhere to your
remote fork.
* You should be able to find the deployed webpage from a successful workflow at
https://your_username.github.io/fre-cli (if you did not change the fork’s name from ``fre-cli``, that is).
* If you’re only editing docs, you can make the turn-around time on your workflow ~3 min faster by
commenting-out the ``pylint`` and ``pytest`` steps in ``create_test_conda_env.yml``, and disable running the
``build_conda.yml`` workflow



118 changes: 118 additions & 0 deletions docs/developer_usage.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
===============
Developer Usage
===============

Developers are free to use the user guide above to familiarize with the CLI and save time from
having to install any dependencies, but development within a Conda environment is heavily
recommended regardless.

Gain access to the repository with ``git clone --recursive [email protected]:NOAA-GFDL/fre-cli.git`` or your fork's
link (recommended) and an SSH RSA key. Once inside the repository, developers can test local changes
by running a ``pip install .`` inside of the root directory to install the ``fre-cli`` package locally
with the newest local changes. Test as a normal user would use the CLI.


Adding New Tools
================


From Other Repositories
-----------------------

Currently, the solution to this task is to approach it using Conda packages. The tool that is being
added must reside within a repository that contains a ``meta.yaml`` that includes Conda dependencies
like the one in this repository and ideally a ``setup.py`` (may be subject to change due to deprecation)
that may include any potentially needed pip dependencies

* Once published as a Conda package, ideally on the `NOAA-GFDL conda channel <https://anaconda.org/NOAA-GFDL>`_,
an addition can be made to the ``run`` section under ``requirements`` in ``meta.yaml`` of the ``fre-cli``
following the syntax ``channel::package``

* On pushes to the main branch, the package located at https://anaconda.org/NOAA-GFDL/fre-cli will automatically
be updated using by the workflow defined in ``.github/workflows/publish_conda.yml``


Checklist
---------

For the new tool you are trying to develop, there are a few criteria to satisfy

1. Create a subdirectory for the tool group inside the ``fre/`` directory; i.e. ``fre/<tool>``

2. Add an ``__init__.py`` inside of ``fre/<tool>``

* typically this file should be empty, but it depends on the ``<tool>``'s needs
* even if empty, the file facillitates module importability and must be present

3. Add a file named ``fre/<tool>/fre<tool>.py``. This will serve as the main entry point for ``fre``
into the ``<tool>``'s functionality

4. Add a ``click`` group named after ``<tool>`` within ``fre/<tool>/fre<tool>.py``

* This ``click`` group will contain all the functionality under the ``<tool>``

5. Create separate files as needed for different commands; do not code out the full
implemetation of ``<tool>`` inside of a ``click`` command within ``fre/<tool>/fre<tool>.py``.

* better yet, consider what structure your tool may need in the future for maintainability's sake
* if you need, specify a ``<subtool>`` like ``fre/<tool>/<subtool>``. ``fre/app`` currently has
this structure

6. Be sure to import the contents of the needed subcommand scripts inside of ``fre<tool>.py``

* i.e. from ``fre.<tool>.toolCommandScript import *``

7. At this point, you can copy and paste the parts of your main ``click`` command from its script
into ``fre<tool>.py`` when implementing the function reflective of the command function

* Everything will remain the same; i.e. arguments, options, etc.

* However, this new function within ``fre<tool>.py`` must a new line after the arguments, options,
and other command components; ``@click.pass_context``

* Along with this, a new argument ``context`` must now be added to the parameters of the command
(preferably at the beginning, but it won't break it if it's not)

8. From here, all that needs to be added after defining the command with a name is
``context.forward(mainFunctionOfToolCommand)``, and done!

9. The last step is to replicate the command in the same way as done in ``fre<tool>.py``
inside of ``fre.py``, but make sure to add ``from fre import <tool>`` and
``from fre.<tool> import *``

Please refer to this issue when encountering naming issues:
`NOAA-GFDL#31 <https://github.com/NOAA-GFDL/fre-cli/issues/31>`_


Example ``fre/`` Directory Structure
------------------------------------

``fre/``
├── ``__init__.py``
├── ``fre.py``
├── ``fre<tool>``
│ ├── ``__init__.py``
│ ├── ``toolCommandScript.py``
│ └── ``fre<tool>.py``


``MANIFEST.in``
---------------

In the case where non-python files like templates, examples, and outputs are to be included in the ``fre-cli`` package,
``MANIFEST.in`` can provide the solution. Ensure that the file exists within the correct folder, and add a line to the
``MANIFEST.in`` file saying something like ``include fre/fre<tool>/fileName.fileExtension``

* For more efficiency, if there are multiple files of the same type needed, the ``MANIFEST.in`` addition can be something
like ``recursive-include fre/fre<tool> *.fileExtension`` which would recursively include every file matching that
``fileExtension`` within the specified directory and its respective subdirectories.


Adding Documentation
--------------------

see section "Documentation-Documentation"




19 changes: 10 additions & 9 deletions docs/index.rst
Original file line number Diff line number Diff line change
@@ -1,23 +1,24 @@
.. Fre-Cli documentation master file, created by
sphinx-quickstart on Wed Mar 6 22:28:21 2024.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
.. Fre-Cli documentation master file, created by sphinx-quickstart on Wed Mar 6 22:28:21 2024.
You can adapt this file completely to your liking, but it should at least contain the root
\`toctree\` directive (no backslashes)
===================================
Welcome to Fre-Cli's documentation!
===================================

.. toctree::
:maxdepth: 2
:maxdepth: 3
:caption: Contents:

setup
usage
subtools
FAQ
developer_usage
contributing_to_doc
badges

Indices and tables
==================

Indices
=======

* :ref:`genindex`
* :ref:`modindex`
Expand Down
38 changes: 20 additions & 18 deletions docs/setup.rst
Original file line number Diff line number Diff line change
@@ -1,32 +1,34 @@
=====
Setup
=====

Need to set up Conda environment first and foremost
Set up Conda environment
========================

various options work quite well:

If on workstation:
module load conda

Create new Conda environment
conda create -n [environmentName]
If on workstation
-----------------

Append necessary channels
conda config --append channels noaa-gfdl
conda config --append channels conda-forge
``module load fre/2024.01``

Run conda install on needed dependencies
conda install noaa-gfdl::fre-cli should install the CLI package located at https://anaconda.org/NOAA-GFDL/fre-cli created from the meta.yaml file

All other dependencies used by the tools are installed along with this install (configured inside the meta.yaml), with the exception of local modules
setup.py file allows fre.py to be ran with fre as the entry point on the command line instead of python fre.py
Create New Conda environment
----------------------------

Enter commands and follow --help messages for guidance (brief rundown of commands also provided below)
if you're at GFDL:
``module load miniforge``

If the user just runs fre, it will list all the command groups following fre, such as run, make, pp, etc. and once the user specifies a command group, the list of available subcommands for that group will be shown
create an empty environment to start
``conda create -n [environmentName]``

Commands that require arguments to run will alert user about missing arguments, and will also list the rest of the optional parameters if --help is executed
activate the empty environment:
``conda activate [environmentName]``

Argument flags are not positional, can be specified in any order as long as they are specified
append necessary channels:
``conda config --append channels noaa-gfdl; conda config --append channels conda-forge;``

Can run directly from any directory, no need to clone repository
install ``fre-cli`` into the activated environment from `the GFDL conda channel https://anaconda.org/NOAA-GFDL/fre-cli`_ :
``conda install noaa-gfdl::fre-cli``

May need to deactivate environment and reactivate it in order for changes to apply
55 changes: 0 additions & 55 deletions docs/subtools.rst

This file was deleted.

Loading

0 comments on commit 3288f22

Please sign in to comment.