Skip to content

Commit

Permalink
Merge pull request #683 from mit-ll-responsible-ai/docs
Browse files Browse the repository at this point in the history
New how-to guide and changelog update
  • Loading branch information
rsokl authored Apr 30, 2024
2 parents e090e7b + 6a68603 commit d692c50
Show file tree
Hide file tree
Showing 5 changed files with 144 additions and 4 deletions.
2 changes: 2 additions & 0 deletions docs/source/api_reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,7 @@ hydra-zen provides specialized auto-config support for values of the following t
- :py:class:`set`
- :py:class:`frozenset`
- :py:func:`dataclasses.dataclass` (note: not compatible with pickling)
- `hydra_zen.wrapper.Zen`

hydra-zen also provides auto-config support for some third-pary libraries:

Expand Down Expand Up @@ -264,3 +265,4 @@ Although Hydra provides some runtime type-checking functionality, it only suppor

beartype.validates_with_beartype
pydantic.validates_with_pydantic
pydantic.pydantic_parser
13 changes: 9 additions & 4 deletions docs/source/changes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,10 @@ chronological order. All previous releases should still be available on pip.

.. _v0.13.0:

----------------------
0.13.0rc1 - 2024-04-01
----------------------
-------------------
0.13.0 - 2024-04-29
-------------------

.. note:: This is documentation for an pre-release version of hydra-zen. You can install this pre-release via `pip install --pre hydra-zen`

This release makes it possible to use `pydantic's <https://docs.pydantic.dev/latest/>`_ runtime type-checking and parsing throughout your hydra-zen app 🚀🚀🚀

Expand Down Expand Up @@ -92,13 +91,19 @@ New Features
- Adds :func:`hydra_zen.third_party.pydantic.pydantic_parser`. See :pull:`666`.
- Adds `_target_wrapper_` as a new, optional argument to `hydra_zen.instantiate`, which enables users to add a custom wrapper to be applied recursively to all targets during instantiation. See :pull:`666`.
- Adds an optional `instantiation_wrapper` to `hydra_zen.zen` (and `hydra_zen.wrappers.Zen`) that will apply a custom wrapper to all targets during instantiation performed by `zen`. See :pull:`666`.
- Adds :ref:`autoconfig <additional-types>` support for :py:class:`collections.defaultdict`. See :pull:`681`.
- Adds autoconfig support for `hydra_zen.wrapper.Zen`. I.e. the output of `zen` can now be passed to `~hydra_zen.just` and `~hydra_zen.builds` to generate Hydra-compatible configs. See :pull:`666`.
- Improved support for passing parameterized generics to `hydra_zen.builds` and `hydra_zen.just`. See :pull:`666`.

Bug Fixes
---------
- Fixes incompatibility between zen-processing features (e.g. ``zen_meta``) and iterative-build patterns. See :pull:`638`.

Documentation
-------------

Added How-To: :ref:`pydantic-parsing`


.. _v0.12.0:

Expand Down
2 changes: 2 additions & 0 deletions docs/source/explanation/type_refinement.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ modify their code's type annotations. Furthermore, hydra-zen preserves
type information such that third-party type checkers can be used to provide more
robust support for more general type annotations.

.. _hydra-type-support:

Hydra's Limited Type Support
----------------------------

Expand Down
130 changes: 130 additions & 0 deletions docs/source/how_to/pydantic_guide.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
.. meta::
:description: Adding pydantic parsing to a Hydra app

.. admonition:: tl;dr

When wrapping your main function with `hydra_zen.zen`, pass in `hydra_zen.third_party.pydantic.pydantic_parser` to the `instantiation_wrapper` argument of
`~hydra_zen.zen`. Additionally, for any `hydra_zen.instantiate` calls you make, pass
this parser in as the `_target_wrapper_` argument.


.. admonition:: Prerequisites

Your must install `pydantic <https://docs.pydantic.dev/latest/>`_ in your Python environment in order to complete this How-To guide.

.. _pydantic-parsing:

========================================================
Add Pydantic Type Checking and Parsing to Your Hydra App
========================================================

Hydra's runtime type checking is :ref:`limited to only a narrow subset of Python's typing features <hydra-type-support>`. By contrast, `Pydantic <https://pydantic-docs.helpmanual.io/>`_ provides much more comprehensive type checking
capabilities; additionally, it is capable of parsing CLI inputs into complex data
structures (e.g., convert a string to a :py:class:`pathlib.Path`). In this how-to
guide, we will **add pydantic parsing to all config-instantiation sites in our
Hydra-app**.


Consider the following simple hydra-zen app:


.. code-block:: python
:caption: Contents of `my_app.py`
from hydra_zen import store
from hydra_zen.third_party.pydantic import pydantic_parser
from typing import Literal
from pydantic import PositiveInt
from dataclasses import dataclass
@dataclass
class Character:
age: PositiveInt = 22
name: str = "Bobby"
def main(
character: Character,
gear: tuple[str, ...] = (),
mode: Literal["easy", "hard"] = "easy",
):
print(f"{character=!r} {gear=!r} {mode=!r}")
if __name__ == "__main__":
from hydra_zen import zen
store(main, hydra_defaults=["_self_", {"character": "base"}])
store(Character, group="character", name="base")
store.add_to_hydra_store()
zen(
main,
# This is the key ingredient
instantiation_wrapper=pydantic_parser,
).hydra_main(
config_name="main",
config_path=None,
version_base="1.3",
)
By specifying `instantiation_wrapper=pydantic_parser` in our `hydra_zen.zen` call, we
insure that all config-instantiation sites in our Hydra-app will use pydantic parsing.
Because of this, we will benefit from the following features:

1. Annotating `mode` with `Literal` will restrict the permitted values for this parameter.
2. We can configure `gear` with a list of strings at the CLI and - because of the `tuple[str, ...]` annotation - it will be coerced into a tuple of strings before being passed to `main`.
3. We can use a `pydantic.PositiveInt` annotation for the `character.age` nested field; this will ensure that the age is a valid value.


Let's check that `mode` is restricted to the values `easy` and `hard`:

.. code-block:: console
:caption: Running the app with an invalid value for `mode`
$ python my_app.py mode='medium'
Traceback (most recent call last):
...
pydantic_core._pydantic_core.ValidationError: 1 validation error for main
mode
Input should be 'easy' or 'hard'
Next, let's see that `gear` can be passed a list, which will be coerced into a tuple:

.. code-block:: console
:caption: Running the app with a list of strings for `gear`
$ python my_app.py gear='[sword,shield]'
character=Character(age=22, name='Bobby') gear=('sword', 'shield') mode='easy'
Finally, let's see that `character.age` is correctly parsed as a `PositiveInt`:

.. code-block:: console
:caption: Running the app with an invalid value for `character.age`
$ python my_app.py character.age=-1
Traceback (most recent call last):
...
Error in call to target '__main__.Character':
1 validation error for Character
age
Input should be greater than 0
In this way, we can use Pydantic to add powerful type checking and parsing to our Hydra
apps.

If your app includes manual calls to `hydra_zen.instantiate`, you can also pass in the
`pydantic_parser` as the `_target_wrapper_` argument to ensure that these
config-instantiation calls also use Pydantic parsing.

.. note::

Keep in mind that this pydantic parsing layer only occurs when we instantiate
configs that have `_target_` fields, and that it uses the annotations of the
`_target_` objects.
1 change: 1 addition & 0 deletions docs/source/how_tos.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ objectives.

how_to/configuring_experiments
how_to/configure_hydra
how_to/pydantic_guide
how_to/use_callbacks
how_to/partial_config
how_to/beartype

0 comments on commit d692c50

Please sign in to comment.