Incorporate NOAA and NCALM to search

docs/examples/additional_lidar.ipynb

@@ -0,0 +1,282 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Additional Aerial LiDAR Datasets\n",
+    "\n",
+    "This notebook will highlight the additional aerial LiDAR datasets besides USGS 3dep that are available in the `coincident` package and how to grab those datasets' footprints using spatial and temporal search parameters.\n",
+    "\n",
+    "Other Supported Catalogs:\n",
+    "- NOAA Coastal LiDAR\n",
+    "- NCALM LiDAR\n",
+    "- NEON LiDAR"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import coincident\n",
+    "import geopandas as gpd"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## OpenTopography API\n",
+    "\n",
+    "`coincident` supports the use of the [OpenTopo /otCatalog API](https://portal.opentopography.org/apidocs/) to access additional aerial LiDAR data.\n",
+    "\n",
+    "\n",
+    "opentopo datasets currently supported includes the [NOAA Coastal LiDAR Catalog](https://coast.noaa.gov/htdata/lidar1_z/) and [NCALM Aerial LiDAR Catalog](https://calm.geo.berkeley.edu/ncalm/dtc.html). \n",
+    "\n",
+    "```{note}\n",
+    "The NCALM Aerial LiDAR Catalog also includes user-submitted flights\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# first, we'll search for NOAA Coastal LiDAR missions in Washington\n",
+    "# we'll inspect 2018 arbitrarily\n",
+    "aoi = gpd.read_file(\n",
+    "    \"https://raw.githubusercontent.com/unitedstates/districts/refs/heads/gh-pages/states/WA/shape.geojson\"\n",
+    ")\n",
+    "date = \"2018\"\n",
+    "aoi.plot();"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "gf_noaa = coincident.search.search(dataset=\"noaa\", intersects=aoi, datetime=[date])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "gf_noaa"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "gf_noaa.explore(column=\"title\", cmap=\"Set1\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# now, let's see if there were any NCALM missions from the same year\n",
+    "gf_ncalm = coincident.search.search(dataset=\"ncalm\", intersects=aoi, datetime=[date])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "gf_ncalm"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "gf_ncalm.explore(column=\"id\", cmap=\"Set3\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## NEON API\n",
+    "\n",
+    "We also support the use of the [NEON Python API](https://www.neonscience.org/resources/learning-hub/tutorials/neon-api-intro-requests-py).\n",
+    "\n",
+    "```{warning}\n",
+    "Searching large areas and/or large time periods in the NEON catalog will take a long time due to the limited spatiotemporal search supported by the API.\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%time\n",
+    "gf_neon = coincident.search.search(dataset=\"neon\", intersects=aoi, datetime=[date])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "gf_neon"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "m = gf_noaa.explore(color=\"blue\")\n",
+    "gf_ncalm.explore(m=m, color=\"black\")\n",
+    "gf_neon.explore(m=m, color=\"deeppink\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## NASA LVIS\n",
+    "\n",
+    "Currently, only a select number of LVIS flghts are supported due to the limited amount of LVIS products hosted on STAC catalogs:\n",
+    "- [ABoVE LVIS L2 Geolocated Surface Elevation Product, Version 1](https://nsidc.org/data/ablvis2/versions/1)\n",
+    "    - Search extent limited to bbox [-158, 48, -104, 72] and 2017-06-29 to 2017-07-17\n",
+    "- [AfriSAR LVIS L2 Geolocated Surface Elevation Product, Version 1](https://nsidc.org/data/aflvis2/versions/1)\n",
+    "    - Search extent limited to bbox [8, -2, 12, 1] and 2016-02-20 to 2016-03-08\n",
+    "\n",
+    "```{warning}\n",
+    "Searching highly complex multi-polygons might break the STAC search, so simplifying your polygon before searching is recommended.\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# first, let's look at the ABoVE product in Canada\n",
+    "aoi = gpd.read_file(\n",
+    "    \"https://gist.githubusercontent.com/M1r1k/d5731bf39e1dfda5b53b4e4c560d968d/raw/c774258085ddc11776591ce95f2240d0fd0657a2/canada_provinces.geo.json\"\n",
+    ")\n",
+    "aoi = aoi[aoi[\"name\"] == \"Yukon\"].reset_index(drop=True)\n",
+    "# reduce complexity of multipolygon input\n",
+    "aoi.geometry = aoi.geometry.convex_hull"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "gf_lvis_ab = coincident.search.search(dataset=\"ablvis2_1\", intersects=aoi)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "print(gf_lvis_ab.shape)\n",
+    "gf_lvis_ab.head(2)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "m = aoi.explore(color=\"gray\")\n",
+    "gf_lvis_ab.clip(aoi).explore(m=m, column=\"datetime\", cmap=\"inferno\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# now, let's look at AfriSAR LiDAR\n",
+    "# this polygon is already simplified\n",
+    "aoi = gpd.read_file(\n",
+    "    \"https://raw.githubusercontent.com/glynnbird/countriesgeojson/refs/heads/master/gabon.geojson\"\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "gf_lvis_af = coincident.search.search(\n",
+    "    dataset=\"aflvis2_1\", intersects=aoi, datetime=[\"2016-02-20\"]\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "print(gf_lvis_af.shape)\n",
+    "gf_lvis_af.head(2)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "gf_lvis_af.explore(column=\"datetime\", cmap=\"inferno\")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "dev",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

docs/examples/index.md

@@ -10,4 +10,5 @@ quickstart
 cascading_search
 sliderule
 contextual_data
+additional_lidar
 ```

docs/user_guide/datasets.md

@@ -5,15 +5,20 @@ Below we provide a short table summarizing datasets that are searchable with
 different providers, the _provider_ column identifies the source of the data
 used by this library.
 
-| Dataset        | Alias      | Type      | Start      | End        | Extent        | Source                                                                      |
-| -------------- | ---------- | --------- | ---------- | ---------- | ------------- | --------------------------------------------------------------------------- |
-| TanDEM-X       | tdx        | SAR       | 2007-07-01 |            | global        | [NASA CSDAP](https://csdap.earthdata.nasa.gov/stac/collections/airbus)      |
-| Maxar Stereo   | maxar      | VHR       | 2007-07-01 |            | global        | [Maxar](https://developers.maxar.com/docs/discovery/)                       |
-| Coperincus DEM | cop30      | SAR       | 2021-04-22 |            | global        | [Microsoft](https://planetarycomputer.microsoft.com/dataset/cop-dem-glo-30) |
-| ICESat-2 ATL06 | atl06      | Altimeter | 2018-10-13 |            | global        | [NASA](https://nsidc.org/data/atl03)                                        |
-| GEDI L2A       | gedi       | Altimeter | 2019-04-04 | 2023-03-17 | mid-latitudes | [NASA](https://lpdaac.usgs.gov/products/gedi02_av002/)                      |
-| 3DEP LiDAR     | 3dep       | LiDAR     | 2000-12-01 |            | CONUS         | [USGS](https://www.usgs.gov/3d-elevation-program)                           |
-| ESA WorldCover | worldcover | LULC      | 2020-01-01 | 2021-12-31 | global        | [Microsoft](https://planetarycomputer.microsoft.com/dataset/esa-worldcover) |
+| Dataset            | Alias      | Type      | Start      | End        | Extent         | Source                                                                      |
+| ------------------ | ---------- | --------- | ---------- | ---------- | -------------- | --------------------------------------------------------------------------- |
+| TanDEM-X           | tdx        | SAR       | 2007-07-01 |            | global         | [NASA CSDAP](https://csdap.earthdata.nasa.gov/stac/collections/airbus)      |
+| Maxar Stereo       | maxar      | VHR       | 2007-07-01 |            | global         | [Maxar](https://developers.maxar.com/docs/discovery/)                       |
+| Coperincus DEM     | cop30      | SAR       | 2021-04-22 |            | global         | [Microsoft](https://planetarycomputer.microsoft.com/dataset/cop-dem-glo-30) |
+| ICESat-2 ATL06     | atl06      | Altimeter | 2018-10-13 |            | global         | [NASA](https://nsidc.org/data/atl03)                                        |
+| GEDI L2A           | gedi       | Altimeter | 2019-04-04 | 2023-03-17 | mid-latitudes  | [NASA](https://lpdaac.usgs.gov/products/gedi02_av002/)                      |
+| 3DEP LiDAR         | 3dep       | LiDAR     | 2000-12-01 |            | CONUS          | [USGS](https://www.usgs.gov/3d-elevation-program)                           |
+| ESA WorldCover     | worldcover | LULC      | 2020-01-01 | 2021-12-31 | global         | [Microsoft](https://planetarycomputer.microsoft.com/dataset/esa-worldcover) |
+| NOAA Coastal LiDAR | noaa       | LiDAR     | 1996-10-09 |            | US Territories | [NOAA](https://coast.noaa.gov/digitalcoast/data/coastallidar.html)          |
+| NCALM LiDAR        | ncalm      | LiDAR     | 2003-05-15 |            | US Territories | [NCALM](https://calm.geo.berkeley.edu/ncalm/dtc.html)                       |
+| NEON LiDAR         | neon       | LiDAR     | 2013-06-01 |            | US Territories | [NEON](https://data.neonscience.org/data-products/DP3.30024.001)            |
+| ABoVE LVIS L2      | ablvis2_1  | LiDAR     | 2017-06-29 | 2017-07-17 | AK and Canada  | [ABoVE LVIS L2](https://nsidc.org/data/ablvis2/versions/1)                  |
+| AfriSAR LVIS L2    | aflvis2_1  | LiDAR     | 2016-02-20 | 2016-03-08 | Gabon          | [AfriSAR LVIS L2](https://nsidc.org/data/aflvis2/versions/1)                |
 
 ## Other data sources
 

pyproject.toml

@@ -120,6 +120,7 @@ report.exclude_also = [
 [tool.codespell]
 # Ignore notebooks with output cells
 skip="pixi.lock,docs/examples/sliderule.ipynb,docs/examples/contextual_data.ipynb"
+ignore-words-list="ALOS"
 
 [tool.mypy]
 files = ["src", "tests"]

src/coincident/datasets/__init__.py

@@ -12,7 +12,15 @@
 
 from __future__ import annotations
 
-from coincident.datasets import csda, maxar, nasa, planetary_computer, usgs
+from coincident.datasets import (
+    csda,
+    maxar,
+    nasa,
+    neon,
+    opentopo,
+    planetary_computer,
+    usgs,
+)
 from coincident.datasets.general import Dataset
 
 # Convenience mapping of string aliases to supported dataset classes
@@ -21,12 +29,17 @@
     usgs.ThreeDEP(),
     nasa.ICESat2(),
     nasa.GEDI(),
+    nasa.ABLVIS2_1(),
+    nasa.AFLVIS2_1(),
     planetary_computer.COP30(),
     planetary_computer.WorldCover(),
     csda.TDX(),
+    opentopo.NOAA(),
+    opentopo.NCALM(),
+    neon.NEON(),
 ]
 
 aliases = [x.alias for x in _datasets]
 _alias_to_Dataset = dict(zip(aliases, _datasets, strict=False))
 
-__all__ = ["Dataset", "usgs", "maxar", "nasa", "planetary_computer", "csda"]
+__all__ = ["Dataset", "usgs", "maxar", "nasa", "planetary_computer", "csda", "opentopo"]