Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spotchecking the notebooks #520

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
337 changes: 129 additions & 208 deletions notebooks/00-quickstart.ipynb

Large diffs are not rendered by default.

28 changes: 14 additions & 14 deletions notebooks/01-advanced-deblending.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -109,9 +109,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"All implemented deblenders follow the same template abstract `Deblender` class. Basically, `Deblender` class standardizes inputs and outputs of the deblender, which allows to simultaneously run multiple deblenders on a generated data and benchmark their performance. \n",
"All implemented deblenders follow the same template: the abstract `Deblender` class. Basically, the `Deblender` class standardizes the inputs and outputs of the deblender, which allows to simultaneously run multiple deblenders on generated data and benchmark their performance. \n",
"\n",
"The `__init__` class has a required argument `max_n_sources`, which is needed to set a fixed shape for the output of the deblender. All deblenders must implement `__deblend__` method, which performs deblending on the i-th example from the `BlendBatch`. Then, we require the output to be packaged into `DeblendExample` data class, which peforms internal validation of the data format -- more on that later. The `__call__` and `batch_call` are internally implemented in the parent class, and user doesn't need to modify them to create a deblender. These two methods implement multiprocessing procedure to efficiently paralelize computation across several CPU cores. Finally `__repr__` class is used for internall bookkeeping. "
"The `__init__` class has a required argument `max_n_sources`, which is needed to set a fixed shape for the output of the deblender. All deblenders must implement `__deblend__` method, which performs deblending on the i-th example from the `BlendBatch`. Then, we require the output to be packaged into `DeblendExample` data class, which performs internal validation of the data format -- more on that later. The `__call__` and `batch_call` methods are internally implemented in the parent class, meaning that the user does not need to modify them when creating a new deblender. These two methods implement a multiprocessing procedure to efficiently parallelize computation across several CPU cores. Finally `__repr__` class is used for internal bookkeeping. "
]
},
{
Expand Down Expand Up @@ -200,7 +200,7 @@
"\n",
"Now comes the crucial step —- implementing the `deblend` method. The first step of any `deblend` method is to index into the `BlendBatch` and select the data that corresponds to the i-th example. In this case, we just need the `blend_image`. \n",
"\n",
"Once we selected the target grayscale image (either by averaging or by fixing a band) we generate random detections based on the image dimensions. Notice that the output coordinates will be in the pixel values, but the requirement for the BTK is to convert output coordinates to `ra` and `dec`, and wrap the detection in `atropy.Table`. Finally, we package the output into an object called `DeblendExample` that conveniently stores our data. "
"Once we have selected the target grayscale image (either by averaging or by choosing a specific band), we generate random detections based on the image dimensions. Notice that the output coordinates will be in the pixel values, but the requirement for the BTK is to convert output coordinates to `ra` and `dec`, and wrap the detection in `astropy.Table`. Finally, we package the output into an object called `DeblendExample` that conveniently stores our data. "
]
},
{
Expand All @@ -209,7 +209,7 @@
"source": [
"# Deblend Example \n",
"\n",
"BTK supports three kinds of blending realated tasks —— **detection**, **segmentation**, and **deblending** of individual sources. Data for all of these tasks can be stored in the `DeblendingExample`. Specifically, we **require** the user to pass catalog with detections, while **segmentation** and **deblended_images** are optional. This is done because **detection** is crucial for doing any other task like (segmentation and deblending), and also we use detections to match the output of the deblender to the ground truth of generated data. \n",
"BTK supports three kinds of deblending-related tasks —— **detection**, **segmentation**, and **deblending** of individual sources. Data for all of these tasks can be stored in the `DeblendingExample`. Specifically, we **require** the user to return a catalog with detections, while **segmentation** and **deblended_images** are optional. This is done because **detection** is crucial for doing any other task, and also we use detections to match the output of the deblender to the ground truth of generated data. \n",
"\n",
"```python\n",
"class DeblendExample:\n",
Expand All @@ -225,10 +225,10 @@
"```\n",
"\n",
"\n",
"The create an instance of the `DeblendExample` we need some of the following parameters:\n",
"- `max_n_sources` maximum number of galaxies on the stamp set in the generator. This should be the same across all examples. \n",
"To create an instance of the `DeblendExample`, we need some of the following parameters:\n",
"- `max_n_sources` - maximum number of galaxies on the stamp set in the generator. This should be the same across all examples. \n",
"- `n_bands` - number of image channels (filters) needed for internal validation\n",
"- `image_size` -- size of the image needed for internal validation\n",
"- `image_size` - size of the image needed for internal validation\n",
"- `catalog` - astropy table that must contain `ra` and `dec` columns with the coordinates of the detected galaxies (see example above)\n",
"- `segmentation` - `np.array` of shape `[self.max_n_sources, self.image_size, self.image_size]` with values 1 if pixel belongs to the ith galaxy and 0 otherwise\n",
"- `deblended_images` - `np.array` of shape `[self.max_n_sources, self.n_bands, self.image_size, self.image_size]` where `deblended_images[i]` is the ith deblended image. If detected less than `self.max_n_sources` remaining images should be zeros. "
Expand All @@ -239,7 +239,7 @@
"metadata": {},
"source": [
"## Running Deblenders\n",
"Let's test the output of the `RandomDetector` on an example batch"
"Let's test the output of the `RandomDetector` on an example batch:"
]
},
{
Expand Down Expand Up @@ -269,7 +269,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Segmentation, and deblended images are None because our deblenders only does the detection. Let's look at the detected coordiantes. Technically, we require only `ra` and `dec` to be saved, but saving `x_peak` and `y_peak` can be useful for visualization purposes."
"Segmentation and deblended images are None since our deblender only does the detection. Let's look at the detected coordinates. Technically, we require only `ra` and `dec` to be saved, but saving `x_peak` and `y_peak` can be useful for visualization purposes."
]
},
{
Expand Down Expand Up @@ -330,7 +330,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"As we can see our detections are indeed random. Now suppose we want to efficeintly run our deblender on the entire batch. We can do so by calling the following command:"
"As we can see our detections are indeed random. Now suppose we want to efficiently run our deblender on the entire batch. We can do so by calling the following command:"
]
},
{
Expand Down Expand Up @@ -358,7 +358,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally if we want to collect more statistics about the performace of our deblender, we can run our deblender on many batches using `DeblendGenerator`. We will also compute recall of our algorithm."
"Finally if we want to collect more statistics about the performance of our deblender, we can run our deblender on many batches using `DeblendGenerator`. We will also compute recall of our algorithm."
]
},
{
Expand Down Expand Up @@ -425,7 +425,7 @@
"source": [
"# Segmentation and Deblending \n",
"\n",
"As another example we can look at the `SepSingleBand` deblender in BTK. This deblender uses SourceExtractor to perform all three tasks — detection, segmentation, and deblending. "
"As an additional example, we can look at the `SepSingleBand` deblender in BTK. This deblender uses SourceExtractor to perform all three tasks — detection, segmentation, and deblending. "
]
},
{
Expand Down Expand Up @@ -647,7 +647,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "btk310_1",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
Expand All @@ -661,7 +661,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.11"
"version": "3.10.12"
}
},
"nbformat": 4,
Expand Down
18 changes: 9 additions & 9 deletions notebooks/01-advanced-generation.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"To write your own sampling function, you should create a subclass of the `SamplingFunction` class available in the `btk.sampling_functions.py` module, or from one of the classes in that file. \n",
"To write your own sampling function, you should create a subclass of the `SamplingFunction` class available in the `btk.sampling_functions` module, or from one of the classes in that file. \n",
"\n",
"Below you can find the implementation of the `DefaultSamplingFunction` that is also available in this module (as an example): \n",
"\n",
Expand Down Expand Up @@ -272,7 +272,7 @@
"source": [
"Your new sampling function should have the arguments in its initializer (`__init__`) that are at least the ones contained in the parent `SamplingFunction` class namely: `max_number`, `min_number`, and `seed`. \n",
"\n",
"It should also implement the `__call__` method, which should take as an argument an `astropy` table corresponding to the full galaxy catalog you are using to produce your simulations and return a smaller `astropy` table correspnoding to a subset of the entries which will be the galaxies forming a galaxy blend. Each time the `__call__` method is used, a new blend should be returned and the randomness needed for this should be derived from the random number generator `self.rng`, which ensures reproducibility in BTK. Finally, the positions of galaxies should be adjusted to be inside the postage stamp and w.r.t to its center (0,0). In the example above, we completely overwrite the galaxies centroids and set them to be random shifts from the center of the stamp up to some `self.max_shift`, but there are other ways of implementing this step to preserve some information on the relative positions of galaxies within the catalog. See, for example, `btk.sampling_functions.RandomSquareSampling` sampling function. For more details on implementing a `__call__` method for a sampling function, see the comments in the example above."
"It should also implement the `__call__` method, which should take as an argument an `astropy` table corresponding to the full galaxy catalog you are using to produce your simulations and return a smaller `astropy` table corresponding to a subset of the entries which will be the galaxies forming a galaxy blend. Each time the `__call__` method is used, a new blend should be returned and the randomness needed for this should be derived from the random number generator `self.rng`, which ensures reproducibility in BTK. Finally, the positions of galaxies should be adjusted to be inside the postage stamp and w.r.t to its center (0,0). In the example above, we completely overwrite the galaxies centroids and set them to be random shifts from the center of the stamp up to some `self.max_shift`, but there are other ways of implementing this step to preserve some information on the relative positions of galaxies within the catalog. See, for example, `btk.sampling_functions.RandomSquareSampling` sampling function. For more details on implementing a `__call__` method for a sampling function, see the comments in the example above."
]
},
{
Expand All @@ -290,7 +290,7 @@
"\n",
"BTK now relies on an external package named `surveycodex`, which provides two types of objects containing those informations. Here is a brief outline of how they work.\n",
"\n",
"A `Survey` is defined as a python dataclass, that is a class with some standard methods for accessing the parameters. Most of these parameters are using the astropy unit system. Here are all the fields relevant to BTK that a survey contains:\n",
"A `Survey` is defined as a Python dataclass, that is a class with some standard methods for accessing the parameters. Most of these parameters are using the astropy unit system. Here are all the fields relevant to BTK that a survey contains:\n",
"\n",
"* `name`: Name of the survey\n",
"* `description` : Description of the survey\n",
Expand All @@ -310,8 +310,8 @@
"* `zeropoint`: Magnitude of an object giving a measured flux of 1 electron per second\n",
"* `extinction`: exponential coefficient describing the absorption of light by the atmosphere.\n",
"* `psf`: PSF for the filter. This can be provided in two ways:\n",
" Providing a `galsim` PSF model, e.g. `galsim.Kolmogorov(fwhm)` or any convolution of such models.\n",
" Providing a *function* which returns a Galsim model when called (with no arguments). This can be used when you you want to randomize the PSF. In the case of the default settings in BTK, we always use the same constant PSF for each filter in a given survey, computed from the `btk.survey._get_default_psf`. This PSF has an atmospheric and an optical component."
" * Providing a `galsim` PSF model, e.g. `galsim.Kolmogorov(fwhm)` or any convolution of such models.\n",
" * Providing a *function* which returns a Galsim model when called (with no arguments). This can be used when you you want to randomize the PSF. In the case of the default settings in BTK, we always use the same constant PSF for each filter in a given survey, computed from the `btk.survey._get_default_psf`. This PSF has an atmospheric and an optical component."
]
},
{
Expand All @@ -320,7 +320,7 @@
"source": [
"Internally, BTK actually uses its own `Survey` and `Filter` objects, which inherit from the `surveycodex` object. The main difference is that the BTK objects can be modified while the `surveycodex` objects cannot. \n",
"\n",
"Surveys are usually imported using the `btk.survey.get_surveys` function, which will create the Survey object(s) from surveycodex (currently, the implemented surveys are LSST, HSC, COSMOS, Euclid, DES and CFHT); it is currently difficult to create new surveys. We encourage you to reach out to the BTK or GalCheat team in github if creating a new survey would be helpful. \n",
"Surveys are usually imported using the `btk.survey.get_surveys` function, which will create the Survey object(s) from surveycodex (currently, the implemented surveys are LSST, HSC, COSMOS, Euclid, DES and CFHT); it is currently difficult to create new surveys. We encourage you to reach out to the BTK or SurveyCodex team on github if creating a new survey would be helpful. \n",
"\n",
"Here is an example of how parameters can be accessed and modified:"
]
Expand Down Expand Up @@ -361,7 +361,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"To use your own PSF or use a randomized PSF, you can create a function that returns the PSF as a `galsim` object given a `Survey` and `Filter` object. For example: "
"To use your own PSF or a randomized PSF, you can create a function that returns the PSF as a `galsim` object given a `Survey` and `Filter` object. For example: "
]
},
{
Expand Down Expand Up @@ -603,7 +603,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "btk310_1",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
Expand All @@ -617,7 +617,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.11"
"version": "3.10.12"
}
},
"nbformat": 4,
Expand Down
123 changes: 55 additions & 68 deletions notebooks/01-advanced-metrics.ipynb

Large diffs are not rendered by default.

211 changes: 71 additions & 140 deletions notebooks/02-advanced-plots.ipynb

Large diffs are not rendered by default.

Binary file modified notebooks/figures/effmatrix.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading