- In soma and compartment reports, add trial number in the extracted dataframe, as in the spikes report.
- In soma and compartment reports, make the exported time relative to the offset, as in the spikes report.
- Create
.zenodo.json
as a workaround to publish the release on Zenodo.
- Technical release.
- Technical release.
- When loading the cache, do not fail if the original simulation campaign and related files are missing [NSETM-2336].
- Fix functional tests.
- Ensure that some methods and functions are called with named parameters. In particular, the following public methods require named parameters:
blueetl.analysis.run_from_file()
blueetl.extract.report.ReportExtractor.from_simulations()
blueetl.repository.Repository.__init__()
blueetl.repository.FilteredRepository.__init__()
- Fix tests with libsonata 0.1.28.
- Skip simulations with empty path in the simulation campaign configuration (fix regression in blueetl 0.8.0).
- Update expected results in functional tests, because the output of statistics.Complexity changed in elephant >= 0.13.0.
- Fix warning when building docs, and include
blueetl.analysis
only.
- Fix version number in the analysis config created by
blueetl convert-spikes
.
- Revert raising an error if the env variable SHMDIR isn't set. Log a more detailed warning instead.
- Allow to access the cache config and the output path of individual analyses with
analysis_config.cache
andanalysis_config.output
, as a shortcut toanalysis_config.cache.path
. - Raise an error if the env variable
SHMDIR
isn't set, instead of logging a warning.
- Fix link to the example configuration in the documentation.
- Add
multi_index
option to the features configuration, to decide whetherreset_index()
should be applied to the features DataFrames. Note: the features cache will be rebuilt, although the resulting DataFrames are unchanged (because the default value of the new option isTrue
).
- Improve performance of report extraction and features calculation by writing partial DataFrames to the shared memory (or temp directory, if shared memory is not available). Both the used memory and the execution time should be lower than before, when processing large DataFrames.
- Use zstd compression instead of snappy when writing parquet files.
- When
repo
is pickled, extract the DataFrames only if they aren't already stored in the cache. - Remove fastparquet extra dependency.
- Add
cache.readonly
, to be able to use an existing cache without exclusive locking [NSETM-2310]. - Add
cache.store_type
, to change the file format (experimental). - Add
cache.skip_features
, to skip writing the features DataFrames [NSETM-2312].
- Deprecate
output
, usecache.path
instead. - Deprecate
clear_cache
, usecache.clear
instead.
- Ensure that invalid cache entries are always deleted from
checksums.cached.yaml
.
- Filter features only when
apply_filter
is called to save some time, but ensure that repo and features are filtered when they are loaded from the cache using a more restrictive filter. - Improve logging in
blueetl.utils.timed()
. - Improve tests coverage.
- Split migration documentation.
- Add
--sort/--no-sort
option toblueetl migrate-config
to sort the root keys of the converted configuration.
- If
node_id
in the neuron_classes configuration is set to an empty list, it's now considered as an empty selection instead of selecting all the neurons.
- Add configuration examples 11 and 12 to the documentation.
- Improve tests coverage.
- Fix method
Repository.missing_simulations()
.
- Support custom node_sets_file in extraction, neuron_classes and trial_steps [NSETM-2226]
- Fix expected data in functional tests.
- Allow to specify
trial_steps_label
to calculate the dynamic offset of trial steps [NSETM-2281]
- Allow to specify
trial_steps_list
instead oftrial_steps_value
andn_trials
[NSETM-2280]
- Temporarily disable
trial_steps_label
[NSETM-2281]
- Add tests for Python 3.12.
- Remove brion dependency in tests.
- Add CLI to convert and import inferred spikes in CSV format.
- Support relative paths in the simulation campaign config.
- Add a simple simulation campaign using a subsampled circuit, to run the Jupyter notebooks in the documentation.
- Update DOI.
- Fix docs build in rtd.
- Update badges.
- Conditionally skip tests requiring bluepy.
- First public release.
- Extend the API of SimulationCampaign (previously SimulationsConfig) to open simulation campaigns.
- Rename SimulationsConfig to SimulationCampaign.
- Support SONATA simulation campaigns, circuits, and reports using bluepysnap.
- Simulation campaigns, circuits, and reports using BlueConfig format aren't supported anymore.
- The analysis configuration accepts
population
andnode_set
, instead oftarget
. - In the
neuron_classes
definition, the query parameters must be moved toquery
,$limit
must be renamed tolimit
,$gids
tonode_id
. - The function
blueetl.core.utils.safe_concat
has been renamed tosmart_concat
. - The module
blueetl.core
has been moved to a separate package,blueetl-core
.
- The function
blueetl.core.utils.smart_concat
usescopy=False
by default, and accepts dictionaries aspd.concat
does. - All the internal calls to
pd.concat
are redirected tosmart_concat
.
- Improve performance of etl.add_conditions.
- Ensure that the package can be installed and used without optional dependencies.
- Support Pandas 2.0. Changed in Pandas 2.0.0: Index can hold all numpy numeric dtypes (except float16). Previously only int64/uint64/float64 dtypes were accepted.
- Add MultiAnalyzer class to support multiple reports [NSETM-2015]
- Allow to resolve windows by reference [NSETM-2015]
- Support combination of parameters in features configuration [NSETM-2091]
- Allow to access the concatenation of features dataframes using the basename [NSETM-2149]
- Add analysis configuration model and validation [NSETM-2099]
- Add blueetl CLI [NSETM-2115]
- Add blueetl.analysis.run_from_file [NSETM-2151]
- Improve performance of report extraction [NSETM-2116]
- Improve performance of features calculation [NSETM-2116]
- Process features in group when possible.
- Add _cached and _filtered private attributes to BaseExtractor.
- Add clear_cache parameter to run_from_file and to the configuration schema [NSETM-2150]
- Allow etl.q to support regular expressions [NSETM-2170]
- Allow etl.q to accept a list of query dicts [NSETM-2162]
- Allow neuron_classes configuration to be defined as a list of query dicts [NSETM-2163]
- The previous analysis configuration format has been replaced by the version 2 to support multiple reports.
- After applying a filter, the indices of the repo DataFrames are reset to remove any gap.
- The function
call_by_simulation
has been refactored and moved intoblueetl.parallel
. - In neuron_classes configuration,
gid
has been renamed to$gids
. - Require Python >= 3.9.
- Raise an exception if there are multiple features dataframes with the same name.
- Enforce the correct dtype in the features dataframes.
- Add
dtypes
parameter toETLBaseAccessor.add_conditions
. - Add
dtypes
andastype
methods toETLIndexAccessor
. - Support filtering by windows or neuron classes for each features configuration [NSETM-2085]
- Deepcopy the params dict passed to the user func.
- Ignore simulations for which BlueConfig no longer exists [NSETM-1967]
- Add optional in-memory filter [NSETM-1965]
- Support subtargets per neuron class [NSETM-2004]
- Add env variable
BLUEETL_SUBPROCESS_LOGGING_LEVEL
to set a logging level in subprocesses. - Improve log of execution times.
- Improve performances of
etl.q
when only a single condition is specified. - Lock the cache used by the Analyzer instance [NSETM-1971]
- Make the function call_by_simulation more flexible.
- Reset the index in the simulations dataframe after applying filters.
- Ensure that RangeIndex is converted to Int64Index in MultiIndexes with Pandas 1.5.0, see https://issues.apache.org/jira/browse/ARROW-17806.
First release including:
- Core Transformations
- Simulation Campaign Configuration
- Spike Analysis (Repository Extraction and Features Collection)