Sync with master. refs: #201

NEWS.md

@@ -1,3 +1,7 @@
+# New in version 8.8
+
+* Added `dballe.Explorer` examples to python HOWTO (#181)
+
 # New in version 8.7
 
 * Fixed the command line documentation of possible input types (#202)

README.md

@@ -53,34 +53,54 @@ The main characteristics of DB-ALL.e are:
  * Does not need backup, since it only contains replicated or derived data.
  * Write access is enabled for its users.
 
-[DB-All.e documentation](https://arpa-simc.github.io/dballe/).
+DB-All.e documentation: https://arpa-simc.github.io/dballe/
 
-Building DB-All.e
------------------
+Installing DB-All.e
+-------------------
 
 DB-All.e is already packaged in both .rpm and .deb formats, and that provides
 easy installation for most Linux distributions.
 
+For CentOS and Fedora, rpm files are hosted in a copr repo:
+https://copr.fedorainfracloud.org/coprs/simc/stable/
+
+For Debian, DB-All.e is available in the testing distribution:
+https://packages.debian.org/testing/dballe
+
+Using docker images with DB-All.e preinstalled is also possible:
+
+```
+docker run -it arpaesimc/fedora:31 /bin/bash
+docker run -it arpaesimc/centos:8 /bin/bash
+```
+
 If you want to build and install DB-All.e yourself, you'll need to install the
 automake/autoconf/libtool packages then you can proceed as in most other Unix 
 software:
 
-  autoreconf -if
-  ./configure
-  make
-  make install
+```
+autoreconf -if
+./configure
+make
+make install
+```
 
+If you're familiar with .rpm and .deb packaging you'll find the packaging 
+files in the `debian` and `fedora` directories.
 
 Getting started
 ---------------
 
 DB-All.e requires a database to run. It can create a SQLite database, or access
-a PostgreSQL or MySQL database. See doc/fapi_connect.md for details about
-connecting to a database.
+a PostgreSQL or MySQL database.
+For details about connecting to a database see:
+https://arpa-simc.github.io/dballe/general_ref/connect.html
 
 Once this is set up, you can initialise the DB-All.e database using the command::
 
-  dbadb wipe --url=sqlite:dballe.sqlite3
+```
+dbadb wipe --url=sqlite:dballe.sqlite3
+```
 
 If you do not already have access to datasets to import, some are available
 from http://www.ncar.ucar.edu/tools/datasets/ after registering (for free) on
@@ -90,22 +110,20 @@ the website.
 Documentation
 -------------
 
+DB-All.e documentation:
+https://arpa-simc.github.io/dballe/
+
 Documentation for all commandline tools can be found in their manpages.  All
 commandline tools also have extensive commandline help that can be accessed
-using the "--help" option.
-
-The Fortran API is documented in the fapi.pdf document.
-
-The C API and all the C internals are documented through Doxygen.
+using the `--help` option.
 
-Administration and maintanance of DB-All.e are covered in the guide.pdf
-document.
+The C API and all the C internals are also documented through Doxygen.
 
 
 Testing DB-All.e
 ----------------
 
-Unit testing can be run using "make check", but it requires an existing DSN
+Unit testing can be run using `make check`, but it requires an existing DSN
 connection to a MySQL database, which should be called 'test'.  Please note
 that unit testing functions will wipe existing DB-All.e tables on the test DSN
 database.
@@ -128,7 +146,7 @@ Contact and copyright information
 
 The author of DB-ALLe is Enrico Zini <[email protected]>
 
-DB-ALLe is Copyright (C) 2005-2018 ARPAE-SIMC <[email protected]>
+DB-ALLe is Copyright (C) 2005-2020 ARPAE-SIMC <[email protected]>
 
 DB-ALLe is licensed under the terms of the GNU General Public License version
 2.  Please see the file COPYING for details.

configure.ac

@@ -1,6 +1,6 @@
 dnl Process this file with autoconf to produce a configure script.
 
-AC_INIT(dballe, [8.6], [[email protected]])
+AC_INIT(dballe, [8.7], [[email protected]])
 AC_CONFIG_SRCDIR([configure.ac])
 AM_INIT_AUTOMAKE([foreign subdir-objects])
 AC_CONFIG_HEADER(config.h)

dballe/db/explorer.cc

@@ -164,21 +164,21 @@ void BaseExplorer<Station>::Update::add_json(core::json::Stream& in)
 template<typename Station> template<typename OStation>
 void BaseExplorer<Station>::Update::add_explorer(const BaseExplorer<OStation>& explorer)
 {
-    this->explorer->_global_summary->add_summary(explorer.global_summary());
+    this->explorer->_global_summary->add_summary(explorer.active_summary());
 }
 template<> template<>
 void BaseExplorer<Station>::Update::add_explorer(const BaseExplorer<Station>& explorer)
 {
     if (this->explorer == &explorer)
         wreport::error_consistency::throwf("Adding an Explorer to itself is not supported");
-    this->explorer->_global_summary->add_summary(explorer.global_summary());
+    this->explorer->_global_summary->add_summary(explorer.active_summary());
 }
 template<> template<>
 void BaseExplorer<DBStation>::Update::add_explorer(const BaseExplorer<DBStation>& explorer)
 {
     if (this->explorer == &explorer)
         wreport::error_consistency::throwf("Adding an Explorer to itself is not supported");
-    this->explorer->_global_summary->add_summary(explorer.global_summary());
+    this->explorer->_global_summary->add_summary(explorer.active_summary());
 }
 
 template<typename Station>

dballe/db/explorer.h

@@ -53,7 +53,7 @@ class BaseExplorer
         /// Load the explorer contents from JSON
         void add_json(core::json::Stream& in);
 
-        /// Merge the contents of another explorer into this one
+        /// Merge the currently selected contents of another explorer into this one
         template<typename OStation>
         void add_explorer(const BaseExplorer<OStation>& explorer);
 

doc/python_how/explorer.rst

@@ -0,0 +1,47 @@
+.. _python_how_explorer:
+
+Load/update/save an Explorer
+============================
+
+This is an example python code that loads the contents of a
+:class:`dballe.Explorer` from a JSON file, adds information from various `BUFR`
+files, and saves is back to JSON::
+
+   #!/usr/bin/python3
+   import dballe
+   import os
+   import sys
+
+   e = dballe.Explorer()
+
+   with e.update() as updater:
+       # Load existing json summary
+       if os.path.exists("xpl.json"):
+           with open("xpl.json", "rt") as fd:
+               updater.add_json(fd.read())
+
+       # Import files listed on command line
+       importer = dballe.Importer("BUFR")
+       for fname in sys.argv[1:]:
+           print(f"Load {fname}…")
+           with importer.from_file(fname) as f:
+               updater.add_messages(f)
+
+   # Write out
+   with open("xpl.json", "wt") as fd:
+       fd.write(e.to_json())
+
+
+Work with a subset of an Explorer
+=================================
+
+This an example that creates a new :class:`dballe.Explorer` with a selection of
+the data of an existing one::
+
+   e = dballe.Explorer()
+   # …fill e…
+   e.set_filter(...)
+
+   e1 = dballe.Explorer()
+   with e1.update() as updater:
+       updater.add_explorer(e)

doc/python_how/index.rst

@@ -7,3 +7,4 @@ Python HOWTOs
    dumpdb
    insert
    convertvars
+   explorer

fedora/SPECS/dballe.spec

@@ -4,7 +4,7 @@
 
 Summary: DB-ALLe is a database for point-based metereological data  (Command line tools)
 Name: dballe
-Version: 8.6
+Version: 8.7
 Release: %{releaseno}%{dist}
 License: GPL
 Group: Applications/Meteo
@@ -316,6 +316,11 @@ mv $RPM_BUILD_ROOT%{_includedir}/*.mod $RPM_BUILD_ROOT%{_fmoddir}
 
 
 %changelog
+* Tue Feb  4 2020 Daniele Branchini <[email protected]> - 8.7-1
+- Fixed the command line documentation of possible input types (#202)
+- Restructured and tested documentation (#204, #205, #206)
+- JSON is now supported for encoding/decoding wherever BUFR and CREX are (#202)
+
 * Mon Dec  9 2019 Daniele Branchini <[email protected]> - 8.6-1
 - Turned a segfault into a proper exception (#197)
 - Parse again '-' as missing (#200)

python/explorer.cc

@@ -590,7 +590,7 @@ struct add_explorer : public MethKwargs<add_explorer<Station>, typename ImplTrai
     typedef typename ImplTraits<Station>::UpdateImpl Impl;
     constexpr static const char* name = "add_explorer";
     constexpr static const char* doc = R"(
-Add the contents of the given Explorer or DBExplorer to the Explorer.
+Add the currently selected contents of the given Explorer or DBExplorer to the Explorer.
 )";
     static PyObject* run(Impl* self, PyObject* args, PyObject* kw)
     {
@@ -627,7 +627,7 @@ struct add_messages : public MethKwargs<add_messages<Station>, typename ImplTrai
     constexpr static const char* doc = R"(
 Add dballe.Message objects to the explorer.
 
-It takes the same messages argument of dballe.DB.import_messages
+It takes the same messages argument of :func:`dballe.DB.import_messages`.
 )";
     [[noreturn]] static void throw_typeerror()
     {

tables/dballe.txt

@@ -476,6 +476,10 @@
  048078 Vitacee                                                          POLLEN/M**3                1            0  24 POLLEN/M**3               1          8
  048079 Corilacee_Carpino                                                POLLEN/M**3                1            0  24 POLLEN/M**3               1          8
  048080 Corilacee_Ostria                                                 POLLEN/M**3                1            0  24 POLLEN/M**3               1          8
+ 048081 Moracee - Broussonetia                                           POLLEN/M**3                1            0  24 POLLEN/M**3               1          8
+ 048082 Moracee - Morus                                                  POLLEN/M**3                1            0  24 POLLEN/M**3               1          8
+ 048083 Leguminose                                                       POLLEN/M**3                1            0  24 POLLEN/M**3               1          8
+ 048084 Arecacee                                                         POLLEN/M**3                1            0  24 POLLEN/M**3               1          8
  048128 Conta Graminacee_Graminacee indistinte                           NUMERIC                    0            0  20 NUMERIC                   0          7
  048129 Conta Betulacee_Ontano nero                                      NUMERIC                    0            0  20 NUMERIC                   0          7
  048130 Conta Betulacee_Betulla                                          NUMERIC                    0            0  20 NUMERIC                   0          7
@@ -556,6 +560,10 @@
  048205 Conta Vitacee                                                    NUMERIC                    0            0  20 NUMERIC                   0          7
  048206 Conta Corilacee_Carpino                                          NUMERIC                    0            0  20 NUMERIC                   0          7
  048207 Conta Corilacee_Ostria                                           NUMERIC                    0            0  20 NUMERIC                   0          7
+ 048208 Conta Moracee - Broussonetia                                     NUMERIC                    0            0  20 NUMERIC                   0          7
+ 048209 Conta Moracee - Morus                                            NUMERIC                    0            0  20 NUMERIC                   0          7
+ 048210 Conta Leguminose                                                 NUMERIC                    0            0  20 NUMERIC                   0          7
+ 048211 Conta Arecacee                                                   NUMERIC                    0            0  20 NUMERIC                   0          7
  049192 Number of particles with diameter <= 0.3 um                      NUMERIC                   -1            0  16 NUMERIC                  -1          5
  049193 Number of particles with diameter > 0.3 <= 0.5 um                NUMERIC                   -1            0  16 NUMERIC                  -1          5
  049194 Number of particles with diameter > 0.5 <= 1.0 um                NUMERIC                   -1            0  16 NUMERIC                  -1          5