Sort entries in bib files.

[marcfehling]

biber offers the option to sort all entries in bib files (see 3.1.7 in biber doc).

Although this brings no potential benefit for publication lists of previous years, I think this would be quite useful for the current year and upcoming ones. It would be a lot easier to find duplicate entries early if we sort by author for example.

I can also imagine including a check like indent from the main repository: We can check if the sorted bib file differs from the one in a PR, and request the author to change it. It is also not difficult to add a sort target to the makefile.

The check in #388 is a good first step to find duplicates. However, it only covers duplicate bibtex keys. With a sorted publication list, it would be easier to find real duplicates.

What do you think? I am confident that I can prepare a patch in a few minutes :)

[marcfehling]

Just out of curiosity: If you run biber --tool publications*.bib in the main folder, it will create a separate file containing all bibtex entries from all years combined, formatted according to the biber default style: publications-1998_bibertool.bib

We can also provide our own stylefile, similar to clang-format.

[marcfehling]

So far, I count 44 duplicates in our publication list. I will have a look a them tomorrow and sort them out (probably finding some false positives on the way)

$ grep -r "*1" publications*
publications-2002.bib:@Article{cockburn.kanschat.ea:local*1,
publications-2002.bib:@Article{hartmann.houston:adaptive*1,
publications-2003.bib:@Article{hartmann.houston:goal-oriented*1,
publications-2003.bib:@Article{kanschat:preconditioning*1,
publications-2004.bib:@Article{joshi.bangerth.ea:adaptive*1,
publications-2006.bib:@Article{bangtsson.lund:comparison*1,
publications-2006.bib:@Article{hartmann.houston:symmetric*1,
publications-2006.bib:@Article{joshi.bangerth.ea:non-contact*1,
publications-2009.bib:@PhDThesis{willems:numerical*1,
publications-2009.bib:@Article{ziegler.brendel.ea:investigation*1,
publications-2010.bib:@Article{leicht.hartmann:error*1,
publications-2011.bib:@Article{freddi.royer-carfagni:variational*1,
publications-2011.bib:@Article{hinze.kunkel.ea:model*1,
publications-2012.bib:@Article{carraro.joos.ea:3d*1,
publications-2012.bib:@Article{efendiev.galvis.ea:robust*1,
publications-2012.bib:@Article{hai:numerical*1,
publications-2013.bib:@Article{bangerth.heister.ea:deal-ii*1,
publications-2013.bib:@Article{bonito.kyza.ea:time-discrete*1,
publications-2013.bib:@Article{frei.richter.ea:eulerian*1,
publications-2013.bib:@Article{hartmann:higher-order*1,
publications-2014.bib:@Article{frei.richter.ea:eulerian*1,
publications-2015.bib:@Article{hai.bause:adaptive*1,
publications-2015.bib:@Article{hai.bause:finite*1,
publications-2015.bib:@PhDThesis{kircheis:structure*1,
publications-2015.bib:@Article{mikelic.wheeler.ea:phase-field*1,
publications-2015.bib:@Article{nochetto.otarola.ea:pde*1,
publications-2015.bib:@Article{zheng.mcclarren:non-oscillatory*1,
publications-2016.bib:@TechReport{lee.mikelic.ea:phase-field*1,
publications-2016.bib:@Article{voll.stollenwerk.ea:computing*1,
publications-2016.bib:@Article{wick:coupling*1,
publications-2017.bib:@Article{arbogast.hesse.ea:mixed*1,
publications-2017.bib:@Article{brisard:reconstructing*1,
publications-2017.bib:@Article{seydel.schuster:identifying*1,
publications-2018.bib:@Article{basak.levitas:nanoscale*1,
publications-2018.bib:@Article{brauss.meir:on*1,
publications-2018.bib:@Article{choo.lee:enriched*1,
publications-2018.bib:@Article{failer.wick:adaptive*1,
publications-2018.bib:@Article{ilio.dorschner.ea:simulation*1,
publications-2018.bib:@PhDThesis{sabawi:discontinuous*1,
publications-2018.bib:@Article{sartori.giuliani.ea:deal2lkit*1,
publications-2019.bib:@Article{hai.bause.ea:modeling*1,
publications-2021.bib:@Article{arndt.bangerth.ea:deal-ii*1,
publications-2021.bib:@Article{pagani.dede.ea:computational*1,
publications-2022.bib:@Article{salvador.regazzoni.ea:role*1,

$ grep -r "*1" publications* | wc
     44      44    2591

[marcfehling]

biber found 109 (!) duplicates when calling biber --tool publications*.bib to build a global publication list. I guess these might be preprints from previous years. Again, I am sure that there are false positives. I will have a look.

WARN - Duplicate entry key: 'bangerth.rannacher:adaptive' in file 'publications-2001.bib', skipping ...
WARN - Duplicate entry key: 'bangerth:adaptive' in file 'publications-2002.bib', skipping ...
WARN - Duplicate entry key: 'hartmann:adaptive' in file 'publications-2002.bib', skipping ...
WARN - Duplicate entry key: 'bangerth.rannacher:adaptive' in file 'publications-2003.bib', skipping ...
WARN - Duplicate entry key: 'cockburn.kanschat.ea:local' in file 'publications-2003.bib', skipping ...
WARN - Duplicate entry key: 'boffi.gastaldi.ea:finite' in file 'publications-2005.bib', skipping ...
WARN - Duplicate entry key: 'cockburn.kanschat.ea:local' in file 'publications-2005.bib', skipping ...
WARN - Duplicate entry key: 'hartmann:adaptive' in file 'publications-2006.bib', skipping ...
WARN - Duplicate entry key: 'hartmann:error' in file 'publications-2007.bib', skipping ...
WARN - Duplicate entry key: 'bangerth:framework' in file 'publications-2008.bib', skipping ...
WARN - Duplicate entry key: 'bangtsson.lund:comparison' in file 'publications-2008.bib', skipping ...
WARN - Duplicate entry key: 'boffi.gastaldi.ea:on' in file 'publications-2008.bib', skipping ...
WARN - Duplicate entry key: 'hartmann:error' in file 'publications-2008.bib', skipping ...
WARN - Duplicate entry key: 'joshi.bangerth.ea:non-contact' in file 'publications-2008.bib', skipping ...
WARN - Duplicate entry key: 'bangerth.joshi:adaptive' in file 'publications-2009.bib', skipping ...
WARN - Duplicate entry key: 'ghysels.samaey.ea:multi-scale' in file 'publications-2009.bib', skipping ...
WARN - Duplicate entry key: 'freddi.fremond:collision' in file 'publications-2010.bib', skipping ...
WARN - Duplicate entry key: 'freddi.royer-carfagni:variational' in file 'publications-2010.bib', skipping ...
WARN - Duplicate entry key: 'freddi.royer-carfagni:variational' in file 'publications-2011.bib', skipping ...
WARN - Duplicate entry key: 'george.gupta.ea:empirical' in file 'publications-2011.bib', skipping ...
WARN - Duplicate entry key: 'leicht.hartmann:error' in file 'publications-2011.bib', skipping ...
WARN - Duplicate entry key: 'cangiani.georgoulis.ea:discontinuous' in file 'publications-2012.bib', skipping ...
WARN - Duplicate entry key: 'hinze.kunkel.ea:model' in file 'publications-2012.bib', skipping ...
WARN - Duplicate entry key: 'hinze.kunkel.ea:pod' in file 'publications-2012.bib', skipping ...
WARN - Duplicate entry key: 'stoll.wathen:all-at-once' in file 'publications-2012.bib', skipping ...
WARN - Duplicate entry key: 'hartmann:higher-order' in file 'publications-2013.bib', skipping ...
WARN - Duplicate entry key: 'kramer:cuda-based' in file 'publications-2013.bib', skipping ...
WARN - Duplicate entry key: 'stoll.wathen:all-at-once' in file 'publications-2013.bib', skipping ...
WARN - Duplicate entry key: 'wallraff.leicht.ea:numerical' in file 'publications-2013.bib', skipping ...
WARN - Duplicate entry key: 'willems:robust' in file 'publications-2013.bib', skipping ...
WARN - Duplicate entry key: 'alexe.sandu:space-time' in file 'publications-2014.bib', skipping ...
WARN - Duplicate entry key: 'fankhauser.wihler.ea:hp-adaptive' in file 'publications-2014.bib', skipping ...
WARN - Duplicate entry key: 'frei.richter.ea:eulerian' in file 'publications-2014.bib', skipping ...
WARN - Duplicate entry key: 'frei.richter.ea:eulerian*1' in file 'publications-2014.bib', skipping ...
WARN - Duplicate entry key: 'heltai.roy.ea:fully' in file 'publications-2014.bib', skipping ...
WARN - Duplicate entry key: 'hintermuller.schiela.ea:length' in file 'publications-2014.bib', skipping ...
WARN - Duplicate entry key: 'kim:analysis' in file 'publications-2014.bib', skipping ...
WARN - Duplicate entry key: 'kumar.noorden.ea:ale-based' in file 'publications-2014.bib', skipping ...
WARN - Duplicate entry key: 'richter.wick:on' in file 'publications-2014.bib', skipping ...
WARN - Duplicate entry key: 'schotzau.schwab.ea:exponential' in file 'publications-2014.bib', skipping ...
WARN - Duplicate entry key: 'willems:robust' in file 'publications-2014.bib', skipping ...
WARN - Duplicate entry key: 'adler.atherton.ea:energy' in file 'publications-2015.bib', skipping ...
WARN - Duplicate entry key: 'bangerth.heister.ea:deal-ii' in file 'publications-2015.bib', skipping ...
WARN - Duplicate entry key: 'el-kurdi.gross.ea:parallel' in file 'publications-2015.bib', skipping ...
WARN - Duplicate entry key: 'frisani.hassan:on' in file 'publications-2015.bib', skipping ...
WARN - Duplicate entry key: 'hai.bause:adaptive' in file 'publications-2015.bib', skipping ...
WARN - Duplicate entry key: 'hai.bause:finite' in file 'publications-2015.bib', skipping ...
WARN - Duplicate entry key: 'mallikarjunaiah.walton:on' in file 'publications-2015.bib', skipping ...
WARN - Duplicate entry key: 'riedlbauer.steinmann.ea:thermomechanical' in file 'publications-2015.bib', skipping ...
WARN - Duplicate entry key: 'tirupathi.hesthaven.ea:multilevel' in file 'publications-2015.bib', skipping ...
WARN - Duplicate entry key: 'adler.emerson.ea:constrained' in file 'publications-2016.bib', skipping ...
WARN - Duplicate entry key: 'bangerth.heister.ea:deal-ii' in file 'publications-2016.bib', skipping ...
WARN - Duplicate entry key: 'kim.zhang:optimized' in file 'publications-2016.bib', skipping ...
WARN - Duplicate entry key: 'kormann:time-space' in file 'publications-2016.bib', skipping ...
WARN - Duplicate entry key: 'li.hartmann:adjoint-based' in file 'publications-2016.bib', skipping ...
WARN - Duplicate entry key: 'nochetto.salgado.ea:diffuse' in file 'publications-2016.bib', skipping ...
WARN - Duplicate entry key: 'vidal-ferrandiz.fayez.ea:moving' in file 'publications-2016.bib', skipping ...
WARN - Duplicate entry key: 'wick:coupling' in file 'publications-2016.bib', skipping ...
WARN - Duplicate entry key: 'chandrashekar.zenk:well-balanced' in file 'publications-2017.bib', skipping ...
WARN - Duplicate entry key: 'deolmi.dahmen.ea:effective' in file 'publications-2017.bib', skipping ...
WARN - Duplicate entry key: 'hai.bause:finite' in file 'publications-2017.bib', skipping ...
WARN - Duplicate entry key: 'harmon.gamba.ea:numerical' in file 'publications-2017.bib', skipping ...
WARN - Duplicate entry key: 'hinze.kunkel.ea:model' in file 'publications-2017.bib', skipping ...
WARN - Duplicate entry key: 'wick:coupling' in file 'publications-2017.bib', skipping ...
WARN - Duplicate entry key: 'bonito.demlow.ea:priori' in file 'publications-2018.bib', skipping ...
WARN - Duplicate entry key: 'cangiani.georgoulis.ea:adaptive' in file 'publications-2018.bib', skipping ...
WARN - Duplicate entry key: 'charnyi.heister.ea:efficient' in file 'publications-2018.bib', skipping ...
WARN - Duplicate entry key: 'galiano.velasco:on' in file 'publications-2018.bib', skipping ...
WARN - Duplicate entry key: 'goll.wick.ea:dopelib' in file 'publications-2018.bib', skipping ...
WARN - Duplicate entry key: 'hai.bause:numerical' in file 'publications-2018.bib', skipping ...
WARN - Duplicate entry key: 'kronbichler.diagne.ea:fast' in file 'publications-2018.bib', skipping ...
WARN - Duplicate entry key: 'lee.mikelic.ea:phase-field' in file 'publications-2018.bib', skipping ...
WARN - Duplicate entry key: 'maier.margetis.ea:generation' in file 'publications-2018.bib', skipping ...
WARN - Duplicate entry key: 'maier.nemilentsau.ea:ultracompact' in file 'publications-2018.bib', skipping ...
WARN - Duplicate entry key: 'maier.rannacher:duality-based' in file 'publications-2018.bib', skipping ...
WARN - Duplicate entry key: 'na.sun:computational' in file 'publications-2018.bib', skipping ...
WARN - Duplicate entry key: 'neitzel.wick.ea:optimal' in file 'publications-2018.bib', skipping ...
WARN - Duplicate entry key: 'verner.garikipati:computational' in file 'publications-2018.bib', skipping ...
WARN - Duplicate entry key: 'wells.banks:using' in file 'publications-2018.bib', skipping ...
WARN - Duplicate entry key: 'arbogast.tao:direct' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'arndt.bangerth.ea:deal-ii' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'berselli.wells.ea:spatial' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'bonetti.cavaterra.ea:nonlinear' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'bruchhauser.schwegler.ea:numerical' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'charnyi.heister.ea:efficient' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'fu.jin.ea:parameter-free' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'gulevich.koshelets.ea:bridging' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'guo.wu.ea:understanding' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'hai.bause:numerical' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'kazemi.vaziri.ea:topology' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'krank.kronbichler.ea:wall' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'kronbichler.kormann.ea:fast' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'liu.reina:dynamic' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'liu.yin:unconditionally' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'maday.marcati:regularity' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'maier.mattheakis.ea:homogenization' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'mikelic.wheeler.ea:phase-field' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'neitzel.wick.ea:optimal' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'odster.kvamsdal.ea:simple' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'wick.wollner:on' in file 'publications-2019.bib', skipping ...
WARN - Duplicate entry key: 'arndt.bangerth.ea:deal-ii' in file 'publications-2020.bib', skipping ...
WARN - Duplicate entry key: 'cangiani.georgoulis.ea:posteriori' in file 'publications-2020.bib', skipping ...
WARN - Duplicate entry key: 'jodlbauer.langer.ea:parallel' in file 'publications-2020.bib', skipping ...
WARN - Duplicate entry key: 'mcbride.davydov.ea:modelling' in file 'publications-2020.bib', skipping ...
WARN - Duplicate entry key: 'araujo.engstrom:on' in file 'publications-2021.bib', skipping ...
WARN - Duplicate entry key: 'arndt.bangerth.ea:deal-ii' in file 'publications-2021.bib', skipping ...
WARN - Duplicate entry key: 'kourakos.harter:simulation' in file 'publications-2021.bib', skipping ...
WARN - Duplicate entry key: 'arndt.bangerth.ea:deal-ii' in file 'publications-2022.bib', skipping ...
WARN - Duplicate entry key: 'jodlbauer.langer.ea:matrix-free' in file 'publications-2022.bib', skipping ...

[marcfehling]

bibcure is also an interesting tool: https://github.com/bibcure/bibcure#bibcure