markdown.Rmd

---
title: "Markdown"
output: 
  html_document: 
    toc: yes
    toc_float: true
---


# Introduction

This page describes how to use markdown to edit texts such as reading notes,
meeting notes, technical documentation or even scientific publications.

[Common Mark](https://commonmark.org/) is an attempt at standardizing markdown
syntax. The common mark [help page](https://commonmark.org/help/) explains the
syntax.

See also the [publish](publish.html) page which mentions Latex tips and other
editing ideas.


# Images

![Image](https://avatars0.githubusercontent.com/u/5647397?v=4&s=460) 


# Pandoc convert to and from other file formats

See also the [bash](bash.html) page.

[Pandoc demos](https://pandoc.org/demos.html) contains a lot of example commands 
to convert text files from one format to another.
[Pandoc user guide](https://pandoc.org/MANUAL.html)

Convert a markdown document to pdf

    pandoc filname.md -o filename.pdf
    # If there are utf-8 encoding issues, use another engine
    pandoc readings_jrc.md -o readings_jrc.pdf --pdf-engine=xelatex

Convert a markdown document to pptx

    pandoc filname.md -o filename.pptx

Convert an odt document to markdown

    pandoc filename.odt -o filename.md 

Convert a Microsoft Word document to markdown

    pandoc filename.docx -o filename.md 

Convert a Microsoft Word document to markdown and **keep images**

    pandoc --extract-media figures input.docx -o output.md

Images are contained in a `media` subfolder in Word documents, the instruction
above leads to a `figure/media` sub folder. To avoid this [use the current
directory path](https://stackoverflow.com/a/52593892/2641825)

    pandoc --extract-media=. input.docx -o output.md

Convert an html code snippets to markdown and display the output at the command line

    echo "Link: <a href="https://www.linphone.org/">Linphone</a>"| pandoc -f html -t markdown
    # Link: [Linphone](https://www.linphone.org/)

Convert an entire web page to markdown

    pandoc https://pandoc.org/help.html -t markdown

[Markdown has two different header
styles](https://pandoc.org/MANUAL.html#headers) The default is to use
setext-style headers for levels 1-2, and then ATX headers. To use ATX style
headers only in markdown output:

    pandoc --atx-headers filename.docx -o filename.md 


## Bash script for bulk conversion

Convert many markdown files with this [bash script for
pandoc](https://github.com/FHefner/pandoc-docs/blob/master/pandoc-docs) 

    for mdfile in *.md; do
    pdffile=${mdfile%.md}.pdf
    echo $pdffile
    done

Convert all docx documents to markdown in the current folder

    for docxfile in *.docx; do
    mdfile=${docxfile%.docx}.md
    echo "converting $docxfile to $mdfile"
    pandoc "$docxfile" -o "$mdfile"
    done

Pandoc can convert to PDF, but not from PDF.

You can also create template files as explained in super user [markdown to latex conversion with a custom preamble](https://superuser.com/questions/356032/markdown-to-latex-conversion-with-a-custom-preamble-using-pandoc/356044#356044)

See also the makefile for pandoc in the programming with bash / makefile
section below.

## Pandoc for research papers


- [How to use Pandoc to produce a research
  paper](https://opensource.com/article/18/9/pandoc-research-paper)
  using the latex template of a journal to specify authorship in the way
  requested by the journal. Then the body of the article is written in
  Markdown.

- Pandoc needs to be called with the `--citeproc` argument to generate a list
  of references from the citations. The bibliography file can be specified in
  the `--bibliography` argument.

      pandoc article.md -o article.pdf --bibliography=article.bib --citeproc

    - See also the YAML front matter section on bibliography. Even if the yaml 
      front matter specifies a bibliography file, the `--citeproc` still needs 
      to be called explicitly in order to process citations and generate a list 
      of references.


# Quarto

Goals:

> The overarching goal of Quarto is to make the process of creating and 
> collaborating on scientific and technical documents dramatically better. We 
> hope to do this in several dimensions:

> Create a writing and publishing environment with great integrated tools 
> for technical content. We want to make authoring with embedded code, 
> equations, figures, complex diagrams, interactive widgets, citations, 
> cross references, and the myriad other special requirements of scientific 
> discourse straightforward and productive for everyone.

> Help authors take full advantage of the web as a connected, interactive 
> platform for communications, while still providing the ability to create 
> excellent printed output from the same document source. Researchers 
> shouldn’t need to choose between LaTeX, MS Word, and HTML but rather be 
> able to author documents that target all of them at the same time.

> Make reproducible research and publications the norm rather than the 
> exception. Reproducibility requires that the code and data required to 
> create a manuscript are an integrated part of it. However, this isn’t 
> often straightforward in practice—Quarto aims to make it easier to adopt 
> a reproducible workflow than not.


## Books that use quarto

- Seen at https://quarto.org/docs/books/

    - https://r4ds.hadley.nz/

    - https://wesmckinney.com/book/

    - https://jjallaire.github.io/visualization-curriculum


## Citations

To cite a reference in the format of "author (year)" from the bibliography, 
use:

    @citationkey

To cite a reference inside brackets in the form of "(author year)", place the 
citation key inside square brackets:

    [@citationkey]

To cite more than one reference:

     [see @doe99, pp. 33-35; also @smith04, chap. 1]. 

See also https://quarto.org/docs/visual-editor/technical.html#citations

    > "Citations go inside square brackets and are separated by semicolons."

The bibliography is located at the end of the document by default.
You can change its location with a `{#refs}` placeholder, as explained in 
https://quarto.org/docs/authoring/footnotes-and-citations.html

```{text}
    ::: {#refs}
    :::
```


## Code chunks

### Python and Jupyter

- Need to set the extension as `.qmd` to enable execution of code chunks

- Virtual environments 
  https://quarto.org/docs/projects/virtual-environments.html

- Debugging information 

    - verbose mode https://quarto.org/docs/troubleshooting/#verbose-mode

    - Log files https://quarto.org/docs/troubleshooting/#log-files

        - Add this to the yaml block

          execute:
              debug: true

        - See log file in `$HOME/.local/share/quarto/logs`

    - Using warnings to debug (it's not ideal, but at least they get printed in 
      the quarto output)

      import warnings
      warnings.warn(f"scenario_dir: {scenario_dir}\nexists: {scenario_dir.exists()}")


## Figures

Quarto documentation on figures https://quarto.org/docs/authoring/figures.html

Here is how to add a figure with caption and cross reference in quarto:

    ![An Elephant](elephant.png){#fig-elephant}

Example of a figure with caption and label, generated from python code. Note 
you would have to use single curly braces in the code chunk, these are only 
here so that this code chunk is not interpreted.

````

    >   ```{{python}}
    >   #| label: fig-plot
    >   #| fig-cap: "Plot 0, 1, 2, 3, 4"
    >   #| eval: false
    >   import matplotlib.pyplot as plt
    >   plt.plot([0,1,1,2,3,3,4], [0,1,3,2,1,3,4], "*c")
    >   plt.show()
    >   ```

````

Refer to the figures with an `@` label:

    see @fig-plot and @fig-elephant


## Tables

[Quarto tables](https://quarto.org/docs/authoring/tables.html)

> " The most commonly used markdown table is known as a pipe table. Pipe tables
> support specifying per column alignment as well as captions. For example:

    | Default | Left | Right | Center |
    |---------|:-----|------:|:------:|
    | 12      | 12   |    12 |   12   |
    | 123     | 123  |   123 |  123   |
    | 1       | 1    |     1 |   1    |


> "For tables produced by executable code cells, include a label with a `tbl-`
> prefix to make them cross-referenceable."


> "For markdown tables, add a caption below the table, then include a #tbl-
> label in braces at the end of the caption. For example:"

    | Col1 | Col2 | Col3 |
    |------|------|------|
    | A    | B    | C    |
    | E    | F    | G    |
    | A    | G    | G    |

    : My Caption {#tbl-letters}

    See @tbl-letters.


### Grid table

- Use grid tables to merge cells 
  https://github.com/quarto-dev/quarto-cli/discussions/2603

    +---------------------+----------+
    | Property            | Earth    |
    +=============+=======+==========+
    |             | min   | -89.2 °C |
    | Temperature +-------+----------+
    | 1961-1990   | mean  | 14 °C    |
    |             +-------+----------+
    |             | max   | 56.7 °C  |
    +-------------+-------+----------+

- Pandoc Grid tables https://pandoc.org/MANUAL.html#extension-grid_tables

- Force a new line  with `\ `
  https://stackoverflow.com/questions/33048001/how-do-i-get-a-hard-line-break-in-a-multiline-table-using-markdown

    +---------------------+----------+
    | Property            | Earth    |
    +=============+=======+==========+
    |             | min   | -89.2 °C |
    | Temperature +-------+----------+
    | 1961\       | mean  | 14 °C    |
    | -1990       +-------+----------+
    |             | max   | 56.7 °C  |
    +-------------+-------+----------+


## Team work

### With git

Co-authors load the quarto document to their machines and make edit suggestions 
as commits.


### With Google Docs

- R markdown and google docs 
  https://www.matthewvanaman.com/posts/2021-03-17-extremely-simple-guide-to-collaborative-writing-between-r-markdown-and-google-docs-users/collab_writing.html
  
    > "At each stage of revision, you, Stewardperson of R, will have written 
    > the initial draft within R Markdown. Once the draft is ready for 
    > revisions from collaborators, you will use R to upload the contents of 
    > your R Markdown document to a Google Doc on your Google Drive. From 
    > there, you and your collaborators use Google Docs to revise. Once 
    > everyone is satisfied with the content, you will download the Google Doc 
    > content back to your R Markdown file, which will rewrite the contents of 
    > the file to include any changes that were made."

- R package trackdown https://claudiozandonella.github.io/trackdown/

    > "Using trackdown, the local .Rmd (or Quarto / .Rnw) file can be uploaded 
    > as a plain-text file to Google Drive."

    > During the collaborative writing/editing of an .Rmd (or Quarto / .Rnw) 
    > document, it is important to employ different workflows for computer code 
    > and narrative text:

        > Code - Collaborative code writing is done most efficiently by 
        > following a traditional Git-based workflow using an online repository 
        > (e.g., GitHub or GitLab).

        > Narrative Text - Collaborative writing of narrative text is done most 
        > efficiently using Google Docs which provides a familiar and simple online 
        > interface that allows multiple users to simultaneously write/edit the 
        > same document.



## Yaml in quarto

- Execution options https://quarto.org/docs/computations/execution-options.html

Option to not show code output

    execute:
      echo: false

# Text formating

## Extra spaces

Add extra spaces

    $~$

Seen at https://stackoverflow.com/a/60876153


# Yaml front matter for styles

## Bibliography

Custom yaml front matter for a markdown document that contains reading notes
with citations. See also the vim.Rmd document on pandoc bibliographies.

    ---
    title: Reading notes
    author: Paul Rougieux
    bibliography: /home/paul/rp/bioeconomy_papers/bibliography/jrc_ispra.bib
    abstract: Reading notes at the JRC
    header-includes: |
        \usepackage{fancyhdr}
        \pagestyle{fancy}
        \fancyhead[CO,CE]{Reading notes}
        \fancyfoot[CO,CE]{JRC}
        \fancyfoot[LE,RO]{\thepage}
    ---
    \tableofcontents{}
    ...file content... this is probably not the way to do it    
    \bibliography{../bibliography/jrc_ispra}
    \bibliographystyle{apalike}

This is probably not the way to do it, the bibliography should be included in the pandoc call? 

Pandoc requires citeproc to process the bibliography, install it with

    sudo apt install pandoc-citeproc

Explicitly call the `--citeproc` argument to generate a list of references
from the citations

    pandoc article.md -o article.pdf --citeproc


## Paper size and margins

Change the margin to 0.5 cm

    ---
    title: "Songs of freedom"
    author: "Bob Marley"
    geometry: margin=0.5cm
    ---

A4 paper

    ---
    title: LETTERA
    papersize: A4
    ---


## Font size

Change the font to 12pt, the max available in the default template.

    ---
    fontsize: 12pt
    ---

To get bigger fonts (up to 20pt) as explained in this [SO
answer](https://stackoverflow.com/a/46055046), you can use the `extsizes`
package by adding `documentclass: extarticle`.

## Page numbers

Adding headers and footers to pandoc suggests using:

    ---
    title: Test
    author: Author Name
    header-includes: |
        \usepackage{fancyhdr}
        \pagestyle{fancy}
        \fancyhead[CO,CE]{This is fancy}
        \fancyfoot[CO,CE]{So is this}
        \fancyfoot[LE,RO]{\thepage}
    abstract: This is a pandoc test . . . 
    ---


## Table of content

Add a table of content at the beginning of the document

    ---
    title: "Work plan"
    geometry: margin=2cm
    toc: yes
    ---

When specified in the yaml front matter only, the table of content appears in pdf output but not in html output. This may be
due to a difference in the latex and html templates? Display the
content of the latex and html templates
    
    pandoc -D latex
    pandoc -D html

However calling the --toc argument does create a table of content in the html file:

    pandoc -s --toc notes.md -o notes.html


## Two column layout and class options

The `openany` option for the book document class, allows opening a chapter on any
page left or right, otherwise the book class forces chapter opening on the left
page only and inserts a blank page.

    ---
    title: "Songs"
    geometry: margin=0.5cm
    documentclass: book
    classoption:
    - twocolumn
    - openany
    ---


# Flavours of markdown

## Common Mark

Stack Overflow 

- The Stack Exchange network used to have a custom flavoured markdown, but they switched to common mark in 2020 [We're switching
  to
  CommonMark](https://meta.stackexchange.com/questions/348746/were-switching-to-commonmark)

    > "Another reason is that this reduces some of the maintenance burdens of our
    > development teams. Instead of maintaining two distinct Markdown renderers, we
    > can now pick something off the shelf and use that instead. With markdig and
    > markdown-it we’ve found two reputable libraries that are beating our own
    > implementations when it comes to performance and functionality. Both are
    > great pieces of software that we're more than happy to use in our product."

- Stack Overflow [markdown editing help](https://stackoverflow.com/editing-help).


## Github flavoured markdown

- [How to add a collapsible
  section](https://gist.github.com/pierrejoubert73/902cc94d79424356a8d20be2b382e1ab)

    <details>
    <summary>Click me</summary>

    ```R
    plot(iris)
    ```
    </details>

> Rules: Have an empty line after the `</summary>` tag or markdown/code blocks
> will not render. Have an empty line after each `</details>` tag if you have
> multiple collapsible sections.


## Gitlab flavoured markdown

https://docs.gitlab.com/ee/user/markdown.html#task-lists

> "To create a task list, follow the format of an ordered or unordered list"

    - [x] Completed task
    - [~] Inapplicable task
    - [ ] Incomplete task
      - [x] Sub-task 1
      - [~] Sub-task 2
      - [ ] Sub-task 3

    1. [x] Completed task
    1. [~] Inapplicable task
    1. [ ] Incomplete task
       1. [x] Sub-task 1
       1. [~] Sub-task 2
       1. [ ] Sub-task 3