DataPreperation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The input for the SRL is the SONAR-1 dataset.\n",
    "\n",
    "We use the CONLL file prepared by Jisk (see https://github.com/Filter-Bubble/stroll), \n",
    "\n",
    "(TODO: also describe how to get from SONAR-1 to this conll files)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "root_dir = '/home/dafne/shared/FilterBubble/SRL/pipeline/'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 0: split CONLLU in train/dev/test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The train file that results from this step can be used to train the stroll tagger. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import conllu\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "conllu_path = os.path.join(root_dir,'gold-conllu')\n",
    "fname_in = os.path.join(conllu_path, 'sonar1_fixed.conll')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "features = [\n",
    "    'ID',\n",
    "    'FORM',\n",
    "    'LEMMA',\n",
    "    'UPOS',\n",
    "    'XPOS',\n",
    "    'FEATS',\n",
    "    'HEAD',\n",
    "    'DEPREL',\n",
    "    'DEPS',\n",
    "    'MISC',\n",
    "    'FRAME',\n",
    "    'ROLE'\n",
    "]\n",
    "features = [f.lower() for f in features]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(fname_in) as fin:\n",
    "    sentences = conllu.parse(fin.read(), fields=features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train size: 23249, dev size: 2907, test size: 2906\n"
     ]
    }
   ],
   "source": [
    "nr_sentences = len(sentences)\n",
    "perm = np.random.permutation(nr_sentences)\n",
    "train_size = int(nr_sentences*0.8)\n",
    "test_size = int(nr_sentences*0.1)\n",
    "dev_size = nr_sentences - train_size - test_size\n",
    "print('Train size: {}, dev size: {}, test size: {}'.format(train_size, dev_size, test_size))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def write_conll(sentences, fn):\n",
    "    with open(fn, 'w') as fout:\n",
    "        for sent in sentences:\n",
    "            fout.write(sent.serialize())\n",
    "            fout.write('\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_set = np.array(sentences)[perm[:train_size]].tolist()\n",
    "dev_set = np.array(sentences)[perm[train_size:train_size+dev_size]].tolist()\n",
    "test_set = np.array(sentences)[perm[train_size+dev_size:]].tolist()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "#write_conll(train_set, os.path.join(conllu_path, 'sonar1_train.conll'))\n",
    "#write_conll(dev_set, os.path.join(conllu_path, 'sonar1_dev.conll'))\n",
    "#write_conll(test_set, os.path.join(conllu_path, 'sonar1_test.conll'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1: from CONLLU with golden annotation to NAF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the [conll2naf](https://github.com/Filter-Bubble/FormatConversions/tree/master/conll2naf) conversion script:\n",
    "```\n",
    "python conll2naf --file_per_sent -o ~/output/path/train ~/input/path/sonar1_train.conll\n",
    "python conll2naf --file_per_sent -o ~/output/path/dev ~/input/path/sonar1_dev.conll\n",
    "python conll2naf --file_per_sent -o ~/output/path/test ~/input/path/sonar1_test.conll\n",
    "```\n",
    "\n",
    "The default is to create one NAF file for the complete conll file, just glueing all sentences together. The option `--file_per_sent` prevents this and writes one NAF file per sentence in the conll file.\n",
    "\n",
    "In theory, we could use these NAF files to train/test the vua-srl tagger but `python nafAlpinoToSRLFeatures.py` gives an error, probably it is missing constituents or uses the wrong tags?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2: extract raw text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "from KafNafParserPy import KafNafParser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "raw_path = os.path.join(root_dir, 'raw')\n",
    "if not os.path.exists(raw_path):\n",
    "    os.mkdir(raw_path)\n",
    "for s in ['train', 'dev', 'test']:\n",
    "    if not os.path.exists(os.path.join(raw_path, s)):\n",
    "        os.mkdir(os.path.join(raw_path, s))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "naf_path = os.path.join(root_dir, 'gold-naf')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "for s in ['train', 'dev', 'test']:\n",
    "    for fname in os.listdir(os.path.join(naf_path, s)):\n",
    "        fpath = os.path.join(naf_path, s, fname)\n",
    "        naf_obj = KafNafParser(fpath)\n",
    "        fname_out = os.path.splitext(fname)[0] + '.txt'\n",
    "        with open(os.path.join(raw_path, s, fname_out), 'w') as fout:\n",
    "            fout.write(naf_obj.get_raw())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3: Run pipeline with StanfordNLP"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this you need [this fork](https://github.com/Filter-Bubble/vu-rm-pip3) of the pipeline which is still under development."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "stanfordnlp_path = os.path.join(root_dir, 'stanfordnlp-naf')\n",
    "if not os.path.exists(stanfordnlp_path):\n",
    "    os.mkdir(stanfordnlp_path)\n",
    "for s in ['train', 'dev', 'test']:\n",
    "    if not os.path.exists(os.path.join(stanfordnlp_path, s)):\n",
    "        os.mkdir(os.path.join(stanfordnlp_path, s))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```bash\n",
    "for fn in $(ls ~/shared/FilterBubble/SRL/pipeline/raw/dev); \n",
    "    do ./scripts/run-pipeline.sh -c cfg/pipeline_stanfordnlp.yml \\\n",
    "        < ~/shared/FilterBubble/SRL/pipeline/raw/dev/$fn  \\\n",
    "        > ~/shared/FilterBubble/SRL/pipeline/stanfordnlp-naf/dev/$fn.naf;\n",
    "done\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And similar for train, test\n",
    "\n",
    "\n",
    "Now we can also compare with files in `gold-naf` to see if StanfordNLP did a good job (do we have an evaluate script for this for the dep parser?). They use the same tags."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4: Run pipeline with Alpino"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alpino_path = os.path.join(root_dir, 'alpino-naf')\n",
    "if not os.path.exists(alpino_path):\n",
    "    os.mkdir(alpino_path)\n",
    "for s in ['train', 'dev', 'test']:\n",
    "    if not os.path.exists(os.path.join(alpino_path, s)):\n",
    "        os.mkdir(os.path.join(alpino_path, s))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```bash\n",
    "for fn in $(ls ~/shared/FilterBubble/SRL/pipeline/raw/dev); \n",
    "    do ./scripts/run-pipeline.sh -c cfg/pipeline_alpino.yml \\\n",
    "        < ~/shared/FilterBubble/SRL/pipeline/raw/dev/$fn  \\\n",
    "        > ~/shared/FilterBubble/SRL/pipeline/alpino-naf/$fn.naf;\n",
    "done\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5: convert files to conll"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So that we can use them to evaluate the stroll parser"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "stanfordnlp_conll_path = os.path.join(root_dir, 'stanfordnlp-conll')\n",
    "if not os.path.exists(stanfordnlp_conll_path):\n",
    "    os.mkdir(stanfordnlp_conll_path)\n",
    "for s in ['train', 'dev', 'test']:\n",
    "    if not os.path.exists(os.path.join(stanfordnlp_conll_path, s)):\n",
    "        os.mkdir(os.path.join(stanfordnlp_conll_path, s))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TODO: extend the naf2conll converter so that it outputs all fields needed by the SRL tagger!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}