05_uebung_allgemein.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_Lizenz: Das folgende Lehrmaterial kann unter einer [CC-BY-SA 4.0](https://creativecommons.org/licenses/by/4.0/legalcode) Lizenz frei verwendet, verbreitet und modifiziert werden._   \n",
    " _Authoren: Jana Lasser (jana.lasser@ds.mpg.de)_  \n",
    " _Das Lehrmaterial wurde im Zuge des Projektes \"Daten Lesen Lernen\", gefördert vom Stifterverband und der Heinz Nixdorf Stiftung erstellt._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Übung 05 - Allgemein: Daten einlesen, bereinigen und mit Histogrammen visualisieren\n",
    "### Praktische Hinweise\n",
    "$\\rightarrow$ Übungen sind dafür gedacht, dass ihr sie euch daheim anseht und versucht, ein paar Aufgaben selbst zu lösen. In den Tutorien könnt ihr euch weiter mit dem Lösungen der Aufgaben beschäftigen und dabei Hilfe von den Tutor*innen bekommen.  \n",
    "\n",
    "$\\rightarrow$ Wenn ein Fehler auftritt:\n",
    "1. Fehlermeldung _lesen_ und _verstehen_\n",
    "2. Versuchen, selbst eine Lösung zu finden ($\\rightarrow$ hier findet das Lernen statt!)\n",
    "3. Das Problem googlen (Stichwort: Stackoverflow) oder den/die Nachbar*in fragen\n",
    "4. Tutor*in fragen\n",
    "\n",
    "$\\rightarrow$ Unter <font color='green'>**HINWEIS**</font> werden Hinweise gegeben, die für die Lösung der Aufgabe hilfreich sind und oftmals auch weiterführende Informationen zur Aufgabe liefern.\n",
    "\n",
    "$\\rightarrow$ Mit **(Optional)** gekennzeichnete Übungsteile sind für besonders schnelle Menschen :-)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name=\"top\"></a>Inhalt\n",
    "---\n",
    "In der nachfolgenden Fallstudie werden wir mehrere Datensätze mit Tweets untersuchen. Dabei vergleichen wir Tweets von \"normalen\" Twitter-Nutzern mit Tweets von Donald Trum und Tweets von russischen Trollen.\n",
    "\n",
    "Als Datenquellen verwenden wir öffentlich zugängliche Datensätze von Tweets:\n",
    "* [\"normale\" Twitter-User](http://help.sentiment140.com/for-students/)\n",
    "* [Donald Trump](https://github.com/mkearney/trumptweets)\n",
    "* [russische Trolle](https://github.com/fivethirtyeight/russian-troll-tweets/)\n",
    "\n",
    "Für die folgenden Übungen gibt es kein Lehrvideo. Deswegen wird es in den Jupyter-Notebooks zu den Übungen mehr Erklärungen und Zwischenschritte geben. Darüber hinaus haben ab dieser Woche auch nicht mehr alle Übungen der verschiedenen Übungsgruppen den exakt gleichen Inhalt, da einige Themenbereiche etwas unterschiedliche Werkzeuge brauchen als andere. Das Konzept ist aber überall das gleiche.  \n",
    "\n",
    "Die aktuelle Übung gliedert sich in drei Teile:\n",
    "\n",
    "* [Daten beschaffen](#daten_beschaffen)\n",
    "* [Daten erkunden](#daten_erkunden)\n",
    "* [Daten bereinigen](#daten_bereinigen)\n",
    "* [Histogramme](#histogramme)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name=\"daten_beschaffen\"></a>1. Daten beschaffen\n",
    "---\n",
    "**A.** Mache dich mit den Datenquellen vertraut: Wer hat die Daten gesammelt und in welchem Kontext publiziert?   \n",
    "**B.** Laden der Daten: Den Datensatz der \"normalen\" User müssen wir erst herunterladen, entpacken und können ihn dann als Pandas DataFrame laden:\n",
    "* gehe auf die Seite http://help.sentiment140.com/for-students/\n",
    "* lade das Archiv ```trainingtestdata``` mit dem Datensatz herunter\n",
    "* entpacke das Archiv\n",
    "* lies die Datei ```training.1600000.processed.noemoticon.csv``` wie folgt ein:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1467810369</th>\n",
       "      <th>Mon Apr 06 22:19:45 PDT 2009</th>\n",
       "      <th>NO_QUERY</th>\n",
       "      <th>_TheSpecialOne_</th>\n",
       "      <th>@switchfoot http://twitpic.com/2y1zl - Awww, that's a bummer.  You shoulda got David Carr of Third Day to do it. ;D</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1467810672</td>\n",
       "      <td>Mon Apr 06 22:19:49 PDT 2009</td>\n",
       "      <td>NO_QUERY</td>\n",
       "      <td>scotthamilton</td>\n",
       "      <td>is upset that he can't update his Facebook by ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>1467810917</td>\n",
       "      <td>Mon Apr 06 22:19:53 PDT 2009</td>\n",
       "      <td>NO_QUERY</td>\n",
       "      <td>mattycus</td>\n",
       "      <td>@Kenichan I dived many times for the ball. Man...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>1467811184</td>\n",
       "      <td>Mon Apr 06 22:19:57 PDT 2009</td>\n",
       "      <td>NO_QUERY</td>\n",
       "      <td>ElleCTF</td>\n",
       "      <td>my whole body feels itchy and like its on fire</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1467811193</td>\n",
       "      <td>Mon Apr 06 22:19:57 PDT 2009</td>\n",
       "      <td>NO_QUERY</td>\n",
       "      <td>Karoli</td>\n",
       "      <td>@nationwideclass no, it's not behaving at all....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1467811372</td>\n",
       "      <td>Mon Apr 06 22:20:00 PDT 2009</td>\n",
       "      <td>NO_QUERY</td>\n",
       "      <td>joy_wolf</td>\n",
       "      <td>@Kwesidei not the whole crew</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   0  1467810369  Mon Apr 06 22:19:45 PDT 2009  NO_QUERY _TheSpecialOne_  \\\n",
       "0  0  1467810672  Mon Apr 06 22:19:49 PDT 2009  NO_QUERY   scotthamilton   \n",
       "1  0  1467810917  Mon Apr 06 22:19:53 PDT 2009  NO_QUERY        mattycus   \n",
       "2  0  1467811184  Mon Apr 06 22:19:57 PDT 2009  NO_QUERY         ElleCTF   \n",
       "3  0  1467811193  Mon Apr 06 22:19:57 PDT 2009  NO_QUERY          Karoli   \n",
       "4  0  1467811372  Mon Apr 06 22:20:00 PDT 2009  NO_QUERY        joy_wolf   \n",
       "\n",
       "  @switchfoot http://twitpic.com/2y1zl - Awww, that's a bummer.  You shoulda got David Carr of Third Day to do it. ;D  \n",
       "0  is upset that he can't update his Facebook by ...                                                                   \n",
       "1  @Kenichan I dived many times for the ball. Man...                                                                   \n",
       "2    my whole body feels itchy and like its on fire                                                                    \n",
       "3  @nationwideclass no, it's not behaving at all....                                                                   \n",
       "4                      @Kwesidei not the whole crew                                                                    "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# zum Speichern von Tabellen\n",
    "import pandas as pd \n",
    "\n",
    "# unter diesem Pfad sollte der Datensatz gespeichert sein\n",
    "pfad_normal = 'daten/training.1600000.processed.noemoticon.csv'\n",
    "tweets_normal_raw = pd.read_csv(pfad_normal, encoding=\"ISO-8859-1\")\n",
    "tweets_normal_raw.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**HINWEIS:** durch das Argument ```encoding``` geben wir explizit an, welchen Standard für die Zeichenkodierung des Textes Pandas verwenden soll, um den Text in dem Datensatz zu interpretieren. Das ist insbesondre wichtig, da Tweets sehr viele Sonderzeichen enthalten können, die nicht im normalen Alphabet abgebildet sind.  \n",
    "Der hier verwendete Standard [ISO 8859-1](https://de.wikipedia.org/wiki/ISO_8859-1) ermöglicht es, den Tweet-Datensatz ohne Fehler einzulesen. Würden wir das encoding nicht explizit angeben, würde Pandas per Default [UTF-8](https://de.wikipedia.org/wiki/UTF-8) verwenden und einen ```UnicodeDecodeError``` produzieren.  \n",
    "\n",
    "Beim Einlesen fällt auf, dass der Datensatz keinen \"Header\" (also eine Kopfzeile mit den Namen der Spalten) enthält. Pandas verwendet automatisch die erste Zeile als Header, was in diesem Fall zu einem Falschen Ergebnis führt, da die erste Zeile bereits Daten enthält. Das können wir umgehen, indem wir beim Laden der Daten einen Header explizit angeben:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sentiment</th>\n",
       "      <th>ID</th>\n",
       "      <th>Date</th>\n",
       "      <th>Query</th>\n",
       "      <th>User</th>\n",
       "      <th>Tweet</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1467810369</td>\n",
       "      <td>Mon Apr 06 22:19:45 PDT 2009</td>\n",
       "      <td>NO_QUERY</td>\n",
       "      <td>_TheSpecialOne_</td>\n",
       "      <td>@switchfoot http://twitpic.com/2y1zl - Awww, t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>1467810672</td>\n",
       "      <td>Mon Apr 06 22:19:49 PDT 2009</td>\n",
       "      <td>NO_QUERY</td>\n",
       "      <td>scotthamilton</td>\n",
       "      <td>is upset that he can't update his Facebook by ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>1467810917</td>\n",
       "      <td>Mon Apr 06 22:19:53 PDT 2009</td>\n",
       "      <td>NO_QUERY</td>\n",
       "      <td>mattycus</td>\n",
       "      <td>@Kenichan I dived many times for the ball. Man...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1467811184</td>\n",
       "      <td>Mon Apr 06 22:19:57 PDT 2009</td>\n",
       "      <td>NO_QUERY</td>\n",
       "      <td>ElleCTF</td>\n",
       "      <td>my whole body feels itchy and like its on fire</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1467811193</td>\n",
       "      <td>Mon Apr 06 22:19:57 PDT 2009</td>\n",
       "      <td>NO_QUERY</td>\n",
       "      <td>Karoli</td>\n",
       "      <td>@nationwideclass no, it's not behaving at all....</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   sentiment          ID                          Date     Query  \\\n",
       "0          0  1467810369  Mon Apr 06 22:19:45 PDT 2009  NO_QUERY   \n",
       "1          0  1467810672  Mon Apr 06 22:19:49 PDT 2009  NO_QUERY   \n",
       "2          0  1467810917  Mon Apr 06 22:19:53 PDT 2009  NO_QUERY   \n",
       "3          0  1467811184  Mon Apr 06 22:19:57 PDT 2009  NO_QUERY   \n",
       "4          0  1467811193  Mon Apr 06 22:19:57 PDT 2009  NO_QUERY   \n",
       "\n",
       "              User                                              Tweet  \n",
       "0  _TheSpecialOne_  @switchfoot http://twitpic.com/2y1zl - Awww, t...  \n",
       "1    scotthamilton  is upset that he can't update his Facebook by ...  \n",
       "2         mattycus  @Kenichan I dived many times for the ball. Man...  \n",
       "3          ElleCTF    my whole body feels itchy and like its on fire   \n",
       "4           Karoli  @nationwideclass no, it's not behaving at all....  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tweets_normal_raw = pd.read_csv(pfad_normal, \\\n",
    "                            names=['sentiment','ID','Date','Query','User','Tweet'],\\\n",
    "                            encoding=\"ISO-8859-1\")\n",
    "tweets_normal_raw.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Die verschiedenen Argumente, um die Funktionsweise der Funktion ```read_csv()``` von Pandas zu modifizieren finden sich in der [Dokumentation](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html) der Funktion. Falls eine Funktion nicht direkt das tut, was du erwartest, ist es oftmals eine gute Idee, als ersten Schritt in der Dokumentation nachzusehen.  \n",
    "\n",
    "Die anderen beiden Datensätze können wir direkt aus dem Internet herunterladen:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/jlasser/virtualenvs/p3.5_venv/lib/python3.5/site-packages/IPython/core/interactiveshell.py:2785: DtypeWarning: Columns (15,30,31,32,36,37,38,42,43,44,45,46,47,50,51,52,56,57,58,59,60,61,62,63,64,65,66,67) have mixed types. Specify dtype option on import or set low_memory=False.\n",
      "  interactivity=interactivity, compiler=compiler, result=result)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>status_id</th>\n",
       "      <th>created_at</th>\n",
       "      <th>user_id</th>\n",
       "      <th>screen_name</th>\n",
       "      <th>text</th>\n",
       "      <th>source</th>\n",
       "      <th>display_text_width</th>\n",
       "      <th>reply_to_status_id</th>\n",
       "      <th>reply_to_user_id</th>\n",
       "      <th>reply_to_screen_name</th>\n",
       "      <th>...</th>\n",
       "      <th>retweet_verified</th>\n",
       "      <th>place_url</th>\n",
       "      <th>place_name</th>\n",
       "      <th>place_full_name</th>\n",
       "      <th>place_type</th>\n",
       "      <th>country</th>\n",
       "      <th>country_code</th>\n",
       "      <th>geo_coords</th>\n",
       "      <th>coords_coords</th>\n",
       "      <th>bbox_coords</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>x1864367186</td>\n",
       "      <td>2009-05-20 22:29:47</td>\n",
       "      <td>x25073877</td>\n",
       "      <td>realDonaldTrump</td>\n",
       "      <td>Read a great interview with Donald Trump that ...</td>\n",
       "      <td>Twitter Web Client</td>\n",
       "      <td>112</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>x9273573134835712</td>\n",
       "      <td>2010-11-29 15:52:46</td>\n",
       "      <td>x25073877</td>\n",
       "      <td>realDonaldTrump</td>\n",
       "      <td>Congratulations to Evan Lysacek for being nomi...</td>\n",
       "      <td>Twitter Web Client</td>\n",
       "      <td>127</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>x29014512646</td>\n",
       "      <td>2010-10-28 18:53:40</td>\n",
       "      <td>x25073877</td>\n",
       "      <td>realDonaldTrump</td>\n",
       "      <td>I was on The View this morning. We talked abou...</td>\n",
       "      <td>Twitter Web Client</td>\n",
       "      <td>139</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>x7483813542232064</td>\n",
       "      <td>2010-11-24 17:20:54</td>\n",
       "      <td>x25073877</td>\n",
       "      <td>realDonaldTrump</td>\n",
       "      <td>Tomorrow night's episode of The Apprentice del...</td>\n",
       "      <td>Twitter Web Client</td>\n",
       "      <td>140</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>x5775731054</td>\n",
       "      <td>2009-11-16 21:06:10</td>\n",
       "      <td>x25073877</td>\n",
       "      <td>realDonaldTrump</td>\n",
       "      <td>Donald Trump Partners with TV1 on New Reality ...</td>\n",
       "      <td>Twitter Web Client</td>\n",
       "      <td>116</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 68 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           status_id           created_at    user_id      screen_name  \\\n",
       "0        x1864367186  2009-05-20 22:29:47  x25073877  realDonaldTrump   \n",
       "1  x9273573134835712  2010-11-29 15:52:46  x25073877  realDonaldTrump   \n",
       "2       x29014512646  2010-10-28 18:53:40  x25073877  realDonaldTrump   \n",
       "3  x7483813542232064  2010-11-24 17:20:54  x25073877  realDonaldTrump   \n",
       "4        x5775731054  2009-11-16 21:06:10  x25073877  realDonaldTrump   \n",
       "\n",
       "                                                text              source  \\\n",
       "0  Read a great interview with Donald Trump that ...  Twitter Web Client   \n",
       "1  Congratulations to Evan Lysacek for being nomi...  Twitter Web Client   \n",
       "2  I was on The View this morning. We talked abou...  Twitter Web Client   \n",
       "3  Tomorrow night's episode of The Apprentice del...  Twitter Web Client   \n",
       "4  Donald Trump Partners with TV1 on New Reality ...  Twitter Web Client   \n",
       "\n",
       "   display_text_width reply_to_status_id reply_to_user_id  \\\n",
       "0                 112                NaN              NaN   \n",
       "1                 127                NaN              NaN   \n",
       "2                 139                NaN              NaN   \n",
       "3                 140                NaN              NaN   \n",
       "4                 116                NaN              NaN   \n",
       "\n",
       "  reply_to_screen_name     ...      retweet_verified  place_url  place_name  \\\n",
       "0                  NaN     ...                   NaN        NaN         NaN   \n",
       "1                  NaN     ...                   NaN        NaN         NaN   \n",
       "2                  NaN     ...                   NaN        NaN         NaN   \n",
       "3                  NaN     ...                   NaN        NaN         NaN   \n",
       "4                  NaN     ...                   NaN        NaN         NaN   \n",
       "\n",
       "   place_full_name place_type country country_code geo_coords coords_coords  \\\n",
       "0              NaN        NaN     NaN          NaN        NaN           NaN   \n",
       "1              NaN        NaN     NaN          NaN        NaN           NaN   \n",
       "2              NaN        NaN     NaN          NaN        NaN           NaN   \n",
       "3              NaN        NaN     NaN          NaN        NaN           NaN   \n",
       "4              NaN        NaN     NaN          NaN        NaN           NaN   \n",
       "\n",
       "  bbox_coords  \n",
       "0         NaN  \n",
       "1         NaN  \n",
       "2         NaN  \n",
       "3         NaN  \n",
       "4         NaN  \n",
       "\n",
       "[5 rows x 68 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Bibliotheken zum Laden von Daten aus dem Internet\n",
    "import requests \n",
    "import io\n",
    "\n",
    "# die URL unter der der Trump-Tweet-Datensatz zu finden ist\n",
    "url_trump = \"https://github.com/mkearney/trumptweets/raw/master/data/trumptweets-1515775693.tweets.csv\"\n",
    "\n",
    "# lies die Information von der in der URL hinterlegten Website aus\n",
    "# diesmal geben wir das encoding schon an dieser Stelle an\n",
    "antwort_trump = requests.get(url_trump).content\n",
    "inhalt_trump = io.StringIO(antwort_trump.decode('ISO-8859-1'))\n",
    "\n",
    "# lade die Daten in ein DataFrame und zeige die ersten paar Zeilen an\n",
    "tweets_trump_raw = pd.read_csv(inhalt_trump)\n",
    "tweets_trump_raw.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>external_author_id</th>\n",
       "      <th>author</th>\n",
       "      <th>content</th>\n",
       "      <th>region</th>\n",
       "      <th>language</th>\n",
       "      <th>publish_date</th>\n",
       "      <th>harvested_date</th>\n",
       "      <th>following</th>\n",
       "      <th>followers</th>\n",
       "      <th>updates</th>\n",
       "      <th>...</th>\n",
       "      <th>account_type</th>\n",
       "      <th>retweet</th>\n",
       "      <th>account_category</th>\n",
       "      <th>new_june_2018</th>\n",
       "      <th>alt_external_id</th>\n",
       "      <th>tweet_id</th>\n",
       "      <th>article_url</th>\n",
       "      <th>tco1_step1</th>\n",
       "      <th>tco2_step1</th>\n",
       "      <th>tco3_step1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>906000000000000000</td>\n",
       "      <td>10_GOP</td>\n",
       "      <td>\"We have a sitting Democrat US Senator on tria...</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>English</td>\n",
       "      <td>10/1/2017 19:58</td>\n",
       "      <td>10/1/2017 19:59</td>\n",
       "      <td>1052</td>\n",
       "      <td>9636</td>\n",
       "      <td>253</td>\n",
       "      <td>...</td>\n",
       "      <td>Right</td>\n",
       "      <td>0</td>\n",
       "      <td>RightTroll</td>\n",
       "      <td>0</td>\n",
       "      <td>905874659358453760</td>\n",
       "      <td>914580356430536707</td>\n",
       "      <td>http://twitter.com/905874659358453760/statuses...</td>\n",
       "      <td>https://twitter.com/10_gop/status/914580356430...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>906000000000000000</td>\n",
       "      <td>10_GOP</td>\n",
       "      <td>Marshawn Lynch arrives to game in anti-Trump s...</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>English</td>\n",
       "      <td>10/1/2017 22:43</td>\n",
       "      <td>10/1/2017 22:43</td>\n",
       "      <td>1054</td>\n",
       "      <td>9637</td>\n",
       "      <td>254</td>\n",
       "      <td>...</td>\n",
       "      <td>Right</td>\n",
       "      <td>0</td>\n",
       "      <td>RightTroll</td>\n",
       "      <td>0</td>\n",
       "      <td>905874659358453760</td>\n",
       "      <td>914621840496189440</td>\n",
       "      <td>http://twitter.com/905874659358453760/statuses...</td>\n",
       "      <td>https://twitter.com/damienwoody/status/9145685...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>906000000000000000</td>\n",
       "      <td>10_GOP</td>\n",
       "      <td>Daughter of fallen Navy Sailor delivers powerf...</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>English</td>\n",
       "      <td>10/1/2017 22:50</td>\n",
       "      <td>10/1/2017 22:51</td>\n",
       "      <td>1054</td>\n",
       "      <td>9637</td>\n",
       "      <td>255</td>\n",
       "      <td>...</td>\n",
       "      <td>Right</td>\n",
       "      <td>1</td>\n",
       "      <td>RightTroll</td>\n",
       "      <td>0</td>\n",
       "      <td>905874659358453760</td>\n",
       "      <td>914623490375979008</td>\n",
       "      <td>http://twitter.com/905874659358453760/statuses...</td>\n",
       "      <td>https://twitter.com/10_gop/status/913231923715...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>906000000000000000</td>\n",
       "      <td>10_GOP</td>\n",
       "      <td>JUST IN: President Trump dedicates Presidents ...</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>English</td>\n",
       "      <td>10/1/2017 23:52</td>\n",
       "      <td>10/1/2017 23:52</td>\n",
       "      <td>1062</td>\n",
       "      <td>9642</td>\n",
       "      <td>256</td>\n",
       "      <td>...</td>\n",
       "      <td>Right</td>\n",
       "      <td>0</td>\n",
       "      <td>RightTroll</td>\n",
       "      <td>0</td>\n",
       "      <td>905874659358453760</td>\n",
       "      <td>914639143690555392</td>\n",
       "      <td>http://twitter.com/905874659358453760/statuses...</td>\n",
       "      <td>https://twitter.com/10_gop/status/914639143690...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>906000000000000000</td>\n",
       "      <td>10_GOP</td>\n",
       "      <td>19,000 RESPECTING our National Anthem! #StandF...</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>English</td>\n",
       "      <td>10/1/2017 2:13</td>\n",
       "      <td>10/1/2017 2:13</td>\n",
       "      <td>1050</td>\n",
       "      <td>9645</td>\n",
       "      <td>246</td>\n",
       "      <td>...</td>\n",
       "      <td>Right</td>\n",
       "      <td>1</td>\n",
       "      <td>RightTroll</td>\n",
       "      <td>0</td>\n",
       "      <td>905874659358453760</td>\n",
       "      <td>914312219952861184</td>\n",
       "      <td>http://twitter.com/905874659358453760/statuses...</td>\n",
       "      <td>https://twitter.com/realDonaldTrump/status/914...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   external_author_id  author  \\\n",
       "0  906000000000000000  10_GOP   \n",
       "1  906000000000000000  10_GOP   \n",
       "2  906000000000000000  10_GOP   \n",
       "3  906000000000000000  10_GOP   \n",
       "4  906000000000000000  10_GOP   \n",
       "\n",
       "                                             content   region language  \\\n",
       "0  \"We have a sitting Democrat US Senator on tria...  Unknown  English   \n",
       "1  Marshawn Lynch arrives to game in anti-Trump s...  Unknown  English   \n",
       "2  Daughter of fallen Navy Sailor delivers powerf...  Unknown  English   \n",
       "3  JUST IN: President Trump dedicates Presidents ...  Unknown  English   \n",
       "4  19,000 RESPECTING our National Anthem! #StandF...  Unknown  English   \n",
       "\n",
       "      publish_date   harvested_date  following  followers  updates    ...      \\\n",
       "0  10/1/2017 19:58  10/1/2017 19:59       1052       9636      253    ...       \n",
       "1  10/1/2017 22:43  10/1/2017 22:43       1054       9637      254    ...       \n",
       "2  10/1/2017 22:50  10/1/2017 22:51       1054       9637      255    ...       \n",
       "3  10/1/2017 23:52  10/1/2017 23:52       1062       9642      256    ...       \n",
       "4   10/1/2017 2:13   10/1/2017 2:13       1050       9645      246    ...       \n",
       "\n",
       "  account_type retweet  account_category new_june_2018     alt_external_id  \\\n",
       "0        Right       0        RightTroll             0  905874659358453760   \n",
       "1        Right       0        RightTroll             0  905874659358453760   \n",
       "2        Right       1        RightTroll             0  905874659358453760   \n",
       "3        Right       0        RightTroll             0  905874659358453760   \n",
       "4        Right       1        RightTroll             0  905874659358453760   \n",
       "\n",
       "             tweet_id                                        article_url  \\\n",
       "0  914580356430536707  http://twitter.com/905874659358453760/statuses...   \n",
       "1  914621840496189440  http://twitter.com/905874659358453760/statuses...   \n",
       "2  914623490375979008  http://twitter.com/905874659358453760/statuses...   \n",
       "3  914639143690555392  http://twitter.com/905874659358453760/statuses...   \n",
       "4  914312219952861184  http://twitter.com/905874659358453760/statuses...   \n",
       "\n",
       "                                          tco1_step1 tco2_step1 tco3_step1  \n",
       "0  https://twitter.com/10_gop/status/914580356430...        NaN        NaN  \n",
       "1  https://twitter.com/damienwoody/status/9145685...        NaN        NaN  \n",
       "2  https://twitter.com/10_gop/status/913231923715...        NaN        NaN  \n",
       "3  https://twitter.com/10_gop/status/914639143690...        NaN        NaN  \n",
       "4  https://twitter.com/realDonaldTrump/status/914...        NaN        NaN  \n",
       "\n",
       "[5 rows x 21 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# die URL unter der der Troll-Tweet-Datensatz zu finden ist\n",
    "url_troll = 'https://github.com/fivethirtyeight/russian-troll-tweets/raw/master/IRAhandle_tweets_1.csv'\n",
    "\n",
    "# lies die Information von der in der URL hinterlegten Website aus\n",
    "# auch hier geben wir das encoding schon an dieser Stelle an\n",
    "antwort_troll = requests.get(url_troll).content\n",
    "inhalt_troll = io.StringIO(antwort_troll.decode('ISO-8859-1'))\n",
    "\n",
    "# lade die Daten in ein DataFrame und zeige die ersten paar Zeilen an\n",
    "tweets_troll_raw = pd.read_csv(inhalt_troll)\n",
    "tweets_troll_raw.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Anfang](#top)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name=\"daten_erkunden\"></a>2. Daten erkunden\n",
    "---\n",
    "Zu Beginn möchten wir ein paar grundlegende Dinge über den Datensatz herausfinden: z.B. die enthaltene Information, die Anzahl der Tweets in jedem der Datensätze und die Anzahl der User.  \n",
    "Die Namen der Spalten eines DataFrame lassen sich über die Variable ```columns``` des DataFrames ansehen:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# eine Liste der im DataFrame \"tweets_troll_raw\" enthaltenen Spalten\n",
    "tweets_troll_raw.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In einem DataFrame können wir auf eine bestimmte Spalte mit ihrem Namen zugreifen:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# zeigt die Spalte mit dem Namen \"author\" im DataFrame \n",
    "# \"tweets_troll_raw\" an\n",
    "tweets_troll_raw.author"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mit der Funktion ```unique()``` lassen sich die einzigartigen Einträge in einer Spalte herausfinden:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['10_GOP', '1488REASONS', '1D_NICOLE_', '1ERIK_LEE', '1LORENAFAVA1',\n",
       "       '2NDHALFONION', '459JISALGE', '4EVER1937', '4EVER_SUSAN',\n",
       "       '4MYSQUAD', '5EMEN5EMENICH', '5L5_5L5', '666STEVEROGERS', '6DRUZ',\n",
       "       '71BILALJAMIL1', 'AAASSSSSHHH', 'AAATEST1234', 'AAAVADAKEDAVRAA',\n",
       "       'AAGN1AA', 'AANTIRACIST', 'AARONALLENALL', 'AARON_M1TCHELL',\n",
       "       'ABALAKOVAYLIA', 'ABBASSHUKRI', 'ABBYLOPTRT', 'ABELLABAB',\n",
       "       'ABIGAILCALLME', 'ABIGAILSSILK', 'ABIISSROSB', 'ABISADMASST',\n",
       "       'ABMERRLINS', 'ABOUTPOLIT', 'ABRAMSREBEKAH', 'ABUNUWASA',\n",
       "       'ABU_UBAIDA2', 'ACAB_ZONE', 'ACAPARELLA', 'ACEJINEV',\n",
       "       'ACHSAHORKHAN', 'ACTIVEMIKR', 'ADALESTRTM', 'ADAMCHAPMANJR',\n",
       "       'ADAMMA_DEFOND', 'ADAM_MATHISSS', 'ADDIE_HOL', 'ADELE_BROCK',\n",
       "       'ADELIINESTRT', 'ADELINE_GS', 'ADELISTRTT', 'ADKEZLATE',\n",
       "       'ADLEESTTT', 'ADLEESWOD', 'ADNNELSTR', 'ADRGREERR', 'ADRIANAMFTTT',\n",
       "       'ADRIASNSS', 'ADRIENNE_GG', 'ADRI_LOW_R', 'AESPARZZA', 'AFEELIUTR',\n",
       "       'AFIFYAMIRA1', 'AFONIA_ZHILCOV', 'AFONINMIXAIL', 'AFRICAVSERDCE',\n",
       "       'AFUCKINSIDE', 'AGAFONME', 'AGAINUKRFAKE', 'AGATA_ALEXEEVA',\n",
       "       'AGITPOLK', 'AGNESGRHM', 'AGNESMLTRT', 'AGNESSTRYT',\n",
       "       'AHMADHUSSEINII', 'AHMADRADJAB', 'AHMADYOUSEF23', 'AHMADYUSUFF03',\n",
       "       'AHMANSOUR91061', 'AHNORDYK', 'AHOGAN_AM', 'AH_VERONIKI',\n",
       "       'AIDEN7757', 'AKOROMYSLOVA', 'ALANISSTRS', 'ALAXXATR', 'ALAXXATRT',\n",
       "       'ALBELITATRTS', 'ALBERTA_HAYNESS', 'ALBERTMORENMORE',\n",
       "       'ALBUQUERQUEON', 'ALDRICH420', 'ALECMOOOOODY', 'ALEESATRS',\n",
       "       'ALEKSANISIMOF', 'ALEKSEY_SOKOL_', 'ALEMICHLESS', 'ALENAZELDINA',\n",
       "       'ALESISTRTT', 'ALESSIAWILLSON', 'ALEXBORONIN', 'ALEXHARRITONOV',\n",
       "       'ALEXISFOXIE', 'ALEXLOVESSHENNA', 'ALEXMMYER', 'ALEXSVLADIMIROV',\n",
       "       'ALEXWARNINGU', 'ALEXXBELYAEV', 'ALEXXDRTRR', 'ALEX_DUGIN',\n",
       "       'ALEX___ANT', 'ALFNEWDAY', 'ALFREDBENBEN', 'ALFREDTHREE',\n",
       "       'ALI27HUSSEIN', 'ALIDANILOVSKAYA', 'ALIISTRR', 'ALINAANTONCHIK',\n",
       "       'ALINALINKI_', 'ALINANOVIKOV022', 'ALINAVIATKOVA', 'ALINA_9_1',\n",
       "       'ALISA_FRATKINA', 'ALISONDAVOL', 'ALIZASSHIELD', 'ALLAHOMORE',\n",
       "       'ALLBERRTOSS', 'ALLFORPETERSONS', 'ALMOSTSAINTGIRL', 'ALODOY',\n",
       "       'ALTMANBELINDA', 'ALTMANCOON', 'ALVA_MC_GHEE', 'ALWAYSHUNGRYBAE',\n",
       "       'ALW_ILL', 'ALYSSABEDOLLA', 'AMALIAHULICK', 'AMALIA_PETTY',\n",
       "       'AMAMCAMCAM', 'AMANDAVGREEN', 'AMBAASTRT', 'AMBBERTHTT',\n",
       "       'AMBERDAVV', 'AMBERLINETR', 'AMBERRPAR', 'AMCONVOICE',\n",
       "       'AMELCREECH', 'AMELIEBALDWIN', 'AMELINAASTR', 'AMERICANALBERT',\n",
       "       'AMIISTRS', 'AMIRAFERRR', 'AMIRAHPOPE', 'AMIRICANWILL',\n",
       "       'AMIYAHSAMUELS', 'AMYGRABMYESSAY', 'AMYMUSICLL', 'AMYYSLEGENTR',\n",
       "       'AMYYSSTS', 'AMYYTHOSSMA', 'ANAAISLEC', 'ANAALESSIS',\n",
       "       'ANASTASIAR777', 'ANASTASSIAZZ', 'ANASTASSITR', 'ANATOLINEMCOV',\n",
       "       'ANATOLRMNFF', 'ANBIVALENTNAYA', 'ANCARICTRS', 'ANDEERLWR',\n",
       "       'ANDEYNESTEROV', 'ANDIOSMARRTRUMP', 'ANDREASEULITZ', 'ANDREBAKIN',\n",
       "       'ANDREBEAULEGER', 'ANDREJFROLOV69', 'ANDREWHEDDD',\n",
       "       'ANDREYYKONDRAT1', 'ANDREY_LISCHINA', 'ANDRISNABRT', 'ANDRIYCCCP',\n",
       "       'ANDROFAG', 'ANDRREETRT', 'ANDRUNINVANYA', 'ANDRYNAUMOV',\n",
       "       'ANDRY_VOLK', 'ANDSBXX30', 'ANDYDREWDANIELS', 'ANDYHASHTAGGER',\n",
       "       'ANDY_PUCHINSKYI', 'ANEELSSTRT', 'ANETTANOVGOROD', 'ANFINRUDSADIE',\n",
       "       'ANFISAMICKEY', 'ANFRAIZER', 'ANGEELISHET', 'ANGEELISTR',\n",
       "       'ANGELABACH991', 'ANGELA_LATTKE', 'ANGELIKAMUSI',\n",
       "       'ANGELIKOROTKOVA', 'ANGELINADARCY_', 'ANGELINALADOVA',\n",
       "       'ANGELISTRR', 'ANGELITHSS', 'ANGELOENOUGH', 'ANGISDOOYL',\n",
       "       'ANGTHEESTR', 'ANG__BOWERS', 'ANIIANTRS', 'ANISACRRUTR',\n",
       "       'ANISILOPE', 'ANISSNOVOTR', 'ANITABR1994', 'ANITALOVEHEART',\n",
       "       'ANJELICAFAMOUS', 'ANKIDINOVAKIRA', 'ANN1EMCCONNELL',\n",
       "       'ANNABEZGUZIKOVA', 'ANNAMARFINA', 'ANNAMINGT', 'ANNAROMAN0',\n",
       "       'ANNA_VYSOKAYA', 'ANNETTETEAL1', 'ANNIEISBEAR', 'ANNIEPOSHES',\n",
       "       'ANNIJONSTR', 'ANNSAM_SAM', 'ANNY_DUBI', 'ANN_HANAH',\n",
       "       'ANN__BOGDANOVA', 'ANORAVERD', 'ANOSSOVV', 'ANTHONYCANLASS',\n",
       "       'ANTHONYCARTR', 'ANTHONYWOODBOY', 'ANTIEHARMON', 'ANTONFROMSPB',\n",
       "       'ANTONHAYHAY', 'ANTONINAPERVAY', 'ANTONOVMENMENCO', 'ANTONRUMSS',\n",
       "       'ANTOSHAKARGIN', 'ANTYSHEVALIDA', 'ANWARGAFUR', 'ANWARJAMIL22',\n",
       "       'ANYA_ISA1282', 'ANZGRI', 'AN_NA__AN', 'AN_N_GASTON',\n",
       "       'APRIILDTRUT', 'APRRILSBRUSS', 'APUHNA', 'ARABMTR', 'ARAGARTRR',\n",
       "       'ARCHIEOLIVERS', 'ARESTOVEVG', 'ARGENTINACATLE1', 'ARIAANST',\n",
       "       'ARIANNA_PER_RY', 'ARIEENSLPE', 'ARIELLE_BROWNIN', 'ARIELSHINING',\n",
       "       'ARINA_RONDAL', 'ARISGRAYS', 'ARKADII_TROFIM', 'ARKADSHIHKIN',\n",
       "       'ARKHIPOVICHIVNA', 'ARKHIPOVTIOVEOF', 'ARMEDFREJYA', 'ARM_2_ALAN',\n",
       "       'ARNOLDPARRK', 'ARONHOLDEN8', 'ARONS_KWENDE', 'ARSENI_ROMAN',\n",
       "       'ARTAUTUMN', 'ARTEMBOGACHENKO', 'ARTEMMPLOTNIKOV', 'ARTEMSSFE',\n",
       "       'ARTEM_POLISHHUK', 'ARTHCLAUDIA', 'ARTKORCHUNOV', 'ARTURBURANOF',\n",
       "       'ARTUR__2014', 'ARTYRTOLSTOV', 'ARYANNAJ0NES', 'ARZHANOVAAA',\n",
       "       'ASDIQARUSSIYA', 'ASHLEECOLLINSS', 'ASHLEEYWES', 'ASHLEYSIMPSN',\n",
       "       'ASHLEY_KRI', 'ASSUNCAOWALLAS', 'ASWWIMMORRIS', 'ASYAARRIVE',\n",
       "       'ATIF_SHAIKH_ME', 'ATLANTA_ONLINE', 'ATULSCARPA', 'AURRLISTR',\n",
       "       'AUSTINLOVESBEER'], dtype=object)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# eine Liste der User-Namen, wobei jeder User genau einmal\n",
    "# vorkommt\n",
    "tweets_troll_raw.author.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Anzahl Troll Tweets: {}'.format(len(tweets_troll_raw)))\n",
    "unique_users = len(tweets_troll_raw.author.unique())\n",
    "print('Anzahl Troll User: {}'.format(unique_users))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Aufgaben\n",
    "**A.** Wie heißen die Spalten in den drei Datensätzen und welche Informationen enthalten sie?  \n",
    "**B.** Sieh dir ein paar Tweets aus den jeweiligen Datensätzen an, indem du auf die Spalte mit dem Tweet-Text und den Index des Tweets zugreifst.  \n",
    "**C.** Wieviele Tweets und User enthält der Datensatz mit \"normalen\" Tweets, wieviele der Datensatz zu den Trump Tweets?  \n",
    "**D.** Was ist die mittlere Anzahl an Tweets je User für den Datensatz mit den normalen Usern und den mit den Trollen?  \n",
    "**E.** **(Optional):** Welcher User im Troll-Datensatz hat die meisten Tweets produziert und wieviele Tweets sind das?  \n",
    "**F.** **(Optional):** der Troll-Datensatz listet auch die Tweet-Sprache. Wieviele Sprachen sind im Datensatz vertreten und wieviele Tweets gibt es je Sprache?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Anfang](#top)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name=\"daten_bereinigen\"></a>3. Daten bereinigen\n",
    "---\n",
    "Offensichtlich gibt es im Datensatz der Trump-Tweets nicht nur Tweets von Trump selbst [@realDonaldTrump](https://twitter.com/realdonaldtrump) sondern auch noch von vier anderen Usern. Im Folgenden wollen wir die Datensätze etwas aufräumen, um sie leichter nutzbar zu machen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# originale Anzahl Tweets\n",
    "print('Anzahl der Tweets vor Bereinigung: {}'.format(len(tweets_trump_raw)))\n",
    "\n",
    "# entferne alle Tweets von Usern, die nicht \"realDonaldTrump\"\n",
    "# sind aus dem Datensatz.\n",
    "mask_realDonald = tweets_trump_raw['screen_name'] == 'realDonaldTrump'\n",
    "tweets_trump_filtered = tweets_trump_raw[mask_realDonald]\n",
    "\n",
    "# wir sehen: es sind gar nicht so viele Tweets verschwunden.\n",
    "print('Anzahl der Tweets nach Bereinigung: {}'.format(len(tweets_trump_filtered)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Es fällt auf, dass die Spalten in den drei Datensätzen unterschiedlich heißen. Die Spalte mit dem User, der einen Tweet erstellt hat, heißt z.B. ```User``` (normale Tweets), ```screen_name``` (Trump Tweets) und ```author``` (Troll Tweets), obwohl sie die gleiche Information - nämlich den Username - enthält. Um die Daten in Zukunft einfacher handhaben zu können, wollen wir sicherstellen, dass alle Spalten in allen Datensätzen gleich heißen. Außerdem löschen wir alle Spalten mit Informationen, die nicht relevant für uns sind. Dafür erstellen wir basierend auf den DataFrame mit den (gefilterten) Rohdaten ein neues DataFrame, das nur die Spalten (mit den richtigen Namen) enthält, die wir brauchen:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# neues, gesäubertes DataFrame\n",
    "tweets_trump = pd.DataFrame({'Date':tweets_trump_filtered['created_at'],\\\n",
    "                             'User':tweets_trump_filtered['screen_name'],\\\n",
    "                             'Tweet':tweets_trump_filtered['text']})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Aufgaben\n",
    "Generell interessieren wir uns für die Spalten, in denen der Username, das Tweet-Datum und der Tweet-Text enthalten sind. Bei den Troll Tweets möchten wir außerdem die Spalten mit Informationen über die Sprache, die Anzahl der Follower und die Anzahl der gefolgten Accounts.  \n",
    "\n",
    "**A.** Bereinige auch die Tweets von den normalen Usern, indem du ein neues DataFrame mit den Spalten ```[Date, User, Text]``` basierend auf den Rohdaten erstellst.  \n",
    "**B.** Bereinige die Troll Twees (wie oben die Trump Tweets), behalte zusätzlich auch die Spalten, die die Informationen über die Tweet-Sprache, die Anzahl der Follower und die Anzahl der gefolgten Accounts enthalten.  \n",
    "**C. (Optional):** Aktuell benutzen wir nur den ersten von 13 Teilen des Troll-Tweet Datensatzes. Wir machen das, da sonst der Datensatz eventuell zu groß werden würde. Wenn du möchtest, kannst du aber auch die anderen 12 Datensätze herunterladen und mit dem ersten zu einem einzigen großen Datensatz zusammenfügen. <font color='green'>**HINWEIS:** Benutze dazu am besten die ```concat()``` Funktion von Pandas ([Dokumentation](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.concat.html)). </font> Wie sind die Tweets in dem gesamten Datensatz strukturiert? Wie wurde der Datensatz in einzelne Teile aufgeteilt?   \n",
    "**D. (Optional):** Das Format der Spalte \"Date\" ist noch in allen drei Datensätzen unterschiedlich. Versuche, alle drei unterschiedlichen Zeit- und Datumsangaben in einen String mit dem Format ```YYYY-MM-DD hh:mm:ss``` (Year-Month-Day hour:minute:second) zu überführen. Noch eleganter ist die Benutzung eines [datetime](https://docs.python.org/2/library/datetime.html) Objektes, um Datum und Uhrzeit zu speichern. \n",
    "\n",
    "Im Folgenden interessieren wir uns für die Länge der einzelnen Tweets. Diese können wir berechnen und in den DataFrames speichern:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "length = []\n",
    "for tweet in tweets_trump['Tweet']:\n",
    "    length.append(len(tweet))\n",
    "\n",
    "tweets_trump['Length'] = length\n",
    "tweets_trump.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**E.** Erstelle eine Funktion, die für ein gegebenes DataFrame die Tweet-Länge berechnet und eine neue Spalte mit der Tweet-Länge zum DataFrame hinzufügt. Wende diese Funktion auf alle drei Tweet-Datensätze an.  \n",
    "**F.** **(Optional)** Erstelle eine zweite Funktion, die die Anzahl der Worte je Tweet berechnet und ebenfalls als zusätzliche Spalte dem DataFrame hinzufügt.  \n",
    "**G.** Speichere alle drei Dataframes als ```.csv``` Datei, indem du die Funktion [to_csv()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_csv.html) des Dataframes benutzt."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Anfang](#top)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name=\"histogramme\"></a>4. Histogramme\n",
    "---\n",
    "Im folgenden Abschnitt werden wir damit beginnen, die gewonnenen Informationen zu visualisieren. Eine sehr einfache Visualisierung ist das Histogramm, eine graphische Darstellung der Häufigkeit eines Merkmals. Zur Illustration erstellen wir eine kleine Visualisierung der Verteilung von Werten in einer Liste:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# eine Liste mit 30 Ganzzahlen, die z.B. das Alter von Personen in einer\n",
    "# Gruppe darstellen können\n",
    "alter = [37, 20, 84,  2, 11, 89, 52, 65, 90, 21, 30, 12, 17, 53, 62, 45, 37,\n",
    "        3,  6, 64, 85,  6, 95, 73, 68, 55, 86, 83, 91,  1]\n",
    "\n",
    "# für Visualisierungen benutzen wir die Blbiothek \"matplotlib.pyplot\"\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Histogramme lassen sich einfach mit der Funktion plt.hist() erstellen\n",
    "plt.hist(alter);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualisierungen sollten immer mit Achsenbeschriftungen versehen werden. Dafür verwenden wir die Funktionen ```plt.xlabel()``` und ```plt.ylabel```. Außerdem können wir der Abbildung mit ```plt.title()``` einen Titel geben. Mit verschiedenen optionalen Argumenten lässt sich das Verhalten von ```plt.hist()``` anpassen:\n",
    "* ```bins``` ändert die Anzahl der Bins\n",
    "* ```range``` ändert den dargestellten Bereich auf der x-Achse\n",
    "* ```rwidth``` ändert die dargestellte bin-Weite \n",
    "* ```color``` ändert die Farbe\n",
    "* ```alpha``` ändert den Alpha-Wert (Durchsichtigkeit) der dargestellten Balken.\n",
    "\n",
    "Plot-Funktionen haben oftmals sehr viele optionale Argumente (auch \"keyword arguments\" genannt). Um herauszufinden, wass die jeweiligen Funktionen alles können, lohnt sich ein Blick in die [Dokumentation](https://matplotlib.org/api/_as_gen/matplotlib.pyplot.hist.html). Außerdem sollte jede Darstellung *immer* mit Achsenbeschriftungen und eventuell einem Titel versehen werden:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAEWCAYAAABliCz2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAE91JREFUeJzt3X20ZXVdx/H3JwZEgUDk+sDDZXxAjazAJrF8SNFMERqXUg0ZUlpTa0mpq5YPyzKtpStbrtRWpk2ogIniAyCySkXFBywhBgmHBxUVBOVBBGTAUga//bH3zeP13jvnDnffO+f+3q+1zrpnP5z9++6zZz5nn9/eZ+9UFZKk1e+nVroASdLyMPAlqREGviQ1wsCXpEYY+JLUCANfkhph4GuiJJlOckeSXfrhTyX5g52grt9Lcv4yt7k2SSVZs5ztanIZ+Bpbko8k+es5xq9PcsOOBE+SJyW5btz5q+obVbVnVd292LYmVf8eVZKXbWe+neLDTzsvA1+LcQrwu0kya/zxwLurattiFuae6dhOAG4BnjdkIzPfmrR6GfhajLOA+wFPmBmR5L7A0cCp/fC9krwhyTeS3JjkbUnu3U97UpLrkrwsyQ3Ae4B/B/bvu2nuSLJ/kp9K8vIkX03ynSTvS7Jvv4x5uzGSvDrJv44M/9i8/R7w3yT5XJKtST6WZL+R+Z+X5Jq+zb9McnWSp871RiS5X5Kzk9ye5ELgobOmPzLJuUluSfKlJL81Mu2oJJf3NXwzyZ/P94Yn2QM4FnghcEiSdfPM99p+u/xj/z7+4xh1nJzkrUn+LcmdwJMXU5smj4GvsVXV/wDv48f3NH8LuLKq/rsf/lvg4cBhwMOAA4BXjcz/QGBf4OB+Oc8AvtV30+xZVd8C/gR4FvCrwP7ArcBblmg1fgf4feD+wG7AnwMkORT4J+C5wIOAvfva5/MW4H/7eZ/fP+iXtQdwLnBa384G4J/6NgDeDvxRVe0FPAr45ALtPBu4A3g/8FG6vf2fUFWvBD4LnNi/jyeOUcfM+/FaYC/g/EXWpglj4GuxTgGOTbJ7P/y8fhx9V89G4CVVdUtVbQVeRxc0M34I/FVVfb//AJnLHwOvrKrrqur7wKv7NpeiC+idVfXlkQ+vw/rxxwIfrqrzq+oHdB9Sc15oqu/6eA7wqqq6s6q20L8HvaOBq6vqnVW1raq+AHwQ+M1++l3AoUl+uqpuraqLF6j3BOD0/pjFacCGJLuOua7bqwPgQ1X1uar6YVX97yJr04Qx8LUoVXU+cDPwrCQPBR5DF0QAU8B9gM1JbktyG/CRfvyMb/fBspCDgTNHlnEFcDfwgCVYhRtGnn8P2LN/vj9w7cyEqvoe8J15ljEFrBmdH7hm5PnBwBEz9ffr8Fy6bzfQfVgcBVyT5NNJfnmuRpIcBDwZeHc/6kPA7sAzF1zD8etg1jqMXZsmkwfNtCNOpduzfwTw0aq6sR9/M/A/wM9W1Tfnee3svea59qKvBZ5fVZ+bPSHJ2gXqupPuA2fGA+ebcQ7X063PTDv3pjteMZdvA9uAg4Ar+3HTI9OvBT5dVb8214ur6r+A9f2e+ol03zQOmmPW4+l2yj48cpx8d7q9/rPmWvSs4QXrmOs1i6hNE8g9fO2IU4GnAn/ISFdGVf0Q+BfgjUnuD5DkgCS/vsCybgTul2TvkXFvA16b5OB+GVNJ1o9R1yXAE9Odq7838IpFrNMHgGOS/EqS3ei6kWafjQRA371yBvDqJPfp+8RH+9bPAR6e5Pgku/aPX0ryM0l2S/LcJHtX1V3A7XTdXHM5AXgNXbfTzOM5wFFJ5vowuhF4yDh1zNXYImvTBDLwtWhVdTXwH8AewNmzJr8MuAr4fJLbgY8zsuc8x7KupDtb52t9t8P+wJv75X4syVbg88ARY9R1LnA6cCmwmS7wxl2ny+gOFr+Xbm//DuAm4PvzvOREuu6gG4CTgXeOLGsr8DS6Yxff6ud5PXCvfpbjgav79+eP6bpZfkySx9J1ybylqm4YeZxN9/4eN0dNb6Y71nFrkn8Yo465bLc2Ta54AxTpJyXZE7gNOKSqvr7S9UhLwT18qZfkmL6LZg/gDcAXgatXtipp6Rj40o+sp+v6+BZwCLCh/AqsVcQuHUlqhHv4ktSIneo8/P3226/Wrl270mVI0sTYvHnzzVU1tf05d7LAX7t2LRdddNFKlyFJEyPJNdufq2OXjiQ1wsCXpEYY+JLUCANfkhph4EtSIwx8SWrEoKdlJrka2Ep384ptVTXn/TglScNbjvPwn1xVNy9DO5KkBdilI0mNGHoPv+huYlHAP1fVptkzJNlId+NrpqenZ08eX+a8OdHS8kJz0srw//eSGHoP//FV9WjgGcALkzxx9gxVtamq1lXVuqmpsS4HIUnaAYMG/syNrKvqJuBM4DFDtidJmt9ggZ9kjyR7zTynu7fmlqHakyQtbMg+/AcAZ6bre1sDnFZVHxmwPUnSAgYL/Kr6GvALQy1fkrQ4npYpSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJaoSBL0mNMPAlqREGviQ1wsCXpEYY+JLUCANfkhph4EtSIwx8SWqEgS9JjTDwJakRBr4kNcLAl6RGGPiS1AgDX5IaYeBLUiMMfElqhIEvSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJasTggZ9klyRfSHLO0G1Jkua3HHv4LwKuWIZ2JEkLGDTwkxwIPBM4ach2JEnbt2bg5b8JeCmw13wzJNkIbASYnp4euBxpwiXDt1E1fBtaEYPt4Sc5GripqjYvNF9VbaqqdVW1bmpqaqhyJKl5Q3bpPA74jSRXA+8FjkzyrwO2J0lawGCBX1WvqKoDq2otsAH4ZFX97lDtSZIW5nn4ktSIoQ/aAlBVnwI+tRxtSZLm5h6+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJaoSBL0mNMPAlqREGviQ1wsCXpEYY+JLUCANfkhph4EtSIwx8SWqEgS9JjTDwJakRBr4kNcLAl6RGGPiS1AgDX5IaYeBLUiMMfElqhIEvSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJaoSBL0mNMPAlqRGDBX6S3ZNcmOS/k1yW5DVDtSVJ2r41Ay77+8CRVXVHkl2B85P8e1V9fsA2JUnzGCzwq6qAO/rBXftHDdWeJGlhg/bhJ9klySXATcC5VXXBkO1JkuY3ZJcOVXU3cFiSfYAzkzyqqraMzpNkI7ARYHp6eshyhpMM30bthF+OXO/h7Izr3apVtL2X5SydqroNOA94+hzTNlXVuqpaNzU1tRzlSFKT5t3DT/JF5u5zD10X/c8vtOAkU8BdVXVbknsDvwa8/p4UK0nacQt16Rx9D5f9IOCUJLvQfZN4X1Wdcw+XKUnaQfMGflVdc08WXFWXAoffk2VIkpbOdvvwkzw7yVeSfDfJ7Um2Jrl9OYqTJC2dcc7S+TvgmKq6YuhiJEnDGecsnRsNe0mafAudpfPs/ulFSU4HzqK7XAIAVXXGwLVJkpbQQl06x4w8/x7wtJHhAgx8SZogC52l8/vLWYgkaVjbPWibZHfgBcDPArvPjK+q5w9YlyRpiY1z0PZdwAOBXwc+DRwIbB2yKEnS0hsn8B9WVX8J3FlVpwDPBI4YtixJ0lIbJ/Dv6v/eluRRwN7A/YcrSZI0hHF+eLUpyX2BvwDOBvYEXjVoVZKkJbfdwK+qk/qnnwEeMmw5kqShjHMtnbuT/G3yo7sAJLl42LIkSUttnD78y/r5PpZk337cMtwCRpK0lMYJ/G1V9VLgJOCzSX4Rb0YuSRNnnIO2Aaiq05NcBpwGTOjNZyWpXeME/h/MPKmqLUmeAKwfriRJ0hDGOUtnc5JfAdaOM78kaec0zrV03gU8FLgEuLsfXcCpA9YlSVpi4+yxrwMOrSoP1ErSBBvnLJ0tdBdPkyRNsHH28PcDLk9yIT+641VVlQduJWmCjBP4rx55HuAJwIZBqpEkDWa7XTpV9WngduBo4GTgSOBtw5YlSVpqC93E/OHAcf3jZuB0IFX15GWqTZK0hBbq0rkS+CxwdFVdBZDkJctSlSRpyS3UpfNs4HrgvCT/kuQpeNE0SZpY8wZ+VZ1VVRuARwLnAS8G7p/krUmetlwFSpKWxjgHbe+sqtOq6hi6G5h/AXjZ4JVJkpbUOD+8+n9VdWtVbaqqpwxVkCRpGIsKfEnS5DLwJakRBr4kNWKwwE9yUJLzklye5LIkLxqqLUnS9g15Q5NtwJ9V1cVJ9gI2Jzm3qi4fsE1J0jwG28Ovquur6uL++VbgCuCAodqTJC1sWW5ZmGQtcDhwwRzTNgIbAaanvTf6omUZfvzsvW+kVWHwg7ZJ9gQ+CLy4qm6fPb0/r39dVa2bmpoauhxJataggZ9kV7qwf3dVnTFkW5KkhQ15lk6AtwNXVNXfD9WOJGk8Q+7hPw44HjgyySX946gB25MkLWCwg7ZVdT5eTlmSdhr+0laSGmHgS1IjDHxJaoSBL0mNMPAlqREGviQ1wsCXpEYY+JLUCANfkhph4EtSIwx8SWqEgS9JjTDwJakRBr4kNcLAl6RGGPiS1AgDX5IaYeBLUiMMfElqhIEvSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJaoSBL0mNMPAlqREGviQ1wsCXpEYY+JLUiMECP8k7ktyUZMtQbUiSxjfkHv7JwNMHXL4kaREGC/yq+gxwy1DLlyQtzpqVLiDJRmAjwPT09ApXo4mRDN9G1fBtTBLf84m34gdtq2pTVa2rqnVTU1MrXY4krVorHviSpOVh4EtSI4Y8LfM9wH8Cj0hyXZIXDNWWJGn7BjtoW1XHDbVsSdLi2aUjSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJaoSBL0mNMPAlqREGviQ1wsCXpEYY+JLUCANfkhph4EtSIwx8SWqEgS9JjTDwJakRBr4kNcLAl6RGGPiS1AgDX5IaYeBLUiMMfElqhIEvSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJaoSBL0mNGDTwkzw9yZeSXJXk5UO2JUla2GCBn2QX4C3AM4BDgeOSHDpUe5KkhQ25h/8Y4Kqq+lpV/QB4L7B+wPYkSQtYM+CyDwCuHRm+Djhi9kxJNgIb+8E7knxpEW3sB9y8wxUuVrJsTW2nbdd7+dteXq43uN7jOnjcGYcM/LFU1SZg0468NslFVbVuiUva6bnebXG92zLkeg/ZpfNN4KCR4QP7cZKkFTBk4P8XcEiSByfZDdgAnD1ge5KkBQzWpVNV25KcCHwU2AV4R1VdtsTN7FBX0CrgerfF9W7LYOudqhpq2ZKknYi/tJWkRhj4ktSIiQz8Vi7ZkOSgJOcluTzJZUle1I/fN8m5Sb7S/73vStc6hCS7JPlCknP64QcnuaDf7qf3JwOsKkn2SfKBJFcmuSLJLze0vV/S/zvfkuQ9SXZfjds8yTuS3JRky8i4ObdxOv/Qr/+lSR59T9qeuMBv7JIN24A/q6pDgccCL+zX9eXAJ6rqEOAT/fBq9CLgipHh1wNvrKqHAbcCL1iRqob1ZuAjVfVI4Bfo1n/Vb+8kBwB/CqyrqkfRneixgdW5zU8Gnj5r3Hzb+BnAIf1jI/DWe9LwxAU+DV2yoaqur6qL++db6f7zH0C3vqf0s50CPGtlKhxOkgOBZwIn9cMBjgQ+0M+y6tY7yd7AE4G3A1TVD6rqNhrY3r01wL2TrAHuA1zPKtzmVfUZ4JZZo+fbxuuBU6vzeWCfJA/a0bYnMfDnumTDAStUy7JJshY4HLgAeEBVXd9PugF4wAqVNaQ3AS8FftgP3w+4raq29cOrcbs/GPg28M6+K+ukJHvQwPauqm8CbwC+QRf03wU2s/q3+Yz5tvGS5t0kBn5zkuwJfBB4cVXdPjqtuvNqV9W5tUmOBm6qqs0rXcsyWwM8GnhrVR0O3Mms7pvVuL0B+j7r9XQfevsDe/CT3R5NGHIbT2LgN3XJhiS70oX9u6vqjH70jTNf6/q/N61UfQN5HPAbSa6m67I7kq5ve5/+6z6szu1+HXBdVV3QD3+A7gNgtW9vgKcCX6+qb1fVXcAZdP8OVvs2nzHfNl7SvJvEwG/mkg19v/XbgSuq6u9HJp0NnNA/PwH40HLXNqSqekVVHVhVa+m27yer6rnAecCx/Wyrcb1vAK5N8oh+1FOAy1nl27v3DeCxSe7T/7ufWfdVvc1HzLeNzwae15+t81jguyNdP4tXVRP3AI4Cvgx8FXjlStcz4Ho+nu6r3aXAJf3jKLr+7E8AXwE+Duy70rUO+B48CTinf/4Q4ELgKuD9wL1Wur4B1vcw4KJ+m58F3LeV7Q28BrgS2AK8C7jXatzmwHvojlPcRfet7gXzbWMgdGclfhX4It1ZTDvctpdWkKRGTGKXjiRpBxj4ktQIA1+SGmHgS1IjDHxJaoSBr6YleVaSSvLIfnjtzFUMkxyW5KiVrVBaOga+WncccH7/d7bD6H73MLaRX4VKOx0DX83qr1H0eLofvmyYNW034K+B305ySZLfTrJHfy3zC/uLm63v5/29JGcn+STdj2eknZJ7I2rZerprz385yXeS/CLwHeguTZzkVXS/bDwRIMnr6C7z8Pwk+wAXJvl4v6xHAz9fVbMveyvtNNzDV8uOo7s4G/3fubp1Rj0NeHmSS4BPAbsD0/20cw177ezcw1eTkuxLdxXOn0tSdHdYKrrrlsz7MuA5VfWlWcs6gu5SxtJOzT18tepY4F1VdXBVra2qg4Cv8+OXot0K7DUy/FHgT/qrOZLk8GWrVloCBr5adRxw5qxxHwReMTJ8HnDozEFb4G+AXYFLk1zWD0sTw6tlSlIj3MOXpEYY+JLUCANfkhph4EtSIwx8SWqEgS9JjTDwJakR/wdhX0+/osaaZwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(alter, color='red', rwidth=0.8, range=[0,100], bins=10, label='alter')\n",
    "plt.xlabel('Alter')\n",
    "plt.ylabel('Anzahl')\n",
    "plt.title('Verteilung des Alters');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Aufgaben\n",
    "**A.** Erstelle ein Histogramm der Follower der Troll-Accounts.  \n",
    "**B.** Mache dich mit den unterschiedlichen Argumenten der Funktion ```plt.hist()``` vertraut, indem du verschiedene Werte und Kombinationen für die Daten ausprobierst. Finde Werte, die die Daten deiner Meinung nach gut und verständlich darstellen. Worauf kommt es dabei an? Versieh die Darstellung mit passenden Beschriftungen.  \n",
    "**C.** Insbesondere das Argument ```bins``` verändert die Darstellung des Histogrammes sehr stark. Was kann schiefgehen, wenn ```bins``` zu niedrig oder zu hoch gewählt wird? Was ist eine gute Wahl für einen Datensatz?  \n",
    "**D.** Visualisiere die Anzahl der gefolgten Accounts (```'Following'```) im selben Histogram. Was für Aussagen über die Troll-Accounts kannst du aus dieser Visualisierung ableiten? <font color='green'>**HINWEIS:** Du kannst einfach in der selben Code-Zelle zwei mal hintereinander die Funktion ```plt.hist()``` (natürlich mit unterschiedlichen Daten) aufrufen, um die Daten übereinandergelegt darzustellen. Du kannst mit der Funktion ```plt.legend()``` eine Legende zur Abbildung hinzufügen. Damit das funktioniert, musst du den einzelen plot-Befehlen vorher ein entsprechendes ```label``` Argument mitgeben.</font> Was fällt dir auf? Worauf musst du achten, damit die beiden Histogramme vergleichbar sind?  \n",
    "**E. (Optional)** Was hat es mit den beiden keyword arguments ```density``` und ```cumulative``` auf sich? Wie verändert sich das Histogram und wie kann die Darstellung dann interpretiert werden?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Anfang](#top)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}