Top_Ranked_Gutenberg_Ebooks_Download.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/SatriaImawan12/Top-Ranked-Gutenberg-Ebooks-Download/blob/main/Top_Ranked_Gutenberg_Ebooks_Download.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BbxOTry0XYwR"
      },
      "source": [
        "# Top Gutenberg Ebooks (yesterday's ranking) download\n",
        "\n",
        "### What is Project Gutenberg? -\n",
        "Project Gutenberg is a volunteer effort to digitize and archive cultural works, to \"encourage the creation and distribution of eBooks\". It was founded in 1971 by American writer Michael S. Hart and is the **oldest digital library.** This longest-established ebook project releases books that entered the public domain, and can be freely read or downloaded in various electronic formats.\n",
        "\n",
        "* **This starter code scrapes the url of the Project Gutenberg's Top 100 ebooks (yesterday's ranking) for identifying the ebook links.**\n",
        "* **It uses BeautifulSoup4 for parsing the HTML and regular expression code for identifying the Top 100 ebook file numbers.**\n",
        "* **It includes a function to take an usser input on how many books to download and then crawls the server to download them in a dictionary object.**\n",
        "* **Finally, it also includes a function to save the downloaded Ebooks as text files in a local directory.**"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KnWW6iyMXYwV"
      },
      "outputs": [],
      "source": [
        "import urllib.request, urllib.parse, urllib.error\n",
        "from bs4 import BeautifulSoup\n",
        "import ssl\n",
        "import re"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6LWdfR0MXYwW"
      },
      "source": [
        "#### Ignore SSL certificate errors"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "v2_V4OxJXYwW"
      },
      "outputs": [],
      "source": [
        "# Ignore SSL certificate errors\n",
        "ctx = ssl.create_default_context()\n",
        "ctx.check_hostname = False\n",
        "ctx.verify_mode = ssl.CERT_NONE"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ry9Z8fR-XYwX"
      },
      "source": [
        "#### Read the HTML from the URL and pass on to BeautifulSoup"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BGbteOi2XYwX",
        "outputId": "04bb6605-8953-489b-e8cc-57c15ad0e7eb"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Opening the file connection to https://www.gutenberg.org/browse/scores/top\n",
            "Connection established and HTML parsed...\n"
          ]
        }
      ],
      "source": [
        "# Read the HTML from the URL and pass on to BeautifulSoup\n",
        "top100url = 'https://www.gutenberg.org/browse/scores/top'\n",
        "url = top100url\n",
        "print(f\"Opening the file connection to {url}\")\n",
        "html = urllib.request.urlopen(url, context=ctx).read()\n",
        "soup = BeautifulSoup(html, 'html.parser')\n",
        "print(\"Connection established and HTML parsed...\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L7u7xf0SXYwY"
      },
      "source": [
        "#### Find all the _'href'_ tags and store them in the list of links"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RvxiVnMYXYwY"
      },
      "outputs": [],
      "source": [
        "# Empty list to hold all the http links in the HTML page\n",
        "lst_links=[]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "69wfK9h-XYwZ"
      },
      "outputs": [],
      "source": [
        "# Find all the href tags and store them in the list of links\n",
        "for link in soup.find_all('a'):\n",
        "    #print(link.get('href'))\n",
        "    lst_links.append(link.get('href'))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ovvAHdz4XYwZ"
      },
      "source": [
        "#### Use regular expression to find the numeric digits in these links. These are the file number for the Top 100 books."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OWQhvOB6XYwZ"
      },
      "outputs": [],
      "source": [
        "# Use regular expression to find the numeric digits in these links. These are the file number for the Top 100 books.\n",
        "# Initialize empty list to hold the file numbers\n",
        "booknum=[]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CKDgPu_hXYwa",
        "outputId": "885f8b6d-17b7-43e4-bcd0-558df0ab3125"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "The file numbers for the top 100 ebooks on Gutenberg are shown below\n",
            "----------------------------------------------------------------------\n",
            "[1, 1, 7, 7, 30, 30, 1513, 2701, 1342, 145, 2641, 84, 100, 37106, 67979, 11, 16389, 6761, 394, 2160, 1259, 5197, 6593, 4085, 2554, 345, 5200, 2542, 28054, 174, 50150, 33283, 98, 2600, 76, 64317, 25344, 1661, 16119, 6130, 1998, 4300, 1400, 844, 2000, 16328, 30254, 1184, 74239, 2591, 27827, 1260, 74243, 74240, 1232, 4363, 1952, 8492, 1080, 24022, 5740, 768, 1727, 74244, 45, 74, 43, 219, 74245, 244, 67098, 31552, 2680, 36034, 16, 1497, 12, 2650, 398, 23, 8800, 3207, 408, 31284, 41, 996, 514, 120, 135, 5827, 36]\n"
          ]
        }
      ],
      "source": [
        "# Number 19 to 119 in the original list of links have the Top 100 books' number.\n",
        "for i in range(19,119):\n",
        "\tlink=lst_links[i]\n",
        "\tlink=link.strip()\n",
        "\t# Regular expression to find the numeric digits in the link (href) string\n",
        "\tn=re.findall('[0-9]+',link)\n",
        "\tif len(n)==1:\n",
        "\t\t# Append the filenumber casted as integer\n",
        "\t\tbooknum.append(int(n[0]))\n",
        "\n",
        "print (\"\\nThe file numbers for the top 100 ebooks on Gutenberg are shown below\\n\"+\"-\"*70)\n",
        "print(booknum)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G6Rkm-0eXYwa"
      },
      "source": [
        "#### Search in the extracted text (using Regular Expression) from the soup object to find the names of top 100 Ebooks (Yesterday's rank)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UMrvvLIYXYwa"
      },
      "outputs": [],
      "source": [
        "start_idx=soup.text.splitlines().index('Top 100 EBooks yesterday')\n",
        "lst_titles_temp=[] # Empty list of Ebook names\n",
        "for i in range(100):\n",
        "    lst_titles_temp.append(soup.text.splitlines()[start_idx+2+i])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OYa1p-R3XYwa",
        "outputId": "90ed2ee3-ef5e-40fa-e7fc-1f018bd6386f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Romeo and Juliet by William Shakespeare \n",
            "Moby Dick\n",
            "Pride and Prejudice by Jane Austen \n",
            "Middlemarch by George Eliot \n",
            "A Room with a View by E\n",
            "Frankenstein\n",
            "The Complete Works of William Shakespeare by William Shakespeare \n",
            "Little Women\n",
            "The Blue Castle\n",
            "Alice\n",
            "The Enchanted April by Elizabeth Von Arnim \n",
            "The Adventures of Ferdinand Count Fathom \n",
            "Cranford by Elizabeth Cleghorn Gaskell \n",
            "The Expedition of Humphry Clinker by T\n",
            "Twenty years after by Alexandre Dumas and Auguste Maquet \n",
            "My Life \n",
            "History of Tom Jones\n",
            "The Adventures of Roderick Random by T\n",
            "Crime and Punishment by Fyodor Dostoyevsky \n",
            "Dracula by Bram Stoker \n",
            "Metamorphosis by Franz Kafka \n",
            "A Doll\n",
            "The Brothers Karamazov by Fyodor Dostoyevsky \n",
            "The Picture of Dorian Gray by Oscar Wilde \n",
            "The Devil is an Ass by Ben Jonson \n",
            "Calculus Made Easy by Silvanus P\n",
            "A Tale of Two Cities by Charles Dickens \n",
            "War and Peace by graf Leo Tolstoy \n",
            "Adventures of Huckleberry Finn by Mark Twain \n",
            "The Great Gatsby by F\n",
            "The Scarlet Letter by Nathaniel Hawthorne \n",
            "The Adventures of Sherlock Holmes by Arthur Conan Doyle \n",
            "Doctrina Christiana \n",
            "The Iliad by Homer \n",
            "Thus Spake Zarathustra\n",
            "Ulysses by James Joyce \n",
            "Great Expectations by Charles Dickens \n",
            "The Importance of Being Earnest\n",
            "Don Quijote by Miguel de Cervantes Saavedra \n",
            "Beowulf\n",
            "The Romance of Lust\n",
            "The Count of Monte Cristo by Alexandre Dumas and Auguste Maquet \n",
            "Dreams of an astronomer by Camille Flammarion \n",
            "Grimms\n",
            "The Kama Sutra of Vatsyayana by Vatsyayana \n",
            "Jane Eyre\n",
            "Many furrows by A\n",
            "The works of the highly experienced and famous chymist\n",
            "The Prince by Niccol\n",
            "Beyond Good and Evil by Friedrich Wilhelm Nietzsche \n",
            "The Yellow Wallpaper by Charlotte Perkins Gilman \n",
            "The King in Yellow by Robert W\n",
            "A Modest Proposal by Jonathan Swift \n",
            "A Christmas Carol by Charles Dickens \n",
            "Tractatus Logico\n",
            "Wuthering Heights by Emily Bront\n",
            "The Odyssey by Homer \n",
            "The pageant of Parliament\n",
            "Anne of Green Gables by L\n",
            "The Adventures of Tom Sawyer\n",
            "The Strange Case of Dr\n",
            "Heart of Darkness by Joseph Conrad \n",
            "A Hausa botanical vocabulary by J\n",
            "A Study in Scarlet by Arthur Conan Doyle \n",
            "Winnie\n",
            "Novo dicion\n",
            "Meditations by Emperor of Rome Marcus Aurelius \n",
            "White Nights and Other Stories by Fyodor Dostoyevsky \n",
            "Peter Pan by J\n",
            "The Republic by Plato \n",
            "Through the Looking\n",
            "Du c\n",
            "The First Book of Adam and Eve by Rutherford Hayes Platt \n",
            "Narrative of the Life of Frederick Douglass\n",
            "The divine comedy by Dante Alighieri \n",
            "Leviathan by Thomas Hobbes \n",
            "The Souls of Black Folk by W\n",
            "Josefine Mutzenbacher by Felix Salten \n",
            "The Legend of Sleepy Hollow by Washington Irving \n",
            "Don Quixote by Miguel de Cervantes Saavedra \n",
            "Little Women by Louisa May Alcott \n",
            "Treasure Island by Robert Louis Stevenson \n",
            "Les Mis\n",
            "The Problems of Philosophy by Bertrand Russell \n",
            "The War of the Worlds by H\n",
            "A Christmas Carol in Prose\n",
            "Emma by Jane Austen \n",
            "Second Treatise of Government by John Locke \n",
            "British birds by Frank E\n",
            "Half a Man\n",
            "The art of money getting \n",
            "Notes from the Underground by Fyodor Dostoyevsky \n",
            "Arabi and his household by Lady Gregory \n",
            "The Wonderful Wizard of Oz by L\n",
            "The Prophet by Kahlil Gibran \n",
            "The Confessions of St\n",
            "Dubliners by James Joyce \n",
            "Le Morte d\n",
            "The Elements of Style by William Strunk \n",
            "Anna Karenina by graf Leo Tolstoy \n"
          ]
        }
      ],
      "source": [
        "# Use regular expression to extract only text from the name strings and append to an empty list\n",
        "lst_titles=[]\n",
        "for i in range(100):\n",
        "    id1,id2=re.match('^[a-zA-Z ]*',lst_titles_temp[i]).span()\n",
        "    lst_titles.append(lst_titles_temp[i][id1:id2])\n",
        "for l in lst_titles:\n",
        "    print(l)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S52B2LGAXYwa"
      },
      "source": [
        "### Define a function that takes an user input of how many top books to download and crawls the server to download"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "WJ6Vau1yXYwa"
      },
      "outputs": [],
      "source": [
        "def download_top_books(num_download=10,verbosity=0):\n",
        "    \"\"\"\n",
        "    Function: Download top N books from Gutenberg.org where N is specified by user\n",
        "    Verbosity: If verbosity is turned on (set to 1) then prints the downloading status for every book\n",
        "    Returns: Returns a dictionary where keys are the names of the books and values are the raw text.\n",
        "    Exception Handling: If a book is not found on the server (due to broken link or whatever reason), inserts \"NOT FOUND\" as the text.\n",
        "    \"\"\"\n",
        "    topEBooks = {}\n",
        "\n",
        "    if num_download<=0:\n",
        "        print(\"I guess no download is necessary\")\n",
        "        return topEBooks\n",
        "\n",
        "    if num_download>100:\n",
        "        print(\"You asked for more than 100 downloads.\\nUnfortunately, Gutenberg ranks only top 100 books.\\nProceeding to download top 100 books.\")\n",
        "        num_download=100\n",
        "\n",
        "    # Base URL for files repository\n",
        "    baseurl= 'http://www.gutenberg.org/files/'\n",
        "\n",
        "    if verbosity==1:\n",
        "        count_done=0\n",
        "        for i in range(num_download):\n",
        "            print (\"Working on book:\", lst_titles[i])\n",
        "\n",
        "            # Create the proper download link (url) from the book id\n",
        "            # You have to examine the Gutenberg.org file structure carefully to come up with the proper url\n",
        "            bookid=booknum[i]\n",
        "            bookurl= baseurl+str(bookid)+'/'+str(bookid)+'-0.txt'\n",
        "            # Create a file handler object\n",
        "            try:\n",
        "                fhand = urllib.request.urlopen(bookurl)\n",
        "                txt_dump = ''\n",
        "                # Iterate over the lines in the file handler object and dump the data into the text string\n",
        "                for line in fhand:\n",
        "                    # Use decode method to convert the UTF-8 to Unicode string\n",
        "                    txt_dump+=line.decode()\n",
        "                # Add downloaded text to the dictionary with keys matching the list of book titles.\n",
        "                # This puts the raw text as the value of the key of the dictionary bearing the name of the Ebook\n",
        "                topEBooks[lst_titles[i]]=txt_dump\n",
        "                count_done+=1\n",
        "                print (f\"Finished downloading {round(100*count_done/num_download,2)}%\")\n",
        "            except urllib.error.URLError as e:\n",
        "                topEBooks[lst_titles[i]]=\"NOT FOUND\"\n",
        "                count_done+=1\n",
        "                print(f\"**ERROR: {lst_titles[i]} {e.reason}**\")\n",
        "    else:\n",
        "        count_done=0\n",
        "        from tqdm import tqdm, tqdm_notebook\n",
        "        for i in tqdm(range(num_download),desc='Download % completed',dynamic_ncols=True):\n",
        "            # Create the proper download link (url) from the book id\n",
        "            # You have to examine the Gutenberg.org file structure carefully to come up with the proper url\n",
        "            bookid=booknum[i]\n",
        "            bookurl= baseurl+str(bookid)+'/'+str(bookid)+'-0.txt'\n",
        "            # Create a file handler object\n",
        "            try:\n",
        "                fhand = urllib.request.urlopen(bookurl)\n",
        "                txt_dump = ''\n",
        "                # Iterate over the lines in the file handler object and dump the data into the text string\n",
        "                for line in fhand:\n",
        "                    # Use decode method to convert the UTF-8 to Unicode string\n",
        "                    txt_dump+=line.decode()\n",
        "                # Add downloaded text to the dictionary with keys matching the list of book titles.\n",
        "                # This puts the raw text as the value of the key of the dictionary bearing the name of the Ebook\n",
        "                topEBooks[lst_titles[i]]=txt_dump\n",
        "                count_done+=1\n",
        "            except urllib.error.URLError as e:\n",
        "                topEBooks[lst_titles[i]]=\"NOT FOUND\"\n",
        "                count_done+=1\n",
        "                print(f\"**ERROR: {lst_titles[i]} {e.reason}**\")\n",
        "\n",
        "    print (\"-\"*40+\"\\nFinished downloading all books!\\n\"+\"-\"*40)\n",
        "\n",
        "    return (topEBooks)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c885NQk0XYwb"
      },
      "source": [
        "#### Test the function with verbosity=0 (default)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mJdDdTENXYwb",
        "outputId": "57fabea5-4b93-4f2e-f420-023b2d39cfc1"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Download % completed: 100%|██████████| 1/1 [00:00<00:00,  1.07it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "----------------------------------------\n",
            "Finished downloading all books!\n",
            "----------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        }
      ],
      "source": [
        "dict_books=download_top_books(1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gxR-4v1dXYwb"
      },
      "source": [
        "#### Show the final dictionary and an example of the downloaded text"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NAVLVcoIXYwb",
        "outputId": "fd1e3cd4-2830-4c1f-fab9-18c5bf5bde88"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\r\n",
            "\r\n",
            "===========================================================\r\n",
            "\r\n",
            "     NOTE:  This file combines the first two Project Gutenberg\r\n",
            "     files, both of which were given the filenumber #1. There are\r\n",
            "     several duplicate files here. There were many updates over\r\n",
            "     the years.  All of the original files are included in the\r\n",
            "     \"old\" subdirectory which may be accessed under the \"More\r\n",
            "     Files\" listing in the PG Catalog of this file. No changes\r\n",
            "     have been made in these original etexts.\r\n",
            "\r\n",
            "===========================================================\r\n",
            "\r\n",
            "\r\n",
            "**Welcome To The World of Free Plain Vanilla Electronic Texts**\r\n",
            "\r\n",
            "**Etexts Readable By Both Humans and By Computers, Since 1971**\r\n",
            "\r\n",
            "*These Etexts Prepared By Hundreds of Volunteers and Donations*\r\n",
            "\r\n",
            "Below you will find the first nine Project Gutenberg Etexts, in\r\n",
            "one file, with one header for the entire file.  This is to keep\r\n",
            "the overhead down, and in response to requests from Gopher site\r\n",
            "keeper to eliminate as much of the headers as possible.\r\n",
            "\r\n",
            "However, for legal and financial reasons, we must request these\r\n",
            "headers be left at the beginning of each file that is posted in\r\n",
            "any general user areas, as Project Gutenberg is run mostly by a\r\n",
            "donation from people like you.\r\n",
            "\r\n",
            "If you see our books posted ANYWHERE without these headers, you\r\n",
            "are requested to send them a note requesting they re-attach the\r\n",
            "header, otherwise they have no legal protection and we have the\r\n",
            "loss of the donations we hope will keep Project Guten\n"
          ]
        }
      ],
      "source": [
        "print(dict_books[lst_titles[0]][:1500])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4M_SySGPXYwb"
      },
      "source": [
        "### Write a function to download and save the downloaded texts"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "qNZ0LJNJXYwc"
      },
      "outputs": [],
      "source": [
        "def save_text_files(num_download=10,verbosity=1):\n",
        "    \"\"\"\n",
        "    Downloads top N books from Gutenberg.org where N is specified by user.\n",
        "    If verbosity is turned on (set to 1) then prints the downloading status for every book.\n",
        "    Asks user for a location on computer where to save the downloaded Ebooks and process accordingly.\n",
        "    Returns status message indicating how many ebooks could be successfully downloaded and saved\n",
        "    \"\"\"\n",
        "\n",
        "    import os\n",
        "\n",
        "    # Download the Ebooks and save in a dictionary object (in-memory)\n",
        "    dict_books=download_top_books(num_download=num_download,verbosity=verbosity)\n",
        "\n",
        "    if dict_books=={}:\n",
        "        return None\n",
        "\n",
        "    # Ask use for a save location (directory path)\n",
        "    savelocation=input(\"Please enter a folder location to save the Ebooks in:\")\n",
        "\n",
        "    count_successful_download=0\n",
        "\n",
        "    # Create a default folder/directory in the current working directory if the input is blank\n",
        "    if (len(savelocation)<1):\n",
        "        savelocation=os.getcwd()+'\\\\'+'Ebooks'+'\\\\'\n",
        "        # Creates new directory if the directory does not exist. Otherwise, just use the existing path.\n",
        "        if not os.path.isdir(savelocation):\n",
        "            os.mkdir(savelocation)\n",
        "    else:\n",
        "        if savelocation[-1]=='\\\\':\n",
        "            os.mkdir(savelocation)\n",
        "        else:\n",
        "            os.mkdir(savelocation+'\\\\')\n",
        "    #print(\"Saving files at:\",savelocation)\n",
        "    for k,v in dict_books.items():\n",
        "        if (v!=\"NOT FOUND\"):\n",
        "            filename=savelocation+str(k)+'.txt'\n",
        "            file=open(filename,'wb')\n",
        "            file.write(v.encode(\"UTF-8\",'ignore'))\n",
        "            file.close()\n",
        "            count_successful_download+=1\n",
        "\n",
        "    # Status message\n",
        "    print (f\"{count_successful_download} book(s) was/were successfully downloaded and saved to the location {savelocation}\")\n",
        "    if (num_download!=count_successful_download):\n",
        "        print(f\"{num_download-count_successful_download} books were not found on the server!\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "background_save": true,
          "base_uri": "https://localhost:8080/"
        },
        "id": "GuhcQI9tXYwc",
        "scrolled": false,
        "outputId": "9893b4db-01b8-4d62-a68d-1969bbb74420"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Working on book: Romeo and Juliet by William Shakespeare \n",
            "Finished downloading 1.11%\n",
            "Working on book: Moby Dick\n",
            "Finished downloading 2.22%\n",
            "Working on book: Pride and Prejudice by Jane Austen \n",
            "Finished downloading 3.33%\n",
            "Working on book: Middlemarch by George Eliot \n",
            "Finished downloading 4.44%\n",
            "Working on book: A Room with a View by E\n",
            "Finished downloading 5.56%\n",
            "Working on book: Frankenstein\n",
            "Finished downloading 6.67%\n",
            "Working on book: The Complete Works of William Shakespeare by William Shakespeare \n",
            "Finished downloading 7.78%\n",
            "Working on book: Little Women\n",
            "Finished downloading 8.89%\n",
            "Working on book: The Blue Castle\n",
            "Finished downloading 10.0%\n",
            "Working on book: Alice\n",
            "Finished downloading 11.11%\n",
            "Working on book: The Enchanted April by Elizabeth Von Arnim \n",
            "Finished downloading 12.22%\n",
            "Working on book: The Adventures of Ferdinand Count Fathom \n",
            "Finished downloading 13.33%\n",
            "Working on book: Cranford by Elizabeth Cleghorn Gaskell \n",
            "Finished downloading 14.44%\n",
            "Working on book: The Expedition of Humphry Clinker by T\n",
            "Finished downloading 15.56%\n",
            "Working on book: Twenty years after by Alexandre Dumas and Auguste Maquet \n",
            "Finished downloading 16.67%\n",
            "Working on book: My Life \n",
            "Finished downloading 17.78%\n",
            "Working on book: History of Tom Jones\n",
            "Finished downloading 18.89%\n",
            "Working on book: The Adventures of Roderick Random by T\n",
            "Finished downloading 20.0%\n",
            "Working on book: Crime and Punishment by Fyodor Dostoyevsky \n",
            "Finished downloading 21.11%\n",
            "Working on book: Dracula by Bram Stoker \n",
            "Finished downloading 22.22%\n",
            "Working on book: Metamorphosis by Franz Kafka \n",
            "Finished downloading 23.33%\n",
            "Working on book: A Doll\n",
            "Finished downloading 24.44%\n",
            "Working on book: The Brothers Karamazov by Fyodor Dostoyevsky \n",
            "Finished downloading 25.56%\n",
            "Working on book: The Picture of Dorian Gray by Oscar Wilde \n",
            "Finished downloading 26.67%\n",
            "Working on book: The Devil is an Ass by Ben Jonson \n",
            "Finished downloading 27.78%\n",
            "Working on book: Calculus Made Easy by Silvanus P\n",
            "Finished downloading 28.89%\n",
            "Working on book: A Tale of Two Cities by Charles Dickens \n",
            "Finished downloading 30.0%\n",
            "Working on book: War and Peace by graf Leo Tolstoy \n",
            "Finished downloading 31.11%\n",
            "Working on book: Adventures of Huckleberry Finn by Mark Twain \n",
            "Finished downloading 32.22%\n",
            "Working on book: The Great Gatsby by F\n",
            "Finished downloading 33.33%\n",
            "Working on book: The Scarlet Letter by Nathaniel Hawthorne \n",
            "Finished downloading 34.44%\n",
            "Working on book: The Adventures of Sherlock Holmes by Arthur Conan Doyle \n",
            "**ERROR: The Adventures of Sherlock Holmes by Arthur Conan Doyle  Not Found**\n",
            "Working on book: Doctrina Christiana \n",
            "Finished downloading 36.67%\n",
            "Working on book: The Iliad by Homer \n",
            "Finished downloading 37.78%\n",
            "Working on book: Thus Spake Zarathustra\n",
            "Finished downloading 38.89%\n",
            "Working on book: Ulysses by James Joyce \n",
            "Finished downloading 40.0%\n",
            "Working on book: Great Expectations by Charles Dickens \n",
            "Finished downloading 41.11%\n",
            "Working on book: The Importance of Being Earnest\n",
            "Finished downloading 42.22%\n",
            "Working on book: Don Quijote by Miguel de Cervantes Saavedra \n",
            "**ERROR: Don Quijote by Miguel de Cervantes Saavedra  Not Found**\n",
            "Working on book: Beowulf\n",
            "Finished downloading 44.44%\n",
            "Working on book: The Romance of Lust\n",
            "Finished downloading 45.56%\n",
            "Working on book: The Count of Monte Cristo by Alexandre Dumas and Auguste Maquet \n",
            "Finished downloading 46.67%\n",
            "Working on book: Dreams of an astronomer by Camille Flammarion \n",
            "Finished downloading 47.78%\n",
            "Working on book: Grimms\n",
            "Finished downloading 48.89%\n",
            "Working on book: The Kama Sutra of Vatsyayana by Vatsyayana \n",
            "Finished downloading 50.0%\n",
            "Working on book: Jane Eyre\n",
            "Finished downloading 51.11%\n",
            "Working on book: Many furrows by A\n",
            "Finished downloading 52.22%\n",
            "Working on book: The works of the highly experienced and famous chymist\n",
            "Finished downloading 53.33%\n",
            "Working on book: The Prince by Niccol\n",
            "Finished downloading 54.44%\n",
            "Working on book: Beyond Good and Evil by Friedrich Wilhelm Nietzsche \n",
            "Finished downloading 55.56%\n",
            "Working on book: The Yellow Wallpaper by Charlotte Perkins Gilman \n",
            "**ERROR: The Yellow Wallpaper by Charlotte Perkins Gilman  Not Found**\n",
            "Working on book: The King in Yellow by Robert W\n",
            "Finished downloading 57.78%\n",
            "Working on book: A Modest Proposal by Jonathan Swift \n",
            "Finished downloading 58.89%\n",
            "Working on book: A Christmas Carol by Charles Dickens \n",
            "Finished downloading 60.0%\n",
            "Working on book: Tractatus Logico\n",
            "Finished downloading 61.11%\n",
            "Working on book: Wuthering Heights by Emily Bront\n",
            "**ERROR: Wuthering Heights by Emily Bront Not Found**\n",
            "Working on book: The Odyssey by Homer \n",
            "Finished downloading 63.33%\n",
            "Working on book: The pageant of Parliament\n",
            "**ERROR: The pageant of Parliament Not Found**\n",
            "Working on book: Anne of Green Gables by L\n",
            "Finished downloading 65.56%\n",
            "Working on book: The Adventures of Tom Sawyer\n",
            "Finished downloading 66.67%\n",
            "Working on book: The Strange Case of Dr\n",
            "**ERROR: The Strange Case of Dr Not Found**\n",
            "Working on book: Heart of Darkness by Joseph Conrad \n",
            "Finished downloading 68.89%\n",
            "Working on book: A Hausa botanical vocabulary by J\n",
            "Finished downloading 70.0%\n",
            "Working on book: A Study in Scarlet by Arthur Conan Doyle \n",
            "Finished downloading 71.11%\n",
            "Working on book: Winnie\n",
            "Finished downloading 72.22%\n",
            "Working on book: Novo dicion\n",
            "Finished downloading 73.33%\n",
            "Working on book: Meditations by Emperor of Rome Marcus Aurelius \n",
            "Finished downloading 74.44%\n",
            "Working on book: White Nights and Other Stories by Fyodor Dostoyevsky \n",
            "Finished downloading 75.56%\n",
            "Working on book: Peter Pan by J\n",
            "Finished downloading 76.67%\n",
            "Working on book: The Republic by Plato \n",
            "Finished downloading 77.78%\n",
            "Working on book: Through the Looking\n",
            "Finished downloading 78.89%\n",
            "Working on book: Du c\n",
            "Finished downloading 80.0%\n",
            "Working on book: The First Book of Adam and Eve by Rutherford Hayes Platt \n",
            "Finished downloading 81.11%\n",
            "Working on book: Narrative of the Life of Frederick Douglass\n",
            "**ERROR: Narrative of the Life of Frederick Douglass Not Found**\n",
            "Working on book: The divine comedy by Dante Alighieri \n",
            "Finished downloading 83.33%\n",
            "Working on book: Leviathan by Thomas Hobbes \n",
            "Finished downloading 84.44%\n",
            "Working on book: The Souls of Black Folk by W\n",
            "Finished downloading 85.56%\n",
            "Working on book: Josefine Mutzenbacher by Felix Salten \n",
            "Finished downloading 86.67%\n",
            "Working on book: The Legend of Sleepy Hollow by Washington Irving \n",
            "**ERROR: The Legend of Sleepy Hollow by Washington Irving  Not Found**\n",
            "Working on book: Don Quixote by Miguel de Cervantes Saavedra \n",
            "Finished downloading 88.89%\n",
            "Working on book: Little Women by Louisa May Alcott \n",
            "Finished downloading 90.0%\n",
            "Working on book: Treasure Island by Robert Louis Stevenson \n",
            "Finished downloading 91.11%\n",
            "Working on book: Les Mis\n",
            "Finished downloading 92.22%\n",
            "Working on book: The Problems of Philosophy by Bertrand Russell \n",
            "**ERROR: The Problems of Philosophy by Bertrand Russell  Not Found**\n",
            "Working on book: The War of the Worlds by H\n",
            "Finished downloading 94.44%\n",
            "Working on book: A Christmas Carol in Prose\n",
            "Finished downloading 95.56%\n",
            "Working on book: Emma by Jane Austen \n",
            "Finished downloading 96.67%\n",
            "Working on book: Second Treatise of Government by John Locke \n",
            "Finished downloading 97.78%\n",
            "Working on book: British birds by Frank E\n",
            "Finished downloading 98.89%\n",
            "Working on book: Half a Man\n",
            "**ERROR: Half a Man Not Found**\n",
            "----------------------------------------\n",
            "Finished downloading all books!\n",
            "----------------------------------------\n"
          ]
        }
      ],
      "source": [
        "save_text_files(90,verbosity=1)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.2"
    },
    "latex_envs": {
      "LaTeX_envs_menu_present": true,
      "autoclose": false,
      "autocomplete": true,
      "bibliofile": "biblio.bib",
      "cite_by": "apalike",
      "current_citInitial": 1,
      "eqLabelWithNumbers": true,
      "eqNumInitial": 1,
      "hotkeys": {
        "equation": "Ctrl-E",
        "itemize": "Ctrl-I"
      },
      "labels_anchors": false,
      "latex_user_defs": false,
      "report_style_numbering": false,
      "user_envs_cfg": false
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}