cookbooks/python/langchain/lc_openai_getting_started.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LangChain + GitHub Models + Azure OpenAI\n",
    "\n",
    "LangChain is a modular framework designed to streamline the development and integration of applications utilizing large language models (LLMs) by providing tools for effective prompt engineering, data handling, and complex workflows.\n",
    "\n",
    "LangChain supports a number of model providers including Azure OpenAI and Cohere. Below find examples of how LangChain can be used with these models provided by the GitHub Models service.\n",
    "\n",
    "\n",
    "## 1. Install dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install langchain-openai\n",
    "%pip install langchain-community\n",
    "%pip install faiss-cpu"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Constructing LangChain's ChatOpenAI class\n",
    "\n",
    "Setting up the OpenAI API key and base URL for LangChain is the same as for OpenAI's Python client. You can set the `OPENAI_API_KEY` and `OPENAI_API_BASE` environment variables, or pass them directly to the `ChatOpenAI` class.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_openai import ChatOpenAI\n",
    "import dotenv\n",
    "import os\n",
    "\n",
    "dotenv.load_dotenv()\n",
    "\n",
    "if not os.getenv(\"GITHUB_TOKEN\"):\n",
    "    raise ValueError(\"GITHUB_TOKEN is not set\")\n",
    "\n",
    "os.environ[\"OPENAI_API_KEY\"] = os.getenv(\"GITHUB_TOKEN\")\n",
    "os.environ[\"OPENAI_BASE_URL\"] = \"https://models.inference.ai.azure.com/\"\n",
    "\n",
    "GPT_MODEL = \"gpt-4o-mini\"\n",
    "\n",
    "llm = ChatOpenAI(model=GPT_MODEL)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you can use the ChatOpenAI class to interact with the OpenAI API. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import Markdown, display\n",
    "\n",
    "response = llm.invoke(\"What is the meaning of life?\")\n",
    "display(Markdown(response.content))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Use a prompt template\n",
    "\n",
    "LangChain has the concept of a prompt template that allows you parameterize the prompt and fill in the parameters with values. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "prompt = ChatPromptTemplate.from_messages([\n",
    "    (\"system\", \"You are a fun assistant that uses a lot of emojis.\"),\n",
    "    (\"user\", \"{input}\")\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we combine the promt with the ChatOpenAI into a chain:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chain = prompt | llm "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then we invoke the chain with the input data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chain.invoke({\"input\": \"when was the first ESC held?\"})\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Add a parsing step to your chain\n",
    "\n",
    "We can also extend the chain to include additional steps, such as parsing the content from the reponse:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.output_parsers import StrOutputParser\n",
    "\n",
    "output_parser = StrOutputParser()\n",
    "chain = prompt | llm | output_parser\n",
    "\n",
    "chain.invoke({\"input\": \"when was the first ESC held?\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Use retrieval to ground the model\n",
    "\n",
    "Now let's ask for some more recent information:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chain.invoke({\"input\": f\"Who won the 2024 ESC?\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks like the model is not up to date with the latest contest. Let's help it out with some content from the wikipedia:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import WebBaseLoader\n",
    "\n",
    "loader = WebBaseLoader(\"https://en.wikipedia.org/wiki/Eurovision_Song_Contest_2024\")\n",
    "\n",
    "docs = loader.load()\n",
    "len(docs[0].page_content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Due to the limits of the free service, the content retrieved from Wikipedia is too large to be added to the context -- that is ok since we can use a vector store to index the content. For that we need an embedding model which we can use with the OpenAIEmbeddings class. Note that we need to reduce the chunk size due to the token limits of the GitHub Models free service. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_openai import OpenAIEmbeddings\n",
    "\n",
    "EMBEDDINGS_MODEL = \"text-embedding-3-small\"\n",
    "\n",
    "# need to constrain the chunk_size to work within the limits of the GitHub Model service\n",
    "embeddings = OpenAIEmbeddings(model=EMBEDDINGS_MODEL,chunk_size=30)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we will use the [FAISS](https://github.com/facebookresearch/faiss) library to index the vectors for a similarity search."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.vectorstores import FAISS\n",
    "from langchain_text_splitters import RecursiveCharacterTextSplitter\n",
    "\n",
    "text_splitter = RecursiveCharacterTextSplitter()\n",
    "documents = text_splitter.split_documents(docs)\n",
    "vector = FAISS.from_documents(documents, embeddings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we can create a document chain that will use some context and the user's question to provide an answer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains.combine_documents import create_stuff_documents_chain\n",
    "\n",
    "prompt = ChatPromptTemplate.from_template(\"\"\"You are a fun assistant that uses a lot of emojis.\n",
    "Answer the following question based only on the provided context:\n",
    "\n",
    "<context>\n",
    "{context}\n",
    "</context>\n",
    "\n",
    "Question: {input}\"\"\")\n",
    "\n",
    "document_chain = create_stuff_documents_chain(llm, prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could just provide the context directly to the chain like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.documents import Document\n",
    "\n",
    "document_chain.invoke({\n",
    "    \"input\": \"Who won 2024 ESC?\",\n",
    "    \"context\": [Document(page_content=\"The top 3 placings in the 2024 Eurovision Song Contest were:\\n1. Switzerland\\n2. Croatia\\n3. Ukraine\")]\n",
    "})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But we really want a retrieval chain that will retrieve the right documents from our vector index and then uses it as context in our prompt like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import create_retrieval_chain\n",
    "\n",
    "retriever = vector.as_retriever()\n",
    "retrieval_chain = create_retrieval_chain(retriever, document_chain)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's ask this one who won the 2024 ESC:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = retrieval_chain.invoke({\"input\": f\"Who won the 2024 ESC?\"})\n",
    "print(response[\"answer\"])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And a few more increasingly detailed questions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = retrieval_chain.invoke({\"input\": \"where did the 2024 ESC take place?\"})\n",
    "print(response[\"answer\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = retrieval_chain.invoke({\"input\": \"who designed the stage for the 2024 ESC?\"})\n",
    "print(response[\"answer\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = retrieval_chain.invoke({\"input\": \"when did the semi-finals for the 2024 ESC take place?\"})\n",
    "print(response[\"answer\"])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "gh-cookbook",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}