add LiteLLM_batch_completions cookbook

Author: ishaan-jaff

cookbook/LiteLLM_batch_completion.ipynb

@@ -0,0 +1,166 @@
+{
+  "nbformat": 4,
+  "nbformat_minor": 0,
+  "metadata": {
+    "colab": {
+      "provenance": []
+    },
+    "kernelspec": {
+      "name": "python3",
+      "display_name": "Python 3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "source": [
+        "# LiteLLM Batch Completions Example\n",
+        "\n",
+        "* This tutorial walks through using `batch_completion`\n",
+        "* Docs: https://docs.litellm.ai/docs/completion/batching"
+      ],
+      "metadata": {
+        "id": "MbLbs1tbISk-"
+      }
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {
+        "id": "Ty6-ko_aDlPF"
+      },
+      "outputs": [],
+      "source": [
+        "!pip install litellm"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "## Import Batch Completion"
+      ],
+      "metadata": {
+        "id": "KGhNJRUCIh1j"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "import litellm\n",
+        "import os\n",
+        "from litellm import batch_completion\n",
+        "\n",
+        "# set your API_KEY\n",
+        "os.environ['ANTHROPIC_API_KEY'] = \"\""
+      ],
+      "metadata": {
+        "id": "LOtI43snDrSK"
+      },
+      "execution_count": 7,
+      "outputs": []
+    },
+    {
+      "cell_type": "markdown",
+      "source": [
+        "## Calling `litellm.batch_completion`\n",
+        "\n",
+        "In the batch_completion method, you provide a list of messages where each sub-list of messages is passed to litellm.completion(), allowing you to process multiple prompts efficiently in a single API call."
+      ],
+      "metadata": {
+        "id": "Xhv92NBaIpaw"
+      }
+    },
+    {
+      "cell_type": "code",
+      "source": [
+        "import litellm\n",
+        "import os\n",
+        "from litellm import batch_completion\n",
+        "\n",
+        "os.environ['ANTHROPIC_API_KEY'] = \"\"\n",
+        "\n",
+        "\n",
+        "responses = batch_completion(\n",
+        "    model=\"claude-2\",\n",
+        "    messages = [\n",
+        "        [\n",
+        "            {\n",
+        "                \"role\": \"user\",\n",
+        "                \"content\": \"good morning? \"\n",
+        "            }\n",
+        "        ],\n",
+        "        [\n",
+        "            {\n",
+        "                \"role\": \"user\",\n",
+        "                \"content\": \"what's the time? \"\n",
+        "            }\n",
+        "        ]\n",
+        "    ]\n",
+        ")\n",
+        "responses"
+      ],
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "yY7GIRLsDywu",
+        "outputId": "009ea67f-95d5-462b-947f-b0d21e60c5bb"
+      },
+      "execution_count": 11,
+      "outputs": [
+        {
+          "output_type": "execute_result",
+          "data": {
+            "text/plain": [
+              "[<ModelResponse at 0x7a164eed4450> JSON: {\n",
+              "   \"choices\": [\n",
+              "     {\n",
+              "       \"finish_reason\": \"stop\",\n",
+              "       \"index\": 0,\n",
+              "       \"message\": {\n",
+              "         \"content\": \" Good morning!\",\n",
+              "         \"role\": \"assistant\",\n",
+              "         \"logprobs\": null\n",
+              "       }\n",
+              "     }\n",
+              "   ],\n",
+              "   \"created\": 1694030351.309254,\n",
+              "   \"model\": \"claude-2\",\n",
+              "   \"usage\": {\n",
+              "     \"prompt_tokens\": 11,\n",
+              "     \"completion_tokens\": 3,\n",
+              "     \"total_tokens\": 14\n",
+              "   }\n",
+              " },\n",
+              " <ModelResponse at 0x7a164eed5800> JSON: {\n",
+              "   \"choices\": [\n",
+              "     {\n",
+              "       \"finish_reason\": \"stop\",\n",
+              "       \"index\": 0,\n",
+              "       \"message\": {\n",
+              "         \"content\": \" I'm an AI assistant created by Anthropic. I don't actually have a concept of the current time.\",\n",
+              "         \"role\": \"assistant\",\n",
+              "         \"logprobs\": null\n",
+              "       }\n",
+              "     }\n",
+              "   ],\n",
+              "   \"created\": 1694030352.1215081,\n",
+              "   \"model\": \"claude-2\",\n",
+              "   \"usage\": {\n",
+              "     \"prompt_tokens\": 13,\n",
+              "     \"completion_tokens\": 22,\n",
+              "     \"total_tokens\": 35\n",
+              "   }\n",
+              " }]"
+            ]
+          },
+          "metadata": {},
+          "execution_count": 11
+        }
+      ]
+    }
+  ]
+}

docs/my-website/docs/completion/batching.md

@@ -1,6 +1,6 @@
-# Batching Completion Calls - batch_completion
+# Batching Completion() Calls 
 
-Batch Completion allows you to pass a batch of completion() requests to process multiple `messages` in a single API call.
+In the batch_completion method, you provide a list of `messages` where each sub-list of messages is passed to `litellm.completion()`, allowing you to process multiple prompts efficiently in a single API call.
 
 ## Example Code
 ```python
@@ -28,7 +28,4 @@ responses = batch_completion(
         ]
     ]
 )
-```
-
-
-In the batch_completion method, you provide a list of `messages` where each sub-list of messages is passed to `litellm.completion()`, allowing you to process multiple prompts efficiently in a single API call.
+```