🦙 docs: fix litellm.md (#2566)

docs/install/configuration/litellm.md

@@ -12,11 +12,13 @@ Use **[LiteLLM Proxy](https://docs.litellm.ai/docs/simple_proxy)** for:
 * Authentication & Spend Tracking Virtual Keys
 
 ## Start LiteLLM Proxy Server 
-### 1. Uncomment desired sections in docker-compose.override.yml
+
+## 1. Uncomment desired sections in docker-compose.override.yml
 The override file contains sections for the below LiteLLM features
 
 Minimum working `docker-compose.override.yml` Example:
-```
+
+```yaml
 litellm:
     image: ghcr.io/berriai/litellm:main-latest
     volumes:
@@ -32,24 +34,29 @@ litellm:
       GOOGLE_APPLICATION_CREDENTIALS: /app/application_default_credentials.json
 ```
 
-#### Caching with Redis
+### Caching with Redis
 Litellm supports in-memory, redis, and s3 caching. Note: Caching currently only works with exact matching.
 
-#### Performance Monitoring with Langfuse
+### Performance Monitoring with Langfuse
 Litellm supports various logging and observability options.  The settings below will enable Langfuse which will provide a cache_hit tag showing which conversations used cache.
 
-### 2. Create a Config for LiteLLM proxy 
+## 2. Create a Config for LiteLLM proxy 
 LiteLLM requires a configuration file in addition to the override file. Within LibreChat, this will be `litellm/litellm-config.yml`. The file 
 below has the options to enable llm proxy to various providers, load balancing, Redis caching, and Langfuse monitoring. Review documentation for other configuration options.
 More information on LiteLLM configurations here: **[docs.litellm.ai/docs/simple_proxy](https://docs.litellm.ai/docs/simple_proxy)**
 
-#### Working Example of incorporating OpenAI, Azure OpenAI, AWS Bedrock, and GCP
+### Working Example of incorporating OpenAI, Azure OpenAI, AWS Bedrock, and GCP
 
 Please note the `...` being a secret or a value you should not share (API key, custom tenant endpoint, etc)
 You can potentially use env variables for these too, ex: `api_key: "os.environ/AZURE_API_KEY" # does os.getenv("AZURE_API_KEY")`
-```yaml
+
-model_list:
+??? abstract "Example A"
+
+    ```yaml
+    model_list:
       # https://litellm.vercel.app/docs/proxy/quick_start
+
+      # Anthropic
       - model_name: claude-3-haiku
         litellm_params:
           model: bedrock/anthropic.claude-3-haiku-20240307-v1:0
@@ -85,6 +92,7 @@ model_list:
           aws_access_key_id: A...
           aws_secret_access_key: ...
 
+      # Llama
       - model_name: llama2-13b
         litellm_params:
           model: bedrock/meta.llama2-13b-chat-v1
@@ -113,7 +121,7 @@ model_list:
           aws_access_key_id: A...
           aws_secret_access_key: ...
 
-
+      # Mistral
       - model_name: mistral-7b-instruct
         litellm_params:
           model: bedrock/mistral.mistral-7b-instruct-v0:2
@@ -135,6 +143,7 @@ model_list:
           aws_access_key_id: A...
           aws_secret_access_key: ...
 
+      # Cohere
       - model_name: cohere-command-v14
         litellm_params:
           model: bedrock/cohere.command-text-v14
@@ -149,6 +158,7 @@ model_list:
           aws_access_key_id: A...
           aws_secret_access_key: ...
 
+      # AI21 Labs
       - model_name: ai21-j2-mid
         litellm_params:
           model: bedrock/ai21.j2-mid-v1
@@ -163,6 +173,7 @@ model_list:
           aws_access_key_id: A...
           aws_secret_access_key: ...
 
+      # Amazon
       - model_name: amazon-titan-lite
         litellm_params:
           model: bedrock/amazon.titan-text-lite-v1
@@ -177,9 +188,7 @@ model_list:
           aws_access_key_id: A...
           aws_secret_access_key: ...
 
-
+      # Azure
-
-
       - model_name: azure-gpt-4-turbo-preview
         litellm_params:
           model: azure/gpt-4-turbo-preview
@@ -210,8 +219,7 @@ model_list:
           api_base: https://tenant-name.openai.azure.com/
           api_key: ...
 
-
+      # OpenAI
-
       - model_name: gpt-4-turbo
         litellm_params:
           model: gpt-4-turbo
@@ -247,51 +255,48 @@ model_list:
           model: gpt-4-vision-preview
           api_key: ...
 
-
+      # Google
-
+      # NOTE: For Google - see above about required auth "GOOGLE_APPLICATION_CREDENTIALS" environment and volume mount
-  # NOTE: For Google - see above about required auth "GOOGLE_APPLICATION_CREDENTIALS" envronment and volume mount
       - model_name: google-chat-bison
         litellm_params:
           model: vertex_ai/chat-bison
           vertex_project: gcp-proj-name
           vertex_location: us-central1
 
-  # NOTE: For Google - see above about required auth "GOOGLE_APPLICATION_CREDENTIALS" envronment and volume mount
       - model_name: google-chat-bison-32k
         litellm_params:
           model: vertex_ai/chat-bison-32k
           vertex_project: gcp-proj-name
           vertex_location: us-central1
 
-  # NOTE: For Google - see above about required auth "GOOGLE_APPLICATION_CREDENTIALS" envronment and volume mount
       - model_name: google-gemini-pro-1.0
         litellm_params:
           model: vertex_ai/gemini-pro
           vertex_project: gcp-proj-name
           vertex_location: us-central1
 
-  # NOTE: For Google - see above about required auth "GOOGLE_APPLICATION_CREDENTIALS" envronment and volume mount
       - model_name: google-gemini-pro-1.5-preview
         litellm_params:
           model: vertex_ai/gemini-1.5-pro-preview-0409
           vertex_project: gcp-proj-name
           vertex_location: us-central1
 
-# NOTE: It may be a good idea to comment out "success_callback", "cache", "cache_params" (both lines under) when you first start until this works!
+    # NOTE: It may be a good idea to comment out "success_callback", "cache", "cache_params" (both lines under) when you first start until this works!
-litellm_settings:
+    litellm_settings:
       success_callback: ["langfuse"]
       cache: True
       cache_params:
         type: "redis"
         supported_call_types: ["acompletion", "completion", "embedding", "aembedding"]
-general_settings:
+    general_settings:
       master_key: sk_live_SetToRandomValue
-```
+    ```
 
-#### Example of a few Different Options (ex: rpm, stream, ollama)
+### Example of a few Different Options (ex: rpm, stream, ollama)
-```yaml
 
-model_list:
+??? abstract "Example B"
+    ```yaml
+    model_list:
       - model_name: gpt-3.5-turbo
         litellm_params:
           model: azure/gpt-turbo-small-eu
@@ -320,25 +325,23 @@ model_list:
           model: openai/mistral                                # use openai/* for ollama's openai api compatibility
           api_base: http://ollama:11434/v1
           stream: True
-litellm_settings:
+    litellm_settings:
       success_callback: ["langfuse"]
       cache: True
       cache_params:
         type: "redis"
         supported_call_types: ["acompletion", "completion", "embedding", "aembedding"]
-general_settings:
+    general_settings:
       master_key: sk_live_SetToRandomValue
-```
+    ```
 
-
+## 3. Configure LibreChat
-
-### 3. Configure LibreChat
 
 Use `librechat.yaml` [Configuration file (guide here)](./ai_endpoints.md) to add Reverse Proxies as separate endpoints.
 
 Here is an example config:
 
-```
+```yaml
 custom:
     - name: "Lite LLM"
       # A place holder - otherwise it becomes the default (OpenAI) key
@@ -358,9 +361,8 @@ custom:
       forcePrompt: false
       modelDisplayLabel: "Lite LLM"
 ```
----
 
-### Why use LiteLLM?
+## Why use LiteLLM?
 
 1. **Access to Multiple LLMs**: It allows calling over 100 LLMs from platforms like Huggingface, Bedrock, TogetherAI, etc., using OpenAI's ChatCompletions and Completions format.