Skip to main content

LLMonitor Tutorial

LLMonitor is an open-source observability platform that provides cost tracking, user tracking and powerful agent tracing.

Use LLMonitor to log requests across all LLM Providers (OpenAI, Azure, Anthropic, Cohere, Replicate, PaLM)​

liteLLM provides callbacks, making it easy for you to log data depending on the status of your responses.

info

We want to learn how we can make the callbacks better! Meet the founders or join our discord

Using Callbacks​

First, sign up to get an app ID on the LLMonitor dashboard.

Use just 2 lines of code, to instantly log your responses across all providers with llmonitor:

litellm.success_callback = ["llmonitor"]
litellm.failure_callback = ["llmonitor"]

Complete code

from litellm import completion

## set env variables
os.environ["LLMONITOR_APP_ID"] = "your-llmonitor-app-id"
# Optional: os.environ["LLMONITOR_API_URL"] = "self-hosting-url"

os.environ["OPENAI_API_KEY"], os.environ["COHERE_API_KEY"] = "", ""

# set callbacks
litellm.success_callback = ["llmonitor"]
litellm.failure_callback = ["llmonitor"]

#openai call
response = completion(
  model="gpt-3.5-turbo", 
  messages=[{"role": "user", "content": "Hi 👋 - i'm openai"}],
  user="ishaan_litellm"
)

#cohere call
response = completion(
  model="command-nightly", 
  messages=[{"role": "user", "content": "Hi 👋 - i'm cohere"}],
  user="ishaan_litellm"
)

Support & Talk to Founders​