I’ve been using sentry for automatically logging the errors and exceptions of my python projects. A few months ago I needed to log some information if a specific condition is true in my side project’s backend, but I wasn’t able to do this with sentry. It apparently can only work when something fails, and you can’t capture log messages if there’s no failure or exception.

I looked for an affordable and user friendly observability tool and settled on using axiom . It has a generous 500GB ingestion on free tier plan, but you can only view the events for the past 30 days time period. So I’ve been exporting the logs every month into a csv file, since I want to be able to view the trend of some behaviours over time.

Today I came across a new observability tool from pydantic team called logfire . It sounds like an excellent tool, and a brilliant way for the team to monetize their work. Logfire has 10M events per month limit on its free tier, and its SDK can be integrated with OpenAI (for logging your LLM API calls), FastAPI, and databases. These three are the main areas that I’d like to have observability around on my projects, and the SDK sounds very easy to implement into any new or existing project:

  1. Create a new Logfire account and setup a project .

  2. Get a write token and set it as an envoironment variable:

LOGFIRE_TOKEN=your-write-token
  1. Send data to Logfire:
import logfire

logfire.configure()
logfire.info("Hello, {name}!", name="world")
  • Integrate with OpenAI:
import logfire
import openai

client = openai.AsyncClient()
logfire.instrument_openai(client) ## Add this

...

async def stream_openai(messages: list[ChatMessage]):
    chunks = await client.chat.completions.create(
        mode1="gpt-4o",
        messages=messages,
        stream=True
    )

...
  • Integrate with FastAPI:
import logfire
from fastapi import FastAPI

app = FastAPI()
logfire.instrument_fastapi(app)

...

@app.get("/1lm/ask/stream/{chat_id}")
async def llm_stream(chat_id: str) -> StreamingResponse:
    ...
  • Integrate with Python’s logging library:
import logging
import logfire

logfire.configure()
LOGGER = logging.getLogger(__name__)
LOGGER.setLevel(logging.INFO)
LOGGER.addHandler(logfire.LogfireLoggingHandler())

Logfire also has a 30 days retention period, but they are planning to add more options to their paid plans.

To view the logs you can view the live dashboard of the project, or run the following query on the explore page:

select * from records

You can use SQL syntax to further adjust the query and get what you’re looking for in the logs.

It’s not possible to export the log data from Logfire’s UI at the moment, but this is possible with their API (more info ).

Comment? Reply via Email or Bluesky or Twitter.