bryan_rodrigues

Profile description

# Relay — every LLM, one interface

relay-llm on GitHub — pip install ai5labs-relay

Apache-2.0 licensed, no enterprise SKU, no closed-source tier. The library
is and stays open source.

Relay is a production-grade Python library that gives developers one interface to every major LLM provider — OpenAI, Anthropic, AWS Bedrock,
Azure, Google Gemini, Vertex AI, Groq, DeepSeek, xAI, Mistral, Cohere,
and 7 more — defined in a single YAML config they check into their repo.

The most distinctive piece is the MCP universal tool layer: any
Model Context Protocol server's tools
work against any provider, including ones without native MCP support.
We translate the schemas at request time so an MCP server you wrote for GPT-4 just works against Claude or Gemini.

## What we ship

• 18 providers, all tested with real wire formats (not OpenAI-compat
  shims for the natives — actual lossless adapters that preserve Anthropic thinking blocks, Gemini grounding, etc.)
  
• Hub-level cache + Anthropic prompt-cache passthrough
  
• Pydantic structured output (compiled per-provider, not text-coerced)
  
• Tiered pricing resolver with provenance on every cost
  
• Cross-provider tool-schema compiler with Mastra-style fallback
  
• OpenTelemetry GenAI semantic conventions (opt-in)
  
• PII redaction, audit logging, pre/post guardrails
  
• 400+ models in the catalog with benchmark scores + a CLI to compare /
  recommend models for your task and budget
  

• 177 unit tests passing on Python 3.10–3.13, mypy --strict clean,
  Sigstore-signed releases via PyPI Trusted Publishing

  Reproducible benchmarks
  show materially faster cold start than LiteLLM (5–19× across runs).

  What your donations fund

  Recurring sponsorship, even small, makes maintenance sustainable:

• Maintainer time — issue triage, PR review, releases, new-provider
  support (a new model or provider lands somewhere every 2-3 weeks in this space)

• Live contract-test budgets — we run nightly tests against real OpenAI / Anthropic / Groq APIs to catch wire-format regressions
  before users do; that costs real money

• Catalog refresh — keeping pricing + capabilities accurate for 400+ models, weekly via CI

• CI compute — matrix testing across 4 Python versions × 3 OSes

• Docs hosting — mkdocs-material site for guides + API reference

  Roadmap (all stays Apache-2.0)

• v0.2 — proxy mode: FastAPI server wrapping the library, for
  teams who want a hosted gateway in their VPC

• v0.3 — multi-tenant primitives, distributed rate limiting,
  audit-log sinks for SIEM

• v1.0 — Rust hot path for sub-millisecond p99 overhead at high
  QPS, while keeping the Python public API identical

  Why Liberapay

  Liberapay's 0% platform fee + recurring-donation model is the right
  shape for OSS infrastructure. Donations are the most direct way to keep the maintainer working on the library full-time, with no gateway,
  no markup, no lock-in.

  Thank you 🙏