Gateway
Portkey Gateway is an open-source AI gateway designed to provide a unified API for interacting with over 1,600 large language models (LLMs) from various providers. It acts as a central routing layer between your application and multiple LLM services, simplifying integration and management. One of its key features is integrated guardrails, offering more than 50 AI guardrails that can be applied to requests and responses for content moderation, PII redaction, and other policy enforcement. This helps ensure compliance and safety without additional tooling. The gateway is built for speed and reliability, supporting features like automatic retries, fallbacks, load balancing, and semantic caching to optimize performance and cost. It aims to provide a fast and friendly API, masking the differences between providers and enabling developers to switch models or providers with minimal code changes. Portkey Gateway is self-hosted and open source, allowing teams to maintain full control over their data and infrastructure. It is part of the broader Portkey ecosystem, which also offers a cloud-based version with additional enterprise features.
- Category
- LLM
- Pricing
- Open Source
Problem solved
Portkey Gateway solves the complexity of integrating and managing multiple LLM providers by offering a single, unified API. It centralizes routing, reliability, and governance concerns such as guardrails, fallbacks, and caching, allowing developers to focus on building AI applications rather than dealing with provider-specific integrations and operational overhead.
Use cases
- Build AI applications that need to switch between multiple LLM providers without code changes
- Add content moderation and PII redaction guardrails to LLM requests and responses
- Improve reliability with automatic retries and fallback models
- Reduce latency and cost through semantic caching of LLM responses
- Centralize API key management and usage tracking for LLM providers
- Load balance traffic across multiple models or providers to avoid rate limits
- Enforce organizational AI policies and compliance in a single place
Audience
- Developers
- ML Engineers
- Data Scientists
- Enterprise
Pros
- Supports 1,600+ LLMs through a single API
- Integrated with 50+ AI guardrails for safety and compliance
- Blazing fast performance with low overhead
- Open source and self-hosted, giving full control
- Includes reliability features like retries, fallbacks, and caching
Cons
- Requires self-hosting and infrastructure management
- May lack some advanced features available in Portkey's cloud offering
- Configuration of guardrails can be complex for beginners
- Limited documentation compared to commercial alternatives
Official website: https://github.com/Portkey-AI/gateway
Canonical LambdaBase page: https://www.lambdabase.com/tools/gateway