Skip to main content
Portkey provides a robust and secure gateway to integrate various Large Language Models (LLMs) into applications, including Together AI’s hosted models. With Portkey, take advantage of features like fast AI gateway access, observability, prompt management, and more, while securely managing API keys through Model Catalog.

​
Quick Start

Get Together AI working in 3 steps: 
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @together-ai provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@together-ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)
Tip: You can also set provider="@together-ai" in Portkey() and use just model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo" in the request.

​
Add Provider in Model Catalog

  1. Go to Model Catalog → Add Provider
  2. Select Together AI
  3. Choose existing credentials or create new by entering your Together AI API key
  4. Name your provider (e.g., together-ai-prod)

Complete Setup Guide →

See all setup options, code examples, and detailed instructions

​
Managing Together AI Prompts

Manage all prompt templates to Together AI in the Prompt Library. All current Together AI models are supported, and you can easily test different prompts. Use the portkey.prompts.completions.create interface to use the prompt in an application.

​
Next Steps

Add Metadata

Add metadata to your Together AI requests

Gateway Configs

Add gateway configs to your Together AI requests

Tracing

Trace your Together AI requests

Fallbacks

Setup fallback from OpenAI to Together AI
For complete SDK documentation:

SDK Reference

Complete Portkey SDK documentation