Skip to main content

​
Quick Start

Get started with Fireworks in under 2 minutes: 
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @fireworks-ai provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@fireworks-ai/accounts/fireworks/models/llama-v3-70b-instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

​
Add Provider in Model Catalog

Before making requests, add Fireworks to your Model Catalog:
  1. Go to Model Catalog → Add Provider
  2. Select Fireworks
  3. Enter your Fireworks API key
  4. Name your provider (e.g., fireworks-ai)

Complete Setup Guide

See all setup options and detailed configuration instructions

​
Fireworks Capabilities

​
Embeddings

Generate embeddings using Fireworks models: 
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

embeddings = portkey.embeddings.create(
    model="thenlper/gte-large",
    input="create vector representation on this sentence"
)

print(embeddings.data[0].embedding)

​
Vision Models

Process images with vision models: 
from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

response = portkey.chat.completions.create(
    model="accounts/fireworks/models/firellava-13b",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text", "text": "Can you describe this image?"},
            {
                "type": "image_url",
                "image_url": {
                    "url": "https://images.unsplash.com/photo-1582538885592-e70a5d7ab3d3"
                }
                        }
                    ]
    }]
        )

print(response.choices[0].message.content)

​
Image Generation

Generate images with Stable Diffusion: 
        from portkey_ai import Portkey
        import base64
        from io import BytesIO
        from PIL import Image

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

        image = portkey.images.generate(
          model="accounts/fireworks/models/stable-diffusion-xl-1024-v1-0",
          prompt="An orange elephant in a purple pond"
        )

Image.open(BytesIO(base64.b64decode(image.data[0].b64_json))).save("generated.png")

​
Function Calling

Use function calling with Fireworks models:

Function Calling Guide

Explore function calling examples and best practices

​
Grammar Mode

Define formal grammars to constrain model outputs: 
        from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

        patient_classification = """
        root      ::= diagnosis
        diagnosis ::= "flu" | "dengue" | "malaria"
        """

response = portkey.chat.completions.create(
    model="accounts/fireworks/models/llama-v3-70b-instruct",
    messages=[{"role": "user", "content": "Patient has fever and chills"}],
    response_format={"type": "grammar", "grammar": patient_classification}
)

print(response.choices[0].message.content)

Fireworks Grammar Guide

Learn more about grammar mode and examples

​
JSON Mode

Force JSON output with optional schema validation: 
        from portkey_ai import Portkey
from pydantic import BaseModel
from typing import List

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@fireworks-ai")

        class Recipe(BaseModel):
            title: str
            description: str
            steps: List[str]

response = portkey.chat.completions.create(
    model="accounts/fireworks/models/llama-v3-70b-instruct",
    messages=[{"role": "user", "content": "Give me a recipe for making Ramen"}],
    response_format={
        "type": "json_object",
                "schema": Recipe.schema_json()
            }
        )

print(response.choices[0].message.content)

Fireworks JSON Mode Guide

Learn more about JSON mode

​
Supported Models

Fireworks provides fast inference for 49+ language models including:
  • Llama models (v3-70b-instruct, v3-8b-instruct)
  • FireLLaVA vision models
  • Stable Diffusion XL image generation
  • Embedding models (gte-large, nomic-embed)
Check Fireworks documentation for the complete model list.

​
Next Steps

Gateway Configs

Add fallbacks, load balancing, and more

Observability

Monitor and trace your Fireworks requests

Prompt Library

Manage and version your prompts

Metadata

Add custom metadata to requests
For complete SDK documentation:

SDK Reference

Complete Portkey SDK documentation