Skip to main content

​
Quick Start

Get started with Inference.net in under 2 minutes: 
from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @inference-net provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@inference-net/llama3",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

​
Add Provider in Model Catalog

Before making requests, add Inference.net to your Model Catalog:
  1. Go to Model Catalog → Add Provider
  2. Select Inference.net
  3. Enter your Inference.net API key
  4. Name your provider (e.g., inference-net)

Complete Setup Guide

See all setup options and detailed configuration instructions

​
Supported Models

Inference.net provides distributed GPU compute for various open-source models including:
  • Llama 3
  • Mistral
  • And other popular open-source models
Check Inference.net’s documentation for the complete model list.

​
Next Steps

Gateway Configs

Add fallbacks, load balancing, and more

Observability

Monitor and trace your Inference.net requests

Prompt Library

Manage and version your prompts

Metadata

Add custom metadata to requests
For complete SDK documentation:

SDK Reference

Complete Portkey SDK documentation