#LlamaIndex

Use LlamaIndex's OpenAI-like integration for arbitrary GetRouter model IDs. Configure and verify the LLM and embedding models separately.

#Install

pip install -U llama-index-llms-openai-like llama-index-embeddings-openai
export GETROUTER_API_KEY="YOUR_API_KEY"

#Configure the LLM

import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="MODEL_ID",
    api_key=os.environ["GETROUTER_API_KEY"],
    api_base="https://getrouter.ai/v1",
    is_chat_model=True,
    context_window=int(os.environ["MODEL_CONTEXT_WINDOW"]),
)

print(llm.complete("Reply only: connected"))

Set MODEL_CONTEXT_WINDOW to the selected model's actual context-window size as an integer. Do not copy a limit from a different model. Only add is_function_calling_model=True after confirming that the selected model supports tool calling; otherwise leave it disabled.

#Configure embeddings

import os
from llama_index.embeddings.openai import OpenAIEmbedding

embed_model = OpenAIEmbedding(
    model=os.environ["EMBEDDING_MODEL_ID"],
    api_key=os.environ["GETROUTER_API_KEY"],
    api_base="https://getrouter.ai/v1",
)

Set EMBEDDING_MODEL_ID to an actual embedding-capable model from the model catalog. Do not assume that the LLM model is also an embedding model. Changing the embedding model or vector dimensions requires rebuilding existing indexes.

#Before production

• Test one LLM request and one embedding request separately.
• Set MODEL_CONTEXT_WINDOW from the selected model's documented capability.
• Pin the embedding model and vector dimensions used by each index.
• Configure concurrency, retries, and cost limits for bulk ingestion.
• Verify Tool / Agent workflows separately against the tool calling guide.