dspy-go uses llm-go for model generation protocols and model metadata. pkg/llms adapts an llm-go Generator to dspy-go’s core.LLM interface so existing modules, agents, and optimizers keep the same integration point.

llms.NewLLM selects the registered provider from the model ID and returns a cached core.LLM:

import (
    "os"

    "github.com/XiaoConstantine/dspy-go/pkg/core"
    "github.com/XiaoConstantine/dspy-go/pkg/llms"
)

llm, err := llms.NewLLM(
    os.Getenv("GEMINI_API_KEY"),
    core.ModelGoogleGeminiFlash,
)
if err != nil {
    return err
}
core.SetDefaultLLM(llm)

You can also initialize the factory and use the core convenience function:

llms.EnsureFactory()
if err := core.ConfigureDefaultLLM(apiKey, core.ModelOpenAIGPT4o); err != nil {
    return err
}

Modules resolve an LLM in this order: a module-local model set with SetLLM, a request-local runtime supplied by core.WithRuntime, then the package default set with core.SetDefaultLLM.

Provider Constructors

Use a dedicated constructor when you need provider-specific endpoint or credential behavior:

ProviderConstructorNotes
AnthropicNewAnthropicLLM(apiKey, modelID)Uses ANTHROPIC_OAUTH_TOKEN first, then apiKey, then ANTHROPIC_API_KEY
Google GeminiNewGeminiLLM(apiKey, modelID)Reads GEMINI_API_KEY when apiKey is empty
OpenAI-compatible chatNewOpenAILLM(modelID, options...)Chat Completions constructor; reads OPENAI_API_KEY if WithAPIKey is omitted
OpenAI API keyNewOpenAI(modelID, apiKey)Chat Completions convenience wrapper; requires an explicit key
OpenAI Codex subscriptionNewOpenAICodexLLM(modelID, options...)Requires an application-owned credential resolver
OllamaNewOllamaLLM(modelID, options...)Requires Ollama’s OpenAI-compatible API
llama.cppNewLlamacppLLM(endpoint)Requires an OpenAI-compatible server
Generic compatible endpointNewOpenAICompatible(provider, modelID, baseURL, options...)LiteLLM, LocalAI, FastChat, LM Studio, and similar servers

The generic NewLLM/registry path uses llm-go’s current native API selection. In particular, official OpenAI models use the Responses API. The NewOpenAI and NewOpenAILLM compatibility constructors continue to use Chat Completions.

The exported model constants in pkg/core and llm-go’s catalog are the source of truth. Provider model availability, context limits, pricing, and rate limits change independently of dspy-go, so verify those details in the provider’s current documentation.

OpenAI-Compatible Endpoints

llm, err := llms.NewOpenAICompatible(
    "litellm",
    core.ModelID("my-proxy-model"),
    "http://localhost:4000",
    llms.WithAPIKey(os.Getenv("LITELLM_API_KEY")),
    llms.WithOpenAITimeout(90*time.Second),
    llms.WithHeader("X-Tenant", "example"),
)

Available OpenAI-compatible options are WithAPIKey, WithOpenAIBaseURL, WithOpenAIPath, WithOpenAITimeout, WithHeader, and WithHTTPClient.

For Ollama, use WithBaseURL, WithAuth, WithTimeout, and WithOpenAIAPI. WithNativeAPI returns an unsupported-operation error because llm-go intentionally targets Ollama’s OpenAI-compatible surface.

Generation Options

Generation settings are call options in pkg/core; they are not provider constructor options:

response, err := llm.Generate(
    ctx,
    "Summarize this incident report.",
    core.WithMaxTokens(800),
    core.WithTemperature(0.2),
    core.WithTopP(0.9),
    core.WithPresencePenalty(0.1),
    core.WithFrequencyPenalty(0.1),
    core.WithStopSequences("END"),
)

Pass the same settings through a module with core.WithGenerateOptions:

result, err := predictor.Process(
    ctx,
    inputs,
    core.WithGenerateOptions(
        core.WithMaxTokens(800),
        core.WithTemperature(0.2),
    ),
)

Support for an individual option remains model- and provider-dependent.

Streaming

Streaming is explicit. Consume the returned channel and call Cancel when the consumer stops early:

stream, err := llm.StreamGenerate(ctx, "Explain the result.")
if err != nil {
    return err
}
defer stream.Cancel()

for chunk := range stream.ChunkChannel {
    if chunk.Error != nil {
        return chunk.Error
    }
    fmt.Print(chunk.Content)
}

Modules can stream with core.WithStreamHandler:

result, err := predictor.Process(ctx, inputs,
    core.WithStreamHandler(func(chunk core.StreamChunk) error {
        fmt.Print(chunk.Content)
        return chunk.Error
    }),
)

Embeddings

llm-go is the generation layer. Embeddings remain in dspy-go because they are part of the compatibility core.LLM contract.

LLM adapterEmbedding client retained by dspy-goDefault embedding model
GeminiGoogle Gen AI SDKgemini-embedding-2
OpenAI and OpenAI-compatibleOpenAI-compatible /v1/embeddingstext-embedding-3-small
Ollama and llama.cppOpenAI-compatible /v1/embeddingsConfigured model ID
Anthropic and OpenAI CodexNot supported—
llms.Adapt(generator)Not attached automatically—
embedding, err := llm.CreateEmbedding(
    ctx,
    "text to embed",
    core.WithModel("gemini-embedding-2"),
)

batch, err := llm.CreateEmbeddings(
    ctx,
    []string{"first", "second"},
    core.WithBatchSize(32),
)

Calling an embedding method on an adapter without an embedding client returns an unsupported-operation error.

OpenAI Codex Subscription

ChatGPT subscription access is separate from OpenAI API-key access. Select the openai-codex provider explicitly and send subscription traffic only to the Codex backend.

llm, err := llms.NewOpenAICodexLLM(
    core.ModelOpenAIGPT54,
    llms.WithOpenAICodexCredentials(
        func(ctx context.Context, rejectedToken string) (llms.OpenAICodexCredentials, error) {
            credential, err := credentialStore.Resolve(ctx, rejectedToken)
            if err != nil {
                return llms.OpenAICodexCredentials{}, err
            }
            return llms.OpenAICodexCredentials{
                AccessToken: credential.AccessToken,
                AccountID:   credential.AccountID,
            }, nil
        },
    ),
)

The resolver runs before each request. After a 401, llm-go invokes it once with the rejected token so the application can coordinate refresh safely. The application owns OAuth login, refresh-token rotation, secure persistence, and cross-process synchronization. dspy-go does not ship interactive OAuth or a credential store.

For static registry/CLI use, select a model such as openai-codex:gpt-6.1-sol and provide the OAuth access token as the API key or OPENAI_OAUTH_TOKEN. Account identity can come from account_id, an ID token, or compatible access-token claims.

Timeouts and Capabilities

Use context deadlines for operation-scoped timeouts:

ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

response, err := llm.Generate(ctx, prompt)

Use llm.Capabilities() instead of assuming that every model offered by a provider supports vision, tools, JSON, or streaming.

Next Steps