Skip to content

What is simulation?

Tenro uses simulation rather than traditional patching to test AI agents. No more brittle mocks, verbose setup, or breaking tests when SDKs update.

The problem with patching LLMs

Traditional tests replace a function with a fake using patching:

# Traditional patching approach
@patch("openai.chat.completions.create")
def test_agent(patched_create):
    patched_create.return_value = MagicMock(
        choices=[MagicMock(message=MagicMock(content="Hello"))]
    )
    result = my_agent.run("Hi")
    assert result == "Hello"

This works, but has problems:

Issue Impact
Brittle Breaks when OpenAI SDK changes internal structure
Verbose Every test needs complex patch setup
Incomplete Doesn't cover tool-call metadata or errors
No verification Can't assert what prompts were sent

Simulation: a better approach

Tenro simulates the provider behaviour, not individual functions:

from tenro import Provider
from tenro.simulate import llm, tool
from tenro.testing import tenro

@tenro
def test_agent():
    # Simulate OpenAI's behaviour
    llm.simulate(Provider.OPENAI, response="Hello")

    result = my_agent.run("Hi")

    # Verify what your agent did
    llm.verify(Provider.OPENAI)
    assert result == "Hello"

The simulation handles:

  • Response shapes: Returns proper ChatCompletion objects
  • Multi-turn responses: Returns the next response for each call
  • Tool-call metadata: Adds tool call data via responses=[ToolCall(...)]
  • Errors: Raises exceptions when using use_http=False

How simulation works

Tenro intercepts LLM provider HTTP calls automatically (no decorators required):

┌─────────────┐     ┌─────────────┐     ┌───────────────┐
│ Your Agent  │ ──▶ │ OpenAI SDK  │ ──▶ │  HTTP Call    │
└─────────────┘     └─────────────┘     │ (intercepted) │
                                        └───────┬───────┘
                                                │
                                                ▼
                                      Simulated Response

Your code uses the real OpenAI SDK. Tenro intercepts at the network layer and returns simulated responses that match the real API exactly. This works with LangChain, CrewAI, Pydantic AI, and the official OpenAI/Anthropic SDKs.

Simulation vs patching

Aspect Traditional Patching Tenro Simulation
Level Function replacement Provider behaviour
Setup Verbose patch chains Simple declarations
SDK compatibility Breaks on updates Works with real SDKs
Multi-turn Manual orchestration Built-in sequencing
Tool calls Custom implementation Tool metadata support
Verification Assert on patched calls Semantic verification

When to use simulation

Simulation is ideal when you need to:

  • Test agent workflows without API costs
  • Verify tool calling sequences
  • Test error handling (rate limits, timeouts)
  • Run fast, deterministic CI tests
  • Avoid flaky tests from real API variability

Example: simulating a conversation

from tenro import Provider
from tenro.simulate import llm, tool
from tenro.testing import tenro

@tenro
def test_multi_turn_conversation():
    # Simulate a back-and-forth (each item = one LLM call)
    llm.simulate(
        provider=Provider.OPENAI,
        responses=[
            "I'll help you with that.",
            "Here's what I found...",
            "Is there anything else?",
        ],
    )

    # Run your agent
    conversation_agent.run("Help me research AI")

    # Verify the conversation happened
    llm.verify_many(Provider.OPENAI, at_least=3)

Each call to the LLM returns the next response in sequence, simulating a multi-turn conversation.

Example: interleaved text and tool calls

Some providers (Anthropic, Gemini) support interleaved content in a single response:

from tenro import LLMResponse, Provider, ToolCall
from tenro.simulate import llm, tool
from tenro.testing import tenro

@tenro
def test_interleaved_response():
    # ONE LLM call with text + tool calls interleaved
    llm.simulate(
        provider=Provider.ANTHROPIC,
        responses=[
            LLMResponse([
                "Let me search for that.",
                ToolCall(search, query="AI trends"),
                "Now checking another source.",
                ToolCall(fetch, url="https://..."),
            ])
        ],
    )

    agent.run("Research AI")

    # Only one LLM call, but multiple tool calls
    llm.verify_many(Provider.ANTHROPIC, count=1)
    tool.verify_many(count=2)

Use LLMResponse when you need text and tool calls in a single atomic response.

Next steps