# Write a provider plugin

> Connect elelem to a new model API by implementing one method, fetch, that streams events back to the agent.


At the end of this guide you will have a working provider, `compat`, that
connects elelem to any server speaking the OpenAI chat completions format,
including tool calls. You will know the whole provider contract: one
factory, one method, three event types.

## Before you start

- elelem and `elelem-builtins` installed. See
  [the Quickstart](/docs/quickstart/).
- A server that speaks `POST /v1/chat/completions`, such as llama.cpp's
  `llama-server`, vLLM, or a hosted API. Step 1 needs no server at all.
- Familiarity with [tool plugins](/guides/write-a-tool/). Providers load
  the same way.

## How the agent uses a provider

A provider is any object with one method:

```ruby
def fetch(messages, tools = [], &block)
```

For each model call, the agent passes the conversation and the tool list,
and the provider calls the block once per event. The agent ignores the
return value. When the model asks for tools, the agent runs them, appends
the results to `messages`, and calls `fetch` again. A turn ends when a
`fetch` produces no tool calls.

**`messages`** is an array of hashes with symbol keys and string roles.
The first is always the system prompt.

```ruby
[
  { role: "system", content: "You are a terminal coding agent.\n\n..." },
  { role: "user", content: "what is in the readme?" },
  { role: "assistant", content: "",
    tool_calls: [{ id: "call_1", name: "read", arguments: { "path" => "README.md" } }] },
  { role: "tool", tool_call_id: "call_1",
    content: "{\"content\":\"# hello\\n\",\"path\":\"README.md\",\"ok\":true}" }
]
```

Assistant messages carry `tool_calls` only while a turn is in progress.
Each `arguments` is a hash, and each tool result `content` is a JSON
string. Once the turn ends, elelem keeps only your prompt and the final
reply in the history.

**`tools`** is an array in OpenAI's function format, also with symbol
keys:

```ruby
[{ type: "function",
   function: { name: "read", description: "Read file",
               parameters: { type: "object", properties: { path: { type: "string" } },
                             required: ["path"] } } }]
```

**Events** are hashes passed to the block. There are three:

| Event | Keys | What elelem does |
|---|---|---|
| `"saying"` | `text` | Appends `text` to the reply. |
| `"thinking"` | `text` | Prints `text` dimmed. Not kept. |
| `"doing"` | `id`, `name`, `arguments` | Records a tool call to run after `fetch` returns. |

`arguments` in a `"doing"` event must be a hash with string keys, which is
what `JSON.parse` gives you. Tools read `args["path"]`. The `id` comes back
to you as `tool_call_id` on the tool's result message.

If `fetch` raises, the agent catches it, prints the error, and ends the
turn with the reply `Error: <message>`.

## 1. Write the smallest provider

Before talking to a network, see the contract work. Create
`.agents/plugins/parrot.rb` in a project that has a `README.md`:

```ruby
module Parrot
  class Provider
    def fetch(messages, tools = [], &block)
      last = messages.last
      if last[:role] == "tool"
        block.call(type: "saying", text: "the tool said: #{last[:content]}")
      elsif (tool = tools.find { |t| t.dig(:function, :name) == "read" })
        block.call(type: "thinking", text: "I should read README.md first.")
        block.call(type: "doing", id: "parrot-1", name: tool.dig(:function, :name),
                                  arguments: { "path" => "README.md" })
      else
        block.call(type: "saying", text: "you said: #{last[:content]}")
      end
    end
  end
end

Elelem.configure do |config|
  config.provider(:parrot) { |**| Parrot::Provider.new }
end
```

`config.provider(:parrot)` registers a factory under the name you pass to
`-p`. elelem calls it once per session with the provider options from the
command line as keyword arguments. Today that is `model:` when you pass
`--model`, so accept `**` even if you ignore them.

```bash
elelem ask -p parrot "what is in the readme?"
```

````text
I should read README.md first.+ read({"path" => "README.md"})
# hello

```
the tool said: {"content":"# hello\n","path":"README.md","ok":true}
```
````

The first line is the `"thinking"` text (dimmed in a terminal) followed by
the tool call. `# hello` is the `read` tool showing you the file. `ask`
prints the reply in a code fence.

That was two model calls. The first emitted a `"doing"` event, elelem ran
`read`, and the second saw the `tool` message and replied.

## 2. Write a real provider

Create `.agents/plugins/compat.rb`:

```ruby
require "json"
require "net/http"
require "uri"

module Compat
  class Provider
    def initialize(base_url:, model:, api_key: nil)
      @uri = URI.join("#{base_url}/", "chat/completions")
      @model = model
      @api_key = api_key
    end

    def fetch(messages, tools = [], &block)
      body = { model: @model, messages: messages.map { |m| normalize(m) } }
      body[:tools] = tools unless tools.empty?
      message = post(body).dig("choices", 0, "message")

      text = message["content"].to_s
      block.call(type: "saying", text: text) unless text.empty?

      Array(message["tool_calls"]).each do |call|
        block.call(type: "doing",
                   id: call["id"],
                   name: call.dig("function", "name"),
                   arguments: JSON.parse(call.dig("function", "arguments") || "{}"))
      end
    end

    private

    def normalize(message)
      return message unless message[:tool_calls]

      message.merge(tool_calls: message[:tool_calls].map do |call|
        { id: call[:id], type: "function",
          function: { name: call[:name], arguments: JSON.generate(call[:arguments]) } }
      end)
    end

    def post(body)
      request = Net::HTTP::Post.new(@uri)
      request["content-type"] = "application/json"
      request["authorization"] = "Bearer #{@api_key}" if @api_key
      request.body = JSON.generate(body)
      response = Net::HTTP.start(@uri.host, @uri.port, use_ssl: @uri.scheme == "https", read_timeout: 600) do |http|
        http.request(request)
      end
      raise "HTTP #{response.code}: #{response.body}" unless response.is_a?(Net::HTTPSuccess)

      JSON.parse(response.body)
    end
  end
end

Elelem.configure do |config|
  config.provider(:compat) do |model: nil, **|
    Compat::Provider.new(
      base_url: ENV.fetch("COMPAT_BASE_URL", "http://localhost:8080/v1"),
      model: model || ENV.fetch("COMPAT_MODEL", "default"),
      api_key: ENV.fetch("COMPAT_API_KEY", nil)
    )
  end
end
```

Three things carry the weight:

- **The factory reads configuration.** elelem has no config file. Every
  provider gem reads environment variables in its factory and lets
  `--model` override the model. Follow the pattern so `-p compat --model x`
  works like the others.
- **`normalize` translates elelem's messages into the API's format.**
  elelem's tool calls are `{ id:, name:, arguments: Hash }`. OpenAI wants
  `{ id:, type: "function", function: { name:, arguments: String } }`. The
  `tool` messages already match OpenAI's shape, so they pass through. Every
  provider has a version of this method: `elelem-anthropic` turns tool
  results into `tool_result` blocks in a `user` message, and
  `elelem-ollama` replaces `tool_call_id` with `tool_name`.
- **The response becomes events.** Text becomes one `"saying"` event and
  each tool call becomes a `"doing"` event with parsed arguments.

Run it against your server:

```bash
COMPAT_BASE_URL=http://localhost:8080/v1 elelem chat -p compat
```

Inside a session, `/provider compat` switches to it, and `/provider` with
no argument lists the providers registered so far. A provider built by
`/provider` gets no options, so the factory falls back to `COMPAT_MODEL`.

## 3. Stream the reply

The example waits for the whole response, then emits one `"saying"` event.
Streaming means calling the block as chunks arrive, so the reply appears
as it is written. The agent does not care how many events you send.

The three published HTTP providers share one shape:

1. Send the request with `stream: true`.
2. Read the body with `response.read_body { |chunk| ... }`, buffer it, and
   split it into events: lines of `data: {...}` for OpenAI and Anthropic,
   one JSON object per line for Ollama.
3. Emit a `"saying"` or `"thinking"` event for each text delta as it
   arrives.
4. Collect tool calls as they stream. OpenAI and Anthropic send the
   arguments as JSON fragments, so append the fragments to a string and
   call `JSON.parse` only at the end. Then emit one `"doing"` event per
   call.

The providers in `elelem-openai` and `elelem-anthropic` are one short file
each and are the best reference.

## 4. Test it without a model

`fetch` is a plain method, so you can call it with a block that records
events:

```ruby
events = []
provider.fetch([{ role: "user", content: "hi" }], []) { |event| events << event }
```

The published providers take an `http:` keyword in their constructor so
specs can swap in a fake that returns canned responses. Do the same once
your provider grows past a sketch.

## 5. Package it as a gem

Name the gem `elelem-compat` and put the plugin at
`lib/elelem/compat.rb`, starting with `require "elelem"`. Provider gems
need no loader file: when you run `-p compat` and no provider of that name
is registered, elelem runs `require "elelem/compat"` and looks again. If
the gem is missing, elelem says so:

```text
elelem: unknown provider: "compat". Available: stub. Run: gem install elelem-compat
```

The gemspec is the same as for a [tool gem](/guides/write-a-tool/), with
`spec.add_dependency "elelem", "~> 0.13"`. To ship a shortcut executable
like `elelem-ollama`, add `exe/elelem-compat`:

```ruby
#!/usr/bin/env ruby
require "elelem/cli"

Signal.trap("INT") { exit 1 }

Elelem::CLI.new(ARGV, provider: "compat").run
```

Then set `spec.bindir = "exe"` and `spec.executables = ["elelem-compat"]`
in the gemspec.

## What you built

Two providers: `parrot`, which shows the event contract with no network,
and `compat`, which connects elelem to any OpenAI-compatible server,
including tool calls. You know the message and tool shapes elelem sends,
the three events it expects back, and how `-p NAME` finds a provider gem.

## Next steps

- [Compare it with elelem-evals](/guides/evals/) against the published
  providers.
- Read the source of [elelem-openai](/plugins/openai/),
  [elelem-anthropic](/plugins/anthropic/), and
  [elelem-ollama](/plugins/ollama/) for streaming implementations.
- The [plugin API reference](/docs/plugin-api/) covers the rest of
  `Elelem.configure`.


Canonical: https://elelem.sh/guides/write-a-provider/