Use your own embedder

Remem 0.5.1 and later can use embeddings computed by your application. This is available in both Community and Business editions. Your model can run locally or behind an API: your application calls it, then sends the resulting vectors to Remem.

With embedding.provider = "external", Remem does not generate embeddings or call an embedding service. It stores your vectors, maintains the search index, and uses them for semantic search and relationship discovery. The default onnx provider still runs the built-in model inside Remem.

1. Configure the server

Choose a model identity and the dimension your embedder returns. For example, for a model that produces 768 components, create external-embeddings.toml:

[embedding]
provider = "external"
external_model_id = "my-embedder-v1"

[vector]
dimension = 768
metric = "cosine"

Replace my-embedder-v1 with an identity for your model and preprocessing settings. Use exactly the same embedding setup for stored memories and queries, including any document/query prefixes your model requires. The identity is recorded with stored vectors; Remem cannot check which model actually generated a vector you send.

The dimension must be between 1 and 4096. Every vector must have that exact number of finite numeric components. With cosine similarity, a vector must also be nonzero. Remem validates vectors and refuses invalid ones rather than padding, truncating, or replacing them. It does not rescale supplied vectors.

Do not use the reserved built-in identity all-MiniLM-L6-v2 as external_model_id, or try to select an external model by changing embedding.model. The external provider uses external_model_id and vector.dimension instead.

Start a server with a new data directory:

# Set REMEM_SERVER_API_KEY to a long random secret before running this.
remem start --config ./external-embeddings.toml \
  --storage.path ./data-external \
  --server.env production

For a checkout using Docker Compose, put the same settings in a compose.external.yml override:

services:
  remem:
    environment:
      REMEM_EMBEDDING_PROVIDER: external
      REMEM_EMBEDDING_EXTERNAL_MODEL_ID: my-embedder-v1
      REMEM_VECTOR_DIMENSION: "768"
      REMEM_VECTOR_METRIC: cosine

Then start it with a fresh, writable data directory and your server credential:

mkdir -p ./remem-data-external
sudo chown 10001:10001 ./remem-data-external
REMEM_DATA_DIR=./remem-data-external \
  docker compose -f docker-compose.yml -f compose.external.yml up --build -d

See Docker Compose for the other credentials required by the Business stack. External mode needs no local model files; the supported server image still contains its bundled model.

2. Embed content and send the vector

Your application must generate the vector before creating the memory. Send the original text as content and the vector as embedding to POST /api/v1/memories. Both are required: the text supports keyword search and remains the content returned to your agent.

The Python example below uses only the standard library for HTTP. Supply an embed(text) function backed by your model’s library or embedding API client; it must return a list of numbers, with 768 components for the configuration above. Replace that function when switching providers; the Remem requests stay the same.

import json
import os
from urllib.request import Request, urlopen

BASE_URL = "http://localhost:4545"
API_KEY = os.environ["REMEM_SERVER_API_KEY"]


def request(method, path, body):
    req = Request(
        BASE_URL + path,
        data=json.dumps(body, allow_nan=False).encode(),
        headers={
            "Authorization": f"Bearer {API_KEY}",
            "Content-Type": "application/json",
        },
        method=method,
    )
    with urlopen(req, timeout=30) as response:
        return json.load(response)


def store_memory(content, embed):
    return request("POST", "/api/v1/memories", {
        "content": content,
        "embedding": embed(content),
        "policy": "long_term",
    })


def search_memories(query, embed, search_type="semantic"):
    return request("POST", "/api/v1/memories/search", {
        "query": query,
        "query_vector": embed(query),
        "search_type": search_type,
        "limit": 5,
    })

Call store_memory("The customer prefers email updates.", embed) with your embedding function. To find it, call search_memories("How should we contact the customer?", embed). For hybrid search, pass search_type="hybrid"; it uses both the query text and your vector. If your model has different document and query encoding methods, pass the document function to store_memory and the query function to search_memories.

Semantic search can also use query_vector without query text. Read matches from the response’s results array; each entry contains memory and score.

3. Keep vectors consistent on every request

Operation	What to send in external mode
Create a memory	content and embedding
Batch create (POST /api/v1/memories:batch)	An embedding for every memory in the batch
Update with a content field (PATCH /api/v1/memories/{id})	The new content and its embedding, even if the text is unchanged
Update only tags or lifecycle fields	Omit content and embedding
Semantic search	search_type: "semantic" and query_vector; query text is optional
Hybrid search	search_type: "hybrid", query, and query_vector
Keyword search	search_type: "keyword" and query; omit query_vector

A vector-only update is refused: an embedding must travel with the content it describes. A server using the default onnx provider also refuses caller-supplied embedding and query_vector fields.

Budgeted recall through POST /api/v1/memories/recall accepts query_vector for semantic or hybrid retrieval too. Send context and token_budget with it. Use search_type: "semantic" or "hybrid" to retrieve by your vector, or "keyword" without a vector for text retrieval.

MCP and the Business console

The current MCP tools have no vector arguments. On an external-provider server, MCP cannot store memories, update their content, or run semantic/hybrid search or recall. Keyword retrieval, listing, fetching, and updates that leave content alone remain available. Use the REST API from an application that supplies vectors for operations that need embeddings.

The Business console does not compute embeddings for you either. Its content-writing and semantic/hybrid search workflows need an integration that supplies vectors; setting the server to external mode does not add an embedding provider to the console.

Changing models and importing data

Keep one embedding model and dimension for a corpus. Do not point a new model at a directory containing vectors from another one. Remem refuses incompatible stored model identities; giving two models the same identity would bypass that check without making their vectors comparable.

To change models, create a fresh data directory, compute new embeddings for the memory content with the new model, and store those memories through the REST API. Simply changing the configuration or running a vector index rebuild does not convert old embeddings: a rebuild uses the canonical vectors already stored on disk.

Snapshot import into external mode requires vectors with the same model identity and dimension as the destination. It cannot re-embed a snapshot from a different model, because the external provider has no embedding function. Plan a content re-ingestion workflow when changing models; creating memories again gives them new IDs and does not preserve the old graph or history.

Relationship discovery uses your stored vectors. Its default cosine threshold of 0.7 was chosen for the built-in model; check whether it creates useful links for your model before relying on it, or disable discovery with REMEM_DISCOVERY_ENABLED=false.