Conduit

K.1

The shape

Conduit is an open-source framework for tuning, deploying, and building applications with open-source models. It abstracts infrastructure into composable, type-safe building blocks. You define models, schemas, and compute requirements in Python. Conduit handles provisioning, deployment, health management, and orchestration.

from conduit import ModelConfig, ComputeProvider
from conduit.runtime import LMLiteBlock

block = LMLiteBlock(
    models=[
        ModelConfig(
            "Qwen/Qwen3-4B",
            max_model_len=1400,
            max_model_concurrency=50,
            model_batch_execute_timeout_ms=1000,
        )
    ],
    compute_provider=ComputeProvider.RUNPOD,
    gpu=GPUS.NVIDIA_L4,
    replicas=2,
)

K.2

Runtime blocks

Pipelines are built from blocks. Each block conforms to Conduit's communication spec, so blocks chain.

K.3

Resource management

The developer declares intent. Conduit does the arithmetic.

Add as many models as needed. If the resources don't fit, Conduit throws an error.

K.4

State

K.5

LM Lite runtime

The inference runtime inside Conduit: a multi-model batching engine.

ModelConfig(
    model_id,                             # HuggingFace model ID
    max_model_len=1400,                   # max tokens per request
    max_model_concurrency=50,             # max concurrent requests per batch
    model_batch_execute_timeout_ms=1000,  # batch timeout
)

K.6

Model Data Language

MDL is the type system. Schemas are Python dataclasses. Conduit compiles them to model-safe type hints, validates payloads on request and response, and guarantees structured output.

from dataclasses import dataclass
from typing import List

@dataclass
class Input:
    name: str
    context: str

@dataclass
class Output:
    response: str
    confidence: float
    tags: List[str]

result = block(
    model_id="Qwen/Qwen3-4B",
    input=Input(name="query", context="..."),
    output=Output,
)
# result is a guaranteed valid Output instance

K.7

Compute abstraction

Every compute provider is an interchangeable container endpoint. The same code runs on any supported provider. Switching providers is a config change, not an application change, and providers can mix within one pipeline.

compute_provider=ComputeProvider.RUNPOD  # or AWS, GCP, LOCAL, ...

Optimized for the Covenant Secure Compute Cloud (2026): private model deployment on controlled infrastructure, end-to-end sovereignty guarantees, cryptographic attestation of execution.

K.8

Lifecycle

block = LMLiteBlock(models=[...], compute_provider=..., replicas=2)

LMLiteBlock.gc()        # clean stale instances after config changes

while not block.ready:
    time.sleep(5)       # safe to send traffic once ready

block.delete()          # destroy the runtime and backing resources

K.9

Interface

Two ways in. OpenAI-style messages for compatibility, typed MDL for guarantees.

result = block(
    model_id="Qwen/Qwen3-4B",
    messages=[{"role": "user", "content": "Hello"}],
    system_message="You are a helpful assistant",
)

result = block(
    model_id="Qwen/Qwen3-4B",
    input=TypedInput(...),
    output=TypedOutput,
)

K.10

Roadmap

Requires Python 3.12.6+. Distributed as a Python package, installed from source.

• [ ]vLLM runtime support (VllmBlock)
• [ ]Local compute provider for on-device execution
• [ ]AWS (EKS / ECS / EC2)
• [ ]GCP (GKE / Vertex / Compute Engine)
• [ ]Azure (AKS / Azure ML)
• [ ]Lambda Labs, Modal, CoreWeave
• [ ]Covenant Secure Compute Cloud integration

PUBLISHED AS OUTPUT 003