Models

uRun has no model class hierarchy to extend. A model is something your code loads inside a @app.function — the same from_pretrained call you'd write anywhere, wrapped in the store's loading primitives so weights download, cache, and stay GPU-resident across calls. @app.function is the surface; the store is how weights get into it.

The one convenience on top: for a model the catalog already covers, urun.serve("<id:variant>") deploys it without you writing any function at all.

The catalog shortcut: urun.model / urun.serve

urun.model() and urun.serve() are the Python front door to the model catalog. urun.model("<id:variant>") resolves a catalog row to a handle you can inspect; urun.serve(x) takes a str | Model and deploys it through the same serve pipeline as the urun serve CLI.

import urun

# Serve by name — sugar for urun.serve(urun.model(name))
urun.serve("qwen-coder:fp8", gpu="l4:1")

# Or resolve first, inspect, then serve
model = urun.model("glm-5.2:UD-IQ2_M")
print(model.engine, model.size_gb)   # "llamacpp", 239
urun.serve(model, gpu="l4:1")

A bare <id> picks the model's default variant; gpu= selects a placement (default = the variant's first placement). That's the whole API: an id string in, a served model out. What runs underneath is a normal uRun app the platform authored for that row — not a special runtime.

When the catalog doesn't have your model, write the app yourself with the loading primitives below — no catalog extension or base class involved.

Loading models in your own app

store.model — one-call HuggingFace load

store.model(url, ...) handles the full lifecycle: download (with auth from Credentials), auto-detect the model class from model_index.json (diffusers) or config.json (transformers), load via from_pretrained, and place on device. Extra kwargs are forwarded to cls.from_pretrained().

import torch
from urun import store

# Auto-detected pipeline from model_index.json
pipe = store.model("hf://stabilityai/stable-diffusion-xl-base-1.0",
                   dtype=torch.float16, device="cuda")

# Explicit class for a sub-component
encoder = store.model("hf://Wan-AI/Wan2.1-T2V-14B",
                      cls=UMT5EncoderModel,
                      subfolder="text_encoder",
                      dtype=torch.bfloat16, device="cuda")

# Large model with accelerate auto-sharding
llm = store.model("hf://meta-llama/Llama-2-70b", device_map="auto")

# Gated model — HF_TOKEN from Credentials handles auth
flux = store.model("hf://black-forest-labs/FLUX.1-Redux-dev")

Key kwargs: cls= (skip auto-detection — needed for research models without standard metadata), subfolder=, device= / device_map= (accelerate sharding), dtype=, allow_patterns= / ignore_patterns= (restrict what downloads).

store.cache — load once, stay resident

Model loading is expensive; sessions are not. Wrap the loader in @store.cache so the first call loads and every later call returns the same object — GPU tensors pinned in GPU memory between calls.

from urun import App, store

app = App("my/imagegen")

@store.cache(prefix="imagegen_pipe", process_local=True, gpu_pin=True)
def load_pipe():
    import torch
    return store.model("hf://stabilityai/stable-diffusion-xl-base-1.0",
                       dtype=torch.float16, device="cuda")

@app.function(gpus="l40s:1")
def generate(prompt: str):
    pipe = load_pipe()          # first call loads; after that, instant
    return pipe(prompt).images[0]

Use process_local=True, gpu_pin=True for hot non-serializable runtime objects like pipelines — the cached value stays inside the worker process and survives warm restarts. This is the pattern every example app uses.

store.repo — models that live in a research repo

Plenty of models aren't a clean from_pretrained away — the architecture lives in a GitHub repo with no package. store.repo pins that repo at a SHA and makes it importable, scoped: inside the with block the repo's modules resolve with normal import syntax; on exit the path entry is removed. No vendoring, no fork.

from urun import App, Dependencies, store

REPO = store.repo("github.com/org/research-model@<commit-sha>")

app = App("my/research")

@store.cache(prefix="research_pipe", process_local=True, gpu_pin=True)
def load_pipe():
    with REPO:                              # importable ONLY inside the block
        from research_model import Pipeline
        return Pipeline.from_pretrained(
            store.path("hf://org/research-model-weights"))

@app.function(gpus="h100:1", deps=Dependencies(python=[*REPO.deps()]))
def generate(prompt: str):
    return load_pipe()(prompt)

REPO.deps() splices the upstream's own dependency list into your function's deps — no hand-kept requirements copy. For patched upstreams, diff= / overlay= / patches= apply your changes at materialization (loud-fail if a hunk doesn't apply); subdir= re-roots monorepo releases. See De-vendoring for the full workflow.

Accelerating a model: urun.compile

The store gets weights loaded; urun.compile shapes how the model runs: one call that transparently accelerates a torch-native model (a bare nn.Module or a whole diffusers pipeline), in place, returning the same object.

import urun

pipe = load_pipe()
urun.compile(pipe)          # fused kernels + torch.compile, in place — same object back

Two phases, both degrade-safe (a failed phase logs once and falls through — the model always keeps working):

1. Kernel swap — structure-detected fused kernels from pluggable providers. Detection is structural (module class shape), never by app or model name; only numerically-safe module classes are swapped by default.
2. torch.compile — applied through the module's own in-place mechanism (nn.Module.compile), so object identity and every isinstance/attribute contract downstream survive. For a diffusers pipeline the compile target is the denoiser (transformer/unet) — the measured hot path.

Key kwargs: mode= (the torch.compile mode; "default" is the platform default, None skips torch.compile), kernels=False (skip the provider phase), dynamic=/fullgraph= (forwarded to torch.compile), regional=True (diffusers' compile_repeated_blocks escape hatch when full-graph tracing fails).

What it actually did is attached to the model as an AccelReport (model.__urun_accel__): which providers swapped which kernels, whether torch.compile applied, the mode, and any degrade notes. urun.compile never raises for an acceleration-shaped failure. It only wraps — dynamo compiles lazily on first call, so your ctx.self_test() pass is the compile warmup.

Boundaries

• There is no user-facing model class hierarchy. You don't subclass a Model to bring your own model — you write a @app.function that loads it.
• urun.serve(name) == urun.serve(urun.model(name)); both deploy via the same serve pipeline as the CLI, and only for catalog rows.
• Loading composes from three store primitives: store.model (HF one-call load), store.cache (residency), store.repo (SHA-pinned research code). They're generic — nothing is special-cased per model.
• urun.compile accelerates in place and never breaks the model — degrade-safe by contract.

Related:

• App anatomy — the @app.function shape the loaded model lives in
• De-vendoring — the full store.repo workflow for upstream research code
• Streaming Pipelines — paced media flow and the pipeline DSL