Sense reference
Reference for Sense, uRun's perception model
This page is the reference for the Sense model. For what Sense is and why you would use it, see the Sense overview.
You can connect to the Sense Model API through a request for async work or a Session for realtime work. How you call the model depends on your connection type.
- In a request, you send one JSON body with the content and questions to assess, and get back one JSON object with the answers.
- In a Session, you use the Sense SDK: you declare a standing set of questions, a cadence, and an initial state, then stream multimedia inputs which Sense evaluates continuously. The SDK yields the answers as waves — one complete answer set per evaluation tick.
Connection
To use the Sense Model API in a request/response pattern for batch or async work, call it with your API key through the URL uRun exposes for TypeSafe Jev:
- TypeSafe Jev -
https://inference.urun.sh/v1/systemone— takes typedstate,questions, and optionalmediain; emitsanswersas a JSON object.
Inputs
| Field | Required | Type | Description |
|---|---|---|---|
state | yes | string or JSON | The content to evaluate. |
media | no | array | Audio, image, and video items attached to the state. |
questions | yes | object | What you want to assess, keyed by the question name you choose. |
-
The
stateis the text your questions judge. It can be a string, a JSON object, or a JSON array.{ "state": "My card was charged twice for the same order.", "questions": { "refund_requested": { "type": "noul", "instructions": "Is the person asking for a refund?" } } }{ "state": { "ticket": "I was charged twice for order A-104.", "order": { "id": "A-104", "amount_usd": 49 }, "refund_policy": "Duplicate charges are eligible for a refund." }, "questions": { "refund_requested": { "type": "noul", "instructions": "Does `ticket` request a refund, and does `refund_policy` allow one for `order`?" } } }{ "state": [ { "from": "person", "text": "I was charged twice for order A-104." }, { "from": "support", "text": "We're checking the charges now." } ], "questions": { "refund_requested": { "type": "noul", "instructions": "Does `[0].text` request a refund?" } } } -
Attach audio, images, and video with
media, which takes an array of up to 32 items totaling up to 20 MB combined when decoded.Each item in the array is an object with the following properties:
Property Required Type Description typeyes string audio,image, orvideo.datayes string Base64-encoded bytes, or a data URL ( data:<mime>;base64,<bytes>).mimeonly when the payload doesn't identify itself string audio/wav,audio/x-wav,audio/wave,audio/mpeg,audio/flac,audio/ogg,image/png,image/jpeg,image/webp,video/mp4,video/webm, orvideo/quicktime. Omit whendatais a data URL (its mime is used) or the bytes are a recognized container (sniffed).start_s,end_sno number Window the media — for example the last 8 seconds of a longer clip. { "state": "A recording of a support call, with a screenshot of the order page.", "media": [ { "type": "audio", "data": "<base64-encoded audio>", "mime": "audio/wav" }, { "type": "image", "data": "<base64-encoded image>", "mime": "image/png" }, { "type": "video", "data": "<base64-encoded video>", "mime": "video/mp4" } ], "questions": { "refund_requested": { "type": "noul", "instructions": "Is the person asking for a refund?" } } }stateis still required when you sendmedia. Use it for context that is not in the media itself, such as who is speaking.
Question types
A question can be a choice, score, or noul. Questions are defined as objects and different types take different properties.
| Property | Applies to | Required | Description |
|---|---|---|---|
type | All types | No — defaults to choice | The question kind: choice, noul, or score. |
instructions | All types | Yes | The question itself, in plain words. Describe the judgment to make. |
criteria | choice, score | Yes — optional for noul | The answers to choose among. For noul, optional { "true": "...", "false": "..." } descriptions. |
-
A
choicequestion asks "which option fits best?".criteriais a map of up to 255 option IDs to descriptions."questions": { "route": { "type": "choice", "instructions": "Which team should handle this message?", "criteria": { "billing": "Payment or refund issues", "technical": "Bugs or integration problems", "sales": "Pricing or account questions" } } } -
A
noulquestion asks "is this true?"criteriais optional and describes what true and false mean. Give every option a description:"questions": { "finished_speaking": { "type": "noul", "instructions": "Has the person finished their turn?", "criteria": { "true": "the person yielded the turn", "false": "the person is only pausing" } } } -
A
scorequestion asks "where does this fall on the defined spectrum?"criteriais an ordered array of 2 to 10 level descriptions, low end first. Each entry is a plain description string; the level's index is its id."questions": { "tone": { "type": "score", "instructions": "How does the person speaking sound right now?", "criteria": [ "well structured thoughts", "pausing a lot", "repeating themselves", "using strong language" ] } }Describe situations in your levels, not degrees. Defining
"angry"as"using strong language"is more specific and therefore better than defining"angry"as"very upset".
Outputs
Different question types return different fields.
| Question type | Returned fields |
|---|---|
choice | choice (the selected option), probabilities (per option), confidence |
noul | noul (probability of yes) |
score | score (weighted level), legend, probabilities (per level), confidence |
Option probabilities are softmax-normalized across the question's options (or levels) and sum to 1 up to floating-point rounding.
Each request with questions returns one JSON object with answers for all the questions. The response also carries usage, the token counts for the call.
"answers": {
"route": {
"type": "choice",
"choice": "billing",
"probabilities": { "billing": 0.9, "technical": 0.1 },
"confidence": 0.8
},
"wants_refund": {
"type": "noul",
"noul": 0.97
},
"urgency": {
"type": "score",
"score": 0.85,
"legend": {
"0": "Low",
"1": "Medium",
"2": "High"
},
"probabilities": { "0": 0.35, "1": 0.45, "2": 0.2 },
"confidence": 0.175
}
}Confidence
choice and score answers return a confidence from 0 to 1.
- For
choice,confidencemeasures how concentrated the answer's probability mass is:1means all probability is on one option.0means the probabilities are spread evenly across all options.
- For
score,confidencemeasures how tightly the probability mass clusters around the most-likely level, counting nearer levels as less of a spread than distant ones:1means all probability is on one level.0means the probabilities are spread evenly across all levels or split between far-apart levels.
In both cases, confidence is a property of the distribution you got back — not a probability that the answer is correct.
Use confidence in your application code to gate how much you trust a decision:
- High — act on it automatically.
- Middle — confirm with a person or gather more state before acting.
- Low — do not act on it.
Limits
Calls that exceed a limit are rejected with an error that notes the limit.
| Field(s) | Limit |
|---|---|
state plus all questions, combined | 524,288 characters |
Options per choice question | 2 to 255 |
Levels per score question | 2 to 10 |
media items | 32 |
media bytes, decoded | 20 MB |
| Context window | 32,768 tokens |
Complete example
Encode media files and define your questions
AUDIO_B64=$(base64 < clip.wav | tr -d '\n')
cat > request.json <<EOF
{
"state": "Audio of a person speaking.",
"media": [
{ "type": "audio", "data": "$AUDIO_B64", "mime": "audio/wav" }
],
"questions": {
"team": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "Payments, refunds",
"technical": "Bugs, outages",
"sales": "Pricing, upgrades"
}
},
"refund_warranted": {
"type": "noul",
"instructions": "Does the duplicate charge qualify for a refund?",
"criteria": {
"true": "the duplicate charge qualifies for a refund under the policy",
"false": "it does not qualify"
}
},
"urgency": {
"type": "score",
"instructions": "How urgent is this request?",
"criteria": [
"can wait",
"should be handled today",
"the person is blocked"
]
}
}
}
EOFSend the content and questions to the Sense Model API.
curl https://inference.urun.sh/v1/systemone \
-H "Authorization: Bearer urun_sk_YOUR_KEY" \
-H "Content-Type: application/json" \
--data-binary @request.jsonThe answer:
{
"model": "sense-latest",
"answers": {
"team": {
"type": "choice",
"choice": "billing",
"probabilities": {
"billing": 0.991048276424408,
"technical": 0.005195490550249815,
"sales": 0.0037562334910035133
},
"confidence": 0.9865724139443727
},
"refund_warranted": {
"type": "noul",
"noul": 0.981
},
"urgency": {
"type": "score",
"score": 1.5,
"legend": { "0": "can wait", "1": "should be handled today", "2": "the person is blocked" },
"probabilities": { "0": 0.1, "1": 0.3, "2": 0.6 },
"confidence": 0.52
}
}
}Was this page helpful?