Suggest edit — attention-gating

Title

Name

Note

---
visibility: public
---

# Attention gating

## Why

In a group chat with four humans and a bot, the bot should not respond every 3 seconds. But it should respond when the conversation is about a bug it can help with, even if no one @mentioned it. That's the "discriminator" lane.

## The ladder

```
 silent ──▶ discriminate-quiet ──▶ discriminate ──▶ mentions-only ──▶ always
     (record)          (LLM)           (LLM)          (strict match)    (no filter)
```

Going right = more responsive. Going left = quieter.

## Rule before LLM

The LLM discriminator is slow and costs money. Hard rules short-circuit:

1. `/picortex <cmd>` slash command → always respond
2. Reply-to a bot message → respond
3. `@picortex` mention → respond
4. Admin override from Jacob's DM → respond
5. Non-text payload → mode-dependent

## Discriminator prompt

Lives at `$CHAT_HOME/.picortex/prompts/discriminator.md`, git-tracked inside the chat's home. Seeded from a default that can be tuned live.

Default (sketched):

> You are deciding whether a chat bot should respond to the most recent message. Only respond if the message:
> - asks a question that a capable coding / research assistant could help with, OR
> - is reacting to a previous message from you, OR
> - contains a task/request that the assistant should pick up.
>
> Do NOT respond if the message is:
> - small talk between humans
> - private conversation you weren't addressed in
> - off-topic or tangential
>
> Return JSON: `{"should_respond": bool, "reason": "short"}`.

Prompt is git-versioned so Jacob can tune per-chat without redeploying.

## Model choice

Default: `claude-3-5-haiku` (cheap, fast). Swap via `chat_config.discriminator_model`.

Cost: ~200 input tokens + 30 output tokens per classification ≈ $0.0005. 1,000 classifications = $0.50. Cheap.

## Failure modes

- LLM returns non-JSON → 1 retry with reformat prompt; if still bad, fail-open for 1:1, fail-closed for groups.
- LLM times out (> 5 s) → same fail policy.
- Cost spike → per-day per-chat budget cap in `chat_config`. Hitting the cap disables `discriminate` until midnight.

## Observability

Every discriminator decision is logged as an `event_type: "discriminator.decision"` with: `chat_id`, `message_id`, `should_respond`, `reason`, `model`, `latency_ms`, `cost_usd`.

## Future

- Personalized discriminator (fine-tune or few-shot on Jacob's own "respond / don't respond" label history). v0.3.
- Hierarchical gating: per-user overrides inside a group. v0.2.

## References

- [Spec 005](../specs/005-attention-gating.md)
- Cortex R4 — the pattern we adopted
- Default template: `/usr/local/share/picortex/discriminator.default.md` (created by installer in S4)