Validation Interface
The interface shown to a validator renders data from the ingestion phase according to transformations in the project's specification (poq.toml). This page describes the evidence and evaluation elements that can be defined via the project's specification.
What Sapien can show a validator
| What it looks like | What it shows |
|---|---|
| A page of text | Formatted writing (Markdown) |
| A picture | One image |
| A snippet of code | A code block |
| A formatted fact sheet | A formatted JSON object |
| A table of facts | A list of labeled facts, side by side |
| A quoted excerpt | Text pulled from a source file |
| A link | A pointer to an exact file, commit, and line |
| Two pictures at once | Shown side by side, as a slider, or as a difference view |
| A picture with boxes drawn on it | Labeled boxes drawn over the important parts |
| A video | With an optional cover image and captions |
| A question-and-answer pair | A prompt and the answer it received, sometimes with a second answer for comparison |
For most of these, the project specification names exactly what to show, for example a particular picture or a particular fact, and Sapien checks that everything required has been provided before anyone sees it. If something is missing when it is time to show a validator, they see a message stating the evidence is missing, instead of a blank space.
How a validator answers
There is one kind of answer control: a row of buttons, and the validator picks one. There is no typing, no sliders, and no selecting multiple options. Only one button can be picked.
What changes is how many buttons there are and what they say.
- A 5-point or 7-point scale, for example from "Strongly Disagree" to "Strongly Agree", evenly spaced.
- A list of custom labels, for example "Low", "Medium", "High", also spaced evenly.
- A list of custom numbers, as long as it includes the lowest and highest possible score.
Regardless of which type is used, every answer is converted to a number from 0 to 100. The buttons are a way of picking a position on that scale. This is what allows Sapien to compare answers to each other and determine whether reviewers agree, even when the questions look different on screen.
A few rules keep this working.
- Every question needs at least two buttons to choose from.
- If an AI is one of the reviewers, the numbers behind the buttons must be whole numbers, with no fractions. A 5-choice scale always satisfies this, but some other combinations do not.
A question can also be skipped automatically. If the answer is already determined by the data, Sapien fills it in and marks it as skipped instead of asking the validator.
Continue
| Page | Description |
|---|---|
| PoQ Artifacts | How poq.md and poq.toml work together |
| How consensus works | How 0-100 rubric answers become a verdict |
poq.toml Specification | Field-level reference for [[validation.evidence]] and [[validation.rubric]] |