poq.md examples
Plain-language prompts in poq.md draft into poq.toml. Examples use a Beatles catalog review theme.
Data & ingestion
One datapoint per CSV row
csv-row
Track annotation CSV
Ingest tracks.csv with one row per datapoint.
Columns: track_id, title, lyric_excerpt, proposed_genre.
One datapoint per JSON file
Use songs/*.json and dotted paths for nested fields.
json-glob
Song JSON glob
Ingest songs/*.json — one JSON file per datapoint.
Map nested fields with dotted paths: metadata.writer, rubric.proposedMood.
Image + CSV join
Join track metadata to album cover files on basename.
image-join
Album cover join
Join annotations.csv to covers/* on album_id ↔ file basename.
Each datapoint shows the cover image plus CSV metadata (album, caption).
Markdown report split
Split liner notes into one datapoint per track heading.
markdown-split
Liner notes split
Split notes/*.md on headings matching ^## TRK-\d+:.
Extract album metadata (title, release year) and per-track fields (writer, key) with regex.
Rubric & scoring
Multi-dimensional rubric
Mix Likert, ordinal, and numeric scales on one track.
multi-rubric
Track rubric
Score each annotation on four dimensions with mixed scale types:
- Validity — 7-point Likert (
influence_gauge): does the genre tag fit? - Impact — ordinal: deep cut → iconic
- Arrangement — ordinal: weak → definitive
- Confidence — numeric 0–100 (
certainty, advisory)
Weighted consensus
Genre fit counts twice; confidence is advisory.
weighted-consensus
Weighted consensus
Genre fit consensus_weight = 2.0. Confidence weight = 0 (advisory only).
Conditional rubric rows
Skip Arrangement in the validator UI when Validity is False positive (consensus_skip.match; filler votes still count toward consensus).
conditional-rubric
Conditional rubric
Skip Arrangement in the validator UI when Validity is False positive (consensus_skip.match; filler votes still count toward consensus).
Validation UI
Image and text evidence
Cover image first, then markdown caption and album label.
image-text
Cover + liner note evidence
Validators see the album cover image first, then the markdown caption below with the album label.
Evidence order: image (cover path), then markdown (caption).
Pinned source link
Link to anthology edition, track file, and lyric line.
source-link
Lyric source link
Show source_link to the anthology at repository + edition, track sourcePath, line lineNumber.
Structured metadata
Song name, album, and writer in a facts panel.
metadata-facts
Track metadata panel
Show song name, album, and writer in a datapoint_facts sidebar.
Ground truth, prefill & difficulty
Hidden golden labels
label is ground truth; validators never see it.
golden-labels
Genre label QC
label is ground truth — validators never see it.
Golden label field: label.
Prefilled hints
Show label_shown; validators judge agreement.
prefill-hints
Prefill disagreement probes
Show label_shown as the model's proposed genre. Validators rate agreement.
Prefill hint: label_shown. Golden label: label (hidden).
Difficulty metadata
Numeric difficulty for routing; stripped from validator UI.
difficulty
Difficulty routing
difficulty (1–5) is metadata for analytics/routing. Strip from validator UI.
Difficulty field: difficulty.
Validators
Validator classes
Junior vs Senior music critics with different pay and priority.
validator-classes
Beatles catalog review
Add Junior and Senior critic classes with different reward_usd and priority.
Flagship albums (tier = iconic): larger panel with fixed senior share plus wildcard slots.
Attribute-based routing
Iconic tier → 5 critics (2 senior); default → 3.
attribute-routing
Tier-based routing
Match on proposed_tier: iconic → 5 validators (2 senior, 3 wildcard). Default → 3.
Wildcard composition
Fill remaining slots with class = "*" at claim time.
wildcard
Wildcard composition
Route composition uses class = "*" so any qualified critic can fill remaining slots at claim time.
Senior count is fixed; wildcards fill the rest.
Escalation ladder
Add senior critic waves until verified.
escalation
Escalation ladder
If the first round does not verify, add 2 senior critics, then another senior wave.
Ordered [[validators.routes.escalation]] steps — no per-step predicate.
Class-specific rewards and stakes
Different reward_usd / stake_usd per critic class.
class-pay
Critic pay and stake
Junior critic: reward_usd = "5.00", stake_usd = "0.00".
Senior critic: reward_usd = "20.00", stake_usd = "5.00".
Qualification gates
Require publication and musicology credential before claim.
qualification
Critic qualification
Require before claim: publication (required), musicology_credential (required), years_listening (optional).
Claim time limits
30 minutes per claim.
claim-ttl
Claim TTL
Critics have 30 minutes per claim ([validators.claims] duration_minutes = 30).
Conflict-of-interest self-decline
Release a claim without penalty when COI applies.
coi-decline
Conflict-of-interest decline
Allow critics to release an active claim without penalty when they reviewed the same album for publication.
[validators.actions] allow_conflict_of_interest_self_decline = true
AI-assisted review
AI validator panel
Three AI model classes review each track before humans.
ai-panel
AI critic panel
Run three AI classes per track (melody-model, lyrics-model, era-model) before humans.
Each class: type = "ai", model, and review prompt.
AI-first with human escalation
AI panel first; contested items escalate to senior critics.
ai-escalation
AI-first with human escalation
Initial route: 3 AI critics. Escalation: +1 senior human, then +2 senior humans if still unverified.
AI-safe scale constraints
Integer-friendly scales when AI critics are present.
ai-scales
AI-safe scales
When AI critics are used, every rubric scale must use integer-friendly anchors:
- Likert:
scale.size = 5only - Ordinal: 5 labels (or any count where
(n-1)divides 100) - Numeric: integer
scale.valuesonly