Spec / open-values-challenge-index
Open Values Challenge Index v0.1
A compact version of the challenge set for sites to read: each worked example with its top picks recomputed from the current category data and the chosen sourced facts filled in. pipeline/build_challenge.js builds it from content/challenge/index.json into app/data/challenge-index.json; change the challenge set, not this file.
- Files built from other files
- Status: draft
- Version: 0.1.0
- Format name in the file:
open-values-challenge-index
Draft. It may still change before it is called stable. A file a build step makes from other files. Change the sources, not this file.
Who writes it and who reads it
Written by pipeline/build_challenge.js. Read by future challenge surface, Kosplora configuration, Socrates Colosseum format seed.
Fields
| Field | What it means | Type |
|---|---|---|
format required | Always "open-values-challenge-index". Tells a reader which Open Values format this is. | always "open-values-challenge-index" |
version required | The version of the index format this file follows. Must start with 0.1.0. | string |
standard required | Always "open-values-standard", the standard this format belongs to. | always "open-values-standard" |
built required | The date of the challenge set this index was built from, copied from that file's updated field, as YYYY-MM-DD. | string |
purpose required | A sentence saying what this index is for. | see nonEmptyString |
generatedFrom required | The challenge set file this index was built from, its date, and its hash, so anyone can check the index against it. | see generatedFrom |
counts required | How many sites, methods, cases, contrary tags and categories the challenge set holds. | see counts |
principles required | Copied unchanged from the challenge set: the rules challenge features follow. | list |
algorithms required | The methods from the challenge set, with only their id, label and summary. | list |
instances required | The sites and apps from the challenge set, copied with their settings. | list |
categories required | Every category that has at least one case, sorted by id, with its criteria. | list |
cases required | Every case from the challenge set, with each view reranked against the current category data and the chosen sourced facts filled in. | list |
A reader MUST ignore fields it does not know, so a file can carry more than this list and still be read. A writer MUST NOT change what a listed field means.
A file that passes
Challenge indexes expose generated instance defaults, category summaries, replayed top picks, and selected sourced contrary tags.
{
"format": "open-values-challenge-index",
"version": "0.1.0",
"standard": "open-values-standard",
"built": "2026-07-07",
"purpose": "Generated compact challenge index.",
"generatedFrom": {
"path": "content/challenge/index.json",
"updated": "2026-07-07",
"sha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
},
"counts": {
"instances": 3,
"algorithms": 2,
"cases": 1,
"contraryTags": 2,
"categories": 1
},
"principles": [
"calm challenge",
"sourced facts"
],
"algorithms": [
{
"id": "opposite-view",
"label": "Opposite view",
"summary": "Rank under opposite weights."
},
{
"id": "steelman",
"label": "Steelman",
"summary": "Surface a sourced contrary fact."
}
],
"instances": [
{
"id": "conscious-consuming",
"label": "Conscious Consuming",
"role": "reference",
"challengeDefault": "available",
"surfaces": [
"verdict"
]
},
{
"id": "kosplora",
"label": "Kosplora",
"role": "learning",
"challengeDefault": "prominent",
"surfaces": [
"resource"
]
},
{
"id": "socrates-colosseum",
"label": "Socrates Colosseum",
"role": "dissent",
"challengeDefault": "primary",
"surfaces": [
"position"
]
}
],
"categories": [
{
"id": "ai-assistants",
"label": "AI assistants",
"type": "Services",
"file": "app/data/ai-assistants.json",
"n": 2,
"criteria": [
{
"key": "openness",
"label": "Openness"
},
{
"key": "safety",
"label": "Safety"
}
]
}
],
"cases": [
{
"id": "sample-challenge",
"category": "ai-assistants",
"label": "Sample challenge",
"reads": "The strongest case the other way: ChatGPT gives managed safety receipts.",
"userView": {
"label": "Open first",
"weights": {
"openness": 5,
"safety": 1
},
"expectedTop": "local-open",
"top": {
"code": "local-open",
"name": "Local Open",
"score": 82,
"coverage": 1,
"why": [
"Openness"
]
}
},
"oppositeView": {
"label": "Safety first",
"weights": {
"openness": 1,
"safety": 5
},
"expectedTop": "chatgpt",
"top": {
"code": "chatgpt",
"name": "ChatGPT",
"score": 80,
"coverage": 1,
"why": [
"Safety"
]
},
"sourceTag": "ai-assistants/chatgpt/safety",
"selectedTag": {
"tag": "ai-assistants/chatgpt/safety",
"axis": "safety",
"score": 82,
"summary": "Managed services publish safety evaluations.",
"source": {
"note": "System cards document safety evaluations.",
"source": "https://example.com/system-card",
"asof": "2026"
}
},
"receipt": {
"tag": "ai-assistants/chatgpt/safety",
"axis": "safety",
"score": 82,
"summary": "Managed services publish safety evaluations.",
"source": {
"note": "System cards document safety evaluations.",
"source": "https://example.com/system-card",
"asof": "2026"
}
}
},
"steelman": {
"forEntity": "local-open",
"axis": "safety",
"sourceTag": "ai-assistants/local-open/safety",
"framing": "The strongest counterpoint is managed safety.",
"line": "The strongest counterpoint is managed safety.",
"selectedTag": {
"tag": "ai-assistants/local-open/safety",
"axis": "safety",
"score": 42,
"summary": "Local models vary by configuration.",
"sour
…
A file that must be rejected
A generated challenge index must carry the selected sourced tag for the steelman fact.
{
"format": "open-values-challenge-index",
"version": "0.1.0",
"standard": "open-values-standard",
"built": "2026-07-07",
"purpose": "Generated compact challenge index.",
"generatedFrom": {
"path": "content/challenge/index.json",
"updated": "2026-07-07",
"sha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
},
"counts": {
"instances": 1,
"algorithms": 1,
"cases": 1,
"contraryTags": 1,
"categories": 1
},
"principles": [
"calm challenge"
],
"algorithms": [
{
"id": "steelman",
"label": "Steelman",
"summary": "Surface a sourced contrary fact."
}
],
"instances": [
{
"id": "conscious-consuming",
"label": "Conscious Consuming",
"role": "reference",
"challengeDefault": "available",
"surfaces": [
"verdict"
]
}
],
"categories": [
{
"id": "ai-assistants",
"label": "AI assistants",
"type": "Services",
"file": "app/data/ai-assistants.json",
"criteria": [
{
"key": "safety",
"label": "Safety"
}
]
}
],
"cases": [
{
"id": "sample-challenge",
"category": "ai-assistants",
"label": "Sample challenge",
"userView": {
"label": "Open first",
"weights": {
"safety": 1
},
"expectedTop": "local-open",
"top": {
"code": "local-open",
"name": "Local Open",
"score": 82,
"coverage": 1,
"why": [
"Safety"
]
}
},
"oppositeView": {
"label": "Safety first",
"weights": {
"safety": 5
},
"expectedTop": "chatgpt",
"top": {
"code": "chatgpt",
"name": "ChatGPT",
"score": 80,
"coverage": 1,
"why": [
"Safety"
]
},
"sourceTag": "ai-assistants/chatgpt/safety",
"selectedTag": {
"tag": "ai-assistants/chatgpt/safety",
"axis": "safety",
"score": 82,
"summary": "Managed services publish safety evaluations.",
"source": {
"note": "System cards document safety evaluations.",
"source": "https://example.com/system-card",
"asof": "2026"
}
}
},
"steelman": {
"forEntity": "local-open",
"axis": "safety",
"sourceTag": "ai-assistants/local-open/safety",
"framing": "The strongest counterpoint is managed safety."
}
}
]
}
The schema
Its address is https://openvaluesstandard.org/schema/challenge-index-v0.1.schema.json. Validators can fetch it there; the file is also in the repository at app/data/standard/schemas/challenge-index.schema.json.
The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are used as described in RFC 2119 and RFC 8174 when, and only when, they appear in capitals.
Whether this format fits your app is your call. The spec says what a file MUST hold. Which formats you read and write, and what your app does with them, is yours to decide.