Spec / open-values-challenge-set
Open Values Challenge Set v0.1
Worked examples for showing a reader the best case for a different choice: for a category, a likely weighting and its opposite, the top pick under each, and the sourced facts to show. It is kept by hand at content/challenge/index.json; the challenge audit reranks every example against the current data, and pipeline/build_challenge.js builds an index from it.
- Files people or tools write
- Status: draft
- Version: 0.1.0
- Format name in the file:
open-values-challenge-set
Draft. It may still change before it is called stable. A file someone writes by hand or with a tool, and keeps.
Who writes it and who reads it
No writer is listed. Read by future challenge surface, Kosplora configuration, Socrates Colosseum format seed.
Fields
| Field | What it means | Type |
|---|---|---|
format required | Always "open-values-challenge-set". Tells a reader which Open Values format this is. | always "open-values-challenge-set" |
version required | The version of the challenge set format this file follows. Must start with 0.1.0. | string |
standard required | Always "open-values-standard", the standard this format belongs to. | always "open-values-standard" |
updated required | The date the file was last changed, as YYYY-MM-DD. | string |
purpose required | A sentence or two saying what this file is for. | see nonEmptyString |
principles required | The rules challenge features follow, one short sentence each, such as "challenge is calm and dismissible". | list |
algorithms required | The methods that pick what to show a reader, each with what it reads, what it produces, and what it must not do. | list |
instances required | The sites and apps that can show challenges, and how visible each makes them by default. | list |
cases required | Worked examples: a category ranked under a reader's likely weighting and under the opposite one, with the expected top pick of each. The challenge audit reranks each case and fails if the top picks differ. | list |
contraryTags required | Sourced facts that a challenge can show: weaknesses of a likely top pick, or strengths of the opposite pick. | list |
A reader MUST ignore fields it does not know, so a file can carry more than this list and still be read. A writer MUST NOT change what a listed field means.
A file that passes
Challenge sets register algorithms, instance defaults, opposing-view fixtures, and sourced contrary tags.
{
"format": "open-values-challenge-set",
"version": "0.1.0",
"standard": "open-values-standard",
"updated": "2026-07-07",
"purpose": "Seed anti-echo-chamber data.",
"principles": [
"calm challenge",
"sourced facts",
"no engagement mechanics"
],
"algorithms": [
{
"id": "opposite-view",
"label": "Opposite view",
"summary": "Rank under an explicit opposite weighting.",
"inputs": [
"category",
"weights"
],
"outputs": [
"expectedTop"
],
"mustNot": [
"Do not imply objective superiority."
]
},
{
"id": "steelman",
"label": "Steelman",
"summary": "Surface a sourced contrary fact.",
"inputs": [
"entity",
"axis"
],
"outputs": [
"framing"
],
"mustNot": [
"Do not use unsourced criticism."
]
}
],
"instances": [
{
"id": "conscious-consuming",
"label": "Conscious Consuming",
"role": "reference",
"challengeDefault": "available",
"surfaces": [
"verdict"
]
},
{
"id": "kosplora",
"label": "Kosplora",
"role": "learning",
"challengeDefault": "prominent",
"surfaces": [
"resource"
]
},
{
"id": "socrates-colosseum",
"label": "Socrates Colosseum",
"role": "dissent",
"status": "format-seed",
"challengeDefault": "primary",
"surfaces": [
"position"
]
}
],
"cases": [
{
"id": "sample-challenge",
"category": "ai-assistants",
"label": "Sample challenge",
"reads": "The strongest case the other way: ChatGPT gives managed safety receipts.",
"userView": {
"label": "Open first",
"weights": {
"openness": 5,
"safety": 1
},
"expectedTop": "local-open"
},
"oppositeView": {
"label": "Safety first",
"weights": {
"openness": 1,
"safety": 5
},
"expectedTop": "chatgpt",
"sourceTag": "ai-assistants/chatgpt/safety"
},
"steelman": {
"forEntity": "local-open",
"axis": "safety",
"sourceTag": "ai-assistants/local-open/safety",
"framing": "The strongest counterpoint is managed safety."
}
}
],
"contraryTags": [
{
"id": "ai-assistants/local-open/safety",
"category": "ai-assistants",
"entity": "local-open",
"axis": "safety",
"polarity": "steelman-weakness",
"score": 42,
"summary": "Local models vary by configuration.",
"source": {
"note": "Local deployments inherit safety from selected models and configuration.",
"source": "https://example.com/local-safety",
"asof": "2026"
}
},
{
"id": "ai-assistants/chatgpt/safety",
"category": "ai-assistants",
"entity": "chatgpt",
"axis": "safety",
"polarity": "opposite-view-strength",
"score": 82,
"summary": "Managed services publish safety evaluations.",
"source": {
"note": "System cards document safety evaluations.",
"source": "https://example.com/system-card",
"asof": "2026"
}
}
]
}
A file that must be rejected
A steelman sourceTag must point to a steelman-weakness tag for the user-view top pick.
{
"format": "open-values-challenge-set",
"version": "0.1.0",
"standard": "open-values-standard",
"updated": "2026-07-07",
"purpose": "Seed anti-echo-chamber data.",
"principles": [
"calm challenge",
"sourced facts",
"no engagement mechanics"
],
"algorithms": [
{
"id": "opposite-view",
"label": "Opposite view",
"summary": "Rank under opposite weights.",
"inputs": [
"category"
],
"outputs": [
"expectedTop"
],
"mustNot": [
"Do not imply objective superiority."
]
},
{
"id": "steelman",
"label": "Steelman",
"summary": "Surface a contrary fact.",
"inputs": [
"entity"
],
"outputs": [
"framing"
],
"mustNot": [
"Do not use unsourced criticism."
]
}
],
"instances": [
{
"id": "conscious-consuming",
"label": "Conscious Consuming",
"role": "reference",
"challengeDefault": "available",
"surfaces": [
"verdict"
]
},
{
"id": "kosplora",
"label": "Kosplora",
"role": "learning",
"challengeDefault": "prominent",
"surfaces": [
"resource"
]
},
{
"id": "socrates-colosseum",
"label": "Socrates Colosseum",
"role": "dissent",
"status": "format-seed",
"challengeDefault": "primary",
"surfaces": [
"position"
]
}
],
"cases": [
{
"id": "sample-challenge",
"category": "ai-assistants",
"label": "Sample challenge",
"reads": "The strongest case the other way: ChatGPT gives managed safety receipts.",
"userView": {
"label": "Open first",
"weights": {
"openness": 5,
"safety": 1
},
"expectedTop": "local-open"
},
"oppositeView": {
"label": "Safety first",
"weights": {
"openness": 1,
"safety": 5
},
"expectedTop": "chatgpt",
"sourceTag": "ai-assistants/chatgpt/safety"
},
"steelman": {
"forEntity": "local-open",
"axis": "safety",
"sourceTag": "ai-assistants/local-open/safety",
"framing": "The strongest counterpoint is managed safety."
}
}
],
"contraryTags": [
{
"id": "ai-assistants/local-open/safety",
"category": "ai-assistants",
"entity": "local-open",
"axis": "safety",
"polarity": "opposite-view-strength",
"score": 42,
"summary": "Local models vary by configuration.",
"source": {
"note": "Local deployments inherit safety from selected models and configuration.",
"source": "https://example.com/local-safety",
"asof": "2026"
}
},
{
"id": "ai-assistants/chatgpt/safety",
"category": "ai-assistants",
"entity": "chatgpt",
"axis": "safety",
"polarity": "opposite-view-strength",
"score": 82,
"summary": "Managed services publish safety evaluations.",
"source": {
"note": "System cards document safety evaluations.",
"source": "https://example.com/system-card",
"asof": "2026"
}
}
]
}
The schema
Its address is https://openvaluesstandard.org/schema/challenge-set-v0.1.schema.json. Validators can fetch it there; the file is also in the repository at app/data/standard/schemas/challenge-set.schema.json.
The key words MUST, MUST NOT, SHOULD, SHOULD NOT and MAY are used as described in RFC 2119 and RFC 8174 when, and only when, they appear in capitals.
Whether this format fits your app is your call. The spec says what a file MUST hold. Which formats you read and write, and what your app does with them, is yours to decide.