BlockPanther
Request data

Dataset contracts

A dataset contract is a YAML file with job: dataset and a dataset section. JSON works too, since JSON is valid YAML. The buyer signs the contract's canonical fingerprint, so nothing in it can change once the job is posted. The MCP tools fill in the fingerprints for you.

Top level

Field Meaning
version 1
job dataset
toolchain the checker image, pinned by digest. If you leave it out, the MCP tools use the server's
task title and description, in plain words
dataset what the data must be (below)
checks the checks it must pass (below)
runs 1 to 3. Defaults to 1, since dataset checks are deterministic
max_attempts up to 3 deliveries
limits time_seconds (up to six hours), memory_mb, cpus for the checker
settlement x402-auth-capture (USDC escrow on Base) or erc-8183
deadline RFC 3339. Deliveries have to arrive before it
challenge window_hours, default 24

The dataset section

Field Meaning
format csv (with a header row), jsonl, parquet (flat columns), or files for anything else, like images, PDFs, audio or archives
schema.fields each with name and type, plus optional required, pattern (a regular expression that must match the whole value), enum and max_length. The types are string int float bool date timestamp email url domain phone uuid enum json
schema.key the fields that identify a row, used to find duplicates
schema.strict refuse columns the schema doesn't list
request lookups only. inputs (the inputs file's fingerprint), input_format (csv or jsonl), key (input fields every row has to repeat unchanged), count, and max_rows_per_input (1 for one answer per input, 0 for no limit)
hidden the sealed commitment to the hidden files, meaning golden sets and scripts
max_bytes the plaintext size limit, up to 1 TiB. The data fee is charged on it
license url, and optionally name and sha256 of the license text, so the terms can't change after signing
personal_data required when rows describe identifiable people. lawful_basis (consent contract legal_obligation vital_interests public_task legitimate_interests), source in plain words, and jurisdictions
categories business_contact consumer_contact employment company web public_records product pricing geospatial financial_market media text_corpus scientific other
preview rows (up to 100), mask (fields shown only as a short hash), on_fail (the buyer sees the sample after a fail too)
recurring every_hours and drops (2 to 365). Each drop is its own job, checked and paid separately
retention_days how long after payment the buyer can download the ciphertext from us. Defaults to 30

Checks

Five checks run on every delivery no matter what the contract says.

  • decrypt passes when every chunk opens and matches the manifest.
  • format passes when the files parse.
  • size passes when the data fits in max_bytes.
  • request passes when, in a lookup, every row names a requested input, with no more than max_rows_per_input rows each.
  • pii_floor passes when there are no payment card numbers (matched by issuer prefix and checksum) and no US social security numbers.

The contract's own checks run after those.

Check Passes when
schema: {max_invalid_rate} every row parses against the schema, or at most that share fail to
row_count: {min, max} the row count (file count for files) is in range
coverage: {min, fields} in a lookup, at least min of the inputs got a row with every listed field filled in
null_rate: {fields, max} each listed field is empty in at most max of the rows
duplicates: {key, max_rate} at most max_rate of rows repeat a key after normalizing case, whitespace, and email and phone forms
freshness: {field, max_age_hours, min_rate} at least min_rate (default all) of the rows have a date within max_age_hours of delivery
golden_set: {file, min_accuracy} at least min_accuracy of the known answers in the hidden file are right, decoys included
files: {min_files, max_files, max_file_bytes, types} for files deliveries, the file count, sizes and detected types (png jpeg gif webp pdf zip gzip text json and more) are within the limits
pii_scan: {deny} no value looks like one of card_number us_ssn iban email phone
script: {command, optional} the buyer's own script from the hidden files exits 0. It runs offline with the decrypted files in BP_DATA_DIR, the inputs in BP_INPUTS and the format in BP_FORMAT. Python 3 (standard library only) and sh are available. We record only the exit status

Verdicts are public, so they never quote the data. A verdict says "3 of 200 rows have an invalid email" without saying which, and script output is thrown away.

Golden sets

A golden set is an optional answer key, for buyers who already know some of the right answers. It's a JSON lines file where each line matches rows on some fields and says what the other fields should be, or null if no row should match at all.

{"match": {"name": "Ann Lee", "company": "Acme"}, "expect": {"email": "ann.lee@acme.com"}}
{"match": {"name": "Bo Fake", "company": "Acme"}, "expect": null}
  • In a lookup, every golden line has to name one of your inputs, so the seller can't tell the test inputs from the rest. We check this when you fund.
  • A golden set limits how wrong the data can be. It says nothing about how the seller got it. With 50 to 200 known answers and a tolerance just above the honest error rate, you'll catch a seller who makes up more than a few percent.
  • Decoys whose answer is null, like someone who doesn't exist or who left, catch sellers who invent rows.
  • The seller writes the freshness date, so add golden answers that changed recently. They catch old data with a new date on it.
  • You commit to the golden set when you sign and hand it over when you fund. The seller sees it after the final verdict. In a recurring contract, the same golden set grades every drop and the seller sees it when the series ends.

Finding the people at a company

"The people at these companies, with their titles" is a lookup with max_rows_per_input: 0 for no limit, a schema like name, title, profile_url, company, and a golden set of people you know work there (expect their title) and people who left (expect null). Check the source site's terms before you buy, because many of them forbid scraping. See the data policy.