Skip to main content

​
Provenance and source files

Soria publishes provenance with the data contract. A dataset description can include:
  • the declared row grain;
  • methodology and update cadence;
  • upstream providers and canonical source links;
  • recursive source relations from the transformation graph; and
  • retained source files that contributed through those relations.
Dataset-level lineage does not imply that every output row maps to one source row. Aggregates may depend on many files or records.

​
Retrieve a source file

  1. Call list_dataset_files in MCP or GET /datasets/{dataset}/files in the API.
  2. Select a returned file_id.
  3. Call get_dataset_source_file or GET /datasets/{dataset}/files/{file_id}.
  4. Choose representation=original or representation=text.
Soria verifies that the file ID belongs to the named dataset’s recorded source lineage before returning it.

​
Original representation

The response includes customer-safe metadata and a signed link to the retained file. The link expires after five minutes. It is not single-use: the client may fetch it more than once before it expires, which supports retries and redirects. Treat the URL as a temporary secret. Returned metadata includes the filename, media type, source and landing URLs when recorded, SHA-256 checksum when available, source relation, and provenance relationship.

​
Text representation

When parsed text is available, Soria returns a bounded text representation. It is capped at 50,000 characters and reports whether it was truncated. If a file has no safe text representation, request the original file instead.

​
Licensed sources

Some Soria datasets contain derived fields backed by separately licensed data. Column access follows the account’s entitlement. Raw third-party licensed source files are not redistributed through MCP or the Data API, even when derived values are available.

​
What the checksum proves

The SHA-256 checksum identifies the retained bytes recorded by Soria. It helps confirm that two downloads are the same artifact. It does not, by itself, prove that a publisher has not replaced a file at the original URL or that a transformed value is numerically correct; those are separate provenance and data-quality checks.