> ## Documentation Index
> Fetch the complete documentation index at: https://docs.soriaanalytics.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Warehouse

> How data is organized and queried in the Soria warehouse — bronze through platinum layers.

Once files are extracted (or, for CSVs, directly after download), you publish them to the warehouse. The warehouse is where your data lives as queryable tables, organized into four layers that build on each other from raw to dashboard-ready.

***

## The four layers

The warehouse uses a layered model where each layer has a specific purpose. You build SQL models to transform data as it moves up the layers.

<CardGroup cols={2}>
  <Card title="Bronze" icon="database">
    Raw source data. Files land here automatically after extraction. One table per group. No transformations applied — this is the data exactly as extracted.
  </Card>

  <Card title="Silver" icon="sparkles">
    Cleaned and typed data. Explicit type casts (no implicit conversions), no JOINs, one source per model. This is where you enforce data types and clean up inconsistencies.
  </Card>

  <Card title="Gold" icon="star">
    Joined data. Combine silver models across sources here. Gold is where cross-source analysis becomes possible.
  </Card>

  <Card title="Platinum" icon="crown">
    Dashboard-ready aggregations. Platinum models can include chart configuration that controls how data appears in dashboards.
  </Card>
</CardGroup>

The pipeline: `bronze → silver → gold → platinum`

***

## Publishing to bronze

When you run `warehouse_manage(publish=True)` for a group, all extracted CSVs for that group are loaded into a bronze table. Column mappings and value mappings are applied automatically at publish time — the warehouse always sees canonical names and values.

```
warehouse_manage(
    group_id="<group_id>",
    publish=True,
    workspace_id="<workspace_id>"
)
```

The output reports the table name, how many files were loaded, and the total rows inserted.

<Note>
  Publishing requires a workspace. Changes don't reach production until you promote the workspace with `workspace_manage(operation="promote", workspace_id="<id>")`.
</Note>

To check the current state of a warehouse table:

```
warehouse_manage(
    group_id="<group_id>",
    status=True
)
```

Status shows the table name, column list, row count, and which files are currently loaded.

### Rebuilding a table

If you need to rebuild a table from scratch — for example, after changing your schema columns — use `force=True`:

```
warehouse_manage(
    group_id="<group_id>",
    publish=True,
    force=True,
    workspace_id="<workspace_id>"
)
```

`force=True` drops and recreates the table before loading.

***

## Querying the warehouse

Use `warehouse_query` to explore data at any layer using DuckDB SQL. This is how you profile data before writing SQL models and verify that published data looks correct.

```
warehouse_query("SELECT * FROM bronze.my_group_name LIMIT 10")
```

Pass `workspace_id` to query a workspace's data (where in-progress changes live). Omit it to query production.

### Useful query patterns

**Profile a table's columns and statistics:**

```sql theme={null}
SUMMARIZE bronze.pa_medicaid_enrollment
```

**Check distinct values in a column:**

```sql theme={null}
SELECT DISTINCT plan_type FROM bronze.pa_medicaid_enrollment
```

**Inspect data types:**

```sql theme={null}
SELECT typeof(enrollment), COUNT(*) FROM bronze.pa_medicaid_enrollment GROUP BY 1
```

**Preview rows:**

```sql theme={null}
SELECT * FROM bronze.pa_medicaid_enrollment LIMIT 10
```

`SUMMARIZE` returns column-level statistics: min, max, approximate distinct count, null count, and more. It's the fastest way to understand a new table.

***

## Building SQL models

Bronze tables are automatically created by the pipeline. Silver, gold, and platinum models are SQL models that you write and save using `sql_model_save`.

Each model is a SQLMesh SQL file with a `MODEL` block that defines its name, layer, and column descriptions. See [SQL Models](/dashboards/sql-models) for how to write and save models.

**Silver model rules:**

* Explicit type casts on every column (`CAST(col AS INTEGER)` or `col::INTEGER`)
* No JOINs — one source per model
* Source CTEs must be `SELECT * FROM source` with no transforms

**Gold model rules:**

* JOINs across silver models are allowed here
* No aggregations — save those for platinum

**Platinum model rules:**

* Aggregations and final calculations
* Can include a `@dashboard` YAML annotation for chart configuration

***

## Table history

Each time you publish and promote, the warehouse records a snapshot. You can inspect this history to understand when data changed and what workspace made the change:

```
warehouse_history("bronze.pa_medicaid_enrollment")
```

The output shows each snapshot version with its timestamp, the workspace that created it, and whether it's been promoted to production.

***

## Removing data

To remove specific files from a warehouse table:

```
warehouse_manage(
    group_id="<group_id>",
    unpublish=True,
    file_ids=["<file_id_1>", "<file_id_2>"],
    workspace_id="<workspace_id>"
)
```

To drop an entire table:

```
warehouse_manage(
    group_id="<group_id>",
    unpublish=True,
    workspace_id="<workspace_id>"
)
```

Omitting `file_ids` drops the whole table.