> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sandmark.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Paging through history

> Pull a full history window in pages of up to 5,000 rows.

A single request returns at most 5,000 rows (`limit` defaults to 500). Hourly data passes 5,000 rows after
about seven months, so quarterly and annual plans need more than one request to fetch the whole window.

Rows come back **newest first**, so you page **backwards**: ask for the newest page, then set `end` to just
before the oldest row you received, and repeat until a page comes back short.

```python Python theme={null}
import os
from datetime import datetime, timedelta

import requests

BASE_URL = "https://api.sandmark.com/volatility"
session = requests.Session()
session.headers["X-API-Key"] = os.environ["SANDMARK_API_KEY"]


def fetch_all(path: str, **params) -> list[dict]:
    """Every row in the history window, newest first."""
    rows, end = [], None
    while True:
        page_params = {**params, "limit": 5000}
        if end:
            page_params["end"] = end
        response = session.get(f"{BASE_URL}{path}", params=page_params, timeout=60)
        response.raise_for_status()
        page = response.json()["data"]
        rows.extend(page)
        if len(page) < 5000:
            return rows
        oldest = datetime.fromisoformat(page[-1]["DATETIME"].replace("Z", "+00:00"))
        end = (oldest - timedelta(seconds=1)).isoformat()


rows = fetch_all(
    "/v1/timeseries/rv", asset="DER.BTC_USD", frequency="1h", lookback_days=30
)
print(len(rows), "rows from", rows[-1]["DATETIME"], "to", rows[0]["DATETIME"])
```

```javascript JavaScript theme={null}
const BASE_URL = "https://api.sandmark.com/volatility";

async function fetchAll(path, params) {
  const rows = [];
  let end;
  for (;;) {
    const query = new URLSearchParams({ ...params, limit: "5000", ...(end && { end }) });
    const response = await fetch(`${BASE_URL}${path}?${query}`, {
      headers: { "X-API-Key": process.env.SANDMARK_API_KEY },
    });
    if (!response.ok) throw new Error(`${response.status} ${await response.text()}`);
    const { data } = await response.json();
    rows.push(...data);
    if (data.length < 5000) return rows;
    end = new Date(Date.parse(data.at(-1).DATETIME) - 1000).toISOString();
  }
}

const rows = await fetchAll("/v1/timeseries/rv", {
  asset: "DER.BTC_USD",
  frequency: "1h",
  lookback_days: "30",
});
console.log(rows.length, "rows");
```

> [!NOTE]
>
> `end` is inclusive, which is why each page ends one second before the previous page's oldest row. Without
> that step the boundary row would come back twice.

## Staying under the rate limit

Each page counts as one request against your [rate limit](https://docs.sandmark.com/errors-and-limits.md#rate-limit) of 60 per minute.
Three years of one hourly series is about six pages, so one series is well within the limit. If you're
backfilling many series at once, space the requests out or retry on `429` using the `Retry-After` header.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.