Skip to main content

Storage

Every run gets its own dataset, key-value store and request queue. You can also create named storages that several runs share. Browse them all under Storage & Proxy → Storage in the Console.

KindHoldsTypical use
DatasetA list of JSON itemsThe results of a scrape: one item per product, post or page
Key-value storeRecords by key, any content typeINPUT, OUTPUT, files, screenshots, crawler state
Request queueURLs to visit, de-duplicatedThe crawl frontier of a crawler

Default and named storages​

  • Default storages are created for each run and have no name. The run's default_dataset_id, default_key_value_store_id and default_request_queue_id point at them.
  • Named storages have a name (letters, digits and dashes, up to 63 characters). Actors open them by name to share data across runs, for example a key-value store called state. POST /api/v3/datasets with { "name": "..." } returns the storage with that name, creating it if needed (201 when created, 200 when it existed).

API paths accept a storage's ID or its name.

Retention​

Unnamed storages of a run are removed when your plan's retention period ends (shown on the Billing page as "Run storage kept N days"). Named storages are kept until you delete them.

To keep a run's results, open the storage and use Name this storage, or rename it with the API:

Name a run's dataset to keep it
curl -s -X PATCH "https://lantfly.com/api/v3/datasets/$DATASET_ID" \
-H "Authorization: Bearer $LANTFLY_TOKEN" \
-H "Content-Type: application/json" \
-d '{ "name": "tiktok-weekly-2026" }'

Send { "name": null } to make it unnamed again. Once a run's storages are removed, the run can no longer be resurrected.

List and delete​

RequestWhat it does
GET /api/v3/datasetsList datasets (also key-value-stores, request-queues), oldest first; offset, limit up to 1,000, desc, unnamed=0 for named only
GET /api/v3/datasets/:datasetOne storage with its counts and public link
DELETE /api/v3/datasets/:datasetDelete the storage and its contents
POST /api/v3/datasets/bulk{ "ids": [...], "action": "delete" }, up to 200 storages

Deleting a run's storage empties it for that run too.

Who can read a storage​

By default only people in your account, API tokens and signed links can read your storages.

  • Datasets have an items_public_url and key-value stores a keys_public_url: signed links anyone can open without a token. They expire after 7 days by default. See Datasets and Signed links.
  • Settings → Login & privacy → General resource access can be set so that anyone who has a dataset or key-value store ID can read its items, keys and records without a token. Writes always need a token. In an organization, only the owner and admins can change it.

Storage from inside an actor​

A running actor uses its run token. It can read and write its own default storages and open (or create) any named storage of the account. It cannot reach another run's unnamed storages. See Actor SDK.