Actor SDK for Node.js

@lantfly/actor-sdk is the runtime for actors written in Node.js. Inside a run it talks to the Lantfly API (/api/v3) with the run token. On your machine it reads and writes files under ./storage, so you can develop without the platform.

It gives you:

• the run input, the default dataset and the default key-value store, plus named storages;
• proxy URLs through the Lantfly proxy;
• pay-per-event charges and the run's status message;
• a graceful stop on abort and before the run's timeout;
• the run outcome record (OUTPUT) with a status per input ("targets"), error rows and block detection;
• helpers for crawlers: Crawlee presets, an HTTP client with browser impersonation, fallback strategies and external tools.

Install

Coming soon to npm

@lantfly/actor-sdk (version 0.4.0) is not published to the npm registry yet. Once it is, install it with the commands below. Until then, npm install fails with a 404 for this package.

npm

Terminal

npm install @lantfly/actor-sdk

yarn

Terminal

yarn add @lantfly/actor-sdk

pnpm

Terminal

pnpm add @lantfly/actor-sdk

Requirements:

• Node.js 22 or newer.
• The package is ESM only ("type": "module" in your package.json).
• Optional peer dependencies, only for the helpers that need them: @crawlee/core (comes with @crawlee/playwright and @crawlee/cheerio) for crawlerPresets, and impit or got-scraping for httpClient.

A minimal actor

src/main.js

import { Actor } from '@lantfly/actor-sdk';

await Actor.main(async () => {
  const { message = 'Hello from Lantfly', count = 3 } = await Actor.getInput();
  Actor.log.info(`Pushing ${count} items`);

  for (let index = 1; index <= count; index++) {
    await Actor.pushData({ index, message, at: new Date().toISOString() });
  }

  await Actor.setValue('OUTPUT', { message, count });
});

Run it locally with node src/main.js. The SDK reads ./storage/key_value_stores/default/INPUT.json and writes items to ./storage/datasets/default/.

Import the SDK before Crawlee

Importing @lantfly/actor-sdk sets CRAWLEE_MEMORY_MBYTES (from the run's memory) and CRAWLEE_STORAGE_DIR=/tmp/crawlee when they are not set yet, so Crawlee sizes its pool from the container limit and not from the host. Put the SDK import first:

src/main.ts

import { Actor, crawlerPresets } from '@lantfly/actor-sdk'; // first
import { PlaywrightCrawler } from '@crawlee/playwright';

Lifecycle

Method	What it does
Actor.main(fn, options?)	Calls init(), runs fn, then flushes buffered items, writes OUTPUT, sets the final status message and exits with code 0. An uncaught error gives status failed and exit code 1.
Actor.init(env?, options?)	Reads the environment and picks the backend: the API inside a run, local files otherwise. options.handleSignals (default true) installs the SIGTERM/SIGINT handlers.
Actor.exit(message?, exitCode = 0)	Flushes, writes OUTPUT and ends the process.
Actor.fail(message)	Ends the run with status failed and exit code 1.
Actor.isAtHome()	true inside a Lantfly run.
Actor.log.debug / info / warning / error(message, data?)	Writes a timestamped log line. data is printed as JSON.

Actor.main options:

Option	Default	Meaning
exit	true	End the process when fn settles. With false, main returns, or rethrows the error after OUTPUT is written.
env	process.env	Environment passed to init().
handleSignals	true	Passed to init().

Call Actor.init() before any other method (or use Actor.main). Otherwise the SDK throws Call Actor.init() before using the SDK.

Input

src/main.js

const input = await Actor.getInput(); // the INPUT record, or {} when there is none

getInput() reads the record named by LANTFLY_INPUT_KEY (default INPUT) from the default key-value store. The platform fills in the default values of your input schema before the run starts.

In TypeScript, pass the input type: await Actor.getInput<{ startUrls: { url: string }[] }>().

Datasets

Method	What it does
Actor.pushData(item | items)	Appends one item or a list to the default dataset.
Actor.openDataset(name?)	Returns a Dataset. Without a name: the run's default dataset. With a name: the account's named dataset, created when missing.
dataset.pushData(item | items)	Same as above, for that dataset.
dataset.flush()	Sends the buffered items now.
dataset.itemCount	Items stored so far that are not error rows.

Items must be plain objects. The SDK buffers them and sends them in batches of 50 (at most 500 per request); main, exit and the stop handlers flush the buffer. One item may be at most 9 MB as JSON: a larger item throws a RangeError and is not sent.

See Datasets for reading items back.

Key-value stores

Method	What it does
Actor.getValue(key)	Reads a record of the default store, or null.
Actor.setValue(key, value, { contentType? })	Writes a record of the default store.
Actor.openKeyValueStore(name?)	Returns a KeyValueStore (default store, or a named store of the account, created when missing).
store.getValue(key) / store.setValue(key, value, options)	Same, for that store.

How setValue stores a value:

Value	Stored as
object, array, number, boolean	JSON (application/json; charset=utf-8)
Buffer	as is, with contentType (default application/octet-stream)
string with a contentType	as is (an already serialized JSON string is not encoded again)
string without a contentType	JSON
null or undefined	the record is deleted

getValue parses JSON records, returns text records (text/*) as strings and anything else as a Buffer.

Keep state across a restart

const store = await Actor.openKeyValueStore('my-crawler-state');
const state = (await store.getValue('STATE')) ?? { page: 1 };
// … work …
await store.setValue('STATE', state);

See Key-value stores.

Request queues

The Node.js SDK has no request queue API of its own. Use Crawlee's request queue inside the container (for example through CheerioCrawler or PlaywrightCrawler). The run still has a default request queue on the platform (LANTFLY_DEFAULT_REQUEST_QUEUE_ID) that you can call through the API with the run token.

Proxy

src/main.js

const proxy = Actor.createProxyConfiguration({ groups: ['RESIDENTIAL'], country: 'VN' });

if (proxy) {
  const url = proxy.newUrl('session-1'); // the same session id keeps the same exit IP
}

Actor.createProxyConfiguration(options) returns a LantflyProxyConfiguration, or null when the run has no proxy.

Option	Meaning
groups	Proxy groups in order of preference, for example ['DATACENTER'] or ['RESIDENTIAL']. Empty: any group.
country	Two-letter country code of the exit IP.
useLantflyProxy	false returns null (no proxy). Handy when you pass the input's proxyConfiguration object straight in.

Member	Meaning
newUrl(sessionId?)	A proxy URL. Pass a session id for a sticky exit IP; without one, every call gets a fresh session.
username(sessionId?)	The proxy user name that carries the parameters, for example groups-RESIDENTIAL,country-VN,session-abc.
groups, country	The configured groups and country (upper case).
check(timeoutMs?)	Asks the proxy once whether it can route these groups and this country.

Actor.proxyStatus() runs check() once per configuration and caches the answer: { ok, reason?, message? }. ok is false only when the proxy cannot route the groups or country at all, refuses the password, or cannot be reached.

With Crawlee:

src/main.js

import { Actor } from '@lantfly/actor-sdk';
import { CheerioCrawler, ProxyConfiguration } from '@crawlee/cheerio';

const lantflyProxy = Actor.createProxyConfiguration({ groups: ['DATACENTER'] });
const crawler = new CheerioCrawler({
  proxyConfiguration: lantflyProxy
    ? new ProxyConfiguration({ newUrlFunction: (sessionId) => lantflyProxy.newUrl(sessionId) })
    : undefined,
  async requestHandler({ request, $ }) {
    await Actor.pushData({ url: request.url, title: $('title').text() });
  },
});

On your machine there is no Lantfly proxy unless you set LANTFLY_PROXY_URL to a plain upstream proxy. {session} in that URL is replaced with the session id. See Proxy for groups and countries.

Pay-per-event charges

src/main.js

await Actor.charge('page_opened');      // one event
await Actor.charge('page_opened', 10);  // ten events

Actor.charge(eventName, count = 1) reports events of your actor's pricing while the run is active. Locally it does nothing. The SDK sends each charge with an idempotency key, so a retry after a lost answer is not charged twice. result_item cannot be charged this way: the platform counts it from the results. See Monetization.

Status message

src/main.js

await Actor.setStatusMessage('Crawled 120 of 500 pages');
await Actor.setStatusMessage('Done', { isStatusMessageTerminal: true });

The status message shows on the run in the Console and in the API (status_message). A terminal message is kept after the run ends. setStatusMessage never throws; a failure is only logged. Actor.main sets a terminal message from the run outcome at the end, for example partial: 12 items from 3 targets (BLOCKED×1).

Stopping and timeouts

When the run is aborted or reaches its timeout, the container gets SIGTERM and about 10 seconds before it is killed. The SDK then:

1. emits aborting to your listeners (at most 3 seconds for them),
2. flushes buffered items,
3. writes OUTPUT (targets that did not finish become failed, "Stopped before finishing"),
4. exits with code 0.

A second signal exits at once.

Before the timeout (LANTFLY_TIMEOUT_AT), the SDK emits aborting with reason: 'timeout' early (Actor.stopBeforeTimeoutMs, default 30 seconds, at most a quarter of the time left) and flushes items. Stop taking new work and let main return.

Member	Meaning
Actor.on('aborting', fn) / Actor.off('aborting', fn)	Listen for the stop. fn gets { reason: 'signal' | 'timeout', signal?, timeLeftMs }.
Actor.isAborting()	true once the run is stopping.
Actor.timeLeftMs()	Milliseconds until the timeout (Infinity without one).
Actor.stopOnAbort(crawler)	Calls crawler.stop() on aborting (running requests finish).
Actor.abort(reason)	Starts the graceful stop yourself.
Actor.stopBeforeTimeoutMs	Lead time before the timeout (default 30000).

Stop a Crawlee crawler cleanly

Actor.stopOnAbort(crawler);
await crawler.run(startUrls);

Run outcome: targets and OUTPUT

Actor.main() writes an OUTPUT record when the run ends, with status, items, errors, targets, proxy and finishedAt. Report each input (a hashtag, a profile, a URL) as a target:

src/main.ts

await Actor.main(async () => {
  const input = await Actor.getInput<{ hashtags: string[] }>();
  for (const tag of input.hashtags) {
    const t = Actor.target(`#${tag}`);
    const res = await http.get(urlFor(tag), { expectJson: true });
    if (res.block === 'NOT_FOUND') await t.notFound();
    else if (res.block === 'PRIVATE') await t.private();
    else if (res.block) await t.blocked(res.block, `HTTP ${res.status}`, res.url);
    else {
      const items = parse(res.json());
      if (items === null) await t.failed('SCHEMA_CHANGED', 'unknown payload', res.url);
      else if (items.length === 0) await t.empty();
      else { await Actor.pushData(items); await t.ok(items.length); }
    }
  }
});

Target method	Meaning
ok(n)	Done with n items (calls add up).
empty()	The source answered that there is nothing.
notFound() / private() / restricted()	The input does not exist, is private, or is age or region restricted.
blocked(code, message, url?)	Blocked after every retry. code: BLOCKED, LOGIN_REQUIRED or RATE_LIMITED.
failed(code, message, url?)	Failed, for example SCHEMA_CHANGED.

The last call on a target wins. A target you never resolve counts as ok when the run pushed items, otherwise empty (or failed when the run was stopped or crashed).

Run status in OUTPUT:

Status	When
ok	Every target is ok, not found or private.
partial	Some items, and some targets blocked or failed.
empty	No items, nothing blocked or failed.
blocked	Targets were blocked and nothing else answered.
failed	The run crashed, or targets failed.

Each non-ok target writes an error row { error, errorMessage, input, url? } to the default dataset. Actor.pushError(code, message, { input, url }) writes one directly. Error codes: BLOCKED, LOGIN_REQUIRED, RATE_LIMITED, NOT_FOUND, PRIVATE, AGE_RESTRICTED, EMPTY, SCHEMA_CHANGED, PROXY_UNAVAILABLE, FAILED.

The exit code is 0 for every status; only an uncaught error exits with 1. If your actor writes OUTPUT itself and never uses targets or error rows, the SDK keeps your OUTPUT.

Crawler helpers

Block detection

classifyBlock({ status, url, body, headers, platform, expectJson }) returns null or one of BLOCKED, LOGIN_REQUIRED, RATE_LIMITED, NOT_FOUND, PRIVATE, AGE_RESTRICTED. It knows common captcha, login and bot-check pages, anti-bot vendors and generic 401/403/404/429 answers. An unknown payload is not a block: report it as SCHEMA_CHANGED, not as zero results.

platform is one of tiktok, instagram, facebook, youtube, threads, x, pinterest, shopee, tiki, lazada, tiktok_shop, google_maps, app_store, google_play, lemon8, reddit, telegram, news, forum, generic.

HTTP client

src/main.js

import { Actor, httpClient } from '@lantfly/actor-sdk';

const proxy = Actor.createProxyConfiguration({ groups: ['DATACENTER'] });
const http = await httpClient({ platform: 'generic', proxy });
const res = await http.get('https://example.com/api/items', { expectJson: true });
if (res.block) { /* blocked after the last retry */ }

httpClient({ platform, proxy }) returns a client with get(url, { expectJson }) and request(url, { method, headers, body }). It impersonates Chrome through impit (add impit to your dependencies; it falls back to got-scraping), keeps a sticky proxy session, backs off, takes a new session on a block, and returns res.block instead of throwing after the last retry.

Crawlee presets

src/main.js

import { Actor, crawlerPresets } from '@lantfly/actor-sdk';
import { PlaywrightCrawler } from '@crawlee/playwright';

const proxy = Actor.createProxyConfiguration({ groups: ['RESIDENTIAL'], country: 'VN' });
const preset = await crawlerPresets({ platform: 'generic', proxy });
const crawler = new PlaywrightCrawler({ ...preset, requestHandler: async ({ page }) => { /* … */ } });

crawlerPresets({ platform, proxy, browser? }) returns crawler options: a session pool with a sticky proxy session per Crawlee session (retired on 401/403/429), 3 retries, a per-platform request rate, desktop Chrome fingerprints, the exit country's time zone, and images, media and fonts blocked. browser is chromium (default, headless), chromium-headful (needs a display) or firefox. browserAvailable(browser) tells whether that browser can run in the image.

Fallback strategies

runStrategies(label, strategies, options) tries several ways to get one target in order (for example an HTTP endpoint, then a browser, then an external tool), stops at the first answer worth stopping for, pushes the items, resolves Actor.target(label) and records every attempt in OUTPUT. selectStrategies(all, requested?) picks and orders them from an input field such as strategies: ["browser", "*"].

Each strategy has a name, a kind (http, browser, browser-headful, browser-firefox, tool, official-api, vendor), an optional enabled() check and a run(ctx) that returns { status: 'ok', items }, empty, not_found, private, restricted, blocked or failed. Before the first strategy that uses the proxy, the SDK checks Actor.proxyStatus() once; when the proxy cannot route the run's groups or country, those strategies are skipped instead of retrying for minutes.

runStrategies option	Meaning
limit	Items wanted; extra items are dropped.
key(item)	De-duplicates items within and across strategies.
mergePartial	Go on after an ok with complete: false until limit (default false).
timeoutMs	Time per attempt (default: the time left divided by the strategies still to try).
push	Push the items with Actor.pushData (default true).
tagItems	Add a strategy field (or a field of this name) to pushed items.
resolveTarget	Resolve Actor.target(label) (default true).

External tools

runTool(cmd, args, options) runs a command-line tool (no shell; arguments as an array) with a timeout, an abort signal, JSON-lines parsing and size limits on its output, and kills its whole process group on timeout. whichTool(name) finds it on PATH. toolProxyArgs(proxyUrl) gives --proxy arguments and HTTPS_PROXY/HTTP_PROXY variables for a tool. Put -- before user input so it cannot be read as an option, and never log the proxy arguments: they contain the proxy password.

Review items

Actors that collect reviews (marketplaces, maps, app stores) can build every row with reviewItem({ id, url, rating, text, author, target, variant, verifiedPurchase, sellerReply, publishedAt, likes, media }). It sets kind: 'review', rounds rating to a whole 1–5 (else null), keeps web media URLs once and keeps a seller reply only when it has text. target.type is product, shop, place or app.

Environment variables

The platform sets these variables in the run container; the SDK reads them through readEnv() (available as Actor.env).

Variable	Meaning	Local default
LANTFLY_IS_AT_HOME	1 inside a Lantfly run	unset: local mode
LANTFLY_API_BASE_URL	The API base URL, with /api/v3
LANTFLY_TOKEN	The run token
LANTFLY_ACTOR_ID	The actor's id	local
LANTFLY_ACTOR_RUN_ID	The run's id	local
LANTFLY_DEFAULT_DATASET_ID	The run's default dataset	default
LANTFLY_DEFAULT_KEY_VALUE_STORE_ID	The run's default key-value store	default
LANTFLY_DEFAULT_REQUEST_QUEUE_ID	The run's default request queue	default
LANTFLY_INPUT_KEY	Key of the input record	INPUT
LANTFLY_MEMORY_MBYTES	Memory of the run, in MB	1024
LANTFLY_TIMEOUT_AT	When the run times out (ISO 8601)	none
LANTFLY_PROXY_HOSTNAME, LANTFLY_PROXY_PORT, LANTFLY_PROXY_PASSWORD	The Lantfly proxy	port 8000

Variables you set yourself on your machine:

Variable	Meaning
LANTFLY_LOCAL_STORAGE_DIR	Local storage folder (default ./storage).
LANTFLY_PROXY_URL	A plain upstream proxy for local runs; {session} in it becomes the session id. Ignored on the platform.
LANTFLY_STRATEGIES	Default strategy order when the input has none (comma list of names or kinds).
LANTFLY_DISABLED_STRATEGIES	Strategies never to run (names or kinds).
LANTFLY_TOOL_<NAME>	Path of a tool for whichTool (for example LANTFLY_TOOL_YT_DLP); 0 or off hides it.

The full list of variables in a run is on Actor environment.

Local storage layout

./storage

storage/
  datasets/default/000000001.json            # one file per item
  key_value_stores/default/INPUT.json        # JSON records: <key>.json
  key_value_stores/default/screenshot        # other records: <key> + <key>.__metadata__.json

Named storages use their name as the folder name.

Exports

Export	Kind
Actor, ActorClass, Dataset, KeyValueStore	runtime
LantflyProxyConfiguration	proxy
readEnv, applyCrawleeEnv	environment
ApiBackend, LocalBackend, ApiError	storage backends
OutcomeTracker, TargetHandle, computeStatus, statusMessage, ERROR_CODES	run outcome
classifyBlock	block detection
crawlerPresets, timezoneForCountry, hasDisplay, browserAvailable, camoufoxAvailable, PLATFORM_RATE_LIMITS	Crawlee presets
runStrategies, selectStrategies, isAuthoritative, usesProxy, STRATEGY_KINDS, DEFAULT_STRATEGY_TIMEOUT_MS	strategies
runTool, whichTool, toolProxyArgs, toolEnvName, ToolError, DEFAULT_TOOL_TIMEOUT_MS	tools
httpClient, LantflyHttpClient, RefusedUrlError, CookieJar, splitSetCookie, withJarCookies	HTTP
reviewItem, reviewRating, reviewCount, reviewTargetStats, searchTermMatches, REVIEW_TARGET_TYPES	review items

Next steps

• Build your first actor
• Actor definition files
• Python SDK