Skills

Chapter 4 of 5

Apify SDK Integration

Integrate Apify into an existing JavaScript/TypeScript or Python application using the apify-client package.

2 minutes · 427 words · 21 sections

Add Apify Actor execution to an existing application. This skill covers the apify-client package for JS/TS and Python, plus the REST API for other languages.

When to Use This Skill

Adding web scraping or automation to an existing app
Calling Apify Actors programmatically from application code
Building a product that uses Apify as a backend service
Integrating Actor results into a data pipeline

Critical: Package Naming

apify-client is the API client for calling Actors from your app. apify is the SDK for building Actors (wrong package for this use case).

Always install apify-client. Never install apify for integration work.

Prerequisites

The user needs an APIFY_TOKEN. Direct them to Console > Settings > Integrations at https://console.apify.com/settings/integrations (opens in a new tab) to create one. If they don’t have an account: https://console.apify.com/sign-up (opens in a new tab) (free, no credit card).

Store the token securely — environment variable or secrets manager, never hardcoded.

Finding the Right Actor

Before writing integration code, find the Actor that fits the user’s needs. Use the MCP tools if available:

search-actors — search the Apify Store by keyword
fetch-actor-details — get the Actor’s input schema, output format, and pricing

Alternatively, browse https://apify.com/store (opens in a new tab). Append .md to any Actor’s Store URL to get its docs in markdown.

JavaScript / TypeScript

Install

npm install apify-client

Synchronous Execution (wait for results)

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('apify/web-scraper').call({
    startUrls: [{ url: 'https://example.com' }],
    maxPagesPerCrawl: 10,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();

.call() blocks until the Actor finishes. Use for short-running Actors (under a few minutes).

Asynchronous Execution (start and poll/retrieve later)

const run = await client.actor('apify/web-scraper').start({
    startUrls: [{ url: 'https://example.com' }],
});

// Poll for completion
const finishedRun = await client.run(run.id).waitForFinish();

// Retrieve results
const { items } = await client.dataset(finishedRun.defaultDatasetId).listItems();

Use .start() + .waitForFinish() for long-running Actors or when you need the run ID immediately.

Retrieving Results

// Dataset items (structured data from pushData)
const { items } = await client.dataset(run.defaultDatasetId).listItems({
    limit: 100,
    offset: 0,
});

// Key-value store (files, screenshots, etc.)
const record = await client.keyValueStore(run.defaultKeyValueStoreId).getRecord('OUTPUT');

Error Handling

try {
    const run = await client.actor('apify/web-scraper').call(input);

    if (run.status !== 'SUCCEEDED') {
        const log = await client.log(run.id).get();
        throw new Error(`Actor failed with status ${run.status}: ${log}`);
    }

    const { items } = await client.dataset(run.defaultDatasetId).listItems();
} catch (error) {
    if (error.message?.includes('not found')) {
        // Actor ID is wrong or Actor was deleted
    } else if (error.statusCode === 401) {
        // Invalid or missing APIFY_TOKEN
    }
    throw error;
}

Python

Install

pip install apify-client

Synchronous Execution

from apify_client import ApifyClient
import os

client = ApifyClient(token=os.environ['APIFY_TOKEN'])

run = client.actor('apify/web-scraper').call(run_input={
    'startUrls': [{'url': 'https://example.com'}],
    'maxPagesPerCrawl': 10,
})

items = client.dataset(run['defaultDatasetId']).list_items().items

Asynchronous Execution

run = client.actor('apify/web-scraper').start(run_input={
    'startUrls': [{'url': 'https://example.com'}],
})

# Poll for completion
finished_run = client.run(run['id']).wait_for_finish()

items = client.dataset(finished_run['defaultDatasetId']).list_items().items

Async Client (asyncio)

from apify_client import ApifyClientAsync

client = ApifyClientAsync(token=os.environ['APIFY_TOKEN'])

run = await client.actor('apify/web-scraper').call(run_input={
    'startUrls': [{'url': 'https://example.com'}],
})

items = (await client.dataset(run['defaultDatasetId']).list_items()).items

REST API (Any Language)

For languages without an official client, use the REST API directly.

Start a Run

POST https://api.apify.com/v2/actors/{actorId}/runs
Authorization: Bearer <APIFY_TOKEN>
Content-Type: application/json

{ "startUrls": [{ "url": "https://example.com" }] }

Get Run Status

GET https://api.apify.com/v2/actor-runs/{runId}
Authorization: Bearer <APIFY_TOKEN>

Get Dataset Items

GET https://api.apify.com/v2/datasets/{datasetId}/items?format=json
Authorization: Bearer <APIFY_TOKEN>

Full API reference: https://docs.apify.com/api/v2 (opens in a new tab)

Best Practices

Set timeouts: Pass timeoutSecs in the Actor input or use waitSecs on .call() to avoid indefinite waits.
Paginate large datasets: Use limit and offset when retrieving dataset items. Default limit is 250K items.
Reuse clients: Create one ApifyClient instance and reuse it across calls.
Handle Actor-specific input: Every Actor has its own input schema. Use fetch-actor-details MCP tool or append .md to the Actor’s Store URL to get the schema before constructing input.

Documentation

Apify API client for JS: https://docs.apify.com/api/client/js (opens in a new tab)
Apify API client for Python: https://docs.apify.com/api/client/python (opens in a new tab)
REST API reference: https://docs.apify.com/api/v2 (opens in a new tab)
Apify docs (LLM-friendly): https://docs.apify.com/llms.txt (opens in a new tab)
Apify docs (full): https://docs.apify.com/llms-full.txt (opens in a new tab)

If the Apify MCP server is available, use search-apify-docs and fetch-apify-docs tools for contextual documentation lookups during development.

Install this repository

$ npx skills add apify/agent-skills

» /plugin marketplace add apify/agent-skills

Skills install per repository, not per chapter — the CLI has no documented per-skill form, so we do not print one.

About this skill

Trigger

Integrate Apify into an existing JavaScript/TypeScript or Python application using the apify-client package. Use when adding web scraping, automation, or data extraction capabilities to an existing app via the Apify API.

The verbatim description from this skill’s front matter — the string an agent matches on to decide whether to load it.

Colophon for this issue

Colophon

How this issue was made

Source

github.com/apify/agent-skills, branch main, last pushed 4 August 2026.

Licence

Discovery

5 chapters found by walking the repository tree for SKILL.md, not by matching a directory convention. One layout observed: skills/*/SKILL.md.

Issue colours

Resolved from a deterministic hash of the owner name. Two accent tones are generated per issue and each is proven against its own ground before it ships: a single accent that passes AA on both light and dark paper is arithmetically impossible.

Heading repairs

1 repair applied to this chapter so the document has one h1 and no skipped levels:

Removed “Apify SDK Integration”, a leading h1 that duplicated the chapter title.

Spec compliance

Every chapter satisfies the Agent Skills specification’s frontmatter rules.

Images

Images inside a chapter come from the upstream repository. Where the author gave no alternative text we mark the image decorative rather than inventing a description — a plausible caption we made up is worse than none for the reader who depends on it.

Marketplace

A plugin manifest is published at .claude-plugin/marketplace.json by Apify, declaring 6 plugins. It is read for editorial metadata only — never as the skill index, which is always the repository tree.

Signal

Install counts come from skills.sh. They measure downloads, not quality, and an unranked repository is not an unread one.

Agent surfaces

The whole issue is available as one markdown document at /apify/agent-skills.md, and each chapter at its own .md URL.

Publication

Set by Skills Docs from the source repository. Body text is Literata at the reader’s chosen size and measure; code is Geist Mono. Nothing on this page was written by us except this paragraph.

Add Apify Actor execution to an existing application. This skill covers the apify-client package for JS/TS and Python, plus the REST API for other languages.

When to Use This Skill

Adding web scraping or automation to an existing app
Calling Apify Actors programmatically from application code
Building a product that uses Apify as a backend service
Integrating Actor results into a data pipeline

Critical: Package Naming

apify-client is the API client for calling Actors from your app. apify is the SDK for building Actors (wrong package for this use case).

Always install apify-client. Never install apify for integration work.

Prerequisites

Store the token securely — environment variable or secrets manager, never hardcoded.

Finding the Right Actor

Before writing integration code, find the Actor that fits the user’s needs. Use the MCP tools if available:

search-actors — search the Apify Store by keyword
fetch-actor-details — get the Actor’s input schema, output format, and pricing

Alternatively, browse https://apify.com/store (opens in a new tab). Append .md to any Actor’s Store URL to get its docs in markdown.

JavaScript / TypeScript

Install

npm install apify-client

Synchronous Execution (wait for results)

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('apify/web-scraper').call({
    startUrls: [{ url: 'https://example.com' }],
    maxPagesPerCrawl: 10,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();

.call() blocks until the Actor finishes. Use for short-running Actors (under a few minutes).

Asynchronous Execution (start and poll/retrieve later)

const run = await client.actor('apify/web-scraper').start({
    startUrls: [{ url: 'https://example.com' }],
});

// Poll for completion
const finishedRun = await client.run(run.id).waitForFinish();

// Retrieve results
const { items } = await client.dataset(finishedRun.defaultDatasetId).listItems();

Use .start() + .waitForFinish() for long-running Actors or when you need the run ID immediately.

Retrieving Results

// Dataset items (structured data from pushData)
const { items } = await client.dataset(run.defaultDatasetId).listItems({
    limit: 100,
    offset: 0,
});

// Key-value store (files, screenshots, etc.)
const record = await client.keyValueStore(run.defaultKeyValueStoreId).getRecord('OUTPUT');

Error Handling

try {
    const run = await client.actor('apify/web-scraper').call(input);

    if (run.status !== 'SUCCEEDED') {
        const log = await client.log(run.id).get();
        throw new Error(`Actor failed with status ${run.status}: ${log}`);
    }

    const { items } = await client.dataset(run.defaultDatasetId).listItems();
} catch (error) {
    if (error.message?.includes('not found')) {
        // Actor ID is wrong or Actor was deleted
    } else if (error.statusCode === 401) {
        // Invalid or missing APIFY_TOKEN
    }
    throw error;
}

Python

Install

pip install apify-client

Synchronous Execution

from apify_client import ApifyClient
import os

client = ApifyClient(token=os.environ['APIFY_TOKEN'])

run = client.actor('apify/web-scraper').call(run_input={
    'startUrls': [{'url': 'https://example.com'}],
    'maxPagesPerCrawl': 10,
})

items = client.dataset(run['defaultDatasetId']).list_items().items

Asynchronous Execution

run = client.actor('apify/web-scraper').start(run_input={
    'startUrls': [{'url': 'https://example.com'}],
})

# Poll for completion
finished_run = client.run(run['id']).wait_for_finish()

items = client.dataset(finished_run['defaultDatasetId']).list_items().items

Async Client (asyncio)

from apify_client import ApifyClientAsync

client = ApifyClientAsync(token=os.environ['APIFY_TOKEN'])

run = await client.actor('apify/web-scraper').call(run_input={
    'startUrls': [{'url': 'https://example.com'}],
})

items = (await client.dataset(run['defaultDatasetId']).list_items()).items

REST API (Any Language)

For languages without an official client, use the REST API directly.

Start a Run

POST https://api.apify.com/v2/actors/{actorId}/runs
Authorization: Bearer <APIFY_TOKEN>
Content-Type: application/json

{ "startUrls": [{ "url": "https://example.com" }] }

Get Run Status

GET https://api.apify.com/v2/actor-runs/{runId}
Authorization: Bearer <APIFY_TOKEN>

Get Dataset Items

GET https://api.apify.com/v2/datasets/{datasetId}/items?format=json
Authorization: Bearer <APIFY_TOKEN>

Full API reference: https://docs.apify.com/api/v2 (opens in a new tab)

Best Practices

Set timeouts: Pass timeoutSecs in the Actor input or use waitSecs on .call() to avoid indefinite waits.
Paginate large datasets: Use limit and offset when retrieving dataset items. Default limit is 250K items.
Reuse clients: Create one ApifyClient instance and reuse it across calls.
Handle Actor-specific input: Every Actor has its own input schema. Use fetch-actor-details MCP tool or append .md to the Actor’s Store URL to get the schema before constructing input.

Documentation

Apify API client for JS: https://docs.apify.com/api/client/js (opens in a new tab)
Apify API client for Python: https://docs.apify.com/api/client/python (opens in a new tab)
REST API reference: https://docs.apify.com/api/v2 (opens in a new tab)
Apify docs (LLM-friendly): https://docs.apify.com/llms.txt (opens in a new tab)
Apify docs (full): https://docs.apify.com/llms-full.txt (opens in a new tab)

If the Apify MCP server is available, use search-apify-docs and fetch-apify-docs tools for contextual documentation lookups during development.