Stage system reference (developer)

The SSI import pipeline is built as a registry of named stages with priorities. The user-facing how it works article covers the conceptual stages; this article is the developer reference for the registration API, filter chain, and parallel-vs-sequential split.

Audience: add-on authors, integrators, and anyone building a custom template.

Registering a stage

Inside a template class (extending SuperSpeedyImports_ImportTemplate, SuperSpeedyImports_PostTemplate, etc.), override registerTemplateStages():

class My_Custom_Template extends SuperSpeedyImports_PostTemplate {
    protected function registerTemplateStages() {
        parent::registerTemplateStages();   // keep the inherited stages

        $this->registerStage(
            'my-custom-stage',                  // name (unique across all stages)
            85,                                 // priority (lower = earlier)
            [$this, 'stage_my_custom_stage'],   // callable
            'Description shown in the admin UI'
        );
    }

    public function stage_my_custom_stage() {
        // your stage logic — runs in $this context
        // $this->table_names['batch'] is the per-import batch table
        // $this->config holds the parsed import config
    }
}

registerStage() is a protected method on the abstract template class (import-template-abstract.php). Add-on plugins (without a custom template) register via the superspeedyimports_register_stages action with the public wrapper addStage(), which takes the same arguments:

add_action('superspeedyimports_register_stages', function($template) {
    $template->addStage(
        'my-addon-stage',
        92,
        function() use ($template) { /* ... */ },
        'My add-on stage'
    );
});

Use addStage() when registering from outside the template class. It calls registerStage() and is public, so add-on code can reach it.

Stage priorities (priority bands)

Lower priority runs first. SSI builds a default stage plan for each import, and these are the priorities it gives the core stages. source-fetch and xml-to-csv are placed relative to load-csv, and post-import-jobs registers at 100000:

You’re free to pick any integer for your stage. Stages sharing a priority run in registration order. If your stage needs to run after the post and postmeta writes but before the image stages, use a priority between 71 and 94.

The superspeedyimports_filter_stages_to_run filter

Every run filters the registered stage list through superspeedyimports_filter_stages_to_run before dispatching. The filter signature:

apply_filters('superspeedyimports_filter_stages_to_run', $stages_assoc, $config, $template);

$stages_assoc is an associative array keyed by stage name. Return a filtered associative array (typically by unset()‘ing keys you want to skip, or building a new array with only the keys you want).

Priority is the standard WP filter priority and controls who-wins-when-multiple-filters-compose.

Built-in filter registrations (priorities)

So a preset shapes ordinary runs, a CLI stage list overrides the preset for one run, and --delete-all has the final say. A CLI list naming a stage the import does not have stops the command before anything runs, with an error listing the import’s stages.

Adding your own filter

To add an extra preset:

add_filter('superspeedyimports_filter_stages_to_run', function($stages, $config, $template) {
    if (!empty($template->additional_options['import_mode'])
        && $template->additional_options['import_mode'] === 'my_custom_mode') {
        return array_intersect_key($stages, array_flip([
            'load-csv', 'match-existing', 'my-custom-stage',
        ]));
    }
    return $stages;
}, 4, 3);  // priority 4 — runs BEFORE the core preset filter at 5

To compose on top of an existing preset:

add_filter('superspeedyimports_filter_stages_to_run', function($stages, $config, $template) {
    // Remove image stages unconditionally
    foreach (['attach-existing-images', 'upload-remote-images', 'attach-gallery-images'] as $img_stage) {
        unset($stages[$img_stage]);
    }
    return $stages;
}, 11, 3);  // priority 11 — runs AFTER the CLI filter at 10

Parallel vs sequential stages

Most stages run sequentially. Only these are marked parallel:

$parallel_stages = [
    'load-csv'                => 'csv',
    'attach-existing-images'  => 'batch',
    'upload-remote-images'    => 'batch',
    'attach-gallery-images'   => 'batch',
    'fix-attributes'          => 'batch',
];

The value ('csv' or 'batch') tells the orchestrator how to segment the work:

• 'csv' — split the CSV file by byte offset and assign each segment to a worker.
• 'batch' – split the batch table by its id range.

The parallel map is fixed inside getStagesWithParallelInfo() on the abstract template class. A stage added through superspeedyimports_register_stages runs sequentially; marking a custom stage parallel means overriding getStagesWithParallelInfo() in your own template. Most add-ons stay sequential.

Don’t parallelise a stage that writes to global state without coordination. The classic violation is wp_insert_term: concurrent workers race on the wp_terms / wp_term_taxonomy UNIQUE keys. SSI recovers from that race (it retries via term_exists() instead of aborting on the duplicate-key error), but it still serialises import-taxonomies by policy, both because the race is cheaper to avoid than to recover from, and because parallelism measured only ~7% faster at 100k.

Add-on authors, take note: superspeedyimports_perf_effective_workers($stage_name, $default) in functions/perf-options.php hard-returns 1 for import-taxonomies before consulting any stage registration or saved worker config. So even if you mark a taxonomy-writing stage parallel, or copy the parallel-stage pattern for import-taxonomies itself, that short-circuit forces it back to a single worker — and the Performance Options Workers dropdown for it is silently ignored.

Hooks that fire during a run

do_action('superspeedyimports_register_stages', $template);    // when stages get registered
do_action('superspeedyimports_template_init', $template);      // right after stage registration
apply_filters('superspeedyimports_stage_presets', $presets, $template);  // import mode other than 'full': preset => allowed stage names
do_action('superspeedyimports_before_import_setup', $import_id);   // every runner: before the run's setup
do_action('superspeedyimports_before_import_stages', $template);   // every runner: just before the first stage
// for each stage (once per segment for a parallel stage):
do_action('superspeedyimports_stage_starting', $stage_name, $template);   // before the stage body
//   (stage callback)
do_action('superspeedyimports_stage_ending', $stage_name, $template);     // after the stage body, before its finalize
// once per stage, on every runner:
do_action('superspeedyimports_stage_finalized', $stage_name, $template);  // after the stage's finalize work
do_action('superspeedyimports_after_import_stages', $template);    // every runner: after the last stage, before the run completes

Use superspeedyimports_register_stages to add or remove stages. Use superspeedyimports_template_init to set up state that all your custom stages need. Use superspeedyimports_stage_starting, superspeedyimports_stage_ending and superspeedyimports_stage_finalized to extend any stage, core or your own, on every runner; work that depends on a stage’s finalize results belongs on superspeedyimports_stage_finalized. The superspeedyimports_before_import_stages and superspeedyimports_after_import_stages pair wraps the stages once per run on every runner (from 3.49.0; before that, on WP-CLI runs only). Browser and scheduled runs execute stages in several requests, so the two may fire in different PHP processes: keep state between them in the database, not in PHP variables.

A stage registered by your own code is dropped by every import-mode preset unless you add it to that preset through superspeedyimports_stage_presets. superspeedyimports_recoverable_stages filters the list of stage names that SSI may re-run whole after a mid-flight death (a browser timeout or a stalled scheduled run); list a stage there only if re-running it from the start is safe and its SQL goes through superspeedyimports_query().

Logging from inside a stage

Every template has a $this->logger (instance of ImportLogger). The key methods:

$this->logger->logStageStart($stage_name);        // already called by the orchestrator
$this->logger->logStageMetric($stage, $key, $value);   // numeric stat shown in the summary
$this->logger->logStageDetail($stage, $msg);      // free-form note attached to the stage row
$this->logger->updateSummaryMetrics($array);      // top-level summary stat
$this->logger->logStageEnd($stage_name);          // already called by the orchestrator

logStageMetric('upsert-relationships', 'product_cat_created', 850000) appears in the CLI summary table AND persists to wp_ssi_import_history_stages.metrics so you can query it post-run.

Re-running a single stage

The orchestrator supports wp ssi <id> <stage-name> for any registered stage. Your custom stage needs to be IDEMPOTENT for this to work — re-running it should produce the same end state whether it’s the first run or the 50th.

Pattern: do all destructive work (DELETE) before the constructive work (INSERT). For relationship-style stages:

-- Delete relationships this stage might have created
DELETE tr FROM wp_term_relationships tr
INNER JOIN wp_super_speedy_imports_batch_<id> it ON tr.object_id = it.post_id
INNER JOIN wp_term_taxonomy tt ON tr.term_taxonomy_id = tt.term_taxonomy_id
WHERE tt.taxonomy = 'my-taxonomy';

-- Recreate them from scratch
INSERT INTO wp_term_relationships (object_id, term_taxonomy_id)
SELECT DISTINCT it.post_id, tt.term_taxonomy_id
FROM wp_super_speedy_imports_batch_<id> it
INNER JOIN wp_terms t ON ...
INNER JOIN wp_term_taxonomy tt ON ...;

A failing re-run after an aborted earlier run leaves the table in the expected end state, not a half-state.

Per-import staging tables

Every stage has access to these via $this->table_names:

These survive between runs on purpose. The runner drops and recreates the batch, terms, hierarchies, duplicate-ID, validation-error, term-metadata, ACF and ambiguous-term tables while setting up a run, before load-csv starts; a WP-CLI run whose stage list leaves out load-csv keeps them. The gallery-images table is recreated when attach-gallery-images runs, and the deletes table when process-deletes runs. Your stage can read them but should treat write access as load-csv’s domain. Don’t add or modify rows from a later stage unless you understand what downstream stages expect.

Useful helpers

superspeedyimports_query($sql, $params = array());    // wrapper around $wpdb->query, fingerprinted
superspeedyimports_get_results($sql, $params);        // wrapper around $wpdb->get_results, fingerprinted; $params has no default, pass array()
superspeedyimports_get_var($sql, $params = array());  // wrapper around $wpdb->get_var, fingerprinted
superspeedyimports_insert($table, $data, $format = null, $params = array(),
           $silent = true, $ignore = false, $on_duplicate_update = ''); // $wpdb insert with INSERT IGNORE / ON DUPLICATE KEY UPDATE options, not fingerprinted

superspeedyimports_is_safe_identifier($name);         // SQL hardening — true if $name is safe to interpolate as a backtick identifier
superspeedyimports_perf_get($stage, $key, $default);  // read per-stage performance options (workers, query_limit, etc.)
superspeedyimports_report_import_error($msg, $fatal); // log an error; if $fatal=true, abort the import

$params in these wrappers is an options array (for example 'purpose' => '...'), not bind values: prepare the SQL with $wpdb->prepare() first. The wrappers run through $wpdb; only load-csv’s row loop switches them to a direct mysqli connection, and not even that when wp-config.php defines SUPERSPEEDYIMPORTS_FORCE_WPDB as true or the site runs the WordPress.org edition.

Use superspeedyimports_query(), superspeedyimports_get_results() and superspeedyimports_get_var() for stage SQL instead of calling $wpdb directly. While a stage runs, they stamp each statement with the stage fingerprint comment /*ssi:<import_id>:<stage>*/ and refresh the run’s heartbeat. That is how SSI tells a stage that is still running from one that has died after a browser request times out. SQL sent straight to $wpdb, or through superspeedyimports_insert(), carries no fingerprint.

Cross-references

• How imports work — the stage pipeline (user-facing) — the same content from the user’s perspective.
• Creating custom templates — for setting up the template class that registers stages.
• Creating add-ons — for stage registration from a plugin without a custom template.
• Stage selection presets — for the user-facing layer that filters this registry.

Written for Super Speedy Imports 3.63.0.