Adding compatibility for other plugins

This guide is for developers who need another WordPress plugin to keep working correctly when Super Speedy Imports writes data. It explains why a plugin can need an integration at all, how to tell which kind of integration it needs, and which extension points to use for each kind. It finishes with a worked example: the WooCommerce Back In Stock Notifications integration built into Super Speedy Imports.

Available since 3.53.0.

If you want to add an option or change how an import behaves, see Creating Add-ons for Super Speedy Imports. If you want to import a new kind of content, see Creating Custom Templates for Super Speedy Imports.

Why compatibility needs adding

Super Speedy Imports writes posts, postmeta and term relationships with bulk SQL: one statement writes the price column for every product in the file, instead of loading and saving each product in turn. That is where the speed comes from.

The cost is that the per-item hooks WordPress and WooCommerce fire when an object is saved do not fire during an import. Among them:

• save_post, save_post_{post_type}, wp_insert_post and transition_post_status
• added_post_meta, updated_post_meta and deleted_post_meta
• set_object_terms
• woocommerce_update_product, woocommerce_product_set_stock_status and woocommerce_variation_set_stock_status

Those hooks are how most plugins find out that anything happened. A plugin that relies on them sees nothing when an import updates 50,000 products, and carries on as if the catalogue had not changed. New terms are the exception: the import creates them with wp_insert_term(), so hooks on term creation do fire.

Most plugins need nothing. A plugin that reads the database when a page is requested, and keeps nothing of its own, sees the imported data straight away. The plugins that need an integration fall into two groups.

Plugins that keep data derived from posts

These plugins build their own tables, indexes or caches from your content, and refresh them when a post is saved. After an import they are stale until something tells them to rebuild. Examples:

• search indexes, such as Super Speedy Search, ElasticPress and Relevanssi
• filter indexes, such as Super Speedy Filters
• WooCommerce’s own product lookup table (wc_product_meta_lookup) and attribute lookup table (wc_product_attributes_lookup)
• term counts
• SEO plugins’ indexables
• related-products caches, sitemaps, product feeds and translation tables

The failure is silent staleness: search misses new products, filter counts are wrong, a price sort uses old prices. Everything looks right in wp-admin, which is why it goes unnoticed.

These plugins do not care what changed. They only need telling to rebuild from the data as it is when the import finishes. That makes them cheap to support.

Plugins that act on changes to posts

These plugins act on a transition, and the transition itself is the information. “Stock is 5” does not tell a back-in-stock notifier that it became 5 from 0. Examples:

• back-in-stock notifiers and price-drop alerts
• stock sync to marketplaces, ERPs and CRMs
• webhook dispatchers and audit logs
• cache purgers that clear only the pages of changed products

The failure is that the action never happens: subscribers are never mailed and the marketplace never hears about the new stock. These plugins usually do something external or irreversible, so the naive fix is dangerous. Saving every product after the import gives them no “before” value, and for a notifier it can mean mail to every subscriber at once.

A plugin can be in both groups: it can keep an index and send alerts. Classify it by what needs doing, not by what the plugin is.

Classifying a plugin

Ask two questions of its code.

1. Does it keep anything derived from your data? Any store counts: a custom table, a custom post type, postmeta, options, transients, an object cache or an external index. Search for CREATE TABLE, dbDelta, register_post_type, update_option, set_transient and wp_remote_. If yes, it needs a rebuild after an import.
2. Does it hook the save-side hooks listed above to do something external or irreversible? Follow each handler and see whether it reaches wp_mail, wp_remote_*, as_schedule_* or a queue of its own. If yes, it needs to be told about each change.

If both answers are no, the plugin needs nothing.

The three runners

An import runs from one of three places: the browser (Run Now in wp-admin, through the REST runner), WP-CLI (wp ssi <id>) and the scheduler. Your integration must behave the same on all three.

The browser and scheduled runners split an import across several PHP requests, each with a time limit. WP-CLI runs it in one process with no time limit. So an integration must keep its state in the database rather than in PHP variables, and any long piece of work must be divisible into short steps. Each extension point below is called on every runner; the differences are in how long one call may take.

Rebuilding derived data after an import

Register an After Import task with the superspeedyimports_after_import_tasks filter. This is a public extension point, and it is the same registry Super Speedy Imports uses for term recounts, the WooCommerce lookup tables, the WooCommerce caches and the Super Speedy Search and Super Speedy Filters rebuilds.

A registered task:

• appears in the import’s After Import section under Other Plugins, as a checkbox, and is saved with the import
• runs in the final After Import stage (post-import-jobs) of every run on every runner, when it is ticked
• is listed on the completion screen and at the end of the CLI output when it did not run, with your command and admin link

The filter receives the task list and the import template, which can be null where there is no import to ask (a listing, for example).

Task keys

The saved choice is stored in the import’s additional options as after_import_<task id>.

A single-call task

This example clears a related-products cache for exactly the posts the import touched. Every run keeps a batch table, one row per source row, whose post_id column holds the post each row created or updated. It is available through $template->table_names['batch'] until the next run of the same import recreates it, so read it during the run.

<?php
/**
 * Plugin Name: Acme related products - Super Speedy Imports compatibility
 */
defined( 'ABSPATH' ) || exit;

add_filter( 'superspeedyimports_after_import_tasks', function ( $tasks, $template ) {
    $tasks['acme-related-cache'] = array(
        'label'        => 'Clear Acme related-products cache',
        'description'  => 'Without this, Acme shows related products cached before the import',
        'group'        => 'other',
        'detect'       => function () {
            return function_exists( 'acme_related_forget' );
        },
        'run'          => function ( $template, $value ) {
            global $wpdb;
            $ids = array_map( 'intval', $wpdb->get_col( $wpdb->prepare(
                'SELECT DISTINCT post_id FROM %i WHERE post_id IS NOT NULL',
                $template->table_names['batch']
            ) ) );
            foreach ( array_chunk( $ids, 1000 ) as $chunk ) {
                acme_related_forget( $chunk );
            }
            return array( 'state' => 'complete', 'cleared' => count( $ids ) );
        },
        'cli'          => 'wp acme related flush',
        'browser_safe' => true,
        'priority'     => 80,
    );
    return $tasks;
}, 10, 2 );

Set browser_safe to true only when the work takes a few seconds at most, because on the browser and scheduled runners it shares a request with the rest of the After Import stage. A run task without it is reported as deferred, with your cli command, on a host where the plugin cannot start background processes, such as Super Speedy Imports Lite or a server with proc_open disabled. Anything that can take longer belongs in a stage_facade.

A run task that throws does not fail the import. The data import has succeeded by then; the task is reported as failed and listed as still to run.

A multi-step task

For a rebuild that can take minutes, such as a full search index, give the task a stage_facade class. Super Speedy Imports calls its advance_import() method again and again until it reports complete:

• $owner is "<import id>:<history id>", which identifies this run. Key your progress on it.
• $runner is cli, ajax (the browser) or scheduled.
• Return an array with state, completed, total and current_step. complete ends the task; failed, blocked or cancelled (with an error_message) ends it with a warning; any other state means “call me again”. A WP_Error is treated as a failure.

On the browser and scheduled runners, each request spends about five seconds calling your step and then hands over to the next request, so every call must do one bounded batch and save its progress. On WP-CLI the calls continue until the task completes. A step that fails becomes a warning on the completion screen and leaves the imported data in place; the operator can retry it from there.

<?php
/**
 * Plugin Name: Acme search index - Super Speedy Imports compatibility
 */
defined( 'ABSPATH' ) || exit;

class Acme_Index_Import_Step {
    /**
     * One bounded step of the rebuild. SSI calls this repeatedly until it reports
     * 'complete'. $owner is "<import_id>:<history_id>"; $runner is cli, ajax or scheduled.
     */
    public static function advance_import( $owner, $runner ) {
        $key   = 'acme_index_rebuild_' . md5( $owner );
        $state = get_option( $key, array( 'done' => 0, 'total' => 3 ) );
        if ( $state['done'] < $state['total'] ) {
            // acme_index_next_batch() would go here: index one bounded batch.
            $state['done']++;
            update_option( $key, $state, false );
        }
        return array(
            'state'        => $state['done'] >= $state['total'] ? 'complete' : 'running',
            'completed'    => $state['done'],
            'total'        => $state['total'],
            'current_step' => 'Indexed batch ' . $state['done'] . ' of ' . $state['total'],
        );
    }
}

add_filter( 'superspeedyimports_after_import_tasks', function ( $tasks ) {
    $tasks['acme-index'] = array(
        'label'        => 'Rebuild Acme search index',
        'description'  => 'Without this, Acme search misses items this import added or changed',
        'group'        => 'other',
        'detect'       => function () {
            return class_exists( 'Acme_Index_Import_Step' );
        },
        'stage_facade' => 'Acme_Index_Import_Step',
        'cli'          => 'wp acme index rebuild',
        'browser_safe' => true,
        'priority'     => 85,
    );
    return $tasks;
} );

Super Speedy Search and Super Speedy Filters integrate this way: their Sspsss_Integration_Rebuild and SSF_Integration_Rebuild classes are their stage facades.

Using the completion hooks instead

When your rebuild runs somewhere else, for example on another server, the superspeedyimports_import_run_succeeded action is the simpler choice. It fires once per successful run on every runner, with the import ID, the history ID and a structured report of counters and timings. The report does not list post IDs. Import completion hooks documents the report and the delivery guarantees.

Reacting to changes per item

A plugin that acts on changes needs three things a rebuild does not:

1. the value before the import wrote the new one, captured before the write
2. delivery after the import succeeds, never for a run that failed
3. delivery once per change, even when a request dies part-way and is retried

Super Speedy Imports provides this through its change-event system. It is internal: there is no public filter for registering a listener, and the registration function is documented in the code as not being a public API. Integrations are written into Super Speedy Imports itself, one plugin at a time, each tested against a named version of that plugin. If you maintain a plugin that acts on changes and want an integration, contact Super Speedy Plugins. The rest of this section explains how the system works, so you know what an integration does and how to contribute one.

For the full build of the two back-in-stock notifier integrations, from capture to the vendor’s email, with a step-by-step guide to adding another adapter, see Building change-listener integrations: the back-in-stock notifiers.

What a third party can do today

Without a change-event integration, a plugin that acts on changes has two sound options using public hooks:

• Reconcile against your own record. Keep, in your plugin, the last value you acted on for each item, such as the stock level last pushed to a marketplace. On superspeedyimports_import_run_succeeded, compare it with the current value for the import’s items and act on the differences. This is the same approach a nightly sync takes, and it survives anything else that changes the data, not only imports.
• Snapshot before the write. superspeedyimports_stage_starting fires before each stage’s body on every runner. A callback for the update-postmeta stage can record the current values of the keys you care about, for the posts in the batch table, into a table of your own. Compare after superspeedyimports_import_run_succeeded. The batch table’s columns are not a public contract, so test against each Super Speedy Imports release you support.

Do not fire woocommerce_product_set_stock_status or similar hooks for every imported product after the import. That reports changes that did not happen, and for a notifier it sends mail for products that were in stock all along.

The wp ssi <id> save-posts command saves each of an import’s items through WordPress and WooCommerce, firing their save hooks. It suits a plugin that only rebuilds on save, and is slow on a large catalogue. It does not give a change-listening plugin the old value.

How a change-event integration works

Each integration is a small adapter in functions/compatibility/. It registers a listener with these parts:

The table is the queue of record. Capture happens in the same stage as the write, so no import mode or single-stage run can separate the two, and a crashed run’s rerun captures nothing twice because of a unique key on run, integration, event type and item.

Delivery

Delivery starts in the After Import stage, only after a successful product import, and never when the operator reruns a single After Import job from the completion screen.

• Inline, within a budget. The import delivers what it can itself rather than relying on WP-Cron. In a web request (browser or scheduled) it spends up to 10 seconds; on WP-CLI it has no limit, and starts up to three extra wp ssi deliver-events processes to share the work where the host allows background processes. The superspeedyimports_change_event_workers filter sets the total, 4 by default.
• Queue fallback. Anything not delivered inline is handed to Action Scheduler, or WP-Cron where Action Scheduler is absent, in chunks of 25. Returning false from superspeedyimports_change_event_deliver_inline skips inline delivery and leaves everything to the queue.
• Sweep. Five minutes after each import, a sweep re-drives anything left undelivered. It does nothing when everything has been delivered.
• Claim-based idempotency. Events are claimed 500 at a time with a conditional UPDATE, so concurrent runners take separate sets, and each item is delivered under a per-item database lock. A delivery interrupted mid-way is reclaimed after 15 minutes.
• Coalescing. Several pending events for one item and event type within a claimed set become a single delivery.
• Supersede. Immediately before delivery, the current value is checked again. If a later import or a manual edit has reversed the change, the event is cancelled as superseded and nothing is sent.
• Age cap. Events still pending after 7 days are cancelled, so an abandoned run never sends notifications weeks later.

wp ssi changes <import id> shows an import’s events and their states, wp ssi deliver-events <import id> delivers anything outstanding, and wp ssi retry-events <import id> [listener id] retries failed deliveries. Each also accepts --history=<id> for one run. See the CLI reference.

Worked example: WooCommerce Back In Stock Notifications

WooCommerce Back In Stock Notifications, the official WooCommerce extension, emails customers who signed up to hear when a product is available again. It listens to woocommerce_product_set_stock_status and woocommerce_variation_set_stock_status, which an import does not fire. The adapter, tested with version 3.3.5, is functions/compatibility/woocommerce-back-in-stock-notifications.php, with the capture query shared with the other notifier adapter in functions/compatibility/restock-notifiers.php.

Detect

The adapter is active only when the extension has initialised: WC_BIS() exists and WC_BIS()->sync is a WC_BIS_Sync object. The extension declines to initialise on a WooCommerce version it does not support, and in that case the adapter stays out of the way.

Option

While the extension is active, every product import shows a WooCommerce Back In Stock Notifications switch in its After Import section, under Notify other plugins, and in the Ready to import? box, saved as woocommerce_back_in_stock_notify. It is ticked unless the operator unticks it, so a daily supplier feed can notify while a one-off correction import stays silent. Non-product imports never show it and never capture.

Capture

The extension acts on every change between the three stock statuses when a product is saved by hand, so the adapter captures all six moves:

The capture records only items the import updated (a new product has no subscribers) and only rows where the source file actually set a stock status. A row whose stock status was filled in by default, such as a price-only file, is never treated as a stock move. The old value is read from wp_postmeta immediately before the _stock_status write.

Deliver

For each event, after the supersede check, the adapter loads the product and calls the extension’s own handler and then its flush:

$sync = WC_BIS()->sync;
$sync->product_stock_status_changed( $product_id, (string) $event->new_value, $product );
$sync->sync();

product_stock_status_changed() is the method the extension hooks to the stock-status actions, and sync() is the work it otherwise runs at shutdown. Calling both inside the per-item lock makes the extension do exactly what it does when an administrator saves that product in wp-admin: it applies its own product-type, minimum-stock, throttle and variation rules, queues its notifications, and sends them through its own Action Scheduler chain. A move to out of stock sends nothing; it re-arms sign-ups for the next restock.

The adapter returns a failure reason, rather than true, when the extension has been deactivated or the product no longer exists.

What the adapter does not do

• It never writes the extension’s tables.
• It never sends mail.
• It has no decline condition, because the extension has no setting that processes the same items concurrently. The Back In Stock Notifier for WooCommerce adapter has one: it holds delivery while that plugin’s own third-party stock scanner, or its “keep subscription entry subscribed after mail” setting, is enabled.

What customers see, and how to check notifications went out, is in Using Back In Stock Notifier with Super Speedy Imports.

Checklist for a new integration

1. Classify the plugin with the two questions above.
2. For derived data, register an After Import task: run for a short job, stage_facade for a long one, default left off.
3. For changes, use the reconcile or snapshot approach, or ask for a change-event adapter in Super Speedy Imports.
4. Test on all three runners: Run Now in the browser, wp ssi <id>, and a scheduled run.
5. Test a run that fails and a run that is stopped. The After Import stage does not run for either, so nothing change-driven should happen unless you hooked the general completion action on purpose.
6. Run the same import twice. The second run must not repeat an action for changes that did not happen.

Related products

Super Speedy ImportsEasy to use. Built for speed.Super Speedy SearchSpeeds up front-end and wp-admin searchSuper Speedy FiltersFast product filters and pretty, indexable permalinks on every archive

Further reading

Written for Super Speedy Imports 3.65.0.