Importing XML and JSON feeds

Super Speedy Imports reads XML, JSON and JSON Lines files as ordinary import sources. You choose the file in New Import, confirm which repeating element holds one record, and then map, preview and run the import exactly as you would a CSV. You don’t need an external converter.

Available since 3.19.0.

Every edition includes XML and JSON sources, Lite included.

Add the file

Open Super Speedy > Super Speedy Imports and start a New Import. You can bring an .xml or .json file in the same three ways as a CSV:

• Upload it from your computer.
• Import from URL: paste a link and click Fetch. The server downloads the file, keeps its .xml or .json extension, and checks it by parsing it. The import remembers the URL and downloads the feed again at the start of every run.
• Pick a file you uploaded before from the file list.

Super Speedy Imports decides what a file is from its contents. A file named .xml that does not contain XML is refused with a message saying so, rather than stored.

Confirm the record path

A CSV has one row per record. An XML or JSON document has to say which repeating element is a record, so New Import reads the file and shows a Record Path (XML) or Array Path (JSON) beside the file settings.

• The path it detected is selected, with a note on how sure it is. When more than one level of the file looks like a list of records, the note asks you to check it.
• The dropdown lists the alternatives it found, each with its field count and record count, for example /catalogue/product - 12 fields, 10,000 records.
• Choose Enter a path manually to type a path yourself, such as /catalogue/product for XML or /data/products for JSON.

Changing the path re-reads the file, and the receipt’s field and record counts update to match. Check the record count against what you expect before you create the import.

A feed with only one record

Detection looks for elements that repeat, so a document holding a single record has nothing to detect. In that case the path row opens straight into manual entry with the reason above it. Type the record’s path and the file is read normally.

JSON Lines

A JSON Lines (NDJSON) file, one JSON object per line, is read as a list of records at the path /. Large JSON and JSON Lines files are read as a stream, one record at a time, so a very large feed does not need to fit in memory.

How fields are named

Super Speedy Imports turns each record into fields you can map:

• A nested value becomes a field named after its leaf, such as sku or price. Where two nested values share a leaf name, both keep their parent’s name to tell them apart, such as dims_w and variant_price.
• XML attributes become fields prefixed attr_, such as attr_id.
• A value that repeats inside one record, such as several <image> elements, is joined into one field with |. The Gallery Images field reads a |-separated list as several images.

A single record can hold up to 16 MiB of data and the file up to 2,000 fields. XML containing a DOCTYPE is refused, which keeps entity-expansion attacks out.

Map and run it like a CSV

After you create the import, the Source row on its settings screen names your own file, for example Source: catalogue.xml (XML file). The field list, mapping controls, completion screen and downloadable summary talk about records and fields. Everything else is the same as a CSV import: automatic column mapping, matching, Preview first item and Preview & Run.

Behind the scenes Super Speedy Imports keeps a working copy of your file in its own format. You never pick or edit that copy, and it never appears in the file list. Your XML or JSON file stays the source.

Updating the feed

Each run reads the feed in a stage called Reading the source file (xml-to-csv in the CLI and History). It compares the file’s contents with the last read:

• unchanged contents are not read again, so a repeated or scheduled run of an unchanged feed skips straight to the import;
• changed contents are always read again, even when the file’s date and size are the same.

An import created with Import from URL downloads the feed again at the start of every run, on the browser, WP-CLI and scheduled runners, before Reading the source file. Any other XML or JSON import can do the same: open it, click Change Source beside its Source row, choose Use a URL, enter the feed’s address and click Save and download. A download is used only if it still reads at the import’s saved record path; otherwise the run stops with the reason and the previous file is kept. Choose Choose an uploaded file in Change Source to go back to an uploaded file. An import created from a URL in a version before 3.41.0 needs its URL entered in Change Source once. See Can SSI import from a URL or remote feed? for how the URL is stored and shown.

To give an import that reads an uploaded file a new version of its feed, upload a file with the same name (in New Import, or in Change Source > Upload a new file), or choose another uploaded XML or JSON file in Change Source > Choose an uploaded file, which lists files with the same fields first. The previous copy is kept in the previous folder inside the imports directory, and the upload receipt names where it went. Nothing is deleted automatically.

Scheduled imports need the paid edition. In Lite, run the import from the browser or with WP-CLI after the new file arrives.

From the command line

Two WP-CLI commands handle XML and JSON sources:

# Read a feed and prepare an import from it
wp ssi convert-source /path/to/catalogue.xml [output.csv] [--record-path=/catalogue/product] [--repeat-separator=|]

# Replace the source of an existing XML- or JSON-backed import
wp ssi replace-source 42 /path/to/new-catalogue.json

convert-source detects whether the file is XML or JSON from its contents and prints the record path, record count, fields and the next command to run, a wp ssi create line naming the prepared file. An import created from it keeps your XML or JSON file as its source and reads it again whenever its contents change. replace-source reads the new file completely before it changes anything, so a file that cannot be used leaves the import as it was.

The older names wp ssi xml-to-csv and wp ssi replace-xml run the same commands. See the CLI reference for every flag.

Migrating an XML job from WP All Import

An XML job migrated from WP All Import keeps its XML as the source, with the record path taken from the job’s own XPath. Use Migrating from WP All Import for that route. Migration Tools is only for WP All Import jobs; an XML or JSON file of your own goes through New Import.

Further reading

Related products

Super Speedy ImportsEasy to use. Built for speed.

Written for Super Speedy Imports 3.64.0.