Changelog

1.84 (21st June 2025)

• Upgraded load-csv again to lower RAM consumption further while handling special characters

1.83 (21st June 2025)

• Re-added previous load-csv ram-optimisation. Fixed now so it handles 4-byte characters like emoticons etc.

1.82 (20th June 2025)

• Upgraded keep-sold SQL to account for cancelled orders, pending orders etc, it will only count as sold actually sold and paid for items
• Keep-sold was also upgraded to work with both traditional and HPOS order processing
• Minor upgrades to CSS to improve visuals for those with long CSV header names (to avoid wrapping as much as possible on 1080p)
• Moved keep-sold & other product related deletes SQL code into the WooCommerce template using filters so it has been removed from the abstract class
• Fixed ocassional collation error at import-taxonomies stage
• Added new filter for posts and postmeta updates https://www.superspeedyplugins.com/kb/super-speedy-imports/advanced/customising-imports-using-sql-filters/
• Upgraded attach-gallery-images stage to handle comma separated images as well as pipe separated
• Added ability to use csv list of stages in the CLI, e.g. wp ssi 1 load-csv,match-existing,update-postmeta
• Consolidated image upload code and in the process fixed initial featured image upload. There were cases where it would fail on first run, then the image would appear on 2nd run.
• Refactored some code to tidy up duplicate code and remove unused code
• Fixed export config to include additional options! (e.g. deletes, continue-on-error)
• Fixed import history to include rows processed - Clicking the history will show more detail about each stage
• Merged some edits from @grambam
  • scan uploads for local images (if you do not have // in the image URL, it will search the uploads folder)
  • import history upgraded to format numeric values
  • fixed running imports from JS (through SSI interface)

1.81 (11th June 2025)

• Added mb_convert_encoding to post_excerpt (short description) too
• Added validated direct SQL for load-csv stage rather than using wpdb to eliminate memory leak - previously, after 800,000 rows of test file, it consumed 16GB! Now it's about 200MB.

1.80 (10th June 2025)

• Added mb_convert_encoding to load-csv stage to handle cases where weird encoding can break $wpdb->insert, this on top of wp_kses_post
• Added full error logging and import abort if any CSV rows fail to import

1.79 (5th June 2025)

• Improved insert special character handling with improved db insert technique
• Added wp_kses_post to post_content and post_excerpt (let me know if this slows down your load-csv stage!)

1.78 (3rd June 2025)

• Upgraded flat-term handler inside load-csv stage to fallback to comma-separated values instead of demanding | separated term names
  • hierarchical taxonomies like product categories still require pipe separation, since categories are far more likely to have commas in their names

1.77 (3rd June 2025)

• Reduced index length further

1.76 (3rd June 2025)

• Added index length restriction for cases where term source values are too large for mysql indexes while retaining performance

1.75 (2nd June 2025)

• Fixed bug in utils so that save-posts and sanitize-terms work correctly now

1.74 (2nd June 2025)

• Upgraded stage detector so you can delete all post entries and it will automatically avoid running update-posts and insert-posts stages
• Added filter so other templates can adjust the stages executed

1.73 (6th May 2025)

• Added a bunch more tests focusing on permalinks
• Fixed bug in insert-posts stage where it now correctly inserts whatever has had a default mapping applied (e.g. post_name)
• Added post_name to the list of required fields (for the batch table, it includes defaults to figure out the permalink)
• Improved the interface code to load the template and check the config before it decides if fields (e.g. post_slug) should be hidden or shown
  • only if you previously used post_slug in the past will post_slug appear - everyone should use post_name to set the slug

1.72 (17th April 2025)

• Fixed bug with product_type warning which was affecting the load-csv stage.

1.71 (12th April 2025)

• Fixed downloadable product support with new fields for download type, expiry, and limits - just add meta: _downloadable=yes and _downloadable_url to use
• Fixed attribute serialization to preserve original attribute order for update items tests
• Fixed product_type taxonomy - refactored so it uses other taxonomy stages
• Refactored taxonomy and postmeta processing loop to use new function to guarantee retrieval of all but exactly the columns required from the imports tables
• Fixed meta updates on 2nd runs and added tests for posts, taxonomy and postmeta updates to different values

1.70 (11th April 2025)

• Improved auto collation handling - the value back from $wpdb->get_charset_collate is often not what is in use in the database, and for importing, being able to join the tables without adding a collation override is important
• Cleaned up a lot of code, mostly involving switching to using the class properties instead of duplicating them into local variables
• Fixed duplicate image upload issue if duplicates were inside the galleries

1.69 (10th April 2025)

• Fixed bug with WooTemplate extending PostTemplate where Woo child was replacing rather than adding to postmeta to update
• Added automated test to confirm custom meta like above is loaded and available

1.68 (8th April 2025)

• Updated JS to show special fields under Post Fields section - these will probably move somewhere else soon, but for now they are back

1.67 (8th April 2025)

• Enhanced image handling:
  • Fixed featured image import to correctly grab the first image if pipe separated
  • Fixed broken code in image gallery upload, previously images were only importing correctly for galleries if you used delayed image import
  • Increased timeout for image downloads from 100ms to 3000ms - if this slows down your import, use Delayed Image Import
• Fixed config export to prevent JSON double-encoding issues
• Added enhanced export functionality for testing:
  • Added featured image URLs to exports
  • Added gallery image URLs to exports
• Added new simple products test with images & gallery images

1.66 (4th April 2025)

• Added .gitattributes file to exclude tests folder from released zip file

1.65 (2nd April 2025)

• Enhanced user interface for better usability:
  • Redesigned file upload interface with improved visual feedback
  • Auto open sections at relevant times, e.g. upload CSV opens the CREATE IMPORT section
  • Added toggle to show/hide unused fields for cleaner interface, defaults to hide when reloading previously saved imports
  • Improved taxonomies sorting in template selection (prioritizing Post and WooCommerce templates for new imports)
  • Other CSS improvements in admin interface
• Added support for CSV files with full paths from CLI imports tool into the admin interface
• Enhanced testing infrastructure:
  • Added verbose mode to test scripts with optional detailed output (--verbose flag)
• Added human-friendly descriptions for postmeta fields

1.64 (1st April 2025)

• Created comprehensive tools management system with standalone classes
• Added new CLI commands for better import/export workflow:
  • Export: wp ssi export ./exports/products.csv product - Export products to CSV file
  • Export with options: wp ssi export ./exports/posts.csv --exclude-dates - Export posts excluding date columns
  • Export taxonomies: wp ssi export-taxonomies product ./taxonomy-export.json - Export product taxonomies
  • Import taxonomies: wp ssi import-taxonomies ./taxonomy-export.json - Import taxonomies from file
  • Load config: wp ssi load-config 10 ./import-config.json - Load config into import ID 10
  • Save config: wp ssi save-config 10 ./export-config.json - Save config from import ID 10
  • Create import: wp ssi create "New Import" ./products.csv WooProductTemplate - Create new import
• Enhanced export functionality with options to exclude date columns
• Improved SQL generation for more consistent and reliable exports
• Added SQL syntax error detection to prevent failed imports
• Improved automated testing infrastructure with better error reporting
• Added comprehensive test suite for regression prevention

1.63 (31st March 2025)

• Added new CLI export command to help with automated testing: wp ssi export $post_type $file_name - no config, it just exports everything from wp_posts, wp_postmeta and all the terms for that post type
• Fixed big issue with fix-attributes - this was only working on 2nd runs, not sure how long for, tests added to prevent it ever happening again
• Fixed issue with hierarchical taxonomies - child leaf was not getting the correct term_taxonomy_id until UPDATE (2nd run)
• Added comprehensive tests to wipe, run, export, re-run, export, diff - more to come to guarantee no regressions and get us to Beta

1.62 (27th March 2025)

• Fixed error handling so that if an actual SQL error occurs then it aborts properly
• Upgraded hard-error reporting so that it uses the :error from WP CLI (it shows in red in the console rather than white)
• Upgraded insert-postmeta stage so that it will skip missing meta columns from the import table which can happen, e.g. if _sale_price was not defined
• Upgraded valid-CSV detection so it works for exports from other places - instead of checking mime type, actually reads the first header row

1.61 (25th March 2025)

• Fixed bug with deletes from refactoring job

1.60 (25th March 2025)

• Added all remaining missing columns for config from wp_posts
  • defaults will be used if they are missing (e.g. author will be the current user, date will be current date)
  • any items NOT configured will guarantee those items will NOT be updated, but may be inserted as required (e.g. author, dates, other sensible defaults)
• Fixed last refactor job - it was working for posts, but not for products
• Added new CPT template in case anyone wants to experiment with post types other than post or product
• Found and fixed 1 place preventing simultaneous import execution (so long as it's different import ids)
• Found and fixed a bunch of other bugs that arose from the refactor job
• In particular, inserts always insert defaults now for any unspecified import whereas updates never will use defaults

1.59 (10th March 2025)

• Minor refactoring
• Added Post template, tested, working
• Separated all product related stuff out into the WooCommerce-specific template
• Added import history to admin interface
• Added import details to admin interface
• Hardened checks for additional options
• Fixed issue with HTML special characters in term names (flat terms & hierarchical)
• Upgraded the sanitize term function to exactly mimic wp_insert_term - should fix any remaining broken term names
• Added new import-taxonomies stage specific to posts and specific to products
• Added fallback for terms failing to insert to grab the duplicate term_id and continue the import
• Added sanitize_terms to clean up the term mess I made
• Added 2 new tables to record ambiguous term names - e.g. if Dave's stuff is a term name, that could be inserted either html encoded or not, and may not be discovered by the join
• Added handler to update the term_taxonomy_id's from the ambiguous terms tables
• Added SSI_RUNNING_IMPORT definition in case anything wants to do or not do stuff when SSI is running
• Batched up the insert-posts command into 100,000 per batch and added performance metrics to trace slow (200 minutes) performance for 2.7 million row CSV file
• Renamed save_posts to save-posts, renamed sanitize_terms to sanitize-terms
• Added --perf option to save_posts to show what's taking time, which function, which plugin

1.58 (4th March 2025)

• Code refactoring of runtime
• Improved performance reporting summary
• Improved logging
• Fixed issue with attributes to ensure it only updates attributes mentioned in the import
• Improved resilience of save_posts for the Woo Product template, now uses wc_get_product and $product->save etc
• Added extra output to end of import to inform admin of how to run the builds for regular woocommerce attributes
• Fixed issue with attribute/taxonomy import - changed to use wc_create_attribute instead of standard wordpress taxonomy code for attributes

1.57 (21st February 2025)

• Fixed force delete not working
• Fixed 'keep sold' items option not being recognised
• Added lots more logging everywhere
• Added import summary table at end of import

1.56 (7th February 2025)

• Fixed progress indicator on save_posts

1.55 (5th February 2025)

• Fixed 'permanently delete' items vs move to trash - thanks to @AdamDar for this one!
• Fixed issue with SQL to keep 'previously sold items'

1.54 (4th February 2025)

• Improved RAM usage for fix-attributes stage
• Added new execution URL which can be useful for profiling with Code Profiler Pro

1.53 (14th January 2025)

• Further improvements to image download to ensure it works on as many hosts as possible
• Fixed fatal error in some paths for image downloads

1.52 (14th January 2025)

• Increased image download timeout
• Fixed issue with source image URLs containing query parameters

1.51 (10th January 2025)

• Fixed CSV upload ajax for brand new users so that the CSV file correctly appears in the dropdown after upload
• Fixed New Import creation for brand new imports, so that the import loads properly after creation
• Fixed deletes - it was referencing the old meta key name, updated to use ssi_updated and deletes are working like before

1.50 (24th December 2024)

• Added continue-on-error functionality for CSV imports - if a row or term fails, it will be logged and the import will continue
• Fixed function editor initialization to properly use full code editor with PHP syntax highlighting

1.49 (23rd December 2024)

• Optimised load-csv stage by eliminating need for get_existing_terms at small cost of extra RAM consumption
• Small optimisation for the insert_flat_terms code by moving index creation on that table to the end of load-csv stage
• Optimised load-csv stage using more memory efficient way of reading the CSV file

1.48 (20th December 2024)

• Added version number to output so I can be sure which version clients are using - some hosts set opcache really high so it can still be using old code on some hosts even if updated through wp-admin

1.47 (20th December 2024)

• Fixed bug with hierarchies being too long - previous limit of 200 characters was lifted (no limit now other than 200 char limit for each term name in the hierarchy)
  • However, this is in contradiction with WordPress - because they uniquify the slug, their uniquify function can create slugs > 200 characters which then fail
  • There's a new filter added to wp_unique_term_slug to reduce the size of the slug and still ensure it's unique so our import can handle any number of X > Y > Z terms so long as each component term name is < 200 characters

1.46 (18th December 2024)

• Fixed memory leak in fix-attributes stage - it was from a call to WordPress core function update_user_meta - even though I only asked to update user meta, it also caches the meta for that item

1.45 (13th December 2024)

• Added PHP directive to display errors to CLI output for cases of some hosting environments having everything off
• Added better debug to failed function eval so you now find out exactly which column has a bad function defined for it

1.44 (12th December 2024)

• Fixed issue with saving PHP functions - was working if the function was still in codemirror/expanded view, otherwise was saving nothing

1.43 (11th December 2024)

• Fixed taxonomy matching bug introduced in 1.35
• Fixed save import broken if taxonomy with function (not csv mapping) was in the import
• Found and fixed a bug affecting terms with parents being mistakenly matched with parentless terms
  • this in turn caused some terms to not be attempted to import and subsequent checks to fail because of these terms being missing
• Added CSV file name for configured template to template info at top of interface
• Added links to let admins see products affected by this import
  • also added dropdown filter to edit.php?post_type=product so users can filter on there to view products last affected by X import
• Big speed boost for upsert-relationships stage - it's as fast as it should be again, e.g. 50,000 items 0.5s now compared to 5+ minutes previously
• Strengthed wp_posts update so it runs one update per column and checks which rows to set based on source being not null and not empty
• Added performance info for load-csv stage which is often the slowest stage

1.42 (6th December 2024)

• Proper fix for tab-separated files - previously I only fixed the import interface, now fixed the actual import run too to handle tab separated files properly

1.41 (6th December 2024)

• Fix for tab-separated files

1.40 (29th November 2024)

• Fix to allow spaces in CSV file names...

1.39 (29th November 2024)

• Added alert if trying to load an import to edit when the CSV file has been deleted - it's currently needed for reference
• Added early-access auto-update prevention mechanism - users have to very clearly and deliberately enable auto-updates to mess up their live site - if you want to enable auto updates on dev or staging, add define('SSI_ALLOW_AUTO_UPDATE', true); to your wp-config.php
• Added run now button to interface for small imports

1.38 (29th November 2024)

• Improved default template to remove unused fields
• Hidden tax and postmeta entries which are meant to be hidden
• Added prevent creation of import with no name
• Added duplicate import name check
• Added new cli parameter list-files - e.g. wp ssi 20 list-files - will list files compatible with this import template
• Added new cli --file parameter - e.g. wp ssi 20 --file="alternative.csv" will use import 20 but against the alternative.csv file - this will barf if the headers are different

1.37 (27th November 2024)

• Fix for new settings to clear the local transient cache when the license key is changed
• Added a new 'recheck now' button on the license key page for when users renew or upgrade

1.36 (21st November 2024)

• Tidy up and standardisation of new license/settings page

1.35 (21st November 2024)

• Speed boost for stage 1 - load-csv stage
• Fix for stage 2 - import-taxonomies - no more reporting error when no error occurred, and fixed bug with duplicate term names in different taxonomies
• Upgraded warning messages inside functions to Fatal Errors - this helps debug failing functions, just fix all the warnings so the code can run

1.34 (20th November 2024)

• Fixed broken function calls - since I added the pretty textareas, the functions were not saving properly
• Updated the JS to use time() for admin so ugosprint definitely gets the latest JS

1.33 (19th November 2024)

• Added _default_attributes hidden field to batch import table
• Use _default_attributes to store first variation from table selected per variable product
• Updates variable product parents efficiently to set the default variation using _default_attributes
• Fixed all slugs to be sanitized - for post_name and for attribute names when they are used as keys

1.32 (16th November 2024)

• Fixed JS/CSS loading issue for certain versions of Scalability Pro (shared Settings file)

1.31 (13th November 2024)

• Fixed issue with additional options (delete items missing from previous import) from appearing multiple times as you loaded each import
• Added code to hide the 'hidden' fields in posts

1.30 (12th November 2024)

• Fixed various warnings

1.29 (12th November 2024)

• Fixed bug where everything was being marked as variable product, even simple products
• Fixed new bug with postmeta _regular_price where it was checking the function rather than the stage table for whether it should upsert or not

1.28 (12th November 2024)

• update-postmeta stage adjusted to increase performance
• Updated post-update and post-insert to not set the value if the source column is null
• Added post-slug field - by default, a sanitized post_title will be used for the post slug, but you can overrule this by setting a value for the post slug - if slug and post title are not set, the post_name will not be changed
• Added variation control into the taxonomy section
• Integrated variation control with previous hard-coded variations list

1.27 (8th November 2024)

• Reduced key size for hierarchical tables to acommodate 4-byte unicode collations
• Removed fallback to try to convert from wpallimport function - it was failing badly

1.26 (8th November 2024)

• Renamed stages to useful names
• Tidied up SQL code especially since no guarantee of execution order
• Fixed some warnings

1.25 (8th November 2024)

• Commented out the eval method for function calling in postmeta

1.24 (8th November 2024)

• Added variable product capability

  • in dev, you need to configure your variation attributes with:

    define('SSI_HARD_CODED_VARIABLE_TAXONOMY_ARRAY', array('pa_size', 'pa_colour'));

1.23 (7th November 2024)

• Added new code to let templates add and alter additional options
• Added additional options section to end of import with save/load/conditional appearance
• Added 3 new delete options to let people configure how they want their deletes handled

1.22 (4th November 2024)

• Fixed bug with flat term import where import failed if term name was empty in csv

1.21 (22nd October 2024)

• Fixed hierarchical taxonomies so they are correctly set with parents in the database - also confirmed existing terms are being retrieved correctly
• Also upgraded multi-term inserts, so if you have e.g. Cat1|Cat2|Cat3 > Cat4 then the item will be allocated to Cat 1, Cat 2 and Cat 4 correctly
• Fixed term inserts and checks for terms when you have the likes of: Salt & Pepper > Salt|Salt & Pepper > Pepper, so it correctly assigns the parent for the 2nd term Pepper to be Salt & Pepper
• Updated code to use the decimal separator correctly for prices

1.20 (3rd October 2024)

• Fixed bug in wp ssi 3 save_posts command - it was just re-running all instead of doing the save_posts

1.19 (30th September 2024)

• Fixed bug for terms with html encodable text in their name (typically &)
• Improved speed by fixing collation issue
  • rather than trusting default collations in WP, i pull the collation from wp_terms since this is what ultimately is joined to on text columns
• Hard errors now correctly stop the script and output the error in RED
• Vastly improved debug and error output to help trace problems more quickly

1.18 (25th September 2024)

• Added gallery image support
• Added checks at start to ensure you have an SKU (for now, there will be other ways of identifying the same item in future)

1.17 (24th September 2024)

• Added featured image import
• Added --verbose mode

1.16 (17th September 2024)

• Fixed the collation issue some users were having
• Added taxonomy export/import as extra option - this makes it far easier for me to replicate exactly what users see
• Fixed an issue with importing terms where the term name was used in another taxonomy
• Tested multiple imports, everything imported fully

1.15 (14th September 2024)

• Added pa_ real name indicators to interface for taxonomy
• Fixed flat taxonomy issue with missing terms after import - all terms are fully loaded now

1.14 (12th September 2024)

• Improved error logging for broken functions so you can see which one needs attention

1.13 (6th September 2024)

• Added optional collation config for those who have mismatching collations in their defaults versus actual tables
  • add the following to your wp-config: define('SSI_COLLATION', 'DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_520_ci');
  • change the collation to whatever your tables are and this will fix collation issues
  • to see your table collations run SHOW TABLE STATUS

1.12 (4th September 2024)

• Fixed stupid mistake of hardcoding my dev table name into stage 2!

1.11 (4th September 2024)

• Much more work and fixes for hiearchical taxonomies
  • tested with flat and hiearchical, tested 5 levels deep
  • only setting leaf term, not all the ancestors
• Fixed attributes
  • product attributes now work properly and are loaded in stage 10
  • this stage may be removed if I can get this back into stage 1 where the array should be defined and where this should work

1.10 (3rd September 2024)

• Speed boost by eliminating duplicates in the _hieararchies table

1.09 (3rd September 2024)

• Code inspected everything related to taxonomies
  • fixed hierarchical taxonomies not being set in destination

1.08 (29th August 2024)

• Added debug for user functions
• Added check for array type to avoid potential fatal error

1.07 (28th August 2024)

• Added export/import for templates
  • export will export to file, but also gives you the export in the textarea to copy/paste
  • import involves pasting the import config in the textarea then hitting import
• Fixed error with decimal place not being sent to the server for new imports
• Changed default mapping types to 'Direct Mapping' rather than 'function' since this should be most of the cases
• Ajaxified everything to reduce confusion for new users

1.06 (13th June 2024)

• Added txt file compatibility
• Added semi-colon separator compatibility
• Tidied up the interface a little
• Added multi-hierachical term capability (so you can have 2+ product cats per item)
• Fixed other taxonomy related bugs
• Optimised the code a bit
• Found a bunch of bugs in taxonomy, fixed more
• Added multi-product_cat compatibility
• Fixed collation and charset so it pulls from $wpdb

1.05 (31st January 2024)

• Fixed additional columns defined in template
  • these are now correctly added to the flat mappings table if the user did not add them
  • the mapData function is properly called now so any template mappings or additional data massage can occur

1.04 (30th January 2024)

• Added commas as decimals handler in the Woo import template
• Added infinite time limit in case wp cli badly configured

1.03 (30th January 2024)

• Fixed import bug to handle dashes in attribute names and postmeta keys

1.02 (30th January 2024)

• Added functions editor to interface
  • also eval these functions at import time through wp cli
  • you must trust people who can edit these functions obviously

1.01 (29th January 2024)

• JS Fix for new imports to load default values for postmeta

1.00 (29th January 2024)

• Very first working version including interface
  • please read the knowledge-base guide to understand how to use this, in particular how to use the function system