How to sync WordPress / WooCommerce staging and production site. A current overview.

Last Updated on: 15th May 2019, 09:14 am

The problem scenario

I have come across many different scenarios and tickets where it would have been mighty helpful to be able to sync two WooCommerce sites. This is one of my favourite subjects so I try to keep tabs on various tools and also set aside time to test promising solutions.

Understanding the sync scenario for WordPress

So what do you need to do to actually sync to WorPress sites? In order to able to sync two WordPress / WooCommerce instances we need to basically sync two things.

  1. the /wp-content/ folder – a file sync job
  2. the complete database – a database sync job

As for number 1) syncing files remotely is relatively simple. Most modern FTP clients can do it fine and have the feature built in.

But: Conflict is always hard.

What is a conflict? When two files / two datasets have the exact same file name, things get difficult. You need to set a deciding factor, for example ‘newer date overwrites older date’.

However there must be one source of truth. What if two developers worked on the same file during the same time period? In that case files need to be merged first and then deployed. Someone has to take a look at both sides and merge the data in a way that still makes sense. This often requires two- and three-way comparison of both files. There are of course tools for this, here are two examples: http://meldmerge.org/ for Linux and a free online tool: http://www.mergely.com/ – I google for “diff merge” to find these.

Merging needs to be done before and in person. And I have yet to see anything that can recognise and handle this complicated process during a file sync procedure.

Where does WooCommerce store it’s data

WooCommerce tries to stay as extendable and compatible as possible. You can glimpse at our coding principles here:
https://docs.woocommerce.com/document/create-a-plugin/

Specifically this is something you should remember: https://docs.woocommerce.com/document/create-a-plugin/#section-6 – custom database tables structures are to be avoided. In short this means the following.

  •  WooCommerce Orders and Products are custom post types and stored in the wp_post table.
  • All relevant order and product data is stored as meta info via custom fields and saved in the wp_postmeta or saved as options via wp_options.

The database sync trouble

The way I understand it, the problem with keeping track of WordPress database changes is the post_ID. As content is added and both sites modified the hierarchy and order of the post_IDs can develop in different ways.

This means that very quickly two sites who seem identical now have different information in posts with the same ID – hope this last bit made sense so far. If not, please let me know in the comments.

So how do you tell what is the source of truth? You cannot really. In that case you would have to write one of the data-sets anew with a different ID. Think of when you copy files on your computer and are asked what to do as identical files names are encountered: When selecting “Copy and keep both files”, your computer appends a ‘-1’ to the file and copies the data over. This would also work well here in a sense that no data is lost.

In the case of database entries however, this breaks the dependencies: for example the link to the user, the corresponding order etc. all this information gets unhinged since the post_ID changes.

Bummer.

When merging you have to account for all these scenarios.
So yeah, merging is uber, mega, final-boss-level difficult.

My tools overview for tackling WooCommerce sync

I have tried many tools, only one does live up to the job thus far.
Lets start with the winner.

Commercial Plugin: WP-Stagecoach
https://wpstagecoach.com/

WP-stagecoach works from my experience and successfully merges a staging and production site. I did test it briefly, but did not test thoroughly with WooCommerce yet. The drawback is that you do not own the staging site. The staging site data is kept on their server.

However they are working on this:
https://wpstagecoach.com/wp-stagecoach-web-hosts/

WordPress Plugin: Database Sync
https://wordpress.org/plugins/database-sync/

I have not tested this yet.
The description reads “can pull or push the entire database between servers with just a click”.

I expect it to ‘just’ do a full database push and pull.

GitHub Plugin: Versionpress
http://versionpress.net/

Versionpress is based on Git.

They have a very good blog article that explains the process of merging and it’s challenges: https://blog.versionpress.net/2015/09/versionpress-2-0-staging/

Further reading: http://docs.versionpress.net/en/sync/cloning
and also http://docs.versionpress.net/en/sync/merging

WordPress plugin (with commercial Pro version): WP DB Migrate Pro.
https://wordpress.org/plugins/wp-migrate-db/.

WP DB Migrate Pro works smoothly and well. It allows you to selectively push and pull database data by tables. But there is no sync functionality available. Deliciousbrains are working on another solution they call ‘datahawk’.

WordPress plugin: WP-staging
https://wordpress.org/plugins/wp-staging/

Fastest site duplication I know of. It duplicates locally which is a big plus as you stay in the same environment. The developer is still working on a feature that allows you to push back changes.

WordPress plugin: WPSiteSync for Content
https://wordpress.org/plugins/wpsitesynccontent/

Briefly tested, no support for WooCommerce yet.

WordPress plugin: Gitium
https://wordpress.org/plugins/gitium/

I haven’t taken a look yet. Seems to cater to a different problem scenario.

WordPress plugin: WP Github Sync
https://wordpress.org/plugins/wp-github-sync/

I haven’t taken a look yet.

WordPress plugin: PushLive
https://wordpress.org/plugins/pushlive/

I haven’t taken a look yet. This one supports WordPress Multisite.

Codecanyon plugins
Yes I also looked there. The only thing I found were two ‘copy’ plugins, but nothing capable of sync.
https://codecanyon.net/item/wp-mirror/4069845
https://codecanyon.net/item/wp-remote-post-replicator/9306623

What is next?

Honestly I do not know. But I will keep my ear to the ground and post as soon as I have something new and viable.
Have I missed something out there? If yes, let me know in the comments.

FAQ

While I was working on my staging site my live site got new orders, that I do not want to overwrite / lose.
How do I push my staging site to live without loosing new orders?
How do I selectively push my data so I maintain new orders? 

As mentioned above WooCommerce uses no custom tables for storing products, customers and orders, but saves its data in the wp_post table along with all other posts. There is no selective table transfer possible. You would have to identify the WooCommerce data amidst the post data and then magically and seamlessly fit it into your live site while preserving / adapting post IDs correctly. Not happening (for now).

The solution to this is to use an import / export routine in addition to your deploy process.

  1. Export all orders / customers / products / all WooCommerce data from your current live site. It does not have to be all the data, you can of course set a date range for the export.
  2. Shutdown / block live site to prevent further orders.
  3. Push staging onto live overwriting it completely
  4. Import data from step 1 to new live site.

This will result in a fully updated live site with all current WooCommerce data on board.

23 responses

  1. Thanks for the overview and the options, DB sync is indeed one of the hardest problems in WordPress.

    VersionPress currently supports vanilla WordPress sites and some simpler configurations. Generic 3rd-party plugin support is currently being worked on (issue #1036) and will be available in v4 later this year. WooCommerce is actually one of the plugins we’d like to test and support first – would you like to help us test it?

    Thanks,
    Borek

    1. Con Schneider Avatar
      Con Schneider

      Hi Borek, thanks for the comment and offer. I have followed the progress of VersionPress since it was only an idea. Therefore I will be happy to contribute.
      How can I help?

      1. When the plugin support is roughly ready, we’d like to get feedback on it and try to support some first plugins. Can I add you to that issue as a WooCommerce expert and keep you in the loop?

        1. Con Schneider Avatar
          Con Schneider

          Absolutely. I got the GitHub notification and will keep myself updated as best as I can. Very interested in following the progress.

          1. So, as I am in this particular boat of a woocom dev site and a live site.. is there any further progress on this project?

          2. Con Schneider Avatar
            Con Schneider

            Hi Scott,

            I haven’t spent a lot of time on the subject lately. But I will checkout https://mergebot.com/ beta soon and sum up my experience in a blog post.

  2. Celenie Singley Avatar
    Celenie Singley

    Hi there. I use SiteGround hosting and have a WordPress Woocommerce store. I am able to make staging sites withe SiteGround, but my concern is that when I push my updated staging site to the live site, that it will overwrite the data concerning my store’s sales that were made while I was updating the staging version. How can I avoid that?

    1. Con Schneider Avatar
      Con Schneider

      Hi Celenie,

      You are correct in your assumption that pushing back changes from your staging site will overwrite the data on your live site.
      Other than the options discussed here, I have not found anything better yet.

      If the changes are not many and there is no option in this post that you like, I would simply apply them manually. I also like using selective database table migrations to move only the data I want. But that requires you to be able to clearly separate the changes you work on from the WooCommerce data.

      1. Celenie Singley Avatar
        Celenie Singley

        Okay – thanks!

    2. Yo. Are there new solution for woocommerce dev and live server merging issue?

      1. Con Schneider Avatar
        Con Schneider

        Honestly this post could use an update. But I have not been keeping track.

  3. Hey Con,

    I’m curious if you have you thoroughly tested Stagecoach with a WP+WooCommerce site at this point, and if so, what your observations were.

    Many thanks,

    Eric

    1. Con Schneider Avatar
      Con Schneider

      Hi Eric,

      I have not yet, but I would be curious as well. If I find the time I’ll do just that and reach out to the team of WP-Stagecoach. This post can certainly use a follow-up at some point soon :).

      Many regards,
      Con

  4. WP All Export with WP All Import, is a great option for staging to prod sync issues. Since you can export just the needed post types, products and orders, can also deal with users as well. Since it supports WooCommerce.

    1. Con Schneider Avatar
      Con Schneider

      Yup. I like WP All Import / Export as well. Good stuff.

  5. Simon Skilling Avatar
    Simon Skilling

    This is one of those issues with Woo that I file under the ‘can’t believe no-one’s sorted this yet’ pile – along with properly customizable email templates.
    In my mind, there are two things needed.
    It’s no doubt much more complex than I realise, but feels like a plugin should be able to handle it, if designed well.
    At the risk of sounding like a Sci-Fi movie premise – a clone of a site, doesn’t know that it’s a clone. So first up – there needs to be a way to tick a box (or some other method) that says ‘I’m the clone’. Then the only other crucial info is, what date/time was it cloned. Then you know all DB records following that point (on the clone) are ones that will need new IDs when merged. (There will probably need to be checks to ensure no duplicate slugs, etc. )
    My other thought is that – when you make the clone (and tell it, it’s the clone) then it adds a prefix to the IDs, in a way that means they are in no risk of being also used on live. Or jump the numbers ahead (e.g. last ID used at point of cloning was 1081, so here on the clone we’ll start using from 1000001082 But I’m not sure DBs work that way…
    My other other thought is for the merging to happen ‘live’ with the two sites constantly polling each other to let them know which IDs are reserved/used. But again, probably not feasible/reliable.

    1. Con Schneider Avatar
      Con Schneider

      Thank you. There are some good ideas here.

  6. Hello, I’m currently involved in trying to support production and staging environments that have WooCommerce as well. Our current Hosting Provider: Liquid Web (Nexcess) (not a plug for their service believe me) recommended WPMERGE.io

    I’ve tried using this plugin only to have my production site merged into my staging site.
    Glad I made backups.

    Came here to see if anyone has had experience using https://wp-staging.com/ as this might
    be an alternative?

    1. Con Schneider Avatar
      Con Schneider

      Hi Kerry,

      Welcome. I only know the free version of WP-Staging so. Hope someone else can pitch in.
      🤞.

  7. We built staging to support this use case on WooCart where we host only WooCommerce stores.

    For a basic user it looks very simple: create staging, do changes and then click publish. Our system downloads orders/customers and publishes the new version.

    We do this by incrementing IDs when staging is created, then download the new orders/customers from the live site before publishing. The only issue we found is with custom plugins that change the database on production while staging is running. We add custom integrations for those (common ones are affiliate plugins, eBay and other marketplace syncing plugins).

    I’d love for you to add us as one of the solutions to your blog post. 🙂

    Dejan

    1. Con Schneider Avatar
      Con Schneider

      Hi Dejan,

      Thank you for your comment. All the articles I wrote were researched by me which took quite some time. However I don’t invest any time into blogging these days so I am not adding any new content to my articles since I don’t publish anything without investing the time to look at it personally first.

      But this articles still gets some traffic and your comment author link does not have any rel attribute. 😉

      Kind regards,

  8. heinrich taitl Avatar
    heinrich taitl

    hi con,
    are there any news regarding this topic?

    1. Con Schneider Avatar
      Con Schneider

      Hi Heinrich,

      I haven’t been keeping tabs on the subject so I can’t say. Is there anything specific you are wondering about?

Leave a Reply

Your email address will not be published. Required fields are marked *

Con Schneider