Update to HPOS, some small stylistic changes

Peter Fabian 2022-09-18 22:18:52 +02:00
parent cad419bffc
commit e402982450
2 changed files with 192 additions and 204 deletions

@ -1,204 +0,0 @@
> 💡 This is a place to collect short 'recipes' for the transition to COT that may be useful by themselves, or possibly for later use as part of an official upgrade guide. Feel free to add to it and edit it.
# What are custom order tables (COT)?
WooCommerce has stored order related data in the post and postmeta tables in the database as a custom WordPress post type so far. This has allowed everyone working in the ecosystem to take advantage of extensive APIs provided by the WordPress core in managing these custom post types.
However, earlier this year, [we announced our plans to migrate to custom tables for orders](https://developer.woocommerce.com/2022/01/17/the-plan-for-the-woocommerce-custom-order-table/). Orders in their own tables will allow the shop to scale more easily (especially for large shops), make the data storage simple and increase reliability.
Generally, WooCommerce has tried to be fully backward compatible with the older versions, but, in this project there will be effort required by developers to take advantage of the project. This is because the underlying data structure has changed fundamentally.
More specifically, instead of using WordPress provided APIs to access order data, we will have to use WooCommerce specific APIs. We [introduced these APIs in WooCommerce version 3.0](https://developer.woocommerce.com/2017/04/04/say-hello-to-woocommerce-3-0-bionic-butterfly/) with the intention of making the transition to custom tables easier.
In this guide, we will document what (if any) changes are required to make an extension, or a custom code compatible with custom order tables.
# Backward compatibility
To make the transition easier for shops, we have tried to be as backward compatible as possible as well. One of the major compatibility issues with this project is that since the underlying data structure was `wp_posts` and `wp_postmeta` tables, circumventing the WC-specific CRUD classes and accessing the data directly using WordPress APIs worked fine.
Now that this has changed, directly reading from these WordPress tables may mean reading an outdated order, and directly writing to these tables may mean updating an order that will not be read. Since this is a pretty significant issue, we have added few mitigations for a transitory period.
## Roll out policies
As we rollout the project, we are going to be following these policies:
1. Custom order tables will not be enabled for any shop by default for now. This means you can continue to safely upgrade WooCommerce regardless of whether you will be opting into custom order tables or not.
2. Shops can opt-in to use the custom order tables (please see the section below _How to enable Custom Order Tables_ below). When they do so, we will enable *data synchronization* (or _sync_, in short) by default to provide additional layer of compatibility:
1. Whenever we write data to custom order tables, we will write the same to the posts table as well. This way, if there are incompatible plugins that still access data directly, they will read updated data from the posts table.
2. Similarly, when we read order from custom order tables, we will read the same order from the posts table as well and compare them. If there is any difference, then itd imply a direct write to posts table and WC will update the order in the custom tables accordingly.
It is important to keep the sync in place until all plugins and custom code are compatible with the new data structure of COT. Any performance penalties will be temporary, resulting from additional inserts present in the sync.
## Switching data source
Additionally, you can switch from using custom order tables to posts tables manually if you see an issue with the custom order tables. To switch back, change the Data store for orders setting in the WC > Settings > Advanced > Custom data stores:
Please note this setting is only available once the custom order tables feature is enabled (please see section How to enable Custom Order Tables below).
## Possibility of downgrading
You can also downgrade the WooCommerce version to a version before custom order tables were launched if the sync was enabled. This is possible because when sync is enabled, we propagate any changes to custom order tables to wp_posts and wp_postmeta tables as well.
# How to enable Custom Order Tables
## Easy way
1. Set up your dev environment according to the [general dev guide](https://github.com/woocommerce/woocommerce/tree/trunk/plugins/woocommerce#readme)
2. Grab and install the zip package from the pre-release page
[COT Release testing](https://github.com/woocommerce/woocommerce/releases/tag/feature-custom-order-table)
4. Add the following snippet to your WordPress installation (e.g. by using [Code Snippets plugin](https://wordpress.org/plugins/code-snippets/)):
```
function enable_cot(){
$order_controller = wc_get_container()
->get('Automattic\WooCommerce\Internal\DataStores\Orders\CustomOrdersTableController');
if( isset( $order_controller ) ) {
$order_controller->show_feature();
}
}
add_action( 'init', 'enable_cot', 99 );
```
5. Create custom tables by running _Create custom order tables_ tool from WC > Status > Tools (once you have them created the tool will change to Delete the custom order tables tool).
6. If you have created the tables previously, please delete the old ones and create new ones again, as the database structure has changed a couple of times.
7. Now you can use settings in WC > Settings > Advanced > Custom data stores to switch data source from posts table to custom table for orders.
## Migrating orders
When getting started and when testing out Custom Order Tables, you may occasionally find that orders are waiting to be synced. For instance, within **WooCommerce ▸ Settings ▸ Advanced ▸ Custom Data Stores** you may see a message like, _"There are 1234 orders pending sync!"_
<img width="827" alt="cot-orders-pending-sync" src="https://user-images.githubusercontent.com/3594411/184413815-73c057b3-7551-4813-a228-cdeddd856ca7.png">
You will amost certainly see this when you are getting started with COT, but depending on your testing patterns you may also see it later on and will wish to clear the message—which can be done by triggering a migration via WP CLI:
```sh
wp wc cot migrate
```
## Using GH
Use the same steps as above, but instead of downloading the zip package (step 2), checkout the `cot-main` branch from the monorepo and build WC using `pnpm -- turbo run build --filter=woocommerce`. Then mount/link the `plugins/woocommerce` subdirectory to your WP plugins directory.
# Supporting Custom Order Tables (COT)
While these backward compatibility policies make it easier for merchants to use the project, for extension developers this means that you have to support both Posts and Custom Order Tables for a period of time.
To help with this, we have provided a few guidelines for extension developers to follow:
## Detecting whether custom order tables are being used in the store:
While the WooCommerce CRUD API will let you support both posts and custom tables without additional effort, in some cases (like when you are writing a SQL query for better performance) you would want to know whether you are using custom order tables or not. In this case, you can use the following pattern
```php
use Automattic\WooCommerce\Utilities\woocommerce;
if ( OrderUtil::custom_orders_table_usage_is_enabled() ) {
// COT usage is enabled.
} else {
// Traditional CPT-based orders are in use.
}
```
## Auditing the code base for direct DB access usage
To support custom order tables, a good place to start is to audit your code base for direct DB access and usage of WordPress APIs that we won't be using any more. You can search using following regexes to perform this audit:
(Note that you will find lots of false positives, but these regexes are pretty thorough and will catch most of the true cases.)
```regexp
(get_post)|(get_post_field)|(get_post_status)|(get_post_type)|(get_post_type_object)|(get_posts)|(metadata_exists)|(get_post_meta)|(get_metadata)|(get_metadata_raw)|(get_metadata_default)|(get_metadata_by_mid)|(wp_insert_post)|(add_metadata)|(add_post_meta)|(wp_update_post)|(update_post_meta)|(update_metadata)|(update_metadata_by_mid)|(delete_metadata)|(delete_post_meta)|(delete_metadata_by_mid)|(delete_post_meta_by_key)|(wp_delete_post)|(wp_trash_post)|(wp_untrash_post)|(wp_transition_post_status)|(clean_post_cache)|(update_post_caches)|(update_postmeta_cache)|(post_exists)
```
Search for the above regex in your source code, and:
1. Go through each match one by one. Most of these will be false positives i.e. they won't be in context of an order.
2. If you see one of these matches are directly accessing or modifying order data, you will need to change it to use WooCommerce's CRUD API instead.
## APIs for getting/setting posts and postmeta
Any code getting posts directly can be converted to a `wc_get_order` call instead:
```php
// Instead of
$post = get_post( $post_id ); // returns WP_Post object.
// use
$order = wc_get_order( $post_id ); // returns WC_Order object.
```
For interacting with metadata, use the `update_`/`add_`/`delete_metadata` methods on the order object, followed by a `save` call. WooCommerce will take care of figuring out which tables are active, and saving data in appropriate locations.
```php
// Instead of following update/add/delete methods, use:
update_post_meta( $post_id, $meta_key_1, $meta_value_1 );
add_post_meta( $post_id, $meta_key_2, $meta_value_2 );
delete_post_meta( $post_id, $meta_key_3, $meta_value_3 );
// use
$order = wc_get_order( $post_id );
$order->update_meta_data( $meta_key_1, $meta_value_1 );
$order->add_meta_data( $meta_key_2, $meta_value_2 );
$order->delete_meta_data( $meta_key_3, $meta_value_3 );
$order->save();
```
💡 Calling the `save()` method is a relatively expensive option, so you may wish to avoid calling it more times than necessary (for example, if you know it will be called later in the same flow, you may wish to avoid additional earlier calls when operating on the same object).
When getting exact type of an order, or checking if given ID is an order, you can use methods from `OrderUtil` class.
```php
// Pattern to check when an ID is an order
shop_order === get_post_type( $post_id ); // or
in_array( get_post_type( $post_type ), wc_get_order_types() );
// replace with:
use Automattic\WooCommerce\Utilities\OrderUtil;
shop_order === OrderUtil::get_order_type( $post_id ); // or
OrderUtil::is_order( $post_id, wc_get_order_types() );
```
## Audit for order administration screen functions
Similar to direct DB access, we have also added new screens for order administration. These screens are very similar to the one you see in the WooCommerce admin currently (except for the fact that they are using custom order tables). Current admin screens are powered by WordPress's custom post type API, which cannot be used anymore. You can use the following regex to perform this audit:
```regexp
(post_updated_messages)|(do_meta_boxes)|(enter_title_here)|(edit_form_before_permalink)|(edit_form_after_title)|(edit_form_after_editor)|(submitpage_box)|(submitpost_box)|(edit_form_advanced)|(dbx_post_sidebar)|(manage_shop_order_posts_columns)|(manage_shop_order_posts_custom_column)
```
You will see a lot of false positives here as well. However, if you do encounter a usage where these methods are called for the order screen then to upgrade them to custom tables, the following changes have to be done:
Instead of a `$post` object of the `WP_Post` class, you will need to use an `$order` object of the `WC_Order` class. If its a filter or an action, then we will implement a similar filter in WooCommerce screen as well which will be passed a WC_Order object instead of post.
The following snippet shows a way to add meta boxes to the legacy order editor screen when legacy orders are in effect, and to the new COT-powered editor screen otherwise:
```php
use Automattic\WooCommerce\Internal\DataStores\Orders\CustomOrdersTableController;
add_action( 'add_meta_boxes', 'xyz_metabox' );
function add_xyz_metabox() {
$screen = wc_get_container()->get( CustomOrdersTableController::class )->custom_orders_table_usage_is_enabled()
? wc_get_page_screen_id( 'shop-order' )
: 'shop_order';
add_meta_box(
'xyz',
'Custom Meta Box',
render_xyz_metabox,
$screen,
'side',
'high'
);
}
```
The above will also change the parameter passed to the metabox to order. So in your metaboxes, you would have to account for both a post or order object that may be passed. We recommend fetching the order object and working with it completely instead of the passed parameter.
```php
function render_xyz_metabox( $post_or_order_object ) {
$order = ( $post_or_order_object instanceof WP_Post ) ? wc_get_order( $post_or_order_object->ID ) : $post_or_order_object;
... rest of the code. $post_or_order_object should not be used directly below this point.
}
```

@ -0,0 +1,192 @@
> 💡 This is a place to collect short 'recipes' for the transition to High-performance Order Storage that may be useful by themselves, or possibly for later use as part of an official upgrade guide. Feel free to add to it and edit it.
# What is High-performance Order Storage (HPOS)?
WooCommerce has stored order-related data in the post and postmeta tables in the database as a custom WordPress post type so far. This has allowed everyone working in the ecosystem to take advantage of extensive APIs provided by the WordPress core in managing orders as custom post types.
However, earlier this year, [we announced the plans to migrate to dedicated tables for orders](https://developer.woocommerce.com/2022/01/17/the-plan-for-the-woocommerce-custom-order-table/). Orders in their own tables will allow the shops to scale more easily, make the data storage simpler and increase reliability. For further details, please check out our [deep dive on the database structure on our dev blog](https://developer.woocommerce.com/2022/09/15/high-performance-order-storage-database-schema/).
Generally, WooCommerce has tried to be fully backward compatible with the older versions, but, as a result of this project, extension developers will be required to make some changes to their plugins to take advantage of the HPOS. This is because the underlying data structure has changed fundamentally.
More specifically, instead of using WordPress-provided APIs to access order data, developers will need to use WooCommerce-specific APIs. We [introduced these APIs in WooCommerce version 3.0](https://developer.woocommerce.com/2017/04/04/say-hello-to-woocommerce-3-0-bionic-butterfly/) to make the transition to HPOS easier.
In this guide, we will document the changes required to make an extension, or a custom code compatible with HPOS.
# Backward compatibility
To make the transition easier for shops and developers alike, we have tried to be as backward compatible as possible. One of the major compatibility issues with this project is that since the underlying data structure was `wp_posts` and `wp_postmeta` tables, circumventing the WC-specific CRUD classes and accessing the data directly using WordPress APIs worked fine.
Now that this has changed, directly reading from these WordPress tables may mean reading an outdated order, and directly writing to these tables may mean updating an order that will not be read. Since this is a pretty significant issue, we have added a few mitigations for the transitory period.
## Roll out policies
As we roll out the project, we are going to be following these policies:
1. High-performance Order Storage will not be enabled for any shop by default for now. This means end users can continue to safely upgrade WooCommerce.
2. Shops can opt-in to use the HPOS (please see the section _How to enable HPOS_ below). When they do so, they can enable *data synchronization* (or _sync_, in short) to provide an additional layer of compatibility:
1. Whenever we write data to HPOS tables, we will write the same to the posts table as well. This way, if there are incompatible plugins that still access data directly in the posts table, they will read updated data from the posts table.
2. Similarly, when we read orders from custom order tables, we will read the same order from the posts table as well and compare them. If there is any difference, then itd imply a direct write to posts table and WC will update the order in the custom tables accordingly.
It is important to keep the sync in place until all plugins and custom code are compatible with the new HPOS. Any performance penalties will be temporary, resulting from additional inserts present during the sync.
## Switching data source
Additionally, you can switch from using HPOS to posts tables manually if you see an issue with the new tables. To switch back, change the Data store for orders setting in the WC > Settings > Advanced > Custom data stores:
Please note this setting is only available once the HPOS feature is enabled (please see the section How to enable HPOS below).
## Possibility of downgrading
You can also downgrade the WooCommerce version to a version before the HPOS feature was launched if the sync was enabled. This is possible because when sync is enabled, we propagate any changes to orders both to HPOS tables and to wp_posts and wp_postmeta tables.
# How to enable HPOS
## Easy way
1. Set up your dev environment according to the [general dev guide](https://github.com/woocommerce/woocommerce/tree/trunk/plugins/woocommerce#readme)
2. Grab and install the zip package from the [HPOS feature release page](https://github.com/woocommerce/woocommerce/releases/tag/feature-custom-order-table)
3. Create the new tables by running _Create custom order tables_ tool from WC > Status > Tools (once you have them created the tool will change from Create to Delete the tables to allow recreating the tables from scratch).
4. If you have created the tables previously, please delete the old ones and create new ones again, as the database structure has changed a couple of times.
5. Now you can use settings in WC > Settings > Advanced > Custom data stores to switch the data source from posts table to custom table for orders and change other related settings.
## Initial sync of orders
When getting started and when testing out HPOS, you may occasionally find that orders are waiting to be synced. For instance, within **WooCommerce > Settings > Advanced > Custom Data Stores** you may see a message like, _"There are 1234 orders pending sync!"_
<img width="827" alt="cot-orders-pending-sync" src="https://user-images.githubusercontent.com/3594411/184413815-73c057b3-7551-4813-a228-cdeddd856ca7.png">
You will almost certainly see this when you are getting started with HPOS, but depending on your testing patterns you may also see it later on. You can either wait for orders to get synchronized via Action Scheduler, or speed things up via WP CLI command:
```sh
wp wc cot sync
```
## Using GH /// TODO: do we still want to advertise this method?
Use the same steps as above, but instead of downloading the zip package (step 2), checkout the `cot-main` branch from the monorepo and build WC using `pnpm -- turbo run build --filter=woocommerce`. Then mount/link the `plugins/woocommerce` subdirectory to your WP plugins directory.
# Supporting High-performance Order Storage
While the backward compatibility policies make it easier for merchants to use the project, for extension developers this means that you have to support both Posts and HPOS for a while.
To help with this, we have provided a few guidelines for extension developers to follow:
## Detecting whether HPOS tables are being used in the store:
While the WooCommerce CRUD API will let you support both posts and custom tables without additional effort most of the time, in some cases (like when you are writing a SQL query for better performance) you would want to know whether the store is using HPOS tables or not. In this case, you can use the following pattern:
```php
use Automattic\WooCommerce\Utilities\woocommerce;
if ( OrderUtil::custom_orders_table_usage_is_enabled() ) {
// HPOS usage is enabled.
} else {
// Traditional CPT-based orders are in use.
}
```
## Auditing the code base for direct DB access usage
To support HPOS tables, a good place to start is to audit your code base for direct DB access and usage of WordPress APIs that shouldn't be used to work with orders anymore. You can search using the following regexp to perform this audit:
```regexp
get_post|get_post_field|get_post_status|get_post_type|get_post_type_object|get_posts|metadata_exists|get_post_meta|get_metadata|get_metadata_raw|get_metadata_default|get_metadata_by_mid|wp_insert_post|add_metadata|add_post_meta|wp_update_post|update_post_meta|update_metadata|update_metadata_by_mid|delete_metadata|delete_post_meta|delete_metadata_by_mid|delete_post_meta_by_key|wp_delete_post|wp_trash_post|wp_untrash_post|wp_transition_post_status|clean_post_cache|update_post_caches|update_postmeta_cache|post_exists
```
Please note that you will find lots of false positives, but this regular expression is quite thorough and should catch most of the true positives.
Search for the above regular expression in your source code, and:
1. Go through the matches one by one and check whether the occurrence relates to an order. Most of the matches will probably be false positives i.e. they won't be related to orders.
2. If you see one of the matches are directly accessing or modifying order data, you will need to change it to use WooCommerce's CRUD API instead.
## APIs for getting/setting posts and postmeta
Any code getting posts directly can be converted to a `wc_get_order` call instead:
```php
// Instead of
$post = get_post( $post_id ); // returns WP_Post object.
// use
$order = wc_get_order( $post_id ); // returns WC_Order object.
```
For interacting with metadata, use the `update_`/`add_`/`delete_metadata` methods on the order object, followed by a `save` call. WooCommerce will take care of figuring out which tables are active, and saving data in appropriate locations.
```php
// Instead of following update/add/delete methods, use:
update_post_meta( $post_id, $meta_key_1, $meta_value_1 );
add_post_meta( $post_id, $meta_key_2, $meta_value_2 );
delete_post_meta( $post_id, $meta_key_3, $meta_value_3 );
// use
$order = wc_get_order( $post_id );
$order->update_meta_data( $meta_key_1, $meta_value_1 );
$order->add_meta_data( $meta_key_2, $meta_value_2 );
$order->delete_meta_data( $meta_key_3, $meta_value_3 );
$order->save();
```
💡 Calling the `save()` method is a relatively expensive operation, so you may wish to avoid calling it more times than necessary (for example, if you know it will be called later in the same flow, you may wish to avoid additional earlier calls when operating on the same object).
When getting exact type of an order, or checking if given ID is an order, you can use methods from `OrderUtil` class.
```php
// Pattern to check when an ID is an order
shop_order === get_post_type( $post_id ); // or
in_array( get_post_type( $post_type ), wc_get_order_types() );
// replace with:
use Automattic\WooCommerce\Utilities\OrderUtil;
shop_order === OrderUtil::get_order_type( $post_id ); // or
OrderUtil::is_order( $post_id, wc_get_order_types() );
```
## Audit for order administration screen functions
As WC can't use the WordPress-provided post list and post edit screens, we have also added new screens for order administration. These screens are very similar to the one you see in the WooCommerce admin currently (except for the fact that they are using HPOS tables). You can use the following regular expression to perform this audit:
```regexp
post_updated_messages|do_meta_boxes|enter_title_here|edit_form_before_permalink|edit_form_after_title|edit_form_after_editor|submitpage_box|submitpost_box|edit_form_advanced|dbx_post_sidebar|manage_shop_order_posts_columns|manage_shop_order_posts_custom_column
```
You will see a lot of false positives here as well. However, if you do encounter a usage where these methods are called for the order screen then to upgrade them to HPOS, the following changes have to be done:
Instead of a `$post` object of the `WP_Post` class, you will need to use an `$order` object of the `WC_Order` class. If its a filter or an action, then we will implement a similar filter in the new WooCommerce screen as well and instead of passing the post object, it will accept a WC_Order object instead.
The following snippet shows a way to add meta boxes to the legacy order editor screen when legacy orders are in effect, and to the new HPOS-powered editor screen otherwise:
```php
use Automattic\WooCommerce\Internal\DataStores\Orders\CustomOrdersTableController;
add_action( 'add_meta_boxes', 'xyz_metabox' );
function add_xyz_metabox() {
$screen = wc_get_container()->get( CustomOrdersTableController::class )->custom_orders_table_usage_is_enabled()
? wc_get_page_screen_id( 'shop-order' )
: 'shop_order';
add_meta_box(
'xyz',
'Custom Meta Box',
render_xyz_metabox,
$screen,
'side',
'high'
);
}
```
The above will also change the parameter passed to the metabox to order. So in your metaboxes, you would have to account for both a post or order object that may be passed. We recommend fetching the order object and working with it completely instead of the passed parameter.
```php
function render_xyz_metabox( $post_or_order_object ) {
$order = ( $post_or_order_object instanceof WP_Post ) ? wc_get_order( $post_or_order_object->ID ) : $post_or_order_object;
... rest of the code. $post_or_order_object should not be used directly below this point.
}
```