woocommerce/plugins/woocommerce-blocks/.github/workflows/php-js-e2e-tests.yml

147 lines
5.3 KiB
YAML
Raw Normal View History

name: E2E tests
Create GitHub actions for automated tests (https://github.com/woocommerce/woocommerce-blocks/pull/3544) * Create end-to-end-tests.yml for E2E testing action * Change actions to run on push instead of PR * Install libstdc++-4.9-dev on E2E tests action * Add correct apt repository for libstc++ * Reconfigure apt-get commands for installing libstdc++ * Remove accidental inclusion of Travis config from E2E tests action * Install libkrb5-dev as part of e2e test action * Run apt commands as sudo * Install gutenberg plugin and e2e testutils * Add environment variables to E2E tests * Rename action and add further config for composer and wp-env * Rename workflow * Add jobs for WP 5.6, 5.5, and 5.4 * Fix YML indentation * Apply 767 permissions to wp-env directory * Run chmod as sudo * Comment 5.6 and 5.6 with GB out to test 5.4 more easily * Remove WP install job, since it should run on each step * Change order of wp-env start and chmod * Reorder commands for 5.4 job * Try running 5.4 tests in isolation * Reenable tests for all WP versions * Move commands out of bash script into a series of commands * Fix indentation on 5.5 job * Re-enable libkrb5-dev install * Clean wp-env before each run & upgrade wp-env to 3.0.0 * Update lock file for wp-env@3.0.0 * Reorder wp-env start and clean commands * Reorder wp-env permissions commands * Reorder wp-env permissions setup for all jobs * Reorder wp-env permissions setup for 5.5 and 5.4 * Ensure correct order for env setup and flush permalinks twice * Update jest snapshots * Remove rewrite flush command from yml * Remove npm build from every step and try it just at the start * Set correct e2e build script * Add jobs for PHP 8 * Specify PHP 8 minor version * Run PHP 8 jobs first * Remove PHP 8 jobs * Add JS Unit tests job * Remove js-unit-tests.js workflow * Remove composer install from every step, add it to its own step * Cache composer files * Bust npm cache to test nodegit * Rename npm cache * Renove npm cache entirely * Revert "Renove npm cache entirely" This reverts commit d6fac6a6ebd9162e48f64daaa8c971320756579e. * Rename npm cache back to how it was * Fix yml indentation * Remove echo from composer cache step * Revert back to composer example * Add PHP Unit tests to workflow * Add PHP Unit tests to workflow * Rename E2E tests workflow and file * deliberately break e2e and unit tests to test workflow 👺 * fix php test, should see e2e fail * revert broken e2e test * Change steps into jobs, rename workflow * Remove Travis workflow file * Add all necessary setup steps to each job * Rename Setup job and remove dependency * Add individual jobs for each E2E test environment * Add npm install and build to setup job * YML syntax fix * Remove error-causing chmod * Rename blocks.ini setup step Fixes a typographical error. * Get the latest stable version of WooCommerce for PHPUnit testing * Add PHP8.0 and PHP5.6 Unit tests * Run composer update for PHP 5.6 and PHP 8.0 * Revert "Run composer update for PHP 5.6 and PHP 8.0" This reverts commit 4f90522d0b52b7a8b9e896e9c783795be9dc5399. * Revert "Add PHP8.0 and PHP5.6 Unit tests" This reverts commit 66e317dec4af6e3a2ac6f78b6efd050e7fc5aa8e. Co-authored-by: Rua Haszard <rua.haszard@automattic.com>
2021-01-05 09:27:22 +00:00
on:
push:
branches: [trunk]
pull_request:
Create GitHub actions for automated tests (https://github.com/woocommerce/woocommerce-blocks/pull/3544) * Create end-to-end-tests.yml for E2E testing action * Change actions to run on push instead of PR * Install libstdc++-4.9-dev on E2E tests action * Add correct apt repository for libstc++ * Reconfigure apt-get commands for installing libstdc++ * Remove accidental inclusion of Travis config from E2E tests action * Install libkrb5-dev as part of e2e test action * Run apt commands as sudo * Install gutenberg plugin and e2e testutils * Add environment variables to E2E tests * Rename action and add further config for composer and wp-env * Rename workflow * Add jobs for WP 5.6, 5.5, and 5.4 * Fix YML indentation * Apply 767 permissions to wp-env directory * Run chmod as sudo * Comment 5.6 and 5.6 with GB out to test 5.4 more easily * Remove WP install job, since it should run on each step * Change order of wp-env start and chmod * Reorder commands for 5.4 job * Try running 5.4 tests in isolation * Reenable tests for all WP versions * Move commands out of bash script into a series of commands * Fix indentation on 5.5 job * Re-enable libkrb5-dev install * Clean wp-env before each run & upgrade wp-env to 3.0.0 * Update lock file for wp-env@3.0.0 * Reorder wp-env start and clean commands * Reorder wp-env permissions commands * Reorder wp-env permissions setup for all jobs * Reorder wp-env permissions setup for 5.5 and 5.4 * Ensure correct order for env setup and flush permalinks twice * Update jest snapshots * Remove rewrite flush command from yml * Remove npm build from every step and try it just at the start * Set correct e2e build script * Add jobs for PHP 8 * Specify PHP 8 minor version * Run PHP 8 jobs first * Remove PHP 8 jobs * Add JS Unit tests job * Remove js-unit-tests.js workflow * Remove composer install from every step, add it to its own step * Cache composer files * Bust npm cache to test nodegit * Rename npm cache * Renove npm cache entirely * Revert "Renove npm cache entirely" This reverts commit d6fac6a6ebd9162e48f64daaa8c971320756579e. * Rename npm cache back to how it was * Fix yml indentation * Remove echo from composer cache step * Revert back to composer example * Add PHP Unit tests to workflow * Add PHP Unit tests to workflow * Rename E2E tests workflow and file * deliberately break e2e and unit tests to test workflow 👺 * fix php test, should see e2e fail * revert broken e2e test * Change steps into jobs, rename workflow * Remove Travis workflow file * Add all necessary setup steps to each job * Rename Setup job and remove dependency * Add individual jobs for each E2E test environment * Add npm install and build to setup job * YML syntax fix * Remove error-causing chmod * Rename blocks.ini setup step Fixes a typographical error. * Get the latest stable version of WooCommerce for PHPUnit testing * Add PHP8.0 and PHP5.6 Unit tests * Run composer update for PHP 5.6 and PHP 8.0 * Revert "Run composer update for PHP 5.6 and PHP 8.0" This reverts commit 4f90522d0b52b7a8b9e896e9c783795be9dc5399. * Revert "Add PHP8.0 and PHP5.6 Unit tests" This reverts commit 66e317dec4af6e3a2ac6f78b6efd050e7fc5aa8e. Co-authored-by: Rua Haszard <rua.haszard@automattic.com>
2021-01-05 09:27:22 +00:00
jobs:
JSE2EWithGutenberg:
name: JavaScript E2E Tests (WP latest with Gutenberg plugin)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Cache node modules
uses: actions/cache@v2
env:
cache-name: cache-node-modules
with:
# npm cache files are stored in `~/.npm` on Linux/macOS
path: ~/.npm
key: ${{ runner.OS }}-build-${{ secrets.CACHE_VERSION }}-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.OS }}-build-${{ secrets.CACHE_VERSION }}-${{ env.cache-name }}-
${{ runner.OS }}-build-${{ secrets.CACHE_VERSION }}-
Fix 404 and 500 errors in E2E test logs, editor and front-end. (https://github.com/woocommerce/woocommerce-blocks/pull/5989) * Change atomic blocks to not use custom webpack build paths we want to always load lazy loaded components from their default paths to avoid having to handle importing components in different ways for regular and atomic component packages * Add footer parts to test themes The Site Editor is expecting the footer parts to be there or it will try to load them anyways and throw a 404. It's not breaking, but it's polluting the console. * Use REST API to tear down the templates Previously, we used a util called `trashAllPosts` which navigated to the post UI and deleted all the posts to tear down any side-effects of template editing tests. However, with a [recent change](https://github.com/WordPress/wordpress-develop/commit/14e20f72b568cfb5a85f57c77eaa538c17a5f302), WP Core removed the UI for those and that made our tests meet a 500 error. Using the REST API should also make everything faster. * Remove deprecated pupeteer waitFor usage was still present in attribute-filter.test.js * Update package-lock.json * pin package versions * Unify all atomic blocks to register on php side * Remove Atomic Blcoks chunk_translation handling from AllBlocks Before it was responsible for enabling translations for all the atomic blocks * Add per atomic block chunk_translation registration * update @wordpress/e2e-test-utils to 6.0.2 * add optional puppeteer * pin workflows node version to 16.13.2 * upgrade package-lock * upgrade package-lock * set react and react-dom as peerDeps * remove atomic block registration Co-authored-by: Lucio Giannotta <lucio.giannotta@a8c.com> Co-authored-by: Luigi <gigitux@gmail.com>
2022-03-10 10:00:23 +00:00
- name: Use Node.js 16.13.2
uses: actions/setup-node@v1
with:
Fix 404 and 500 errors in E2E test logs, editor and front-end. (https://github.com/woocommerce/woocommerce-blocks/pull/5989) * Change atomic blocks to not use custom webpack build paths we want to always load lazy loaded components from their default paths to avoid having to handle importing components in different ways for regular and atomic component packages * Add footer parts to test themes The Site Editor is expecting the footer parts to be there or it will try to load them anyways and throw a 404. It's not breaking, but it's polluting the console. * Use REST API to tear down the templates Previously, we used a util called `trashAllPosts` which navigated to the post UI and deleted all the posts to tear down any side-effects of template editing tests. However, with a [recent change](https://github.com/WordPress/wordpress-develop/commit/14e20f72b568cfb5a85f57c77eaa538c17a5f302), WP Core removed the UI for those and that made our tests meet a 500 error. Using the REST API should also make everything faster. * Remove deprecated pupeteer waitFor usage was still present in attribute-filter.test.js * Update package-lock.json * pin package versions * Unify all atomic blocks to register on php side * Remove Atomic Blcoks chunk_translation handling from AllBlocks Before it was responsible for enabling translations for all the atomic blocks * Add per atomic block chunk_translation registration * update @wordpress/e2e-test-utils to 6.0.2 * add optional puppeteer * pin workflows node version to 16.13.2 * upgrade package-lock * upgrade package-lock * set react and react-dom as peerDeps * remove atomic block registration Co-authored-by: Lucio Giannotta <lucio.giannotta@a8c.com> Co-authored-by: Luigi <gigitux@gmail.com>
2022-03-10 10:00:23 +00:00
node-version: 16.13.2
- name: Npm install and build
run: |
npm ci
FORCE_REDUCED_MOTION=true npm run build:e2e-test
- name: blocks.ini setup
run: |
echo -e 'woocommerce_blocks_phase = 3\nwoocommerce_blocks_env = tests' > blocks.ini
- name: Get Composer Cache Directory
id: composer-cache
run: |
echo "::set-output name=dir::$(composer config cache-files-dir)"
- uses: actions/cache@v2
with:
path: ${{ steps.composer-cache.outputs.dir }}
key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }}
restore-keys: |
${{ runner.os }}-composer-
- name: Set up PHP
uses: shivammathur/setup-php@v2
with:
php-version: 7.4
coverage: none
tools: composer
- name: Composer install
run: |
composer install
- name: E2E Tests (WP latest with Gutenberg plugin)
env:
WOOCOMMERCE_BLOCKS_PHASE: 3
run: |
chmod -R 767 ./ #needed for permissions issues
JSON='{"plugins": ["https://downloads.wordpress.org/plugin/woocommerce.latest-stable.zip","https://github.com/WP-API/Basic-Auth/archive/master.zip","https://downloads.wordpress.org/plugin/gutenberg.latest-stable.zip", "."] }'
echo $JSON > .wp-env.override.json
npm run wp-env start
npm run wp-env clean all
npm run test:e2e
- name: Upload artifacts on failure
if: ${{ failure() }}
uses: actions/upload-artifact@v2
with:
Critical flow: Can see correct tax (https://github.com/woocommerce/woocommerce-blocks/pull/5900) * Tax is displayed correctly e2e test * Add extra utils to shopper * add single e2e test script oin package.json * Feedback changes from Niels * Test tax is correct on summary page * Check if CI test passes * Longer timeout for tax test * change timeout in jest config * increate jest timeout * Test * Test * Test * setTimeout outside of tests to 120000 * Address raluca's feedback and timeout everywheregaa * Increase timeout on for search box * set jest timeout to 120000 * Raluca's suggestions * Debug * Increase search timeout to 5000 * Only run Tax e2e tests in CI for quicker debug - REMOVE THIS * use waitforSelector * 30s delay for toMatchElement * Let'shope this works * Remove toMatchElement * waitForSelector * Run debug test only * Debug tax test * debug taxes * shopper using toMatchElement again * Remove the 30000 timeout delay from toMatchElement * Run all tests but only once * Remove tax test * Revert "Remove tax test" This reverts commit 7db34120e1e91f4fd26514fdb9525cac2a6066bb. * Delete the minicart test * Revert "Revert "Remove tax test"" This reverts commit 31dd654e52f37fcd02ccd25336958248bef495c6. * Revert "Delete the minicart test" This reverts commit 5595f0834a975d097a300bc0c7cbf643caa8b764. * tax test * block addToCart * Put back block functions * Refactor block.addToCart * mini-cart e2e test uses shopper.block.emptyCart() * only run shopper tests in CI * Run frontend and shopper tests and change waitFor -> waitForTimeout * Run frontend tests in CI * Run CI e2e tests again * Skip legacy template blocks * Skip legacy template test but run all others * Switch to storefront instead of twentytwentyone after legacy-template-blocks tests finish * Tidy up * fix typo * Fix ESLint issue Co-authored-by: Saad Tarhi <saad.trh@gmail.com> Co-authored-by: Saad Tarhi <saad.tarhi@automattic.com>
2022-03-08 14:50:16 +00:00
name: e2e-with-gutenberg-test-report
path: reports/e2e
JSE2ETests:
name: JavaScript E2E Tests (latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Cache node modules
uses: actions/cache@v2
env:
cache-name: cache-node-modules
with:
# npm cache files are stored in `~/.npm` on Linux/macOS
path: ~/.npm
key: ${{ runner.OS }}-build-${{ secrets.CACHE_VERSION }}-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.OS }}-build-${{ secrets.CACHE_VERSION }}-${{ env.cache-name }}-
${{ runner.OS }}-build-${{ secrets.CACHE_VERSION }}-
Fix 404 and 500 errors in E2E test logs, editor and front-end. (https://github.com/woocommerce/woocommerce-blocks/pull/5989) * Change atomic blocks to not use custom webpack build paths we want to always load lazy loaded components from their default paths to avoid having to handle importing components in different ways for regular and atomic component packages * Add footer parts to test themes The Site Editor is expecting the footer parts to be there or it will try to load them anyways and throw a 404. It's not breaking, but it's polluting the console. * Use REST API to tear down the templates Previously, we used a util called `trashAllPosts` which navigated to the post UI and deleted all the posts to tear down any side-effects of template editing tests. However, with a [recent change](https://github.com/WordPress/wordpress-develop/commit/14e20f72b568cfb5a85f57c77eaa538c17a5f302), WP Core removed the UI for those and that made our tests meet a 500 error. Using the REST API should also make everything faster. * Remove deprecated pupeteer waitFor usage was still present in attribute-filter.test.js * Update package-lock.json * pin package versions * Unify all atomic blocks to register on php side * Remove Atomic Blcoks chunk_translation handling from AllBlocks Before it was responsible for enabling translations for all the atomic blocks * Add per atomic block chunk_translation registration * update @wordpress/e2e-test-utils to 6.0.2 * add optional puppeteer * pin workflows node version to 16.13.2 * upgrade package-lock * upgrade package-lock * set react and react-dom as peerDeps * remove atomic block registration Co-authored-by: Lucio Giannotta <lucio.giannotta@a8c.com> Co-authored-by: Luigi <gigitux@gmail.com>
2022-03-10 10:00:23 +00:00
- name: Use Node.js 16.13.2
uses: actions/setup-node@v1
with:
Fix 404 and 500 errors in E2E test logs, editor and front-end. (https://github.com/woocommerce/woocommerce-blocks/pull/5989) * Change atomic blocks to not use custom webpack build paths we want to always load lazy loaded components from their default paths to avoid having to handle importing components in different ways for regular and atomic component packages * Add footer parts to test themes The Site Editor is expecting the footer parts to be there or it will try to load them anyways and throw a 404. It's not breaking, but it's polluting the console. * Use REST API to tear down the templates Previously, we used a util called `trashAllPosts` which navigated to the post UI and deleted all the posts to tear down any side-effects of template editing tests. However, with a [recent change](https://github.com/WordPress/wordpress-develop/commit/14e20f72b568cfb5a85f57c77eaa538c17a5f302), WP Core removed the UI for those and that made our tests meet a 500 error. Using the REST API should also make everything faster. * Remove deprecated pupeteer waitFor usage was still present in attribute-filter.test.js * Update package-lock.json * pin package versions * Unify all atomic blocks to register on php side * Remove Atomic Blcoks chunk_translation handling from AllBlocks Before it was responsible for enabling translations for all the atomic blocks * Add per atomic block chunk_translation registration * update @wordpress/e2e-test-utils to 6.0.2 * add optional puppeteer * pin workflows node version to 16.13.2 * upgrade package-lock * upgrade package-lock * set react and react-dom as peerDeps * remove atomic block registration Co-authored-by: Lucio Giannotta <lucio.giannotta@a8c.com> Co-authored-by: Luigi <gigitux@gmail.com>
2022-03-10 10:00:23 +00:00
node-version: 16.13.2
- name: Npm install and build
run: |
npm ci
FORCE_REDUCED_MOTION=true npm run build:e2e-test
- name: blocks.ini setup
run: |
echo -e 'woocommerce_blocks_phase = 3\nwoocommerce_blocks_env = tests' > blocks.ini
- name: Get Composer Cache Directory
id: composer-cache
run: |
echo "::set-output name=dir::$(composer config cache-files-dir)"
- uses: actions/cache@v2
with:
path: ${{ steps.composer-cache.outputs.dir }}
key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }}
restore-keys: |
${{ runner.os }}-composer-
- name: Set up PHP
uses: shivammathur/setup-php@v2
with:
php-version: 7.4
coverage: none
tools: composer
- name: Composer install
run: |
composer install
- name: E2E Tests (WP latest)
env:
WOOCOMMERCE_BLOCKS_PHASE: 3
run: |
npm run wp-env start
npm run wp-env clean all
npm run test:e2e
Critical flow: Can see correct tax (https://github.com/woocommerce/woocommerce-blocks/pull/5900) * Tax is displayed correctly e2e test * Add extra utils to shopper * add single e2e test script oin package.json * Feedback changes from Niels * Test tax is correct on summary page * Check if CI test passes * Longer timeout for tax test * change timeout in jest config * increate jest timeout * Test * Test * Test * setTimeout outside of tests to 120000 * Address raluca's feedback and timeout everywheregaa * Increase timeout on for search box * set jest timeout to 120000 * Raluca's suggestions * Debug * Increase search timeout to 5000 * Only run Tax e2e tests in CI for quicker debug - REMOVE THIS * use waitforSelector * 30s delay for toMatchElement * Let'shope this works * Remove toMatchElement * waitForSelector * Run debug test only * Debug tax test * debug taxes * shopper using toMatchElement again * Remove the 30000 timeout delay from toMatchElement * Run all tests but only once * Remove tax test * Revert "Remove tax test" This reverts commit 7db34120e1e91f4fd26514fdb9525cac2a6066bb. * Delete the minicart test * Revert "Revert "Remove tax test"" This reverts commit 31dd654e52f37fcd02ccd25336958248bef495c6. * Revert "Delete the minicart test" This reverts commit 5595f0834a975d097a300bc0c7cbf643caa8b764. * tax test * block addToCart * Put back block functions * Refactor block.addToCart * mini-cart e2e test uses shopper.block.emptyCart() * only run shopper tests in CI * Run frontend and shopper tests and change waitFor -> waitForTimeout * Run frontend tests in CI * Run CI e2e tests again * Skip legacy template blocks * Skip legacy template test but run all others * Switch to storefront instead of twentytwentyone after legacy-template-blocks tests finish * Tidy up * fix typo * Fix ESLint issue Co-authored-by: Saad Tarhi <saad.trh@gmail.com> Co-authored-by: Saad Tarhi <saad.tarhi@automattic.com>
2022-03-08 14:50:16 +00:00
- name: Upload artifacts on failure
if: ${{ failure() }}
uses: actions/upload-artifact@v2
with:
Critical flow: Can see correct tax (https://github.com/woocommerce/woocommerce-blocks/pull/5900) * Tax is displayed correctly e2e test * Add extra utils to shopper * add single e2e test script oin package.json * Feedback changes from Niels * Test tax is correct on summary page * Check if CI test passes * Longer timeout for tax test * change timeout in jest config * increate jest timeout * Test * Test * Test * setTimeout outside of tests to 120000 * Address raluca's feedback and timeout everywheregaa * Increase timeout on for search box * set jest timeout to 120000 * Raluca's suggestions * Debug * Increase search timeout to 5000 * Only run Tax e2e tests in CI for quicker debug - REMOVE THIS * use waitforSelector * 30s delay for toMatchElement * Let'shope this works * Remove toMatchElement * waitForSelector * Run debug test only * Debug tax test * debug taxes * shopper using toMatchElement again * Remove the 30000 timeout delay from toMatchElement * Run all tests but only once * Remove tax test * Revert "Remove tax test" This reverts commit 7db34120e1e91f4fd26514fdb9525cac2a6066bb. * Delete the minicart test * Revert "Revert "Remove tax test"" This reverts commit 31dd654e52f37fcd02ccd25336958248bef495c6. * Revert "Delete the minicart test" This reverts commit 5595f0834a975d097a300bc0c7cbf643caa8b764. * tax test * block addToCart * Put back block functions * Refactor block.addToCart * mini-cart e2e test uses shopper.block.emptyCart() * only run shopper tests in CI * Run frontend and shopper tests and change waitFor -> waitForTimeout * Run frontend tests in CI * Run CI e2e tests again * Skip legacy template blocks * Skip legacy template test but run all others * Switch to storefront instead of twentytwentyone after legacy-template-blocks tests finish * Tidy up * fix typo * Fix ESLint issue Co-authored-by: Saad Tarhi <saad.trh@gmail.com> Co-authored-by: Saad Tarhi <saad.tarhi@automattic.com>
2022-03-08 14:50:16 +00:00
name: e2e-test-report
path: reports/e2e