woocommerce/plugins/woocommerce-blocks/.github/workflows/php-js-e2e-tests.yml

169 lines
6.5 KiB
YAML
Raw Normal View History

name: E2E tests
Create GitHub actions for automated tests (https://github.com/woocommerce/woocommerce-blocks/pull/3544) * Create end-to-end-tests.yml for E2E testing action * Change actions to run on push instead of PR * Install libstdc++-4.9-dev on E2E tests action * Add correct apt repository for libstc++ * Reconfigure apt-get commands for installing libstdc++ * Remove accidental inclusion of Travis config from E2E tests action * Install libkrb5-dev as part of e2e test action * Run apt commands as sudo * Install gutenberg plugin and e2e testutils * Add environment variables to E2E tests * Rename action and add further config for composer and wp-env * Rename workflow * Add jobs for WP 5.6, 5.5, and 5.4 * Fix YML indentation * Apply 767 permissions to wp-env directory * Run chmod as sudo * Comment 5.6 and 5.6 with GB out to test 5.4 more easily * Remove WP install job, since it should run on each step * Change order of wp-env start and chmod * Reorder commands for 5.4 job * Try running 5.4 tests in isolation * Reenable tests for all WP versions * Move commands out of bash script into a series of commands * Fix indentation on 5.5 job * Re-enable libkrb5-dev install * Clean wp-env before each run & upgrade wp-env to 3.0.0 * Update lock file for wp-env@3.0.0 * Reorder wp-env start and clean commands * Reorder wp-env permissions commands * Reorder wp-env permissions setup for all jobs * Reorder wp-env permissions setup for 5.5 and 5.4 * Ensure correct order for env setup and flush permalinks twice * Update jest snapshots * Remove rewrite flush command from yml * Remove npm build from every step and try it just at the start * Set correct e2e build script * Add jobs for PHP 8 * Specify PHP 8 minor version * Run PHP 8 jobs first * Remove PHP 8 jobs * Add JS Unit tests job * Remove js-unit-tests.js workflow * Remove composer install from every step, add it to its own step * Cache composer files * Bust npm cache to test nodegit * Rename npm cache * Renove npm cache entirely * Revert "Renove npm cache entirely" This reverts commit d6fac6a6ebd9162e48f64daaa8c971320756579e. * Rename npm cache back to how it was * Fix yml indentation * Remove echo from composer cache step * Revert back to composer example * Add PHP Unit tests to workflow * Add PHP Unit tests to workflow * Rename E2E tests workflow and file * deliberately break e2e and unit tests to test workflow 👺 * fix php test, should see e2e fail * revert broken e2e test * Change steps into jobs, rename workflow * Remove Travis workflow file * Add all necessary setup steps to each job * Rename Setup job and remove dependency * Add individual jobs for each E2E test environment * Add npm install and build to setup job * YML syntax fix * Remove error-causing chmod * Rename blocks.ini setup step Fixes a typographical error. * Get the latest stable version of WooCommerce for PHPUnit testing * Add PHP8.0 and PHP5.6 Unit tests * Run composer update for PHP 5.6 and PHP 8.0 * Revert "Run composer update for PHP 5.6 and PHP 8.0" This reverts commit 4f90522d0b52b7a8b9e896e9c783795be9dc5399. * Revert "Add PHP8.0 and PHP5.6 Unit tests" This reverts commit 66e317dec4af6e3a2ac6f78b6efd050e7fc5aa8e. Co-authored-by: Rua Haszard <rua.haszard@automattic.com>
2021-01-05 09:27:22 +00:00
on:
push:
branches: [trunk]
pull_request:
Create GitHub actions for automated tests (https://github.com/woocommerce/woocommerce-blocks/pull/3544) * Create end-to-end-tests.yml for E2E testing action * Change actions to run on push instead of PR * Install libstdc++-4.9-dev on E2E tests action * Add correct apt repository for libstc++ * Reconfigure apt-get commands for installing libstdc++ * Remove accidental inclusion of Travis config from E2E tests action * Install libkrb5-dev as part of e2e test action * Run apt commands as sudo * Install gutenberg plugin and e2e testutils * Add environment variables to E2E tests * Rename action and add further config for composer and wp-env * Rename workflow * Add jobs for WP 5.6, 5.5, and 5.4 * Fix YML indentation * Apply 767 permissions to wp-env directory * Run chmod as sudo * Comment 5.6 and 5.6 with GB out to test 5.4 more easily * Remove WP install job, since it should run on each step * Change order of wp-env start and chmod * Reorder commands for 5.4 job * Try running 5.4 tests in isolation * Reenable tests for all WP versions * Move commands out of bash script into a series of commands * Fix indentation on 5.5 job * Re-enable libkrb5-dev install * Clean wp-env before each run & upgrade wp-env to 3.0.0 * Update lock file for wp-env@3.0.0 * Reorder wp-env start and clean commands * Reorder wp-env permissions commands * Reorder wp-env permissions setup for all jobs * Reorder wp-env permissions setup for 5.5 and 5.4 * Ensure correct order for env setup and flush permalinks twice * Update jest snapshots * Remove rewrite flush command from yml * Remove npm build from every step and try it just at the start * Set correct e2e build script * Add jobs for PHP 8 * Specify PHP 8 minor version * Run PHP 8 jobs first * Remove PHP 8 jobs * Add JS Unit tests job * Remove js-unit-tests.js workflow * Remove composer install from every step, add it to its own step * Cache composer files * Bust npm cache to test nodegit * Rename npm cache * Renove npm cache entirely * Revert "Renove npm cache entirely" This reverts commit d6fac6a6ebd9162e48f64daaa8c971320756579e. * Rename npm cache back to how it was * Fix yml indentation * Remove echo from composer cache step * Revert back to composer example * Add PHP Unit tests to workflow * Add PHP Unit tests to workflow * Rename E2E tests workflow and file * deliberately break e2e and unit tests to test workflow 👺 * fix php test, should see e2e fail * revert broken e2e test * Change steps into jobs, rename workflow * Remove Travis workflow file * Add all necessary setup steps to each job * Rename Setup job and remove dependency * Add individual jobs for each E2E test environment * Add npm install and build to setup job * YML syntax fix * Remove error-causing chmod * Rename blocks.ini setup step Fixes a typographical error. * Get the latest stable version of WooCommerce for PHPUnit testing * Add PHP8.0 and PHP5.6 Unit tests * Run composer update for PHP 5.6 and PHP 8.0 * Revert "Run composer update for PHP 5.6 and PHP 8.0" This reverts commit 4f90522d0b52b7a8b9e896e9c783795be9dc5399. * Revert "Add PHP8.0 and PHP5.6 Unit tests" This reverts commit 66e317dec4af6e3a2ac6f78b6efd050e7fc5aa8e. Co-authored-by: Rua Haszard <rua.haszard@automattic.com>
2021-01-05 09:27:22 +00:00
jobs:
JSE2EWithGutenberg:
strategy:
fail-fast: false
matrix:
part: [1, 2, 3, 4, 5]
name: JavaScript E2E Tests (WP latest with Gutenberg plugin)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Cache node_modules
id: cache-node-modules
uses: actions/cache@v3
env:
cache-name: cache-node-modules
with:
path: node_modules
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
- name: Setup node version and npm cache
uses: actions/setup-node@v3
with:
node-version-file: '.nvmrc'
cache: 'npm'
- name: Install Node Dependencies
if: steps.cache-node-modules.outputs.cache-hit != 'true'
run: npm ci --no-optional
- name: Build Assets
run: FORCE_REDUCED_MOTION=true npm run build:e2e-test
- name: blocks.ini setup
run: |
echo -e 'woocommerce_blocks_phase = 3\nwoocommerce_blocks_env = tests' > blocks.ini
- name: Get Composer Cache Directory
id: composer-cache
run: |
echo "dir=$(composer config cache-files-dir)" >> $GITHUB_OUTPUT
- uses: actions/cache@v3
with:
path: ${{ steps.composer-cache.outputs.dir }}
key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }}
restore-keys: |
${{ runner.os }}-composer-
- name: Set up PHP
uses: shivammathur/setup-php@v2
with:
php-version: '8.0'
coverage: none
tools: composer
- name: Composer install
run: |
composer install
- name: E2E Tests (WP latest with Gutenberg plugin)
env:
WOOCOMMERCE_BLOCKS_PHASE: 3
GUTENBERG_EDITOR_CONTEXT: 'gutenberg'
run: |
node ./bin/wp-env-with-gutenberg.js
npm run wp-env start
npm run wp-env:config && npx cross-env NODE_CONFIG_DIR=tests/e2e/config wp-scripts test-e2e --config tests/e2e/config/jest.config.js --listTests > ~/.jest-e2e-tests
npx cross-env JEST_PUPPETEER_CONFIG=tests/e2e/config/jest-puppeteer.config.js cross-env NODE_CONFIG_DIR=tests/e2e/config wp-scripts test-e2e --config tests/e2e/config/jest.config.js --runInBand --runTestsByPath $( awk 'NR % 5 == ${{ matrix.part }} - 1' < ~/.jest-e2e-tests )
- name: Upload artifacts on failure
if: ${{ failure() }}
uses: actions/upload-artifact@v3
with:
name: e2e-with-gutenberg-test-report-${{matrix.part}}
Critical flow: Can see correct tax (https://github.com/woocommerce/woocommerce-blocks/pull/5900) * Tax is displayed correctly e2e test * Add extra utils to shopper * add single e2e test script oin package.json * Feedback changes from Niels * Test tax is correct on summary page * Check if CI test passes * Longer timeout for tax test * change timeout in jest config * increate jest timeout * Test * Test * Test * setTimeout outside of tests to 120000 * Address raluca's feedback and timeout everywheregaa * Increase timeout on for search box * set jest timeout to 120000 * Raluca's suggestions * Debug * Increase search timeout to 5000 * Only run Tax e2e tests in CI for quicker debug - REMOVE THIS * use waitforSelector * 30s delay for toMatchElement * Let'shope this works * Remove toMatchElement * waitForSelector * Run debug test only * Debug tax test * debug taxes * shopper using toMatchElement again * Remove the 30000 timeout delay from toMatchElement * Run all tests but only once * Remove tax test * Revert "Remove tax test" This reverts commit 7db34120e1e91f4fd26514fdb9525cac2a6066bb. * Delete the minicart test * Revert "Revert "Remove tax test"" This reverts commit 31dd654e52f37fcd02ccd25336958248bef495c6. * Revert "Delete the minicart test" This reverts commit 5595f0834a975d097a300bc0c7cbf643caa8b764. * tax test * block addToCart * Put back block functions * Refactor block.addToCart * mini-cart e2e test uses shopper.block.emptyCart() * only run shopper tests in CI * Run frontend and shopper tests and change waitFor -> waitForTimeout * Run frontend tests in CI * Run CI e2e tests again * Skip legacy template blocks * Skip legacy template test but run all others * Switch to storefront instead of twentytwentyone after legacy-template-blocks tests finish * Tidy up * fix typo * Fix ESLint issue Co-authored-by: Saad Tarhi <saad.trh@gmail.com> Co-authored-by: Saad Tarhi <saad.tarhi@automattic.com>
2022-03-08 14:50:16 +00:00
path: reports/e2e
- name: Archive flaky tests report
uses: actions/upload-artifact@v3
if: always()
with:
name: flaky-tests-report-${{ matrix.part }}
path: flaky-tests
if-no-files-found: ignore
JSE2ETests:
name: JavaScript E2E Tests (latest)
strategy:
fail-fast: false
matrix:
part: [1, 2, 3, 4, 5]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Cache node_modules
id: cache-node-modules
uses: actions/cache@v3
env:
cache-name: cache-node-modules
with:
path: node_modules
key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
- name: Setup node version and npm cache
uses: actions/setup-node@v3
with:
node-version-file: '.nvmrc'
cache: 'npm'
- name: Install Node Dependencies
if: steps.cache-node-modules.outputs.cache-hit != 'true'
run: npm ci --no-optional
- name: Build Assets
run: FORCE_REDUCED_MOTION=true npm run build:e2e-test
- name: blocks.ini setup
run: |
echo -e 'woocommerce_blocks_phase = 3\nwoocommerce_blocks_env = tests' > blocks.ini
- name: Get Composer Cache Directory
id: composer-cache
run: |
echo "dir=$(composer config cache-files-dir)" >> $GITHUB_OUTPUT
- uses: actions/cache@v3
with:
path: ${{ steps.composer-cache.outputs.dir }}
key: ${{ runner.os }}-composer-${{ hashFiles('**/composer.lock') }}
restore-keys: |
${{ runner.os }}-composer-
- name: Set up PHP
uses: shivammathur/setup-php@v2
with:
php-version: '8.0'
coverage: none
tools: composer
- name: Composer install
run: |
composer install
- name: E2E Tests (WP latest)
env:
WOOCOMMERCE_BLOCKS_PHASE: 3
run: |
npm run wp-env start
npm run wp-env:config && npx cross-env NODE_CONFIG_DIR=tests/e2e/config wp-scripts test-e2e --config tests/e2e/config/jest.config.js --listTests > ~/.jest-e2e-tests
npx cross-env JEST_PUPPETEER_CONFIG=tests/e2e/config/jest-puppeteer.config.js cross-env NODE_CONFIG_DIR=tests/e2e/config wp-scripts test-e2e --config tests/e2e/config/jest.config.js --runInBand --runTestsByPath $( awk 'NR % 5 == ${{ matrix.part }} - 1' < ~/.jest-e2e-tests )
Critical flow: Can see correct tax (https://github.com/woocommerce/woocommerce-blocks/pull/5900) * Tax is displayed correctly e2e test * Add extra utils to shopper * add single e2e test script oin package.json * Feedback changes from Niels * Test tax is correct on summary page * Check if CI test passes * Longer timeout for tax test * change timeout in jest config * increate jest timeout * Test * Test * Test * setTimeout outside of tests to 120000 * Address raluca's feedback and timeout everywheregaa * Increase timeout on for search box * set jest timeout to 120000 * Raluca's suggestions * Debug * Increase search timeout to 5000 * Only run Tax e2e tests in CI for quicker debug - REMOVE THIS * use waitforSelector * 30s delay for toMatchElement * Let'shope this works * Remove toMatchElement * waitForSelector * Run debug test only * Debug tax test * debug taxes * shopper using toMatchElement again * Remove the 30000 timeout delay from toMatchElement * Run all tests but only once * Remove tax test * Revert "Remove tax test" This reverts commit 7db34120e1e91f4fd26514fdb9525cac2a6066bb. * Delete the minicart test * Revert "Revert "Remove tax test"" This reverts commit 31dd654e52f37fcd02ccd25336958248bef495c6. * Revert "Delete the minicart test" This reverts commit 5595f0834a975d097a300bc0c7cbf643caa8b764. * tax test * block addToCart * Put back block functions * Refactor block.addToCart * mini-cart e2e test uses shopper.block.emptyCart() * only run shopper tests in CI * Run frontend and shopper tests and change waitFor -> waitForTimeout * Run frontend tests in CI * Run CI e2e tests again * Skip legacy template blocks * Skip legacy template test but run all others * Switch to storefront instead of twentytwentyone after legacy-template-blocks tests finish * Tidy up * fix typo * Fix ESLint issue Co-authored-by: Saad Tarhi <saad.trh@gmail.com> Co-authored-by: Saad Tarhi <saad.tarhi@automattic.com>
2022-03-08 14:50:16 +00:00
- name: Upload artifacts on failure
if: ${{ failure() }}
uses: actions/upload-artifact@v3
with:
name: e2e-test-report-${{matrix.part}}
Critical flow: Can see correct tax (https://github.com/woocommerce/woocommerce-blocks/pull/5900) * Tax is displayed correctly e2e test * Add extra utils to shopper * add single e2e test script oin package.json * Feedback changes from Niels * Test tax is correct on summary page * Check if CI test passes * Longer timeout for tax test * change timeout in jest config * increate jest timeout * Test * Test * Test * setTimeout outside of tests to 120000 * Address raluca's feedback and timeout everywheregaa * Increase timeout on for search box * set jest timeout to 120000 * Raluca's suggestions * Debug * Increase search timeout to 5000 * Only run Tax e2e tests in CI for quicker debug - REMOVE THIS * use waitforSelector * 30s delay for toMatchElement * Let'shope this works * Remove toMatchElement * waitForSelector * Run debug test only * Debug tax test * debug taxes * shopper using toMatchElement again * Remove the 30000 timeout delay from toMatchElement * Run all tests but only once * Remove tax test * Revert "Remove tax test" This reverts commit 7db34120e1e91f4fd26514fdb9525cac2a6066bb. * Delete the minicart test * Revert "Revert "Remove tax test"" This reverts commit 31dd654e52f37fcd02ccd25336958248bef495c6. * Revert "Delete the minicart test" This reverts commit 5595f0834a975d097a300bc0c7cbf643caa8b764. * tax test * block addToCart * Put back block functions * Refactor block.addToCart * mini-cart e2e test uses shopper.block.emptyCart() * only run shopper tests in CI * Run frontend and shopper tests and change waitFor -> waitForTimeout * Run frontend tests in CI * Run CI e2e tests again * Skip legacy template blocks * Skip legacy template test but run all others * Switch to storefront instead of twentytwentyone after legacy-template-blocks tests finish * Tidy up * fix typo * Fix ESLint issue Co-authored-by: Saad Tarhi <saad.trh@gmail.com> Co-authored-by: Saad Tarhi <saad.tarhi@automattic.com>
2022-03-08 14:50:16 +00:00
path: reports/e2e
- name: Archive flaky tests report
uses: actions/upload-artifact@e448a9b857ee2131e752b06002bf0e093c65e571 # v2.2.2
if: always()
with:
name: flaky-tests-report-${{ matrix.part }}
path: flaky-tests
if-no-files-found: ignore