woocommerce/.github/workflows/smoke-test-release.yml

693 lines
31 KiB
YAML
Raw Normal View History

2021-08-11 02:38:41 +00:00
name: Smoke test release
on:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
release:
types: [released, prereleased, published]
workflow_dispatch:
inputs:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
tag:
description: 'WooCommerce Release Tag'
required: true
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
concurrency:
group: ${{ github.workflow }}-${{ github.event.release.tag_name || inputs.tag }}
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
cancel-in-progress: true
permissions: {}
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
env:
E2E_WP_LATEST_ARTIFACT: e2e-wp-latest--run-${{ github.run_number }}
E2E_UPDATE_WC_ARTIFACT: e2e-update-wc--run-${{ github.run_number }}
FORCE_COLOR: 1
2021-08-11 02:38:41 +00:00
jobs:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
get-tag:
name: Get WooCommerce release tag
permissions:
contents: read
runs-on: ubuntu-20.04
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
outputs:
tag: ${{ steps.get-tag.outputs.tag }}
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
created: ${{ steps.created-at.outputs.created }}
steps:
- name: Validate tag
if: ${{ github.event_name == 'workflow_dispatch' }}
env:
GH_TOKEN: ${{ secrets.E2E_GH_TOKEN }}
run: gh release view "${{ inputs.tag }}" --repo=woocommerce/woocommerce
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Get tag from triggered event
id: get-tag
env:
RELEASE_TAG: ${{ github.event.release.tag_name || inputs.tag }}
run: |
echo "Triggered event: ${{ github.event_name }}"
echo "Tag from event: $RELEASE_TAG"
echo "tag=$RELEASE_TAG" >> $GITHUB_OUTPUT
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Verify woocommerce.zip asset
env:
GH_TOKEN: ${{ secrets.E2E_GH_TOKEN }}
RELEASE_TAG: ${{ steps.get-tag.outputs.tag }}
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
run: |
ASSET_NAMES=$(gh release view $RELEASE_TAG --repo woocommerce/woocommerce --json assets --jq ".assets[].name")
if [[ $ASSET_NAMES == *"woocommerce.zip"* ]]
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
then
echo "$RELEASE_TAG has a valid woocommerce.zip asset."
exit 0
fi
echo "$RELEASE_TAG does not have a valid woocommerce.zip asset."
exit 1
- name: Get 'created-at' of WooCommerce zip
id: created-at
env:
GH_TOKEN: ${{ secrets.E2E_GH_TOKEN }}
run: echo "created=$(gh release view ${{ steps.get-tag.outputs.tag }} --json assets --jq .assets[0].createdAt --repo woocommerce/woocommerce)" >> $GITHUB_OUTPUT
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
e2e-update-wc:
name: Test WooCommerce update
runs-on: ubuntu-20.04
needs: [get-tag]
permissions:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
contents: read
env:
ALLURE_REPORT_DIR: ${{ github.workspace }}/plugins/woocommerce/tests/e2e-pw/test-results/allure-report
ALLURE_RESULTS_DIR: ${{ github.workspace }}/plugins/woocommerce/tests/e2e-pw/test-results/allure-results
steps:
- uses: actions/checkout@v3
- name: Setup WooCommerce Monorepo
uses: ./.github/actions/setup-woocommerce-monorepo
2022-05-12 13:34:55 +00:00
with:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
install-filters: woocommerce
build: false
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Download and install Chromium browser.
working-directory: plugins/woocommerce
run: pnpm exec playwright install chromium
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Run 'Update WooCommerce' test.
2022-05-12 13:34:55 +00:00
working-directory: plugins/woocommerce
env:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
ADMIN_PASSWORD: ${{ secrets.RELEASE_TEST_ADMIN_PASSWORD }}
ADMIN_USER: ${{ secrets.RELEASE_TEST_ADMIN_USER }}
BASE_URL: ${{ secrets.RELEASE_TEST_URL }}
CUSTOMER_PASSWORD: ${{ secrets.RELEASE_TEST_CUSTOMER_PASSWORD }}
CUSTOMER_USER: ${{ secrets.RELEASE_TEST_CUSTOMER_USER }}
DEFAULT_TIMEOUT_OVERRIDE: 120000
GITHUB_TOKEN: ${{ secrets.E2E_GH_TOKEN }}
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
UPDATE_WC: ${{ needs.get-tag.outputs.tag }}
run: |
pnpm exec playwright test \
--config=tests/e2e-pw/playwright.config.js \
update-woocommerce.spec.js
- name: Generate 'Update WooCommerce' test report.
if: success() || failure()
working-directory: plugins/woocommerce
run: pnpm exec allure generate --clean ${{ env.ALLURE_RESULTS_DIR }} --output ${{ env.ALLURE_REPORT_DIR }}
- name: Configure AWS credentials
if: success() || failure()
uses: aws-actions/configure-aws-credentials@v1-node16
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
with:
aws-region: ${{ secrets.REPORTS_AWS_REGION }}
aws-access-key-id: ${{ secrets.REPORTS_AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.REPORTS_AWS_SECRET_ACCESS_KEY }}
- name: Upload Allure files to bucket
if: success() || failure()
run: |
aws s3 sync ${{ env.ALLURE_RESULTS_DIR }} \
${{ secrets.REPORTS_BUCKET }}/artifacts/${{ github.run_id }}/${{ env.E2E_WP_LATEST_ARTIFACT }}/allure-results \
--quiet
aws s3 sync ${{ env.ALLURE_REPORT_DIR }} \
${{ secrets.REPORTS_BUCKET }}/artifacts/${{ github.run_id }}/${{ env.E2E_WP_LATEST_ARTIFACT }}/allure-report \
--quiet
- name: Publish E2E Allure report
if: success() || failure()
env:
GITHUB_TOKEN: ${{ secrets.REPORTS_TOKEN }}
ENV_DESCRIPTION: wp-latest
run: |
gh workflow run publish-test-reports-release.yml \
-f created_at="${{ needs.get-tag.outputs.created }}" \
-f run_id=${{ github.run_id }} \
-f run_number=${{ github.run_number }} \
-f release_tag=${{ needs.get-tag.outputs.tag }} \
-f artifact="${{ env.E2E_WP_LATEST_ARTIFACT }}" \
-f env_description="${{ env.ENV_DESCRIPTION }}" \
-f test_type="e2e" \
--repo woocommerce/woocommerce-test-reports
- name: Archive 'Update WooCommerce' test report
if: success() || failure()
uses: actions/upload-artifact@v3
with:
name: ${{ env.E2E_UPDATE_WC_ARTIFACT }}
path: |
${{ env.ALLURE_RESULTS_DIR }}
${{ env.ALLURE_REPORT_DIR }}
if-no-files-found: ignore
retention-days: 5
api-wp-latest:
name: API on WP Latest
runs-on: ubuntu-20.04
needs: [get-tag, e2e-update-wc]
permissions:
contents: read
env:
ALLURE_REPORT_DIR: ${{ github.workspace }}/plugins/woocommerce/tests/api-core-tests/api-test-report/allure-report
ALLURE_RESULTS_DIR: ${{ github.workspace }}/plugins/woocommerce/tests/api-core-tests/api-test-report/allure-results
API_WP_LATEST_ARTIFACT: api-wp-latest--run-${{ github.run_number }}
steps:
- uses: actions/checkout@v3
- name: Setup WooCommerce Monorepo
uses: ./.github/actions/setup-woocommerce-monorepo
with:
install-filters: woocommerce
build: false
- name: Run API tests.
working-directory: plugins/woocommerce
env:
BASE_URL: ${{ secrets.RELEASE_TEST_URL }}
USER_KEY: ${{ secrets.RELEASE_TEST_ADMIN_USER }}
USER_SECRET: ${{ secrets.RELEASE_TEST_ADMIN_PASSWORD }}
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
run: pnpm exec playwright test --config=tests/api-core-tests/playwright.config.js hello
- name: Generate API Test report.
if: success() || failure()
working-directory: plugins/woocommerce
run: pnpm exec allure generate --clean ${{ env.ALLURE_RESULTS_DIR }} --output ${{ env.ALLURE_REPORT_DIR }}
- name: Configure AWS credentials
if: success() || failure()
uses: aws-actions/configure-aws-credentials@v1-node16
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
with:
aws-region: ${{ secrets.REPORTS_AWS_REGION }}
aws-access-key-id: ${{ secrets.REPORTS_AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.REPORTS_AWS_SECRET_ACCESS_KEY }}
- name: Upload Allure files to bucket
if: success() || failure()
run: |
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
aws s3 cp ${{ env.ALLURE_RESULTS_DIR }} \
${{ secrets.REPORTS_BUCKET }}/artifacts/${{ github.run_id }}/${{ env.API_WP_LATEST_ARTIFACT }}/allure-results \
--recursive \
--quiet
aws s3 cp ${{ env.ALLURE_REPORT_DIR }} \
${{ secrets.REPORTS_BUCKET }}/artifacts/${{ github.run_id }}/${{ env.API_WP_LATEST_ARTIFACT }}/allure-report \
--recursive \
--quiet
- name: Publish API Allure report
if: success() || failure()
env:
GITHUB_TOKEN: ${{ secrets.REPORTS_TOKEN }}
ENV_DESCRIPTION: wp-latest
run: |
gh workflow run publish-test-reports-release.yml \
-f created_at="${{ needs.get-tag.outputs.created }}" \
-f run_id=${{ github.run_id }} \
-f run_number=${{ github.run_number }} \
-f release_tag=${{ needs.get-tag.outputs.tag }} \
-f artifact="${{ env.API_WP_LATEST_ARTIFACT }}" \
-f env_description="${{ env.ENV_DESCRIPTION }}" \
-f test_type="api" \
--repo woocommerce/woocommerce-test-reports
- name: Archive API test report
if: success() || failure()
uses: actions/upload-artifact@v3
with:
name: ${{ env.API_WP_LATEST_ARTIFACT }}
path: |
${{ env.ALLURE_RESULTS_DIR }}
${{ env.ALLURE_REPORT_DIR }}
if-no-files-found: ignore
retention-days: 5
e2e-wp-latest:
name: E2E on WP Latest
runs-on: ubuntu-20.04
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
needs: [get-tag, api-wp-latest]
permissions:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
contents: read
env:
ALLURE_REPORT_DIR: ${{ github.workspace }}/plugins/woocommerce/tests/e2e-pw/test-results/allure-report
ALLURE_RESULTS_DIR: ${{ github.workspace }}/plugins/woocommerce/tests/e2e-pw/test-results/allure-results
steps:
- uses: actions/checkout@v3
2022-05-12 13:34:55 +00:00
- name: Setup WooCommerce Monorepo
uses: ./.github/actions/setup-woocommerce-monorepo
2022-05-12 13:34:55 +00:00
with:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
install-filters: woocommerce
build: false
2022-05-12 13:34:55 +00:00
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Download and install Chromium browser.
working-directory: plugins/woocommerce
run: pnpm exec playwright install chromium
- name: Run E2E tests
env:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
ADMIN_PASSWORD: ${{ secrets.RELEASE_TEST_ADMIN_PASSWORD }}
ADMIN_USER: ${{ secrets.RELEASE_TEST_ADMIN_USER }}
ADMIN_USER_EMAIL: ${{ secrets.RELEASE_TEST_ADMIN_USER_EMAIL }}
BASE_URL: ${{ secrets.RELEASE_TEST_URL }}
CUSTOMER_PASSWORD: ${{ secrets.RELEASE_TEST_CUSTOMER_PASSWORD }}
CUSTOMER_USER: ${{ secrets.RELEASE_TEST_CUSTOMER_USER }}
DEFAULT_TIMEOUT_OVERRIDE: 120000
E2E_MAX_FAILURES: 25
RESET_SITE: true
timeout-minutes: 60
working-directory: plugins/woocommerce
run: pnpm exec playwright test --config=tests/e2e-pw/ignore-plugin-tests.playwright.config.js
- name: Download 'e2e-update-wc' artifact
if: success() || failure()
uses: actions/download-artifact@v3
with:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
name: ${{ env.E2E_UPDATE_WC_ARTIFACT }}
path: plugins/woocommerce/tmp
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Add allure-results from 'e2e-update-wc'
if: success() || failure()
working-directory: plugins/woocommerce
run: cp -r tmp/allure-results tests/e2e-pw/test-results
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Generate E2E Test report.
if: success() || failure()
working-directory: plugins/woocommerce
run: pnpm exec allure generate --clean ${{ env.ALLURE_RESULTS_DIR }} --output ${{ env.ALLURE_REPORT_DIR }}
2022-05-12 13:34:55 +00:00
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Configure AWS credentials
if: success() || failure()
uses: aws-actions/configure-aws-credentials@v1-node16
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
with:
aws-region: ${{ secrets.REPORTS_AWS_REGION }}
aws-access-key-id: ${{ secrets.REPORTS_AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.REPORTS_AWS_SECRET_ACCESS_KEY }}
- name: Upload report to bucket
if: success() || failure()
run: |
aws s3 sync ${{ env.ALLURE_RESULTS_DIR }} \
${{ secrets.REPORTS_BUCKET }}/artifacts/${{ github.run_id }}/${{ env.E2E_WP_LATEST_ARTIFACT }}/allure-results \
--quiet
aws s3 sync ${{ env.ALLURE_REPORT_DIR }} \
${{ secrets.REPORTS_BUCKET }}/artifacts/${{ github.run_id }}/${{ env.E2E_WP_LATEST_ARTIFACT }}/allure-report \
--quiet
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Publish E2E Allure report
if: success() || failure()
env:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
GITHUB_TOKEN: ${{ secrets.REPORTS_TOKEN }}
ENV_DESCRIPTION: wp-latest
run: |
gh workflow run publish-test-reports-release.yml \
-f created_at="${{ needs.get-tag.outputs.created }}" \
-f run_id=${{ github.run_id }} \
-f run_number=${{ github.run_number }} \
-f release_tag=${{ needs.get-tag.outputs.tag }} \
-f artifact="${{ env.E2E_WP_LATEST_ARTIFACT }}" \
-f env_description="${{ env.ENV_DESCRIPTION }}" \
-f test_type="e2e" \
--repo woocommerce/woocommerce-test-reports
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Archive E2E test report
if: success() || failure()
uses: actions/upload-artifact@v3
with:
name: ${{ env.E2E_WP_LATEST_ARTIFACT }}
path: |
${{ env.ALLURE_RESULTS_DIR }}
${{ env.ALLURE_REPORT_DIR }}
if-no-files-found: ignore
retention-days: 5
get-wp-versions:
name: Get WP L-1 & L-2 version numbers
needs: [get-tag]
runs-on: ubuntu-20.04
permissions:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
contents: read
outputs:
matrix: ${{ steps.get-versions.outputs.versions }}
tag: ${{ needs.get-tag.outputs.tag }}
created: ${{ needs.get-tag.outputs.created }}
steps:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Create dirs
run: |
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
mkdir script
mkdir repo
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Checkout
uses: actions/checkout@v3
with:
path: repo
- name: Copy script to get previous WP versions
run: cp repo/plugins/woocommerce/tests/e2e-pw/utils/wordpress.js script
- name: Install axios
working-directory: script
run: npm install axios
- name: Get version numbers
id: get-versions
uses: actions/github-script@v6
with:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
script: |
const { getPreviousTwoVersions } = require('./script/wordpress');
const versions = await getPreviousTwoVersions();
console.log(versions);
core.setOutput('versions', versions);
test-wp-versions:
name: Test against ${{ matrix.version.description }} (${{ matrix.version.number }})
runs-on: ubuntu-20.04
needs: [get-wp-versions]
strategy:
matrix: ${{ fromJSON(needs.get-wp-versions.outputs.matrix) }}
env:
API_ALLURE_REPORT_DIR: ${{ github.workspace }}/plugins/woocommerce/tests/e2e-pw/test-results/api/allure-report
API_ALLURE_RESULTS_DIR: ${{ github.workspace }}/plugins/woocommerce/tests/e2e-pw/test-results/api/allure-results
API_WP_LATEST_X_ARTIFACT: api-${{ matrix.version.env_description }}--run-${{ github.run_number }}
E2E_ALLURE_REPORT_DIR: ${{ github.workspace }}/plugins/woocommerce/tests/e2e-pw/test-results/e2e/allure-report
E2E_ALLURE_RESULTS_DIR: ${{ github.workspace }}/plugins/woocommerce/tests/e2e-pw/test-results/e2e/allure-results
E2E_WP_LATEST_X_ARTIFACT: e2e-${{ matrix.version.env_description }}--run-${{ github.run_number }}
permissions:
contents: read
steps:
- name: Checkout WooCommerce repo
uses: actions/checkout@v3
2022-05-12 13:34:55 +00:00
- name: Setup WooCommerce Monorepo
uses: ./.github/actions/setup-woocommerce-monorepo
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Launch WP Env
working-directory: plugins/woocommerce
run: pnpm run env:test
- name: Download release zip
env:
GH_TOKEN: ${{ secrets.E2E_GH_TOKEN }}
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
run: gh release download ${{ needs.get-wp-versions.outputs.tag }} --dir tmp
- name: Replace `plugins/woocommerce` with unzipped woocommerce release build
run: unzip -d plugins -o tmp/woocommerce.zip
- name: Downgrade WordPress version to ${{ matrix.version.number }}
working-directory: plugins/woocommerce
run: |
pnpm exec wp-env run tests-cli "wp core update --version=${{ matrix.version.number }} --force"
pnpm exec wp-env run tests-cli "wp core update-db"
- name: Verify environment details
working-directory: plugins/woocommerce
run: |
pnpm exec wp-env run tests-cli "wp core version"
pnpm exec wp-env run tests-cli "wp plugin list"
pnpm exec wp-env run tests-cli "wp theme list"
pnpm exec wp-env run tests-cli "wp user list"
- name: Run API tests.
id: api
working-directory: plugins/woocommerce
env:
ALLURE_RESULTS_DIR: ${{ env.API_ALLURE_RESULTS_DIR }}
run: pnpm exec playwright test --config=tests/api-core-tests/playwright.config.js hello
- name: Generate API Allure report.
if: success() || failure()
working-directory: plugins/woocommerce
run: pnpm exec allure generate --clean ${{ env.API_ALLURE_RESULTS_DIR }} --output ${{ env.API_ALLURE_REPORT_DIR }}
- name: Configure AWS credentials
if: success() || failure()
uses: aws-actions/configure-aws-credentials@v1-node16
2022-05-12 13:34:55 +00:00
with:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
aws-region: ${{ secrets.REPORTS_AWS_REGION }}
aws-access-key-id: ${{ secrets.REPORTS_AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.REPORTS_AWS_SECRET_ACCESS_KEY }}
- name: Upload API Allure artifacts to bucket
if: success() || failure()
run: |
aws s3 sync ${{ env.API_ALLURE_RESULTS_DIR }} \
${{ secrets.REPORTS_BUCKET }}/artifacts/${{ github.run_id }}/${{ env.API_WP_LATEST_X_ARTIFACT }}/allure-results \
--quiet
aws s3 sync ${{ env.API_ALLURE_REPORT_DIR }} \
${{ secrets.REPORTS_BUCKET }}/artifacts/${{ github.run_id }}/${{ env.API_WP_LATEST_X_ARTIFACT }}/allure-report \
--quiet
2022-05-12 13:34:55 +00:00
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Publish API Allure report
if: success() || failure()
env:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
GITHUB_TOKEN: ${{ secrets.REPORTS_TOKEN }}
ENV_DESCRIPTION: ${{ matrix.version.env_description }}
run: |
gh workflow run publish-test-reports-release.yml \
-f created_at="${{ needs.get-wp-versions.outputs.created }}" \
-f run_id=${{ github.run_id }} \
-f run_number=${{ github.run_number }} \
-f release_tag=${{ needs.get-wp-versions.outputs.tag }} \
-f artifact="${{ env.API_WP_LATEST_X_ARTIFACT }}" \
-f env_description="${{ env.ENV_DESCRIPTION }}" \
-f test_type="api" \
--repo woocommerce/woocommerce-test-reports
- name: Archive API Allure reports
if: success() || failure()
uses: actions/upload-artifact@v3
with:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
name: ${{ env.API_WP_LATEST_X_ARTIFACT }}
path: |
${{ env.API_ALLURE_RESULTS_DIR }}
${{ env.API_ALLURE_REPORT_DIR }}
if-no-files-found: ignore
retention-days: 5
- name: Download and install Chromium browser.
if: success() || failure()
working-directory: plugins/woocommerce
run: pnpm exec playwright install chromium
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Run E2E tests.
if: |
success() ||
( failure() && steps.api.conclusion == 'success' )
timeout-minutes: 60
id: e2e
env:
USE_WP_ENV: 1
E2E_MAX_FAILURES: 15
FORCE_COLOR: 1
ALLURE_RESULTS_DIR: ${{ env.E2E_ALLURE_RESULTS_DIR }}
DEFAULT_TIMEOUT_OVERRIDE: 120000
working-directory: plugins/woocommerce
run: pnpm exec playwright test --config=tests/e2e-pw/playwright.config.js
- name: Generate E2E Allure report.
if: success() || failure()
working-directory: plugins/woocommerce
run: pnpm exec allure generate --clean ${{ env.E2E_ALLURE_RESULTS_DIR }} --output ${{ env.E2E_ALLURE_REPORT_DIR }}
- name: Upload E2E Allure artifacts to bucket
if: success() || failure()
run: |
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
aws s3 sync ${{ env.E2E_ALLURE_RESULTS_DIR }} \
${{ secrets.REPORTS_BUCKET }}/artifacts/${{ github.run_id }}/${{ env.E2E_WP_LATEST_X_ARTIFACT }}/allure-results \
--quiet
aws s3 sync ${{ env.E2E_ALLURE_REPORT_DIR }} \
${{ secrets.REPORTS_BUCKET }}/artifacts/${{ github.run_id }}/${{ env.E2E_WP_LATEST_X_ARTIFACT }}/allure-report \
--quiet
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Archive E2E Allure reports
if: success() || failure()
uses: actions/upload-artifact@v3
with:
name: ${{ env.E2E_WP_LATEST_X_ARTIFACT }}
path: |
${{ env.E2E_ALLURE_RESULTS_DIR }}
${{ env.E2E_ALLURE_REPORT_DIR }}
if-no-files-found: ignore
retention-days: 5
2022-05-12 13:34:55 +00:00
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Publish E2E Allure report
if: success() || failure()
2022-05-12 13:34:55 +00:00
env:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
GITHUB_TOKEN: ${{ secrets.REPORTS_TOKEN }}
ENV_DESCRIPTION: ${{ matrix.version.env_description }}
run: |
gh workflow run publish-test-reports-release.yml \
-f created_at="${{ needs.get-wp-versions.outputs.created }}" \
-f run_id=${{ github.run_id }} \
-f run_number=${{ github.run_number }} \
-f release_tag=${{ needs.get-wp-versions.outputs.tag }} \
-f artifact="${{ env.E2E_WP_LATEST_X_ARTIFACT }}" \
-f env_description="${{ env.ENV_DESCRIPTION }}" \
-f test_type="e2e" \
--repo woocommerce/woocommerce-test-reports
test-php-versions:
name: Test against PHP ${{ matrix.php_version }}
runs-on: ubuntu-20.04
needs: [get-tag]
strategy:
matrix:
php_version: ['7.4', '8.1']
env:
API_ALLURE_REPORT_DIR: ${{ github.workspace }}/plugins/woocommerce/tests/api-core-tests/test-results/allure-report
API_ALLURE_RESULTS_DIR: ${{ github.workspace }}/plugins/woocommerce/tests/api-core-tests/test-results/allure-results
API_ARTIFACT: api-php-${{ matrix.php_version }}--run-${{ github.run_number }}
E2E_ALLURE_REPORT_DIR: ${{ github.workspace }}/plugins/woocommerce/tests/e2e-pw/test-results/allure-report
E2E_ALLURE_RESULTS_DIR: ${{ github.workspace }}/plugins/woocommerce/tests/e2e-pw/test-results/allure-results
E2E_ARTIFACT: e2e-php-${{ matrix.php_version }}--run-${{ github.run_number }}
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Setup WooCommerce Monorepo
uses: ./.github/actions/setup-woocommerce-monorepo
- name: Launch WP Env
working-directory: plugins/woocommerce
env:
WP_ENV_PHP_VERSION: ${{ matrix.php_version }}
run: pnpm run env:test
- name: Verify PHP version
working-directory: .github/workflows/scripts
env:
EXPECTED_PHP_VERSION: ${{ matrix.php_version }}
run: bash verify-php-version.sh
- name: Download release zip
env:
GH_TOKEN: ${{ secrets.E2E_GH_TOKEN }}
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
run: gh release download ${{ needs.get-tag.outputs.tag }} --dir tmp
- name: Replace `plugins/woocommerce` with unzipped woocommerce release build
run: unzip -d plugins -o tmp/woocommerce.zip
- name: Run API tests.
id: api
working-directory: plugins/woocommerce
env:
ALLURE_RESULTS_DIR: ${{ env.API_ALLURE_RESULTS_DIR }}
run: pnpm exec playwright test --config=tests/api-core-tests/playwright.config.js hello
- name: Generate API Allure report.
if: success() || failure()
working-directory: plugins/woocommerce
run: pnpm exec allure generate --clean ${{ env.API_ALLURE_RESULTS_DIR }} --output ${{ env.API_ALLURE_REPORT_DIR }}
- name: Configure AWS credentials
if: success() || failure()
uses: aws-actions/configure-aws-credentials@v1-node16
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
with:
aws-region: ${{ secrets.REPORTS_AWS_REGION }}
aws-access-key-id: ${{ secrets.REPORTS_AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.REPORTS_AWS_SECRET_ACCESS_KEY }}
- name: Upload API Allure artifacts to bucket
if: success() || failure()
run: |
aws s3 sync ${{ env.API_ALLURE_RESULTS_DIR }} \
${{ secrets.REPORTS_BUCKET }}/artifacts/${{ github.run_id }}/${{ env.API_ARTIFACT }}/allure-results \
--quiet
aws s3 sync ${{ env.API_ALLURE_REPORT_DIR }} \
${{ secrets.REPORTS_BUCKET }}/artifacts/${{ github.run_id }}/${{ env.API_ARTIFACT }}/allure-report \
--quiet
- name: Publish API Allure report
if: success() || failure()
env:
GITHUB_TOKEN: ${{ secrets.REPORTS_TOKEN }}
ENV_DESCRIPTION: php-${{ matrix.php_version }}
run: |
gh workflow run publish-test-reports-release.yml \
-f created_at="${{ needs.get-tag.outputs.created }}" \
-f run_id=${{ github.run_id }} \
-f run_number=${{ github.run_number }} \
-f release_tag=${{ needs.get-tag.outputs.tag }} \
-f artifact="${{ env.API_ARTIFACT }}" \
-f env_description="${{ env.ENV_DESCRIPTION }}" \
-f test_type="api" \
--repo woocommerce/woocommerce-test-reports
- name: Archive API Allure reports
if: success() || failure()
uses: actions/upload-artifact@v3
with:
name: ${{ env.API_ARTIFACT }}
path: |
${{ env.API_ALLURE_RESULTS_DIR }}
${{ env.API_ALLURE_REPORT_DIR }}
if-no-files-found: ignore
retention-days: 5
- name: Download and install Chromium browser.
working-directory: plugins/woocommerce
run: pnpm exec playwright install chromium
- name: Run E2E tests.
if: |
success() ||
( failure() && steps.api.conclusion == 'success' )
timeout-minutes: 60
env:
USE_WP_ENV: 1
E2E_MAX_FAILURES: 15
FORCE_COLOR: 1
ALLURE_RESULTS_DIR: ${{ env.E2E_ALLURE_RESULTS_DIR }}
DEFAULT_TIMEOUT_OVERRIDE: 120000
working-directory: plugins/woocommerce
run: pnpm exec playwright test --config=tests/e2e-pw/playwright.config.js
- name: Generate E2E Allure report.
if: success() || failure()
working-directory: plugins/woocommerce
run: pnpm exec allure generate --clean ${{ env.E2E_ALLURE_RESULTS_DIR }} --output ${{ env.E2E_ALLURE_REPORT_DIR }}
- name: Upload E2E Allure artifacts to bucket
if: success() || failure()
run: |
aws s3 sync ${{ env.E2E_ALLURE_RESULTS_DIR }} \
${{ secrets.REPORTS_BUCKET }}/artifacts/${{ github.run_id }}/${{ env.E2E_ARTIFACT }}/allure-results \
--quiet
aws s3 sync ${{ env.E2E_ALLURE_REPORT_DIR }} \
${{ secrets.REPORTS_BUCKET }}/artifacts/${{ github.run_id }}/${{ env.E2E_ARTIFACT }}/allure-report \
--quiet
- name: Archive E2E Allure reports
if: success() || failure()
uses: actions/upload-artifact@v3
with:
name: ${{ env.E2E_ARTIFACT }}
path: |
${{ env.E2E_ALLURE_RESULTS_DIR }}
${{ env.E2E_ALLURE_REPORT_DIR }}
if-no-files-found: ignore
retention-days: 5
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
- name: Publish E2E Allure report
if: success() || failure()
env:
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
GITHUB_TOKEN: ${{ secrets.REPORTS_TOKEN }}
ENV_DESCRIPTION: php-${{ matrix.php_version }}
run: |
Enable "Smoke test release" workflow (#36598) * First pass at updating release test workflow * Add changelog * Set dir env variables * Update to workflow * Fix indent * Fix indent * Clean up indent * Re-order steps * Change order of jobs * Added common php versions * Update pipeline * Update some labels * Simplify for testing * Update paths * Create tmp folder * Fix path * Paths * Try outputting some debugging * Add step ID back * Remove working directory * Another path tweak * Add API release tests * Add k6 tests * Add PHP tests * Launch wp-env during PHP tests * Try default values * Tweak some settings, add WP testing * Tweak some settings * Re-order e2e steps * Update step descriptions * Reorganize tests, add plugin tests * Enable only e2e job * Initial set up to run against release smoke test site * Fix syntax * Temporarily disable update wc spec * Temporarily disable downloading woocommerce zip * Download release zip using tag name * Fix wrong job name * Fix wrong job name * Fix dir * Delete fetch-asset-id.js * REfactor update-woocommerce spec * Add error handling * Download release zip by tag * Refactor update woo spec to download zip by tag * Correct job name * fail test on invalid tag * Enable all e2e tests * Run api tests before e2e tests * Fix job dependency * Add customer credentials to api job * Separate job for WC Update * Combine e2e allure-results, then report * Enable report job * Fix context * Change job and artifact names * Use test s3 path * Minor job name change * Upload artifacts to bucket * Correct s3 path * Add quiet option * Retain video on failures * Finalize s3 path * Try WP latest-1 * Revert to wp latest * Refine search for woocommerce zip asset * Get created-at * Specify repo in gh command * Slugify env description * Trim space * Sync with bucket instead of copy * Remove invalid --recursive flag * Re-add missing step to combine e2e results from update wc test * Ensure artifact upload on test failure * Enable all e2e tests on WP latest * Retain existing data before updating WC * Make test compatible with 'Canceled' and 'Cancelled' * Set env_desc as env var * Re-add deleted file * Fix UPDATE_WC in daily smoke test workflow * Add tracing in global setup * Remove tracing * Temporarily run only basic spec * Job for WP Latest-1 & 2 * Fix "Required input 'created_at' not provided" * Minor rename * Remove install filter * Install deps in get-wp-matrix * Delete get-wp-versions.js * Add get-wp-versions.js to e2e-pw folder * REname file * REfactor * Refactor script for getting WP prev versions * Update job dependencies * Temporarily remove disabled jobs * Allow e2e-wp-latest after api test failure * Update L-1 & L-2 job deps * Fix report-wp-latest * Fix failing api test * Make get-wp-versions quicker * Publish report immediately after test * Test reporting in e2e-update-wc * Fix missing parameter * Fix env_desc, re-enable other jobs * Enable all e2e tests * Minor job name change * Fix flaky test * Add php version testing * stringify php versions * Re-enable all e2e tests * Up timeout to 2min * Remove PHP 8.0 * Add missing conditionals * Fix php version verification script * Fix starting dir * Fix flakiness * Skip e2e if api failed * Verify woocommerce.zip early * Add token * Delete test summary on github for the meantime * Use default playwright config * More meaningful variable names * Update step titles based on review * Use expect.poll() * Minor spacing corrections * Use `stable-check` endpoint, delete unnecessary loop * Update locators to be JN-compatible * Fix erroneous getting of release tag * Fix conflict of "No thanks" button locator with that of WP Mail Logging's * Update github-script action to v6 * Revert to 'Cancelled' * Remove unnecessary step * Provide missing env variables --------- Co-authored-by: Jon Lane <jon.lane@automattic.com> Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 04:11:46 +00:00
gh workflow run publish-test-reports-release.yml \
-f created_at="${{ needs.get-tag.outputs.created }}" \
-f run_id=${{ github.run_id }} \
-f run_number=${{ github.run_number }} \
-f release_tag=${{ needs.get-tag.outputs.tag }} \
-f artifact="${{ env.E2E_ARTIFACT }}" \
-f env_description="${{ env.ENV_DESCRIPTION }}" \
-f test_type="e2e" \
--repo woocommerce/woocommerce-test-reports