Commit Graph

608 Commits

Author SHA1 Message Date
nigeljamesstevenson c37080f29f
Add workflow to run API, k6 performance, and E2E tests on trunk after PR merge (#37033) 2023-03-22 09:44:36 +00:00
rodelgc 5f8aa3671c Merge branch 'trunk' into e2e/release-plugins-0 2023-03-22 17:29:08 +08:00
rodelgc 65c698a386 Remove unnecessary dash 2023-03-18 08:37:39 +08:00
rodelgc a80f65cc11 Add run number 2023-03-18 08:36:22 +08:00
rodelgc 45071f2f03 Simplify upload scripts 2023-03-18 08:29:56 +08:00
rodelgc 2110d011ae More accurate step name 2023-03-18 08:02:43 +08:00
rodelgc ad8d2f9b44 More readable artifact names 2023-03-17 20:31:10 +08:00
rodelgc 9503868d71 Set environment descriptions 2023-03-17 20:16:01 +08:00
rodelgc b95b9000ce Fix artifact name 2023-03-17 18:13:43 +08:00
rodelgc bc44562329 Fix dependent job id 2023-03-17 18:07:32 +08:00
rodelgc 6a2239183d Remove install-filters in plugin tests 2023-03-17 17:55:55 +08:00
rodelgc 99e243a347 Shorten job name 2023-03-17 17:53:55 +08:00
rodelgc b7cb3f8284 Correct if condition 2023-03-17 17:52:11 +08:00
rodelgc b2e7478a30 Only run hello API tests 2023-03-17 17:48:26 +08:00
rodelgc f7c7a2c451 Add plugin tests 2023-03-17 17:47:28 +08:00
Vedanshu Jain ade1540ece Merge branch 'trunk' into hpos/unit-tests 2023-03-17 13:39:19 +05:30
rodelgc 76dca5548b Use separate files 2023-03-17 15:41:06 +08:00
rodelgc 94eff753be Specify allure report and results dirs 2023-03-16 16:09:53 +08:00
rodelgc be550fec19 Use e2e and api composite actions in wp and php version tests 2023-03-16 15:13:41 +08:00
rodelgc 8b44038699 Run e2e-wp-latest 2023-03-16 13:38:29 +08:00
rodelgc 10029d0007 Skip e2e-wp-latest for now 2023-03-16 11:13:00 +08:00
rodelgc f89dd7e2c5 Use `run-api-tests` action 2023-03-16 11:08:46 +08:00
rodelgc e914281652 Revert to using E2E_WP_LATEST_ARTIFACT 2023-03-16 10:00:49 +08:00
rodelgc 4ca522a16d Fix env var reference 2023-03-16 09:49:28 +08:00
rodelgc 891942ac7c Correct artifact name 2023-03-16 09:45:33 +08:00
rodelgc 71c008eb7e Use e2e composite actions on update wc test 2023-03-16 08:52:33 +08:00
jonathansadowski adffb280f8
Remove broken workflow for bypassing changelog-only PRs (#37216) 2023-03-15 17:20:13 -05:00
Tam Mullen 562fede1e2
Update the CI workflow to include WP 6.0 and 5.9 (#37180)
Update the CI workflow to include WP 6.0 and 5.9 matching our L-2 support policy
2023-03-15 15:49:33 +00:00
rodelgc b1ed8511dc
Update k6 job name
Co-authored-by: Leif Singer <leif@automattic.com>
2023-03-13 08:37:30 +08:00
Vedanshu Jain f497146de9
Merge branch 'trunk' into hpos/unit-tests 2023-03-10 15:21:57 +05:30
jonathansadowski 4ebd35b41d
Update automation bypass to work correctly, consider actions bot. (#37045) 2023-03-09 17:10:33 -06:00
Sam Seay bec3ec18ac
Update Syncpack and use it to pin @wordpress packages to wp-6.0 (#37034) 2023-03-09 10:13:25 +13:00
rodelgc b60cc128fd
Allow running of "Smoke test release" on draft releases (#36997)
* Conditionally use authorization token when getting WC Zip download URL.

* Exponse error message

* Handle undefined authorization token more gracefully

* Specify GH token in workflow

* Add changelog

* Use E2E_GH_TOKEN instead of default github.token

* Provide GitHub token to other jobs

* Fix getting of tag name from triggered event

* Use "inputs" context instead

* Add release version to workflow name

* Correct concurrency group

* Fix workflow name

* Add --archive option

* Fix output setting

* Fix script for verifying asset

* Remove unnecessary 'uses' line

* Remove node 12 deprecation messages

* Re-add "published" release type
2023-03-08 10:23:56 +01:00
rodelgc 46300131ff
Correct `UPDATE_WC` value in the "Smoke test daily" workflow (#37049)
* Correct UPDATE_WC value

* Add changelog
2023-03-03 13:58:21 +01:00
Rodel ad61d474d5 Correct step id 2023-03-02 22:01:52 +08:00
rodelgc a5d8898ba1 Add in files from working branch 2023-03-02 17:57:41 +08:00
Paul Sealock da1dce8f19
Package Release: handle initial release case (#36922) 2023-02-28 09:36:38 +13:00
rodelgc ea1925c841
Enable "Smoke test release" workflow (#36598)
* First pass at updating release test workflow

* Add changelog

* Set dir env variables

* Update to workflow

* Fix indent

* Fix indent

* Clean up indent

* Re-order steps

* Change order of jobs

* Added common php versions

* Update pipeline

* Update some labels

* Simplify for testing

* Update paths

* Create tmp folder

* Fix path

* Paths

* Try outputting some debugging

* Add step ID back

* Remove working directory

* Another path tweak

* Add API release tests

* Add k6 tests

* Add PHP tests

* Launch wp-env during PHP tests

* Try default values

* Tweak some settings, add WP testing

* Tweak some settings

* Re-order e2e steps

* Update step descriptions

* Reorganize tests, add plugin tests

* Enable only e2e job

* Initial set up to run against release smoke test site

* Fix syntax

* Temporarily disable update wc spec

* Temporarily disable downloading woocommerce zip

* Download release zip using tag name

* Fix wrong job name

* Fix wrong job name

* Fix dir

* Delete fetch-asset-id.js

* REfactor update-woocommerce spec

* Add error handling

* Download release zip by tag

* Refactor update woo spec to download zip by tag

* Correct job name

* fail test on invalid tag

* Enable all e2e tests

* Run api tests before e2e tests

* Fix job dependency

* Add customer credentials to api job

* Separate job for WC Update

* Combine e2e allure-results, then report

* Enable report job

* Fix context

* Change job and artifact names

* Use test s3 path

* Minor job name change

* Upload artifacts to bucket

* Correct s3 path

* Add quiet option

* Retain video on failures

* Finalize s3 path

* Try WP latest-1

* Revert to wp latest

* Refine search for woocommerce zip asset

* Get created-at

* Specify repo in gh command

* Slugify env description

* Trim space

* Sync with bucket instead of copy

* Remove invalid --recursive flag

* Re-add missing step to combine e2e results from update wc test

* Ensure artifact upload on test failure

* Enable all e2e tests on WP latest

* Retain existing data before updating WC

* Make test compatible with 'Canceled' and 'Cancelled'

* Set env_desc as env var

* Re-add deleted file

* Fix UPDATE_WC in daily smoke test workflow

* Add tracing in global setup

* Remove tracing

* Temporarily run only basic spec

* Job for WP Latest-1 & 2

* Fix "Required input 'created_at' not provided"

* Minor rename

* Remove install filter

* Install deps in get-wp-matrix

* Delete get-wp-versions.js

* Add get-wp-versions.js to e2e-pw folder

* REname file

* REfactor

* Refactor script for getting WP prev versions

* Update job dependencies

* Temporarily remove disabled jobs

* Allow e2e-wp-latest after api test failure

* Update L-1 & L-2 job deps

* Fix report-wp-latest

* Fix failing api test

* Make get-wp-versions quicker

* Publish report immediately after test

* Test reporting in e2e-update-wc

* Fix missing parameter

* Fix env_desc, re-enable other jobs

* Enable all e2e tests

* Minor job name change

* Fix flaky test

* Add php version testing

* stringify php versions

* Re-enable all e2e tests

* Up timeout to 2min

* Remove PHP 8.0

* Add missing conditionals

* Fix php version verification script

* Fix starting dir

* Fix flakiness

* Skip e2e if api failed

* Verify woocommerce.zip early

* Add token

* Delete test summary on github for the meantime

* Use default playwright config

* More meaningful variable names

* Update step titles based on review

* Use expect.poll()

* Minor spacing corrections

* Use `stable-check` endpoint, delete unnecessary loop

* Update locators to be JN-compatible

* Fix erroneous getting of release tag

* Fix conflict of "No thanks" button locator with that of WP Mail Logging's

* Update github-script action to v6

* Revert to 'Cancelled'

* Remove unnecessary step

* Provide missing env variables

---------

Co-authored-by: Jon Lane <jon.lane@automattic.com>
Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 12:11:46 +08:00
Vedanshu Jain 7f140c77ca Use latest WP version. 2023-02-24 17:02:36 +05:30
Vedanshu Jain 86e5f13180 Modify name to reflet HPOS tests + use latest WP.. 2023-02-24 17:02:05 +05:30
Vedanshu Jain 5c2ccb563b Run HPOS test in a matrix along with post tests. 2023-02-24 16:57:51 +05:30
Leif Singer 1b74871bae
Assign milestone only if a PR touches WooCommerce Core (#36897)
* Assign milestone only if a PR touches WooCommerce Core

* add woocommerce-admin to list of paths
2023-02-21 18:21:29 +01:00
jonathansadowski e8a170165a
Update the release date for code freeze action readme generation (#36877)
Update the release date for code freeze action
2023-02-20 18:00:14 +13:00
Sam Seay 79fa0b9413
Fix permissions for NPM package prepare script (#36732)
Contents write is needed to write to pull requests.
2023-02-14 00:27:50 +13:00
nigeljamesstevenson 7c84271ee2
update smoke test daily workflow to run api tests first (#36494) 2023-02-02 12:04:47 +00:00
nigeljamesstevenson add0c4a5be update after code review 2023-01-31 16:29:15 +00:00
Sam Seay 341541964c
Update action dependencies to remove set-output and save-state warnings (#36549) 2023-01-27 20:26:54 +13:00
nigeljamesstevenson 7cf1599208 update smoke test daily workflow to run api tests first 2023-01-18 17:12:55 +00:00
Vedanshu Jain 8079c157bc Remove commented out code for cleaniliness. 2023-01-18 13:03:17 +05:30
Sam Seay d181008ac4
Fix TS issues and lint the release post tool by running `tsc` (#36412) 2023-01-18 18:43:22 +13:00
Sam Seay b1f80c1271
Create a manually triggered workflow to release WooCommerce Beta Tester (#36387) 2023-01-18 18:42:46 +13:00
Roy Ho d935b0137a
Add necessary permissions for code freeze workflows (#36399)
* Add necessary permissions for code freeze workflows

* Checkout pnpm-lock.yaml to prevent issues
2023-01-16 06:12:25 -08:00
Néstor Soriano 443b563335
Upgrade PHPUnit to v8 and stop using patched PHPUnit 7 for PHP 8 (#36273)
* Upgrade PHPUnit to v8, and remove usage of PHPUnit7 fork in CI

Once the minimum WordPress version supported by WooCommerce is 5.9
and the minimum PHP version supported is 7.2, we can switch to
PHPUnit 7 and the patched PHPUnit 7 with support for PHP 8 is
no longer needed.

* Resolve PHPUnit 8 deprecations in tests

* Add changelog file

* Fix code formatting
2023-01-16 20:58:09 +13:00
Sam Seay 262b2ecebd
Don't run CI checks on PRs that are just changelog changes (#36280) 2023-01-12 14:09:36 +13:00
Vedanshu Jain 475f21a985 Disable posting comment since it conflicts with normal E2E report comment. 2023-01-11 10:19:25 +05:30
Vedanshu Jain b3bd207baa Enable COT Tests for all PRs. 2023-01-11 09:31:11 +05:30
Roy Ho 929fd09fb6
Add permissions to issues endpoint (#36344) 2023-01-09 12:43:33 -08:00
Roy Ho 422b24770b
Add workflow for bumping wp l-2 version support (#36312) 2023-01-09 05:36:54 -08:00
Roy Ho 25f051cad2
Add PR write permissions (#36263) 2023-01-02 20:56:06 -08:00
Roy Ho 16386238cc
Add minimum required permissions for workflow token (#36200)
* Add minimum required permissions to e2e-tests

* Add minimum token permission to more workflows
2023-01-02 09:28:11 -08:00
rodelgc 3daeac9a69
Migrate E2E daily shopper tests (#35873)
Enable shopper E2E tests on Daily smoke test workflow
2022-12-28 18:28:17 +08:00
Vedanshu Jain 34ac47969c
Add cherry-pick support for git merge strategy. (#35927)
* Add changelog.

* Add merge strategy support to cherry-picker tool.
2022-12-22 09:26:14 +05:30
jonathansadowski bea954c2e5
Update code-freeze actions to Monday 22 days before release (#36023)
* Update code-freeze actions to Monday 22 days before release

* Fix incorrect params in workflow dispatch
2022-12-20 10:51:41 -06:00
rodelgc c4260c935f
Consistent folder structure for E2E and API test results (#35907)
Consistent folder structure for E2E and REST API test results
2022-12-19 14:33:22 +08:00
Sam Seay 03d52ff13b
Another set of migrations from set-output to GITHUB_OUTPUT (#35843)
Co-authored-by: Roy Ho <roykho77@gmail.com>
2022-12-16 11:41:03 +13:00
Sam Seay 7224c76053
Add a set_output function to script that uses the new GITHUB_OUTPUT (#35894) 2022-12-13 10:27:42 +13:00
rodelgc 93b6d358f4
Migrate merchant daily smoke tests to Playwright (#35492)
* Checkout files

* Remove reverted permalinks code

* Enable cron

* More accurate job names

* More accurate test summary title
2022-12-06 11:22:49 +08:00
Sam Seay 43143b848f
Remove some usages of set-output (#35799)
* Migrate straight script usage of set-output to write to GITHUB_OUTPUT
* Use file_put_contents in php scripts to set GITHUB_OUTPUT
2022-12-06 09:35:49 +13:00
Paul Sealock 2c03d6cc82
Live Branches: Avoid running job on community PRs (#35795) 2022-12-06 06:28:02 +13:00
Roy Ho aa96739964
Add s flag to show sniff codes (#35802) 2022-12-05 06:21:26 -08:00
Sam Seay 659894f851
Use workflow dispatch that now works with GITHUB_TOKEN (#35705) 2022-12-01 21:36:26 +13:00
Jonathan Lane fd7fc355fa
Fix/e2e environment test failures (#35679) 2022-11-22 16:16:42 -07:00
jonathansadowski 973961fbd5
Add workflow for building live branches (#35344)
* Add workflow for building live branches

* Fix inadvertent newline

* Only run on PR for default types
2022-11-15 13:58:27 -06:00
jonathansadowski f9e7a5a3fb
Increase timeout for changelogger (#35584) 2022-11-15 09:05:09 -06:00
Roy Ho 9d6fe84b47
Fail check when template updated without version (#35510)
* Fail check when template updated without version

* Update logic
2022-11-09 08:39:22 -08:00
Roy Ho 254fbd994f
Remove post merge comment about adding testing instructions (#35498) 2022-11-04 11:39:13 -07:00
Roy Ho ce133089b9
Fix random failing changelog entry (#35425) 2022-11-04 09:11:26 -07:00
Paul Sealock 19f0410bc1
PHPCS: Lint only changes (#35407)
Add `sirbrillig/phpcs-changed` to run PHPCS only on changes, not entire files
2022-11-01 07:53:01 +13:00
Alejandro López Ariza 4cb9bc45c4
Updated COT plugin used to set up the local environment with COT enabled. (#34990)
* Updated COT plugin used to set up the local environment with COT enabled.

* Enable HPOS in the e2e environment using env var (#35057)

* Updated COT plugin used to set up the local environment with COT enabled.

* Used env var to enable HPOS

* Disable HPOS for performance tests

* Updated permissions

* Set ENABLE_HPOS to 1

Co-authored-by: Alex López <alex.lopez@automattic.com>

Co-authored-by: Jamel Noel Reid <MrJnrman@users.noreply.github.com>
2022-10-26 09:29:41 -07:00
jonathansadowski 1807fb20a4
Fix reference to HEAD to work with community PRs (#35315) 2022-10-25 16:36:46 -05:00
Sam Seay 197c43e557
Remove phpcs Github action in favor of running phpcs directly (#35237) 2022-10-25 20:17:38 +13:00
rodelgc 39b472be7a
Set paths for `allure-results`, `test-results.json`, and save state files to be inside their respective E2E or API folders (#35206)
* Set paths to e2e-pw

* Delete and untrack storage state files

* Add changelog

* Checkout updated version of smoke test daily workflow

* Allow setting allure output paths to e2e-pw folder using environment variables

* Set allure output paths to be inside api-core-tests/api-test-report folder

* Remove unnecessary TODO comment
2022-10-24 16:53:46 -07:00
rodelgc 64320a2c5c
Migrate Daily smoke tests to Playwright (#35114)
* Checkout updated files

* Checkout more files

* Remove invalid php-version input

* Add changelog

* Update pnpm-lock.yaml

* Re-add accidentally deleted filter value

* Remove unnecessary TODO comment
2022-10-21 09:07:53 -07:00
Sam Seay 4fc33dc3ef
Add back the code reference updates action (#35140)
Add back the code reference updates action.
2022-10-20 13:52:03 +13:00
Roy Ho 53cb2ada67
Revise logic to update changelog.txt entries (#35086)
* Add logic to update the Stable version after a release

* Revise post release automation to only update changelog
2022-10-18 15:44:20 -07:00
jonathansadowski d0e277e80b
Add PR_ASSIGN_TOKEN to contributors action (#35137)
Adds PR_ASSIGN_TOKEN with repo permissions to the community contributors actions so that the bot has permission to make team assignments
2022-10-18 09:20:22 -05:00
jonathansadowski 1eecefb715
Update version of auto-assign-reviewer (#35104)
* Update version of auto-assign-reviewer

Update version of auto-assign-reviewer action to one that fixes the bug with team reviewers.

* Update community pr assigner config for team syntax

The new version of community pr assigner uses `team: ` prefix to designate teams.
2022-10-14 14:23:39 -05:00
jonathansadowski 699e14d4c7
Action: Assign community PRs a reviewer (#35041)
* Community contrib action: Set output with contributor status

* Assign community PRs a reviewer

* Create configuration for community PR labeler
2022-10-13 14:19:36 -05:00
Roy Ho 8db5b7be8b
Dev/update code freeze workflow (#35078)
* Update changelog command post PNPM update

* Add version bump to trunk for next dev cycle

* Remove reviewer part
2022-10-13 11:23:03 -07:00
Sam Seay 55b49cb50a
Fix and consolidate linting across the monorepo (#35012) 2022-10-12 15:05:01 +13:00
Roy Ho eef006dad1
Remove PHP 7.2 and set 7.4 as default (#35021) 2022-10-11 12:55:36 -07:00
Sam Seay f9bea12589
Disable lint in CI for now since its broken (#35010) 2022-10-10 17:06:37 +13:00
Sam Seay eb6fa6367f
Label community contributed PRs (#34971) 2022-10-10 11:34:52 +13:00
Jonathan Lane aa438e5741
Fix/update playwright and test fixes (#34790)
* Update Playwright 1.22.1 -> 1.26.0

* Update page title locator

* Wait for View cart link after adding to cart

* Add a wait when applying second coupon

* Fix flakey coupon test

* Ensure Store is in US

* Add changelog

* Revert lock file commit

This reverts commit b75192f53c.

* Updated lock file

* Added command for Playwright test execution

* Resolve merge conflict

* Resolve merge conflict

* Fixed pnpm-lock.json

* Update PW Selectors (#34959)

* Updated PW selectors

* Updated selector

* Removed extra command

* Ensure Billing country is set

* Fix failing payment setup test

* Old test is still running in actions

* Adds colour output for Playwright in GH Actions

* Fix payment test

* Lock file update

* Skipping a test for now

Co-authored-by: Jon Lane <jon.lane@automattic.com>
Co-authored-by: jamelreid <jnoelreid@gmail.com>
Co-authored-by: Jamel Noel Reid <MrJnrman@users.noreply.github.com>
2022-10-07 12:39:39 -06:00
rodelgc f8ea46b086
Implement `maxFailures` and warning messages (#33977)
* Add `maxFailures`

* Intentionally fail a test for testing the failure limit

* Add changelog

* Add step to get total number of E2E tests

* Increase max failures

* Add warnings to the test summary.
Also moved some stuff around.

* Update changelog

* Use env var to set `maxFailures` value instead of hard-coding it

* Make sure env var is converted to a number

* Allow maxFailures even if not on CI

* Show incomplete e2e run warning only when E2E_MAX_FAILURES is set

* Safely escape new lines in comment body

* Fix returned stringified summary

* Remove unnecessary newline escaping

* Fix incorrect reference to E2E_MAX_FAILURES

* Minor formatting change

* Shorten warning messages

* Minor change in warning message

* Intentionally fail only 1 test

* Remove intentional failures

* Removed duplicate env declaration

* Correct pnpm command and playwright config file location

* Simplify reference to CI variable

* Remove unnecessary if condition

* Remove references to the old E2E_PLAYWRIGHT flag

* Remove references to the old E2E_PPTR_SUMMARY_PATH variable

* Test for 2 warnings

* Revert to correct locator

* Test for only 1 warning

* Pass all tests
2022-10-05 10:44:20 -07:00
Roy Ho 17c8a7fb0a
Remove failing jobs (#34902)
Remove failing jobs"
2022-10-04 05:36:58 -07:00
nigeljamesstevenson 456dd02352
update workflows to use the playwright api-core-tests instead of the … (#34935)
* update workflows to use the playwright api-core-tests instead of the equivalent supertest versions

* add changelog file
2022-10-03 10:49:42 -06:00
nigeljamesstevenson 75c91ccdc0
E2e pw/migrate api core tests to use playwright 34126 (#34835)
* playwright api-core-tests

* playwright api-core-tests

* playwright api-core-tests

* Workflow updates

* Workflow test updates

* workflow test update

* Workflow testing

* Workflow testing

* workflow test updates

* workflow test updates

* Workflow update test

* Workflow update test restore

* add gitignore for api core tests output dir

* Workflow test with playwright api tests

* Workflow restore to original

* update readme and playwright config
2022-10-03 07:41:27 -06:00
Sam Seay 59e0c80f3c
Update the pnpm constraint to 7.12.2+ now auto-install is fixed. (#34872) 2022-10-03 10:52:42 +13:00
Roy Ho 2c5d6b95cb
Replace expired token with generic GH token (#34901) 2022-09-30 07:48:38 -07:00