* Add workflow to remind PR reviewers to also review the testing instructions.
* Update review-testing-instructions GitHub Actions workflow to also mentioning teams in the comment.
* Update review-testing-instructions GitHub Actions workflow to find if the comment it posts was already posted and - in that case - update it instead of creating a new one.
* Pin find-comment and create-or-update-comment thirdparty GitHub Actions to full length commit SHA instead of tag for better security.
* Conditionally use authorization token when getting WC Zip download URL.
* Exponse error message
* Handle undefined authorization token more gracefully
* Specify GH token in workflow
* Add changelog
* Use E2E_GH_TOKEN instead of default github.token
* Provide GitHub token to other jobs
* Fix getting of tag name from triggered event
* Use "inputs" context instead
* Add release version to workflow name
* Correct concurrency group
* Fix workflow name
* Add --archive option
* Fix output setting
* Fix script for verifying asset
* Remove unnecessary 'uses' line
* Remove node 12 deprecation messages
* Re-add "published" release type
We recently added team names to the `focus: ...` labels:
https://github.com/woocommerce/woocommerce/labels?q=focus
The PR labeler relies on exact label names, however. This PR updates the PR labeler to conform to the new string.
* First pass at updating release test workflow
* Add changelog
* Set dir env variables
* Update to workflow
* Fix indent
* Fix indent
* Clean up indent
* Re-order steps
* Change order of jobs
* Added common php versions
* Update pipeline
* Update some labels
* Simplify for testing
* Update paths
* Create tmp folder
* Fix path
* Paths
* Try outputting some debugging
* Add step ID back
* Remove working directory
* Another path tweak
* Add API release tests
* Add k6 tests
* Add PHP tests
* Launch wp-env during PHP tests
* Try default values
* Tweak some settings, add WP testing
* Tweak some settings
* Re-order e2e steps
* Update step descriptions
* Reorganize tests, add plugin tests
* Enable only e2e job
* Initial set up to run against release smoke test site
* Fix syntax
* Temporarily disable update wc spec
* Temporarily disable downloading woocommerce zip
* Download release zip using tag name
* Fix wrong job name
* Fix wrong job name
* Fix dir
* Delete fetch-asset-id.js
* REfactor update-woocommerce spec
* Add error handling
* Download release zip by tag
* Refactor update woo spec to download zip by tag
* Correct job name
* fail test on invalid tag
* Enable all e2e tests
* Run api tests before e2e tests
* Fix job dependency
* Add customer credentials to api job
* Separate job for WC Update
* Combine e2e allure-results, then report
* Enable report job
* Fix context
* Change job and artifact names
* Use test s3 path
* Minor job name change
* Upload artifacts to bucket
* Correct s3 path
* Add quiet option
* Retain video on failures
* Finalize s3 path
* Try WP latest-1
* Revert to wp latest
* Refine search for woocommerce zip asset
* Get created-at
* Specify repo in gh command
* Slugify env description
* Trim space
* Sync with bucket instead of copy
* Remove invalid --recursive flag
* Re-add missing step to combine e2e results from update wc test
* Ensure artifact upload on test failure
* Enable all e2e tests on WP latest
* Retain existing data before updating WC
* Make test compatible with 'Canceled' and 'Cancelled'
* Set env_desc as env var
* Re-add deleted file
* Fix UPDATE_WC in daily smoke test workflow
* Add tracing in global setup
* Remove tracing
* Temporarily run only basic spec
* Job for WP Latest-1 & 2
* Fix "Required input 'created_at' not provided"
* Minor rename
* Remove install filter
* Install deps in get-wp-matrix
* Delete get-wp-versions.js
* Add get-wp-versions.js to e2e-pw folder
* REname file
* REfactor
* Refactor script for getting WP prev versions
* Update job dependencies
* Temporarily remove disabled jobs
* Allow e2e-wp-latest after api test failure
* Update L-1 & L-2 job deps
* Fix report-wp-latest
* Fix failing api test
* Make get-wp-versions quicker
* Publish report immediately after test
* Test reporting in e2e-update-wc
* Fix missing parameter
* Fix env_desc, re-enable other jobs
* Enable all e2e tests
* Minor job name change
* Fix flaky test
* Add php version testing
* stringify php versions
* Re-enable all e2e tests
* Up timeout to 2min
* Remove PHP 8.0
* Add missing conditionals
* Fix php version verification script
* Fix starting dir
* Fix flakiness
* Skip e2e if api failed
* Verify woocommerce.zip early
* Add token
* Delete test summary on github for the meantime
* Use default playwright config
* More meaningful variable names
* Update step titles based on review
* Use expect.poll()
* Minor spacing corrections
* Use `stable-check` endpoint, delete unnecessary loop
* Update locators to be JN-compatible
* Fix erroneous getting of release tag
* Fix conflict of "No thanks" button locator with that of WP Mail Logging's
* Update github-script action to v6
* Revert to 'Cancelled'
* Remove unnecessary step
* Provide missing env variables
---------
Co-authored-by: Jon Lane <jon.lane@automattic.com>
Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
* Remove check to skip testing instructions if needed from PR template and add final check to confirm that testing instructions were added.
* Update Pull Request template documentation to be consistent with the removal of the checkbox to skip testing instructions.
---------
Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
* Upgrade PHPUnit to v8, and remove usage of PHPUnit7 fork in CI
Once the minimum WordPress version supported by WooCommerce is 5.9
and the minimum PHP version supported is 7.2, we can switch to
PHPUnit 7 and the patched PHPUnit 7 with support for PHP 8 is
no longer needed.
* Resolve PHPUnit 8 deprecations in tests
* Add changelog file
* Fix code formatting
* Updated COT plugin used to set up the local environment with COT enabled.
* Enable HPOS in the e2e environment using env var (#35057)
* Updated COT plugin used to set up the local environment with COT enabled.
* Used env var to enable HPOS
* Disable HPOS for performance tests
* Updated permissions
* Set ENABLE_HPOS to 1
Co-authored-by: Alex López <alex.lopez@automattic.com>
Co-authored-by: Jamel Noel Reid <MrJnrman@users.noreply.github.com>
* Set paths to e2e-pw
* Delete and untrack storage state files
* Add changelog
* Checkout updated version of smoke test daily workflow
* Allow setting allure output paths to e2e-pw folder using environment variables
* Set allure output paths to be inside api-core-tests/api-test-report folder
* Remove unnecessary TODO comment
* Update version of auto-assign-reviewer
Update version of auto-assign-reviewer action to one that fixes the bug with team reviewers.
* Update community pr assigner config for team syntax
The new version of community pr assigner uses `team: ` prefix to designate teams.
* Update Playwright 1.22.1 -> 1.26.0
* Update page title locator
* Wait for View cart link after adding to cart
* Add a wait when applying second coupon
* Fix flakey coupon test
* Ensure Store is in US
* Add changelog
* Revert lock file commit
This reverts commit b75192f53c.
* Updated lock file
* Added command for Playwright test execution
* Resolve merge conflict
* Resolve merge conflict
* Fixed pnpm-lock.json
* Update PW Selectors (#34959)
* Updated PW selectors
* Updated selector
* Removed extra command
* Ensure Billing country is set
* Fix failing payment setup test
* Old test is still running in actions
* Adds colour output for Playwright in GH Actions
* Fix payment test
* Lock file update
* Skipping a test for now
Co-authored-by: Jon Lane <jon.lane@automattic.com>
Co-authored-by: jamelreid <jnoelreid@gmail.com>
Co-authored-by: Jamel Noel Reid <MrJnrman@users.noreply.github.com>
* Add `maxFailures`
* Intentionally fail a test for testing the failure limit
* Add changelog
* Add step to get total number of E2E tests
* Increase max failures
* Add warnings to the test summary.
Also moved some stuff around.
* Update changelog
* Use env var to set `maxFailures` value instead of hard-coding it
* Make sure env var is converted to a number
* Allow maxFailures even if not on CI
* Show incomplete e2e run warning only when E2E_MAX_FAILURES is set
* Safely escape new lines in comment body
* Fix returned stringified summary
* Remove unnecessary newline escaping
* Fix incorrect reference to E2E_MAX_FAILURES
* Minor formatting change
* Shorten warning messages
* Minor change in warning message
* Intentionally fail only 1 test
* Remove intentional failures
* Removed duplicate env declaration
* Correct pnpm command and playwright config file location
* Simplify reference to CI variable
* Remove unnecessary if condition
* Remove references to the old E2E_PLAYWRIGHT flag
* Remove references to the old E2E_PPTR_SUMMARY_PATH variable
* Test for 2 warnings
* Revert to correct locator
* Test for only 1 warning
* Pass all tests
* playwright api-core-tests
* playwright api-core-tests
* playwright api-core-tests
* Workflow updates
* Workflow test updates
* workflow test update
* Workflow testing
* Workflow testing
* workflow test updates
* workflow test updates
* Workflow update test
* Workflow update test restore
* add gitignore for api core tests output dir
* Workflow test with playwright api tests
* Workflow restore to original
* update readme and playwright config
* Add 2 new GitHub Actions workflows to include COT E2E and API automated tests in CI.
* Fix typo in GitHub Actions workflow name.
* Update new PR COT test workflow to trigger with any PR.
* Update new cot-pr-build-and-e2e-tests.yml GitHub Actions workflow to use the Monorepo Setup Action.
* Updated 'cot-build-and-e2e-tests-daily.yml' to use the new WooCommerce Monorepo action and 'cot-pr-build-and-e2e-tests.yml' to trigger on PRs labeled with the custom order tables tag.
* Remove -- -- from scripts that fixed issue in pnpm 6, fix a couple TS issues in sep packages
* Minor fixes to analyzer scripts and doco based on pnpm 7
* Add dompurify types to data package to avoid TS errors
* Remove pnpx in favor of pnpm exec
* Modify the code analyzer to respect pnpm version if present.
* Update instructions for running recursive lint, add comments to explain
* Added Monorepo Setup Action
This action will handle the installation, building, and caching for all
projects within the monorepo. It has inputs for skipping builds and
filtering so that only specific packages are installed and built.
* Removed Test Caching
Caching the results of tests, while an interesting way to avoid
unnecessary execution, seems error-prone. We can't adequately
capture the environment such that we can rely on this working.
For instance, changes in PHP version might break PHPUnit tests,
but, cached runs will be shared across different workflows.
It seems better to just not cache it and rely on people only testing
code that has actually changed in the pull request.
* Added Input Parsing
This SHOULD allow for passing multiple filters as an array argument.
* Updated Workflows
This fully utilizes the new action across all of the workflows.
* Always Cache Build Output
Since there may be other commands run that populate the cache, we
should make sure that we are always caching just in case.
* Add GitHub action and pre-push hook for syncpack for react and react-dom
* Fix typo in pre-push script
* Don't execute suggested command to fix mismatches automatically.
* Update react and react-dom to 17.0.2
* Move changelog check to last, so that pre-push still fails if check isn't successful
* Add changelog files
* Correct issue with syncpack.yml referencing an internal action
* Add command to top-level package.json for fixing mismatches
* Update syncpack action to only run when package.json files are affected
* Update workflow to just run syncpack list-mismatches
* Add version pinning to syncpackrc
This also adds a new command `major_minor` that contains some of the old logic of `getPluginData` to determine the latest major/minor version of WooCommerce
* Renamed e2e to e2e-legacy
* Moved e2e folder to tests/e2e
* Updated gitignore
* Updted init script to work with wp-env
* Updated readme
* Updated commands to match new test structure
* Updated workflow to work with new test structure
* Added changelog
* turned off alternate wp cron in test
* Fixed path to playwright config
* Renamed e2e to e2e-pw
* Renamed e2e-legacy to e2e
* Updated e2e-pw path
* Fixed workflow
* Fixed csv test
* Updated .gitignore
* Removed puppeteer specific step
* Change package json commands and turbo repo config to allow for greatly simplifying running turbo
* Remove require-turbo script which was causing caching issues
* Colorize CI output from scripts
* Add missing changelog script to extend-cart-checkout-block
* Update woocommerce admin scripts to align more with monorepo
* Add changelog
* Add dev script back in
* Add install-if-deps-outdated back into build script
* Remove --force from dev script and add WC_ADMIN_PHASE to turbo cache key
* Remove install-if-deps-outdated from build
* Add composer install step to pr unit tests workflow
* Remove old install scripts, that are not needed anymore
* Make sure client changes are part of the cache hit
* Add composer install to code coverage and code sniff as well
* Add composer_no_dev input to install-build workflow
* Add sass dependency to woocommerce/admin/client turbo cache
* Split changelog script from code-freeze
* Remove hardcoded repository info from testing
* Remove generation of changelog.txt file from automation
* Correct comments and add back write of readme
* Simplify if expression
* Call the new workflow in Woo Test Reports repo for publishing PR test reports
* Rename Playwright-related steps
* Archive Puppeteer screenshots only when Playwright flag is off
* Correct step name and if condition in Puppeteer steps
* Run only a single Playwright test for testing
* Correct workflow name
* Add commit message as input
* Remove invalid COMMIT_MESSAGE input
* Add root_dir input
* Update s3 root dir input
* Update link to full E2E test report
* Re-enable all Playwright tests
Remove change file normalization from code freeze action
This PR updates the code freeze action to remove the the change file normalization, as it was causing issues in this run of the action.
It should no longer be necessary now that we require valid change files and will soon have a pre-push hook to require that.
* Update github action to equire valid changelog file
* Update github action for changelogger to install depedencies and composer
* Make valid changelog files that were invalid
* Final updates for Playwright
* Update config
* Add uuid dependency
* Increase retries to 2
* Update selectors on shipping page
* Use baseURL instead of hard-coded URL for API
* Clarify comment
* Use baseURL instead of hard-coded URL
* Check to see if an order was created before attempting to delete it
* Add changelog
* Turn on Playwright tests in GitHub
* Increase timeout for CI execution
* Update configuration options (minor edit)
* Fix for checkout flaky test
* Parse orderId from URL
* Check for substring
* Streamline email tests
* Remove .only
* Only clear email logs of messages for test
* Get orderId from page element
* Fix for test not waiting for reset
* Add in second wait for Performance section
* Change significance from minor to patch
Co-authored-by: Jon Lane <jon.lane@automattic.com>
* Temporarily toggle Playwright feature flag on
* Add job for test summary
* Add dummy run command
* Debug object contents
* Test getting json from prev step output
* Try toJSON()
* Test access to keys
* Try using context
* Use separate file
* Debug steps
* Try dummy job summary
* Minor job rename
* Add script for preparing job summary
* Fix context syntax error
* Correct artifact path
* List contents of artifacts folder
* inspect API summary contents
* inspect apisummary
* Remove dummy step for listing artifacts folder contents
* Parse number to string
* Correct SHA
* Try printing job summary on next step
* Create PR comment
* Minor step rename to test PR comment update
* Add another line break
* Test for puppeteer tests
* Test full Playwright test run
* Test full Puppeteer e2e run
* Fix spacing inconsistency
* Specify other test result categories in PPTR run
* Rename script file
* Minor refactor and documentation on script file
* Minor addition to script documentation
* Add phpcs-changed
* Put report flag value in quotes
* Use phpcs action workflow instead
* Remove no longer used phpcs script
* Remove phpcs ignore rule
* Use proper name for code sniffer step
This commit changes it to woocommerce/client/admin. This is an invalid NPM package name and an invalid Composer package name. This will prevent conflicts but also identify it as a component of Core.
* Added event trigger for pre-releases
When a woocommece pre-release occurs, this workflow will trigger a testing event in another repo defined using project secrets.
* Set to only fire on prereleases
* Updated run condition
* Fixed run condition
This replaces all `nx` commands with `turbo` commands and removes Nx from the repository. All of the `project.json` files have been removed and any commands that broke with Turborepo have been adjusted.
In order to ease the identification of private packages within the monorepo, this adds an `internal-` prefix to all of the current private packages. This makes it immediately clear when reviewing `packages/js` what is and isn't a private package.
* Separate job for api test
* Remove unnecessary comment
* Use ubuntu 20.04 in job "publish-test-report"
* Simplify syntax in "if" condition
* Temporarily add push trigger for testing
* Temporarily disable running jest e2e tests for testing
* Remove unnecessary OR operator
* try: disable certain jobs for testing
* try: run 'hello' api test for testing
* Implement timeout override in API tests
* Allow workflow to override timeout of api tests
* Correct parsing of timeout override
* try: temporarily disable e2e job
* try: make api test verbose for troubleshooting
* try: temporarily disable some install steps for testing
* Correct smoke test env vars
* try: run all api tests
* Re-enable install-build steps
* Correct smoke test env var values
* Temporarily disable running all e2e tests after WC upgrade
* Remove unnecessary flag for daily smoke test runs
* Use different workflow for publishing daily smoke test report
* Use env vars instead of github expressions in accessing job outputs
* Correct multiline syntax
* Correct api_artifact input value
* Take only the first line of commit message
* Correct passing of input value
* Re-enable jobs before submitting for review
* Separate job for api test
* Remove unnecessary comment
* Use ubuntu 20.04 in job "publish-test-report"
* Simplify syntax in "if" condition
* Temporarily add push trigger for testing
* Temporarily disable running jest e2e tests for testing
* Remove unnecessary OR operator
* try: disable certain jobs for testing
* try: run 'hello' api test for testing
* Implement timeout override in API tests
* Allow workflow to override timeout of api tests
* Correct parsing of timeout override
* try: temporarily disable e2e job
* try: make api test verbose for troubleshooting
* try: temporarily disable some install steps for testing
* Correct smoke test env vars
* try: run all api tests
* Re-enable install-build steps
* Correct smoke test env var values
* Temporarily disable running all e2e tests after WC upgrade
* Remove unnecessary flag for daily smoke test runs
* Use different workflow for publishing daily smoke test report
* Use env vars instead of github expressions in accessing job outputs
* Correct multiline syntax
* Correct api_artifact input value
* Take only the first line of commit message
* Correct passing of input value
* Re-enable jobs before submitting for review
* Add push trigger for final testing
* Re-enable e2e tests, remove push trigger before merge
* Fix spacing
* Correct slack channel id
Sam Seay
Chi-Hsuan Huang
Paul Sealock
rodelgc
Barry Hughes
jonathansadowski
William Staley
nigeljamesstevenson
Alejandro López Ariza
Tam Mullen
Vedanshu Jain
louwie17
Leif Singer
Leif Singer
nigeljamesstevenson
Roy Ho
Néstor Soriano
Jonathan Lane
Christopher Allford
Jamel Noel Reid
Lucas Bustamante
Jonathan Sadowski
Jon Lane