Commit Graph

37 Commits

Author SHA1 Message Date
rodelgc ea1925c841
Enable "Smoke test release" workflow (#36598)
* First pass at updating release test workflow

* Add changelog

* Set dir env variables

* Update to workflow

* Fix indent

* Fix indent

* Clean up indent

* Re-order steps

* Change order of jobs

* Added common php versions

* Update pipeline

* Update some labels

* Simplify for testing

* Update paths

* Create tmp folder

* Fix path

* Paths

* Try outputting some debugging

* Add step ID back

* Remove working directory

* Another path tweak

* Add API release tests

* Add k6 tests

* Add PHP tests

* Launch wp-env during PHP tests

* Try default values

* Tweak some settings, add WP testing

* Tweak some settings

* Re-order e2e steps

* Update step descriptions

* Reorganize tests, add plugin tests

* Enable only e2e job

* Initial set up to run against release smoke test site

* Fix syntax

* Temporarily disable update wc spec

* Temporarily disable downloading woocommerce zip

* Download release zip using tag name

* Fix wrong job name

* Fix wrong job name

* Fix dir

* Delete fetch-asset-id.js

* REfactor update-woocommerce spec

* Add error handling

* Download release zip by tag

* Refactor update woo spec to download zip by tag

* Correct job name

* fail test on invalid tag

* Enable all e2e tests

* Run api tests before e2e tests

* Fix job dependency

* Add customer credentials to api job

* Separate job for WC Update

* Combine e2e allure-results, then report

* Enable report job

* Fix context

* Change job and artifact names

* Use test s3 path

* Minor job name change

* Upload artifacts to bucket

* Correct s3 path

* Add quiet option

* Retain video on failures

* Finalize s3 path

* Try WP latest-1

* Revert to wp latest

* Refine search for woocommerce zip asset

* Get created-at

* Specify repo in gh command

* Slugify env description

* Trim space

* Sync with bucket instead of copy

* Remove invalid --recursive flag

* Re-add missing step to combine e2e results from update wc test

* Ensure artifact upload on test failure

* Enable all e2e tests on WP latest

* Retain existing data before updating WC

* Make test compatible with 'Canceled' and 'Cancelled'

* Set env_desc as env var

* Re-add deleted file

* Fix UPDATE_WC in daily smoke test workflow

* Add tracing in global setup

* Remove tracing

* Temporarily run only basic spec

* Job for WP Latest-1 & 2

* Fix "Required input 'created_at' not provided"

* Minor rename

* Remove install filter

* Install deps in get-wp-matrix

* Delete get-wp-versions.js

* Add get-wp-versions.js to e2e-pw folder

* REname file

* REfactor

* Refactor script for getting WP prev versions

* Update job dependencies

* Temporarily remove disabled jobs

* Allow e2e-wp-latest after api test failure

* Update L-1 & L-2 job deps

* Fix report-wp-latest

* Fix failing api test

* Make get-wp-versions quicker

* Publish report immediately after test

* Test reporting in e2e-update-wc

* Fix missing parameter

* Fix env_desc, re-enable other jobs

* Enable all e2e tests

* Minor job name change

* Fix flaky test

* Add php version testing

* stringify php versions

* Re-enable all e2e tests

* Up timeout to 2min

* Remove PHP 8.0

* Add missing conditionals

* Fix php version verification script

* Fix starting dir

* Fix flakiness

* Skip e2e if api failed

* Verify woocommerce.zip early

* Add token

* Delete test summary on github for the meantime

* Use default playwright config

* More meaningful variable names

* Update step titles based on review

* Use expect.poll()

* Minor spacing corrections

* Use `stable-check` endpoint, delete unnecessary loop

* Update locators to be JN-compatible

* Fix erroneous getting of release tag

* Fix conflict of "No thanks" button locator with that of WP Mail Logging's

* Update github-script action to v6

* Revert to 'Cancelled'

* Remove unnecessary step

* Provide missing env variables

---------

Co-authored-by: Jon Lane <jon.lane@automattic.com>
Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 12:11:46 +08:00
jonathansadowski e8a170165a
Update the release date for code freeze action readme generation (#36877)
Update the release date for code freeze action
2023-02-20 18:00:14 +13:00
jonathansadowski bea954c2e5
Update code-freeze actions to Monday 22 days before release (#36023)
* Update code-freeze actions to Monday 22 days before release

* Fix incorrect params in workflow dispatch
2022-12-20 10:51:41 -06:00
Sam Seay 03d52ff13b
Another set of migrations from set-output to GITHUB_OUTPUT (#35843)
Co-authored-by: Roy Ho <roykho77@gmail.com>
2022-12-16 11:41:03 +13:00
Sam Seay 7224c76053
Add a set_output function to script that uses the new GITHUB_OUTPUT (#35894) 2022-12-13 10:27:42 +13:00
rodelgc 93b6d358f4
Migrate merchant daily smoke tests to Playwright (#35492)
* Checkout files

* Remove reverted permalinks code

* Enable cron

* More accurate job names

* More accurate test summary title
2022-12-06 11:22:49 +08:00
jonathansadowski 973961fbd5
Add workflow for building live branches (#35344)
* Add workflow for building live branches

* Fix inadvertent newline

* Only run on PR for default types
2022-11-15 13:58:27 -06:00
rodelgc 64320a2c5c
Migrate Daily smoke tests to Playwright (#35114)
* Checkout updated files

* Checkout more files

* Remove invalid php-version input

* Add changelog

* Update pnpm-lock.yaml

* Re-add accidentally deleted filter value

* Remove unnecessary TODO comment
2022-10-21 09:07:53 -07:00
jonathansadowski 699e14d4c7
Action: Assign community PRs a reviewer (#35041)
* Community contrib action: Set output with contributor status

* Assign community PRs a reviewer

* Create configuration for community PR labeler
2022-10-13 14:19:36 -05:00
Sam Seay eb6fa6367f
Label community contributed PRs (#34971) 2022-10-10 11:34:52 +13:00
rodelgc f8ea46b086
Implement `maxFailures` and warning messages (#33977)
* Add `maxFailures`

* Intentionally fail a test for testing the failure limit

* Add changelog

* Add step to get total number of E2E tests

* Increase max failures

* Add warnings to the test summary.
Also moved some stuff around.

* Update changelog

* Use env var to set `maxFailures` value instead of hard-coding it

* Make sure env var is converted to a number

* Allow maxFailures even if not on CI

* Show incomplete e2e run warning only when E2E_MAX_FAILURES is set

* Safely escape new lines in comment body

* Fix returned stringified summary

* Remove unnecessary newline escaping

* Fix incorrect reference to E2E_MAX_FAILURES

* Minor formatting change

* Shorten warning messages

* Minor change in warning message

* Intentionally fail only 1 test

* Remove intentional failures

* Removed duplicate env declaration

* Correct pnpm command and playwright config file location

* Simplify reference to CI variable

* Remove unnecessary if condition

* Remove references to the old E2E_PLAYWRIGHT flag

* Remove references to the old E2E_PPTR_SUMMARY_PATH variable

* Test for 2 warnings

* Revert to correct locator

* Test for only 1 warning

* Pass all tests
2022-10-05 10:44:20 -07:00
jonathansadowski dc2c7a4996
Add workflow to label community PRs and Issues. (#34309)
* Add workflow to label community PRs and Issues.

* Move most of the logic for the script to check for community contributors into a separate module
2022-08-23 14:01:12 -05:00
Jamel Noel Reid b85819f2a9
Fix broken API tests (#34345)
* Bumped jest to v27

* Used jest in packaged node_modules

* Jest bump

* Removed puppeteer related code

* Updated CHANGELOG

* Added changelog
2022-08-17 11:04:41 -07:00
jonathansadowski 6670af0d64
Split changelog steps from code-freeze action (#34027)
* Split changelog script from code-freeze

* Remove hardcoded repository info from testing

* Remove generation of changelog.txt file from automation

* Correct comments and add back write of readme
2022-07-21 17:58:23 -05:00
rodelgc 9e576c2208
Upload API and E2E test reports to S3 bucket (#33843)
* Simplify if expression

* Call the new workflow in Woo Test Reports repo for publishing PR test reports

* Rename Playwright-related steps

* Archive Puppeteer screenshots only when Playwright flag is off

* Correct step name and if condition in Puppeteer steps

* Run only a single Playwright test for testing

* Correct workflow name

* Add commit message as input

* Remove invalid COMMIT_MESSAGE input

* Add root_dir input

* Update s3 root dir input

* Update link to full E2E test report

* Re-enable all Playwright tests
2022-07-14 14:15:20 -07:00
rodelgc aed4763414
Post test result summary in workflow run and in PR comment using github-actions[bot] (#33675)
* Temporarily toggle Playwright feature flag on

* Add job for test summary

* Add dummy run command

* Debug object contents

* Test getting json from prev step output

* Try toJSON()

* Test access to keys

* Try using context

* Use separate file

* Debug steps

* Try dummy job summary

* Minor job rename

* Add script for preparing job summary

* Fix context syntax error

* Correct artifact path

* List contents of artifacts folder

* inspect API summary contents

* inspect apisummary

* Remove dummy step for listing artifacts folder contents

* Parse number to string

* Correct SHA

* Try printing job summary on next step

* Create PR comment

* Minor step rename to test PR comment update

* Add another line break

* Test for puppeteer tests

* Test full Playwright test run

* Test full Puppeteer e2e run

* Fix spacing inconsistency

* Specify other test result categories in PPTR run

* Rename script file

* Minor refactor and documentation on script file

* Minor addition to script documentation
2022-07-06 11:17:51 -06:00
Roy Ho 98cf4e7160
Add PR numbers to changelog entries on code freeze process (#33627)
* Add PR numbers to changelog entries on code freeze process

* Add markdown links to PR
2022-06-28 11:15:39 -07:00
jonathansadowski 626a409e83
Release automation: generate changelog during code freeze (#33046)
* Update code freeze script to generate changelog PRs

* Add changelog file

Co-authored-by: jonathan <jonathan@jonathans-MacBook-Pro.local>
2022-06-16 14:44:07 -05:00
jonathansadowski 7eb22eabde
Update code freeze script exit code on failure (#33340)
Update code freeze exit code
2022-06-07 18:13:18 -05:00
Jonathan Sadowski bf6b16972c Update PR template and post-merge comment for changelogger 2022-04-13 10:16:33 -05:00
jonathansadowski 1cc61de404
Merge pull request #31873 from woocommerce/add/release-code-freeze
Add github workflow to automatically enforce the code freeze.
2022-02-25 16:14:49 -06:00
Jonathan Sadowski ad201cde88 Improve messaging when milestone or release branch already exist 2022-02-25 15:02:15 -06:00
Jonathan Sadowski 439ae1f17b Update code-freeze script to base its decision on latest release 2022-02-25 12:22:07 -06:00
Barry Hughes 31884d9052
Update wording. 2022-02-17 09:06:29 -08:00
barryhughes a5a712fbd0 Update references to labels in the post merge commit. 2022-02-16 15:39:20 -08:00
Jonathan Sadowski 49915391d0 Update newlines to PHP_EOL in post-request-shared.php 2022-02-11 08:32:29 -06:00
jamelreid 79a3e8f7ba Fixed typo 2022-02-10 15:22:03 -05:00
Jonathan Sadowski 18a5d3a751 Remove confusing trailing dots near version numbers; use PHP_EOL; add newline to EOF 2022-02-10 08:51:26 -06:00
Jonathan Sadowski 757209ba37 Add github workflow to automatically enforce the code freeze. 2022-02-09 13:43:05 -06:00
jamelreid 0164d219b8 Updated asset index 2022-02-04 08:16:41 -05:00
jamelreid 67f4720e0d Script to fetch an asset id for a release 2022-02-04 07:30:57 -05:00
Nestor Soriano 6fb769bc9d
Refactor the PR post-merge processing GitHub action
Common code has been extracted to a "post-request-shared.php" script.
2021-08-03 11:33:33 +02:00
Nestor Soriano b3617a3115
Extend post-merge GH action to add a comment suggestion next steps. 2021-08-03 10:07:24 +02:00
Nestor Soriano 9ffddbac1f
Use 'pull_request_target' in the automatic milestoning action
The original event used, 'pull_request', runs in the context of the
pull request. In the case of pull requests created from forks this means
that a read-only GitHub token is used for API calls, and thus the
call used to assign the milestone fails.

The fix is using the 'pull_request_target' event, which runs in the
context of the base branch, and thus with a read-write token. See:
https://docs.github.com/en/actions/reference/events-that-trigger-workflows#pull_request_target

Also, added a warning message when no milestone without a release
branch is found and thus the newest existing milestone is assigned
(this is a fallback that should normally not happen).
2021-07-29 16:47:27 +02:00
Nestor Soriano 68fcfa9cf6
Add logging to the GH action for assigning a milestone to merged PRs
If the GH API call to assign the milestone to a merged PR returns an
error, the data returned from the call will be dumped to the console,
and thus will be readable by looking at the action execution log.
2021-07-28 15:20:16 +02:00
Néstor Soriano 3ada9fc562
Change GH API query for milestones to get only the open ones.
Co-authored-by: Vedanshu Jain <vedanshu.jain.2012@gmail.com>
2021-04-19 14:46:45 +02:00
Nestor Soriano a0c965964b
Add a GitHub action for automatic assignment of milestone to merged PRs
The milestone is chosen as follows:
1. List all the open milestones with a title of "X.Y.Z"
2. Sort them desc using verson_compare
3. foreach them in order, and take the last one not having a
   corresponding release branch ("release/X.Y")
2021-04-15 17:16:00 +02:00