Commit Graph

101 Commits

Author SHA1 Message Date
Rodel Calasagsag 2aefd9a301 Modify if condition 2023-09-01 12:12:36 +08:00
Rodel Calasagsag 08eab1046c Additional skip check 2023-09-01 11:39:10 +08:00
Rodel Calasagsag 0207bbff97 Handle skipped plugin tests 2023-09-01 11:35:37 +08:00
Rodel Calasagsag 23dce606c5 Merge branch 'trunk' into e2e/slack-daily-plugins 2023-08-29 15:50:20 +08:00
nigeljamesstevenson 879136404d
Updates smoke-test-daily.yml workflow to correctly remove test execution (#39904)
Updates smoke-test-daily.yml workflow to correctly remove additional test execution as previous revert PR #39889 missed a step
2023-08-25 18:06:46 +08:00
Rodel Calasagsag 4a9361da46 Merge branch 'trunk' into e2e/slack-daily-plugins 2023-08-25 14:42:58 +08:00
nigeljamesstevenson e0c569baff
revert #39803 Add additional Perf Test Daily run (#39889)
revert #39803
2023-08-24 13:27:58 -07:00
Rodel Calasagsag 0573492a74 Minor spacing corrections 2023-08-23 08:45:58 +08:00
Tam Mullen 661c8a1d47 Updated perf test env var 2023-08-18 10:32:35 +01:00
Tam Mullen fdd1196b58 Updated YML file 2023-08-18 09:54:04 +01:00
Tam Mullen 7efb1b5327 Added workflow job for w express perf tests 2023-08-18 09:44:29 +01:00
Rodel Calasagsag 876a0eb6b9 Get actual run date 2023-08-17 11:37:42 +08:00
Rodel Calasagsag c1771cb372 Add missing required env vars 2023-08-17 10:42:13 +08:00
Rodel Calasagsag dbbb2fb6ca Allow Slack block steps to run on e2e failure 2023-08-16 18:02:06 +08:00
Rodel Calasagsag 77277cdfbf Change slack channel id 2023-08-16 16:44:11 +08:00
Rodel Calasagsag c092494c1e Remove comments 2023-08-16 16:41:12 +08:00
Rodel Calasagsag d14d63114b Fix artifact name 2023-08-16 08:43:49 +08:00
Rodel Calasagsag 7317e60bbe Minor spacing fix 2023-08-15 16:53:05 +08:00
Rodel Calasagsag 2932e11ae2 Add job to send slack summary 2023-08-15 16:52:20 +08:00
Rodel Calasagsag eb458bbdf3 Remove unnecessary env var 2023-08-15 16:51:24 +08:00
Rodel Calasagsag 4211c1424f Add step to create and archive plugin test slack blocks 2023-08-15 16:51:06 +08:00
Rodel Calasagsag 58af912bfa Specify job outputs 2023-08-15 16:48:22 +08:00
Rodel Calasagsag a6039aa1b6 Use composite actions in plugin tests job 2023-08-15 16:47:20 +08:00
Rodel Calasagsag 92e15b9def Use composite actions in k6 job 2023-08-15 16:44:50 +08:00
Rodel Calasagsag c40b6e7e7e Use composite actions in E2E test job 2023-08-15 16:44:12 +08:00
Rodel Calasagsag 425063bf0e Use composite actions in API tests job 2023-08-15 16:42:33 +08:00
Jon Lane 9c571c97ce Update API URL in workflow and run all API tests 2023-07-20 14:26:59 -07:00
nigeljamesstevenson ff50a69256
Remove the unnecessary `daily.playwright.config` (#38336) 2023-05-20 00:49:07 +01:00
rodelgc 58f113ec47 Remove daily playwright config 2023-05-18 00:21:26 +08:00
rodelgc 149f83aa2e Enable all jobs in daily workflow 2023-05-13 09:31:55 +08:00
rodelgc 57c6e4b63b Use pnpm script from package.json 2023-05-12 14:31:45 +08:00
rodelgc 56db033947 REplace repo field with slug 2023-05-12 14:29:37 +08:00
rodelgc e71537b118 Disable reporting 2023-05-12 13:05:52 +08:00
rodelgc 244c216c76 Update report job matrix 2023-05-12 12:32:31 +08:00
rodelgc 93bb793178 Comment all other jobs 2023-05-12 12:27:18 +08:00
rodelgc 074c8b871a Test Gutenberg and its nightly build 2023-05-12 12:21:23 +08:00
rodelgc a91e48f6ee Run only Gutenberg tests 2023-05-11 18:19:19 +08:00
rodelgc 46300131ff
Correct `UPDATE_WC` value in the "Smoke test daily" workflow (#37049)
* Correct UPDATE_WC value

* Add changelog
2023-03-03 13:58:21 +01:00
rodelgc ea1925c841
Enable "Smoke test release" workflow (#36598)
* First pass at updating release test workflow

* Add changelog

* Set dir env variables

* Update to workflow

* Fix indent

* Fix indent

* Clean up indent

* Re-order steps

* Change order of jobs

* Added common php versions

* Update pipeline

* Update some labels

* Simplify for testing

* Update paths

* Create tmp folder

* Fix path

* Paths

* Try outputting some debugging

* Add step ID back

* Remove working directory

* Another path tweak

* Add API release tests

* Add k6 tests

* Add PHP tests

* Launch wp-env during PHP tests

* Try default values

* Tweak some settings, add WP testing

* Tweak some settings

* Re-order e2e steps

* Update step descriptions

* Reorganize tests, add plugin tests

* Enable only e2e job

* Initial set up to run against release smoke test site

* Fix syntax

* Temporarily disable update wc spec

* Temporarily disable downloading woocommerce zip

* Download release zip using tag name

* Fix wrong job name

* Fix wrong job name

* Fix dir

* Delete fetch-asset-id.js

* REfactor update-woocommerce spec

* Add error handling

* Download release zip by tag

* Refactor update woo spec to download zip by tag

* Correct job name

* fail test on invalid tag

* Enable all e2e tests

* Run api tests before e2e tests

* Fix job dependency

* Add customer credentials to api job

* Separate job for WC Update

* Combine e2e allure-results, then report

* Enable report job

* Fix context

* Change job and artifact names

* Use test s3 path

* Minor job name change

* Upload artifacts to bucket

* Correct s3 path

* Add quiet option

* Retain video on failures

* Finalize s3 path

* Try WP latest-1

* Revert to wp latest

* Refine search for woocommerce zip asset

* Get created-at

* Specify repo in gh command

* Slugify env description

* Trim space

* Sync with bucket instead of copy

* Remove invalid --recursive flag

* Re-add missing step to combine e2e results from update wc test

* Ensure artifact upload on test failure

* Enable all e2e tests on WP latest

* Retain existing data before updating WC

* Make test compatible with 'Canceled' and 'Cancelled'

* Set env_desc as env var

* Re-add deleted file

* Fix UPDATE_WC in daily smoke test workflow

* Add tracing in global setup

* Remove tracing

* Temporarily run only basic spec

* Job for WP Latest-1 & 2

* Fix "Required input 'created_at' not provided"

* Minor rename

* Remove install filter

* Install deps in get-wp-matrix

* Delete get-wp-versions.js

* Add get-wp-versions.js to e2e-pw folder

* REname file

* REfactor

* Refactor script for getting WP prev versions

* Update job dependencies

* Temporarily remove disabled jobs

* Allow e2e-wp-latest after api test failure

* Update L-1 & L-2 job deps

* Fix report-wp-latest

* Fix failing api test

* Make get-wp-versions quicker

* Publish report immediately after test

* Test reporting in e2e-update-wc

* Fix missing parameter

* Fix env_desc, re-enable other jobs

* Enable all e2e tests

* Minor job name change

* Fix flaky test

* Add php version testing

* stringify php versions

* Re-enable all e2e tests

* Up timeout to 2min

* Remove PHP 8.0

* Add missing conditionals

* Fix php version verification script

* Fix starting dir

* Fix flakiness

* Skip e2e if api failed

* Verify woocommerce.zip early

* Add token

* Delete test summary on github for the meantime

* Use default playwright config

* More meaningful variable names

* Update step titles based on review

* Use expect.poll()

* Minor spacing corrections

* Use `stable-check` endpoint, delete unnecessary loop

* Update locators to be JN-compatible

* Fix erroneous getting of release tag

* Fix conflict of "No thanks" button locator with that of WP Mail Logging's

* Update github-script action to v6

* Revert to 'Cancelled'

* Remove unnecessary step

* Provide missing env variables

---------

Co-authored-by: Jon Lane <jon.lane@automattic.com>
Co-authored-by: Jonathan Lane <lanej0@users.noreply.github.com>
2023-02-27 12:11:46 +08:00
nigeljamesstevenson add0c4a5be update after code review 2023-01-31 16:29:15 +00:00
nigeljamesstevenson 7cf1599208 update smoke test daily workflow to run api tests first 2023-01-18 17:12:55 +00:00
Roy Ho 16386238cc
Add minimum required permissions for workflow token (#36200)
* Add minimum required permissions to e2e-tests

* Add minimum token permission to more workflows
2023-01-02 09:28:11 -08:00
rodelgc 3daeac9a69
Migrate E2E daily shopper tests (#35873)
Enable shopper E2E tests on Daily smoke test workflow
2022-12-28 18:28:17 +08:00
rodelgc c4260c935f
Consistent folder structure for E2E and API test results (#35907)
Consistent folder structure for E2E and REST API test results
2022-12-19 14:33:22 +08:00
rodelgc 93b6d358f4
Migrate merchant daily smoke tests to Playwright (#35492)
* Checkout files

* Remove reverted permalinks code

* Enable cron

* More accurate job names

* More accurate test summary title
2022-12-06 11:22:49 +08:00
rodelgc 39b472be7a
Set paths for `allure-results`, `test-results.json`, and save state files to be inside their respective E2E or API folders (#35206)
* Set paths to e2e-pw

* Delete and untrack storage state files

* Add changelog

* Checkout updated version of smoke test daily workflow

* Allow setting allure output paths to e2e-pw folder using environment variables

* Set allure output paths to be inside api-core-tests/api-test-report folder

* Remove unnecessary TODO comment
2022-10-24 16:53:46 -07:00
rodelgc 64320a2c5c
Migrate Daily smoke tests to Playwright (#35114)
* Checkout updated files

* Checkout more files

* Remove invalid php-version input

* Add changelog

* Update pnpm-lock.yaml

* Re-add accidentally deleted filter value

* Remove unnecessary TODO comment
2022-10-21 09:07:53 -07:00
Sam Seay 04b74c1053
Upgrade to pnpm 7 (#34661)
* Remove -- -- from scripts that fixed issue in pnpm 6, fix a couple TS issues in sep packages
* Minor fixes to analyzer scripts and doco based on pnpm 7
* Add dompurify types to data package to avoid TS errors
* Remove pnpx in favor of pnpm exec
* Modify the code analyzer to respect pnpm version if present.
* Update instructions for running recursive lint, add comments to explain
2022-09-16 09:21:42 +12:00
Christopher Allford a49f23abfd
Optimize Workflow Caching (#34607)
* Added Monorepo Setup Action

This action will handle the installation, building, and caching for all
projects within the monorepo. It has inputs for skipping builds and
filtering so that only specific packages are installed and built.

* Removed Test Caching

Caching the results of tests, while an interesting way to avoid
unnecessary execution, seems error-prone. We can't adequately
capture the environment such that we can rely on this working.
For instance, changes in PHP version might break PHPUnit tests,
but, cached runs will be shared across different workflows.

It seems better to just not cache it and rely on people only testing
code that has actually changed in the pull request.

* Added Input Parsing

This SHOULD allow for passing multiple filters as an array argument.

* Updated Workflows

This fully utilizes the new action across all of the workflows.

* Always Cache Build Output

Since there may be other commands run that populate the cache, we
should make sure that we are always caching just in case.
2022-09-13 13:55:03 +12:00
Tam Mullen def11b77d9
Update daily smoke test to add k6 smoke tests (#33658)
* Add k6 test scenario to the daily smoke test action.

* Add blank patch changelog for daily smoke update
2022-06-29 16:01:37 -06:00