Commit Graph

129 Commits

Author SHA1 Message Date
Adrian Moldovan 7fb40ffc0b
[testing workflows] Migrate release checks to ci.yml (#48113)
* Deprecate pr-smoke-test.yml

* Add configs for each environment

* Add scripts

* Add arguments to playwright command

* Update pnpm scripts and existing e2e projects

* Fix shellcheck warning

* Update test env start for Gutenberg stable run

* Add changelog

* Remove unused var

* Set the USE_WP_ENV var

* Configure daily e2e jobs

* Rename some steps

* Add release checks jobs and deprecate  smoke-test-release.yml

* Deprecate  smoke-test-release.yml

* Add tests-release-checks.yml to trigger ci.yml on push to tags

* Add the pull_request trigger to test the workflow

* Remove the pull_request trigger to test the workflow

* Add the pull_request trigger to test the workflow

* Add the tag input in ci.yml

* Remove import and use core reference

* Only build if build-plugin is true

* Use release-artifact instead

* Split setup-monorepo step

* Use tag: 'nightly' to test

* Use inputs.tag directly in the wp-env setup step

* Add script to override the wp-env config

* Run the env config script in ci

* Set the artifact name

* Fix workflow syntax

* Add changelog

* Validate RELEASE_TAG and ARTIFACT_NAME

* Use real ref type and name

* Use real ref type and name

* Use real ref type and name

* Use github context instead

* Test with 9.0.0-beta.2

* Add push to tags trigger directly in ci.yml

* Configure one job to run with nightly-checks

* Remove empty lines

* Format events array one element per line

* Format events array one element per line
2024-06-05 13:02:54 -03:00
Jonathan Lane e8e1391fa0
Add annotation for test PR e2e test run results (#48029)
* Add annotation for test results

* Minor change to trigger e2e tests

* Fix command, add emoji

* Update emoji

* Add changelog

* Update from PR feedback

* Revert "Add changelog"

This reverts commit 03656848eb.

* Revert "Minor change to trigger e2e tests"

This reverts commit 64e3a87b2d.

---------

Co-authored-by: Jon Lane <jon.lane@automattic.com>
2024-06-04 13:03:38 -07:00
Adrian Moldovan 9b251ce318
[testing workflows] Add test environments and deprecate the daily e2e workflow (#48101)
* Deprecate pr-smoke-test.yml

* Add configs for each environment

* Add scripts

* Add arguments to playwright command

* Update pnpm scripts and existing e2e projects

* Fix shellcheck warning

* Update test env start for Gutenberg stable run

* Add changelog

* Remove unused var

* Set the USE_WP_ENV var

* Configure daily e2e jobs

* Rename some steps
2024-06-03 14:12:45 -07:00
Adrian Moldovan 97e7f86a15
Add the slack-test-report util (#47805)
* Add the slack-test-report util

* Error if the required GitHub context variables are not set

* Error if the required SLACK_CHANNEL env var is not set

* Use a more generic message to be able to report on jobs that are not tests, like build

* Add the alert-on-failure job in ci

* Run for pull_request to test the job

* Setup monorepo first

* Force a linting error

* Better logging

* Use inputs.trigger as report name if set

* Force an e2e test failure

* Set the commit message in the notification

* Use INPUT_TRIGGER for all notifications

* Revert test changes
2024-05-24 13:18:54 -07:00
Adrian Moldovan 835625c329
Fix reporting step for daily e2e runs (#47764)
* Add condition on github.event_name for lint jobs

* Update a file to trigger a lint job

* Update condition to run only on pull_request

* Revert test change

* Always run the Slack notification step

* Add pull_request test trigger

* Update report step

* Remove branch filter

* Try with push event and branch filter

* Pass secrets to ci.yml

* Update testIgnore pattern for Gutenberg project

* Add changelog

* Remove the test push trigger

* One more test run

* Removed push test trigger
2024-05-23 09:12:10 -07:00
Adrian Moldovan f1489892aa
[ci] Only run lint jobs on pull_request event (#47652) 2024-05-23 07:16:07 -07:00
Adrian Moldovan 13ab9007f6
[testing workflows] Add workflow_call trigger in CI workflow - support daily and other types of runs (#47612)
* Add workflow_call event

* Add tests-daily-run.yml

* Remove workflow version

* Update input value

* Remove secrets from caller workflow

* Rename job

* Check context

* Check context

* Check workflow_call inputs

* Override github.event_name with inputs.trigger

* Fix input name

* Add config for daily e2e tests with Gutenberg

* Update env commands for Gutenberg run

* Add disableHpos env variable in ci-utils

* Revert the trigger to push

* Revert the env variable setting in ci-jobs

* Revert the env variable setting in ci-jobs

* Rebuild utils

* Update test env start command

* Define the Gutenberg Playwright project

* Use path relative to workspace root

* Update events for test projects

* Update events for test projects

* Add changelogs

* Remove the e2e tests with Gutenberg tests from daily runs

* Set the schedule and workflow_dispatch triggers

* Add --project default flag to only run the default tests in CI

* Renamed snapshots to match the new Playwright project name
2024-05-22 11:04:44 -07:00
Adrian Moldovan 0522511a72
[testing worklflows] Consolidate the Slack channel used for test reports (#47611) 2024-05-22 11:01:51 -03:00
Adrian Moldovan cf6e01d0d6
[ci-jobs] Add disableHpos test environment variable (#47619)
* Add disableHpos test env variable

* Update events for HPOS disabled tests

* Add pull_request event to test the env

* Remove the pull_request event

* Add changelog

* Make the disableHpos var a boolean

* Also set the env in ci.yml

* Remove console log statement

* Print woocommerce_custom_orders_table_enabled

* Run test setup script again to check output

* Capture env setup output and print it

* Get woocommerce_custom_orders_table_enabled value in test env script

* Get more details on HPOS setting

* Remove the DISABLE_HPOS logic from test env setup script

* Use DISABLE_HPOS instead of ENABLE_HPOS everywhere

* Turn off no-console and jest/no-standalone-expect eslint rules for api core tests

* Test API tests with HPOS disabled

* Fix eslint error

* Clean up debug statements

* Remove pull_request test trigger
2024-05-21 18:33:43 -03:00
Adrian Moldovan ccba59ce05
[ci] Only evaluate jobs statuses for pull request event (#47615)
* Condition evaluate status job by github event

* Test reverse condition

* Test with push event

* Revert condition to match pull_request
2024-05-21 18:28:24 -03:00
Adrian Moldovan 9b8063f6b2
[testing workflows] Add optional jobs in CI workflow (#47414)
* Include performance tests job in the needed jobs for evaluation

* Use Github api to get the workflow jobs status

* Evaluate project-jobs

* See results

* Check for cancelled jobs

* Add evaluate-project-jobs.js and rename optional jobs

* Fix script path

* Checkout first

* And then fix the path again.

* Update env variable name and be more detailed in what are the required variables

* Use a flat matrix env variable

* Update to trigger test jobs

* Use the jobs name to evaluate optional requirement

* Fix conditions

* Prettier print

* Test lint job failure

* Remove unused MATRIX env variable

* Fix test

* Remove unused MATRIX variable check

* Only run my account e2e tests

* Remove unused function

* Nicer console printing

* Revert change that triggers failing lint job

* Force failing e2e test

* Revert e2e test command and forced failure

* Added test data and test mode

* Added more test data

* Fixed lint errors and warnings

* Exclude .github folder from eslintignore

* Change to trigger everything

* Revert change to trigger everything
2024-05-16 13:31:17 -07:00
Jonathan Lane ad19ff51a3
Add Buildkite reporting for e2e PR and daily runs (#47558)
* Add Buildkite reporting for e2e PR runs

* Set up Buildkite reporting on daily runs

---------

Co-authored-by: Jon Lane <jon.lane@automattic.com>
2024-05-16 15:31:21 -03:00
Adrian Moldovan 7b96dfca67
[e2e tests] Remove BuildKite reporter (#47449)
* Remove BuildKite reporter

* Changelog

* Remove BuildKite reporter from api tests
2024-05-14 10:25:17 +01:00
Adrian Moldovan 981fca33d6
[testing workflows] Fix for multi-line commit message not triggering tests (#47353) 2024-05-10 10:12:36 -03:00
Jonathan Lane 396c29d3ec
Install and set up BuildKite test analytics (#47202)
Co-authored-by: Jon Lane <jon.lane@automattic.com>
Co-authored-by: Adrian Moldovan <3854374+adimoldovan@users.noreply.github.com>
2024-05-10 11:47:23 +03:00
Adrian Moldovan de91a0881d
[ci-jobs] Add support for github event and include HPOS disabled e2e tests in ci (#46922)
* Update env setup script to disable hpos

* Use DISABLE_HPOS instead of ENABLE_HPOS

* Log HPOS state

* Configure project for core e2e with HPOS disabled

* Add event argument for ci-jobs util

* Add event option for test jobs

* Add support for github events

* Add changelog

* Configure HPOS tests to run on pull_request

* Fix utils tests for undefined commandVars

* Added some tests for the event configuration

* Revert event for HPOS e2e to push

* Use matrix name in artifact name to avoid duplication

* Test with pull_request event

* Use job-index for unique artifacts names

* Revert event for HPOS e2e to push

* Add api tests for HPOS disabled

* Use unique artifact name for api tests

* Revert event for HPOS disabled api tests to push

* Rebuild monorepo utils to fix a merge conflict

* Updated wording
2024-05-01 10:36:24 -07:00
dependabot[bot] 3d8e20c62c
Bump octokit from 2.1.0 to 3.1.2 (#42891)
* Bump octokit from 2.1.0 to 3.1.2

Bumps [octokit](https://github.com/octokit/octokit.js) from 2.1.0 to 3.1.2.
- [Release notes](https://github.com/octokit/octokit.js/releases)
- [Commits](https://github.com/octokit/octokit.js/compare/v2.1.0...v3.1.2)

---
updated-dependencies:
- dependency-name: octokit
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Rename env variable

* Add changelog

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jon Lane <jon.lane@automattic.com>
2024-04-26 10:52:25 -03:00
Jonathan Lane 9888ee7f4b
Updated deprecated gh-script action to node 20 variant (#46673)
* Updated deprecated gh-script action to node 20 variant

* Minor change in e2e test to trigger CI

* Revert e2e test change

---------

Co-authored-by: Jon Lane <jon.lane@automattic.com>
2024-04-17 12:31:16 +01:00
Adrian Moldovan 1fee0439d3
[testing workflows] Move metrics job to ci.yml (#46471)
* Move deprecated workflows to deprecated folder

* Configure new test job for metrics

* Add changelog

* Update start command

* Update start command

* Archive metrics results

* Fix script path

* Fix name in archive results condition

* Add the actual run command for pull_request event

* Run with pnpm --filter

* Test run for push event

* Revert test conditions

* Add WP_ARTIFACTS_PATH env variable

* Update path to readme.txt

* Revert github event conditions

* Add CODEVITALS_PROJECT_TOKEN env

* Set WP_ARTIFACTS_PATH on job level

* Add blocks templates in the changes list
2024-04-12 11:46:01 -03:00
Adrian Moldovan 9df545046a
[test reporting] Include list of failed tests in Slack notifications (#46360) 2024-04-10 10:10:08 +03:00
Adrian Moldovan 4e9fc5a90f
[testing workflows] Tweak conditions for Slack alerts and reports publishing jobs (#46031) 2024-04-02 14:40:24 -03:00
Adrian Moldovan 7d6d2c94dd
Include e2e, api and performance tests in ci.yml (#45190)
Co-authored-by: Christopher Allford <6451942+ObliviousHarmony@users.noreply.github.com>
Co-authored-by: Ron Rennick <ron@ronandandrea.com>
2024-03-27 17:21:49 +02:00
jonathansadowski a7f9139b33
Fix post-merge CI / update ci-jobs util to make base-ref optional (#44055)
* Update ci-jobs util to make base-ref optional

* Fix linting error

* Update baseRef option to be easier to read in ci.yml
2024-01-26 13:52:55 -06:00
Christopher Allford 31c170f80c
Added Missing Lint Build (#43665)
Unfortunately, it looks like we need to build in order
to be able to lint. This makes sense since some
projects will lint imports from others that haven't
been built yet.
2024-01-15 12:30:54 -08:00
Christopher Allford 758df4854d
Use `ci-jobs` Utility For `ci.yml` Matrix (#43532)
This adds support for using the `pnpm utils ci-jobs` command in our `ci.yml` file. One of the bigger benefits to this change too is that we're now distributing a bundled version of the utils tool. This lets us run it without actually having to install the repo and will let us speed up any workflows that currently do.
2024-01-12 20:32:14 -08:00
Christopher Allford be290984b9
`setup-woocommerce-monorepo` Action in Highlight Changes Workflow (#42740) 2023-12-12 21:33:52 -08:00
Christopher Allford e5eff12f79 Fixed Wrong Equality Operator
Too used to strict equality I guess :)
2023-12-09 14:28:40 -08:00
Christopher Allford b0d896c3e3 Refactored Monorepo Setup Action
This makes some adjustments for consistency in style as well as making
improvements to usability and performance. It also updates all of the
pinned hashes to pull in any fixes that have been made.
2023-12-09 14:28:37 -08:00
Christopher Allford 95c33afa8a Cache Fix Attempt 2023-12-09 14:26:36 -08:00
Christopher Allford e40e63a75c Fixed Matrix Generation Failure Evaluation 2023-12-09 14:26:36 -08:00
Christopher Allford 783efb70b1 Cache Fix Attempt 2023-12-09 14:26:31 -08:00
Christopher Allford 8c6b414fe0
Revert "Refactoring Task Infrastructure" (#41928)
Revert "Refactored Task Infrastructure (#41202)"

This reverts commit 96a973b9fa.
2023-12-06 11:36:12 -08:00
Christopher Allford 96a973b9fa
Refactored Task Infrastructure (#41202)
* Fixed NPM packages with invalid names.
* Renamed plugin packages.
* Standardized package NPM scripts.
* Replaced Turborepo with Wireit.
2023-12-06 09:32:32 -08:00
Christopher Allford 88969742f2
Regenerate PNPM Lock File & Fix Errors (#41830) 2023-12-05 00:36:30 -08:00
Christopher Allford fbbca9c3e0
Refactored Linting & Unit Testing Workflows (#40572) 2023-10-19 11:24:20 -07:00
Sam Seay 6e93e8d35e
Add shard to the matrix name in PHP unit tests for clarity in GH UI (#39747) 2023-08-17 17:23:35 +12:00
Leif Singer 77af817444
Shard unit test runs (#39362) 2023-07-27 14:06:55 +02:00
Leif Singer 147db933be
Revert "Shard unit test runs" (#39344)
Revert "Shard unit test runs (#39302)"

This reverts commit 9ee44947e0.
2023-07-20 18:36:01 +02:00
Leif Singer 9ee44947e0
Shard unit test runs (#39302)
closes #38973
2023-07-20 10:01:28 +02:00
Christopher Allford dedf728d51
Support PHPUnit Tests In `wp-env` (#38265)
You can now run unit tests with `test:unit:env` after using
`env:test` to start the environment.
2023-05-15 09:47:50 -07:00
Paul Sealock 64c2526b98
Update WooCommerce support policy to L-1 (#37970) 2023-04-27 08:41:31 +12:00
Chi-Hsuan Huang 8b99b3f099
Drop WP 5.9 from CI (#37807) 2023-04-20 09:47:30 +08:00
Tam Mullen 562fede1e2
Update the CI workflow to include WP 6.0 and 5.9 (#37180)
Update the CI workflow to include WP 6.0 and 5.9 matching our L-2 support policy
2023-03-15 15:49:33 +00:00
Roy Ho 16386238cc
Add minimum required permissions for workflow token (#36200)
* Add minimum required permissions to e2e-tests

* Add minimum token permission to more workflows
2023-01-02 09:28:11 -08:00
Roy Ho eef006dad1
Remove PHP 7.2 and set 7.4 as default (#35021) 2022-10-11 12:55:36 -07:00
Sam Seay 04b74c1053
Upgrade to pnpm 7 (#34661)
* Remove -- -- from scripts that fixed issue in pnpm 6, fix a couple TS issues in sep packages
* Minor fixes to analyzer scripts and doco based on pnpm 7
* Add dompurify types to data package to avoid TS errors
* Remove pnpx in favor of pnpm exec
* Modify the code analyzer to respect pnpm version if present.
* Update instructions for running recursive lint, add comments to explain
2022-09-16 09:21:42 +12:00
Christopher Allford a49f23abfd
Optimize Workflow Caching (#34607)
* Added Monorepo Setup Action

This action will handle the installation, building, and caching for all
projects within the monorepo. It has inputs for skipping builds and
filtering so that only specific packages are installed and built.

* Removed Test Caching

Caching the results of tests, while an interesting way to avoid
unnecessary execution, seems error-prone. We can't adequately
capture the environment such that we can rely on this working.
For instance, changes in PHP version might break PHPUnit tests,
but, cached runs will be shared across different workflows.

It seems better to just not cache it and rely on people only testing
code that has actually changed in the pull request.

* Added Input Parsing

This SHOULD allow for passing multiple filters as an array argument.

* Updated Workflows

This fully utilizes the new action across all of the workflows.

* Always Cache Build Output

Since there may be other commands run that populate the cache, we
should make sure that we are always caching just in case.
2022-09-13 13:55:03 +12:00
Roy Ho fc06be8296
Add flag to state it is in dev mode so tools are installed like phpun… (#34258)
Add flag to state it is in dev mode so tools are installed like phpunit closes #34158
2022-08-11 12:58:36 -07:00
Sam Seay 5becf47d60
Simplify turbo commands (#34055)
* Change package json commands and turbo repo config to allow for greatly simplifying running turbo
* Remove require-turbo script which was causing caching issues
* Colorize CI output from scripts
* Add missing changelog script to extend-cart-checkout-block
2022-08-05 12:03:17 +12:00
Christopher Allford 6d0fc0d6ac
Expand Repository Development Documentation (#33423)
This commit reworks some of the documentation in the monorepo to better represent how to get started and develop projects in it.
2022-06-15 21:55:25 -07:00
Christopher Allford b7931409f2
Migrate From Nx to Turborepo (#33079)
This replaces all `nx` commands with `turbo` commands and removes Nx from the repository. All of the `project.json` files have been removed and any commands that broke with Turborepo have been adjusted.
2022-06-09 14:40:07 -07:00
Roy Ho b0914e5531
Increase timeout limits for workflows (#33272) 2022-05-31 07:58:54 -07:00
Jonathan Sadowski af575532b5 Update the CI workflow to include WP 5.9 and 5.8 matching our L-2 support policy 2022-05-25 16:00:44 -05:00
roykho 1042f22cf8
Extract cache and build to rest of workflows 2022-05-17 11:48:24 -07:00
roykho 7056cf93b6
Add additional pnpm cache dir 2022-05-16 06:01:53 -07:00
roykho cf5c47eb09
Cache Composer global store 2022-05-13 13:43:33 -07:00
roykho ab58263e80
Add unique secrets cache key to enable clearing of cache 2022-05-04 15:20:31 -07:00
roykho 45a036f35b
Match minor and patch 2022-05-02 10:08:52 -07:00
roykho f2e4a686f1
Lock pnpm to specific version 2022-05-02 09:39:40 -07:00
roykho 57c1c0bc1c
Freeze Ubuntu release to 20.04 for best practice and consistency 2022-04-26 18:28:56 -07:00
roykho fc705cff77
Use glob pattern for hash key 2022-04-26 07:47:50 -07:00
roykho bd096a6edc
Use glob pattern to cache multiple nested Composer vendor paths 2022-04-26 07:07:15 -07:00
roykho 6e5d61da0b
Test pr smoke test workflow 2022-04-21 11:29:53 -07:00
Paul Sealock ab673dd14b Call Core's script from root level 2022-04-06 15:27:37 +12:00
Nestor Soriano 25bf59d5e7
Merge branch 'trunk' into bump-required-php-to-7.2 2022-03-29 10:45:07 +02:00
Christopher Allford 14e852b4b2
Merge WooCommerce Admin Unit Tests (#32287)
This folds the unit tests from WooCommerce Admin into the core test suite, enabling them to run.
2022-03-28 12:53:33 -07:00
Nestor Soriano 819494f2ed
Bump WooCommerce minimum required PHP version to 7.2
This includes:

- Changing "Requires PHP" in woocommerce.php
- Changing "require-php" and "config-platform-php" in composer.json
- Bumping pelago/emogrifier to ^6.0 in composer.json
- Bumping maxmind-db/reader to ^1.6 in composer.json
- Bumping composer/installers to ^1.9 in composer.json
  (bump to 2.0 not possible due to conflict with Admin package)
- Removing jobs for 7.0/.1 unit test runs in CI
- Explicitly specifying the PHPUnit version to be 7.5,
  otherwise it will install v8 which isn't compatible with
  WordPress <5.9
2022-03-24 14:08:11 +01:00
Chi-Hsuan Huang a720ccb42a Add "Build Admin feature config" step to ci.yml 2022-03-24 14:00:43 +08:00
Felipe Santos a8679b041d
Update the WP version for tests and WooCommerce 2022-03-03 11:38:27 +00:00
roykho f15f75e9cf
No need to install Nx globally 2021-11-17 08:54:14 -08:00
roykho f940d049d6
Use Nx commands for CI 2021-11-16 18:17:35 -08:00
roykho 27127ea1fc Fix directory path for CI workflow 2021-10-19 10:35:47 +13:00
Nestor Soriano 0ae984e6cc
Fix unit tests failing in PHP 8 in CI
The problem was that PHP 8 requires setting up a different version of
PHPUnit, and after doing that two versions were in place, one in
'vendor/phpunit' and another one in 'bin/composer/phpunit'; the
autoloader was picking the former but it should pick the later.
2021-08-24 15:36:06 +02:00
Claudio Sanches 37068f198f Improve continue on error 2021-08-09 13:55:53 -03:00
Claudio Sanches 175cc34ee7 Allow failures for WP nightly unit tests 2021-08-09 13:41:53 -03:00
Claudio Sanches 9a9a855701 Update conditional 2021-08-09 13:22:01 -03:00
Claudio Sanches ef9f511387 Run composer update for wp nightly 2021-08-09 13:00:29 -03:00
Claudio Sanches 4fea3f4dc1 Update unit test WP matrix 2021-08-09 12:33:39 -03:00
Claudio Sanches 87a18620eb Added workflow to run CI in trunk and release/** branches 2021-03-13 21:29:15 -03:00