Commit Graph

912 Commits

Author SHA1 Message Date
jonathansadowski b8c2cf8cde
Update ci.yml to be dispatchable and skippable (#45331)
Co-authored-by: Adrian Moldovan <3854374+adimoldovan@users.noreply.github.com>
2024-07-09 16:50:57 +03:00
Vladimir Reznichenko 68532ce433
CI: update linting jobs to skip build step (#49193)
In this PR, we introduce changes to linting and PHPUnit jobs in order to speed them up: skip or minimize the build step.
2024-07-08 08:39:39 +02:00
Adrian Moldovan cef644078e
Remove the matrix.testEnv condition from the build step (#49149) 2024-07-04 14:50:49 +03:00
Adrian Moldovan 32b7dd646a
Updated ignored jobs list for evaluation script (#49141) 2024-07-04 14:05:15 +03:00
Adrian Moldovan 62c74d0ed4
[e2e tests] Add support for external sites in CI (#49017) 2024-07-04 12:15:43 +03:00
Vladimir Reznichenko 7959060850
Monorepo: disable autoloading optimization by default (take 2) (#48980)
This PR is the second attempt to speed up monorepo installation by removing autoloading optimization setting from the configuration and performing the optimization where it is appropriate (composer post-install hooks and when building releases).

In the installation context, the composer post-install hook will execute a background task to dump the optimized autoloading, while the installation process completes other tasks.
2024-07-04 10:12:52 +02:00
Vladimir Reznichenko f8c95a8db8
CI: caching tune (part one - relocate playwright caching) (#49081)
In this PR we ar relocating playwright downloads caching into the monorepo action (behind input parameter). That consolidates monorepo-related caching and also reduces the time fetching the cache action in external workflows.
2024-07-04 06:55:24 +02:00
Adrian Moldovan f50119bd1b
Update action/pnpm-setup version (#49116) 2024-07-03 15:59:15 -03:00
Adrian Moldovan 36c28b6ad2
Move buildkite-test-collector to devDependencies (#49051) 2024-07-03 11:31:40 +03:00
Adrian Moldovan 94b93d40ff
[testing workflows] Removed deprecated reporting CI steps (#49025)
* Removed deprecated reporting steps

* Test change

* Revert test change
2024-07-02 15:40:05 -03:00
Vladimir Reznichenko f1b974a910
Revert "Monorepo: don't optimize composer autoloading by default. (#48971)" (#48982)
This reverts commit 76b1bd4ae7.
2024-07-01 15:13:25 +02:00
Vladimir Reznichenko 76b1bd4ae7
Monorepo: don't optimize composer autoloading by default. (#48971)
This PR drops composer autoloader optimization for the WooCommerce plugin to speed up the build process. The autoloader still gets optimized in `plugins/woocommerce/bin/build-zip.sh` file.
2024-07-01 13:55:36 +02:00
Vladimir Reznichenko 6b121e2f34
CI: caching tweaks in GitHub actions (#48865)
* Tooling: cache pnp deps

* Tooling: try fixing cache errors

* Tooling: try to parallelize js builds a bit

* Tooling: try to parallelize js builds a bit

* Tooling: try to parallelize js builds a bit

* Tooling: try to parallelize js builds a bit

* Tooling: try to parallelize js builds a bit

* Tooling: try to parallelize js builds a bit

* Tooling: try to parallelize js builds a bit

* Tooling: try to parallelize js builds a bit

* Tooling: try to parallelize js builds a bit

* Tooling: added pre-build steps to solve concurrency issues

* Tooling: limit workers

* Tooling: revert to original build command

* Tooling: playwright related tweaks

* Tooling: minor tweaks.

* Tooling: fix caching path.

* Tooling: trigger php builds to test deps caching.

* Tooling: trigger php builds to test deps caching.

* Tooling: lock caching to monorepo setup, so it doesn't slow down build steps.

* Tooling: lock Playwright cache to build, as build is pre-requisite for running tests.

* Tooling: added caching build-folders.

* Tooling: added caching build-folders.

* Tooling: added caching build-folders.

* Revert "Tooling: added caching build-folders."

This reverts commit c57b9dff13.

* Tooling: added caching build-folders.

* Tooling: cleanup sweep.

* Tooling: cleanup sweep.

* Tooling: added a changelog entry.

* Tooling: cleanup.

* Tooling: try speedup live branches build.

* Tooling: try speedup live branches build.

* Tooling: cleanup.

* Tooling: try speedup live branches build.

* Tooling: try speedup live branches build.
2024-06-28 14:50:32 +01:00
Adrian Moldovan b098a393b8
[test reports] Optimize the reporting job - don't duplicate reports and remove Allure report (#48612)
* * Use run_attempt in the artifact name

* Remove the Allure report steps

* Test change to trigger tests

* Only run the publish report step if there is an artifact-id

* Only run the publish report step if there is an artifact-id

* Revert test changes

* Also condition the deprecated publish report step
2024-06-19 09:37:50 -07:00
Adrian Moldovan f2fd4968f8
[ci] Always run the resolve artifacts path (#48508) 2024-06-17 10:16:04 -07:00
Adrian Moldovan 3fb5d40a43
[test reports] Send the run attempt to the reports workflow (#48519) 2024-06-17 10:14:32 -07:00
Adrian Moldovan 90398903ab
Fix core api tests results path and artifacts naming pattern (#48490)
* Fix api tests results path

* Add changelog

* Run Core API tests - HPOS disabled on pull_request

* Update the artifacts name pattern to avoid conflicts

* Revert pull_request trigger
2024-06-14 09:03:43 -07:00
Adrian Moldovan 1f534d0280
[e2e tests] Update the report configuration for core e2e tests and add a new workflow call for report publishing (#48424)
* Add a call to a new report workflow in woocommerce-test-reports repo

* Test change to trigger tests

* Update report title

* Test change to trigger tests

* Check REPORT_TITLE

* Add quotes on all arguments

* Update reporting configuration to all core e2e jobs

* Add changelog

* Use GITHUB_HEAD_REF for pull_request

* Set REPORT_TITLE for workflow dispatch

* Updated trigger from daily-e2e to daily-checks
2024-06-12 13:38:37 -07:00
Adrian Moldovan 1fe724b6ef
Remove deprecated workflows, actions and related scripts (#48364) 2024-06-12 10:21:47 +03:00
Jonathan Lane e5e51a41b8
Update Playwright to 1.44.1 from 1.41.1 (#48291)
Co-authored-by: Jon Lane <jon.lane@automattic.com>
Co-authored-by: Bart Kalisz <bartlomiej.kalisz@gmail.com>
Co-authored-by: Adrian Moldovan <3854374+adimoldovan@users.noreply.github.com>
2024-06-11 21:52:48 +03:00
Adrian Moldovan d63da14787
Skip the install and build steps for changelog auto add workflow (#48368)
Co-authored-by: github-actions <github-actions@github.com>
2024-06-11 20:02:56 +03:00
Adrian Moldovan f3a7c0a6ec
[ci] Skip build for lint jobs (#48234) 2024-06-11 20:01:43 +03:00
Adrian Moldovan a7f634d5ef
[blocks e2e] Move Blocks e2e into ci.yml (#48224) 2024-06-11 19:57:23 +03:00
Ron Rennick 5650f88514
add community check to live branch workflow (#48190)
Co-authored-by: Ron Rennick <ronald.rennick@automattic.com>
2024-06-11 13:48:00 -03:00
Adrian Moldovan 1533afdca3
[e2e tests] Fix e2e test results path and report name (#48320)
* Fix the report name sent to woocommerce-test-reports workflow

* Add test type

* Test change

* Remove test type

* Merge artifacts to keep allure-results

* Overwrite artifacts if one already exists with the same name

* Test changes to trigger tests

* Fix allure-results and .state paths

* Filter tests for faster feedback

* Debug artifacts content

* Use name instead of pattern

* Use name instead of pattern

* Don't use an intermediate path

* Use artifact name and path instead of only pattern

* Use artifact name and path instead of only pattern

* Use artifact name and path instead of only pattern

* Remove testMatch filter to run all tests

* Revert test change

* Add changelog
2024-06-10 11:21:46 -07:00
Adrian Moldovan cd0884a294
[ci] Fetch all pages when evaluating jobs conclusions (#48315)
Use pagination to get all workflow jobs when evaluating the jobs conclusions
2024-06-10 11:17:54 -07:00
Adrian Moldovan f0478c3f60
[ci] Evaluate jobs: update the ignore list with regex expressions (#48268) 2024-06-07 09:34:55 -07:00
Adrian Moldovan 8fc6970b90
[ci] Merge test jobs into a single generic job (#48175) 2024-06-06 21:50:52 +03:00
Adrian Moldovan 26158ee6b4
[testing workflows] Replace push to tags trigger with release published and updated (#48209)
Update push to tag triggers with release trigger
2024-06-06 09:51:02 -07:00
Manish Menaria a746332e01
WooCommerce Blocks: Change label for E2E flaky tests (#48223)
Change label for flaky tests
2024-06-06 20:51:23 +05:30
Adrian Moldovan 7fb40ffc0b
[testing workflows] Migrate release checks to ci.yml (#48113)
* Deprecate pr-smoke-test.yml

* Add configs for each environment

* Add scripts

* Add arguments to playwright command

* Update pnpm scripts and existing e2e projects

* Fix shellcheck warning

* Update test env start for Gutenberg stable run

* Add changelog

* Remove unused var

* Set the USE_WP_ENV var

* Configure daily e2e jobs

* Rename some steps

* Add release checks jobs and deprecate  smoke-test-release.yml

* Deprecate  smoke-test-release.yml

* Add tests-release-checks.yml to trigger ci.yml on push to tags

* Add the pull_request trigger to test the workflow

* Remove the pull_request trigger to test the workflow

* Add the pull_request trigger to test the workflow

* Add the tag input in ci.yml

* Remove import and use core reference

* Only build if build-plugin is true

* Use release-artifact instead

* Split setup-monorepo step

* Use tag: 'nightly' to test

* Use inputs.tag directly in the wp-env setup step

* Add script to override the wp-env config

* Run the env config script in ci

* Set the artifact name

* Fix workflow syntax

* Add changelog

* Validate RELEASE_TAG and ARTIFACT_NAME

* Use real ref type and name

* Use real ref type and name

* Use real ref type and name

* Use github context instead

* Test with 9.0.0-beta.2

* Add push to tags trigger directly in ci.yml

* Configure one job to run with nightly-checks

* Remove empty lines

* Format events array one element per line

* Format events array one element per line
2024-06-05 13:02:54 -03:00
Manish Menaria 307d893549
WooCommerce Blocks: Added a GitHub Action to create issues for flaky E2E tests (#47758)
* Add flaky test reporting and issue creation for Blocks

This commit introduces the functionality for identifying and reporting flaky tests within our CI workflow.
- The reporter captures flaky test results and saves them to `flaky-tests` directory.
- If a test fails initially but passes upon retries, it is marked as flaky and a corresponding report is generated.
- The workflow creates GitHub issues for flaky tests, aiding in better visibility and tracking of these intermittent issues.

1. **Workflow Changes**:
   - **Archive flaky test reports**: Adds a step to archive flaky test reports generated during the test runs.
   - **Merge flaky test reports**: Introduces a new step to merge all flaky test reports from different shards.
   - **Create GitHub issues for flaky tests**: Implements a job that uses the `@wordpress/report-flaky-tests` package to create GitHub issues for detected flaky tests when a PR is not in draft state and flaky test reports are present.

2. **New Flaky Test Reporter**:
   - Adds `flaky-tests-reporter.ts` that defines a custom Playwright reporter for identifying and recording flaky tests.

3. **Playwright Configuration**:
   - Updates `playwright.config.ts` to include the new flaky tests reporter in the list of reporters when running in CI.

4. **Demo Tests**:
   - Adds `demo.spec.ts` containing sample tests to validate the flaky test reporting functionality.

Implementing this feature will help us track and address flaky tests more efficiently, ensuring more stable and reliable test suites.

* Remove draft condition for testing

* 'Merge failures artifacts' job shouldn't touch flaky test artifacts

* Try fixing if condition

* Add required permissions

* Let's try providing all the permissions

* Let's try providing all the permissions

* Create flaky tests issues when PR is merged with trunk

* Update if condition

* Run action for specific paths only

* Add changefile(s) from automation for the following project(s): woocommerce-blocks

* Remove Demo file

* Remove Demo file

* Use commit hash to use exact version of the script

* Replace label with team: Kirigami & Origami

---------

Co-authored-by: github-actions <github-actions@github.com>
2024-06-05 19:49:28 +05:30
Sam Seay 2d4295469d
Remove all usage of WOOCOMMERCE_BLOCKS_PHASE and introduce BUNDLE_EXPERIMENTAL_BLOCKS (#47807) 2024-06-05 20:14:16 +12:00
Jonathan Lane e8e1391fa0
Add annotation for test PR e2e test run results (#48029)
* Add annotation for test results

* Minor change to trigger e2e tests

* Fix command, add emoji

* Update emoji

* Add changelog

* Update from PR feedback

* Revert "Add changelog"

This reverts commit 03656848eb.

* Revert "Minor change to trigger e2e tests"

This reverts commit 64e3a87b2d.

---------

Co-authored-by: Jon Lane <jon.lane@automattic.com>
2024-06-04 13:03:38 -07:00
Adrian Moldovan 9b251ce318
[testing workflows] Add test environments and deprecate the daily e2e workflow (#48101)
* Deprecate pr-smoke-test.yml

* Add configs for each environment

* Add scripts

* Add arguments to playwright command

* Update pnpm scripts and existing e2e projects

* Fix shellcheck warning

* Update test env start for Gutenberg stable run

* Add changelog

* Remove unused var

* Set the USE_WP_ENV var

* Configure daily e2e jobs

* Rename some steps
2024-06-03 14:12:45 -07:00
Sam Seay b04c8bb9ec
Add path ignore to assess bundle action to reduce how often it runs (#48058) 2024-06-01 00:46:44 +12:00
Sam Seay de5d73061d
Introduce asset size check action to CI (#47880) 2024-05-31 05:08:28 +12:00
jonathansadowski f56d3e3ee0
Update PR template and auto change log with better no-entry support (#44023)
* Update PR template and auto change log

* Ignore .github md files in linting

* Rebuilt monorepo-utils to solve conflict

* Updated regex pattern to include uppercase X

* Move the changelog entry not required checkbox on top of the description.

* Return the comment even if shouldAutomateNoChangelog is false

* Updated tests

* Rebuilt monorepo-utils

---------

Co-authored-by: Adrian Moldovan <3854374+adimoldovan@users.noreply.github.com>
2024-05-29 17:06:59 -03:00
Sam Seay 91f91b5d82
Use github proxy to install WC beta tester in playground previews (#47837)
Use github proxy to install beta tester in playground links.
2024-05-27 14:19:56 -03:00
Adrian Moldovan 97e7f86a15
Add the slack-test-report util (#47805)
* Add the slack-test-report util

* Error if the required GitHub context variables are not set

* Error if the required SLACK_CHANNEL env var is not set

* Use a more generic message to be able to report on jobs that are not tests, like build

* Add the alert-on-failure job in ci

* Run for pull_request to test the job

* Setup monorepo first

* Force a linting error

* Better logging

* Use inputs.trigger as report name if set

* Force an e2e test failure

* Set the commit message in the notification

* Use INPUT_TRIGGER for all notifications

* Revert test changes
2024-05-24 13:18:54 -07:00
Adrian Moldovan c97ef43812
[e2e tests] Only run the default project in daily e2e runs (#47771)
Only run default project in daily workflow
2024-05-23 09:38:20 -07:00
Adrian Moldovan 835625c329
Fix reporting step for daily e2e runs (#47764)
* Add condition on github.event_name for lint jobs

* Update a file to trigger a lint job

* Update condition to run only on pull_request

* Revert test change

* Always run the Slack notification step

* Add pull_request test trigger

* Update report step

* Remove branch filter

* Try with push event and branch filter

* Pass secrets to ci.yml

* Update testIgnore pattern for Gutenberg project

* Add changelog

* Remove the test push trigger

* One more test run

* Removed push test trigger
2024-05-23 09:12:10 -07:00
Adrian Moldovan f1489892aa
[ci] Only run lint jobs on pull_request event (#47652) 2024-05-23 07:16:07 -07:00
Bart Kalisz 3b328156a0
Blocks E2E: Cancel running jobs on new commits (#47753) 2024-05-23 11:51:06 +02:00
Adrian Moldovan 13ab9007f6
[testing workflows] Add workflow_call trigger in CI workflow - support daily and other types of runs (#47612)
* Add workflow_call event

* Add tests-daily-run.yml

* Remove workflow version

* Update input value

* Remove secrets from caller workflow

* Rename job

* Check context

* Check context

* Check workflow_call inputs

* Override github.event_name with inputs.trigger

* Fix input name

* Add config for daily e2e tests with Gutenberg

* Update env commands for Gutenberg run

* Add disableHpos env variable in ci-utils

* Revert the trigger to push

* Revert the env variable setting in ci-jobs

* Revert the env variable setting in ci-jobs

* Rebuild utils

* Update test env start command

* Define the Gutenberg Playwright project

* Use path relative to workspace root

* Update events for test projects

* Update events for test projects

* Add changelogs

* Remove the e2e tests with Gutenberg tests from daily runs

* Set the schedule and workflow_dispatch triggers

* Add --project default flag to only run the default tests in CI

* Renamed snapshots to match the new Playwright project name
2024-05-22 11:04:44 -07:00
Adrian Moldovan 0522511a72
[testing worklflows] Consolidate the Slack channel used for test reports (#47611) 2024-05-22 11:01:51 -03:00
Moon f57de64560
Fix broken WooCommerce Beta Tester release action (#47593)
Install and build the core -- removed woocomemrce beta install since it is part of build:zip command
2024-05-21 15:28:00 -07:00
Adrian Moldovan cf6e01d0d6
[ci-jobs] Add disableHpos test environment variable (#47619)
* Add disableHpos test env variable

* Update events for HPOS disabled tests

* Add pull_request event to test the env

* Remove the pull_request event

* Add changelog

* Make the disableHpos var a boolean

* Also set the env in ci.yml

* Remove console log statement

* Print woocommerce_custom_orders_table_enabled

* Run test setup script again to check output

* Capture env setup output and print it

* Get woocommerce_custom_orders_table_enabled value in test env script

* Get more details on HPOS setting

* Remove the DISABLE_HPOS logic from test env setup script

* Use DISABLE_HPOS instead of ENABLE_HPOS everywhere

* Turn off no-console and jest/no-standalone-expect eslint rules for api core tests

* Test API tests with HPOS disabled

* Fix eslint error

* Clean up debug statements

* Remove pull_request test trigger
2024-05-21 18:33:43 -03:00
Adrian Moldovan ccba59ce05
[ci] Only evaluate jobs statuses for pull request event (#47615)
* Condition evaluate status job by github event

* Test reverse condition

* Test with push event

* Revert condition to match pull_request
2024-05-21 18:28:24 -03:00
Adrian Moldovan 3edf17bf22
[testing workflows] Move PHP tests with WP nighly back into ci.yml (#47568)
* Move the PHP WP: nightly tests back into ci.yml

* Add changelog
2024-05-21 18:23:38 -03:00