Skip to content

[Incomplete - for discussion] Add bridge-cache plugin to populate cache without running action #5190

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

benkeen
Copy link

@benkeen benkeen commented Apr 9, 2025

*** This is incomplete. For discussion only. Will add proper doc later.

The goal is to add a plugin which allows users to run a Rush command, but only populate the cache from a previous run of that action, not actually run the command. In other words, it relies on a build having already taken place.

For instance:

rush build --set-cache-only
rush test --only packageX --set-cache-only
rush lint --to packageY --set-cache-only

Each of these would populate the cache for one or more package actions from whatever was on disk. The command itself (build, test, lint) won't get run.

Why?

We're using a different build orchestrator to handle some high-volume builds (BuildXL), primarily because it's able to farm out work across agents, not just processes. As a build orchestrator, it does exactly the same work as what Rush would have done, but has a totally different cache to Rush. We rely on our current (Rush) pipeline populating the remote cache so our local builds are snappy. Without the feature in this PR, we'd need a separate pipeline to populate the Rush cache - which is very wasteful + expensive.

Comment on lines 76 to 78
if (!targetProjects.includes(associatedProject.packageName)) {
return;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend creating a Set<Operation> that contains all operations where operation.enabled was true prior to cancel and just check for membership in that set here.

@benkeen benkeen force-pushed the benkeen/cache-bridge-plugin branch from a457102 to c3fdcc9 Compare April 10, 2025 16:38
@iclanton iclanton moved this from Needs triage to In Progress in Bug Triage Apr 30, 2025
@@ -81,6 +81,7 @@ export class PluginManager {
tryAddBuiltInPlugin('rush-amazon-s3-build-cache-plugin');
tryAddBuiltInPlugin('rush-azure-storage-build-cache-plugin');
tryAddBuiltInPlugin('rush-http-build-cache-plugin');
tryAddBuiltInPlugin('rush-bridge-cache-plugin');
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This plugin shouldn't be installed by default, so make sure this gets removed before checkin.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

3 participants