NIFI-15696 - Add Rebase operation for versioned process groups by pvillard31 · Pull Request #10994 · apache/nifi

pvillard31 · 2026-03-11T12:59:22Z

Summary

NIFI-15696 - Add Rebase operation for versioned process groups

When a versioned process group is at version N with local modifications (state LOCALLY_MODIFIED_AND_STALE), upgrading to a newer version requires reverting all local changes first, then changing version, then manually re-applying every customization. This is error-prone, causes unnecessary downtime, and discourages users from upgrading.

This change adds a "Rebase" operation that upgrades a versioned process group to a newer registry version while automatically preserving compatible local changes — analogous to git rebase.

How it works:

The rebase operation has two phases:

Analysis (read-only): Computes a three-way diff between the local flow, the base registry version (N), and the target registry version. Each local change is classified as Compatible (will be preserved), Conflicting (same field changed both locally and upstream to different values), or Unsupported (change type not yet supported by a rebase handler). Rebase is only allowed when all local changes are compatible.
Execution: Deep-clones the target version snapshot, overlays compatible local changes onto it to produce a merged snapshot, then synchronizes the process group to the merged snapshot using the existing FlowUpdateResource async infrastructure. The VCI is set to the clean target version so that subsequent state checks correctly report LOCALLY_MODIFIED.

Supported change types (for this first iteration, this will evolve over time):

POSITION_CHANGED, SIZE_CHANGED, BENDPOINTS_CHANGED — cosmetic, always compatible (local wins)
PROPERTY_CHANGED, PROPERTY_ADDED — compatible unless upstream changed the same property on the same component to a different value (convergent changes where both sides set the same value are accepted)
COMMENTS_CHANGED — compatible unless upstream also changed comments on the same component
Unsupported change types (examples: COMPONENT_ADDED, COMPONENT_REMOVED) block the rebase with a clear message. The handler architecture is extensible — adding support for a new change type requires only implementing a single RebaseHandler class.

We follow a similar approach as what already exists for the new API endpoints:

GET /versions/rebase-analysis/process-groups/{id}?targetVersion={version}
returns the three-way analysis
POST /versions/rebase-requests/process-groups/{id}
initiates async rebase execution
GET/DELETE /versions/rebase-requests/{id}
polling and cleanup (reuses FlowUpdateResource)

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Apache NiFi Jira issue created

Pull Request Tracking

Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000
Pull request contains commits signed with a registered key indicating Verified status

Pull Request Formatting

Pull Request based on current revision of the main branch
Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

Build completed using ./mvnw clean install -P contrib-check
- JDK 21
- JDK 25

Licensing

New dependencies are compatible with the Apache License 2.0 according to the License Policy
New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

Documentation formatting appears as expected in rendered files

github-actions · 2026-07-10T00:42:09Z

Automated review is marking this PR as stale due to lack of updates in the past four months. This PR will be closed in 15 days if the stale label is not removed. This stale label and automated closure does not indicate a judgement of the PR, just lack of reviewer bandwidth and helps us keep the PR queue more manageable. If you would like this PR re-opened you can do so and a committer can remove the stale label. Or you can open a new PR. Try to help review other PRs to increase PR review bandwidth which in turn helps yours.

exceptionfactory

Thanks for putting together this new feature @pvillard31. The general implementation pattern looks good, and the Handler strategy for various types of changes appears to provide a good foundation for iteration.

I noted a handful of syntactic recommendations, and raised a couple questions about concurrency.

exceptionfactory · 2026-07-20T20:45:22Z

+            description = "For a Process Group that is under Version Control, this will perform a rebase analysis by comparing "
+                    + "local modifications against upstream changes between the current version and the specified target version. "
+                    + "The analysis determines whether a rebase is allowed or if there are conflicts.",


Multiline strings can be used instead of concatenation

exceptionfactory · 2026-07-20T20:46:20Z

+                    @SecurityRequirement(name = "Write - /process-groups/{uuid}"),
+                    @SecurityRequirement(name = "Read - /{component-type}/{uuid} - For all encapsulated components"),
+                    @SecurityRequirement(name = "Write - /{component-type}/{uuid} - For all encapsulated components"),
+                    @SecurityRequirement(name = "Write - if the template contains any restricted components - /restricted-components"),


This reference to restricted components is no longer applicable and should be removed

exceptionfactory · 2026-07-20T20:49:10Z

+            return new ClassifiedDifference(difference, RebaseClassification.COMPATIBLE, null, null);
+        }
+
+        public static ClassifiedDifference conflicting(final FlowDifference difference, final String conflictCode, final String conflictDetail) {


Should the conflictCode be declared as an enum for reuse across the implementation?

exceptionfactory · 2026-07-20T20:49:55Z

+import java.util.SortedSet;
+import java.util.TreeSet;
+
+public class RebaseEngine {


I recommend defining an interface and making this the standard implementation to clarify the public contract for engine operations

exceptionfactory · 2026-07-20T20:50:48Z

+            case PROPERTY_CHANGED -> type.name() + ":" + componentId + ":" + fieldName.orElse("");
+            case POSITION_CHANGED, SIZE_CHANGED, COMMENTS_CHANGED, DESCRIPTION_CHANGED -> type.name() + ":" + componentId;
+            case BENDPOINTS_CHANGED -> type.name() + ":" + componentId;
+            default -> type.name() + ":" + componentId + ":" + fieldName.orElse("");


The concatenation here is a bit hard to read, recommend defining one or more format strings and reusing

exceptionfactory · 2026-07-20T20:54:01Z

+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+
+public class TestPropertyChangedRebaseHandler {


Since these are net-new classes, recommend using Test as the suffix instead of the prefix for the class. The public modifiers can also be removed

exceptionfactory · 2026-07-20T20:54:27Z

+
+    @Test
+    public void testNoUpstreamConflict() {
+        final VersionedProcessor processor = createProcessorWithProperty("proc-a", "propX", "oldVal");


There are a number of repeated string values in this test that should be moved to static final variables

exceptionfactory · 2026-07-20T20:55:02Z

        </dependency>
+        <dependency>
+            <groupId>com.fasterxml.jackson.core</groupId>
+            <artifactId>jackson-databind</artifactId>


Is this dependency needed if the ObjectMapper deep copy is removed?

exceptionfactory · 2026-07-20T20:56:53Z

    private static final int VALIDATION_WAIT_MILLIS = 50;
    private static final String ROOT_PROCESS_GROUP = "RootProcessGroup";

+    private final Map<String, VersionedProcessGroup> rebaseCleanSnapshots = new ConcurrentHashMap<>();


Are there any concerns about concurrent rebase requests for the same Process Group?

exceptionfactory · 2026-07-20T20:59:31Z

+        final ProcessGroupEntity result = serviceFacade.updateProcessGroupContents(revision, groupId, versionControlInfo, flowSnapshot, idGenerationSeed,
                verifyNotModified, false, updateDescendantVersionedFlows);
+
+        serviceFacade.resetVersionControlSnapshotAfterRebase(groupId);


It seems like a failure to update the Process Group contents would fail to reset the snapshot, leaving the snapshot entry in the Map within the Standard Service Facade. Perhaps putting this in a finally block would be sufficient?

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

…rebase Review feedback (exceptionfactory): - Introduce RebaseEngine interface with StandardRebaseEngine implementation - Replace String conflict codes with RebaseConflictCode enum - Remove Jackson round-trip deep clone; engine now merges onto a target snapshot supplied by the caller, and the facade fetches a separate clean target copy for the VCI reset (drops jackson-databind dependency) - Guarantee the retained clean rebase snapshot is always released via a finally block; clearRebaseSnapshot separates cleanup from VCI reset - Use text blocks for the rebase endpoint operation descriptions - Reuse format-string constants for conflict keys and analysis fingerprint - Remove the non-applicable restricted-components security requirement - Fail fast when a component identifier cannot be resolved - Rename new test classes to the Test suffix, drop public modifiers, and extract repeated test string literals into constants Rebase adaptation: - Use VersionedComponentFlowMapper and the current StandardFlowComparator constructor; drop the removed ProcessGroup.getAncestorServiceIds usage Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

- Add a read-only RebaseEngine.classify() path so analysis does not mutate the target snapshot while execution still builds the merged snapshot - Reject local changes when the target version removed the affected component, and treat upstream property removal on the same component/property as a property conflict - Remove the request-node-only rebaseCleanSnapshots map and reset the Version Control Information to the clean target on every node using a dedicated rebase replication endpoint - Add system coverage for removed components, preserved local modifications, and the full rebase suite on a two-node cluster - Restore update request polling annotations and add defensive handler guards Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

pvillard31 · 2026-07-21T12:01:56Z

Thanks for the review @exceptionfactory - it had been a while since I submitted the PR so I addressed your feedback + the rebase on a commit and I made some additional changes in another commit on top.

pvillard31 added the enhancement label Mar 11, 2026

pvillard31 force-pushed the NIFI-15696 branch from a4b1a1c to e5341cb Compare March 11, 2026 17:58

github-actions Bot added the Stale label Jul 10, 2026

pvillard31 removed the Stale label Jul 10, 2026

exceptionfactory requested changes Jul 20, 2026

View reviewed changes

NIFI-15696 - Add Rebase operation for versioned process groups

d76792b

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

pvillard31 force-pushed the NIFI-15696 branch from e5341cb to d76792b Compare July 21, 2026 06:37

pvillard31 added 2 commits July 21, 2026 09:09

pvillard31 requested a review from exceptionfactory July 21, 2026 11:59

Uh oh!

Conversation

pvillard31 commented Mar 11, 2026

Summary

Tracking

Issue Tracking

Pull Request Tracking

Pull Request Formatting

Verification

Build

Licensing

Documentation

Uh oh!

github-actions Bot commented Jul 10, 2026

Uh oh!

exceptionfactory left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pvillard31 commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants