feat: Horizontal scaling - #285
Draft
saini-prakhyat wants to merge 8 commits into
Draft
Conversation
A single provider pod's terraform throughput is the scaling ceiling; running N instances requires each to watch a disjoint set of Workspaces and elect its own leader independently. --watch-label-selector scopes the manager cache to matching Workspaces, via cache.Options.ByObject rather than DefaultLabelSelector -- the latter would filter every cached type (ProviderConfigs, Secrets, the leader election Lease itself), not just Workspaces. --leader-election-id replaces the previously hardcoded lease name. Without a distinct value per instance, all instances' pods would race for one lease cluster-wide, leaving every instance but one with zero active reconcilers -- not degraded HA, a silent total outage for those shards. Both flags default to today's exact values, so an unmodified single-instance deployment is unaffected. Signed-off-by: Prakhyat Saini <sainiprakhyat045@gmail.com>
Relabeling a Workspace to a new shard mid-apply is already safe today (no destroy is triggered, and the Terraform state lock serializes any overlap), but it's noisy -- the new owner fails against the held lock -- and a crash leaves a stale lock needing a manual force-unlock. Adds a per-Workspace ownership claim, stored in a dedicated coordination.k8s.io/v1 Lease (internal/claims), guarding every Apply and Destroy. A fresh foreign claim defers the run; a stale one is stolen, with ForceUnlock (internal/terraform) automatically clearing the dead owner's state lock first. Off by default (--enable-ownership-claims); when off, this is a pure pass-through with no behavioral change. Signed-off-by: Prakhyat Saini <sainiprakhyat045@gmail.com>
Signed-off-by: Prakhyat Saini <sainiprakhyat045@gmail.com>
Signed-off-by: Pavithra CP <cppavithra05@gmail.com>
Co-authored-by: Paul Joseph <k.paul.joseph@gmail.com>
…lector crash Signed-off-by: Tejas M D <tejasmd22@gmail.com>
…belled mid-apply Signed-off-by: Sumanth Bhat <sumanth1198@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of your changes
Fixes #
I have:
make reviewableto ensure this PR is ready for review.How has this code been tested