Skip to content

[kafka] Add basic Produce compatibility - #4187

Draft
gyang94 wants to merge 2 commits into
apache:mainfrom
gyang94:codex/kafka-pr2-basic-produce
Draft

[kafka] Add basic Produce compatibility#4187
gyang94 wants to merge 2 commits into
apache:mainfrom
gyang94:codex/kafka-pr2-basic-produce

Conversation

@gyang94

@gyang94 gyang94 commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Purpose

Linked issue: #4185

Depends on #4186.

This is PR 2 of 4 for the Basic Produce compatibility milestone. The intended review delta is the single tip commit 12e1897a8; GitHub shows the PR1 commit as well until #4186 is merged and this branch is rebased onto the updated main branch.

Brief change log

  • Add Kafka Produce request parsing and response construction.
  • Add the gateway-backed Produce backend and record transcoding path.
  • Support Produce acknowledgement modes, including acks=all.
  • Wake delayed follower fetches after append so replicated acknowledgements can complete.
  • Add focused Kafka handler and server delayed-action coverage.

Tests

  • Kafka module: 41 tests passed.
  • Server regression set: 23 tests passed across ReplicaTest, DelayedActionQueueTest, and DelayedFetchLogTest.
  • Reactor build, Checkstyle, and Spotless validation passed.

API and Format

The change adds internal Kafka Produce compatibility types and reuses existing Fluss log formats. It does not change the Fluss public client API or storage format.

Documentation

No standalone user documentation change in this layer; the umbrella issue describes the staged milestone.

Stack

Generative AI disclosure: Codex (GPT-5.6) assisted with implementation and commit organization. Human review is required before merge.

Introduce request dispatch, ApiVersions, metadata discovery, and topic lifecycle operations as the reviewable foundation for subsequent Produce support.

Co-Authored-By: Codex <noreply@openai.com>

AI-Model: gpt-5.6-sol

AI-Contributed/Feature: 2698/2698

AI-Contributed/UT: 1160/1160
Support raw and string Produce requests, validate acks=all semantics, and wake delayed follower fetches after successful appends.

Co-Authored-By: Codex <noreply@openai.com>

AI-Model: gpt-5.6-sol

AI-Contributed/Feature: 1204/1204

AI-Contributed/UT: 2066/2066
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant