[Bug] When MongoDB processes a transaction, simultaneously performing update and delete operations can trigger a nullnode exception

[bulolo]

Search before asking

• I searched in the issues and found nothing similar.

Paimon version

• paimon all version

Compute Engine

flink

Minimal reproduce step

• run cdc

mongodb_sync_database
--warehouse s3://lakedata/rayflow-paimon
--database saleorder
--mongodb_conf hosts=${mongo-hosts}
--mongodb_conf username=${mongo-cdc-username}
--mongodb_conf password=${mongo-cdc-password}
--mongodb_conf database=saleorder
--mongodb_conf heartbeat.interval.ms=60000
--mongodb_conf connection.options=connectTimeoutMS=3600000&socketTimeoutMS=3600000&maxConnecting=50&maxIdleTimeMS=120000&minPoolSize=20&maxPoolSize=100&serverSelectionTimeoutMS=600000&retryReads=true&retryWrites=true
--catalog_conf s3.endpoint=${s3-endpoint}
--catalog_conf s3.path.style.access=true
--catalog_conf s3.access-key=${s3-access-key}
--catalog_conf s3.secret-key=${s3-secret-key}
--table_conf bucket=2
--table_conf changelog-producer=input
--table_prefix ods_
--including_tables orders|order_aribas|order_item_allots|order_stock_out_items|order_stock_outs|order_reduce_items|order_journals|order_items
--table_conf sink.parallelism=2
--table_conf snapshot.time-retained=6h
--table_conf snapshot.num-retained.min=10
--table_conf snapshot.num-retained.max=1000

• When MongoDB processes a transaction, simultaneously performing update and delete operations can trigger a nullnode exception

What doesn't meet your expectations?

Problem Description: When MongoDB processes a transaction, simultaneously performing update and delete operations can trigger a nullnode exception. The reproduced phenomenon is that the timestamps of the two operations in the opLog are too close, causing Paimon-CDC to collect errors

Anything else?

• Our temporary approach is to add a judgment

Are you willing to submit a PR?

• I'm willing to submit a PR!

[zhang-arvin]

I'd like to work on this issue.

[bulolo]

is there any way to suport mongo6
now,paimon is only suport mongo4