Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions deploy/docker/Dockerfile-full
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,13 @@ FROM debian:bookworm-slim

COPY ./deploy/docker/docker-entrypoint.sh /usr/bin/docker-entrypoint.sh
COPY --from=builder /go/kuiper/_build/kuiper-* /kuiper/
COPY --from=builder /usr/local/lib/libduckdb.so /usr/local/lib/

ARG DEBIAN_FRONTEND="noninteractive"

RUN apt-get update \
&& apt-get install -y pkg-config libczmq-dev wget \
&& ldconfig \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

Expand Down
26 changes: 26 additions & 0 deletions extensions/impl/sql/sqldatabase/driver/duckdb.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
// Copyright 2022-2025 EMQ Technologies Co., Ltd.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

//go:build (!no_base || duckdb) && !no_duckdb

Copilot AI Dec 11, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation for SQL source plugin (docs/en_US/guide/sources/plugin/sql.md line 14 and docs/zh_CN/guide/sources/plugin/sql.md line 13) states that the plugin supports sqlserver\postgres\mysql\sqlite3\oracle drivers by default. With the addition of DuckDB as a base driver (using the !no_base build tag), the documentation should be updated to include DuckDB in the list of default supported drivers.

Copilot uses AI. Check for mistakes.

Copilot AI Dec 11, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation for SQL sink plugin (docs/en_US/guide/sinks/plugin/sql.md line 10 and docs/zh_CN/guide/sinks/plugin/sql.md line 8) states that the plugin supports sqlserver\postgres\mysql\sqlite3\oracle drivers by default. With the addition of DuckDB as a base driver, the documentation should be updated to include DuckDB in the list of default supported drivers.

Copilot uses AI. Check for mistakes.

Copilot AI Dec 11, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SQL source plugin documentation (docs/en_US/guide/sources/plugin/sql.md line 82-87) includes a table showing URL samples for MySQL, SQL Server, Postgres, and SQLite. With DuckDB being added as a supported database, consider adding a DuckDB URL sample to this table to help users understand the connection string format for DuckDB.

Copilot uses AI. Check for mistakes.

package driver

/*
#cgo LDFLAGS: -lduckdb -lstdc++
*/
import "C"

import (
_ "github.com/duckdb/duckdb-go/v2" // DuckDB driver
)
94 changes: 54 additions & 40 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ require (
github.com/denisenkom/go-mssqldb v0.12.3
github.com/dolthub/go-mysql-server v0.18.1
github.com/dop251/goja v0.0.0-20240828124009-016eb7256539
github.com/duckdb/duckdb-go/v2 v2.5.3
github.com/eclipse/paho.golang v0.21.0
github.com/eclipse/paho.mqtt.golang v1.5.1
github.com/edgexfoundry/go-mod-core-contracts/v4 v4.0.1
Expand All @@ -46,7 +47,7 @@ require (
github.com/jinzhu/now v1.1.5
github.com/jmrobles/h2go v0.5.0
github.com/keepeye/logrus-filename v0.0.0-20190711075016-ce01a4391dd1
github.com/klauspost/compress v1.17.11
github.com/klauspost/compress v1.18.0
github.com/lf-edge/ekuiper/contract/v2 v2.3.1
github.com/lib/pq v1.10.9
github.com/mattn/go-adodb v0.0.1
Expand All @@ -72,7 +73,7 @@ require (
github.com/sijms/go-ora/v2 v2.8.19
github.com/sirupsen/logrus v1.9.3
github.com/snowflakedb/gosnowflake v1.13.3
github.com/stretchr/testify v1.10.0
github.com/stretchr/testify v1.11.1
github.com/thda/tds v0.1.7
github.com/trinodb/trino-go-client v0.316.0
github.com/u2takey/ffmpeg-go v0.5.0
Expand All @@ -86,17 +87,17 @@ require (
github.com/yisaer/file-rotatelogs v0.0.0-20240926070915-3a4d03835c68
github.com/ziutek/mymysql v1.5.4
go.nanomsg.org/mangos/v3 v3.4.2
go.opentelemetry.io/otel v1.34.0
go.opentelemetry.io/otel v1.37.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.29.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.29.0
go.opentelemetry.io/otel/sdk v1.34.0
go.opentelemetry.io/otel/trace v1.34.0
go.opentelemetry.io/otel/sdk v1.37.0
go.opentelemetry.io/otel/trace v1.37.0
go.uber.org/automaxprocs v1.5.3
golang.org/x/exp v0.0.0-20240909161429-701f63a606c0
golang.org/x/exp v0.0.0-20250408133849-7e4ce0ab07d0
golang.org/x/text v0.31.0
google.golang.org/genproto/googleapis/api v0.0.0-20240823204242-4ba0660f739c
google.golang.org/grpc v1.67.1
google.golang.org/protobuf v1.36.1
google.golang.org/genproto/googleapis/api v0.0.0-20250707201910-8d1bb00bc6a7
google.golang.org/grpc v1.75.0
google.golang.org/protobuf v1.36.8
gopkg.in/ini.v1 v1.67.0
gopkg.in/yaml.v3 v3.0.1
modernc.org/ql v1.4.7
Expand All @@ -107,32 +108,48 @@ require (

require (
github.com/DataDog/zstd v1.4.5 // indirect
github.com/apache/arrow-go/v18 v18.0.0 // indirect
github.com/apache/arrow-go/v18 v18.4.1 // indirect
github.com/cockroachdb/errors v1.11.1 // indirect
github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b // indirect
github.com/cockroachdb/redact v1.1.5 // indirect
github.com/cockroachdb/tokenbucket v0.0.0-20230807174530-cc333fc44b06 // indirect
github.com/duckdb/duckdb-go-bindings v0.1.23 // indirect
github.com/duckdb/duckdb-go-bindings/darwin-amd64 v0.1.23 // indirect
github.com/duckdb/duckdb-go-bindings/darwin-arm64 v0.1.23 // indirect
github.com/duckdb/duckdb-go-bindings/linux-amd64 v0.1.23 // indirect
github.com/duckdb/duckdb-go-bindings/linux-arm64 v0.1.23 // indirect
github.com/duckdb/duckdb-go-bindings/windows-amd64 v0.1.23 // indirect
github.com/duckdb/duckdb-go/arrowmapping v0.0.26 // indirect
github.com/duckdb/duckdb-go/mapping v0.0.25 // indirect
github.com/envoyproxy/go-control-plane/envoy v1.32.4 // indirect
github.com/fxamacker/cbor/v2 v2.7.0 // indirect
github.com/getsentry/sentry-go v0.18.0 // indirect
github.com/go-jose/go-jose/v4 v4.1.1 // indirect
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/golang/snappy v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/asmfmt v1.3.2 // indirect
github.com/kr/pretty v0.3.1 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/minio/asm2plan9s v0.0.0-20200509001527-cdd76441f9d8 // indirect
github.com/minio/c2goasm v0.0.0-20190812172519-36a3d3bbc4f3 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/rogpeppe/go-internal v1.13.1 // indirect
github.com/spiffe/go-spiffe/v2 v2.5.0 // indirect
github.com/zeebo/errs v1.4.0 // indirect
go.opentelemetry.io/auto/sdk v1.1.0 // indirect
golang.org/x/telemetry v0.0.0-20251008203120-078029d740a8 // indirect
)

require (
cel.dev/expr v0.16.0 // indirect
cloud.google.com/go v0.115.1 // indirect
cloud.google.com/go/auth v0.9.1 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.4 // indirect
cloud.google.com/go/compute/metadata v0.5.0 // indirect
cloud.google.com/go/iam v1.2.0 // indirect
cloud.google.com/go/longrunning v0.6.0 // indirect
cel.dev/expr v0.24.0 // indirect
cloud.google.com/go v0.121.0 // indirect
cloud.google.com/go/auth v0.16.0 // indirect
cloud.google.com/go/auth/oauth2adapt v0.2.8 // indirect
cloud.google.com/go/compute/metadata v0.7.0 // indirect
cloud.google.com/go/iam v1.5.0 // indirect
cloud.google.com/go/longrunning v0.6.6 // indirect
cloud.google.com/go/spanner v1.67.0 // indirect
dario.cat/mergo v1.0.1 // indirect
filippo.io/edwards25519 v1.1.0 // indirect
Expand All @@ -145,14 +162,13 @@ require (
github.com/ClickHouse/ch-go v0.65.0 // indirect
github.com/DATA-DOG/go-sqlmock v1.4.1 // indirect
github.com/GoogleCloudPlatform/grpc-gcp-go/grpcgcp v1.5.0 // indirect
github.com/JohnCGriffin/overflow v0.0.0-20211019200055-46fa312c352c // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver v1.4.2 // indirect
github.com/Masterminds/semver/v3 v3.3.0 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/alicebob/gopher-json v0.0.0-20230218143504-906a9b012302 // indirect
github.com/andybalholm/brotli v1.1.1 // indirect
github.com/apache/thrift v0.21.0 // indirect
github.com/andybalholm/brotli v1.2.0 // indirect
github.com/apache/thrift v0.22.0 // indirect
github.com/apapsch/go-jsonmerge/v2 v2.0.0 // indirect
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
github.com/avast/retry-go v3.0.0+incompatible // indirect
Expand Down Expand Up @@ -181,9 +197,8 @@ require (
github.com/btnguyen2k/consu/reddo v0.1.8 // indirect
github.com/btnguyen2k/consu/semita v0.1.5 // indirect
github.com/bufbuild/protocompile v0.14.1 // indirect
github.com/census-instrumentation/opencensus-proto v0.4.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/cncf/xds/go v0.0.0-20240723142845-024c85f92f20 // indirect
github.com/cncf/xds/go v0.0.0-20250501225837-2ac532fd4443 // indirect
github.com/couchbase/go-couchbase v0.1.1 // indirect
github.com/couchbase/gomemcached v0.3.1 // indirect
github.com/couchbase/goutils v0.1.2 // indirect
Expand All @@ -201,8 +216,7 @@ require (
github.com/edsrzf/mmap-go v1.1.0 // indirect
github.com/elastic/go-sysinfo v1.8.1 // indirect
github.com/elastic/go-windows v1.0.0 // indirect
github.com/envoyproxy/go-control-plane v0.13.0 // indirect
github.com/envoyproxy/protoc-gen-validate v1.1.0 // indirect
github.com/envoyproxy/protoc-gen-validate v1.2.1 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/fsnotify/fsnotify v1.8.0
github.com/fullsailor/pkcs7 v0.0.0-20190404230743-d7302db945fa // indirect
Expand All @@ -211,7 +225,7 @@ require (
github.com/go-faster/errors v0.7.1 // indirect
github.com/go-kit/kit v0.13.0 // indirect
github.com/go-logfmt/logfmt v0.6.0 // indirect
github.com/go-logr/logr v1.4.2 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
github.com/go-openapi/analysis v0.23.0 // indirect
Expand All @@ -230,17 +244,17 @@ require (
github.com/go-resty/resty/v2 v2.13.1 // indirect
github.com/go-sourcemap/sourcemap v2.1.4+incompatible // indirect
github.com/go-zookeeper/zk v1.0.3 // indirect
github.com/goccy/go-json v0.10.4 // indirect
github.com/goccy/go-json v0.10.5 // indirect
github.com/godbus/dbus v0.0.0-20190726142602-4481cbc300e2 // indirect
github.com/godror/knownpb v0.1.2 // indirect
github.com/golang-sql/civil v0.0.0-20190719163853-cb61b32ac6fe // indirect
github.com/golang-sql/sqlexp v0.1.0 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/google/flatbuffers v24.12.23+incompatible // indirect
github.com/google/flatbuffers v25.2.10+incompatible // indirect
github.com/google/pprof v0.0.0-20240903155634-a8630aee4ab9 // indirect
github.com/google/s2a-go v0.1.8 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
github.com/googleapis/gax-go/v2 v2.13.0 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.6 // indirect
github.com/googleapis/gax-go/v2 v2.14.1 // indirect
github.com/gorilla/schema v1.4.1 // indirect
github.com/gorilla/securecookie v1.1.2 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.22.0 // indirect
Expand Down Expand Up @@ -273,7 +287,7 @@ require (
github.com/josharian/intern v1.0.0 // indirect
github.com/kardianos/osext v0.0.0-20190222173326-2bc1f35cddc0 // indirect
github.com/kataras/go-events v0.0.3 // indirect
github.com/klauspost/cpuid/v2 v2.2.9 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/lestrrat-go/strftime v1.1.0 // indirect
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
Expand Down Expand Up @@ -353,25 +367,25 @@ require (
go.mongodb.org/mongo-driver v1.16.1 // indirect
go.mozilla.org/pkcs7 v0.0.0-20200128120323-432b2356ecb1 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.52.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.53.0 // indirect
go.opentelemetry.io/otel/metric v1.34.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.60.0 // indirect
go.opentelemetry.io/otel/metric v1.37.0 // indirect
go.opentelemetry.io/proto/otlp v1.3.1 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
golang.org/x/crypto v0.45.0 // indirect
golang.org/x/mod v0.29.0 // indirect
golang.org/x/net v0.47.0 // indirect
golang.org/x/oauth2 v0.27.0 // indirect
golang.org/x/oauth2 v0.30.0 // indirect
golang.org/x/sync v0.18.0 // indirect
golang.org/x/sys v0.38.0 // indirect
golang.org/x/term v0.37.0 // indirect
golang.org/x/time v0.6.0
golang.org/x/time v0.11.0
golang.org/x/tools v0.38.0 // indirect
golang.org/x/xerrors v0.0.0-20240903120638-7835f813f4da // indirect
google.golang.org/api v0.195.0 // indirect
google.golang.org/genproto v0.0.0-20240827150818-7e3bb234dfed // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240903143218-8af14fe29dc1 // indirect
google.golang.org/api v0.230.0 // indirect
google.golang.org/genproto v0.0.0-20250303144028-a0af3efb3deb // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250707201910-8d1bb00bc6a7 // indirect
gopkg.in/go-jose/go-jose.v2 v2.6.3 // indirect
gopkg.in/jcmturner/aescts.v1 v1.0.1 // indirect
gopkg.in/jcmturner/dnsutils.v1 v1.0.1 // indirect
Expand Down
Loading
Loading