conf-yaml: support glob patterns in include directive (v4) - #16031
Open
ssam18 wants to merge 1 commit into
Open
Conversation
SCConfYamlHandleInclude only accepted explicit filenames. The rule-files directive has supported shell-style glob expansion via glob(3) for years, and asymmetry with include has been blocking clean drop-in conf.d/ style configuration directories. Refactor SCConfYamlHandleInclude into a thin wrapper that resolves the path against conf_dirname and, if the input contains glob metacharacters (*, ?, [), expands it with glob(3) and includes each match in lexicographic order. A pattern that matches no files is logged as a warning and not treated as an error, so a drop-in directory can be empty without breaking startup. Literal paths skip glob() and call the existing per-file inclusion logic directly, preserving current behaviour. Glob expansion needs glob(3), which is not available everywhere. Windows builds have no glob.h, so an include pattern used to fall through and be opened as a literal filename, failing with a confusing errno. On such platforms the pattern is now logged as a warning and skipped instead. Literal includes are unaffected. The behaviour is verified by the config-includes-glob-order suricata-verify test, which checks that every file matching a pattern is loaded and that matches are included in deterministic lexicographic order. Document the new behaviour, including the platform limitation, in doc/userguide/configuration/includes.rst. Feature: OISF#8427.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #16031 +/- ##
==========================================
- Coverage 83.04% 83.02% -0.02%
==========================================
Files 1002 1002
Lines 276722 276746 +24
==========================================
- Hits 229794 229762 -32
- Misses 46928 46984 +56
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
jasonish
self-requested a review
August 18, 2026 16:58
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Continuation of #15937 (v4). See "Changes since #15937" below; no code changes since v3.
Contribution style:
https://docs.suricata.io/en/latest/devguide/contributing/contribution-process.html
Our Contribution agreements:
https://suricata.io/about/contribution-agreement/
Changes (if applicable):
https://redmine.openinfosecfoundation.org/projects/suricata/issues
Link to ticket: https://redmine.openinfosecfoundation.org/issues/8427
Previous PR: #15937
Describe changes:
SCConfYamlHandleIncludenow expands shell-style glob patterns (*,?,[) viaglob(3)when the include path contains them. Each match is loaded in lexicographic (sorted) order; literal paths bypassglob()and take the existing per-file path. A pattern matching zero files is logged as a warning, not an error, so drop-inconf.d/-style directories may be empty. The precedent for theglob(3)shape andHAVE_GLOB_Hgating isrule-files:insrc/detect-engine-loader.c.Changes since #15937
No code changes. The 26 suricata-verify failures on #15937's CI were not caused by this change: that CI run checked out the companion suricata-verify branch (OISF/suricata-verify#3250, via the
SV_BRANCHline below), which was based on a June 5 suricata-verifymaster, 119 commits behind. The failing tests (bug-8489-*,detect-email-*,mime-dec-*,ftp-reply-received-02,http2-keywords2) were all tests whose expectations were updated on suricata-verifymasteron July 21 to match newer suricatamainbehavior, so the stale checkout ran old expectations against current code. The glob tests themselves passed, andmainwas green on the same days.main.masterand resubmitted as config: test deterministic glob include ordering (v3) suricata-verify#3281 (superseding config: test deterministic glob include ordering (v2) suricata-verify#3250);SV_BRANCHbelow now points there. No changes to the test itself.Changes since #15574 (carried over from v3)
Addresses @jasonish's review: "Document and accept that globbing doesn't work on Windows for now. Ideally with a warning message if the patterns are found."
Windows has no
glob.h, soHAVE_GLOB_His undefined and an include pattern fell through to be opened as a literal filename, failing with a confusing errno:A pattern is now detected regardless of
HAVE_GLOB_Hand, on platforms withoutglob(3), logged and skipped instead:Literal includes are unaffected, including the existing hard error for a missing literal file.
Documented the platform limitation in
doc/userguide/configuration/includes.rstand in the function's doxygen comment.Rebased onto current
main.Skip vs. error. An unsupported pattern is a warning that continues, reading "accept that globbing doesn't work on Windows" as a documented platform limitation rather than a startup failure. The tradeoff is that a Windows user gets a partially-loaded config with only a warning. Happy to make it a hard error with an explicit message instead if you prefer that.
The suricata-verify test skips on builds without
glob(3)by probingsrc/autoconf.hforHAVE_GLOB_H, so it no longer depends on the Windows runner incidentally failing the probe redirect.Verification
Built with
HAVE_GLOB_Hundefined to exercise the Windows path directly:glob(3)Error: Failed to open ... Invalid argument, exit 1Warning: ... not supported on this platform, exit 0On a normal build: all matched files load in lexicographic order, a no-match pattern still warns without failing, literal includes still work, a missing literal file still errors, and all 9
ConfYamlunit tests pass. suricata-verify:===> config-includes-glob-order: OK, andSKIPPEDon a build withHAVE_GLOB_Hundefined.The Windows CI failures on #15574 dated 2026-06-07 also predate the suricata-verify test rewrite of 2026-06-12, which removed the
suricata.yamlthat broke suricata-verify's setup step; theUbuntu 24.04 (afpacket IPS tests in namespaces)failure there was a transient Codecov CLI GPG error, unrelated to the change.Provide values to any of the below to override the defaults.
SV_BRANCH=OISF/suricata-verify#3281