Backport8 8731 8799 8995 v3 - #16190
Closed
catenacyber wants to merge 495 commits into
Closed
Conversation
Add flushing logic driven off of the file contexts. This is a simpler solution that removes the need for logger registration changes. Overview: Use the heartbeat-driven thread to periodically flush all registered EVE contexts via a global flush list. The global flush list is a mutex-protected TAILQ of LogFileFlushEntry nodes; each node points to a LogFileCtx. Mutex = log_file_flush_mutex Periodic flushing performed by a thread according to the heartbeat.output-flush-interval [1,60]. LogFileFlushAll() is invoked to initiate flushing of registered LogFileCtx structs; each struct's fp_mutex is obtained while the flush occurs to synchronize with LogFileWrite activity. Interacts with file-rotation via the fp_mutex. Deadlock prevention: the log_file_flush_mutex must be obtained before the fp_mutex. Issue: 8286 (cherry picked from commit a78911f)
Remove packet-based flush logic in favor of simpler solution Issue: 8286 (cherry picked from commit d0ba1c4)
Remove log flush functions and update registration logic as context-based flushing doesn't require it. Issue: 8286 (cherry picked from commit 1923ca1)
Update output flushing description to reflect EVE based approach in documentation and config template. Added: Provide update callout for out-of-tree output plugins. Issue: 8286 (cherry picked from commit e7dc0d8)
Update the public suffix list to the latest release. Ticket: OISF#8360
When libraries add their own -march flags to CFLAGS, let them. But if -march=native should be added (it should by default), add it as the last -march flag to "win". This ensures, that even if a library brings their own minimal -march, it will not downgrade the instruction set for the whole Suricata. This happened with DPDK which, when enabled, downgraded the instruction set to -march=corei7 or -march=haswell. When building packages with --disable-gccmarch-native, the instruction set is auto-upgraded to instruction set requested by libraries. E.g. from RPM builds: `CFLAGS="-march=x86_64" ./configure --enable-dpdk` upgrades instruction set to DPDK's corei7 architecture. Ticket: 8169 (cherry picked from commit 0cb88d6)
Add a file containing rules to match bittorrent protocol events. Ticket: 8421 (cherry picked from commit 42123cc)
Add a file containing rules to match SNMP protocol events. Ticket: 8421 (cherry picked from commit cff3d64)
util-logopenfile.c:837:11: error: initializing 'char *' with an expression of type 'const char *' discards qualifiers [-Werror,-Wincompatible-pointer-types-discards-qualifiers]
837 | char *dot = strrchr(base, '.');
| ^ ~~~~~~~~~~~~~~~~~~
util-logopenfile.c:848:13: error: assigning to 'char *' from 'const char *' discards qualifiers [-Werror,-Wincompatible-pointer-types-discards-qualifiers]
848 | dot = strrchr(original_name, '.');
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 errors generated.
(cherry picked from commit cd1c596)
app-layer-ftp.c:278:14: error: initializing 'uint8_t *' (aka 'unsigned char *') with an expression of type 'const void *' discards qualifiers
[-Werror,-Wincompatible-pointer-types-discards-qualifiers]
278 | uint8_t *lf_idx = memchr(input->buf + input->consumed, 0x0a, input->len);
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
app-layer-ftp.c:1392:11: error: initializing 'char *' with an expression of type 'const char *' discards qualifiers
[-Werror,-Wincompatible-pointer-types-discards-qualifiers]
1392 | char *c = strchr(buffer, '\n');
| ^ ~~~~~~~~~~~~~~~~~~~~
2 errors generated.
(cherry picked from commit c21275c)
app-layer-smtp.c:566:14: error: initializing 'uint8_t *' (aka 'unsigned char *') with an expression of type 'const void *' discards qualifiers
[-Werror,-Wincompatible-pointer-types-discards-qualifiers]
566 | uint8_t *lf_idx = memchr(input->buf + input->consumed, 0x0a, input->len);
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
(cherry picked from commit 5aee5ff)
conf-yaml-loader.c:87:8: error: assigning to 'char *' from 'const char *' discards qualifiers [-Werror,-Wincompatible-pointer-types-discards-qualifiers]
87 | ep = strrchr(filename, '\\');
| ^ ~~~~~~~~~~~~~~~~~~~~~~~
conf-yaml-loader.c:89:12: error: assigning to 'char *' from 'const char *' discards qualifiers [-Werror,-Wincompatible-pointer-types-discards-qualifiers]
89 | ep = strrchr(filename, '/');
| ^ ~~~~~~~~~~~~~~~~~~~~~~
2 errors generated.
datasets.c:159:11: error: initializing 'char *' with an expression of type 'const char *' discards qualifiers
[-Werror,-Wincompatible-pointer-types-discards-qualifiers]
159 | char *got_colon = strchr(line, ':');
| ^ ~~~~~~~~~~~~~~~~~
1 error generated.
(cherry picked from commit 90ce1c5)
decode.c:767:19: error: initializing 'char *' with an expression of type 'const char *' discards qualifiers
[-Werror,-Wincompatible-pointer-types-discards-qualifiers]
767 | char *dot = strchr(DEvents[i].event_name, '.');
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
(cherry picked from commit a45a70b)
detect-app-layer-protocol.c:160:11: error: initializing 'char *' with an expression of type 'const char *' discards qualifiers
[-Werror,-Wincompatible-pointer-types-discards-qualifiers]
160 | char *sep = strchr(arg, ',');
| ^ ~~~~~~~~~~~~~~~~
1 error generated.
(cherry picked from commit 3564f2f)
detect-pcre.c:419:11: error: initializing 'char *' with an expression of type 'const char *' discards qualifiers [-Werror,-Wincompatible-pointer-types-discards-qualifiers]
419 | char *fcap = strstr(regexstr, "flow:");
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~~
detect-pcre.c:420:11: error: initializing 'char *' with an expression of type 'const char *' discards qualifiers [-Werror,-Wincompatible-pointer-types-discards-qualifiers]
420 | char *pcap = strstr(regexstr, "pkt:");
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~
detect-pcre.c:421:11: error: initializing 'char *' with an expression of type 'const char *' discards qualifiers [-Werror,-Wincompatible-pointer-types-discards-qualifiers]
421 | char *acap = strstr(regexstr, "alert:");
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~
3 errors generated.
(cherry picked from commit 0a4685f)
util-magic.c:436:11: error: initializing 'char *' with an expression of type 'const char *' discards qualifiers [-Werror,-Wincompatible-pointer-types-discards-qualifiers]
436 | char *str = strstr(result, "OpenDocument Text");
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
util-path.c:251:11: error: initializing 'char *' with an expression of type 'const char *' discards qualifiers
[-Werror,-Wincompatible-pointer-types-discards-qualifiers]
251 | char *final = strrchr(path, DIRECTORY_SEPARATOR);
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
(cherry picked from commit 72e3d7a)
When we got a bind without the first fragment flag, and did not set any_frag in the signature, the signature always matched, whatever the uuid value Ticket: 8457 (cherry picked from commit e388888)
Other parsers (SMB, POP3, ENIP) already raise a too_many_transactions event when a flow exceeds its per-flow transaction limit. FTP had the limit wired up through app-layer.protocols.ftp.max-tx but just dropped the excess transaction on the floor with a "FTP does not set events yet..." TODO, so no event was raised when the limit was reached. Add the FtpEventTooManyTransactions variant, a stock rule at sid 2232002, and raise the event from FTPTransactionCreate when the live-tx count goes past ftp_config_maxtx. Issue: 8489 (cherry picked from commit 9ea2e29)
The first version of the event set done=true on state->curr_tx and returned NULL when the limit hit. curr_tx is usually the active command just received — marking it done mid-request throws off response matching, and subsequent commands in the flow stop getting logged. SMB behavior is mirrored here: Walk the tx list, find the oldest tx that isn't done, mark it done and tag it with the event, then fall through and create the new tx so the flow parsing continues. One stale tx gets reaped per overflow so memory stays bounded. Issue: 8489 (cherry picked from commit 5ddd808)
Add missing FTP configuration value to suricata.yaml.in showing the default value. Issue: 8489 (cherry picked from commit 86fe20d)
Issue: 8489 (cherry picked from commit c48bb1b)
Ticket: 8336 When a packet has flag PKT_PSEUDO_DETECTLOG_FLUSH, we do not expect to rerun detection on the same tx and direction again So, do not set mpm_in_progress whose purpose is to not store the state as we will run again. Allows transactional bidirectional signatures to work on thse log+flush pair of packets (cherry picked from commit 1ba4114)
Ticket: 8336 At the end of a TLS handshake, in IDS mode, the client acks, and we parse the server hello and use tls.encryption-handling to know what to do next (for example bypass) Everything is parsed, but we have not run detection yet on neither side. So, in IDS mode, we need to first flush the client side, as the comment on the function already stated. (cherry picked from commit cefd134)
As we expect a second log+flush packet in the other direction Ticket: 8336 (cherry picked from commit cb62d6d)
NotRequest and NotResponse event type have never been used. Unsolicited is also unused, however there is a rule for it so removing support would be a breaking change at this time. (cherry picked from commit 879846e)
Fedora 42 is EOL.
Mapping an app-layer progress state to its generic request-/response- hook alias was hardcoded in multiple places. This commit adds a wrapper to unify it to a single function. It returns the config-form (hyphenated) name, or NULL for an intermediate state. Ticket: 8770 (cherry picked from commit 1aa0259)
Every hook has a built-in default policy, but expressing anything other than the built-in meant naming each hook explicitly. This commit adds a `default-policy` setting that covers all hooks below it. For any hook the most specific setting present wins. Ticket: 8770 (cherry picked from commit 0aaa80d)
Validate the resolved scope against the class of hook it is being applied to and fail at startup if it does not fit. Ticket: 8770 (cherry picked from commit 092ae27)
Ticket: 8678 In FilePruneFile() `window * 3` was computed in uint32_t arithmetic. The guard exists to ensure `file->size > window`, so on wrap around `file->size - window` underflows and content_inspected is set to a bogus value, breaking file inspection. Make window uint64_t so the multiplication cannot wrap. (cherry picked from commit f3910d5)
Ticket: 8678 With an inspect window of 0xAAAAAAAB the `window * 3` guard wrapped to 1 in uint32_t arithmetic, so it passed for a 64 byte file and content_inspected was set to `file->size - window`, an underflow. (cherry picked from commit 8e551f2)
The fn to flush the ring is supposed to flush the entire ring buffer but it only resets the buffer up to the "length" bytes. Fix this calculation to correctly reflect the number of bytes that need to be reset. Ticket: 8826 (cherry picked from commit 389700e)
Setting a cpu-affinity path with --set, like --set threading.cpu-affinity.worker-cpu-set.threads=28 creates an intermediate node under cpu-affinity whose val is NULL. With a legacy list format cpu-affinity, AffinitySetupLoadFromConfig() red the set name from the node val and passed it to GetAffinitySetName(), which then dereferenced the NULL pointer. Address it by simply checking for the NULL. Ticket: OISF#6735. (cherry picked from commit 54504ed)
Zero the result when a byte_math right shift count reaches 64, the width of the uint64_t being shifted, so the operation no longer depends on behavior C11 6.5.7p3 leaves undefined. The left shift case has done this since 473ca6d; the right shift case was left unguarded. DetectByteMathDoMatch() shifted by whatever count it was handed. On x86_64 the hardware masks the count to its low six bits, so a count of 64 became a shift of 0 and returned the extracted value unchanged instead of 0. That value is stored in det_ctx->byte_values[] and feeds any byte_test, isdataat, or content offset later in the signature, so the signature's verdict follows from an arithmetic result the standard does not define. The count reaches the shift from the wire. When byte_math names a variable for rvalue, DetectEngineContentInspectionInternal() reads it out of det_ctx->byte_values[] at detect-engine-content-inspection.c:614, where a preceding byte_extract stored bytes taken from the payload, so one payload byte of 0x40 sets the count to 64. Issue: 8845 Issue: 8902 (cherry picked from commit e5d035f)
Warn from DetectByteMathSetup() when byte_math pairs << or >> with a literal rvalue of 64 or more, naming the signature by sid. Such a shift gives 0 for every packet: DetectByteMathDoMatch() zeroes the result once the count reaches 64, the width of the uint64_t being shifted. rvalue was bounded only to the u32 range, so the rule loaded and ran a shift whose result was 0 whatever the packet held. The rule still loads. main rejects it in 89d09c4, the commit this one is derived from, but a released branch should not stop loading rules that loaded before. Passing byte_math to --strict-rule-keywords turns the warning into a load failure, the same escalation classtype and reference apply to values they cannot check. A variable rvalue draws no warning. It resolves to a preceding byte_extract or byte_math result, whose value is known only once the rule runs, so the guard in DetectByteMathDoMatch() stays the only check on that path. Issue: 8845 Issue: 8902 (cherry picked from commit 89d09c4)
Record beside the existing note about division by zero that << and >> give 0 when rvalue is 64 or more, that a rule giving such an rvalue as a number still loads with a warning, and that --strict-rule-keywords makes it fail to load instead. rvalue can name a byte_extract or byte_math variable rather than a number, and then the count is not known until the rule runs, so the rule text does not say whether it will reach 64. The note on main stops at the result, because 89d09c4 rejects such a rule there instead of warning about it. Issue: 8845 Issue: 8902 (cherry picked from commit 4a64778)
The crash handler built its stack trace by advancing a pointer with the snprintf return value. That value is the length the output would have had, not what was written, so once the trace filled msg the pointer moved past the end of the buffer. Every size argument after that, SC_LOG_MAX_LOG_MSG_LEN - (temp - msg), was negative and converted to a huge size_t, and snprintf went on writing into the stack past msg. With a deep enough stack the handler faults before it can log the crash it exists to report. The frame name lookup also compared unw_get_proc_name() against UNW_ENOMEM, but libunwind returns the negated code, so the test never fired. A lookup that failed with UNW_ENOINFO fell through to the success branch and printed the name buffer, which libunwind had not written. Track space used with offset and use that on each snprintf call. Advance by MIN(cw, remaining - 1) to prevent overruns. The loop stops when the buffer is full and closes the trace with "..." so a clipped message is not read as a whole stack. Failed symbol lookups now print "[unknown]:". -UNW_ENOMEM is not one of them, since libunwind returns a truncated but usable name with it. name starts as "?" so nothing is printed from it either way. Ticket: 8846 (cherry picked from commit cd7f775)
Just separate the steps in parsing, and deduplicate code (cherry picked from commit 52c0154)
And does not consider automatically a port after first colon Ticket: 8778 (cherry picked from commit b0b353d)
Rust website has changed the location of the installer. Additionally their installer now includes automatic append to .bashrc to source the correct path to Rust binaries. Cbindgen as an instruction is also removed as it is not needed when building from the downloaded and packaged Suricata. Ticket: 8344 (cherry picked from commit d7bf52e)
if :authority header is absent Ticket: 8774 (cherry picked from commit f5e44ce)
Ticket: 8774 (cherry picked from commit 0b0606b)
nfs4_res_secinfo_no_name() used a local var "i2" to walk through the security-flavor array entries, but returned the pre-walk var "i" in the Ok at the end. Every byte of the flavor array was therefore left unconsumed in the input stream and re-interpreted by count next iteration as a subsequent compound operation. Addressed by returning i2 instead of i so all flavor bytes are consumed. Ticket: OISF#8860. (cherry picked from commit 9b491a1)
Ticket: 8688
Ticket: 6478 (cherry picked from commit d0cc39c)
Ticket: 8731 Adds a configurable limit to string lengths to avoid to retain too much memory for too long, and avoid producing log events that are too big (cherry picked from commit f9515dc)
And log it with from_utf8_lossy Since we may now cut a too big failure_reason breaking a valid utf-8 string Ticket: 8731 (cherry picked from commit c608165)
Suricata never removed the unix command socket file after shutdown. The socket path (sockettarget) is now preserved in the struct UnixCommand_, because it is needed on shutdown to close the socket. The cleanup is done in the shutdown path of UnixManager(), which is reached uniformly whether shutdown is triggered by a fatal error or a signal (e.g. SIGTERM) via THV_KILL. This also applies for a clean shutdown via 'suricatasc -c shutdown'. Ticket: 8799 (cherry picked from commit 011f523)
A server can reject a BDAT command before the client sends the advertised chunk. Suricata would keep waiting for those bytes and treat the rest of the SMTP session as chunk data. Keeps subsequent SMTP commands and message data in sync with the server. Ticket: 8995 (cherry picked from commit 51ac400)
catenacyber
requested review from
a team,
inashivb,
jasonish,
jufajardini and
victorjulien
as code owners
September 5, 2026 16:46
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
14 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to ticket: https://redmine.openinfosecfoundation.org/issues/
https://redmine.openinfosecfoundation.org/issues/8732
https://redmine.openinfosecfoundation.org/issues/8802
https://redmine.openinfosecfoundation.org/issues/8996
Describe changes:
SV_BRANCH=OISF/suricata-verify#3356
#16189 with missing json schema commit cherry-picked