Backport 8.0.x: firewall: add default-policy to policy config v1

doc/userguide/devguide/extending/app-layer/transactions.rst

@@ -73,7 +73,7 @@ Rule Matching
 Transaction progress is also used for certain keywords to know what is the minimum state before we can expect a match: until that, Suricata won't even try to look for the patterns.
 
 As seen in ``DetectAppLayerMpmRegister`` that has ``int progress`` as parameter, and ``DetectAppLayerInspectEngineRegister``, which expects ``int tx_min_progress``, for instance. In the code snippet,
-``HTTP2StateDataClient``, ``HTTP2StateDataServer`` and ``0`` are the values passed to the functions - in the last
+``HTTP2ProgData``, ``HTTP2ProgData`` and ``0`` are the values passed to the functions - in the last
 example, for ``FTPDATA``,
 the existence of a transaction implies that a file is being transferred. Hence the ``0`` value.
 
@@ -86,14 +86,14 @@ the existence of a transaction implies that a file is being transferred. Hence t
         .
         DetectAppLayerMpmRegister("file_data", SIG_FLAG_TOSERVER, 2,
                 PrefilterMpmFiledataRegister, NULL,
-                ALPROTO_HTTP2, HTTP2StateDataClient);
+                ALPROTO_HTTP2, HTTP2ProgData);
         DetectAppLayerMpmRegister("file_data", SIG_FLAG_TOCLIENT, 2,
                 PrefilterMpmFiledataRegister, NULL,
-                ALPROTO_HTTP2, HTTP2StateDataServer);
+                ALPROTO_HTTP2, HTTP2ProgData);
         .
         .
         DetectAppLayerInspectEngineRegister("file_data",
-            ALPROTO_HTTP2, SIG_FLAG_TOCLIENT, HTTP2StateDataServer,
+            ALPROTO_HTTP2, SIG_FLAG_TOCLIENT, HTTP2ProgData,
             DetectEngineInspectFiledata, NULL);
         DetectAppLayerInspectEngineRegister(
                 "file_data", ALPROTO_FTPDATA, SIG_FLAG_TOSERVER, 0, DetectEngineInspectFiledata, NULL);

doc/userguide/firewall/firewall-design.rst

@@ -62,7 +62,7 @@ Application layer tables
 ~~~~~~~~~~~~~~~~~~~~~~~~
 
 If applayer is available, rules from the following tables apply. The tables for the
-application layer are per app layer protocol and per protocol state. e.g. ``http:request_line``.
+application layer are per app layer protocol and per protocol state. e.g. ``http1:request_line``.
 
 
 .. table::
@@ -327,28 +327,57 @@ of :ref:`engine analysis<config:engine-analysis>`.
 Default policies
 ================
 
-Each hook has a default policy. By default ``packet:filter`` enforces a ``drop:packet`` policy and the
-``app:filter`` hooks applies ``drop:flow``.
+Each hook has a default policy applied to traffic that no firewall rule handled.
+By default ``packet.filter`` enforces ``drop:packet``, ``packet.pre-flow`` and
+``packet.pre-stream`` enforce ``accept:hook``, and every ``app`` hook enforces
+``drop:flow``.
 
-The policies can be configured in ``firewall`` block in the config.
-
-Example for ``packet:filter``, to use reject instead of drop::
+Defaults are configured in the ``firewall.policies`` block. A ``default-policy``
+may be given at several levels; for any hook the most specific present setting
+wins::
 
     firewall:
       policies:
-        packet-filter: [ "reject:packet" ]
-
-
-Example for DNS::
+        default-policy: ["accept:hook"]     # global fallback (all hooks)
+        packet:
+          default-policy: ["drop:packet"]   # fallback for packet hooks
+          filter:     ["drop:packet"]
+          pre-flow:   ["accept:hook"]
+          pre-stream: ["accept:hook"]
+        app:
+          default-policy: ["drop:flow"]     # fallback for all app hooks
+          dns:
+            default-policy: ["drop:flow"]   # fallback for dns hooks
+            request-started: ["accept:hook"]
+            request-complete: ["drop:flow", "alert"]
+            response-started: ["accept:tx"]
+
+Protocols whose hooks are grouped into sub states, such as HTTP/2, take an extra
+level for the sub state name::
 
     firewall:
       policies:
-        dns:
-          request-started: ["accept:hook"]
-
-          # Drop and alert on all DNS requests that are not allowed in
-          # firewall.rules.
-          request-complete: ["drop:flow", "alert"]
-
-          # Accept all responses.
-          response-started: ["accept:tx"]
+        app:
+          http2:
+            default-policy: ["drop:flow"]   # fallback for all http2 hooks
+            stream:
+              default-policy: ["drop:flow"] # fallback for http2 stream hooks
+              request-started: ["accept:hook"]
+            global:
+              request-started: ["accept:hook"]
+
+Precedence:
+
+* packet hook: ``packet.<hook>`` > ``packet.default-policy`` >
+  ``policies.default-policy`` > built-in (``drop:packet`` or ``accept:hook``)
+* app hook: ``app.<proto>.<hook>`` > ``app.<proto>.default-policy`` >
+  ``app.default-policy`` > ``policies.default-policy`` > built-in (``drop:flow``)
+* app hook in a sub state: ``app.<proto>.<sub state>.<hook>`` >
+  ``app.<proto>.<sub state>.default-policy`` > ``app.<proto>.default-policy`` >
+  ``app.default-policy`` > ``policies.default-policy`` > built-in (``drop:flow``)
+
+An action scope must be valid for the hook it is applied to. For example,
+defining ``accept:tx`` as a global default policy will fail to start Suricata,
+because ``packet`` policies do not accept ``tx``.
+Cover such hooks with a more specific setting so the incompatible default never
+reaches them.

doc/userguide/firewall/firewall-example.rst

@@ -62,35 +62,36 @@ HTTP example with partially using default policies
 
     firewall:
       policies:
-        http:
-          request-started:
-            - "accept:hook"
-          request-line:
-            - "drop:flow"
-            - "alert"
-          request-headers:
-            - "drop:flow"
-            - "alert"
-          request-body:
-            - "accept:hook"
-          request-trailer:
-            - "accept:hook"
-          request-complete:
-            - "accept:hook"
-
-          response-started:
-            - "accept:hook"
-          response-line:
-            - "drop:flow"
-            - "alert"
-          response-headers:
-            - "accept:hook"
-          response-body:
-            - "accept:hook"
-          response-trailer:
-            - "accept:hook"
-          response-complete:
-            - "accept:hook"
+        app:
+          http1:
+            request-started:
+              - "accept:hook"
+            request-line:
+              - "drop:flow"
+              - "alert"
+            request-headers:
+              - "drop:flow"
+              - "alert"
+            request-body:
+              - "accept:hook"
+            request-trailer:
+              - "accept:hook"
+            request-complete:
+              - "accept:hook"
+
+            response-started:
+              - "accept:hook"
+            response-line:
+              - "drop:flow"
+              - "alert"
+            response-headers:
+              - "accept:hook"
+            response-body:
+              - "accept:hook"
+            response-trailer:
+              - "accept:hook"
+            response-complete:
+              - "accept:hook"
 
 
 ::

doc/userguide/rules/app-layer.rst

@@ -10,7 +10,11 @@ Match on the detected app-layer protocol.
 
 Syntax::
 
-    app-layer-protocol:[!]<protocol>(,<mode>);
+    app-layer-protocol:[!]<protocol>[,<qualifier>]...;
+    app-layer-protocol:[!]<proto1>|<proto2>[|...|<protoN>][,<qualifier>]...;
+
+Each ``<qualifier>`` is either a ``<mode>`` (at most one, see below) or the
+``exact`` option, in any order.
 
 Examples::
 
@@ -21,6 +25,12 @@ Examples::
     app-layer-protocol:http,to_server; app-layer-protocol:tls,to_client;
     app-layer-protocol:http2,final; app-layer-protocol:http1,original;
     app-layer-protocol:unknown;
+    app-layer-protocol:unknown|tls;
+    app-layer-protocol:unknown|tls|http;
+    app-layer-protocol:!tls|http;
+    app-layer-protocol:tls|http,either;
+    app-layer-protocol:dns,exact;
+    app-layer-protocol:tls|dns,either,exact;
 
 A special value 'failed' can be used for matching on flows in which
 protocol detection failed. This can happen if Suricata doesn't know
@@ -42,12 +52,92 @@ By default, (if no mode is specified), the mode is ``direction``.
 .. note:: when negation is used, like ``!http``, it will not match on the
    "unknown" state in the flow.
 
+Protocol equivalences and the ``exact`` option
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+By default a value matches its related protocols as well as itself. For
+example ``http`` matches ``http1`` and ``http2``, ``dns`` also matches
+``doh2`` (DNS over HTTP/2), and ``dcerpc`` also matches ``smb``. This is the
+long-standing behaviour and keeps existing rules working.
+
+Add the ``exact`` qualifier to match strictly, with no equivalences: the
+flow's protocol must equal the configured value exactly. ``exact`` applies to
+all values in the list and can be combined with a mode::
+
+    app-layer-protocol:dns,exact;          # matches dns only, not doh2
+    app-layer-protocol:tls|dns,either,exact;
+
+Because ``exact`` disables all equivalences, the generic ``http`` value is not
+expanded to ``http1``/``http2`` either. A flow is never classified as the
+generic ``http``, so ``app-layer-protocol:http,exact`` can never match and is
+rejected at rule load; use ``http1`` or ``http2`` instead.
+
 Here is an example of a rule matching non-http traffic on port 80:
 
 .. container:: example-rule
 
     alert tcp any any -> any 80 (msg:"non-HTTP traffic over HTTP standard port"; flow:to_server; app-layer-protocol:!http,final; sid:1; )
 
+Multi-value form
+~~~~~~~~~~~~~~~~
+
+The ``app-layer-protocol`` keyword also accepts a pipe-separated (``|``) list
+of protocol values. A rule matches when the flow's resolved application-layer
+protocol equals **any** value in the list (logical OR).
+
+Syntax::
+
+    app-layer-protocol:[!]<proto1>|<proto2>[|...|<protoN>](,<mode>);
+
+Using ``|`` for the list keeps the optional trailing ``,<mode>`` qualifier
+unambiguous, so the single-value ``<protocol>,<mode>`` form is unchanged.
+
+Examples::
+
+    app-layer-protocol:unknown|tls;
+    app-layer-protocol:unknown|tls|http;
+    app-layer-protocol:tls|http,either;
+    app-layer-protocol:!tls|http;
+
+The ``unknown|<proto>`` detection-window idiom
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When Suricata has not yet classified a flow's protocol (the "detection
+window"), the flow's app-layer protocol is ``unknown``. Once protocol
+detection completes, the protocol transitions to its classified value
+(e.g., ``tls``, ``http``). Including ``unknown`` in a multi-value list
+allows a single rule to cover both the detection window and the confirmed
+protocol::
+
+    app-layer-protocol:unknown|tls;
+
+This rule matches during the detection window (while the protocol is still
+``unknown``) **and** after classification (when the protocol is ``tls``).
+If the flow is classified to a protocol not in the list (e.g., ``http``),
+the rule stops matching once the protocol is classified; in firewall mode the
+flow is then handled by the default policy if no other rule accepts it.
+
+Negated multi-value (NOR semantics)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When the multi-value form is negated with ``!``, it implements NOR semantics
+across the entire list: the rule matches when the resolved application-layer
+protocol is **known** AND matches **none** of the listed values.
+
+Example::
+
+    app-layer-protocol:!tls|http;
+
+This matches when the flow's protocol is known and is neither ``tls`` nor
+``http`` (e.g., it matches ``dns``, ``ssh``, ``smtp``, etc.).
+
+.. note:: Negated multi-value rules do not match during the detection window
+   (when the protocol is still ``unknown``). This prevents false positives
+   before protocol classification is complete.
+
+.. note:: The value ``unknown`` cannot appear in a negated list. The parser
+   rejects ``!unknown`` and ``!unknown|tls`` at rule-load time.
+
 .. _proto-detect-bail-out:
 
 Bail out conditions

doc/userguide/upgrade.rst

@@ -46,6 +46,14 @@ Deprecations
 Upgrading to 8.0.5
 ------------------
 
+Keyword Changes
+~~~~~~~~~~~~~~~
+
+- HTTP2 keywords have now better progress defined, with the http2 transaction progress
+  being split per direction. This means that some rules should match sooner,
+  some rules will have less false negatives, and some rules will trigger once per transaction
+  instead of twice (one time for each direction)
+
 Other Changes
 ~~~~~~~~~~~~~
 

etc/schema.json

@@ -8276,6 +8276,10 @@
             "type": "object",
             "additionalProperties": true
         },
+        "sub_state": {
+            "type": "string",
+            "description": "Transaction type or sub state."
+        },
         "suricata_version": {
             "type": "string"
         },

rust/cbindgen.toml

@@ -88,7 +88,9 @@ include = [
     "FtpRequestCommand",
     "FtpStateValues",
     "FtpDataStateValues",
-    "HTTP2TransactionState",
+    "HTTP2TxProgress",
+    "HTTP2TxGlobalProgress",
+    "HTTP2TxType",
     "DataRepType",
 ]
 

rust/src/applayer.rs

@@ -144,6 +144,12 @@ pub struct AppLayerTxData {
     ///
     detect_progress_ts: u8,
     detect_progress_tc: u8,
+    #[doc = " Type of transaction. Meaning is defined by the parser. Used to\n select a state machine. 0 means it is not used."]
+    pub tx_type: u8,
+    #[doc = " End of TX progress values\n\n toserver end of tx progress value"]
+    pub tx_type_eop_ts: u8,
+    #[doc = " toclient end of tx progress value"]
+    pub tx_type_eop_tc: u8,
 
     de_state: *mut DetectEngineState,
     pub events: *mut core::AppLayerDecoderEvents,
@@ -198,6 +204,9 @@ impl AppLayerTxData {
             flags: 0,
             detect_progress_ts: 0,
             detect_progress_tc: 0,
+            tx_type: 0,
+            tx_type_eop_ts: 0,
+            tx_type_eop_tc: 0,
             de_state: std::ptr::null_mut(),
             events: std::ptr::null_mut(),
             txbits: std::ptr::null_mut(),
@@ -225,6 +234,9 @@ impl AppLayerTxData {
             detect_progress_ts: 0,
             detect_progress_tc: 0,
             flags,
+            tx_type: 0,
+            tx_type_eop_ts: 0,
+            tx_type_eop_tc: 0,
             de_state: std::ptr::null_mut(),
             events: std::ptr::null_mut(),
             txbits: std::ptr::null_mut(),