Skip to content

DOCS-1738 - Document 256KB log message size support - #6881

Open
JV0812 wants to merge 5 commits into
mainfrom
DOCS-1738-256kb-log-message-size
Open

DOCS-1738 - Document 256KB log message size support#6881
JV0812 wants to merge 5 commits into
mainfrom
DOCS-1738-256kb-log-message-size

Conversation

@JV0812

@JV0812 JV0812 commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator

Purpose of this pull request

This pull request documents the increase of the maximum log message size from 64KB to 256KB, calls out the known limitations, and adds a service release note.

Changes:

  • Updated max-message-size references from 64KB to 256KB (and _size truncation from 65536 to 262144) in search-large-messages.md, collect-multiline-logs.md, cloud-syslog-source/index.md, and qualys-vmdr-source.md.
  • Added a Known limitations section to search-large-messages.md (LogCompare/LogReduce, UI 25-messages-per-page, Cloud SIEM, FER).
  • Added LogCompare/LogReduce truncation + response-time notes on logcompare.md and detect-patterns-with-logreduce.md.
  • Clarified in fer-limitations.md that the 64KB cumulative field cap applies regardless of message size.
  • New service release note blog-service/2026-07-09-search.md.

Open items to confirm before marking ready

  • FER cumulative field limit stays 64KB (documented as such) — confirm not raised.
  • Cloud SIEM limitation wording ("might not process messages larger than 64KB correctly"; ref SUMO-257908).
  • Release note date/category (2026-07-09 / Search).

Select the type of change

  • Minor Changes - Typos, formatting, slight revisions
  • Update Content - Revisions, updating sections
  • New Content - New features, sections, pages, tutorials
  • Site and Tools - .clabot, version updates, maintenance, dependencies, new packages for the site (Docusaurus, Gatsby, React, etc.)

Ticket (if applicable)

https://sumologic.atlassian.net/browse/DOCS-1738 (epic: SUMO-288529)

Update max log message size references from 64KB to 256KB across search,
collection, and source docs; add a Known limitations section and
LogCompare/LogReduce truncation notes; clarify the FER 64KB cap; and add a
service release note.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@JV0812 JV0812 self-assigned this Jul 9, 2026
@cla-bot cla-bot Bot added the cla-signed Contributor approved, listed in .clabot file label Jul 9, 2026
Comment thread docs/search/get-started-with-search/search-basics/search-large-messages.md Outdated
Comment thread blog-service/2026-07-09-search.md Outdated
Update the Log Search messages-table limitation to the GA behavior (table
shows up to 64KB after expansion; full message via Log Message Inspector),
and drop the unverified large-log source examples from the release note.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@JV0812
JV0812 requested a review from lei-sumo July 21, 2026 08:36
@JV0812
JV0812 marked this pull request as ready for review July 21, 2026 08:37
@kimsauce

Copy link
Copy Markdown
Collaborator

Holding off on approval — the PR description lists three open items still unconfirmed:

  • FER cumulative field limit stays 64KB (documented as such) — confirm not raised.
  • Cloud SIEM limitation wording ("might not process messages larger than 64KB correctly"; ref SUMO-257908).
  • Release note date/category (2026-07-09 / Search).

Could you confirm these and check the boxes before this goes up for merge? The doc mechanics (links, cross-references, consistency of the 64KB→256KB updates across all affected files) all check out — just want the factual claims nailed down first.

JV0812 added 2 commits July 28, 2026 11:33
Adds known-limitation notes for Slack's message truncation and
Jira Cloud's 32,767-character issue description limit, called out
directly on each webhook connection page and cross-linked from the
central 256KB known limitations list, since larger messages make
both limits more likely to be hit.
Adds Slack's 40,000-character hard limit, splits the search-large-
messages known limitations into Sumo Logic platform vs. downstream
webhook connection limitations for clarity, previews both in the
intro, and tightens the release note with the same distinction.
Comment thread blog-service/2026-07-09-search.md Outdated
We're excited to announce that Sumo Logic now supports a maximum log message size of **256KB**, up from 64KB, so large single-line logs are ingested without being split as often.

**Sumo Logic platform limitations:**
* LogCompare, LogReduce, Cloud SIEM parsing, and Field Extraction Rules still process messages at 64KB.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this Field Extraction Rule limitation accurate? This makes it sound like FER will only work if the log is <64kb. I thought the limit was in the volume extracted?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FER can work with 256KB logs, it just cannot extract cumulative 64kb values for all fields

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — the bullet was misleading. Per lei-sumo's clarification (FER works with 256KB logs, it's the cumulative extracted-field size that's capped at 64KB, not the message itself), removed this summarized bullet entirely rather than try to compress it further. See 4a6dd93cb. — via Claude Code

These features process large messages differently within Sumo Logic itself:

- **LogCompare and LogReduce**. These operators truncate raw 256KB messages to 64KB before matching and grouping the logs into signatures, so content beyond 64KB is not considered. This can also affect response time when you run them against large messages. Learn more in [LogReduce](/docs/search/behavior-insights/logreduce/) and [LogCompare](/docs/search/behavior-insights/logcompare/).
- **Log Search messages table**. The messages table displays up to 64KB of a message, even after you expand it. To view a complete message larger than 64KB, use the [Log Message Inspector](/docs/search/get-started-with-search/search-page/log-message-inspector).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After this feature is released, the message table will display messages up to approximately 25 KB.

@JV0812, could you please update this?

Also, please add a note that messages larger than 25 KB containing JSON values as strings will not be rendered as JSON fields by default in the table. To view them as structured JSON, users will need to open the message in the Message Explorer.
YOu can read more on this thread https://sumologic.slack.com/archives/C09NL5MKJUS/p1784608962471479?thread_ts=1779369449.778359&cid=C09NL5MKJUS

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated — the messages table limit is now documented as 25,000 characters (GA behavior), and added a note that JSON values in messages beyond that size aren't rendered as JSON fields by default in the table; the Log Message Inspector shows the full message and structured JSON. See 4a6dd93cb. — via Claude Code

Comment thread blog-service/2026-07-09-search.md
**Downstream webhook connection limitations:**
* Slack (40,000 characters) and Jira Cloud (32,767 characters) enforce their own limits, which larger messages are more likely to reach.

[Learn more](/docs/search/get-started-with-search/search-basics/search-large-messages/).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from a discussion with @kevin-sumo, we suggest to skip the above limitation paragraphs, but just mention to find limitations using this link.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — removed both summarized limitation sections and replaced with a link to the Known limitations section of the main doc. See 4a6dd93cb. — via Claude Code

## Known limitations
The Jira Cloud issue description field has a hard limit of 32,767 characters, enforced at the database level. If a payload variable such as `{{ResultsJson}}` pushes the **Issue Description** past this limit, Jira Cloud returns a `400 Bad Request` error and the issue is not created. To avoid this, reference individual result fields (for example, `{{ResultsJson.fieldname}}`) instead of the full `{{ResultsJson}}` object, or otherwise keep the payload within the limit.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JV0812 We did a fix for this, the behavior is that Issue description will be truncated at 32KB and issue will still be created. Could you update this accordingly?

@lei-sumo lei-sumo Aug 6, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is also truncated with "…", similar as the slack webhook.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated — Known limitations now says the description is truncated with "..." and the issue is still created, matching the Slack webhook behavior, instead of a 400 Bad Request failure. See 4a6dd93cb. — via Claude Code

Alert notifications sent to these tools can also be affected by character limits that are fixed by the tool itself, independent of Sumo Logic's message size. Larger messages make these limits more likely to be reached:

- **Slack webhook connections**. Slack has a hard limit of 40,000 characters per message. Content beyond this limit, such as a large `{{ResultsJson}}` value, is truncated with "…" in the notification. Learn more in [Known limitations](/docs/alerts/webhook-connections/slack/#known-limitations).
- **Jira Cloud webhook connections**. The Jira Cloud issue description field has a hard limit of 32,767 characters. A payload variable such as `{{ResultsJson}}` that exceeds this limit returns a `400 Bad Request` error. Learn more in [Known limitations](/docs/alerts/webhook-connections/jira-cloud/#known-limitations).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to update this as well: issue can be created with a truncated payload.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated — this bullet now says the Jira Cloud payload is truncated with "..." and the issue is still created, instead of returning a 400 Bad Request. See 4a6dd93cb. — via Claude Code

@lei-sumo lei-sumo left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

during GNG meeting, we made another pass and left several comments.

…elease note

- Correct the Log Search messages table limit from 64KB to the GA value of
  25,000 characters, and note that JSON values in larger messages aren't
  rendered as JSON fields by default (per ssharma-sumo).
- Correct Jira Cloud webhook behavior: oversized descriptions are truncated
  with "..." and the issue is still created, not a 400 Bad Request failure
  (per lei-sumo).
- Drop the release note's summarized limitations bullets, which mischaracterized
  the Field Extraction Rule limit as a message-size cap rather than a
  cumulative-field-size cap (per kevin-sumo/lei-sumo), and link to the doc's
  Known limitations section instead.
These features process large messages differently within Sumo Logic itself:

- **LogCompare and LogReduce**. These operators truncate raw 256KB messages to 64KB before matching and grouping the logs into signatures, so content beyond 64KB is not considered. This can also affect response time when you run them against large messages. Learn more in [LogReduce](/docs/search/behavior-insights/logreduce/) and [LogCompare](/docs/search/behavior-insights/logcompare/).
- **Log Search messages table**. The messages table displays up to 25,000 characters of a message, even after you expand it. If a message larger than 25,000 characters contains JSON values as strings, those values are not rendered as JSON fields by default in the table. To view a complete message, or to view large JSON values as structured JSON, use the [Log Message Inspector](/docs/search/get-started-with-search/search-page/log-message-inspector).

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AyanGhatak @ssharma-sumo can you review and confirm the accuracy of this updated paragraph?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed Contributor approved, listed in .clabot file

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants