Limit log event length in Log4J2 JSON Layout to prevent hitting docker log driver limit
Currently the docker logging driver has a limit of 16KB, which is unlikely to change.
We would like to use Log4J2's JSON layout in our applications, because this layout allows us to build very targeted Dashboards in our centralized Splunk-Log-Database.
However, in edge-cases the stacktrace-representation in json can produce very large Log-events exceeding the docker-line-limit of 16KB. In this case the message is corrupted by the docker logging driver. The log line arriving in Splunk looks as follows (Note that the docker json file logger embeds our log event into its own json stucture):
"log":"16 KB of log event here, truncated at end","stream":"stdout","time":"2018-11-15T15:33:56.939598109Z"remaining part of log event appended here
I would like to configure Log4J2 to limit the log event size to 16KB in all circumstances, but it seems not supported (no such parameter).
How can we prevent receiving corrupted log-events in Splunk?
docker log4j2
asked Nov 16 '18 at 9:55
fabfab
91811326
add a comment |
Currently the docker logging driver has a limit of 16KB, which is unlikely to change.
We would like to use Log4J2's JSON layout in our applications, because this layout allows us to build very targeted Dashboards in our centralized Splunk-Log-Database.
However, in edge-cases the stacktrace-representation in json can produce very large Log-events exceeding the docker-line-limit of 16KB. In this case the message is corrupted by the docker logging driver. The log line arriving in Splunk looks as follows (Note that the docker json file logger embeds our log event into its own json stucture):
"log":"16 KB of log event here, truncated at end","stream":"stdout","time":"2018-11-15T15:33:56.939598109Z"remaining part of log event appended here
I would like to configure Log4J2 to limit the log event size to 16KB in all circumstances, but it seems not supported (no such parameter).
How can we prevent receiving corrupted log-events in Splunk?
docker log4j2
asked Nov 16 '18 at 9:55
fabfab
91811326
add a comment |
Currently the docker logging driver has a limit of 16KB, which is unlikely to change.
We would like to use Log4J2's JSON layout in our applications, because this layout allows us to build very targeted Dashboards in our centralized Splunk-Log-Database.
However, in edge-cases the stacktrace-representation in json can produce very large Log-events exceeding the docker-line-limit of 16KB. In this case the message is corrupted by the docker logging driver. The log line arriving in Splunk looks as follows (Note that the docker json file logger embeds our log event into its own json stucture):
"log":"16 KB of log event here, truncated at end","stream":"stdout","time":"2018-11-15T15:33:56.939598109Z"remaining part of log event appended here
I would like to configure Log4J2 to limit the log event size to 16KB in all circumstances, but it seems not supported (no such parameter).
How can we prevent receiving corrupted log-events in Splunk?
docker log4j2
asked Nov 16 '18 at 9:55
fabfab
91811326
Currently the docker logging driver has a limit of 16KB, which is unlikely to change.
We would like to use Log4J2's JSON layout in our applications, because this layout allows us to build very targeted Dashboards in our centralized Splunk-Log-Database.
However, in edge-cases the stacktrace-representation in json can produce very large Log-events exceeding the docker-line-limit of 16KB. In this case the message is corrupted by the docker logging driver. The log line arriving in Splunk looks as follows (Note that the docker json file logger embeds our log event into its own json stucture):
"log":"16 KB of log event here, truncated at end","stream":"stdout","time":"2018-11-15T15:33:56.939598109Z"remaining part of log event appended here
I would like to configure Log4J2 to limit the log event size to 16KB in all circumstances, but it seems not supported (no such parameter).
How can we prevent receiving corrupted log-events in Splunk?
docker log4j2
docker log4j2
asked Nov 16 '18 at 9:55
fabfab
91811326
asked Nov 16 '18 at 9:55
fabfab
91811326
asked Nov 16 '18 at 9:55
fabfab
91811326
asked Nov 16 '18 at 9:55
fabfab
91811326
asked Nov 16 '18 at 9:55
fabfab
91811326
91811326
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53335345%2flimit-log-event-length-in-log4j2-json-layout-to-prevent-hitting-docker-log-drive%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53335345%2flimit-log-event-length-in-log4j2-json-layout-to-prevent-hitting-docker-log-drive%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
