Many HTTP 304 responses results in fewer GET requests
I have a Django development server hosting a web-page that real-time (ish) displays information gathered from numerous servers I watch over. This web-page is still in development, so I am currently using the built-in web host provided with Django, started on an Ubuntu host with:
python3 manage.py runserver IP:Port
On the same ubuntu host there is a python script continuously reaching out to the monitored servers and formatting the responses into a .html file which the client reloads within a <div>
every minute. The general functionality of the page the client accesses is as follows:
<div id="status" style="width:100%; height: 1000"></div>
<script>
$('#status').load("% static 'alerts/status.html' %");
setInterval(function()
$('#status').load("% static 'alerts/status.html' %");
, 60000);
</script>
...so the page loads the status.html
file within the division on page-load, and then reloads it every minute. This has been working great, however, I have noticed looking at the Django log, that if status.html has not changed after ten status 304 (Not Modified) responses, the time waited between requests begins to roll-off. That is to say, instead of waiting 1 minute, it waits 2 minutes, then 5 minutes, and so on (roughly, I forget the actual rate of roll-off).
Now the issue I'm facing is that my server went down over the weekend (unrelated), but the display screen I had the web-page up on stayed active, so it rolled off so much that it seems it has completely broken, refusing to download the latest status.html, even when I force Chrome to reload everything and not use the cache (ctrl + R
or shift + F5
).
I tried researching this roll-off but couldn't find any information on it. I assume this is something built into Google Chrome (the browser I'm using) to save bandwidth when the page is not changing but my status page is a couple kilobytes at most and the 304 responses are already saving the little bandwidth that is so if there's a way to completely disable this roll-off for production that would be ideal.
In any case, any information on why I'm seeing this behavior / where it's coming from would be much appreciated as I can't seem to find any documentation on it. The closest thing I found was from Google's developer documentation on caching here. It mentions the ability to define maximum-age and no-cache behavior, so I could force the client to redownload status.html every minute, but this seems messy. While that would work in my specific scenario given status.html is a couple kilobytes at most, just disabling this roll-off behavior would do the trick and would keep unnecessary bandwidth down.
django google-chrome django-staticfiles http-caching
add a comment |
I have a Django development server hosting a web-page that real-time (ish) displays information gathered from numerous servers I watch over. This web-page is still in development, so I am currently using the built-in web host provided with Django, started on an Ubuntu host with:
python3 manage.py runserver IP:Port
On the same ubuntu host there is a python script continuously reaching out to the monitored servers and formatting the responses into a .html file which the client reloads within a <div>
every minute. The general functionality of the page the client accesses is as follows:
<div id="status" style="width:100%; height: 1000"></div>
<script>
$('#status').load("% static 'alerts/status.html' %");
setInterval(function()
$('#status').load("% static 'alerts/status.html' %");
, 60000);
</script>
...so the page loads the status.html
file within the division on page-load, and then reloads it every minute. This has been working great, however, I have noticed looking at the Django log, that if status.html has not changed after ten status 304 (Not Modified) responses, the time waited between requests begins to roll-off. That is to say, instead of waiting 1 minute, it waits 2 minutes, then 5 minutes, and so on (roughly, I forget the actual rate of roll-off).
Now the issue I'm facing is that my server went down over the weekend (unrelated), but the display screen I had the web-page up on stayed active, so it rolled off so much that it seems it has completely broken, refusing to download the latest status.html, even when I force Chrome to reload everything and not use the cache (ctrl + R
or shift + F5
).
I tried researching this roll-off but couldn't find any information on it. I assume this is something built into Google Chrome (the browser I'm using) to save bandwidth when the page is not changing but my status page is a couple kilobytes at most and the 304 responses are already saving the little bandwidth that is so if there's a way to completely disable this roll-off for production that would be ideal.
In any case, any information on why I'm seeing this behavior / where it's coming from would be much appreciated as I can't seem to find any documentation on it. The closest thing I found was from Google's developer documentation on caching here. It mentions the ability to define maximum-age and no-cache behavior, so I could force the client to redownload status.html every minute, but this seems messy. While that would work in my specific scenario given status.html is a couple kilobytes at most, just disabling this roll-off behavior would do the trick and would keep unnecessary bandwidth down.
django google-chrome django-staticfiles http-caching
add a comment |
I have a Django development server hosting a web-page that real-time (ish) displays information gathered from numerous servers I watch over. This web-page is still in development, so I am currently using the built-in web host provided with Django, started on an Ubuntu host with:
python3 manage.py runserver IP:Port
On the same ubuntu host there is a python script continuously reaching out to the monitored servers and formatting the responses into a .html file which the client reloads within a <div>
every minute. The general functionality of the page the client accesses is as follows:
<div id="status" style="width:100%; height: 1000"></div>
<script>
$('#status').load("% static 'alerts/status.html' %");
setInterval(function()
$('#status').load("% static 'alerts/status.html' %");
, 60000);
</script>
...so the page loads the status.html
file within the division on page-load, and then reloads it every minute. This has been working great, however, I have noticed looking at the Django log, that if status.html has not changed after ten status 304 (Not Modified) responses, the time waited between requests begins to roll-off. That is to say, instead of waiting 1 minute, it waits 2 minutes, then 5 minutes, and so on (roughly, I forget the actual rate of roll-off).
Now the issue I'm facing is that my server went down over the weekend (unrelated), but the display screen I had the web-page up on stayed active, so it rolled off so much that it seems it has completely broken, refusing to download the latest status.html, even when I force Chrome to reload everything and not use the cache (ctrl + R
or shift + F5
).
I tried researching this roll-off but couldn't find any information on it. I assume this is something built into Google Chrome (the browser I'm using) to save bandwidth when the page is not changing but my status page is a couple kilobytes at most and the 304 responses are already saving the little bandwidth that is so if there's a way to completely disable this roll-off for production that would be ideal.
In any case, any information on why I'm seeing this behavior / where it's coming from would be much appreciated as I can't seem to find any documentation on it. The closest thing I found was from Google's developer documentation on caching here. It mentions the ability to define maximum-age and no-cache behavior, so I could force the client to redownload status.html every minute, but this seems messy. While that would work in my specific scenario given status.html is a couple kilobytes at most, just disabling this roll-off behavior would do the trick and would keep unnecessary bandwidth down.
django google-chrome django-staticfiles http-caching
I have a Django development server hosting a web-page that real-time (ish) displays information gathered from numerous servers I watch over. This web-page is still in development, so I am currently using the built-in web host provided with Django, started on an Ubuntu host with:
python3 manage.py runserver IP:Port
On the same ubuntu host there is a python script continuously reaching out to the monitored servers and formatting the responses into a .html file which the client reloads within a <div>
every minute. The general functionality of the page the client accesses is as follows:
<div id="status" style="width:100%; height: 1000"></div>
<script>
$('#status').load("% static 'alerts/status.html' %");
setInterval(function()
$('#status').load("% static 'alerts/status.html' %");
, 60000);
</script>
...so the page loads the status.html
file within the division on page-load, and then reloads it every minute. This has been working great, however, I have noticed looking at the Django log, that if status.html has not changed after ten status 304 (Not Modified) responses, the time waited between requests begins to roll-off. That is to say, instead of waiting 1 minute, it waits 2 minutes, then 5 minutes, and so on (roughly, I forget the actual rate of roll-off).
Now the issue I'm facing is that my server went down over the weekend (unrelated), but the display screen I had the web-page up on stayed active, so it rolled off so much that it seems it has completely broken, refusing to download the latest status.html, even when I force Chrome to reload everything and not use the cache (ctrl + R
or shift + F5
).
I tried researching this roll-off but couldn't find any information on it. I assume this is something built into Google Chrome (the browser I'm using) to save bandwidth when the page is not changing but my status page is a couple kilobytes at most and the 304 responses are already saving the little bandwidth that is so if there's a way to completely disable this roll-off for production that would be ideal.
In any case, any information on why I'm seeing this behavior / where it's coming from would be much appreciated as I can't seem to find any documentation on it. The closest thing I found was from Google's developer documentation on caching here. It mentions the ability to define maximum-age and no-cache behavior, so I could force the client to redownload status.html every minute, but this seems messy. While that would work in my specific scenario given status.html is a couple kilobytes at most, just disabling this roll-off behavior would do the trick and would keep unnecessary bandwidth down.
django google-chrome django-staticfiles http-caching
django google-chrome django-staticfiles http-caching
edited Nov 13 at 6:19
Kevin Christopher Henry
22.5k46361
22.5k46361
asked Nov 12 at 20:19
Eric
446
446
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
The problem here is that the response for status.html
doesn't have an explicit cache expiration header. In the absence of such a header, the browser is free to use its own algorithm (such as the roll-off you're seeing) to choose an expiration time. From RFC 7234:
Since origin servers do not always provide explicit expiration times, a cache MAY assign a heuristic expiration time when an explicit time is not specified.... This specification does not provide specific algorithms.
So the solution is straightforward: assign an explicit cache expiration time.
Implementing this solution is unfortunately not trivial using Django's staticfiles app. A better default for this app would be to not cache the results at all, but that solution was deferred pending a merger with whitenoise.
Solutions include using a different server (like nginx); using a different app (like whitenoise); or using static views directly rather than the staticfiles app (see this question for a few approaches).
Thanks for the complete explanation. I decided, instead of fighting with Django and risking messing up my other views, to just disable the cache completely there with$.ajaxSetup (cache: false);
in the<script>
in my question. My view is only a few kilobytes pulled once a minute so it's not worth the struggle to save bandwidth on. If I (or someone else) has this issue with a higher-bandwidth scenario the suggested solutions are very helpful though so I'm accepting this as the answer.
– Eric
Nov 13 at 18:28
add a comment |
Try this:
// Page reload every 60 seconds
setInterval(function()
location.reload();
, 60000);
I am having difficulties getting this to use the cache. I have reduced the update time to 2s for testing and added 'Expires' = '10' as a header to the django view for this page - I expected this to result in 5 status 304s before another status 200 but instead it is downloading status.html every time. Do you know what else I'm missing? I have also tried usingreload(false)
to be sure it shouldn't use cache.
– Eric
Nov 12 at 23:23
Also - ideally I would only have to re-download status.html, this as-is downloads the whole page each time (so .css and .js files that aren't changing are now being re-downloaded when they don't have to). Is there any way to restrict this to the<div>
in question?
– Eric
Nov 12 at 23:25
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53269502%2fmany-http-304-responses-results-in-fewer-get-requests%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
The problem here is that the response for status.html
doesn't have an explicit cache expiration header. In the absence of such a header, the browser is free to use its own algorithm (such as the roll-off you're seeing) to choose an expiration time. From RFC 7234:
Since origin servers do not always provide explicit expiration times, a cache MAY assign a heuristic expiration time when an explicit time is not specified.... This specification does not provide specific algorithms.
So the solution is straightforward: assign an explicit cache expiration time.
Implementing this solution is unfortunately not trivial using Django's staticfiles app. A better default for this app would be to not cache the results at all, but that solution was deferred pending a merger with whitenoise.
Solutions include using a different server (like nginx); using a different app (like whitenoise); or using static views directly rather than the staticfiles app (see this question for a few approaches).
Thanks for the complete explanation. I decided, instead of fighting with Django and risking messing up my other views, to just disable the cache completely there with$.ajaxSetup (cache: false);
in the<script>
in my question. My view is only a few kilobytes pulled once a minute so it's not worth the struggle to save bandwidth on. If I (or someone else) has this issue with a higher-bandwidth scenario the suggested solutions are very helpful though so I'm accepting this as the answer.
– Eric
Nov 13 at 18:28
add a comment |
The problem here is that the response for status.html
doesn't have an explicit cache expiration header. In the absence of such a header, the browser is free to use its own algorithm (such as the roll-off you're seeing) to choose an expiration time. From RFC 7234:
Since origin servers do not always provide explicit expiration times, a cache MAY assign a heuristic expiration time when an explicit time is not specified.... This specification does not provide specific algorithms.
So the solution is straightforward: assign an explicit cache expiration time.
Implementing this solution is unfortunately not trivial using Django's staticfiles app. A better default for this app would be to not cache the results at all, but that solution was deferred pending a merger with whitenoise.
Solutions include using a different server (like nginx); using a different app (like whitenoise); or using static views directly rather than the staticfiles app (see this question for a few approaches).
Thanks for the complete explanation. I decided, instead of fighting with Django and risking messing up my other views, to just disable the cache completely there with$.ajaxSetup (cache: false);
in the<script>
in my question. My view is only a few kilobytes pulled once a minute so it's not worth the struggle to save bandwidth on. If I (or someone else) has this issue with a higher-bandwidth scenario the suggested solutions are very helpful though so I'm accepting this as the answer.
– Eric
Nov 13 at 18:28
add a comment |
The problem here is that the response for status.html
doesn't have an explicit cache expiration header. In the absence of such a header, the browser is free to use its own algorithm (such as the roll-off you're seeing) to choose an expiration time. From RFC 7234:
Since origin servers do not always provide explicit expiration times, a cache MAY assign a heuristic expiration time when an explicit time is not specified.... This specification does not provide specific algorithms.
So the solution is straightforward: assign an explicit cache expiration time.
Implementing this solution is unfortunately not trivial using Django's staticfiles app. A better default for this app would be to not cache the results at all, but that solution was deferred pending a merger with whitenoise.
Solutions include using a different server (like nginx); using a different app (like whitenoise); or using static views directly rather than the staticfiles app (see this question for a few approaches).
The problem here is that the response for status.html
doesn't have an explicit cache expiration header. In the absence of such a header, the browser is free to use its own algorithm (such as the roll-off you're seeing) to choose an expiration time. From RFC 7234:
Since origin servers do not always provide explicit expiration times, a cache MAY assign a heuristic expiration time when an explicit time is not specified.... This specification does not provide specific algorithms.
So the solution is straightforward: assign an explicit cache expiration time.
Implementing this solution is unfortunately not trivial using Django's staticfiles app. A better default for this app would be to not cache the results at all, but that solution was deferred pending a merger with whitenoise.
Solutions include using a different server (like nginx); using a different app (like whitenoise); or using static views directly rather than the staticfiles app (see this question for a few approaches).
answered Nov 13 at 6:13
Kevin Christopher Henry
22.5k46361
22.5k46361
Thanks for the complete explanation. I decided, instead of fighting with Django and risking messing up my other views, to just disable the cache completely there with$.ajaxSetup (cache: false);
in the<script>
in my question. My view is only a few kilobytes pulled once a minute so it's not worth the struggle to save bandwidth on. If I (or someone else) has this issue with a higher-bandwidth scenario the suggested solutions are very helpful though so I'm accepting this as the answer.
– Eric
Nov 13 at 18:28
add a comment |
Thanks for the complete explanation. I decided, instead of fighting with Django and risking messing up my other views, to just disable the cache completely there with$.ajaxSetup (cache: false);
in the<script>
in my question. My view is only a few kilobytes pulled once a minute so it's not worth the struggle to save bandwidth on. If I (or someone else) has this issue with a higher-bandwidth scenario the suggested solutions are very helpful though so I'm accepting this as the answer.
– Eric
Nov 13 at 18:28
Thanks for the complete explanation. I decided, instead of fighting with Django and risking messing up my other views, to just disable the cache completely there with
$.ajaxSetup (cache: false);
in the <script>
in my question. My view is only a few kilobytes pulled once a minute so it's not worth the struggle to save bandwidth on. If I (or someone else) has this issue with a higher-bandwidth scenario the suggested solutions are very helpful though so I'm accepting this as the answer.– Eric
Nov 13 at 18:28
Thanks for the complete explanation. I decided, instead of fighting with Django and risking messing up my other views, to just disable the cache completely there with
$.ajaxSetup (cache: false);
in the <script>
in my question. My view is only a few kilobytes pulled once a minute so it's not worth the struggle to save bandwidth on. If I (or someone else) has this issue with a higher-bandwidth scenario the suggested solutions are very helpful though so I'm accepting this as the answer.– Eric
Nov 13 at 18:28
add a comment |
Try this:
// Page reload every 60 seconds
setInterval(function()
location.reload();
, 60000);
I am having difficulties getting this to use the cache. I have reduced the update time to 2s for testing and added 'Expires' = '10' as a header to the django view for this page - I expected this to result in 5 status 304s before another status 200 but instead it is downloading status.html every time. Do you know what else I'm missing? I have also tried usingreload(false)
to be sure it shouldn't use cache.
– Eric
Nov 12 at 23:23
Also - ideally I would only have to re-download status.html, this as-is downloads the whole page each time (so .css and .js files that aren't changing are now being re-downloaded when they don't have to). Is there any way to restrict this to the<div>
in question?
– Eric
Nov 12 at 23:25
add a comment |
Try this:
// Page reload every 60 seconds
setInterval(function()
location.reload();
, 60000);
I am having difficulties getting this to use the cache. I have reduced the update time to 2s for testing and added 'Expires' = '10' as a header to the django view for this page - I expected this to result in 5 status 304s before another status 200 but instead it is downloading status.html every time. Do you know what else I'm missing? I have also tried usingreload(false)
to be sure it shouldn't use cache.
– Eric
Nov 12 at 23:23
Also - ideally I would only have to re-download status.html, this as-is downloads the whole page each time (so .css and .js files that aren't changing are now being re-downloaded when they don't have to). Is there any way to restrict this to the<div>
in question?
– Eric
Nov 12 at 23:25
add a comment |
Try this:
// Page reload every 60 seconds
setInterval(function()
location.reload();
, 60000);
Try this:
// Page reload every 60 seconds
setInterval(function()
location.reload();
, 60000);
answered Nov 12 at 20:26
Scott Skiles
5431723
5431723
I am having difficulties getting this to use the cache. I have reduced the update time to 2s for testing and added 'Expires' = '10' as a header to the django view for this page - I expected this to result in 5 status 304s before another status 200 but instead it is downloading status.html every time. Do you know what else I'm missing? I have also tried usingreload(false)
to be sure it shouldn't use cache.
– Eric
Nov 12 at 23:23
Also - ideally I would only have to re-download status.html, this as-is downloads the whole page each time (so .css and .js files that aren't changing are now being re-downloaded when they don't have to). Is there any way to restrict this to the<div>
in question?
– Eric
Nov 12 at 23:25
add a comment |
I am having difficulties getting this to use the cache. I have reduced the update time to 2s for testing and added 'Expires' = '10' as a header to the django view for this page - I expected this to result in 5 status 304s before another status 200 but instead it is downloading status.html every time. Do you know what else I'm missing? I have also tried usingreload(false)
to be sure it shouldn't use cache.
– Eric
Nov 12 at 23:23
Also - ideally I would only have to re-download status.html, this as-is downloads the whole page each time (so .css and .js files that aren't changing are now being re-downloaded when they don't have to). Is there any way to restrict this to the<div>
in question?
– Eric
Nov 12 at 23:25
I am having difficulties getting this to use the cache. I have reduced the update time to 2s for testing and added 'Expires' = '10' as a header to the django view for this page - I expected this to result in 5 status 304s before another status 200 but instead it is downloading status.html every time. Do you know what else I'm missing? I have also tried using
reload(false)
to be sure it shouldn't use cache.– Eric
Nov 12 at 23:23
I am having difficulties getting this to use the cache. I have reduced the update time to 2s for testing and added 'Expires' = '10' as a header to the django view for this page - I expected this to result in 5 status 304s before another status 200 but instead it is downloading status.html every time. Do you know what else I'm missing? I have also tried using
reload(false)
to be sure it shouldn't use cache.– Eric
Nov 12 at 23:23
Also - ideally I would only have to re-download status.html, this as-is downloads the whole page each time (so .css and .js files that aren't changing are now being re-downloaded when they don't have to). Is there any way to restrict this to the
<div>
in question?– Eric
Nov 12 at 23:25
Also - ideally I would only have to re-download status.html, this as-is downloads the whole page each time (so .css and .js files that aren't changing are now being re-downloaded when they don't have to). Is there any way to restrict this to the
<div>
in question?– Eric
Nov 12 at 23:25
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53269502%2fmany-http-304-responses-results-in-fewer-get-requests%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown