how to understand arrival rate about apache storm disruptor queue
up vote
0
down vote
favorite
About storm metric. I do not understand the relationship between send queue arrival rate and receive queue arrival rate.
For example, when open ACK, if a spout receive one tuple , and it emit one tuple. whether the RQ arrival rate : SQ arrival rate = 1:2?
Besides, if system not stable. this Equation may be change?
apache-storm
edited Nov 10 at 14:27
Saurabh
27.3k1491154
asked Apr 24 '17 at 8:30
wendyhusband
12
add a comment |
up vote
0
down vote
favorite
About storm metric. I do not understand the relationship between send queue arrival rate and receive queue arrival rate.
For example, when open ACK, if a spout receive one tuple , and it emit one tuple. whether the RQ arrival rate : SQ arrival rate = 1:2?
Besides, if system not stable. this Equation may be change?
apache-storm
edited Nov 10 at 14:27
Saurabh
27.3k1491154
asked Apr 24 '17 at 8:30
wendyhusband
12
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
About storm metric. I do not understand the relationship between send queue arrival rate and receive queue arrival rate.
For example, when open ACK, if a spout receive one tuple , and it emit one tuple. whether the RQ arrival rate : SQ arrival rate = 1:2?
Besides, if system not stable. this Equation may be change?
apache-storm
edited Nov 10 at 14:27
Saurabh
27.3k1491154
asked Apr 24 '17 at 8:30
wendyhusband
12
About storm metric. I do not understand the relationship between send queue arrival rate and receive queue arrival rate.
For example, when open ACK, if a spout receive one tuple , and it emit one tuple. whether the RQ arrival rate : SQ arrival rate = 1:2?
Besides, if system not stable. this Equation may be change?
apache-storm
apache-storm
edited Nov 10 at 14:27
Saurabh
27.3k1491154
asked Apr 24 '17 at 8:30
wendyhusband
12
edited Nov 10 at 14:27
Saurabh
27.3k1491154
asked Apr 24 '17 at 8:30
wendyhusband
12
edited Nov 10 at 14:27
Saurabh
27.3k1491154
edited Nov 10 at 14:27
Saurabh
27.3k1491154
edited Nov 10 at 14:27
Saurabh
27.3k1491154
27.3k1491154
asked Apr 24 '17 at 8:30
wendyhusband
12
asked Apr 24 '17 at 8:30
wendyhusband
12
asked Apr 24 '17 at 8:30
wendyhusband
12
12
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
Spout instances in Storm do not have a receive queue (only a send queue)? I assume you are referring to bolts?
Although it is a little old this article by Michael Noll gives a good overview of the internal queues within the workers.
To answer your question. The ratio between the queues will not always be 2:1. The disruptor queues report their metrics averaged over the user configurable topology.builtin.metrics.bucket.size.secs so this will obscure some of the difference. Also all metrics are subject to a sample ratio, set by the topology.stats.sample.rate config variable - which by default is only 20% of transferred tuples, this can also cause the reported numbers to be off.
Also, depending on the code in your bolts, 1 input tuple may produce many output tuples so you would have to take this into account in any ratios you were calculating.
You refer to the stability of an equation in your question. The arrival rate is not based on any queuing theory equation and is simply the number of tuples that are put on the queue in a metric.bucket period divided by the period length in seconds. However, Storm does report a queue sojourn time metric. This is based on a very simple queuing theory equation that is not reliable for unstable queue systems and should be avoided.
answered May 8 '17 at 20:26
Tom Cooper
18129
thanks a lot. sorry for lack of timely reply. Spout instances in Storm have a receive queue for receive the tuple form acker instances. therefore, the number of tuple that spout received should be equal to it send.
– wendyhusband
Jul 11 '17 at 14:32
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
Spout instances in Storm do not have a receive queue (only a send queue)? I assume you are referring to bolts?
Although it is a little old this article by Michael Noll gives a good overview of the internal queues within the workers.
To answer your question. The ratio between the queues will not always be 2:1. The disruptor queues report their metrics averaged over the user configurable topology.builtin.metrics.bucket.size.secs so this will obscure some of the difference. Also all metrics are subject to a sample ratio, set by the topology.stats.sample.rate config variable - which by default is only 20% of transferred tuples, this can also cause the reported numbers to be off.
Also, depending on the code in your bolts, 1 input tuple may produce many output tuples so you would have to take this into account in any ratios you were calculating.
You refer to the stability of an equation in your question. The arrival rate is not based on any queuing theory equation and is simply the number of tuples that are put on the queue in a metric.bucket period divided by the period length in seconds. However, Storm does report a queue sojourn time metric. This is based on a very simple queuing theory equation that is not reliable for unstable queue systems and should be avoided.
answered May 8 '17 at 20:26
Tom Cooper
18129
thanks a lot. sorry for lack of timely reply. Spout instances in Storm have a receive queue for receive the tuple form acker instances. therefore, the number of tuple that spout received should be equal to it send.
– wendyhusband
Jul 11 '17 at 14:32
add a comment |
up vote
0
down vote
Spout instances in Storm do not have a receive queue (only a send queue)? I assume you are referring to bolts?
Although it is a little old this article by Michael Noll gives a good overview of the internal queues within the workers.
To answer your question. The ratio between the queues will not always be 2:1. The disruptor queues report their metrics averaged over the user configurable topology.builtin.metrics.bucket.size.secs so this will obscure some of the difference. Also all metrics are subject to a sample ratio, set by the topology.stats.sample.rate config variable - which by default is only 20% of transferred tuples, this can also cause the reported numbers to be off.
Also, depending on the code in your bolts, 1 input tuple may produce many output tuples so you would have to take this into account in any ratios you were calculating.
You refer to the stability of an equation in your question. The arrival rate is not based on any queuing theory equation and is simply the number of tuples that are put on the queue in a metric.bucket period divided by the period length in seconds. However, Storm does report a queue sojourn time metric. This is based on a very simple queuing theory equation that is not reliable for unstable queue systems and should be avoided.
answered May 8 '17 at 20:26
Tom Cooper
18129
thanks a lot. sorry for lack of timely reply. Spout instances in Storm have a receive queue for receive the tuple form acker instances. therefore, the number of tuple that spout received should be equal to it send.
– wendyhusband
Jul 11 '17 at 14:32
add a comment |
up vote
0
down vote
up vote
0
down vote
Spout instances in Storm do not have a receive queue (only a send queue)? I assume you are referring to bolts?
Although it is a little old this article by Michael Noll gives a good overview of the internal queues within the workers.
To answer your question. The ratio between the queues will not always be 2:1. The disruptor queues report their metrics averaged over the user configurable topology.builtin.metrics.bucket.size.secs so this will obscure some of the difference. Also all metrics are subject to a sample ratio, set by the topology.stats.sample.rate config variable - which by default is only 20% of transferred tuples, this can also cause the reported numbers to be off.
Also, depending on the code in your bolts, 1 input tuple may produce many output tuples so you would have to take this into account in any ratios you were calculating.
You refer to the stability of an equation in your question. The arrival rate is not based on any queuing theory equation and is simply the number of tuples that are put on the queue in a metric.bucket period divided by the period length in seconds. However, Storm does report a queue sojourn time metric. This is based on a very simple queuing theory equation that is not reliable for unstable queue systems and should be avoided.
answered May 8 '17 at 20:26
Tom Cooper
18129
Spout instances in Storm do not have a receive queue (only a send queue)? I assume you are referring to bolts?
Although it is a little old this article by Michael Noll gives a good overview of the internal queues within the workers.
To answer your question. The ratio between the queues will not always be 2:1. The disruptor queues report their metrics averaged over the user configurable topology.builtin.metrics.bucket.size.secs so this will obscure some of the difference. Also all metrics are subject to a sample ratio, set by the topology.stats.sample.rate config variable - which by default is only 20% of transferred tuples, this can also cause the reported numbers to be off.
Also, depending on the code in your bolts, 1 input tuple may produce many output tuples so you would have to take this into account in any ratios you were calculating.
You refer to the stability of an equation in your question. The arrival rate is not based on any queuing theory equation and is simply the number of tuples that are put on the queue in a metric.bucket period divided by the period length in seconds. However, Storm does report a queue sojourn time metric. This is based on a very simple queuing theory equation that is not reliable for unstable queue systems and should be avoided.
answered May 8 '17 at 20:26
Tom Cooper
18129
answered May 8 '17 at 20:26
Tom Cooper
18129
answered May 8 '17 at 20:26
Tom Cooper
18129
answered May 8 '17 at 20:26
Tom Cooper
18129
18129
thanks a lot. sorry for lack of timely reply. Spout instances in Storm have a receive queue for receive the tuple form acker instances. therefore, the number of tuple that spout received should be equal to it send.
– wendyhusband
Jul 11 '17 at 14:32
add a comment |
thanks a lot. sorry for lack of timely reply. Spout instances in Storm have a receive queue for receive the tuple form acker instances. therefore, the number of tuple that spout received should be equal to it send.
– wendyhusband
Jul 11 '17 at 14:32
thanks a lot. sorry for lack of timely reply. Spout instances in Storm have a receive queue for receive the tuple form acker instances. therefore, the number of tuple that spout received should be equal to it send.
– wendyhusband
Jul 11 '17 at 14:32
thanks a lot. sorry for lack of timely reply. Spout instances in Storm have a receive queue for receive the tuple form acker instances. therefore, the number of tuple that spout received should be equal to it send.
– wendyhusband
Jul 11 '17 at 14:32
add a comment |
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f43583249%2fhow-to-understand-arrival-rate-about-apache-storm-disruptor-queue%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
