Myujth

I am having issues with concurrency when writing JSON out from my Spring Boot WAR app deployed to Tomcat 8. In the screenshot from AppDynamics there seems to be a considerable wait when the jackson library is performing _flushBuffer.

This issue arises under load testing for even a small amount (< 10) users.

I have configured the messageConverters in my configuration class.

 @Override
 public void configureMessageConverters(List<HttpMessageConverter<?>> converters) 
 converters.add(new MappingJackson2HttpMessageConverter(
 new Jackson2ObjectMapperBuilder().dateFormat(new SimpleDateFormat("yyyy-MM-dd'T'HH:mmZ")).mixIn(LiquidAssignment.class,
 InventoryProviderAssignmentMixin.class)
 .deserializerByType(ActionData.class, new ActionDataDeserializer()).build()));
 converters.add(new MappingJackson2XmlHttpMessageConverter());
 

I am using
Spring Boot 1.5.4
Java 1.8
Jackson 2.9.7
Tomcat 8.5.33

Looking at the source code of UTF8JsonGenerator._flushBuffer(), there is no indication of LockSupport.parkNanos(). So it has probably been inlined by the JIT compiler from OutputStream.write().

My guess is it's the place where – for your application – Tomcat typically waits until the client has accepted all the output (expect for the last piece that fits into the typical connection buffer size) before it can close the connection.

We have had bad experience with slow clients in the past. Until they have retrieved all the output, they block a thread in Tomcat. And blocking a few dozens threads in Tomcat seriously reduces the throughput of a busy web app.

Increasing the number of threads isn't the best option as the blocked threads also occupy a considerable amount of memory. So what you want is that Tomcat can handle a request as quickly as possible and then move on to the next request.

We have solved the problem by configuring our reverse proxy, which we always had in front of Tomcat, to immediately consume all output from Tomcat and deliver it to the client at the client's speed. The reverse proxy is very efficient at handling slow clients.

In our case, we have used nginx. We also looked at Apache httpd. But at the time, it wasn't capable of doing it.

Additional Note

Clients that unexpectedly disconnect also look like slow clients to the server as it takes some time until it has been fully established that the connection is broken.