We started the usual things like upgrading the core components and making more logs that the Tomcat can output. After all these we got rows in the localhost log saying “java.lang.OutOfMemoryError: Java heap space” but with no other information like rows of code, etc. We continued to optimize the memory usage by flushing the input and output streams in the code and others. After that we received this log:
SEVERE: Servlet.service() for servlet jsp threw exception
java.lang.OutOfMemoryError: Java heap space
This was the first time when we had a log where there was a specific row number indicating the error. After looking at the code, we found that the row was in logic, where we assign a field from the database to a string variable. It was nothing unusual, so we decided to check the field in the database table. It was a “text” field in a PostgreSQL database that we access using JDBC driver. After a SELECT length(‘field’) command, we estimated that the biggest field data is about 1 MB! So, if we wanted to take the whole table that would be about 400 MB data fetched in a single click that invoked this method. After that we were able to finally reproduce the server crash, clicking several times on the button that invoked the method fetching this abnormally large amount of data. We had to revise the code so that we take only one field when needed. This solved the crashing issue.
The lesson is that we need to make sure we are not requesting too much data using the SQL statements. Even if we are getting a small amount of columns like in this case, we need to have in mind that one of them could contain a lot of information and make the system crash – especially with lots of concurrent users.