log4j ... Dynamic Log File Name

Background:
We developed a Bulk Loader component using core Java api's and used log4j for writing all the developer debugs, info, warn etc messages to a log file. The Bulk Loader, as the name suggests, sent bulk requests to application which enabled Load Testing. The Bulk Loader was developed in such a way that multiple instances could run on the same machine : this enabled controlling the load pushed to the application from various Bulk Loader instances.


Problem scenario:
During testing, if was found that all the Bulk Loader instances running on one machine, used the same log file for writing their messages to. This made the log file illegible. Messages from different instances were interwoven into a single log file which made debugging a very tough job. A solution was needed for each instance to create and write to its own log file.

Method 1: Modify the java code for setting the correct log file name to the FileAppender

1. Get hold of the all Appenders  from the Logger object
2. Loop through each and find the Appender of interest. This will typically be the FileAppender
3. Call the setFile( String fileName ) api for setting desired log file name
4. Call the activateOptions for Log4j framework to pick the newly set log file name

Though most of us would like and prefer to use the above method, there are two stark limitations :

1. There can be multiple File Appenders defined in the log4j.xml. For example, my log4j.xml has two RollingFileAppenders; one with name as Process the other with Audit. In such cases, the logic of setting the log file name dynamically starts getting complex if lot of conditional checks
2. If the source code of api responsible for initializing the Logger is not available (this could be due to the fact that you, as a part of Services team, have the ability to invoke an exposed log( String message, int level ) method and initialization and configuration of the Logger is taken care by the Product team), then you definitely cannot get hold to any of the Appenders and change the log file name.

Method 2: Modify the log4j.xml and Use -D switch
This method does not require any code changes and hence no need to recompile. This method is only related to configuration of log4j.xml and the start command of java program.

• Open the log4j.xml and introduce a variable in front of the log file name definition. The variable name should be wrapped with special characters as follows: ${variable name}
• Following is the snippet from my log4j.xml

<appender name="Process" class="org.apache.log4j.DailyRollingFileAppender">              <param name="File" value="./logs/${log_file_name}_Log.txt"/>          <appender name="AuditAppender" class="org.apache.log4j.RollingFileAppender">          <param name="File" value="./logs/${log_file_name}_Audit_log.txt"/>

• Modify the start command of java application as below:

java -Dlog_file_name=Items -cp %CLASSPATH% HelloWorld

• This will create two log files by the name Items_Log.txt and Items_Audit.txt in the logs folder

Hope this post was helpful. Happy Logging !