Contact Us 1-800-596-4880
  1. Homepage
  2. Mule Runtime (4.8)
  3. Develop Mule Applications
  4. Configure Logging and Debug Applications

  5. Configure Logging

Configuring Logging

MuleSoft provides application logs and a runtime log to help you monitor and troubleshoot your Mule applications and the Mule instance.

App Logs

Every app that you build in Studio comes with its own log4j2.xml file. The log contains information about any errors raised in the app (unless you have app logic to handle those errors). It also contains anything you want to explicitly log, if you build the logic in the app.

Mule automatically logs multiple messages and specific elements in your app flows to help you debug and keep track of events. You can also include the Logger component anywhere in a flow and set it up to output any message you want.

You can view an app log as follows:

  • If you’re running an app from Anypoint Studio, the output from the app log is visible in Anypoint Studio’s console window.

  • If you’re using Mule from the command line to run an app, the app log is visible in your OS console.

By default, Mule stores application log files in the following location: MULE_HOME/logs/<app-name>.log. You can customize the log file path in the log4j2.xml file of the Mule app.

Runtime Log

The runtime log (mule_ee.log) contains information about the apps and lifecycle events. For example, the log records an entry when a Mule service or app starts, deploys, stops, or undeploys.

The runtime log configuration is located in the log4j2.xml file, in the /conf directory. You can customize this file when running Mule on-premises.

View the Runtime Log in Anypoint Studio:

Click Anypoint Studio > About Anypoint Studio > Installation Details > Configuration > View Error Log:

An Anypoint Studio error log viewer interface

Change the Log Level

You can change the runtime log level by modifying the level value of the following packages in the log4j2.xml file:

  <AsyncLogger name="org.mule" level="INFO"/>
  <AsyncLogger name="com.mulesoft" level="INFO"/>

Accepted values are: DEBUG, ERROR, INFO, TRACE, and WARN.

To change the log level only for specific Connectors, see Enable Verbose Logging.

Configure Logging

You can create a configuration file to define:

  • What kinds of messages to log

  • How to log the messages (asynchronously or synchronously)

  • Where to log the messages, such as to the console or disk, or to an endpoint or database

Mule uses slf4j, which is a logging facade that discovers and uses a logging strategy from the classpath, such as Apache Log4j 2 or the JDK Logger. By default, Mule includes Log4j 2, which is configured with a file called log4j2.xml.

Synchronous Versus Asynchronous Logging

By default, Mule logs messages asynchronously. When logging synchronously, the execution of the thread that is processing your message is interrupted to wait for the log message to be fully handled before it can continue:

A synchronous logging in a threaded environment

When logging asynchronously, the logging operation occurs in a separate thread, so the message can process before the logging completes:

An asynchronous logging in a threaded environment

In most situations, MuleSoft recommends that you use asynchronous logging because it substantially improves the throughput and lowers the latency of message processing.

When asynchronous logging is used, some actions might not be logged if there is a system crash. This situation occurs because log writing is performed on a separate thread that runs independently of other actions. See Exception Handling with Asynchronous Logging for information on mitigating this issue.

If you use app logs as audit trails, configure your app to always use synchronous logging. This prevents the loss of any log messages.

Mule 4 introduced thread-local functionality to optimize the memory usage of log4j. To enhance the use of this feature, you can set the system property AsyncLoggerConfig.RingBufferSize (see Configuring Properties). 20000 is the suggested value, but you need to tune it to meet your needs.

Performance tests

The chart below shows the performance difference between synchronous and asynchronous logging, and how much latency increased as more concurrent messages were added. In this test, an app logged about one million messages, using an increasingly higher amount of threads on each run. Each transaction resulted in 1000 messages.

A line chart comparing latency versus concurrency

As indicated by the chart, the results of logging asynchronously are significantly closer to the results of not logging at all.

Configuring Custom Logging Settings

By default, logging in Mule is done asynchronously and at a level greater than or equal to INFO. The default log level discards log messages at the DEBUG or TRACE level.

To use synchronous logging, adjust the logging level or define custom categories. You can configure these properties using the $MULE_HOME/conf/log4j2.xml file, which specifies how the logger behaves. If you don’t edit this file, Mule uses the default properties.

In Anypoint Studio, log4j2.xml appears in the src/main/resources path of each Mule object.

The default configuration defines all loggers, including the root logger, as asynchronous. You can override this configuration at the domain or app level. To override this configuration at the app level, add a logConfigFile entry to the mule-artifact.json file. For example:

{
  "minMuleVersion": "4.0.0",

  "logConfigFile": "../../../test-classes/resources/logging/custom-log4j2.xml"
}

The following is the default configuration for the log4j2.xml file:

<Configuration>
    <Appenders>
        <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout pattern="%-5p %d [%t] %c: %m%n"/>
        </Console>
    </Appenders>

    <Loggers>

        <!-- CXF is used heavily by Mule for web services -->
        <AsyncLogger name="org.apache.cxf" level="WARN"/>

        <!-- Apache Commons tend to make a lot of noise which can clutter the log-->
        <AsyncLogger name="org.apache" level="WARN"/>

        <!-- Reduce startup noise -->
        <AsyncLogger name="org.springframework.beans.factory" level="WARN"/>

        <!-- Mule classes -->
        <AsyncLogger name="org.mule" level="INFO"/>
        <AsyncLogger name="com.mulesoft" level="INFO"/>

        <AsyncRoot level="INFO">
            <AppenderRef ref="Console"/>
        </AsyncRoot>
    </Loggers>

</Configuration>

Configuring Logs for the Runtime Manager Agent

This configuration is only valid when using the Runtime Manager agent 1.5.2 and later.

To log the Runtime Manager Agent state in a location other than the default mule_agent.log file, configure the $MULE_HOME/conf/log4j2.xml file to include a new Log4j 2 Appender called mule-agent-appender. If included, the Runtime Manager Agent plugin uses this appender to log its state.

The log4j2.xml file should include something like the following snippet to enable this functionality:

<Configuration>
    <Appenders>

      (...)

        <RollingFile name="mule-agent-appender" fileName="${env:MULE_HOME}/logs/custom_mule_agent.log" filePattern="${env:MULE_HOME}/logs/custom_mule_agent.log-%d{MM-dd-yyyy}.log.gz">
            <PatternLayout>
                <Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
            </PatternLayout>
            <Policies>
                <TimeBasedTriggeringPolicy />
                <SizeBasedTriggeringPolicy size="250 MB"/>
            </Policies>
        </RollingFile>
    </Appenders>

    <Loggers>

        (...)

        <AsyncLogger name="com.mulesoft.agent" additivity="TRUE" level="ALL">
          <AppenderRef ref="mule-agent-appender" />
        </AsyncLogger>

        <AsyncRoot level="INFO">
          <AppenderRef ref="Console"/>
        </AsyncRoot>

</Configuration>

The above example makes the Runtime Manager agent log its state to a rolling log file in $MULE_HOME/logs/custom_mule_agent.log. This file rolls over on a daily interval until it reaches a 250MB size.

For other Log4j 2 appender configurations, see the Apache Log4j 2 documentation.

Exception Handling with Asynchronous Logging

If you’re using asynchronous logging and experience a system crash that might have caused incomplete logs, the LMAX ExceptionHandler handler can help. By default, Mule registers this handler to log events to the disk, console, and logs/mule_ee.log. To provide your own exception handler, set the system property AsyncLoggerConfig.ExceptionHandler to the canonical name of a class that implements the interface.

The default exception handler is shown below:

/*
 * Copyright (c) MuleSoft, Inc.  All rights reserved.  http://www.mulesoft.com
 * The software in this package is published under the terms of the CPAL v1.0
 * license, a copy of which has been included with this distribution in the
 * LICENSE.txt file.
 */
package org.mule.runtime.module.launcher.api.log4j2;

import com.lmax.disruptor.ExceptionHandler;

import org.apache.logging.log4j.status.StatusLogger;

/**
 * Implementation of {@link com.lmax.disruptor.ExceptionHandler} to be used when async loggers fail to log their messages. It will
 * log this event using the {@link org.apache.logging.log4j.status.StatusLogger}
 *
 * @since 3.6.0
 */
public class AsyncLoggerExceptionHandler implements ExceptionHandler {

  private static final StatusLogger LOGGER = StatusLogger.getLogger();

  @Override
  public void handleEventException(Throwable ex, long sequence, Object event) {
    LOGGER.error("Failed to asynchronously log message: " + event, ex);
  }

  @Override
  public void handleOnStartException(Throwable ex) {
    LOGGER.error("Failed to start asynchronous logger", ex);
  }

  @Override
  public void handleOnShutdownException(Throwable ex) {
    LOGGER.error("Failed to stop asynchronous logger", ex);
  }
}

There is a performance-reliability trade-off between asynchronous and synchronous logging. If the risk of losing log messages is a serious issue, then configure your loggers to be synchronous. You can have a mix of both synchronous and asynchronous logging.

Configuration Reloading

Mule runtime uses Log4j 2 as the framework for logging. To add or change the monitoring interval at which Log4j 2 checks for changes, add the monitorInterval to the log4j2.xml configuration file:

  1. In the Mule installation folder, open conf/log4j2.xml.

  2. Add <Configuration monitorInterval="15"> after the first line. For example:

    <?xml version="1.0" encoding="UTF-8"?>
<Configuration monitorInterval="15">
    <Appenders>
       <Console name="Console" target="SYSTEM_OUT">
           <PatternLayout pattern="%-5p %d [%t] %c: %m%n"/>
       </Console>
    </Appenders>
...

    In this example, Log4j 2 checks for changes every 15 seconds.

  3. Save the file, and restart the Mule server.

For more information, see the Apache documentation on automatically configuring Log4j 2.

Debugging Projects that Use the HTTP Connector

To debug projects that use the HTTP connector, you might want to make the logging more verbose than usual and track all of the behavior of the HTTP connector’s http-listener and http-request operations in your project. See HTTP Connector Troubleshooting Guide for information.

Request and Response Logging for SOAP

  1. Open the log4j2.xml file in src/main/resources and add the HttpMessageLogger to the config:

    <!-- HTTP is used by Mule for dispatching to web services -->
<AsyncLogger name="org.mule.service.http.impl.service.HttpMessageLogger" level="DEBUG" />

  2. Save your project.

Troubleshooting Logging

I don’t see any logging output

Set the log4j2.xml at the root of your classpath.

I reconfigured Log4j 2, but nothing happened

This issue happens because there is another log4j2.xml file on your classpath that is getting picked up before your modified one. To find out which configuration file Log4j 2 is using, add the following switch when starting Mule (or add it to the container startup script if you are embedding Mule):

-M-Dlog4j.debug=true

This switch writes the Log4j 2 startup information, including the location of the configuration file being used, to stdout. You must remove that configuration file before your modified configuration can work.