Centralize IBM ACE Platform, Application and Exception Logging

Introduction

After implementing the ESB pattern with IBM IIB/ACE for so many years, a very common question is about how to provide better and more efficient mechanism for

• IBM ACE and MQ Console and Error Logs
• IBM ACE Application Logs
• IBM ACE Application Exceptions

Especially starting from IBM ACE v11, applications (Bar files) can be deployed into independent Integration Server (IS) containers without the centralized Node component. The common issues usually include :

• Dispersed Log and Exception entries on multiple servers when using Trace Node and Log4j SupportPac.
• Log and Exception entries not necessarily tied to the Message Flows, Local Environment and Container Locations.
• Using Trace Node which uses local file I/O and usually no standard formats. Local file usage is also not ideal for container deployment.
• For historical Applications, the “common” or “shared” Logging and Exception Handling flows are statically packaged with the application Bar files.

Framework

For these years, I implemented couple of different versions of Framework or SBB to address the issues. Here, I would present one of those.

The environment :

1. uses IBM Standardized ACE and MQ Container images.
2. deploys and configures Applications with image overrides folders.
3. centralizes all container logs to fluent-bit container from other containers.
4. uses Logstash, OpenSearch and OpenSearch Dashboard to ingest and store all the logging entries.

Components Diagram :

And the corresponding Sequence Diagram:

The Framework provides couple useful features:

1. Support IIB v10, ACE v11 and ACE v12.
2. Shared Library deployed per Execution Group. So no need to package with the Application Bar files anymore. Any updates to the Shared Libraries are independent from those Applications and being effective immediately.
3. Each Log and Exception entries from different Message Flows can be correlated with a rqUID, either user provided or automatically generated.
4. All Log and Exception entries include information on Node, Execution Group, Applications, Local Environment, Message Body and customized logging info.
5. Intermediary queues required for the Framework can be flexibly located with IBM MQ Client Connections.
6. With default fluentd package with the IBM standard images, it can also pull all the IBM ACE and MQ console and error logs into OpenSearch.
7. Ingest, collect and centralize all the logging and exceptions entries asynchronously into OpenSearch. (My previous version used database too)
8. Uses Logstash Pipelines to pull to Log and Exceptions entries into OpenSearch to facilitate configuration and allow parallel/multithreading processing.
9. OpenSearch Indices can be split into daily partition indices to improve querying and better utilize and archive storage.
10. Support personnel can query and filter logging entries from a centralized dashboard (OpenSearch Dashboard or Kibana).
11. With the default JDBC Plugins, support personnel can also use standard SQL to select entries from any JDBC compliant workbench.