Configuring Ephemeral instances in the Enterprise Dashboard

Configuring Ephemeral instances in the Enterprise Dashboard

In FusionReactor 8.1 we introduced the Ephemeral Data Service (EDS). This service allows you to register your transient / short-lived instances to the Enterprise Dashboard so they can all be monitored from a single location. Configuring Ephemeral instances in the Enterprise Dashboard is relatively straightforward.

Using EDS, you see each instance in your dashboard and have the ability to group these instances together to quickly see the health of either a single instance or group of instances.

Specifically, with EDS you also gain access to secure tunneling, this means when your instance spins up and registered with your Enterprise dashboard, a secured tunnel is created between the dashboard.

This allows data to flow between the instance and the dashboard to power metrics, but more importantly, it allows the Dashboard to view the user interface of the connected instance via a proxy.

Using the proxy means you can access the user interface of the running instance without having to expose a port to access FusionReactor, meaning your application can be locked down and secure without limiting access to your APM data.

Before we begin!

It is important to note that EDS ephemeral instances will only function if the instance is not already registered within the Enterprise Dashboard.

If you are converting to using EDS and have previously added instances to your Enterprise Dashboard manually or using the Dynamic Configuration using the -Dfrregisterwith JVM arguments, the servers must be removed from the Enterprise dashboard.

To do this you can go to the Manage Servers page and delete the instances.

To confirm if you have non-ephemeral instances registered you can look for the entry for the instance in the Manage Servers page of FusionReactor. Ephemeral instances will not display here.

How to Configure the EDS?

Below is a full guide for running a dashboard and application instances via Docker.

Configuring the EDS requires you to set a few java arguments on both the instance running the Enterprise dashboard and the application instances.

We will be using Docker and Docker-compose to create our ephemeral instances to run the EDS setup.

Java Arguments

Passing Java arguments into your JVM can be done in a number of different ways and changes depending on the application server you are running. For a list of all the common locations see this doc.

The full list of available arguments for the EDS can be found here.

For our example we will be using only the required arguments:

• -Dfr.ed.ds.enable=true – This argument allows the dashboard to use EDS connections
• -Xmx512m – This argument increases the memory of the dashboard instance, as the secured tunnels and metrics are held in memory
• -Dfr.ed.ds.target=fram:2106 – This argument tells the application instance which dashboard to register to

We will also be using the following arguments listed here to configure our instance settings automatically. These arguments are optional:

• -Dfrlicense=X – This argument sets the license key used by the instance
• -Dfradminpassword – This argument sets the administrator password for the instance

Docker Images

In our example, we created 2 simple docker images and a docker-compose file to run our EDS setup. All the files used in our example is available in our GitHub repository

Our FRAM instance uses an OpenJDK base image and the FusionReactor NoJRE installer. We are using a FRAM instance as it does not use an additional server seat so we do not need to pay for an extra license to run it.

Our Dockerfile contains the following:

FROM openjdk:12

ADD https://intergral-dl.s3.amazonaws.com/FR/Latest/FusionReactor_unix_nojre.tar.gz /opt

RUN tar -xvf /opt/FusionReactor_unix_nojre.tar.gz -C /opt

ADD start.sh /opt/fusionreactor/start.sh

EXPOSE 8087

WORKDIR /opt/fusionreactor/

CMD /opt/fusionreactor/start.sh

This docker image downloads the latest NoJRE installer, adds our custom start script with the required Java arguments and exposes the FRAM port 8087.

In the injected start.sh file we have:

#!/bin/bash

dir=`pwd`
LICENSE=${LICENSE:='XXXX-XXXX-XXXX-XXXX-XXXX'}
PASS=${PASS:='admin'}
java -javaagent:"$dir/instance/FRAM/fusionreactor.jar"=name=FRAM,address=:8087 -Dfrnopointcuts -Dfrstartupdelay=0 -Djava.awt.headless=true -Xmx512m -Dfr.ed.ds.enable=true -Dfrlicense=${LICENSE} -Dfradminpassword=${PASS} -cp "$dir/instance/FRAM/fusionreactor.jar" com.intergral.fusionreactor.agent.service.Service

In this bash file, we set the instance password and license key based on the environment variables LICENSE and PASS, and have our EDS server arguments in place.

Our Tomcat instance uses the tomcat base image. The Dockerfile contains:

FROM tomcat

RUN mkdir -p /opt/fusionreactor/instance/tomcat

ADD https://intergral-dl.s3.amazonaws.com/FR/Latest/fusionreactor.jar /opt/fusionreactor/instance/tomcat

ADD https://s3-us-west-1.amazonaws.com/intergral-dl/FR/Latest/libfrjvmti_x64.so /opt/fusionreactor/instance/tomcat

ADD runTomcat.sh /opt

RUN chmod =x /opt/runTomcat.sh

CMD /opt/runTomcat.sh

This file takes the base tomcat image, downloads the 2 FusionReactor files and adds a custom tomcat start script that allows us to set dynamic java arguments. The installation of FusionReactor is based off a manual installation.

The tomcat start script looks like:

#!/bin/bash

LICENSE=${LICENSE:='XXXX-XXXX-XXXX-XXXX-XXXX'}
PASS=${PASS:='admin'}
TARGET=${TARGET:='fram'}
GROUPS=${GROUPS:='group1'}
export JAVA_OPTS="$JAVA_OPTS -javaagent:/opt/fusionreactor/instance/tomcat/fusionreactor.jar=address=8088,name=tomcat -agentpath:/opt/fusionreactor/instance/tomcat/libfrjvmti_x64.so -Dfrlicense=${LICENSE} -Dfr.ed.ds.target=${TARGET}:2106 -Dfradminpassword=${PASS} -Dfr.ed.ds.groups=${GROUPS}"
/usr/local/tomcat/bin/catalina.sh run

This file passes in the FusionReactor start arguments, the license key, admin password and the EDS target and groups are based off the environment variables LICENSE, PASS, TARGET and GROUPS.

Running the EDS setup

To run the EDS setup, we are using docker-compose. This allows us to configure services with environment variables and exposed ports, without needing to add them to our docker run commands each time.

Our docker-compose yml file is:

version: '3'
services:
    fram:
      image: fram
      container_name: fram
      ports:
        - "8187:8087"
      environment:
        - LICENSE=YOUR_LICENSE_HERE
        - PASS=admin1
    tomcat:
      image: edtomcat
      depends_on:
        - fram
      environment:
        - LICENSE=YOUR_LICENSE_HERE
        - PASS=admin2
        - TARGET=fram
        - GROUPS=production,tomcats

In this file we have our license key, passwords, target and groups specified and a dependency ensuring the FRAM dashboard is running before the tomcat instances deploy.

To run our EDS setup we can run docker-compose up –scale tomcat=X to run as many copies of the application as we require. Provided the memory option of -Xmx=512M has been set in the FRAM instance, it is possible for the dashboard to monitor around 10,000 instances.

Features of the EDS Enterprise Dashboard

With the EDS dashboard running, as instances start, stop or restart they will be removed and re-added to the dashboard as soon as they are running. This means as your instances scale up and down you can always get an accurate picture of the health of your infrastructure.

Group and Instance Health

The grouped cubes colour will change as your group health transitions from Error > Warning > OK.

The colour of the instance cube will indicate the health of the instance, you see the used heap memory, used CPU, number of active requests and number of active database calls with a quick glance.

The cubes will transition between OK, warning and error states based on configurable limits.

For the instance you can see an overview of the web metrics, without having to navigate to the instance to view the data.

Instance User Interface with Proxy

For each instance, there is a link that will use the tunnel proxy to give you access to the instances user interface, without any ports of the application being exposed to the internet.

To access the instance through the tunnel, you can click the Arrow icon on the instance.

Using the tunnel applications could be completely locked down behind a firewall, load balancer or AWS security group and you can still access the FusionReactor data to diagnose any issues you are facing.