Automate Enterprise Application Monitoring with Ansible

 In Blog

Bvc7Zh5IEAAmPM9Enterprise Applications and Distributed Architectures

Enterprise applications are typically segregated into multiple, physically and logically independent, functional areas called tiers. As opposed to single-tiered monoliths, the multi-tier pattern enforces a distributed architecture that allows for enhanced maintainability and greater scalability, resulting in increased availability, resilience and performance. Typically, enterprise applications adhere to the classical 3-tier architecture: presentation tier, business logic tier and data tier:

Their logical and physical separateness makes distributed applications available for scale-out (also referred to as horizontal scaling). This means that a tier itself gets distributed across multiple physical servers to effectively distribute the load that otherwise needs to be handled by a single node. In such a scenario, communication between adjacent tiers is handled by load balancers which relay incoming requests to a particular node in the cluster:

Load-Balanced Cluster

Distributed Architectures: The Traceability Dilemma

Distributed architectures have undeniable complexities. One matter that is often undervalued is end-to-end traceability: how to determine data flows in a multi-tiered environment that spans multiple servers? As I have described in an earlier article on the good parts and the not-so-good parts of logging, using only traditional logging, you most probably can’t. Good application monitoring solutions, such as Dynatrace, on the other hand, allow you to figure out exactly what your users were doing throughout their visits and assess the performance of your multi-tiered application on whichever nodes involved, as described in How to Approach Application Failures in Production.

Now, what does it take to obtain these insights? Agent based application monitoring solutions like Dynatrace, AppDynamics and New Relic, require you to load an agent library with your application runtime, such as the Java Runtime Environment (JRE) or the .NET Common Language Runtime (.NET CLR).

An example: monitoring a Java process with Dynatrace requires you to load our native agent through the java command’s -agentpath:pathname[=options] option and additionally specify the name under which the agent shall register itself at the Dynatrace Collector together with the hostname of the latter, as shown below. More often than not, the -agentpath string is provided to a Java process via some environment variable, such as JAVA_OPTS.

-agentpath:/opt/dynatrace/agent/lib/libdtagent.so=name=java-agent,collector=dynatrace.company.com

While this can easily be accomplished for 1 or 2 agents, what would companies do who need to deploy hundreds to thousands of agents? Exactly, automate all that stuff away! Even if all you need is a handful of agents, automation will help you to produce predictable outcomes in all your environments. And if the automation tool doesn’t get into your way, even better. This is where Ansible comes into play!

Introduction to Ansible

Ansible is a radically simple, yet powerful, IT automation engine for environment and infrastructure provisioning, configuration management, application deployment and much more. Its fresh, new agentless approach allows you to effectively orchestrate entire environments without the need to install any prerequisite dependencies on the machines under management – all you need is SSH for Linux hosts or WinRM for Windows hosts as transport layer. Ansible comes with “batteries included”: more than 250 modules are bundled with Ansible’s core that allow you to get even complex automation projects done without the need for custom scripting.

Concept #1: Inventories

Ansible provisions groups of servers at once. Groups, such as groups of web servers, application servers and database servers, are defined in an inventory. Typically, an inventory is a text file that is expressed in an INI-like format that could look like so (note the use of numeric and alphabetic ranges):

# file: production
[balancers]
www.example.com

[webservers]
www[0-1].example.com

[appservers]
app[a:c].example.com

[databases]
db.example.com

[monitoring]
dynatrace.example.com

More information on Ansible inventories can be found in the official documentation.

Concept #2: Playbooks

In Ansible, playbooks define the policies your machines under management shall enforce. They are the place for you to lay out your tasks by referencing Ansible modules. The following example playbook installs the Apache Tomcat application server using the apt module with the package name tomcat7 on all hosts that belong to the group appservers, as defined in an inventory. Ansible will try to connect to each machine via SSH, using the username deploy in this case. Ansible playbooks are written in YAML.

# file: appservers.yml

– hosts: appservers
tasks:
– name: Install Apache Tomcat
apt: name=tomcat7
remote_user: deploy
sudo: yes

An Ansible playbook is executed via the ansible-playbook -i <inventory> <playbook.yml> command. Hence, we could execute our playbook like so: ansible-playbook -i production appservers.yml. More information on Ansible playbooks can be found in the official documentation.

Concept #3: Roles

Ansible roles are the best way to organize your playbooks. Roles mandate a file structure for related content, such as variables, tasks, files, handlers, etc. that enforces the automatic inclusion of your assets. Other than that, roles can be shared conveniently either via GitHub or Ansible Galaxy, Ansible’s hub for finding and sharing roles. Inside playbooks, roles provide for a clean abstraction and separation of concerns:

# file: appservers.yml

– hosts: appservers
roles:
– role: common
– role: tomcat7
remote_user: deploy
sudo: yes

More information on Ansible roles can be found, again, in the official documentation.

Introduction to Ansible Galaxy

Roles can be downloaded from Ansible Galaxy using the ansible-galaxy install username.rolename command. Additionally, you could provide a newline-separated list of dependencies in a file requirements.txt and provide that to ansible-galaxy like so: ansible-galaxy install -r requirements.txt.

Automated Deployment of Dynatrace Agents

Let’s assume that our presentation and business logic tiers are made up of physically separated instances of Apache HTTP Servers and Apache Tomcats, respectively, as sketched out below. Now, in order to get full end-to-end visibility into this environment, we need to install a Dynatrace Agent on each machine and make sure that both Apache HTTP Servers and Apache Tomcats load these agents and are configured to relay their data to a Dynatrace Collector nearby:

3-Tier Application Example

Next, let’s look at how the automated installation of Dynatrace Agents could be solved using the Ansible concepts described further up.

#1: Define an Ansible Inventory

# file: production
[webservers]
www[0-1].example.com

[appservers]
app[a:c].example.com

#2: Define an Ansible Playbook

# file: playbook.yml

– hosts: webservers
roles:
– role: dynatrace.Dynatrace-Apache-HTTPServer-Agent
remote_user: deploy

– hosts: appservers
roles:
– role: dynatrace.Dynatrace-Tomcat-Agent
dynatrace_tomcat_agent_env_var_file_name: /usr/share/tomcat7/bin/catalina.sh
remote_user: deploy

#3: Get Roles from Ansible Galaxy

Create a file requirements.txt with the content below and download the dependencies into the roles directory via ansible-galaxy install -p roles -r requirements.txt:

dynatrace.Dynatrace-Apache-HTTPServer-Agent
dynatrace.Dynatrace-Tomcat-Agent

#4: Run the Ansible Playbook

Finally, we execute our playbook via ansible-playbook -i production playbook.yml. At the end of the run, Ansible will present you an overview of actions taken and any errors which have occurred:

ansible-playbook –i production playbook.yml

PLAY [webservers] *************************************************************

PLAY [appservers] *************************************************************

PLAY RECAP *************************************************************
www0.example.com: ok=27 changed=27 unreachable=0   failed=0
www1.example.com: ok=27 changed=27 unreachable=0   failed=0
appa.example.com: ok=2  changed=2   unreachable=0   failed=0
appb.example.com: ok=2  changed=2   unreachable=0   failed=0
appc.example.com: ok=2  changed=2   unreachable=0   failed=0

Last but not least, a restart of the application tiers will be required so that the agents will be loaded into the environment. After that, you should already see the agents connected to the Dynatrace Server. On our example environment, this took less than 60 seconds!

Dynatrace Agents Connected

Visualization of Flow in Dynatrace

Please note that the Ansible roles we used in our examples support a variety of variables whose default values you will typically want to override to suit your environment. More in-depth information on the available roles can be found in our Tutorials on Automated Deployments with Ansible.

Conclusions

The enabling of end-to-end monitoring for enterprise applications, no matter how complex they are, can easily be automated away. Not only does automation produce deterministic and consistent outcomes particularly in huge environments, it can also save you a lot of time. At Dynatrace, we offer Ansible roles for the installation of and integration with all our products – and that’s just the beginning: Puppet modules and Chef cookbooks are coming soon.

Recent Posts

Leave a Comment

Start typing and press Enter to search