Automate Enterprise Application Monitoring with Ansible
Enterprise applications are typically segregated into multiple, physically and logically independent, functional areas called tiers. As opposed to single-tiered monoliths, the multi-tier pattern enforces a distributed architecture that allows for enhanced maintainability and greater scalability, resulting in increased availability, resilience and performance. Typically, enterprise applications adhere to the classical 3-tier architecture: presentation tier, business logic tier and data tier:
Their logical and physical separateness makes distributed applications available for scale-out (also referred to as horizontal scaling). This means that a tier itself gets distributed across multiple physical servers to effectively distribute the load that otherwise needs to be handled by a single node. In such a scenario, communication between adjacent tiers is handled by load balancers which relay incoming requests to a particular node in the cluster:
Distributed Architectures: The Traceability Dilemma
Distributed architectures have undeniable complexities. One matter that is often undervalued is end-to-end traceability: how to determine data flows in a multi-tiered environment that spans multiple servers? As I have described in an earlier article on the good parts and the not-so-good parts of logging, using only traditional logging, you most probably can’t. Good application monitoring solutions, such as Dynatrace, on the other hand, allow you to figure out exactly what your users were doing throughout their visits and assess the performance of your multi-tiered application on whichever nodes involved, as described in How to Approach Application Failures in Production.
Now, what does it take to obtain these insights? Agent based application monitoring solutions like Dynatrace, AppDynamics and New Relic, require you to load an agent library with your application runtime, such as the Java Runtime Environment (JRE) or the .NET Common Language Runtime (.NET CLR).
An example: monitoring a Java process with Dynatrace requires you to load our native agent through the java command’s -agentpath:pathname[=options] option and additionally specify the name under which the agent shall register itself at the Dynatrace Collector together with the hostname of the latter, as shown below. More often than not, the -agentpath string is provided to a Java process via some environment variable, such as JAVA_OPTS.
While this can easily be accomplished for 1 or 2 agents, what would companies do who need to deploy hundreds to thousands of agents? Exactly, automate all that stuff away! Even if all you need is a handful of agents, automation will help you to produce predictable outcomes in all your environments. And if the automation tool doesn’t get into your way, even better. This is where Ansible comes into play!
Introduction to Ansible
Ansible is a radically simple, yet powerful, IT automation engine for environment and infrastructure provisioning, configuration management, application deployment and much more. Its fresh, new agentless approach allows you to effectively orchestrate entire environments without the need to install any prerequisite dependencies on the machines under management – all you need is SSH for Linux hosts or WinRM for Windows hosts as transport layer. Ansible comes with “batteries included”: more than 250 modules are bundled with Ansible’s core that allow you to get even complex automation projects done without the need for custom scripting.
Concept #1: Inventories
Ansible provisions groups of servers at once. Groups, such as groups of web servers, application servers and database servers, are defined in an inventory. Typically, an inventory is a text file that is expressed in an INI-like format that could look like so (note the use of numeric and alphabetic ranges):
# file: production
More information on Ansible inventories can be found in the official documentation.
Concept #2: Playbooks
In Ansible, playbooks define the policies your machines under management shall enforce. They are the place for you to lay out your tasks by referencing Ansible modules. The following example playbook installs the Apache Tomcat application server using the apt module with the package name tomcat7 on all hosts that belong to the group appservers, as defined in an inventory. Ansible will try to connect to each machine via SSH, using the username deploy in this case. Ansible playbooks are written in YAML.
# file: appservers.yml
– hosts: appservers
– name: Install Apache Tomcat
An Ansible playbook is executed via the ansible-playbook -i <inventory> <playbook.yml> command. Hence, we could execute our playbook like so: ansible-playbook -i production appservers.yml. More information on Ansible playbooks can be found in the official documentation.
Concept #3: Roles
Ansible roles are the best way to organize your playbooks. Roles mandate a file structure for related content, such as variables, tasks, files, handlers, etc. that enforces the automatic inclusion of your assets. Other than that, roles can be shared conveniently either via GitHub or Ansible Galaxy, Ansible’s hub for finding and sharing roles. Inside playbooks, roles provide for a clean abstraction and separation of concerns:
# file: appservers.yml
– hosts: appservers
– role: common
– role: tomcat7
More information on Ansible roles can be found, again, in the official documentation.
Introduction to Ansible Galaxy
Roles can be downloaded from Ansible Galaxy using the ansible-galaxy install username.rolename command. Additionally, you could provide a newline-separated list of dependencies in a file requirements.txt and provide that to ansible-galaxy like so: ansible-galaxy install -r requirements.txt.
Automated Deployment of Dynatrace Agents
Let’s assume that our presentation and business logic tiers are made up of physically separated instances of Apache HTTP Servers and Apache Tomcats, respectively, as sketched out below. Now, in order to get full end-to-end visibility into this environment, we need to install a Dynatrace Agent on each machine and make sure that both Apache HTTP Servers and Apache Tomcats load these agents and are configured to relay their data to a Dynatrace Collector nearby:
Next, let’s look at how the automated installation of Dynatrace Agents could be solved using the Ansible concepts described further up.
#1: Define an Ansible Inventory
# file: production
#2: Define an Ansible Playbook
# file: playbook.yml
– hosts: webservers
– role: dynatrace.Dynatrace-Apache-HTTPServer-Agent
– hosts: appservers
– role: dynatrace.Dynatrace-Tomcat-Agent
#3: Get Roles from Ansible Galaxy
Create a file requirements.txt with the content below and download the dependencies into the roles directory via ansible-galaxy install -p roles -r requirements.txt:
#4: Run the Ansible Playbook
Finally, we execute our playbook via ansible-playbook -i production playbook.yml. At the end of the run, Ansible will present you an overview of actions taken and any errors which have occurred:
ansible-playbook –i production playbook.yml
PLAY [webservers] *************************************************************
PLAY [appservers] *************************************************************
PLAY RECAP *************************************************************
www0.example.com: ok=27 changed=27 unreachable=0 failed=0
www1.example.com: ok=27 changed=27 unreachable=0 failed=0
appa.example.com: ok=2 changed=2 unreachable=0 failed=0
appb.example.com: ok=2 changed=2 unreachable=0 failed=0
appc.example.com: ok=2 changed=2 unreachable=0 failed=0
Last but not least, a restart of the application tiers will be required so that the agents will be loaded into the environment. After that, you should already see the agents connected to the Dynatrace Server. On our example environment, this took less than 60 seconds!
Please note that the Ansible roles we used in our examples support a variety of variables whose default values you will typically want to override to suit your environment. More in-depth information on the available roles can be found in our Tutorials on Automated Deployments with Ansible.
The enabling of end-to-end monitoring for enterprise applications, no matter how complex they are, can easily be automated away. Not only does automation produce deterministic and consistent outcomes particularly in huge environments, it can also save you a lot of time. At Dynatrace, we offer Ansible roles for the installation of and integration with all our products – and that’s just the beginning: Puppet modules and Chef cookbooks are coming soon.