Category Whitepapers and Guides
In this step-by-step guide, we will see how to set-up a Puppet Master in Amazon Web Services, and how to use it to create two other AWS instances.
We will then use Puppet to configure these two instances, one will be a MySQL Database Server and the other an Apache Web Server.
Note that in this Guide, we use the eu-central-1 / Frankfurt zone.
If you intend to use a different zone, you will have to change the ami-id in the appropriate places in the scripts.
Create the AWS Master Instance
Install Puppet Enterprise
chmod 400 Webdev-forest.pem
ssh -i Webdev-forest.pem ubuntu@[public hostname]
sudo su
vim /etc/hosts
"127.0.0.1 localhost master.puppet.vm master puppet"
hostname master.puppet.vm
wget -O puppet-installer.tar.gz "https://pm.puppetlabs.com/cgi-bin/download.cgi?dist=ubuntu&rel=14.04&arch=amd64&ver=latest"
tar -xf puppet-installer.tar.gz
./puppet-enterprise-<version>-ubuntu-14.04-amd64/puppet-enterprise-installer
The puppet master is now all set, so let’s take care of the agents.
Launch the agent nodes with Puppet
mkdir ~/create_instances
vim ~/create_instances/create.pp
$pe_master_hostna$::fqdn # Get the master's fqdn me = $facts['ec2_metadata']['hostname'] # Get the hostname of the master $pe_master_ip = $facts['ec2_metadata']['local-ipv4'] # Get the ip of the master $pe_master_fqdn = # Set the default for the security groups Ec2_securitygroup { region => 'eu-central-1', # Replace by the region in which your puppet master is ensure => present, vpc => 'My VPC', # Replace by the name of your VPC } # Set the default for the instances Ec2_instance { region => 'eu-central-1', # Replace by the region in which your puppet master is key_name => 'Webdev-forest', # Replace by the name of your key if you chose something else ensure => 'running', image_id => 'ami-87564feb', # ubuntu-trusty-14.04-amd64-server-20160114.5 (ami-87564feb) instance_type => 't2.micro', tags => { 'OS' => 'Ubuntu Server 14.04 LTS', 'Owner' => 'Michel Lebeau' # Replace by your name }, subnet => 'My Subnet', # Replace by the name of your Subnet } # Set up the security group for the webserver ec2_securitygroup { 'web-sg': description => 'Security group for web servers', ingress => [{ # Open the port 22 to be able to SSH into, replace by your.ip/32 to secure it better protocol => 'tcp', port => 22, cidr => '0.0.0.0/0' },{ # Open the port 80 for HTTP protocol => 'tcp', port => 80, cidr => '0.0.0.0/0' }, ], } # Set up the security group for the database server ec2_securitygroup { 'db-sg': description => 'Security group for database servers', ingress => [{ # Open the port 22 to be able to SSH into, replace by your.ip/32 to secure it better protocol => 'tcp', port => 22, cidr => '0.0.0.0/0' },{ # Open the port 3306 to be able to access mysql protocol => 'tcp', port => 3306, cidr => '0.0.0.0/0' }, ], } # Set up the instances, assign the security groups and provide user data that will be executed at the end of the initialization ec2_instance { 'webserver': security_groups => ['web-sg'], user_data => template('/root/create_instances/templates/webserver.sh.erb'), } ec2_instance { 'dbserver': security_groups => ['db-sg'], user_data => template('/root/create_instances/templates/dbserver.sh.erb'), }
You can find the VPC and subnet in the VPC section of AWS, please note that Puppet expects the name of the VPc and subnet, the ID will not work.
mkdir ~/create_instances/templates
vim ~/create_instances/templates/webserver.sh.erb
#!/bin/bash PE_MASTER='<%= @pe_master_hostname %>' echo "<%= @pe_master_ip %> <%= @pe_master_fqdn %>" >> /etc/hosts # Download the installation script from the master and execute it curl -sk https://$PE_MASTER:8140/packages/current/install.bash | /bin/bash -s agent:certname=webserver
vim ~/create_instances/templates/dbserver.sh.erb
#!/bin/bash PE_MASTER='<%= @pe_master_hostname %>' echo "<%= @pe_master_ip %> <%= @pe_master_fqdn %>" >> /etc/hosts # Download the installation script from the master and execute it curl -sk https://$PE_MASTER:8140/packages/current/install.bash | /bin/bash -s agent:certname=dbserver
/opt/puppetlabs/puppet/bin/gem install aws-sdk-core retries
mkdir ~/.aws/
vim ~/.aws/credentials
[default] aws_access_key_id = # Paste here your Access Key ID aws_secret_access_key = # Paste here your Secret Access Key ID region = # Specify your region, optional
puppet module install puppetlabs-aws
puppet apply /root/create_instances/create.pp
[root@master ~]# puppet apply /root/create_instances/create.pp Notice: Compiled catalog for master.puppet.vm in environment production in 0.11 seconds Notice: /Stage[main]/Main/Ec2_instance[webserver]/ensure: changed absent to running Notice: /Stage[main]/Main/Ec2_instance[dbserver]/ensure: changed absent to running Notice: Applied catalog in 25.15 seconds
Configure Apache and MySQL using Roles and Profiles
Now, we have two running Puppet Agent nodes communicating with our Puppet Enterprise Master. Only a few steps more and we will enjoy our new website!
Create The Database and and the Webserver Roles
The Roles will define the business logic of our applications, and will be composed by one or more profiles.
cd /etc/puppetlabs/code/environments/production/
mkdir -p modules/roles/manifests
vim modules/roles/manifests/dbserver.pp
# Role for a Database Server class roles::dbserver { # Include the mysql profile include profiles::mysql }
vim modules/roles/manifests/webserver.pp
# Role for a Web Server class roles::webserver { # Include the apache profile include profiles::apache }
Create the Apache and MySQL Profiles
Now, we will create our Profiles which will define the application stack for Aache and MySQL
mkdir -p modules/profiles/manifests
vim modules/profiles/manifests/apache.pp
# Install and configure an Apache server class profiles::apache { # Install Apache and configure it class { 'apache': mpm_module => 'prefork', docroot => '/var/www', } # Install the PHP mod include apache::mod::php # Install php5-mysql for PDO mysql in PHP package { 'php5-mysql': ensure => installed, } # Get the index.php file from the master and place it in the document root file { '/var/www/index.php': ensure => file, source => 'puppet:///modules/profiles/index.php', owner => 'root', group => 'root', mode => '0755', } # Declare the exported resource @@host { 'webserver': ip => $::ipaddress, host_aliases => [$::hostname, $::fqdn] ,pin } # Collect the exported resources Host <<||>> }
vim modules/profiles/manifests/mysql.pp
# Install and configure a MySQL server class profiles::mysql { # Install MySQL Server and configure it class {'mysql::server': root_password => 'p4ssw0rd', remove_default_accounts => true, restart => true, override_options => { mysqld => { bind_address => '0.0.0.0', 'lower_case_table_name' => 1, } } } # Copy the sql script from the puppet master to the /tmp directory file { 'mysql_populate': ensure => file, path => '/tmp/populate.sql', source => 'puppet:///modules/profiles/populate.sql', } -> # Only once the file has been copied, use it to populate a new database mysql::db { 'cats': user => 'forest', password => 'p4ssw0rd2', grant => ['SELECT', 'UPDATE', 'INSERT', 'DELETE'], host => '%', # You can replace by 'webserver' to make it more secure, # but you might have to flush your hosts in mysql for it # to be taken into account sql => '/tmp/populate.sql', } # Declare the exported resources @@host { $::hostname: ip => $::ipaddress, host_aliases => [$::fqdn, 'database'] , } # Collect the exported resources Host <<||>> }
mkdir modules/profiles/files
vim modules/profiles/files/populate.sql
USE `cats`; CREATE TABLE `family` ( `id` mediumint(8) unsigned NOT NULL auto_increment, `Name` varchar(255) default NULL, `Age` mediumint default NULL, PRIMARY KEY (`id`) ) AUTO_INCREMENT=1; INSERT INTO `family` (`Name`,`Age`) VALUES ("Hasad",6),("Uma",5),("Breanna",17),("Macaulay",14),("Colton",11),("Serina",16),("Emery",13),("Christian",7),("Vladimir",16),("Wang",13); INSERT INTO `family` (`Name`,`Age`) VALUES ("Hermione",12),("Yoshio",9),("Hilel",10),("Autumn",6),("Solomon",7),("Briar",6),("Armand",9),("Alyssa",1),("Shelby",1),("Yasir",15); INSERT INTO `family` (`Name`,`Age`) VALUES ("Wallace",1),("Yoshio",5),("Pascale",6),("Dalton",17),("Trevor",9),("Joan",10),("Zephr",14),("Neville",3),("Nicole",4),("Halee",14); INSERT INTO `family` (`Name`,`Age`) VALUES ("Wayne",15),("Maile",8),("Alfonso",9),("Neve",6),("Heidi",16),("Mona",11),("Mollie",16),("Audra",16),("Karyn",12),("Acton",17); INSERT INTO `family` (`Name`,`Age`) VALUES ("Xyla",1),("Cole",6),("Blossom",9),("Sybill",4),("Lavinia",4),("Keely",14),("Gwendolyn",15),("Trevor",10),("Acton",12),("Christine",10); INSERT INTO `family` (`Name`,`Age`) VALUES ("Stone",17),("Erich",12),("Elijah",10),("Emerson",14),("Rafael",8),("Scott",17),("Olympia",13),("Nehru",14),("Casey",8),("Michael",3); INSERT INTO `family` (`Name`,`Age`) VALUES ("Montana",8),("Heidi",11),("Edward",13),("Xenos",1),("Venus",9),("Malik",5),("Madeline",2),("Sacha",8),("Whitney",13),("Eagan",8); INSERT INTO `family` (`Name`,`Age`) VALUES ("Lewis",2),("Guinevere",17),("Oliver",6),("Jana",7),("Rachel",2),("Ariel",7),("Pamela",6),("Medge",11),("Clare",10),("Meghan",8); INSERT INTO `family` (`Name`,`Age`) VALUES ("Stone",10),("Chase",4),("Vladimir",17),("Grace",11),("Damon",15),("Ferdinand",11),("Veronica",14),("Wesley",13),("Zelda",15),("Eugenia",6); INSERT INTO `family` (`Name`,`Age`) VALUES ("Carlos",9),("Cherokee",14),("Theodore",3),("Tanisha",11),("Grant",7),("Xyla",6),("Austin",11),("Madison",4),("Kasper",7),("Andrew",10);
vim modules/profiles/files/index.php
<?php echo "<h1>Our small cat family</h1>"; echo "<table style='border: solid 1px black;'>"; echo "<tr><th>Id</th><th>Name</th><th>Age</th></tr>"; class TableRows extends RecursiveIteratorIterator { function __construct($it) { parent::__construct($it, self::LEAVES_ONLY); } function current() { return "<td style='width:150px;border:1px solid black;'>" . parent::current(). "</td>"; } function beginChildren() { echo "<tr>"; } function endChildren() { echo "</tr>" . "\n"; } } $host = "database"; $port = "3306"; $username = "forest"; $password = "p4ssw0rd2"; $dbname = "cats"; try { $conn = new PDO("mysql:host=$host;port=$port;dbname=$dbname", $username, $password); $conn->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION); $stmt = $conn->prepare("SELECT id, Name, Age FROM family"); $stmt->execute(); // set the resulting array to associative $result = $stmt->setFetchMode(PDO::FETCH_ASSOC); foreach(new TableRows(new RecursiveArrayIterator($stmt->fetchAll())) as $k=>$v) { echo $v; } } catch(PDOException $e) { echo "Error: " . $e->getMessage(); } $conn = null; echo "</table>"; ?>
puppet module install puppetlabs-apache
puppet module install puppetlabs-mysql
Classify our Nodes
vim manifests/site.pp
node 'dbserver'{ include roles::dbserver } node 'webserver'{ include roles::webserver } node default { }
Puppet can be run using various methods, with the CLI, using MCollective or by using the Web Console for example. In this case we are going to use MCollective:
root@master:~# su - peadmin peadmin@master:~$ mco puppet runonce -v -I webserver -I dbserver * [ ============================================================> ] 2 / 2 webserver : OK {:summary=> "Started a Puppet run using the '/opt/puppetlabs/bin/puppet agent --onetime --no-daemonize --color=false --show_diff --verbose --splay --splaylimit 120' command", :initiated_at=>1471353250} dbserver : OK {:summary=> "Started a Puppet run using the '/opt/puppetlabs/bin/puppet agent --onetime --no-daemonize --color=false --show_diff --verbose --splay --splaylimit 120' command", :initiated_at=>1471353250} ---- rpc stats ---- Nodes: 2 / 2 Pass / Fail: 2 / 0 Start Time: 2016-08-16 13:14:11 +0000 Discovery Time: 0.00ms Agent Time: 142.88ms Total Time: 142.88ms peadmin@master:~$
To check if Puppet run successfully in the nodes and the changes that were applied to them, login to the Web Console and go to Configuration > Overview
Now paste the public address of your webserver in your favourite browser and voilà, you are done! Note that if you get a Error: SQLSTATE[HY000] [2005] Unknown MySQL server host ‘database’ (2), you should try to run puppet using mco another time, as the exported resources haven’t been collected. This happens if the ip from the dbserver is not exported before when the webserver collects its resources. Running puppet another time will collect it.
Please note that if you terminate an AWS instance and start another with the create.pp script, it will have the same certname as the one that has been terminated, however the IP will differ. In order for Puppet to run correcty in this case, on the master execute:
puppet cert clean <certname>
With <certname> being either dbserver or webserver,
Related articles
Puppet Enterprise is one of the leading continuous delivery technologies, building on its heritage in infrastructure automation with the addition of Puppet Application Orchestration. Forest Technologies are proud partners of Puppet Labs and experts in delivering rapid value to our customers’ digital transformation initiatives using Puppet Enterprise.