Tag Archives: puppet

Deleting old releases from rails app server using puppet

Posted on by
Reply

Problem

You would like to automatically delete old releases from your rails app server using a puppet crontab resource.

Solution

Add the following to your modules/crontab/manifests/init.pp file and modify some of the values to what you need:

 # use // to suppress warning: Warning: Unrecognised escape sequence '\;'
  cron { remove_old_releases:
    command   => "test $(find /var/www/your/path/to/ror/app/ -maxdepth 1 -type d | wc -l) -gt 10 && find /var/www/your/path/to/ror/app/ -maxdepth 1 -type d -mtime +14 -exec rm -rf '{}' \\;",
    user      => ubuntu,
    hour      => 0,
    minute    => 15
  }

You should need to change the following:

/var/www/your/path/to/ror/app/ to your application’s release path
-gt 10 this is the number of minimum old releases you want to keep
-mtime +14 releases older than 14 days are deleted

UPDATE

There is chance that when you do not calculate the frequency of the releases right the above script can potentially delete all of your releases, so you end up with no releases 🙁

A better solution to keep the -n number of releases (based on the solution from here:

find /var/www/your/path/to/ror/app/* -type d -printf '%T@ %p\n' | sort -nr | tail -n+6 | cut -f 2- -d " "  | xargs -i rm -rf {}

and the explanation of each step:

  • find /var/www/your/path/to/ror/app/* -maxdepth 0 -type d -printf ‘%T@ %p\n’ find all the directories (-maxdepth 0 -type d) in the search path (/var/www/your/path/to/ror/app/*) excluding the . and .. directories (/*) and add to them the time information (‘%T %p\n’)
  • sort -nr sort them in numeric and reverse order – newer first, oldest last
  • tail -n+6 keep only the first 5 directories, starting from line 6 (tail -n+6)
  • cut -f 2- -d ” “ Remove the first field (ie the date information) keeping the file from the second field using the space as the delimiter (-d ” “)
  • xargs -i rm -rf {} Pass the rm -rf command to delete the directory for each line produced in the previous steps

So the puppet script should be as follows (including the escape characters \ for the % and the additional \):

 # use // to suppress warning: Warning: Unrecognised escape sequence '\;'
  cron { remove_old_releases:
    command   => "find /var/www/your/path/to/ror/app/* -maxdepth 0 -type d -printf '\\%T@ \\%p\\n' | sort -nr | tail -n+6 | cut -f 2- -d ' ' | xargs -i rm -rf '{}' \\;",
    user      => ubuntu,
    hour      => 0,
    minute    => 15
  }

Error in AWS when trying to include puppet modules (Error: Could not find class apache2 for …)

Posted on by
Reply

Problem

You are trying to work with puppet modules in AWS, after installing puppet as a gem, but when you try to put the modules and manifests outside the main manifests/site.pp file you get an error similar to following:

Error: Could not find class apache2 for ip-xx-xx-xx-xx.eu-west-1.compute.internal on node ip-xx-xx-xx-xx.eu-west-1.compute.internal

Solution

Create a file for the puppet configuration in /etc/puppet/puppet.conf and add the paths to your own manifests and modules, like :

$aws sudo vi /etc/puppet/puppet.conf

and add your paths:

[main]
manifest = /home/ubuntu/puppet/manifests/site.pp
modulepath = /etc/puppet/modules:/usr/share/puppet/modules:/home/ubuntu/puppet/modules

sudo: puppet: command not found – when trying to use puppet with rvm in aws

Posted on by
2

When trying to use the 

rvmsudo puppet apply --verbose manifests/site.pp

command in a aws instance to follow the instruction from ‘Deploying Rails’, the following error complains that the puppet command is not found, like:

sudo: puppet: command not found

After some searching and having a look at the blog post here and using the rvm notes in the aws instance and looking at this: 

  * On some systems (like Ubuntu) rvmsudo requires following changes to work properly:
      http://stackoverflow.com/questions/257616/sudo-changes-path-why

It seems that for security reason the sudo command resets the path in the system.

You can actually see that this the case by running the command to display the path of the normal user:

aws$ echo 'echo $PATH' | sh

and the corresponding one for the sudo:

aws$ echo 'echo $PATH' | sudo sh

So a workaround to make it work is to add your group into the exemptions group in the visudo file, which means editing the visudo file:

sudo visudo

and adding the group that your user you are working with belongs to:

Defaults        env_reset
Defaults        exempt_group=name_of_your_group(or admin in some systems)

Writing a Passenger Module for Puppet – Vagrant

Posted on by
Reply

Following the steps described in the “Deploying Rails Applications” book and using the currently latest puppet installation described here, the actual puppet files had to be modified slightly in order to make it work as expected.

The main issues requiring the changes were as follow:

  • System wide rvm installation – different paths, and the use of the rvmsudo command instead of sudo
  • Use of latest puppet gem (3.0.1) – ruby 1.9.3 is supported
  • Use of Debian/Ubuntu system – different way of configuring passenger with apache2

So the full file for the passenger module (puppet/modules/passenger/manifests/init.pp) is:

class passenger {

  # required for passenger gem
  package {
    "apache2-prefork-dev":
      ensure => installed
  }

  # required for passenger-apache-module
  package {
    "libcurl4-openssl-dev":
      ensure => installed
  }

  exec  {
    "gem install passenger -v=3.0.18":
      user    => vagrant,
      group   => vagrant,
      alias   => "install_passenger",
      require => Package["apache2"],
      unless  => "ls /usr/local/rvm/gems/ruby-1.9.3-p327@rails3_2_9/bin/passenger",
      before  => Exec["passenger_apache_module"]
  }

  exec  {
    "passenger-install-apache2-module --auto":
      user    => root,
      group   => root,
      alias   => "passenger_apache_module",
      require => Package["apache2-prefork-dev","libcurl4-openssl-dev"],
      unless  => "ls /usr/local/rvm/gems/ruby-1.9.3-p327@rails3_2_9/gems/passenger-3.0.18/ext/apache2/mod_passenger.so",
      before  => File["passenger_conf"]
  }

  file  {
    "/etc/apache2/mods-available/passenger.conf":
      mode    => 644,
      owner   => root,
      group   => root,
      alias   => "passenger_conf",
      source  => "/home/vagrant/puppet/modules/passenger/files/passenger.conf",
      before  => File["passenger_load"]
  }

  file {
    "/etc/apache2/mods-available/passenger.load":
      mode    => 644,
      owner   => root,
      group   => root,
      alias   => "passenger_load",
      notify  => Service["apache2"],
      source  => "/home/vagrant/puppet/modules/passenger/files/passenger.load",
      before  => Exec["enable_passenger"]
  }

  exec {
    "a2enmod passenger":
      user    => root,
      alias   => "enable_passenger"
  }
}

Going through the file one of the first things to notice is the declaration about the “apache2-prefork-dev” package installation

 package {
    "apache2-prefork-dev":
      ensure => "installed"
  }

The reason for this package, is that when trying to compile the passenger apache module (passenger-install-apache2-module), the script will complain that the following packages are missing:

apache2-prefork-dev
libapr1-dev
libaprutil1-dev

By installing the first of the packages (apache2-prefork-dev) the other two are installed automatically as dependencies.

NOTE In trying to install it in an Ubuntu AMI image in AWS EC2 there is also an additional package requirement because the installer complains about:

To install Curl development headers with SSL support:  Please run apt-get install libcurl4-openssl-dev or libcurl4-gnutls-dev, whichever you prefer.

So there is the additional package added as:

# required for the passenger_apache_module
package {
  "libcurl4-openssl-dev":
    ensure  => installed
}

Further down there are two file resources named “passenger_conf” and “passenger_load”.
Looking at the structure in debian for the apache2 configuration it seems that apache2 in debian needs two different files for each module that can be loaded one with named as module.conf and another as module.load to be placed into /etc/apache2/mods-available.

So we can create these two files and placed them in the files folder, as:

puppet/modules/passenger/files/passenger.conf


  PassengerRoot /usr/local/rvm/gems/ruby-1.9.3-p327@rails3_2_9/gems/passenger-3.0.18
  PassengerRuby /usr/local/rvm/wrappers/ruby-1.9.3-p327@rails3_2_9/ruby
  PassengerUseGlobalQueue on
  PassengerMaxPoolSize  5
  PassengerPoolIdleTime 900
  PassengerMaxRequests 10000

and, puppet/modules/passenger/files/passenger.load

LoadModule passenger_module /usr/local/rvm/gems/ruby-1.9.3-p327/gems/passenger-3.0.11/ext/apache2/mod_passenger.so

Which you can actually see if you compile manually the passenger apache2 module as they are described in the output on successful compilation:

The Apache 2 module was successfully installed.

Please edit your Apache configuration file, and add these lines:                                                                                                                                                                                

   LoadModule passenger_module /usr/local/rvm/gems/ruby-1.9.3-p327@rails3_2_9/gems/passenger-3.0.18/ext/apache2/mod_passenger.so
   PassengerRoot /usr/local/rvm/gems/ruby-1.9.3-p327@rails3_2_9/gems/passenger-3.0.18
   PassengerRuby /usr/local/rvm/wrappers/ruby-1.9.3-p327@rails3_2_9/ruby

After you restart Apache, you are ready to deploy any number of Ruby on Rails                                                                                                                                                                   
applications on Apache, without any further Ruby on Rails-specific                                                                                                                                                                              
configuration!                                                                                                                                                                                                                                  

Press ENTER to continue.

Finally the last section of the init.pp file enables the passenger module with the a2enmod apache command:

  exec {
    "a2enmod passenger":
      user    => root,
      alias   => "enable_passenger"
  }

A couple of useful apache2 commands when testing are:

  • To test what modules are availble for loading: 
    a2enmod

    If run after creating the two passenger files passenger.conf and passenger.load, then the passenger module should appear in the list

  • To see the currently enabled modules: 
    apache2ctl -M

    Which again if it is run after the passenger modules has been enabled with a2enmod passenger then it should show an entry with 

    passenger_module (shared)

Could not find puppet (>= 0) amongst [] (Gem::LoadError)

Posted on by
Reply

Problem

When you try to apply the puppet configuration into a vagrant box and using systemwide rvm installation (if you follow the instructions here), you have the following error:

vagrant@precise64:~$ sudo puppet apply --verbose puppet/manifests/site.pp
/usr/local/rvm/rubies/ruby-1.9.3-p327/lib/ruby/site_ruby/1.9.1/rubygems/dependency.rb:247:in `to_specs': Could not find puppet (>= 0) amongst [] (Gem::LoadError)
from /usr/local/rvm/rubies/ruby-1.9.3-p327/lib/ruby/site_ruby/1.9.1/rubygems/dependency.rb:256:in `to_spec'
from /usr/local/rvm/rubies/ruby-1.9.3-p327/lib/ruby/site_ruby/1.9.1/rubygems.rb:1231:in `gem'
from /usr/local/rvm/gems/ruby-1.9.3-p327/bin/puppet:18:in `
'
from /usr/local/rvm/gems/ruby-1.9.3-p327/bin/ruby_noexec_wrapper:14:in `eval'
from /usr/local/rvm/gems/ruby-1.9.3-p327/bin/ruby_noexec_wrapper:14:in `
'

Solution

Use rvmsudo instead of sudo to make sure that the paths are correct:

vagrant@precise64:~$ rvmsudo puppet apply --verbose puppet/manifests/site.pp

Vagrant – rvm – ruby – puppet installation instructions

Posted on by
Reply

Problem

In order to install a vagrant box with rvm and puppet following the examples in the ‘Deploying Rails’ book, the updated steps needed are described below.
The two main differences are a) rvm system wide installation as recommended here, and b) the puppet 3.0.1 version support of ruby 1.9.3 here

Solution

So below are the steps followed to make it work.

  1. Remove system ruby 
    sudo rm -rf /opt/vagrant_ruby/
  2. Remove vagrant_ruby.sh script as it adds a path that no longer exists: 
    sudo rm /etc/profile.d/vagrant_ruby.sh
  3. Install rvm with system wide installation: 
    curl -L get.rvm.io | sudo bash -s stable
  4. Add vagrant user and any other additional user to the rvm group: 
    sudo usermod --append --groups rvm vagrant (or ubuntu)
  5. Exit and login back in to install ruby 1.9.3-p327 (current) 
    vm$ exit
$ vagrant ssh
vm$ rvm install 1.9.3-p327
  6. install necessary ruby dependencies suggestest by previous command: 
    sudo apt-get install build-essential openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison subversion pkg-config
  7. Creat a gemset to be used with the desired rvm ruby version: 
    rvm gemset create rails3_2_11
  8. Use the created gemset to install puppet gem: 
    rvm gemset use rails3_2_11
  9. Install puppet as the vagrant user and NOT as sudo: 
    gem install puppet -v 3.0.1
  10. install the puppet user and group as per book instructions: 
    vm$ sudo useradd --comment "Puppet" --no-create-home --system --shell /bin/false puppet
  11. Optionally create a puppet directory to put manifests and modules and create a .rvmrc file to use specified gemset, by adding the following line to puppet/.rvmrc: 
    rvm gemset use rails3_2_11
  12. use rvmsudo instead of sudo to apply the puppet scripts: 
    rvmsudo puppet apply --verbose puppet/manifests/site.pp