Perl IO with IO::All

Working with Files and Directories in Perl Programming is not really a big deal. But there is a  module that ease this in a cool way.

If you want to do IO operations in Perl i advice you to take a look at IO::All.

File Operations

Opening a file in the traditional way is done with the open() function and than you can iterate over the lines.

my @lines;
open my $fh, "<", "the-file";
@lines = <$fh>; 
close $fh;

With IO::All you can simplify this to one line

my @lines = io("the-file")->getlines;

If you want to load a complete file into a string you have to do the following without IO::All:

my $content;
open my $fh, "<", "the-file";
{
    local $/;
    $content = <$fh>; 
} 
close $fh;

Compared to IO::All it looks a bit too much:

my $content = io("the-file")->all;

You can also use the "<" operator (you might already know from bash) to read a file.

my $content < io("the-file");

Writing to a file is also really easy and straight forward compared to the standard way.

open my $fh, ">", "the-file";
print $fh "Hello World\n"; 
close $fh;

With IO::All it is - as you may expect - only one line.

"Hello World\n" > io("the-file");

If you want to append to a file you can just use the ">>" operator.

"Hello World\n" >> io("the-file");

Special File Operations

With IO::All it is also easy to manipulate files without much code.

For example if you want to edit a special line inside a file you can do this with the following code:

my $file = io("the-file");
$file->[10] = "New content in Line eleven";
print $file->[20];

Directory Operations

 If you want to read the contents of a directory in the past you had code like this:

my @contents; 
opendir my $dh, "/dir";
while(my $entry = readdir($dh)) {    
    next if($entry eq "." || $entry eq "..");
    push @contents, $entry;
} 
closedir $dh;

With IO::All you can reduce this to a one liner:

my @contents = io("/dir")->all;

With this function it is also easy to traverse subdirectories.

my @contents = io("/dir")->all(0);

Or, if you don't want to go through every subdirectory you can specify the depth.

my @contents = io("/dir")->all(2);

Setting hostname and domainname under Solaris 10

Just a short post howto set the hostname and the domainname under Solaris 10. It took a while until i found the solution, hence this post.

echo your-shiny-hostname >/etc/nodename
echo your-shiny-domain >/etc/defaultdomain

now reboot and it is done.

Puppet, iterating over a hash

Today i fiddled about how to iterate over a hash in a puppet function (define). Okay, it is a bit .. scary but it works ;-)

define mymodule::myfunction($options = {}) {
     $cli_parameter = inline_template(
          "<% options.each_pair { |key, val| %> --<%= key %>=<%= val %><% } %>");
     // ...
}

Managing Servers with (R)?ex

Rex is a tool to automate the management of servers. In this blog post i will just post an example of managing some webservers with mod_jk and tomcat.

Every webserver has the same file system structure and the same os (but this is negligible).


To prepare a new webserver i've written a task to install all the needed packages, copy the configuration files and start all the required services.


This is my Rexfile.

use lib 'lib';
user "root";      # set your username
password "pass";  # set your password
pass_auth;        # enable password authentication

# put your server in this group
group "www" => "www[01..05]";

# now load every module via ,,require''
require Rex::mysite;

I've splittet the different tasks in some smaller modules.

This is the module for all the base tasks. (Saved under "lib/Rex/mysite.pm")

package Rex::mysite;

use Rex::Commands;
use Rex::Commands::Fs;
use Rex::Commands::Run;
use Rex::Commands::Pkg;
use Rex::Commands::Upload;
use Rex::Hardware;

use Data::Dumper;

use Rex::mysite::plain;

desc "Prepare system";
task "prepare", group => 'www', sub {

   my %host_info = Rex::Hardware->get(qw/ Host /);

   if($host_info{'Host'}->{'operatingsystemrelease'} =~ m/^5/) {
      install file => "/etc/apt/sources.list", {
         source => "files/sources.list.lenny",
      };
   }
   else {
      install file => "/etc/apt/sources.list", {
         source => "files/sources.list.squeeze",
      };
   }

   if(! is_dir("/root/.ssh")) {
      mkdir "/root/.ssh";
   }

   upload "files/authorized_keys", "/root/.ssh/authorized_keys";

   run "apt-get update";

   install package => "vim";
   install package => "less";

   remove package => "avahi-daemon";
};


1;

And now i have a task to install the tomcat server and configure the apache servers with mod_jk.  (Saved under "lib/Rex/mysite/plain.pm"). With the needs function i define a dependency to the prepare Task in the module Rex::mysite. So this task will be executed always in the same context (same ssh connection) of the tomcat Task in this module.

package Rex::mysite::plain;

use Rex::Commands;
use Rex::Commands::Fs;
use Rex::Commands::Run;
use Rex::Commands::Pkg;
use Rex::Commands::Upload;
use Rex::Commands::Service;

use Data::Dumper;

desc "Prepare Tomcat";
task "tomcat", sub {
   needs Rex::mysite "prepare";

   run "echo sun-java6-jdk shared/accepted-sun-dlj-v1-1 select true | /usr/bin/debconf-set-selections";
   run "echo sun-java6-jre shared/accepted-sun-dlj-v1-1 select true | /usr/bin/debconf-set-selections";

   install package => [
                        "sun-java6-jdk",
                        "libapache2-mod-jk",
                        "sc-tomcat-6.0.28",
                        "sc-tomcat-juli-6.0.28",
                        "sc-tomcat-log4j-1.2.16"
                     ];

   unlink "/etc/apache2/sites-enabled/000-default";
   upload "files/000-default", "/etc/apache2/sites-enabled/000-default";
   upload "files/jk.conf", "/etc/apache2/mods-enabled/jk.conf";

   install file => "/etc/apache2/workers.properties", {
      source    => "files/workers.properties.tpl",
      on_change => service(apache2 => "restart"),
   };

};

1;

The files i uploaded in these tasks are not realy interesting. They are just simple configuration files. Except the file workers.properties.tpl. This i a template file and looks like this.

worker.list=lb,jkstatus

worker.w1.port=8009
worker.w1.host=<%+ $::Network->{"networkconfiguration"}->{"eth0"}->{"ip"} %>
worker.w1.type=ajp13
worker.w1.lbfactor=1
worker.w1.reference=worker.template

worker.lb.type=lb
worker.lb.balance_workers=w1
worker.lb.max_reply_timeouts=10

worker.lb.sticky_session=false
worker.jkstatus.type=status

worker.template.type=ajp13
worker.template.connection_pool_timeout=600
worker.template.connection_pool_minsize=0

This is a template file. During the upload process Rex will parse this template and replace <%+ $::Network->{"networkconfiguration"}->{"eth0"}->{"ip"} %> with the ip of the networkdevice eth0.

With these tasks i can update the configuration of all my servers realy simple. If a new server needs to be installed i can just run the task on this server.

$ rex -H mynewserver mysite:plain:tomcat