Bricolage daily activity summary. -- New Internationalist Blog

The Tech Blog

Bricolage daily activity summary.

Posted by Charlie Harvey on Thursday, February 5, 2009 5 comments

Before I start I should preface this with a big "yuk". Its an extremely inelegant way to achieve what I want.

By default Bricolage sends email notifications with root relative URLs /like/this. Some time ago the NI techs were chatting about how it’d be nice to be able to click the URLs from the emails. This is a partial solution to that.

We’re lucky enough to be running a dovecot imap server for our email. I filter bric email notifications into a single folder. Well, dovecot stores files inna maildir stylee. Which means they can be parsed like regular files.

So, I’ve built a script that trundles through my Bricolage folder checking for new files, parses them and prints a daily activity summary. Here’s the code:

#!/usr/bin/perl
use strict;
use warnings;
use Email::Simple;
use Data::Dumper; # Used for testing

my $MAILDIR = '/home/charlie/Maildir/my_bric_folder'; # directory where files are stored
my %baseurl_lookup = ('New Internationalist main site'=>'http://newint.org','New Internationalist blog'=>'http://b
my @emails = glob "$MAILDIR/*"; # all our emails
my %hash; # 2d hash of story details keyed on {section}{action}{storytitle}

for(@emails) {
  my $age = -M; 
  next if($age>1); # ignore old emails
  chomp;
  open IN, $_ or die ("Couldn't open $_\n$!");
  $/=undef;
  my $text = ;
  close IN; 
  $/="\n";
  my $email = Email::Simple->new($text);
  my $subject = $email->header('subject');
  my $body = $email->body;
  # Only certain subject lines require any action
  created_story($body)    if($subject =~ /A story was just created/); 
  published_story($body)  if($subject =~ /A story was just published/); 
}
#print Dumper %hash;

# Print out daily summary
for(keys %hash) {
  print "\n\n$_\n=============================\n";
  my $section=$_;
  for(keys %{$hash{$section}}) {
    print "\n$_ stories\n-----------------\n";
    my $action = $_; 
    for(keys %{$hash{$section}{$action}}) {
    print "\n$_ stories\n-----------------\n";
    my $action = $_; 
    for(keys %{$hash{$section}{$action}}) {
      print "* " . $_ . ", by " . 
        $hash{$section}{$action}{$_}{'author'} . " " . lc($action) . " on " .
        $hash{$section}{$action}{$_}{'date'} . " at " .
        $hash{$section}{$action}{$_}{'url'} . "\n";
    }
  }
}


# Parse "story created" email 
sub created_story {
  my $body = shift;
  my @body = split(/[\r\n]+/,$body);
  $body = $body[0];
  $body =~ /A story by the name of "([^"]+)" was created by ([\w\s]+) and lives at ([^\s]+) \(([^\)]+)\) and is sc
  my ($title,$author,$location,$section,$date) = ($1,$2,$3,$4,$5);
  my $action = "Created";
  my $url = $baseurl_lookup{$section} . $location;
  $hash{$section}{$action}{$title} = {'url' => $url, 'date' => $date, 'author' => $author };
}

# Parse "story published" email
sub published_story {
  my $body = shift;
  my @body = split /[\r\n]+/,$body; # strips empty lines
  # line 1
  $body[0] =~ /A story by the name of "([^"]+)" was published on ([\d\-]+ [\d\:]+) by ([^\.]+)./;
  my ($title,$date,$author) = ($1,$2,$3);
  # line 2
  $body[1] =~ /You can view a live copy of this story at ([^\(]+)\(([^\)]+)\)$/;
  my($location,$section) = ($1,$2);
  $location =~s/\s$//;
  my $action = "Published";
  my $url = $baseurl_lookup{$section} . $location;
  # print "S: $section A: $action T: $title U: $url D: $date\n\n"; # testing
  $hash{$section}{$action}{$title} = {'url' => $url, 'date' => $date, 'author' => $author };
}

Yep, its pretty gnarly. But it does the job. I guess the ideal would be to rip out the parsing code and set it up as a filter in postfix. But that’s for another day.

To install it I set up a simple crontab, using crontab -e:

0       0       *       *       * /path/to/bric_activity_summary.pl  | mail -s "FYI: Daily Bricolage activity summary" [email address scrubbed]

And now we can get daily summaries that look like this:

New Internationalist blog
=============================

Published stories
-----------------
* Cartoon Capture and Storage, by Bricolage Administrator published on 2009-02-03 09:45:09 at http://blog.newint.org/cantankerousfrank/2009/02/02/cartoon-capture-and-storage/
* Stealth wars, by Sokari Ekine published on 2009-02-05 01:25:10 at http://blog.newint.org/majority/2009/02/05/stealth-wars/
* Democratic Republic of Congo, by Sokari Ekine published on 2009-02-05 01:25:10 at http://blog.newint.org/tag/democratic-republic of congo/
* conflict, by Sokari Ekine published on 2009-02-05 01:25:10 at http://blog.newint.org/tag/conflict/
* BBC, by Sokari Ekine published on 2009-02-05 01:25:10 at http://blog.newint.org/tag/bbc/
* Singularity, by Bricolage Administrator published on 2009-02-04 09:45:07 at http://blog.newint.org/cantankerousfrank/2009/02/03/singularity/

Created stories
-----------------
* Cartoon Capture and Storage, by Bricolage Administrator created on 2009-02-02 06:13:00 at http://blog.newint.org/cantankerousfrank/2009/02/02/cartoon-capture-and-storage/
* Stealth wars, by Sokari Ekine created on 2009-02-05 01:20:06 at http://blog.newint.org/majority/2009/02/05/stealth-wars/
* Singularity, by Bricolage Administrator created on 2009-02-03 04:21:00 at http://blog.newint.org/cantankerousfrank/2009/02/03/singularity/


New Internationalist main site
=============================

Created stories
-----------------
* Fire in the soul, by Theo Sundh created on 2009-02-03 07:32:00 at http://newint.org/publications/fiction/poetry/

| See posts tagged with: Information Technology IT technology Web