cpanm for lazy people

This way individual projects with specific perl requirements avoid
borking the global install.


## I'm going to use MediaWiki::API to load, so I'll install local::lib
## using cpanm.

## Step 1, install a local cpanm (and some deps)

curl -k -L http://cpanmin.us \
   | perl - -l local-lib App::cpanminus


## Step 2, install local::lib

./local-lib/bin/cpanm \
   -l local-lib local::lib

perl -Ilocal-lib/lib/perl5/ -Mlocal::lib=`pwd`/local-lib \
   > local-lib.sh

source local-lib.sh


## Step 3, install MediaWiki::API

cpanm MediaWiki::API




## ENJOY

Portable shebang (for Perl)

What do you do when you have a Perl script that starts with:

#!/somewhere/stupid/perl

?


Just run the script with perl like this:

$ perl my_stupid_scipt.pl


How do you avoid writing scripts like that? Do the following:


#!/usr/bin/env perl

Deparse an ABI file...

#!/usr/bin/perl -w

use strict;

my $stadenApp = "~/staden-linux-1-6-0/linux-bin/getABIfield";

for my $inputFile (@ARGV){

unless($inputFile =~ /\/(KN654-BAC(\d\d)-(\d\d)_([A-H]\d\d)-(FP|RP))\.ab[1i]$/){

warn "ignoring $inputFile\n";

next;

}

my $fileName = $1;

my $wellId = $4;

open APP, "-|", "$stadenApp -a -t $inputFile"

or die "failed to exec $stadenApp : $!\n";

my %data;

while(){

## Skip some 'data rows' that we can ignore for now.

next unless

/^$inputFile (\S{4}) (\d+) (.*)$/;

chomp;

$data{"$1 $2"} .= $3;

}

if (keys %data == 0){

die "wha? \n"

}

print

join("\t",

$fileName,

$data{"SMPL 1"}, # Trace name (unique id)

$data{"RunN 1"}, # Run name

$wellId,

$data{"TUBE 1"}, # Well id

$data{"GTyp 1"}, # Polymer name

$data{"MCHN 1"}, # Instrument name

$data{"MODL 1"}, # Instrument model

$data{"PDMF 1"}, # Mobility file

$data{"CMNT 1"}, # Comment

## Signal to noise ratio for the 4 dyes

split(/ /, $data{"S/N% 1"}),

## Times of the various 'events'

$data{"RUND 1"}. " ". $data{"RUNT 1"},

$data{"RUND 2"}. " ". $data{"RUNT 2"},

$data{"RUND 3"}. " ". $data{"RUNT 3"},

$data{"RUND 4"}. " ". $data{"RUNT 4"},

## More...

$data{"APrV 1"}, # Analysis protocol version number

$data{"CTNM 1"}, # Plate Name

$data{"CTID 1"}, # Plate barcode

$data{"ASPt 1"}, # Basecall start scan

$data{"AEPt 1"}, # Basecall stop scan

$data{"SPAC 1"}, # Base spaceing

), "\n";

}

here it is wrapped in Perl

#!/usr/bin/perl -w

use strict;

## Set up the Perl modules we need

## Read the command line options

use Getopt::Long;

## Read in the fasta sequence file

use Bio::SeqIO;

## Run the blast query

use LWP::UserAgent;

#use HTTP::Cookies;

## Parse the results

use HTML::Strip;

use Bio::SearchIO;

## Set up the net stuff

## Set up a 'web user agent' object (something like a browser).

my $ua = LWP::UserAgent->new;

## Configure the user agent:

## Response time out (in seconds)

$ua->timeout(10);

## How the user agent should handle cookies:

#$ua->cookie_jar({ file => "lwpcookies.txt", autosave => 1})

## Note the following URI is the form behind the login page:

## http://yh.genomics.org.cn/potato/login.jsp

my $loginUri = 'http://yh.genomics.org.cn/potato/check.jsp';

## Use the user agent to login to the webserver

my $login = $ua->

post( $loginUri, [ username => 'test', password => 'abc123' ]);

#warn $login->as_string, "\n";

#exit;

## Don't know why the above dosn't trigger writing a cookie to the

## cookie.jar! (So cookie code is commented out.)

## Grab the cookie manually

die "CANT FIND MY COOKIES!!!!\n"

unless $login->as_string =~ /^Set-Cookie: JSESSIONID=(\w*);/sm;

my $JSESSIONID = $1;

## Manuall set the cookie in the user agents header, so it will be

## used for all subsequent communication.

$ua->default_header('Set-Cookie' => $JSESSIONID);

## The following URI is the form behind the blast page:

## http://yh.genomics.org.cn/potato/search.jsp

my $blastUri = 'http://yh.genomics.org.cn:8099/blast.cgi';

## Now we are set to loop through the sequences in the fasta sequence

## file and blast each one!

## Process the command line

my $fastaFile;

GetOptions(

"fasta=s" => \$fastaFile,

)

or die "failed to parse command line options!\n";

die "please pass a fasta file to blast\n"

unless $fastaFile && -e $fastaFile;

## Process the fasta file

my $s = Bio::SeqIO->

new( -file => $fastaFile,

-format => 'fasta'

);

while(my $seq = $s->next_seq){

## Lets just dump the results!

my $outFile = 'Scratch/'. $seq->id. '.blast.html';

if(-s $outFile ){

warn "skipping, file exists : $outFile\n";

next;

}

warn "doing ". $seq->id. " (". $seq->length. ")\n";

#warn $seq->seq, "\n";

open(OUT, '>', $outFile)

or die "cant open $outFile\n";

## Blast it!

## Set any blast options you want here!

my $blast = $ua->

post( $blastUri, [ DATALIB => 'potato',

SEQUENCE => $seq->seq ]);

#warn $blast->as_string, "\n";

#exit;

print OUT $blast->as_string;

}

__END__

## below we already moved onto processing the results, which is too much for one script.

## Get the results (scalar) as a file handle for processing with

## SearchIO. Here we also strip out the HTML from the results.

my $string = HTML::Strip->new->parse( $blast->as_string );

open my $fh, '<', \$string

or die "FUCK!\n";

my $in =

Bio::SearchIO->new( -format => "blast", -fh => $fh );

while( my $result = $in->next_result ) {

while( my $hit = $result->next_hit ) {

while( my $hsp = $hit->next_hsp ) {

## Apply some basic filtering

if( $hsp->length('total') > 50 ) {

if ( $hsp->percent_identity >= 75 ) {

print

join("\t",

$hit->name(),

$hsp->percent_identity,

$hsp->length('total'),

# No. mismatches?

$hsp->percent_identity('total'),

# No. gaps

$hsp->gaps('total'),

$hsp->start('query'),

$hsp->end('query'),

$hsp->start('hit'),

$hsp->end('hit'),

$hsp->evalue(),

$hsp->score(),

), "\n";

}

}

}

}

}

last;

}

warn "OK\n";

Problems with your local perl install?

Try the following uber 1337 commands (based on this):

mv ~/perl5 ~/notperl5
wget -O - http://cpanmin.us | perl - -L ~/perl5 --self-upgrade local::lib


ZOMG!



OK, now add your ~/perl5 dir into git...
cd ~/perl5/ ; git init ; git add . ; git commit -m 'initial import'

This will help you keep things in order over time.


Example:
cpanm Bio::DB::Sam


You can then use git to see exactly what you did, and commit if it worked OK. i.e.
git add . ; git commit -m 'BioPerl and Bio::DB::Sam installed'

(Once you have confirmed that it's working.)


Once your local::lib is in git, you can do lots of stuff like if something is breaking, just edit it in there to insert debugging or whatever and then just reset --hard to get rid of it after you figure out what is going on, etc.



Thanks to rbuels on irc://irc.freenode.net/#bioperl for helping with this!
(All the above is his doing).

Perl wrapper for R

This Perl wrapper for R fills in some missing functionality (line number and context on error) and automatically 'dumps' the output of the script.

I can be improved in many ways, but here it is, as is....

   1. #! /usr/bin/perl -w
   2.
   3. unless (@ARGV){
   4.   warn "\n";
   5.   warn "Runs an R script with STDERR dumped to STDOUT\n\n";
   6.   warn "Usage: $0 \n\n";
   7.   exit(1);
   8. }
   9.  
  10. my $rFile = shift @ARGV;
  11.  
  12. die "Stubbornly refusing to run R on a file that doesn't end .R!\n"
  13.   unless $rFile =~ /\.R$/;
  14.  
  15. my $oFile = `basename $rFile .R`; chomp( $oFile );
  16.  
  17. my $args = join(" ", @ARGV);
  18.  
  19. if ($args){
  20.   $args = "--args $args";
  21. }
  22.  
  23. warn "Using: R $args < $rFile > $oFile.dump\n";
  24.  
  25.  
  26.  
  27. open ( OH, ">$oFile.dump" )
  28.   or die "cant : $! \n";
  29.  
  30. my $pid =
  31.   open( PH, "R $rFile -q --vanilla $args < $rFile 2>&1 |" )
  32.     or die "cant : $? : $! \n";
  33.  
  34. my $lineNumber;
  35.  
  36. while(){
  37.   $lineNumber++ if /^(\>|\+)/o;
  38.   print OH;
  39. }
  40.  
  41. close( OH );
  42. close( PH );
  43.  
  44. if ($?) {
  45.   #warn "killed by $?\n";
  46.   warn "FAILED AT LINE $lineNumber\n";
  47.   system("tail -n 5 $oFile.dump");
  48.   exit(1);
  49. }
  50.  
  51. warn "OK (NO ERRORS DETECTED)\n";

vpnc timeout problem solved with Perl

Or ... EXPECT PAIN!!!

I've seen lots of people complaining about a time out problem with vpnc. This happens to me even if I'm ping-ing like crazy or not, so its nothing to do with activity on the connection (I think).

The problem with automatically re-connecting was the need to interactively type a password at prompt. Note that this was not my "Xauth password". For some reason my organisation requires an extra password (there are many ways a VPN can be configured).

If you just need to type one username and password, its probable that you can set up your config file to contain:

Xauth username meUserName
Xauth password meSekretPassword


With the above you could just automatically restart vpnc every time it drops out (i.e. using bash). I have this extra password to deal with though, so its a bit harder. Here is the solution achieved using Perl, Expect [1], and a lot of help from the friendly mst in irc://irc.freenode.net/#Perl

1. use Expect;
3. while(1){
4. $e=Expect->new("vpnc", ("--no-detach", "/home/me/my.VPN.conf"));
5. $e->debug(1);
6. $e->log_file("log");
7. if($e->expect(15, "-re", "Password for VPN")){
8. $e->send("sekret pwd\n");
9. $e->interact
10. }
11. else{
12. die "fail!!!\n"
13. }
14. print "woop\n";
15. sleep 1
16. }

[1] http://search.cpan.org/~rgiersig/Expect-1.21/Expect.pod


Take care,
Dan.