Monday, 8 August 2011

Learning pg

SHOW TABLES is
SELECT table_schema, table_name FROM information_schema.tables ;

SHOW DATABASES is
SELECT table_schema, COUNT(*) AS num_tables FROM information_schema.tables
GROUP BY table_schema;


use /? to find out how to do it properly.

Monday, 27 June 2011

Deparse an ABI file...

#!/usr/bin/perl -w

use strict;

my $stadenApp = "~/staden-linux-1-6-0/linux-bin/getABIfield";


for my $inputFile (@ARGV){
unless($inputFile =~ /\/(KN654-BAC(\d\d)-(\d\d)_([A-H]\d\d)-(FP|RP))\.ab[1i]$/){
warn "ignoring $inputFile\n";
next;
}

my $fileName = $1;
my $wellId = $4;
open APP, "-|", "$stadenApp -a -t $inputFile"
or die "failed to exec $stadenApp : $!\n";
my %data;
while(){
## Skip some 'data rows' that we can ignore for now.
next unless
/^$inputFile (\S{4}) (\d+) (.*)$/;
chomp;
$data{"$1 $2"} .= $3;
}
if (keys %data == 0){
die "wha? \n"
}
print
join("\t",
$fileName,
$data{"SMPL 1"}, # Trace name (unique id)
$data{"RunN 1"}, # Run name
$wellId,
$data{"TUBE 1"}, # Well id
$data{"GTyp 1"}, # Polymer name
$data{"MCHN 1"}, # Instrument name
$data{"MODL 1"}, # Instrument model
$data{"PDMF 1"}, # Mobility file
$data{"CMNT 1"}, # Comment
## Signal to noise ratio for the 4 dyes
split(/ /, $data{"S/N% 1"}),
## Times of the various 'events'
$data{"RUND 1"}. " ". $data{"RUNT 1"},
$data{"RUND 2"}. " ". $data{"RUNT 2"},
$data{"RUND 3"}. " ". $data{"RUNT 3"},
$data{"RUND 4"}. " ". $data{"RUNT 4"},
## More...
$data{"APrV 1"}, # Analysis protocol version number
$data{"CTNM 1"}, # Plate Name
$data{"CTID 1"}, # Plate barcode
$data{"ASPt 1"}, # Basecall start scan
$data{"AEPt 1"}, # Basecall stop scan
$data{"SPAC 1"}, # Base spaceing
), "\n";
}

here it is wrapped in Perl

#!/usr/bin/perl -w

use strict;

## Set up the Perl modules we need

## Read the command line options
use Getopt::Long;

## Read in the fasta sequence file
use Bio::SeqIO;

## Run the blast query
use LWP::UserAgent;
#use HTTP::Cookies;

## Parse the results
use HTML::Strip;
use Bio::SearchIO;






## Set up the net stuff

## Set up a 'web user agent' object (something like a browser).

my $ua = LWP::UserAgent->new;



## Configure the user agent:

## Response time out (in seconds)
$ua->timeout(10);

## How the user agent should handle cookies:
#$ua->cookie_jar({ file => "lwpcookies.txt", autosave => 1})



## Note the following URI is the form behind the login page:
## http://yh.genomics.org.cn/potato/login.jsp

my $loginUri = 'http://yh.genomics.org.cn/potato/check.jsp';



## Use the user agent to login to the webserver

my $login = $ua->
post( $loginUri, [ username => 'test', password => 'abc123' ]);

#warn $login->as_string, "\n";
#exit;



## Don't know why the above dosn't trigger writing a cookie to the
## cookie.jar! (So cookie code is commented out.)

## Grab the cookie manually

die "CANT FIND MY COOKIES!!!!\n"
unless $login->as_string =~ /^Set-Cookie: JSESSIONID=(\w*);/sm;

my $JSESSIONID = $1;



## Manuall set the cookie in the user agents header, so it will be
## used for all subsequent communication.

$ua->default_header('Set-Cookie' => $JSESSIONID);



## The following URI is the form behind the blast page:
## http://yh.genomics.org.cn/potato/search.jsp

my $blastUri = 'http://yh.genomics.org.cn:8099/blast.cgi';





## Now we are set to loop through the sequences in the fasta sequence
## file and blast each one!

## Process the command line

my $fastaFile;

GetOptions(
"fasta=s" => \$fastaFile,
)
or die "failed to parse command line options!\n";

die "please pass a fasta file to blast\n"
unless $fastaFile && -e $fastaFile;



## Process the fasta file

my $s = Bio::SeqIO->
new( -file => $fastaFile,
-format => 'fasta'
);

while(my $seq = $s->next_seq){
## Lets just dump the results!
my $outFile = 'Scratch/'. $seq->id. '.blast.html';
if(-s $outFile ){
warn "skipping, file exists : $outFile\n";
next;
}
warn "doing ". $seq->id. " (". $seq->length. ")\n";
#warn $seq->seq, "\n";
open(OUT, '>', $outFile)
or die "cant open $outFile\n";
## Blast it!
## Set any blast options you want here!
my $blast = $ua->
post( $blastUri, [ DATALIB => 'potato',
SEQUENCE => $seq->seq ]);
#warn $blast->as_string, "\n";
#exit;
print OUT $blast->as_string;
}




__END__

## below we already moved onto processing the results, which is too much for one script.
## Get the results (scalar) as a file handle for processing with
## SearchIO. Here we also strip out the HTML from the results.
my $string = HTML::Strip->new->parse( $blast->as_string );
open my $fh, '<', \$string
or die "FUCK!\n";

my $in =
Bio::SearchIO->new( -format => "blast", -fh => $fh );
while( my $result = $in->next_result ) {
while( my $hit = $result->next_hit ) {
while( my $hsp = $hit->next_hsp ) {

## Apply some basic filtering
if( $hsp->length('total') > 50 ) {
if ( $hsp->percent_identity >= 75 ) {
print
join("\t",
$hit->name(),
$hsp->percent_identity,
$hsp->length('total'),
# No. mismatches?
$hsp->percent_identity('total'),
# No. gaps
$hsp->gaps('total'),
$hsp->start('query'),
$hsp->end('query'),
$hsp->start('hit'),
$hsp->end('hit'),
$hsp->evalue(),
$hsp->score(),
), "\n";
}
}
}
}
}
last;
}

warn "OK\n";

Thursday, 26 May 2011

Scripting against a remote HTML blast server

## Scripting against the CN Solexa blast server:



## See "blast.form.html", the HTML dump generated by "blast.jsp". This
## file shows how the names for the various options.



## The blast server page
URL=http://.../blast.jsp
URL=http://.../blast.cgi

## Get this from firefox (Note, expires after about 5 min!)
JSESSIONID=
JSESSIONID=BE55C37071E8CE748053DBCC47FCA3F3
JSESSIONID=1D179D4B6EF55555D344E388D6ABEA19
JSESSIONID=8921525F177EE917E5A27D3FB0CC577D

## Sequence file to submit to the form
SEQFILE=test.seq
SEQFILE=C02HBa0008G02.seq



## All other settings will be default ...

## Don't forget to redirect the result somewhere sensible

curl \
--cookie JSESSIONID=$JSESSIONID \
--form SEQFILE=@$SEQFILE \
--form DATALIB=potato \
$URL \
> $SEQFILE.cn.blast



## Try grabbing (lots of) XML output

JSESSIONID=AE961CF9BF250CCF656832143701BE59
SEQFILE=test.seq

curl \
--cookie JSESSIONID=$JSESSIONID \
--form SEQFILE=@$SEQFILE \
--form DATALIB=potato \
--form ALIGNMENT_VIEW=7 \
--form DESCRIPTIONS=500 \
--form ALIGNMENTS=500 \
$URL \
> $SEQFILE.cn.blast.xml



## Test the XML format using BioPerl

perl -e 'use Bio::SearchIO;
while($r = Bio::SearchIO->
new(
-file=>"$SEQFILE.cn.blast.xml",
-format=>blastxml,
)->next_result){
print $r->query_name, "\n";
while( $h = $r->next_hit ) {
print "\t", $h->name, "\n";
}
}
'


## Looks good!

Monday, 9 May 2011

You don't want to do that... do you? git svn git svn

git rm --cached -n -r `find ./ -type d -name .svn`

Monday, 1 November 2010

Problems with your local perl install?

Try the following uber 1337 commands (based on this):

mv ~/perl5 ~/notperl5
wget -O - http://cpanmin.us | perl - -L ~/perl5 --self-upgrade local::lib


ZOMG!



OK, now add your ~/perl5 dir into git...
cd ~/perl5/ ; git init ; git add . ; git commit -m 'initial import'

This will help you keep things in order over time.


Example:
cpanm Bio::DB::Sam


You can then use git to see exactly what you did, and commit if it worked OK. i.e.
git add . ; git commit -m 'BioPerl and Bio::DB::Sam installed'

(Once you have confirmed that it's working.)


Once your local::lib is in git, you can do lots of stuff like if something is breaking, just edit it in there to insert debugging or whatever and then just reset --hard to get rid of it after you figure out what is going on, etc.



Thanks to rbuels on irc://irc.freenode.net/#bioperl for helping with this!
(All the above is his doing).

Tuesday, 8 June 2010

Testing MediaWiki (MW) & Semantic MediaWiki (SMW)

Long time coming...


http://www.mediawiki.org/wiki/How_to_become_a_MediaWiki_hacker#Testing



First, set up the MW testing environment:

Grab the latest MW (1.15 branch):

Somewhere in the web-directory:
svn co http://svn.wikimedia.org/svnroot/mediawiki/branches/REL1_15/phase3/ MW1_15

Configured using all the default settings.


Next, grab some extensions:
http://svn.wikimedia.org/svnroot/mediawiki/branches/REL1_15/extensions/


Write some tests using http://seleniumhq.org/

1) Install the Selenium-IDE (Firefox plugin)