R-squared

Building (hackishly) rubygems from CMake-based tarballs

allan — Thu, 19 Jan 2012 02:06:12 +0000

Here is my attempt was making a user-space bundle install of the SimGrid’s Ruby bindings. I created a gemspec for the source tarball in a hackish way. Basically I just made a system “cmake .” call in the extconf.rb file when building the gem extensions. Here’s the result:

$ cat Gemfile
source :rubygems

gem “simgrid-ruby”
$ SIMGRID_ROOT=/opt/simgrid-3.6.2 bundle install –path vendor
…
…
make install
[ 66%] Built target SG_ruby
[100%] Built target SG_rubydag
Install the project…
– Install configuration: “”
CMake Error at cmake_install.cmake:47 (FILE):
file cannot create directory: /usr/local/ruby/1.9.1/x86_64-linux. Maybe
need administrative privileges.

make: *** [install] Error 1

-edit Gemfile
$ cat Gemfile
source :rubygems

gem “simgrid-ruby”, :path => “~/tmp/simgrid-ruby” # where I have
simgrid-ruby.git cloned
$ SIMGRID_ROOT=/opt/simgrid-3.6.2 bundle install
/home/aespinosa/LocalDisk/opt/ruby/lib/ruby/1.9.1/yaml.rb:56:in `’:
Using simgrid-ruby (0.0.4890e3f) from source at ~/tmp/simgrid-ruby
Using bundler (1.0.21)
Updating .gem files in vendor/cache
* simgrid-ruby at `~/tmp/simgrid-ruby` will not be cached.
Your bundle is complete! It was installed into ./vendor
$ cd vendor/ruby/1.9.1/gems/simgrid-ruby-0.0.4890e3f/examples/
$ bundle exec ruby MasterSlave.rb
Tremblay:Master:(1) 0.000000] [ruby/INFO] args[0]=20
[Tremblay:Master:(1) 0.000000] [ruby/INFO] args[1]=50000000
[Tremblay:Master:(1) 0.000000] [ruby/INFO] args[2]=1000000
[Tremblay:Master:(1) 0.000000] [ruby/INFO] args[3]=4
[Tremblay:Master:(1) 0.000000] [ruby/INFO] Master Sending Task_0 to
slave 0 with Compute Size 50000000.0
[Tremblay:Master:(1) 0.215872] [ruby/INFO] Master Sending Task_1 to
slave 1 with Compute Size 50000000.0
[Tremblay:Master:(1) 0.381834] [ruby/INFO] Master Sending Task_2 to
slave 2 with Compute Size 50000000.0
…
…
[Bourassa:Slave:(2) 6.234851] [ruby/INFO] Slave ‘slave 0′ done
executing task Task_16.
[Fafard:Slave:(4) 6.243210] [ruby/INFO] Slave ‘slave 2′ done executing
task Task_18.
[Ginette:Slave:(5) 6.759426] [ruby/INFO] Slave ‘slave 3′ done
executing task Task_19.
[Tremblay:Master:(1) 6.772657] [ruby/INFO] Master : Everything’s Done
Simulation time : 6.77266
$

Sort of works! Basically since rubygems installed the entire manifest (I placed
everything in the tarball) on the gem directory, the lib/*.so and lib/*.rb files
somehow fall in place.
Problems arise in the build when the gem installer cannot replace the
RUBYARCHDIR string in the Makefile to install it in isolation
(https://github.com/rubygems/rubygems/blob/master/lib/rubygems/ext/builder.rb)

I made a fork of a repository with the gemspec package in
https://github.com/aespinosa/simgrid-ruby . For now this
hackish-setup satisfies my needs of a user-space installation of
simgrid-ruby bindings.

ADEM on github

allan — Mon, 03 May 2010 20:18:18 +0000

For my first Ruby project, I reimplemented ADEM from scratch. ADEM is tool for automatically installing applications on the Open Science Grid via pacman. Here is a glimpse of the interface.


ADEM is a tool for deploying and managing software on the Open Science Grid.

  Usage:
    adem command [options]

  Examples:
    adem config --display
    adem sites --update
    adem app --avail

  Further help:
    adem config -h/--help        Configure ADEM
    adem sites -h/--help         Manipulate the site list
    adem app -h/--help           Application installation
    adem help                    This help message

The original subversion repository is found in https://svn.ci.uchicago.edu/svn/vdl2/SwiftApps/adem-osg. A Github repository is also mirrored in git://github.com/aespinosa/adem.git

Happy grid computing!

Correspondence on presidentiable’s science agenda

allan — Mon, 26 Apr 2010 04:03:16 +0000

A few days ago, I sent facebook messages to all of the presidential candidates. It was hard to distinguish between fan-generated accounts and the real ones. Amazed by my own facebook stalking skills, I got replies from Dick Gordon and Eddie Villanueva. Here was my message to all of them:

hi,

I’m deciding on my ballot as an absentee voter. as a scientist, please point me to pages/documents about your science & innovation agenda. I sent this message to all the candidates’ facebook pages.

also, please make a response aside from “increase the R & D” budget. it’s like saying ‘world peace’ in a beauty pageant.

thank you

Response from Richard Gordon. Most of the initiatives he mentioned are highly googlable press articles. This guy know his technology stuff:

Allan, kindly look up track record on the net. My Subic cybercity program as well as GIS and GPS mapping implemented in the 90′s. Check also Project 143 and the geo-hazard mapping initiatives with the Red Cross to address disasters and calamities. V-12 program for Filipinos abroad to help promote tourism with telcos using sms is another initiative. Also go to www. senate.gov.ph for transcript of debates re my position on agricultural modernization, and ST and IT development. Thanks.

Response from Eddie Villanueva. I think the reply was referring to a section his platform page. Some of the party’s stand reitarated the current government’s policy such as mother tongue-based education. Also, I’m still waiting for a reply on the email address he gave in the response:

hello! please email your concern to bagongpilipinas@gmail.com
you may also want to visit http://broeddie.ph for the information that you might need.

Bangon Pilipinas!

I will update this page as soon as I get a response from the other candidates, provided that it is their real facebook account that I corresponded to.

SwiftScript Vim syntax file

allan — Fri, 16 Apr 2010 01:05:48 +0000

One weekend I was reading an article howto made a Vim syntax file. In application I decided to make one for the Swift workflow system. Most of the script contains simple word matches for some Swift keywords. Then I copied the matching rules for comments from the C syntax files in the standard Vim distribution. For a preview, checkout the screenshot below which uses the desert256.vim colorscheme in a gnome-terminal:

Fig. a Swift workflow syntax highlighted

The syntax file can be downloaded from my graduate student code-shanty page.

memcache-ing everything in an ActiveRecord model.

allan — Wed, 14 Apr 2010 01:04:18 +0000

I was writing a script that was interactive with a database. The script was being invoked 400 times by a java program. It wasn’t too happy forking too many processes at a time. First I thought that mysql can only handle so many remote connections. Here is my attempt to reduce database load to almost 0 percent.

cache_fu was too dependent on being installed in Rails environment. Commenting out the code which referred to Rails variables made the ruby interpreter not complain. But according to the memcached server logs, nothing was being cached at all! I ended up using the low-level access API fauna.

Wrapping all the ActiveRecord::Base.find…() operations in the fauna-documented recipe, all of the SELECT * statements are now being cached. But the problem that remains is that the models still make ‘SHOW FIELDS..’ queries to the database whenever the object is first invoked. And since the script is being invoked 400 times, only bandwidth and round trip times were saved by the caching effort.

I poked through the metaprogramming examples in the pick-axe book and resulted to overriding my models like this:

require 'memcached'
require 'digest/md5'
require 'active_record'

class Variation < ActiveRecord::Base
  def self.digest
    Digest::MD5.hexdigest(self.to_s).to_s
  end

  def self.columns=(cached_columns)
    @cached_columns= cached_columns
    def self.columns
      @cached_columns
    end
    @cached_columns
  end
end

cache = Memcached.new
begin
  Variation.columns = cache.get(Variation.digest)
rescue Memcached::NotFound
  Variation.columns
  cache.set(Variation.digest, Variation.columns)
end
cache = nil

Now everything is cached!

Pasting Excel Charts in Word via Crossover Linux

allan — Wed, 06 Jan 2010 01:42:44 +0000

Pasting a chart from MS Excel 2007 to Word 2007 will yield a To insert a chart, you must first close any open dialog boxes or cancel editing mode in Microsoft Office Excel error. This bug is found in Crossover Linux Pro v8.0.0. I assume that this will also occur in publicly available versions of Wine.

An alternative solution aside from booting to Windows is to use the Paste Special feature. Paste it as a Picture (Enhanced Metafile). This will preserve the vector information of the chart so zooming in and out of the document object wont’ be a problem.

FASTA splitting with BioRuby

allan — Wed, 09 Dec 2009 22:33:58 +0000

In reference to my previous post, here’s the splitter using BioRuby. Note that I also changed the outer loop to one file per iteration instead of some crazy rules of when to create the file.

#!/usr/bin/env ruby
#
# Script: dumpseq.rb [file] [N] [prefix]
# Description: Splits a fasta file evenl across N files.  dumps files in the
#              [prefix]  directory
require 'bio'
require 'fileutils'

include Bio


seqs =  FlatFile.open(ARGV[0])
ncpus = ARGV[1].to_i
prefix = ARGV[2]

# Remove and hardwire n_seqs if you know beforehand the number of sequences in
# a file.  Saves readtime
n_seqs = 0
seqs.each do |seq|
 n_seqs += 1
end
seqs.rewind

overflow = n_seqs % ncpus
split_size = n_seqs / ncpus

ncpus.times do |i|
  filename = sprintf "%s/D%07d/seq%07d.fasta", prefix, i, i
  FileUtils.mkdir_p File.dirname(filename)
  dump = File.new(filename, "w")
  split_size.times do |j|
    dump << seqs.next_entry.to_s
  end
  if i < overflow 
    dump << seqs.next_entry.to_s
  end
  dump.close
end

Splitting bioinformatics FASTA files

allan — Fri, 09 Oct 2009 00:44:49 +0000

I keep forgetting where my scripts were in my home directories. Below is my ruby script to split a large FASTA [1] sequence into N sequences per file:

#!/usr/bin/env ruby
#
# Script: dumpseq.rb
# Description: Parses the a BLAST Fasta file and dumps each sequence to a 
#              file.
# Usage: dumpseq.rb [fasta_file]

require 'fileutils'


fasta_db  = File.new(ARGV[0])

sno = 0
d = 0

file = nil

while true
  x = fasta_db.readline("\n>").sub(/>$/, "")
  x =~ />(.*)\n/
  if sno % 2 == 0 # 2 seqs per query
    file.close if file != nil
    dir = sprintf("D%04d000", d / 1000)
    FileUtils.mkdir_p dir
    # short filenames
    fname = sprintf "SEQ%07d.fasta", d
    d += 1
    file = File.new("#{dir}/#{fname}","w")
  end
  file << x
  sno += 1
  fasta_db.ungetc ?>
end

Its pretty hackish-looking. But then I found out that BioRuby [2] wrappers for parsing FASTA files.

[1] http://en.wikipedia.org/wiki/Fasta
[2] http://www.bioruby.org

Why do research in the Philippines?

allan — Sun, 08 Mar 2009 02:01:30 +0000

Because installing grid computing middleware can get you to this:

7^th PANDA Grid Workshop, Bohol, Philippines, May 4 – 8, 2009
organised by
Ateneo de Manila University
Sponsored also by EPSRC, IoP, PPARC and the Royal Society of Edinburgh

The aim of the workshop is to bring together grid administrators and software developers in an informal setting, involving open discussions. The focus will include grid maintenance and monitoring and data production with PandaRoot.

Organising committee:

Rafael P. Saldana (Ateneo)
Kilian Schwarz (GSI)
Dan Protopopescu (Glasgow)

Contact person:

Dr. Dan Protopopescu,
University of Glasgow
Glasgow G12 8QQ
Scotland, UK
Tel: +44-141-330-55321

Address:

Holy Name University,
Lesage and Gallares Streets,
6300 Tagbilaran City,
Bohol, Philippines

Let’s look at the itinerary:

Tagbilaran City (May 3, 4, 5)

Metro Centre Hotel and Convention Center
Pres. Carlos P. Garcia Avenue
Tagbilaran City, Bohol
Philippines, 6300
Website: www.metrocentrehotel.com

Panglao Island (May 6, 7, 8, 9, 10)

Bohol Beach Club
Bo. Bolod, Panglao Island, Bohol 6340
Website: www.boholbeachclub.com.ph

Shet, gusto kong umuwi!!!

Adding git-svn support from source

allan — Sun, 25 Jan 2009 16:17:52 +0000

Having workstations where you don’t have root access either means contacting support for installation or building your own software from source to get the latest version.

I started using git for code produced in my work. The build was successful with a simple “./configure; make ; make install” series of steps except for supporting access to subversion repositories. It was looking for the perl module SVN::Core to be able to function successfully. Googling about it will land you to the Alien::SVN CPAN module page. Its dependencies can be installed with the standard “install Module::Name” invocation in the CPAN shell. But the main package does not properly install in this environment. It is probably because of the tarball not containing the standard Makefile.PL. It has Build.PL instead. This script generates the Build that compiles the subversion library and its bindings. Then it generates a Makefile from Makefile.PL in the src/subversion/subversion/bindings/swig/perl/native directory. Below is the output of the script:

[Alien-SVN-1.4.6.0]$ ./Build
Running make
Running make swig-pl-lib
make: Nothing to be done for `swig-pl-lib'.
Running /usr/bin/perl Makefile.PL INSTALLDIRS=site
Writing Makefile for SVN::_Core
Writing Makefile.client for SVN::_Client
Writing Makefile.delta for SVN::_Delta
Writing Makefile.fs for SVN::_Fs
Writing Makefile.ra for SVN::_Ra
Writing Makefile.repos for SVN::_Repos
Writing Makefile.wc for SVN::_Wc
Running make
gcc -c  -I$HOME/local/include/apr-0
...

The command /usr/bin/perl Makefile.PL INSTALLDIRS=site generates a build environment to install in /usr. This is not favorable for installation in userspace since you do not have permission to write on that directory. So this command will be rerun /usr/bin/perl Makefile.PL PREFIX=$USERDIR, where $USERDIR is the destination directory you want to.

Now you can successfully clone subversion repositories!