Archive for the 'Theoretical Hacks' Category

Behavior-driven development of research code

I’ve been reading a lot of papers on optimizing DAG-based workflows and supporting papers on scheduling for my research. Most of the time, I just characterized existing software code on existing systems. This is my first time writing scheduler code from scratch. I started by programming by wishful thinking but got stumped on how a scheduler workflow actually looks like.

In parallel, I have been reading the The Cucumber Book for use in a startup I’m bootstrapping with some friends. I thought that maybe it can help my graduate thought process as well. So I decided to drive my scientific-methodish thinking on a Gherkin file. So here it goes:

Feature: Pipeline workflow optimizations
  We hypothesize that a optimizing a pipeline will have a better load balance
  than a data-aware scheduler.

  Scenario: Load balance comparison of DAG optimization and data-aware scheduler
    Given A pipeline workload with parametarized data
    When We obtain the load from a data-aware scheduler
    And We obtain the load from the DAG-optimized version
    Then DAG-optimized load is more balanced

Here I used the “Then” clause to describe my hypothesis and the “Given” and “When” clauses to describe the experiment that attempts to verify the hypothesis. In my BDD thought process based on the chapter, “Working with Legacy Code”, if “Then” succeeds we accept the hypothesis else we reject the hypothesis and write new Given and When steps.

If you take a look at my Git commit history, I wrote a lower-level .feature file first that describes how a data-aware schedule should work. Taking a short walk and looking back at my Gherkin features, my ‘stakeholder hat’ started to kick in when I read the features again. Hopefully I’m Cuking from the outside correctly.

Why do research in the Philippines?

Because installing grid computing middleware can get you to this:

7th PANDA Grid Workshop, Bohol, Philippines, May 4 – 8, 2009
organised by
Ateneo de Manila University
Sponsored also by EPSRC, IoP, PPARC and the Royal Society of Edinburgh
The aim of the workshop is to bring together grid administrators and software developers in an informal setting, involving open discussions. The focus will include grid maintenance and monitoring and data production with PandaRoot.

Organising committee:

    Rafael P. Saldana (Ateneo)
    Kilian Schwarz (GSI)
    Dan Protopopescu (Glasgow)

Contact person:


    Holy Name University,
    Lesage and Gallares Streets,
    6300 Tagbilaran City,
    Bohol, Philippines

Let’s look at the itinerary:

Tagbilaran City (May 3, 4, 5)

Metro Centre Hotel and Convention Center
Pres. Carlos P. Garcia Avenue
Tagbilaran City, Bohol
Philippines, 6300

Panglao Island (May 6, 7, 8, 9, 10)

Bohol Beach Club
Bo. Bolod, Panglao Island, Bohol 6340

Shet, gusto kong umuwi!!!

Adding git-svn support from source

Having workstations where you don’t have root access either means contacting support for installation or building your own software from source to get the latest version.

I started using git for code produced in my work. The build was successful with a simple “./configure; make ; make install” series of steps except for supporting access to subversion repositories. It was looking for the perl module SVN::Core to be able to function successfully. Googling about it will land you to the Alien::SVN CPAN module page. Its dependencies can be installed with the standard “install Module::Name” invocation in the CPAN shell. But the main package does not properly install in this environment. It is probably because of the tarball not containing the standard Makefile.PL. It has Build.PL instead. This script generates the Build that compiles the subversion library and its bindings. Then it generates a Makefile from Makefile.PL in the src/subversion/subversion/bindings/swig/perl/native directory. Below is the output of the script:

[Alien-SVN-]$ ./Build
Running make
Running make swig-pl-lib
make: Nothing to be done for `swig-pl-lib'.
Running /usr/bin/perl Makefile.PL INSTALLDIRS=site
Writing Makefile for SVN::_Core
Writing Makefile.client for SVN::_Client
Writing for SVN::_Delta
Writing Makefile.fs for SVN::_Fs
Writing Makefile.ra for SVN::_Ra
Writing Makefile.repos for SVN::_Repos
Writing Makefile.wc for SVN::_Wc
Running make
gcc -c  -I$HOME/local/include/apr-0

The command /usr/bin/perl Makefile.PL INSTALLDIRS=site generates a build environment to install in /usr. This is not favorable for installation in userspace since you do not have permission to write on that directory. So this command will be rerun /usr/bin/perl Makefile.PL PREFIX=$USERDIR, where $USERDIR is the destination directory you want to.

Now you can successfully clone subversion repositories!

Chicago Startup Factory

The event is a collaboration between the GSB and CS Department. The group hopes to create technology-heavy startups and businesses unlike when you gather a bunch of pure business people who can’t make a business plan other than canned food, a network of juice/ shake stands, etc. The speaker for the Startup Factory talk was Adarsh Arora, CEO of Athena Security and Co-Founder of Lisle Technology Partners. I took some of his striking ideas about innovating and generating business plans around technology:

  • never sell more than one innovation – his rationale for this was that the market cannot catch-up with all of your ideas. I have not thought of this deeply because [1] I have yet to have a really brilliant idea, and [2] most busines models I saw are too caught up in selling this one unique idea that they don’t bother to look at the other types (probably they are bad ideas in the first place).
  • interdisciplinary collaboration – now this is more familiar to my school of thought. As what we always say in the Ateneo Innovation Center, today’s problems are so complex that you need to apply every type of paradigm to be able to attack the problem from different angles and come up with a brilliant solution.

Adarsh also discussed four types of companies [1] wishful thinking (you have enough deep connections to get angel funding), [2] historical precedence – selling technology to improve a process, [3] intuitive jump – pure luck; with democratization of technology, YouTube and Ebay became a big thing even though video sharing and online auctions were almost non-existent web services during their time, and [4] sure technology – you know that there is a need for it in the future (e.g. Y2K “bug”).

Follow-up events to this is an Entrepenuerial Brainstorming Session with GSB and CS students and an Introduction to creating application on the iPhone. Apple’s development platform makes it so easy for anyone to distribute an app and sell it over iTunes (or AppleStore?) enabling you to earn several thousand dollars in a few months.

Oh, and they had free pizza during the talk 🙂

Great Chicago Book Sale

I quickly grabbed my bike after coming from a seminar class and arrived 10 minutes before the closing time! Within a short span of time and by relying on my semi-rare impulsiveness of buying, I got these two titles foer 5 USD (buy-one-take-one):

W. T. Welford, Useful Optics (Chicago Lectures in Physics). University Of Chicago Press, October 1991.

Students and professionals alike have long felt the need of a modern source of practical advice on the use of optical tools in scientific research. Walter T. Welford’s _Useful Optics_ meets this need. Welford offers a succinct review of principles basic to the construction and use of optics in physics. His lucid explanations and clear illustrations will particularly help those whose interests lie in other areas but who nevertheless must understand enough about optics to create the experimental apparatus necessary to their research. Consistently emphasizing applications and practical points of design, Welford covers a host of topics: mirrors and prisms, optical materials, aberration, the limits of image formation and resolution, illumination for image-forming systems, laser beams, interference and interferometry, detectors and light sources, holography, and more. The final chapter deals with putting together an experimental optics system. Many areas of the physical sciences and engineering increasingly demand an appreciation of optics. Welford’s _Useful Optics_ will prove indispensable to any researcher trying to develop and use effective optical apparatus. Walter T. Welford (1916-1990) was professor of physics at Imperial College of Science, Technology and Medicine from 1951 until his death. He was a Fellow of the Royal Society and of the Optical Society of America.  Link to []

T. P. Hughes, Human-Built World: How to Think about Technology and Culture (science * culture).    University Of Chicago Press, May 2005.

To most people, technology has been reduced to computers, consumer goods, and military weapons; we speak of “technological progress” in terms of RAM and CD-ROMs and the flatness of our television screens. In Human-Built World, thankfully, Thomas Hughes restores to technology the conceptual richness and depth it deserves by chronicling the ideas about technology expressed by influential Western thinkers who not only understood its multifaceted character but who also explored its creative potential.

Hughes draws on an enormous range of literature, art, and architecture to explore what technology has brought to society and culture, and to explain how we might begin to develop an “ecotechnology” that works with, not against, ecological systems. From the “Creator” model of development of the sixteenth century to the “big science” of the 1940s and 1950s to the architecture of Frank Gehry, Hughes nimbly charts the myriad ways that technology has been woven into the social and cultural fabric of different eras and the promises and problems it has offered. Thomas Jefferson, for instance, optimistically hoped that technology could be combined with nature to create an Edenic environment; Lewis Mumford, two centuries later, warned of the increasing mechanization of American life.

Such divergent views, Hughes shows, have existed side by side, demonstrating the fundamental idea that “in its variety, technology is full of contradictions, laden with human folly, saved by occasional benign deeds, and rich with unintended consequences.” In Human-Built World, he offers the highly engaging history of these contradictions, follies, and consequences, a history that resurrects technology, rightfully, as more than gadgetry; it is in fact no less than an embodiment of human values. Link to []

Even information can be found in the UChicago Press site.

Technology for the masses’ sari-saris

The sari-sari store is a key economic and social installation in Filipino communities.  It allows the community easy access to basic good. The term sari-sari is Filipino for “various kinds”.  See more from its wikipedia entry.

Mark Ruiz of Hapinoy shares some updates on their company deploying a POS system for these sari-sari store. Technology is not only a key component in streamlining systems of today’s enterprises but it is also needed by people like Nanay Delia. Below is the image from his blog post:

Nanay Delia learning a Point-of-Sale System for her Sari-Sari Store. Whatever doubts of the applicability of technology for the smallest unit of retail were laid to rest after the training session. The potential of becoming more efficient in pricing controls, inventory management, and accounting were key benefits that were very much appreciated. [description from Mark Ruiz blog.

Midwest Grid School 2008 @UChicago

Yey! Malapit na malapit lang 🙂 More information about the program and registration information is found in its OSG Page. Workshop dates from Sept 17-19.

The Open Science Grid (OSG), the TeraGrid and the Computation Institute of The University of Chicago present a three day intensive course in grid computing and its application to scientific discovery.

The course introduces the techniques of grid and distributed computing for science and engineering fields, with hands-on training in the use of national grid computing resources. The workshop introduces essential skills that will be needed by researchers in the natural and applied sciences, engineering, and computer science to conduct and support large-scale computation and data analysis in emerging grid and distributed computing environments.

School participants will work with grid computing experts during the 3-day training. The workshop will focus on enabling the use of the national cyberinfrastructure –The Open Science Grid and TeraGrid– to perform large-scale computations and data-intensive processing in the your field of research. Participants will learn to use grids of thousands of processors and will be able to continue to use these resources for their research after the class. We encourage you to bring your research project to us for suggestions and help in porting your application to the grid. We would like to offer you support in transitioning your application to this platform.