Jordan Mendler’s Blog

We all want to use open-source tools whenever possible, but I simply got fed up with OpenOffice being so buggy that it would take me hours to make a decent presentation that would not display properly on other systems. Then, of course, there are the crappy Linux UML tools which come nowhere close to OmniGraffle.

Unfortunately, the only things worse than these Linux tools, are Mac OSX and Windows. So what is the solution? A hybrid approach of running Mac OSX through VMware on my Linux workstation when I need to use OmniGraffle, and running Microsoft Office through Wine. Surprisingly, the whole thing actually works really well and could be setup in only a few hours.

I hope OpenOffice and Dia could catch up, but in the meantime here are the links if anyone is interested:

• Office 2007 on Ubuntu 9.10
• Mac OSX 10.6 on VMware 7
• And of course, http://btjunkie.org will be handy.

Screenshot from my dual-monitor Ubuntu workstation:

Read the rest of this entry »

1) Create a file containing all of your jobs. For example I will call it /home/jordan/array.jobs.in:

echo foo > /home/jordan/1
gzip /home/jordan/1
echo bar > /home/jordan/2
gzip /home/jordan/2

2) Submit your array job, changing your jobfile to the proper file below. You can also add SGE options between the qsub and -t, such as -N name, -q all.q or whatever you desire:

jobfile=/home/jordan/array.jobs.in ; echo ‘$(head -n $SGE_TASK_ID’ $jobfile ‘| tail -n 1)’ | qsub -t 1-$(cat $jobfile | wc -l)

NOTE: The above line contains single-quotes, not back-ticks.

For testing, bmatches should take only one or two minutes for 6000 reads on 8-cores.

time bfast match -r /scratch0/tmp/jordan/solexa/truncated/bfast.verysmall.fastq -f /scratch0/bfast/genomes/custom/hg18.chrX.31000000-38000000/hg18.fa -n 8 > /scratch0/tmp/jordan/bfast_test_2009-12-30.test/expected.verysmall

# Server
- Add the following to /etc/gdm/custom.conf:

[xdmcp]
Enable=true
DisplaysPerHost=10 # Maximum number of sessions per remote host

- Restart GDM (sudo service gdm restart)

# Client
- sudo apt-get install tsclient xnest
- start tsclient from command line or Internet->Terminal Server Client from start menu
- Enter IP/hostname of the remote computer and select XDMCP as your protocol
- Hit connect and away you go!

# Preface:
After installing Ubuntu, the graphical interface did not work, and I had no way of getting out of the command line interface. Trying to load GDM would simply result in a blank screen. Searching the net did not provide anything useful, so I had to do the following to get graphics working.

# Install Jockey to fetch the proprietary drivers
sudo apt-get install jockey-gtk

# See what drivers are available
sudo jockey-text -l
xorg:nvidia-173 - NVIDIA accelerated graphics driver (Proprietary, Disabled, Not in use)
xorg:nvidia-185 - NVIDIA accelerated graphics driver (Proprietary, Disabled, Not in use)

# Tell Jockey to enable the properiatry drivers
sudo jockey-text -e xorg:nvidia-185

# Write an xorg.conf file that uses proprietary driver
sudo nvidia-xconfig

# Restart GDM
sudo service gdm restart

# From Gnome you may want to reconfigure the driver if you have a non-standard setup, such as dual monitors:
sudo nvidia-settings

In case cpan2dist isn’t working out for RPM’s, try cpan2rpm:

# Installation of cpan2rpm
wget http://ftp.arix.com/cpan2rpm-2.028.tar.gz
tar -xvzf cpan2rpm-2.028.tar.gz
cd cpan2rpm-2.028
./cpan2rpm –mk-rpm-dirs=~/rpm

# Patch cpan2rpm: Recent version of Pod::Text no longer inherit from Pod::Parser, so use Parser instead, by applying the following patch
perl -pi -e ’s/Pod::Text/Pod::Parser/’ ./cpan2rpm-2.028/cpan2rpm

# Build an RPM
cpan2rpm Parallel::ForkManager

# Deb: Ubuntu/Debian
sudo apt-get install build-essential
cpan CPANPLUS::Dist::Deb

# Build and install the packages of your choosing
sudo cpan2dist –format CPANPLUS::Dist::Deb –buildprereq –install Parallel::ForkManager
sudo cpan2dist –format CPANPLUS::Dist::Deb –install Cache::Memcached::libmemcached # Don’t
build prereqs and skip tests

# RPM: CentOS, RedHat, etc
cpan CPANPLUS::Dist::RPM
sudo cpan2dist –format CPANPLUS::Dist::RPM –buildprereq –install Parallel::ForkManager

# Caveats
## Ignore Lists: Sometimes modules are provided by other modules, so an ignore list is needed. An example is the following modules that are provided by pathtools and the base perl distribution.
Cwd
Test
Test::Harness
Test::More
File::Path
File::Spec

sudo cpan2dist –verbose –skiptest –ignorelist /home/jordan/.cpanplus/ignore_jordan –format CPANPLUS::Dist::Deb –buildprereq –install CPANPLUS::Dist::RPM

Sample /etc/sysconfig/iptables

# Generated by iptables-save v1.2.9 on Wed Oct 22 12:47:53 2008
*nat
:PREROUTING ACCEPT [15869337:1053832946]
:POSTROUTING ACCEPT [606720:36324459]
:OUTPUT ACCEPT [985251:59398276]
-A POSTROUTING -o eth0 -j MASQUERADE
COMMIT
# Completed on Wed Oct 22 12:47:53 2008
# Generated by iptables-save v1.2.9 on Wed Oct 22 12:47:53 2008
*filter
:INPUT ACCEPT [200298:12118535]
:FORWARD ACCEPT [12139545091:14012344684963]
:OUTPUT ACCEPT [16669982531:17010307011265]
COMMIT
# Completed on Wed Oct 22 12:47:53 2008

Set IP Forwarding until reboot:

echo 1 > /proc/sys/net/ipv4/ip_forward

Enable Forwarding permanently (after reboot):

echo “net.ipv4.ip_forward = 1″ >> /etc/sysctl.conf

From the command line:

$ pfexec pkg install SUNWipkg
$ pkg image-update -v
$ reboot

Preface:
After receiving a call from the folks at Gluster, I am making another attempt at prototyping a GlusterFS storage cluster. Here are some of the steps I am taking to do.

Build the RPM

# Install Fuse:
yum -y install fuse-devel byacc libtool
# Make sure you install the needed dependencies by looking at the spec file
# Build throws BDB errors, so skip it. It also has a weird default setup with apache 1.3, so just skip that too for now.
rpmbuild -ta glusterfs-2.0.0rc1.tar.gz –without bdb –without modglfs

Install the RPM on both client and server nodes

yum -y install fuse
rpm -Uvh glusterfs-2.0.0rc1-1.i386.rpm

Start the server on all of your nodes (see below for sample vol files)

mkdir /state/partition1/gluster_export # or whatever your export dir is
glusterfs -f /home/jordan/projects/gluster/glusterfs-server.vol

On client nodes, install mounting requirements

yum -y install fuse dkms-fuse # From rpmforge
modprobe fuse # Load fuse module

Mount gluster on the client nodes

mkdir /mnt/gluster
mount -t glusterfs /home/jordan/projects/gluster/glusterfs-client.vol /mnt/gluster/

Basic Configuration for Sample Server spec

# Configuration for each server to export storage
volume posix
type storage/posix
option directory /state/partition1/gluster_export
end-volume

volume locks
type features/locks
subvolumes posix
end-volume

volume brick
type performance/io-threads
option thread-count 4 # Each of my test servers is 4 core
subvolumes locks
end-volume

volume server
type protocol/server
option transport-type tcp
option auth.addr.brick.allow *
subvolumes brick
end-volume

Basic Configuration for Sample Client spec

# Sample client to aggregate 8 storage nodes
volume r1
type protocol/client
option transport-type tcp
option remote-host compute-0-1
option remote-subvolume brick
end-volume

volume r2
type protocol/client
option transport-type tcp
option remote-host compute-0-2
option remote-subvolume brick
end-volume

volume r3
type protocol/client
option transport-type tcp
option remote-host compute-0-3
option remote-subvolume brick
end-volume

volume r4
type protocol/client
option transport-type tcp
option remote-host compute-0-4
option remote-subvolume brick
end-volume

volume r5
type protocol/client
option transport-type tcp
option remote-host compute-0-5
option remote-subvolume brick
end-volume

volume r6
type protocol/client
option transport-type tcp
option remote-host compute-0-6
option remote-subvolume brick
end-volume

volume r7
type protocol/client
option transport-type tcp
option remote-host compute-0-7
option remote-subvolume brick
end-volume

volume r8
type protocol/client
option transport-type tcp
option remote-host compute-0-8
option remote-subvolume brick
end-volume

volume distribute
type cluster/distribute
subvolumes r1 r2 r3 r4 r5 r6 r7 r8
end-volume