upgrade from 10.04 to 12.04 "not recommended" until july

Was hoping to upgrade my Lucid Lynx laptop to Precise Pangolin today.

Though I was an unflinching Windows user before, I simply fell in love with Ubuntu (and a few other distros like Linux Mint - which happens to be derived from Ubuntu) and started using it in dual boot mode. Eventually, I became less and less dependent on the Windoze drug. Once the withdrawal symptoms (couldn't play my favorite PC games) passed, I realized I didn't need Win at all.

I got rid of my dual-boot and deployed Lucid Lynx and Mint Julia on my work and personal laptops, respectively and there's been no looking back. I only needed Win7 once in the last year, and that was when I had to do some WinPhone7 development+testing at work. (And perhaps a few times more when I gave in to my gaming craving :P)

Lucid has been an awesome, fast, dependable companion, but I heard really good stuff about PP since its recent release and I eagerly wanted to give it a test drive.

However, just when I was about to plug in an external drive for a pre-upgrade backup, I read this on the official upgrade docs:

Upgrading from Ubuntu 10.04 LTS to Ubuntu 12.04 LTS

It is generally recommended that users of Ubuntu 10.04 LTS wait until the first point release, due in July, before upgrading.

Furthermore, seems that there was a major bug in upgrading from LL to PP from CD where one of the Ubuntu developers has this to say:

It's OK for you to try it, but we aren't going to encourage it because
in our experience the extra testing from early-adopter upgraders before
.1 is important in making sure that upgrades are really solid for
everyone by the time we turn it on by default.

Oh well. A couple more months to go before I get PP goodness on my lappy. But I will try a PP live boot from pen drive some time soon.

install sun-java6-jdk on lucid x64

I tried the standard repo/method:

sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
sudo apt-get update
sudo apt-get install sun-java6-jdk

which did work for me on my laptop. But I couldn't get this to work, for the life of me, on my lucid x64 server, for weird reasons:

Package sun-java6-jre is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package sun-java6-jre has no installation candidate

I had to resort to a custom ppa for sun-java6-jdk that worked just fine
https://launchpad.net/~ferramroberto/+archive/java

source: http://ubuntuguide.net/install-sun-java-6-jrejdk-from-ppa-in-ubuntu-11-04

linux: network manager keeps removing the nameserver i set in resolv.conf!

I want to always have Google's public DNS servers (8.8.8.8, 8.8.4.4) in my /etc/resolv.conf

However my WiFi settings are set to DHCP (for both office and home) and whenever I get a DHCP address, I also get the DNS servers configured on the DHCP server. These server settings overwrite my /etc/resolv.conf and I lose the

Having to set the nameservers manually in /etc/resolv.conf on each DHCP connection is too painful, so I found a simple and effective solution to this problem:

$ sudo vim /etc/dhcp3/dhclient.conf

# ensure the following line is uncommented in dhclient.conf:

# prepend domain-name-servers 8.8.8.8,8.8.4.4;

$ sudo service network-manager restart

This configures the DHCP client to always prepend the list of DNS servers obtained from DHCP with "8.8.8.8, 8.8.4.4" - and it works like a charm!

does I3P leak memory?

I was suspecting my tomcat6 test app (I3P) was guilty of a memleak.

So I installed munin and munin-node on my box so that I could get pretty memory curves. The default munin setup doesn't plot graphs for specific processes out of the box. I eventually figured out that 'multips_memory' was the plugin for the job.

I was trying to get munin's multips_memory plugin to show me the RSS (Resident Set Size, not the other one :P - see 'man ps') of tomcat6. I wasn't getting any values in munin's multips_memory for "tomcat6" because multips_memory only checks the command name (which isn't 'tomcat' in my case). Tomcat's command line is a huge mess:

/usr/lib/jvm/java-6-sun/bin/java -Djava.util.logging.config.file=/var/lib/tomcat6/conf/logging.properties -Djava.awt.headless=true -Xmx128m -XX:+UseConcMarkSweepGC -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms512m -Xmx512m -Djava.endorsed.dirs=/usr/share/tomcat6/endorsed -classpath /usr/share/tomcat6/bin/bootstrap.jar -Dcatalina.base=/var/lib/tomcat6 -Dcatalina.home=/usr/share/tomcat6 -Djava.io.tmpdir=/tmp/tomcat6-tmp org.apache.catalina.startup.Bootstrap start 
 
The command name is thus /usr/lib/jvm/java-6-sun/bin/java which doesn't contain the string "tomcat" that I'm interested in. So that's why multips_memory couldn't find tomcat6 on my system... Great, here comes another tweakathon :P

So I went about adjusting the multips_memory code to suit my purposes:

[SNIP]
.
.
.
    ps -eo $monitor,args | gawk '
BEGIN              { total = "U"; } # U = Unknown. 
/grep/             { next; }
/'"$name"'/        { total = total + ($1*1024); }
END                { print "'"$fieldname"'.value", total; }'
done
.
.
.
[/SNIP]

What the above snippet does is match against the entire command+args of the processes against the string of interest using gawk. In other words, I'm now using "ps -eo args" instead of "ps -eo comm" which is necessary to find "tomcat6" in /usr/bin/java's arguments. Also, I changed the gawk regex match to search the entire line (not just $2 since the matching args for my search string could be $3 or $4 etc). I also made the search more inclusive by matching a substring instead of the exact string name  (removed the ^ and $ from the regular expression).

It works like a charm. Munin is now reporting beautiful (and worrying) memory graphs for my selected processes:

And then it struck me that this is pretty handy code. I sometimes need to see how much RAM a particular process is taking.

Every so often I see bad stuff about the tomcat app I'm testing:

4 Nov, 2011 12:58:23 AM com.aaa.bbb.application.modules.AppLauncherModule$1 uncaughtException SEVERE: Uncaught exception from Thread[Timer-0,5,main] java.lang.OutOfMemoryError: Java heap space

in the tomcat logs [/var/log/tomcat6/catalina.out]. The OOM forces a "sudo service tomcat6 restart" - really it must be a memleak.

So here's the result of this effort - a shell script inspired by the multips_memory munin plugin that that tells you how much memory all the instances of Java (e.g.) are consuming. See the usage() below for more details.

#!/bin/bash
#
# meminfo: A simple utility to display the memory usage of given process(es)
#
###############################################################################

# Default values of arguments:
VERBOSE=0
MEMORY_TYPE=rss
ARGS_TYPE=args
THIS_PROGRAM=$(basename $0)



usage()
{
cat << EOF

USAGE: $THIS_PROGRAM [arguments]

SUMMARY: A simple utility to display the memory consumption of given process(es) on this machine.

ARGUMENTS: (all optional)
    -h      Show this message
    -a      The type of arguments specified in 'ps -o'. Can be either 'args' or 'comm' (default: args)
                args: match against the full command name + argument 
                comm: match against the command name only
    -m      Specify memory type (default: rss) - see "man ps"
    -p      The process_string: can be simply a name or a regular expression.
                This argument is optional: if not supplied, all processes are considered.
    -v      Verbose mode: show debugging information

Each line of 'ps -e' is matched against the string using gawk: so you may have to escape special characters like '/' and ':" etc for gawk regex matching.

EXAMPLES:
$THIS_PROGRAM tomcat6
[show tomcat6 RSS memory usage]

$THIS_PROGRAM java
[show total memory usage by all java processes]

$THIS_PROGRAM -m vsz \\/usr\\/bin\\/java.*eclipse.*
[show memory VSZ taken by eclipse]

$THIS_PROGRAM ".usr.bin.java.+eclipse.+"
[simpler version of the above example]

$THIS_PROGRAM "\/usr\/bin\/java -Djava.library.path=\/usr\/lib\/jni -Dosgi.requiredJavaVersion=1.5 -XX:MaxPermSize=256m -Xms40m -Xmx512m -jar \/home\/ambar\/workspace\/tools\/eclipse\/\/plugins\/org.eclipse.equinox.launcher_1.2.0.v20110502.jar"
[very-specific command and args]

EOF
}


while getopts "hvm:a:p:" OPTION
do
    case $OPTION in
        h)
            usage
            exit 0
            ;;
        v)
            VERBOSE=1
            ;;
        a)
            ARGS_TYPE=$OPTARG
            ;;
        m)
            MEMORY_TYPE=$OPTARG
            ;;
        p)
            PROCESS_STRING=$OPTARG
            ;;
        ?)
            usage
            exit 1
            ;;
    esac
done



# another way to set default values, not needed here though
#: ${MEMORY_TYPE:=rss}
#: ${ARGS_TYPE:=args}
#: ${VERBOSE:=0}


if [[ -z $PROCESS_STRING ]]  # if no process name, then override ARGS_TYPE to args so that we calculate the FULL memory usage of all processes
then
    ARGS_TYPE=args
fi

ps -eo $MEMORY_TYPE,$ARGS_TYPE | gawk '
BEGIN                   { total = "U"; } # U = Unknown. 
/'$THIS_PROGRAM'/       { next; }
/grep/                  { next; }
/'"$PROCESS_STRING"'/   { total = total + ($1*1024); if('$VERBOSE'==1) {print "\n\t", $0; print "\tCUMULATIVE USAGE: ", total} }
END                     { mbs = total/(1024*1024); printf("\nTotal '$MEMORY_TYPE' memory used by all '$PROCESS_STRING' processes: %d bytes == %11.3f MB\n", total, mbs); }'

I learnt quite a bit: it's the first time I used getopts, basename, and integrated an awk script (that consumes bash variables) in a shell script. The whole endeavor seemed like a pointless digression at first, but now I think it was totally worth my time :)

And now it's time to show off the results of this little adventure:

[02:43:27] /var/log/tomcat6 $ meminfo -p eclipse -a args -m vsz

Total vsz memory used by all eclipse processes: 5109342208 bytes ==    4872.648 MB
[02:43:36] /var/log/tomcat6 $ meminfo -p tomcat6

Total rss memory used by all tomcat6 processes: 474861568 bytes ==     452.863 MB
[02:43:41] /var/log/tomcat6 $ meminfo 

Total rss memory used by all  processes: 4642054144 bytes ==    4427.008 MB

Ubuntu!!!

A person with ubuntu is open and available to others, affirming of others, does not feel threatened that others are able and good, for he or she has a proper self-assurance that comes from knowing that he or she belongs in a greater whole and is diminished when others are humiliated or diminished, when others are tortured or oppressed.

-- Archbishop Desmond Tutu, 1999