groovy: how to print the classpath

Found this really helpful blog post that got me printing out the classpath in one of my Groovy classes. I was debugging why my logback.groovy configuration wasn't getting picked up (because, inded it wasn't on the classpath!)

From the logback FAQ:

Where should the configuration files such as logback.groovy, logback-test.xml or logback.xml be located on the classpath?

Configuration files such as logback.groovy, logback-test.xml or logback.xml can be located directly under any folder declared in the class path. For example, if the class path reads "c:/java/jdk15/lib/rt.jar;c:/mylibs/" then the logback.xml file should be located directly under "c:/mylibs/", that is as "c:/mylibs/logback.xml". Placing it under a sub-folder of c:/mylibs/, say, c:/mylibs/other/, will not work.

Here's how to print the classpath, thanks to the original poster:

def printClassPath(classLoader) {
  println "$classLoader"
  classLoader.getURLs().each {url->
     println "- ${url.toString()}"
  }
  if (classLoader.parent) {
     printClassPath(classLoader.parent)
  }
}
printClassPath this.class.classLoader

how to fix groovy console warnings

Was getting these annoying messages repeatedly in groovysh and groovyConsole:

java.util.prefs.BackingStoreException: java.lang.ClassCastException: org.apache.xerces.jaxp.DocumentBuilderFactoryImpl cannot be cast to javax.xml.parsers.DocumentBuilderFactory

Solution: delete xml-apis and xercesImpl jars from your ~/.groovy/lib directory:

enkomion [15:20:56] ~/.groovy/lib $ rm xml-apis-1.4.01.jar
enkomion [15:20:59] ~/.groovy/lib $ rm xercesImpl-2.10.0.jar

unmet dependencies in aptitude: when all else fails

A routine update of system packages on 12.04 failed with errors of a broken package due to unmet dependencies. I upgraded from 10.04 to 12.04 a month ago, and things have been working relatively well since then, but somehow something broke in postgresql:

dpkg: dependency problems prevent configuration of postgresql-9.1:
 postgresql-client-9.1 (9.1.6-0ubuntu12.04) breaks postgresql-9.1 (<< 9.1.6-0ubuntu12.04) and is installed.
  Version of postgresql-9.1 to be configured is 9.1.5-1~lucid3.
dpkg: error processing postgresql-9.1 (--configure):
 dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
                                                                                                          Errors were encountered while processing:
 postgresql-9.1
E: Sub-process /usr/bin/dpkg returned an error code (1)

So I tried to fix this with all the tools I was aware of, but to no avail, and then finally gave up and decided to postgresql altogether since I didn't need it right now.

enkomion [19:14:55] ~ $ sudo apt-get purge postgresql-common
Reading package lists... Done
Building dependency tree       
Reading state information... Done
You might want to run 'apt-get -f install' to correct these:
The following packages have unmet dependencies:
 postgresql-9.1 : Depends: postgresql-common (>= 115~) but it is not going to be installed
 postgresql-client-9.1 : Breaks: postgresql-9.1 (< 9.1.6-0ubuntu12.04) but 9.1.5-1~lucid3 is to be installed
E: Unmet dependencies. Try 'apt-get -f install' with no packages (or specify a solution).


enkomion [19:15:07] ~ $ sudo apt-get install -f 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies... Done
The following packages were automatically installed and are no longer required:
  libutouch-evemu1 libutouch-geis1 libutouch-frame1 libutouch-grail1 libtrove-java
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
  postgresql-9.1
Suggested packages:
  oidentd ident-server locales-all
The following packages will be upgraded:
  postgresql-9.1
1 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
1 not fully installed or removed.
Need to get 0 B/4,298 kB of archives.
After this operation, 5,915 kB disk space will be freed.
Do you want to continue [Y/n]? Y
dpkg: dependency problems prevent configuration of postgresql-9.1:
 postgresql-client-9.1 (9.1.6-0ubuntu12.04) breaks postgresql-9.1 (<< 9.1.6-0ubuntu12.04) and is installed.
  Version of postgresql-9.1 to be configured is 9.1.5-1~lucid3.
dpkg: error processing postgresql-9.1 (--configure):
 dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
                                                                                                          Errors were encountered while processing:
 postgresql-9.1
E: Sub-process /usr/bin/dpkg returned an error code (1)

Even the usual "sudo apt-get update" and manually trying to do "sudo apt-get purge postgresql*" didn't work.

dpkg: dependency problems prevent configuration of postgresql-9.1:
 postgresql-client-9.1 (9.1.6-0ubuntu12.04) breaks postgresql-9.1 (<< 9.1.6-0ubuntu12.04) and is installed.
  Version of postgresql-9.1 to be configured is 9.1.5-1~lucid3.
dpkg: error processing postgresql-9.1 (--configure):
 dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
                                                                                                          Errors were encountered while processing:
 postgresql-9.1
E: Sub-process /usr/bin/dpkg returned an error code (1)

I was frustrated with trying to fix this manually, and just wanted to get rid of all postgresql* packages altogether from my system. But even this wasn't happening via aptitude, and was becoming increasingly painful. Finally found the solution at http://dabase.com/e/01169/ which saved the day.

enkomion [19:20:10] ~ $ sudo dpkg --force-depends --purge postgresql*
(Reading database ... 361318 files and directories currently installed.)
Removing postgresql-server-dev-9.0 ...
Removing postgresql-client-9.0 ...
Removing postgresql-client-8.4 ...
Removing postgresql-client ...
Removing postgresql ...
Removing postgresql-common ...
 * No PostgreSQL clusters exist; see "man pg_createcluster"
Removing 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common'
Purging configuration files for postgresql-common ...
dpkg: warning: while removing postgresql-common, directory '/var/lib/postgresql' not empty so not removed.
Removing postgresql-client-9.1 ...
Removing postgresql-client-common ...
Purging configuration files for postgresql-client-common ...
Processing triggers for ureadahead ...
Processing triggers for man-db ...

enkomion [19:34:30] ~ $ sudo apt-get install -f
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libutouch-evemu1 libutouch-geis1 libutouch-frame1 libutouch-grail1 libtrove-java
Use 'apt-get autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.

All postgresql* packages finally removed without fuss :)

 So this is the solution for forcefully removing broken packages:

dpkg --force-depends --purge java-gcj-compat-dev
apt-get -f install

java: create maps the easy way with guava

Traditional:

private HashMap indexMap = new HashMap();

indexMap.put("low", 1);
indexMap.put("medium",2);

indexMap.put("high",3);

Better (via Google Guava):

ImmutableMap indexMap = ImmutableMap.builder()
            .put("low", 0) 
            .put("medium", 1) 
            .put("high", 2) 
            .build();

source

nothing like a good ol' flame war to cure the blues

Great article by Martin Fowler:
http://java.dzone.com/articles/martin-fowler-orm-hate

Crisp, cool and informative as usual. 


But then things started to warm up a little in the comments section:

Frederic Bellier replied on Wed, 2012/05/09 - 9:17pm 

Woah!!! The world has come to an end.
Martin Fowler who I usualy consider as an influencial professional is now writting misinformed articles.
Where to start? There are just too many untruth and mis-information in this article - I cannot go over them all.
But for everyone interested in this topic. ORM does not work and will never work. I have 20 years experience in this field and have build some of the largest systems around where performance is paramount. I can tell you ORM cannot possibly work becuase it automatically means the access to the data will be much slower. This is not due to the incompetence of the dev involved but because of the ill conceived architecture of ORM. I have had the top engineers who build the ORM tools from the vendors telling me this.
The idea to conect an object graph in memory and a relational data structure is simply flawed. The network inefficiencies resulting from that approach are astronomical.
Moreover, ORM is based on a false premise - that SQL is hard. In fact SQL is simple. And for everyone out there interested in computer science. LEARN SQL. It is simple fast and cannot be replaced by anything else since it is a SET language to work on Set data. Replacing on standard Set language by a multitude of proprietary languages (one for each vendors) is of course silly.
And by the way - iBatis is not an ORM tool. It is actually a very good tool - this one works. It simply allows you to populate your objects from a result set.
Putting iBatis in the same categories as Hibernate only shows MF ignorance in this topic. I guess it is time to retire.
FB

Some pretty decent and well-informed arguments, but it does smell slightly trollish. Hmmm, maybe I should put some popcorn in the microwave.


And yes, I was right! The fanboys are IRKED!!! :D

Jammer Man replied on Thu, 2012/05/10 - 11:53am  in response to:FredericBell 

In your 20 years of experience you obviously haven't learned very much at all. I'd pick MF's opinion over some garden-variety d0uche troll like yourself any day.

ROFLMAO!!! Haven't laughed this hard in a long time!!! Thank you God for tech flame wars! :D

groovy: logback configuration

Logback configuration is much easier in groovy than in xml:

import ch.qos.logback.classic.encoder.PatternLayoutEncoder
import ch.qos.logback.core.FileAppender
import ch.qos.logback.core.ConsoleAppender
import static ch.qos.logback.classic.Level.DEBUG
import ch.qos.logback.core.status.OnConsoleStatusListener


/*
 * see http://mrhaki.blogspot.in/2010/09/grassroots-groovy-configure-logback.html
 * and http://logback.qos.ch/manual/groovy.html 
 * for logback.groovy examples
 */

appender("FILE", FileAppender) 
{
    def ts = timestamp("yyyy-MM-dd'_'HH-mm-ss")
    
    file = "./logs/trybeTest.${ts}.log"
    append = true
    encoder(PatternLayoutEncoder) 
    {
        pattern = "%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"
    }
}


appender("CONSOLE", ConsoleAppender) 
{
    //append=true
    encoder(PatternLayoutEncoder) 
    {
      pattern = "%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n"
    }
}

//logger "scratchpad", DEBUG, ["CONSOLE"]
root DEBUG, ["FILE", "CONSOLE"]

upgrade from 10.04 to 12.04 "not recommended" until july

Was hoping to upgrade my Lucid Lynx laptop to Precise Pangolin today.

Though I was an unflinching Windows user before, I simply fell in love with Ubuntu (and a few other distros like Linux Mint - which happens to be derived from Ubuntu) and started using it in dual boot mode. Eventually, I became less and less dependent on the Windoze drug. Once the withdrawal symptoms (couldn't play my favorite PC games) passed, I realized I didn't need Win at all.

I got rid of my dual-boot and deployed Lucid Lynx and Mint Julia on my work and personal laptops, respectively and there's been no looking back. I only needed Win7 once in the last year, and that was when I had to do some WinPhone7 development+testing at work. (And perhaps a few times more when I gave in to my gaming craving :P)

Lucid has been an awesome, fast, dependable companion, but I heard really good stuff about PP since its recent release and I eagerly wanted to give it a test drive.

However, just when I was about to plug in an external drive for a pre-upgrade backup, I read this on the official upgrade docs:

Upgrading from Ubuntu 10.04 LTS to Ubuntu 12.04 LTS

It is generally recommended that users of Ubuntu 10.04 LTS wait until the first point release, due in July, before upgrading.

Furthermore, seems that there was a major bug in upgrading from LL to PP from CD where one of the Ubuntu developers has this to say:

It's OK for you to try it, but we aren't going to encourage it because
in our experience the extra testing from early-adopter upgraders before
.1 is important in making sure that upgrades are really solid for
everyone by the time we turn it on by default.

Oh well. A couple more months to go before I get PP goodness on my lappy. But I will try a PP live boot from pen drive some time soon.

bash: simple script to send alerts if service is down

Here's a simple shell script I cooked up to alert me if my REST service went down for any reason. This was never meant for a production system (where we have nagios and other robust alerting mechanisms) but for an integration/sandbox environment. An external team (subcontractor) was integrating with our REST service deployed on this sandbox and we couldn't afford to have downtime, or the team's productivity would suffer.

It worked like a charm, took just 30 mins to write, test and deploy. Since the external REST service was deployed on a public IP [otherwise how could the external team reach it - we couldn't allow them to VPN in], this script could potentially run from any machine anywhere.

Saving this script here since it was so simple and useful.

#!/bin/bash
#
# sandbox_sanity_check.sh: a simple utility to alert relevant folks if our integration (i.e. sandbox-ext) Trybe Service goes down
# sandbox-ext is in the /etc/hosts file
targetbox=sandbox-ext
# who should we send alert mails to?
recipients="ambar@xyz.com, rakesh@xyz.com, sandeep@xyz.com"
while [ 1 ] 
do
    logfile=sandbox.`date '+%A'`.log

    curl --silent "http://${targetbox}/trybe/v1/config/TEST_067e6162-3b6f.2L_20k_60k?uid=%7B%22aid%22:%22889835751ebf3e49%22%7D&api_key=shared_key&api_nonce=8nk9pbnhacfvgc&api_ts=1333042920376&channel_id=1&api_sig=aba00fdd0058e00111b286c6356f2a70" | grep "trialConfig"

    if [ $? -eq 0 ] 
    then
        echo "[INFO] [`date '+%d_%m_%Y_%H-%M-%S'`] getConfig succeeded " | tee -a ${logfile} ; echo;  
    else
        echo "[ERROR] [`date '+%d_%m_%Y_%H-%M-%S'`] getConfig FAILED... here is the curl output:" | tee -a ${logfile} ; echo; 
        curl -v "http://${targetbox}/trybe/v1/config/TEST_067e6162-3b6f.2L_20k_60k?uid=%7B%22aid%22:%22889835751ebf3e49%22%7D&api_key=shared_key&api_nonce=8nk9pbnhacfvgc&api_ts=1333042920376&channel_id=1&api_sig=aba00fdd0058e00111b286c6356f2a70" >> ${logfile} 2>&1
        echo | tee -a ${logfile};
# send alert email to $recipients using good ol' mutt
        mutt -s "[sandbox checker]: getConfig FAILED" ${recipients}  < /var/local/sandbox.mail.message 
    fi  

    sleep 30
done

The Ten Minute Test Plan

Great idea from James Whittaker, Test Director at Google. Inspiring read, want to see if we can put this to good use:
http://goo.gl/Tocgs

SQA: The Oracle Problem

Testing involves examining the behaviour of a system in order to discover potential faults.

The problem of determining the desired correct behaviour for a given input is called the Oracle Problem. 

Since manual testing is expensive and time consuming there has been a great deal of work on automation and part automation of Software Testing. However, the problem of automating the Oracle remains a bottleneck that inhibits progress in increased automation for test effectiveness and efficiency.


source

structured, semistructured, unstructured data

Structured data is data that is organized into entities that have a defined format, such as XML documents or database tables that conform to a particular predefined schema. This is the realm of the RDBMS.

Semi-structured data, on the other hand, is looser, and though there may be a schema, it is often ignored, so it may be used only as a guide to the structure of the data: for example, a spreadsheet, in which the structure is the grid of cells, although the cells themselves may hold any form of data.

Unstructured data does not have any particular internal structure: for example, plain text or image data.

Source:  Hadoop: The Definitive Guide

how to get the android_id from an avd/emulator

Just a couple of commands take care of this. What's nice is that this can be automated from the shell, and can thus be included in scripts and in CI (e.g. in Jenkins)

[16:23:59] ~ $ adb -s emulator-5556 pull  /data/data/com.android.providers.settings/databases/settings.db
194 KB/s (24576 bytes in 0.123s)

[16:38:59] ~ $ sqlite3 settings.db "select value from secure where name='android_id'"
9774d56d682e549c

The output of the sqlite3 command is the required android_id:  9774d56d682e549c

source

"dependency injection" is such a big phrase for a simple idea

Dependency Injection is a 25-dollar term for a 5-cent concept.
Dependency injection means giving an object its instance variables. Really. That's it.

This is the best explanation of DI that I've yet heard.

source: James Shore

fix mp3 replay gain (AKA normalize) quick and easy

Some things are SO much easier on Linux than Windoze. Don't need an audio editor, even an open-source one like audacity to perform this routine task on Ubuntu.

sudo apt-get install mp3gain

dir=/music/EckhartTolle-StillnessSpeaks
 
# fix replay gain for all songs in $dir
for i in `ls -1 $dir`; do mp3gain -r -k $i; done

where:
-r: radio mode: apply track gain automatically
-k: prevent audio clipping

install sun-java6-jdk on lucid x64

I tried the standard repo/method:

sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
sudo apt-get update
sudo apt-get install sun-java6-jdk

which did work for me on my laptop. But I couldn't get this to work, for the life of me, on my lucid x64 server, for weird reasons:

Package sun-java6-jre is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package sun-java6-jre has no installation candidate

I had to resort to a custom ppa for sun-java6-jdk that worked just fine
https://launchpad.net/~ferramroberto/+archive/java

source: http://ubuntuguide.net/install-sun-java-6-jrejdk-from-ppa-in-ubuntu-11-04

svn: how to ignore certain directories recursively

Wanted to use something like the .gitignore method to recursively ignore all files and folders inside a certain path, which in my case is "troy/RobosoftWrapperTest/branches/Build06/Test/robotium/bin"

Here's how you do it (thanks to SO) in svn:

svn propset --recursive svn:ignore '*' troy/RobosoftWrapperTest/branches/Build06/Test/robotium/bin  

Here's how you can confirm/view the changed properties:

$ svn propget svn:ignore -R
RobosoftWrapperTest/branches/Build06/Test/robotium/bin - *

RobosoftWrapperTest/branches/Build06/Test/robotium/bin/res/drawable-ldpi - *

RobosoftWrapperTest/branches/Build06/Test/robotium/bin/res/drawable-mdpi - *

RobosoftWrapperTest/branches/Build06/Test/robotium/gen - *

[...]

"mimeomorphic" actions?

Discovered a brilliant site on the art and science of software testing: developsense.com

Encountered a new word while reading about the nature of testing on this blog: "mimeographic."

The usual define: mimeomorphic google query didn't yield the usual dictionary/wikipedia definition, so had to look around a bit more.

Google searches led to connections to AI and perhaps even phenomenology! The terms, "mimeomorphic action" and "polimorphic action" seem to have been coined in a computer science book:


Collins, H. M.; Kusch, M., (1998) The Shape of Actions: What Humans and Machines Can Do , Cambridge, Mass: MIT Press.

This book develops and explores the idea of polimorphic and mimeomorphic actions in great detail following on from the initial distinction which was explored in Artificial Experts and then called regular actions and machine-like actions. Cases dealt with include bicycle riding, love-letter writing, writing in general, McDonalds and the mechanisation of air-pumps.

(source)

More info from scrounging around the Net:

What can humans do? What can machines do? How do humans delegate actions to machines? 

Polimorphic actions (such as writing a love letter) are ones that community members expect to vary with social context. Mimeomorphic actions (such a swinging a golf club) do not vary. Although machines cannot act, they can mimic mimeomorphic actions. Mimeomorphic actions are thus the crucial link between what humans can do and what machines can do.

(source)

 And finally from the point-of-contact website itself (developsense.com):

In The Shape of Actions, Collins and Kusch describe key differences between two kinds of intentional human actions that they call mimeomorphic and polimorphic. In both words, “morph” refers to shape, or form. “Mimeo-” refers to copying. (The grey-haired among us may remember that stencil printing machines used to be called mimeographs.) Mimeomorphic actions are actions that we want to do the same way every time, almost as though we were machines. Collins and Kusch use the example of a golf swing, a kind of action in which we want to eliminate variation and emphasize precision, regularity, and smoothness. “Poli-” is a pun, referring to two similar-sounding Greek roots. The Greek word polys refers to many, much, or several. The Greek polis—a different word entirely— literally means the city, so Collins and Kusch use “poli-” to emphasize the collective and diversified nature of human actions. Polimorphic actions are naturally and appropriately variable, and are rooted in social and human interactions and goals. Conversation is a canonical example of polimorphic action.

(source)

soapui: logging request and response to a file

Note: this code snippet only works within a groovy script assertion for a test step. It will not work for a groovy script test step itself, because messageExchange is not available in a script test step :P

def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )

def d = new Date()
newdate = d.format("yyyy-MM-dd'T'HH-mm-ss")


def myOutFile = "/home/ambar/.loadui/ambar/TrybeSoapUITest." + newdate + ".log"
def f = new File(myOutFile)

f.write("STARTING LOG...\n\n", "UTF-8")

def rawRequest = new String( messageExchange.getRawRequestData() )
f.append("\n\n\n[INFO] RAW Request Data: " + rawRequest)

/*context.currentStep.properties.each { key, value ->
  f.append("${key} : ${value.value}\n")
}*/

//f.append("\n\n\n")

//def requestHeaders = messageExchange.getRequestHeaders()
//f.append("[INFO] request headers: " + requestHeaders)

f.append("\n\n[INFO] response: " + messageExchange.getResponseContent() )
f.append("\n\n[INFO] time taken: " + messageExchange.timeTaken)

assert messageExchange.timeTaken < 10000

nginx: simple proxy_pass usage

Problem: send incoming requests containing "/trybe" in the URL on the "enkomion" hostname to the  Trybe REST server locally hosted on tomcat (which is listening on port 8080).

Example:
input="http://enkomion/trybe/..."
output="http://localhost:8080/trybe/..."
(simply have to convert enkomion to localhost:8080 in the hostname part of the URL, but only for /trybe/ location)

Solution:

server{

        ...

        server_name localhost enkomion

        ...

        location /trybe {     
  proxy_pass http://pts;
        }

        ...

}