A new beginning..

After a bit over two and a half years at WSO2, I am starting my own consulting practice today as an independent consultant! The past couple of years has been both very interesting and challenging.

I worked mostly on the Apache Synapse Enterprise Service Bus (ESB) and built the WSO2 branded version of it as WSO2 ESB, and took both of these through many releases. The development of the non-blocking http/s transport was the most challenging problem we solved, that allowed Synapse to scale to support thousands of concurrent connections utilizing a small thread pool of workers. The NIO transport now supports constant memory throttling, utilizing some of the very cool features of the Apache HttpComponents/NIO. I learn't a lot about the complexities of NIO and the issues with the HTTP/S protocol from Oleg Kalnichevski who contributes immensely to HttpComponenets. I've also worked on implementing the JMS transport, the Mail transport and the File system transport for Synapse and Axis2, and also built an abstract transport framework that allows one to write a new transport with minimal effort.

Ruwan, Indika, Chathura, Upul, Evanthika, Chanaka, Saliya and Dinuka were my team mates from WSO2, and both Sanjiva and Paul helped me immensely during these years to reach my targets. We beat both proprietary ESB's as well as other open source ESB's during three rounds of performance testing we carried out, and these tests have now become a defacto standard for ESB's, followed by both BEA (now Oracle) and Mule who ran the same tests.

I will continue to work on Apache Synapse and be available for consulting, training and development assistance around Apache Synapse, open source middleware, enterprise Java and performance testing and tuning. Thus, I have setup my consulting partnership http://adroitlogic.org which is officially starting today.. This is the first time that I am leaving permanent employment after working for over 10 years for IBM Sri Lanka, the National Institute of Business Management, Virtusa (Nasdaq:VRTU) and WSO2.. but I am confident that I will reach my objectives, while enjoying more free time with my kids and family.

So I leave my boats behind
Leave them on familiar shores
Set my heart upon the deep
Follow you again, my Lord!
[http://blog.360.yahoo.com/blog-EEadyvswaKcb8ZXz15O_?p=1]

Why doesn't an executor reuse a thread after completing your task?

This could happen if your task throws a RuntimeException.. then the Java Executor will not reuse that thread and would create a new thread. Consider the following example, if you comment the line "throw new RuntimeException("Runtime");" you will get the following output:

This is some thread : TG-ID-1
This is some thread : TG-ID-2
...
This is some thread : TG-ID-20
This is some thread : TG-ID-20
This is some thread : TG-ID-20
This is some thread : TG-ID-20
...
...
This is some thread : TG-ID-20
..
This is some thread : TG-ID-2
This is some thread : TG-ID-1

But if you leave the RuntimeException, your outpur will be:
This is some thread : TG-ID-1
...
This is some thread : TG-ID-71
This is some thread : TG-ID-72
This is some thread : TG-ID-73
...
etc..

import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;

public class TestExecutor {

    public static void main(String[] args) {
        new TestExecutor().test();
    }

    private void test() {
        BlockingQueue blockingQueue = new LinkedBlockingQueue();
        Executor executor = new ThreadPoolExecutor(
            20, 100, 5,
            TimeUnit.SECONDS,
            blockingQueue,
            new NativeThreadFactory(new ThreadGroup("My-T-grp"), "TG-ID"));

        for (int i=0; i<100; i++) {
            executor.execute(new Runnable() {
                public void run() {
                    System.out.println("This is some thread : " + Thread.currentThread().getName());
                    throw new RuntimeException("Runtime");
                }
            });
        }
    }


    public class NativeThreadFactory implements
        ThreadFactory {

        final ThreadGroup group;
        final AtomicInteger count;
        final String namePrefix;

        public NativeThreadFactory(final ThreadGroup group, final String namePrefix) {
            super();
            this.count = new AtomicInteger(1);
            this.group = group;
            this.namePrefix = namePrefix;
        }

        public Thread newThread(final Runnable runnable) {
            StringBuffer buffer = new StringBuffer();
            buffer.append(this.namePrefix);
            buffer.append('-');
            buffer.append(this.count.getAndIncrement());
            Thread t = new Thread(group, runnable, buffer.toString(), 0);
            t.setDaemon(false);
            t.setPriority(Thread.NORM_PRIORITY);
            return t;
        }

    }
}

Major bug with Ohloh makes it almost useless?!

Apache Synapse became a Top Level Project (TLP) of the Apache Software Foundation (ASF) a few months back, and thus we had to change our SVN url. Although our SVN contains history spanning back to almost three (3) years back, Ohloh only sees the project as only 5 months old!

I have complained about this to Ohloh 4 months back, at which time they stated

..We are unable to follow development activity across branches or directory moves..As far as I can tell there is no workaround. This affects a lot of projects, and I desperately want to get this fixed, but it will take some time.

Like they claimed, this affects many many projects! For example check Apache HttpComponents, ServiceMix, MINA..etc! This list will go on and on, since each new project started at the ASF will first go through incubation and then graduation, followed optionally by a move to become a TLP, and at each of these stages, the SVN url could possibly change.

Since the code metrics reported for so many of thes open source projects are just plain wrong, how can any of the information presented by ohloh be useful to anyone?

I sure hope someone at ohloh recognises the importance of this, and fixes this ASAP!

Limiting permissions to Java programs with policies

I wanted to limit the permissions for a Java program to only connect to a certain host.. and the way to do this is by defining your own policy file, and passing this as an argument to the Java command as follows:

temp.policy
grant {
//permission java.net.SocketPermission "69.147.112.160:80","connect,resolve";
permission java.io.FilePermission "<>", "read, write, delete, execute";
permission java.util.PropertyPermission "*", "read, write";
permission java.lang.RuntimePermission "*";
permission java.lang.reflect.ReflectPermission "*";
};

The above policy will not grant the process any SocketPermissions, however, if you only wanted it to connect to a specific IP address or hostname, you can uncomment the first line, and edit it appropriately.

For more information refer to: http://java.sun.com/developer/onlineTraining/Programming/JDCBook/appA.html

Now to start your program using the above policy file, use:

java -cp some.jar -Djava.security.manager -Djava.security.policy=temp.policy MainClass