So will the Sun JDK be known as the Oracle JDK in the future?

I guess Oracle could infuse some of the JRockit tricks into the Sun JVMs too.. But I wish IBM bought Sun!

http://www.oracle.com/sun/index.html

Sun's Garbage First Collector Largely Eliminates Low Latency/High Throughput Tradeoff

Can't wait to try this with JDK 6 u14!

In a recent podcast, James Gosling highlights the importance of G1 for certain kinds of large-scale Java applications, such as financial exchanges, which are characterised by large amounts of live heap data and considerable thread-level parallelism, and are often run on high-end multi-core processors:

"...the deep hidden secret about many of these Java apps is that they don't really use databases. Instead of databases they use huge amounts of RAM and push the garbage collector like mad because they cannot afford to touch the disk ever. When you are doing many, many thousands of transactions per second it's all about keeping everything in RAM, using hash tables, getting as many cores focused on the transactions as possible, and they usually have big issues about transaction latency."

More details: http://www.infoq.com/news/2009/04/g1

How do you effectively communicate with a Free and Open Source community (FOSS) of users and developers, to get the great support you need?

It ain't rocket science, but many people people may still be new to the world of Free and Open Source software! Many enterprise architects and development leads come from an environment where they have been paying large amounts of money to commercial software vendors, fooled by nicely attired sales folks who promises dreams, but deliver nightmares. Then sometime later you end up realizing that you are stuck into a well crafted vendor lock-in, and attempts to free your self sometimes makes you realize that the support teams of the commercial vendors rarely knows the code they are talking about to support you in the first place. They would call you within a hour of you raising an issue, just to ensure the SLA is adhered.. but sometimes you have no option when they fail to deliver a solution, and they say that the feature will be available next year, or when you upgrade to the next version, or when you buy the additional feature pack! In the worst case, you may know whats wrong or find a trivial bug, but you never see the code to fix it yourself in an hour, and have to wait for weeks for the next service pack! and ain't that fun?

FOSS - No free beer if you are drunk!
Welcome to the world of Free and Open Source software development for a difference.. but don't expect free beer if you are drunk!

Don't be rude or demand answers
You may be forced to using an open source solution since your company can no longer afford the exorbitant license fees of the commercial equivalents.. but in the world of FOSS, you cannot "demand" free support, or think that rude behavior or comments will yield you the results you need by frightened developers. The FOSS projects are driven around close groups of developers and users, and at the Apache Software Foundation, we even value community over the code.

We treat everyone equally, and don't care if you are a big shot
You may be a very intelligent and highly regarded employee of your company, and even be its CTO.. but we don't really care at all about your title! We treat all individuals the same, irrespective of their title, country or age.

Do not underestimate the intelligence of others, you may be disappointed
Never underestimate the intelligence of others in the community whom you do not know about, they could probably be much more experienced and a lot smarter than you in that domain, especially if they are contributing developers.

Do not expect to be bottle-fed, or to be read nursery rhymes
They are not obliged to answer all of your questions, or in great detail when they do, or write everything down with great documentation.. they are only obliged to share the code freely with the whole world, so that you could also fix up anything wrong, and contribute back to the project.

Consider supporting the project
If you want to improve the documentation, consider hiring a technical writer on your payroll.. If some new feature is critical to you alone, see if you could sponsor its development. Try to help as much as you can with testing.. especially when nearing a release

Remember, you are not paying for the free support
The developers are not paid by the project to provide you support, and they do it voluntarily in their own free time. If you want better response times, you are free to select a vendor that offers typical commercial support over the project. Remember this when you write an email to a free mailing list asking for help, and is about to write something rude

If its broken, fix it
If something is broken, help fix it. If the documentation is not complete, help improve it. If there seems to be a bug in the code, raise an issue with all required information to reproduce and fix the issue - if you are smart to solve it your self, fix it and send a patch, and show your true colors. When you write an email to a mailing list, show a little courtesy, attach files instead of pasting large amounts of unreadable code into the message body, use a new subject line when relevant, and don't keep on pasting new mail on top of old mailing threads.

Don't feel shy.. and show a little respect
Don't feel shy to appreciate help and say 'Thank You'. And to conclude, show a little respect, and see others respecting you back.

PS: More links
I've found a golden list of rules to start with Open Source here [http://schlitt.info/opensource/blog/0541_10_golden_rules_for_starting_with_open_source.html] and a good document on "How To Ask Questions the Smart Way" here [http://linuxmafia.com/faq/Essays/smart-questions.html] for those who still might need more material

Getting back the iPhone!

My first generation iPhone running firmware 1.1.4 hacked to work in Sri Lanka, started to give trouble from a few months back, and taking almost a day of being kept switched off to magically recover! I never understood why a 8hr switch off was different to a 10 minute switch off.. but then again, people even suggest keeping the iPhone in the freezer to make it recover :).. I only tried that for 5 minutes..

Anyway on the 10th of February, I upgraded to the new version 2.2.1 firmware.. and immediately realized I had done something really bad.. by Wifi was not working at all.. I then tried down grading to 2.2 - but still no luck.. then I tried going back to v.1.1.4 by even installing an old iTunes version and downloading the ipsw image files.. but trying quite a lot of hints, I was never able to get my modem firmware downgraded.. it had got to 04.05.04_G and I was stuck! Then things got even worse.. the phone would not ring.. but silently record missed calls.. then I gave up everything, and was waiting for someone to find a fix patiently.. I realized that the phone does ring, when its kept unlocked - but that soon drains the battery.. another option was keeping it connected to the charger.. so I did both of these until today the 6th of March..

.. and today, I felt lucky! suddenly.. I felt my phone ring again!.. and I tried it a few times to make sure.. and yes, it has started to work again :).. then I tried my Wifi.. and thats back too! I hope this will help someone else who is like me to get back their iPhones.. the secret being patience! I'm happy to have a real ringing phone back with me.. and I will not try any more upgrades for a loooong time.. its currently running 2.2.1(5H11), Carrier 2.9 and Modem firmware 04.05.04_G

The Secrets of Housing Loans and Taxation

Ever wondered how much a housing loan would really help you get your OWN house, AND save you tax? Well I've been looking for answers on this topic from many years ago, but was only able to fully figure out everything once I got my own housing loan and tax file some years ago.. All this time I have been secretly giving out information to my friends on this, and today I am going to make it publicly available..

First for some history on this topic, I wrote a tiny Java Script EMI calculator few years back, and its still available on my GeoCities website, which I haven't updated for quite sometime :)

I've also blogged about "How to Get a Loan in 3 days" previously, and would recommend reading it for anyone who is considering this in Sri Lanka

So now for the goodies..
The worksheet has been made available in M Excel format - to support those who still pays Microsoft for software provided free by many others, and is much better most of the times.. It allows you to first key in your EMI details - i.e. how much you want to borrow, how much interest the financial institution is asking for, and the duration of the loan, and then you can calculate your monthly EMI (Equated Monthly Installment)

Search Google for more information on what EMI is and means - its a very fair scheme, where you pay interest for what you really get in a loan. Each month you pay the same amount, but during the start of the repayment, most of this money is counted as interest, and a little as capital repayment, and towards the end of the loan, most will be counted as capital repayment, and less as interest. Thus if you take an EMI loan for 20 years, and pay it for 3 years and want to exit, you still would have to pay back a large sum of the amount you've borrowed - but hey - its a FAIR calculation.. a bank is not your relative, to give you free money!

Next, enter your salary details, and see how much you will save as Tax. BUT this comes at a price.. you will need to open a Tax File and ask for a salary direction to your employer, so that PAYE tax will not be deducted from you each month.

A Tax file is like a Driving License in Sri Lanka, they never expires :).. so thereafter, each year you WILL have to send your tax information to the Inland Revenue Department. These people sometimes even ask people who has died to keep filing tax as well, probably to cover up the massive VAT fraud the officials of this department helped themselevs into. But then again, officials in Sri Lanka rarely resign, even AFTER they are found guily by the supreme court!.. but we know of one who had to bite the dust finally!

So finally .. here is the goodie I promised.
http://people.apache.org/~asankha/personal/Salaries_and_Tax_DEMO.xls

Please let me know if you have any corrections, or suggestions for improvement.. and feel free to share it with others, but please do respect the copyright notice and the name of the original author.

Dead men may not tell tales.. But dead JVMs do!

During the last couple of days, I was troubleshooting a critical server crash encountered by an airline, on behalf of a software consulting firm which requested my help. The symptoms were that WebSphere 6.0.2 was crashing with an Out of Memory (OOM) condition, sometimes reaching 100% (or a very high) CPU utilization level in addition. Although you can profile an application with excellent profilers such as JProbe, it becomes much more difficult when you are unable to re-create the issues in any other environment but they keep happening on the live production instances, to which you only have limited access.

Luckily the IBM JVM's generate a Portable Heap Dump (PHD) on an OOM, and has an array of extremely helpful tools to analyze information from heap/thread dumps and and other information offline. Thus it is still possible to detect memory leaks in applications, even when they only occur in live production systems. Sometimes the cause would be heap fragmentation, where even if a considerable percentage of memory is still available, a contiguous chunk of the size required cannot be freed.



The above image shows the IBM Heap Analyzer, detecting a memory leak by the WebSphere DRS / Session Replication, where 923MB of heap has been consumed by 14,011 HashMap#Entry objects held onto by the WebSphere Data Replication Service, used for HTTP session replication.

It is also interesting to look for the use of Xalan 2.6.0 by any application code, as I have at earlier instances found memory leaks that are typically more difficult to trace - but which occurs primarily due to a well known bug in 2.6.0 of Xalan.

A project for Christmas 2008

During the last Christmas holidays, I wanted to use my vacation time to build a corner sofa set for use at home.. however, due to many unexpected obligations, I couldn't finally find any time for it.. So this year, I got a deadline to either build it before the 24th, or to buy it! Since I finally had to get someone else to build the pantry cupboards (which I wanted to do - if I had the time) as well, I decided not to let this opportunity pass by.. after all, I am working as an independent consultant now, and it was holidays around the world anyway!

So I started to look for a simple software to design the structure, the Google Sketchup came in very handy! I also used the measurements I took some time ago from the corner sofa set bought for the WSO2 office in Colombo. I had to spend on the fabric, wood and the foam and it cost around the equivalent of $500.. it took about a week to come into shape, and finally my wife helped with the sewing of the fabric and the cushions. I did stay up all night on the 23rd, to ensure that everything was ready by the morning of the 24th :)

Here are some pics taken during the construction.. if you need the Sketchup files, let me know


Finishing off with padding and foam..




Making the frame..



Cutting the wood.. with the use of support 'templates' to get correct edges with the circular saw..

A new beginning..

After a bit over two and a half years at WSO2, I am starting my own consulting practice today as an independent consultant! The past couple of years has been both very interesting and challenging.

I worked mostly on the Apache Synapse Enterprise Service Bus (ESB) and built the WSO2 branded version of it as WSO2 ESB, and took both of these through many releases. The development of the non-blocking http/s transport was the most challenging problem we solved, that allowed Synapse to scale to support thousands of concurrent connections utilizing a small thread pool of workers. The NIO transport now supports constant memory throttling, utilizing some of the very cool features of the Apache HttpComponents/NIO. I learn't a lot about the complexities of NIO and the issues with the HTTP/S protocol from Oleg Kalnichevski who contributes immensely to HttpComponenets. I've also worked on implementing the JMS transport, the Mail transport and the File system transport for Synapse and Axis2, and also built an abstract transport framework that allows one to write a new transport with minimal effort.

Ruwan, Indika, Chathura, Upul, Evanthika, Chanaka, Saliya and Dinuka were my team mates from WSO2, and both Sanjiva and Paul helped me immensely during these years to reach my targets. We beat both proprietary ESB's as well as other open source ESB's during three rounds of performance testing we carried out, and these tests have now become a defacto standard for ESB's, followed by both BEA (now Oracle) and Mule who ran the same tests.

I will continue to work on Apache Synapse and be available for consulting, training and development assistance around Apache Synapse, open source middleware, enterprise Java and performance testing and tuning. Thus, I have setup my consulting partnership http://adroitlogic.org which is officially starting today.. This is the first time that I am leaving permanent employment after working for over 10 years for IBM Sri Lanka, the National Institute of Business Management, Virtusa (Nasdaq:VRTU) and WSO2.. but I am confident that I will reach my objectives, while enjoying more free time with my kids and family.

So I leave my boats behind
Leave them on familiar shores
Set my heart upon the deep
Follow you again, my Lord!
[http://blog.360.yahoo.com/blog-EEadyvswaKcb8ZXz15O_?p=1]

Why doesn't an executor reuse a thread after completing your task?

This could happen if your task throws a RuntimeException.. then the Java Executor will not reuse that thread and would create a new thread. Consider the following example, if you comment the line "throw new RuntimeException("Runtime");" you will get the following output:

This is some thread : TG-ID-1
This is some thread : TG-ID-2
...
This is some thread : TG-ID-20
This is some thread : TG-ID-20
This is some thread : TG-ID-20
This is some thread : TG-ID-20
...
...
This is some thread : TG-ID-20
..
This is some thread : TG-ID-2
This is some thread : TG-ID-1

But if you leave the RuntimeException, your outpur will be:
This is some thread : TG-ID-1
...
This is some thread : TG-ID-71
This is some thread : TG-ID-72
This is some thread : TG-ID-73
...
etc..

import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;

public class TestExecutor {

    public static void main(String[] args) {
        new TestExecutor().test();
    }

    private void test() {
        BlockingQueue blockingQueue = new LinkedBlockingQueue();
        Executor executor = new ThreadPoolExecutor(
            20, 100, 5,
            TimeUnit.SECONDS,
            blockingQueue,
            new NativeThreadFactory(new ThreadGroup("My-T-grp"), "TG-ID"));

        for (int i=0; i<100; i++) {
            executor.execute(new Runnable() {
                public void run() {
                    System.out.println("This is some thread : " + Thread.currentThread().getName());
                    throw new RuntimeException("Runtime");
                }
            });
        }
    }


    public class NativeThreadFactory implements
        ThreadFactory {

        final ThreadGroup group;
        final AtomicInteger count;
        final String namePrefix;

        public NativeThreadFactory(final ThreadGroup group, final String namePrefix) {
            super();
            this.count = new AtomicInteger(1);
            this.group = group;
            this.namePrefix = namePrefix;
        }

        public Thread newThread(final Runnable runnable) {
            StringBuffer buffer = new StringBuffer();
            buffer.append(this.namePrefix);
            buffer.append('-');
            buffer.append(this.count.getAndIncrement());
            Thread t = new Thread(group, runnable, buffer.toString(), 0);
            t.setDaemon(false);
            t.setPriority(Thread.NORM_PRIORITY);
            return t;
        }

    }
}

Major bug with Ohloh makes it almost useless?!

Apache Synapse became a Top Level Project (TLP) of the Apache Software Foundation (ASF) a few months back, and thus we had to change our SVN url. Although our SVN contains history spanning back to almost three (3) years back, Ohloh only sees the project as only 5 months old!

I have complained about this to Ohloh 4 months back, at which time they stated

..We are unable to follow development activity across branches or directory moves..As far as I can tell there is no workaround. This affects a lot of projects, and I desperately want to get this fixed, but it will take some time.

Like they claimed, this affects many many projects! For example check Apache HttpComponents, ServiceMix, MINA..etc! This list will go on and on, since each new project started at the ASF will first go through incubation and then graduation, followed optionally by a move to become a TLP, and at each of these stages, the SVN url could possibly change.

Since the code metrics reported for so many of thes open source projects are just plain wrong, how can any of the information presented by ohloh be useful to anyone?

I sure hope someone at ohloh recognises the importance of this, and fixes this ASAP!