[0xCAFEFEED]

Backward compatibility is a challenge for everyone from the guy writing the kernel to guy building the application.

Raymond Chen has written a lot of anecdotes on compatibility issues in windows. (See Compatibility Constraints and Handling Compatibility Hacks.) He has also authored a book with the same title as that of his blog: The old new thing, which I am sure will have even more interesting tidbits.

Even application developers have to worry about the API they expose and its backward compatibility. The fine people at viewvc believed the URL scheme used by viewvc is part of their "API" and had a thorough discussion on what changes could be made to the URL scheme before and after the 1.0 release!

Being in the middle tier, we seem to face more than our fair share of compatibility issues. First, the product is based on the specifications which are written in English. Each vendor is free to interpret it the way they want. Eventually this leads to customer queries like: "But this works in [plug the customer's current vendor name here]!" This results in new flags in the product configuration files. Over the years we have brought up such ambiguities in the expert groups and the newer versions of the specs clarify the interpretation.

The other category of issues arise from ensuring compatibility with the innumerable combinations of databases, drivers, operating systems and JVMs. JV talks about an issue we recently faced wherein upgrading the VM from 1.4.2_10 to 1.4.2_11 breaks our product. JDK 1.4 introduced a new feature of disconnected sockets. The new implementation was supposed to be backward compatible. However, the implementation had a bug (2126509 ). In the process of fixing this bug in 1.4.2_11, (I believe) they introduced a new bug which breaks backward compatibility.

Such explicit problems are easier to manage than the unwritten contracts. While debugging I rely heavily on the exceptions raised. For example when you cast an object to a type, if there is a ClassCastException, the message generally has the class name of the object. So, in your code, if you are casting to java.util.List and the exception message is java.util.String, you know you are casting a String to a List. Also if you are casting to foo.bar.Klass and the message name is also foo.bar.Klass, then it is a class loading issue. The object you have loaded is of the same class but loaded from a different classloader. Looking at the exception message I could figure out whether it was a genuinely wrong cast or a ClassLoader issue. Unfortunately in JDK 1.4 ClassCastExceptions do not have a message. For a developer this is an annoyance, but is it worth logging a bug?

Here is a sample output with different jdks:

D:\code\41>java -cp PS30\tp\classes ClassCastTest
1.5.0_06
java.lang.ClassCastException: java.lang.String
        at ClassCastTest.main(ClassCastTest.java:25)

D:\code\41>java -cp PS30\tp\classes ClassCastTest
1.4.2_10
java.lang.ClassCastException
        at ClassCastTest.main(ClassCastTest.java:25)

D:\code\41>java -cp PS30\tp\classes ClassCastTest
1.3.1_16
java.lang.ClassCastException: java.lang.String
        at ClassCastTest.main(ClassCastTest.java:25)

Would you write unit tests for the output messages?

  ¶ 1:43 AM | (0) comments  | trackbacks

Deepak has finally started blogging at Deep into Java. He has been sharing some neat tricks like Bootstrapping static fields within enums. Thanks for the tips buddy ... look forward to more of them.  ¶ 1:30 AM | (1) comments  | trackbacks

I booted my laptop early this morning to get my daily dose of Google alerts. Navigating through the alerts I ended up at: The Museum of Modern Betas and browsing through its entries I chanced upon Google's firefox extension for detecting phishing: Safe Browsing. While I was going through their site, I noticed the yahoo notification window show "Deeps is now online" and I was thinking to myself "What is this guy doing online so early in the morn?" (And may I ask what are YOU doing online?!) Installing Google SafeBrowsing seems to be fraught with its own problems. The SafeBrowsing home page says that it can be installed as part of google toolbar only. Antitrust I say! Some sites say the download is available only in US. Further googling revealed the URL: http://dl.google.com/firefox/google-safebrowsing.xpi

While I was installing the plugin (firefox waits for a couple of seconds before enabling the install button ... i wonder why?!), I see a message from deeps:

Deeps: http://www.geocities.com/junebug585

...and he logs out. I promptly clicked on the link which showed the page:

"Hmmm ... password" I say. I ping deeps ..."Dood ... it's asking for password". No response. Being a stickler for online security (ask my wife on how i nag her into setting a different password for each site and make sure she does not note down her passwords anywhere!) I think to myself... "Hey! This could be a phishing site!" (what with me looking at Google SafeBrowsing site just a few mins ago). "Very well", I tell myself, "... the url is Yahoo! Geocities, the logos and the layout looks ok ... Why would deep try to phish my yahoo account details ... what the hell .. let's try to login". Key in the user id pass nothing happens. Back to Yahoo home page. "D'oh! Have I been phished?!" with this nagging doubt go have a shower ... come back... notice deeps has replied to my message:

Deeps: what login?
Deeps: did u a get any message from me?
Deeps: i did not send it..

"Oh God! I have been phished!! Change the password quickly." Chirpy wifey: "Breakfast time!" "Oh God! Imagine after all my taunts ... I've been phished ... how am I going to tell her this?! Have I really been phished?!" So I decide to go back to the site and verify. LiveHTTPHeaders shows:

http://www2.fiberbit.net/form/mailto.cgi
POST /form/mailto.cgi HTTP/1.1
Host: www2.fiberbit.net
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://www.geocities.com/junebug585/?200614
Content-Type: application/x-www-form-urlencoded
Content-Length: 138
Mail_From=GOD&Mail_To=jawsy1%40gmail.com&Mail_Subject=Gift&Next_Page=http%3A%2F%2Fwww.yahoo.com
&.pd=fpctx_ver%253d0&login=asdf&passwd=asdf

HTTP/1.x 302 Moved Temporarily
Date: Thu, 14 Sep 2006 04:01:30 GMT
Server: Apache/1.3.26 (Unix) mod_perl/1.26
Location: http://www.yahoo.com
Content-Type: text/html; charset=iso-8859-1
X-Cache: MISS from downloads.pramati.com
X-Cache-Lookup: MISS from downloads.pramati.com:3128
Connection: close

"Oh no! I really have been phished! Bugger has mailed my yahoo password to himself!! Change the password ... change the password ... change the password!". Wifey, annoyed after waiting for me at the b'fast table: "Wot'chu doin buster?!" ... where do I hide my face now?!

The phisher is not Deeps... some one phished his id and sent me the message. Maybe it is not even junebug585 whose geocities site was used to phish for my account details ... maybe that id was also phished and misused. Maybe it is not even jawsy1@gmail.com where my userid/pass have been mailed ... maybe that id/pass also have been phished?! I wonder who all will receive a message from my id now and be phished!

Imagine the coincidence, I am finicky abt internet security, I was just checking out Google's SafeBrowsing and I got phished! "I took a chance typing my yahoo id, surely I wouldn't have taken a chance if the site asked for my bank account-nos/user-ids" I rationalize. But still ... what if the phisher downloaded all my password reminder mails from my yahoo mailbox??!!

All the best dad!

... now the painful part of reporting abuse to yahoo and gmail ...

---

Update: Looks like I am not the first! ... and looks like google pages are also being used for phishing. Google: yahoo geocities phishing

---

Update 1: Yahoo!'s soln for phishing?! (via deeps):

---------- Forwarded message ----------
From: Kalyan K Kumar
Date: Sep 14, 2006 11:04 AM
Subject: phishing
To: sammelan

keep an eye on those yahoo login look alike geocities links. don't enter
yahoo password anywhere
other than login.yahoo.com
you can setup a sign in seal to protect partly.

http://protect.login.yahoo.com/

  ¶ 9:01 AM | (7) comments  | trackbacks

Rakesh sent me a mail the other day asking why the finalize method is protected. Well, here's what I think:

The finalize method is invoked by the JVM/GarbageCollector on Objects which are no longer referenced. So, if I were the guy who designed the finalize method, ideally I would want it to be private. Just like why the writeObject method of the Serializable interface is private. These methods are not meant to be called by other user objects, these methods are invoked only by the JVM runtime classes only. So it would make sense to make them private, and have special handling in the VM to invoke these methods. This is ok in case of the writeObject method, however, making finalize method private would have other implications.

Assume we had decided to go ahead and allow the finalize method to be private. Consider the following classes

public class SuperClass
{
    private HeavyResource resource = new HeavyResource();
 
    private void finalize()
        throws Throwable
    {
        resource.shutdown();
    }
}

public class SubClass
    extends SuperClass
{
    private AnotherResource another = new AnotherResource();
 
    public SubClass(){
        super(); //Added for clarity
    }

    private void finalize()
        throws Throwable
    {
        another.shutdown();
    }
}

Since SubClass extends the SuperClass, when we create an instance of SubClass, we would also have created an instance of HeavyResource and an instance of AnotherResource. However, when this instance of SubClass is being finalized, we shutdown only the instance of AnotherResource. The shutdown method of HeavyResource would not be called.

The recommended practice for finalize methods is that in the finally block of the finalize method one should call the finalize method of the super class. (That's right, in the constructor the first statement has to be the call to super so for destructor/finalize the sequence should be reversed and super should be called last). So our SubClass would look something like:

public class SubClass
    extends SuperClass
{
    private AnotherResource another = new AnotherResource();
 
    public SubClass(){
        super(); //Added for clarity
    }

    private void finalize()
        throws Throwable
    {
        try{
            another.shutdown();
        }finally{
            super.finalize();//Compilation error here
        }
    }
}

This would have worked, but unfortunately this won't even compile. You can not call the private method of the super class. There are couple of ways I can think of to solve this problem.

One solution of-course would be to relax our requirements and make the access modifier of the finalize method protected and hope people are sensible enough not to call it!

Another solution could be to allow calls to super.finalize(), even if the finalize method of the super class has private access modifier, in the Java Language Specification (JLS).

My preferred solution would be to have compilers automatically add a try finally block and insert a call to super.finalize() method in the finally block. (Modifying byte code to add a try finally block is a nightmare (as compared to adding a call to super()), but that's a separate discussion!) This would be similar and consistent with the way compilers add the call to super() as the first statement of a constructor if it does not already exist. (You can use javap -c ClassName to look at the byte code generated by the compiler, but I prefer to use JClasslib.)

The Java language specification, (3rd ed, Section 12.6) does mention that the call to super.finalize is not injected automatically and provides a hint as to why:

The fact that class Object declares a finalize method means that the finalize method for any class can always invoke the finalize method for its superclass. This should always be done, unless it is the programmer's intent to nullify the actions of the finalizer in the superclass. (Unlike constructors, finalizers do not automatically invoke the finalizer for the superclass; such an invocation must be coded explicitly.)

It appears this was done ``So that the programmer can nullify the actions of the finalizer in the superclass.'' But thanks to this choice, developers today use tools like PMD which warn them about empty finalize methods and when the finalize does not call the same method of the super class.

Update (16 Jan 2007): It appears that the designers of C# learnt from Java's mistakes and decided to make constructor and destructor symmetric. In C#, the destructor of the super-class is called whether the destructor of the sub-class was successfuly completed or not. From "Section 16.3: How exceptions are handled" of the C# language specification 1.2:

Exceptions that occur during destructor execution are worth special mention. If an exception occurs during destructor execution, and that exception is not caught, then the execution of that destructor is terminated and the destructor of the base class (if any) is called. If there is no base class (as in the case of the object type) or if there is no base class destructor, then the exception is discarded.

  ¶ 11:30 AM | (2) comments  | trackbacks

[or JavaScript chorma-coding]

While posting Java sources on this blog, I used to wish that the browsers could take care of chroma-coding Java sources just the way they take care of chroma-coding XML files. It would eliminate the hassle of exporting the sources in HTML format. All one would have to do is paste the Java source in a pre block and viola! you are done with it!

Inspired by the "live" pages on Blogger, Feed Burner and GMail, I decided to experiment and see if JavaScript and CSS could be used to achieve this. After couple of hours of struggling and about 200 lines of re-factored code, I managed to get it to work!

Below is the HTML colorized version of ClassParser.java from the BCEL sources. Check out the curly braces ... they are live! Clicking on them will show or hide the enclosed block! Now for the surprise ... if you look at the HTML source for the above part, you will see only plain java source. All the colorization and parenthesis matching has been done in jscc.js, jsccJAVA.js and jscc.css referred to in the head of the HTML file. Pretty neat huh?!

package org.apache.bcel.classfile;

/* ====================================================================
 * The Apache Software License, Version 1.1
 *
 * Copyright (c) 2001 The Apache Software Foundation.  All rights
 * reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions
 * are met:
 *
 * 1. Redistributions of source code must retain the above copyright
 *    notice, this list of conditions and the following disclaimer.
 *
 * 2. Redistributions in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in
 *    the documentation and/or other materials provided with the
 *    distribution.
 *
 * 3. The end-user documentation included with the redistribution,
 *    if any, must include the following acknowledgment:
 *       "This product includes software developed by the
 *        Apache Software Foundation (http://www.apache.org/)."
 *    Alternately, this acknowledgment may appear in the software itself,
 *    if and wherever such third-party acknowledgments normally appear.
 *
 * 4. The names "Apache" and "Apache Software Foundation" and
 *    "Apache BCEL" must not be used to endorse or promote products
 *    derived from this software without prior written permission. For
 *    written permission, please contact apache@apache.org.
 *
 * 5. Products derived from this software may not be called "Apache",
 *    "Apache BCEL", nor may "Apache" appear in their name, without
 *    prior written permission of the Apache Software Foundation.
 *
 * THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
 * WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
 * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
 * DISCLAIMED.  IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION OR
 * ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
 * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
 * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
 * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
 * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 * SUCH DAMAGE.
 * ====================================================================
 *
 * This software consists of voluntary contributions made by many
 * individuals on behalf of the Apache Software Foundation.  For more
 * information on the Apache Software Foundation, please see
 * ;.
 */

import  org.apache.bcel.Constants;
import  java.io.*;
import  java.util.zip.*;

/***
 * Wrapper class that parses a given Java .class file. The method 
 * parse returns a 
 * JavaClass object on success. When an I/O error or an
 * inconsistency occurs an appropiate exception is propagated back to
 * the caller.
 *
 * The structure and the names comply, except for a few conveniences,
 * exactly with the 
 * JVM specification 1.0. See this paper for
 * further details about the structure of a bytecode file.
 *
 * @version $Id: ClassParser.java,v 1.5 2002/08/09 13:09:31 mdahm Exp $
 * @author M. Dahm 
 */
public final class ClassParser {
  private DataInputStream file;
  private ZipFile         zip;
  private String          file_name;
  private int             class_name_index, superclass_name_index;
  private int             major, minor; // Compiler version
  private int             access_flags; // Access rights of parsed class
  private int[]           interfaces; // Names of implemented interfaces
  private ConstantPool    constant_pool; // collection of constants
  private Field[]         fields; // class fields, i.e., its variables
  private Method[]        methods; // methods defined in the class
  private Attribute[]     attributes; // attributes defined in the class
  private boolean         is_zip; // Loaded from zip file

  private static final int BUFSIZE = 8192;

  /***
   * Parse class from the given stream.
   *
   * @param file Input stream
   * @param file_name File name
   */
  public ClassParser(InputStream file, String file_name) {
    this.file_name = file_name;

    String clazz = file.getClass().getName(); // Not a very clean solution ...
    is_zip = clazz.startsWith("java.util.zip.") || clazz.startsWith("java.util.jar.");

    if(file instanceof DataInputStream) // Is already a data stream
      this.file = (DataInputStream)file;
    else
      this.file = new DataInputStream(new BufferedInputStream(file, BUFSIZE));
  }

  /*** Parse class from given .class file.
   *
   * @param file_name file name
   * @throws IOException
   */
  public ClassParser(String file_name) throws IOException
  {    
    is_zip = false;
    this.file_name = file_name;
    file = new DataInputStream(new BufferedInputStream
			       (new FileInputStream(file_name), BUFSIZE));
  }

  /*** Parse class from given .class file in a ZIP-archive
   *
   * @param file_name file name
   * @throws IOException
   */
  public ClassParser(String zip_file, String file_name) throws IOException
  {    
    is_zip = true;
    zip = new ZipFile(zip_file);
    ZipEntry entry = zip.getEntry(file_name);
  		   
    this.file_name = file_name;

    file = new DataInputStream(new BufferedInputStream(zip.getInputStream(entry),
						       BUFSIZE));
  }

  /***
   * Parse the given Java class file and return an object that represents
   * the contained data, i.e., constants, methods, fields and commands.
   * A ClassFormatException is raised, if the file is not a valid
   * .class file. (This does not include verification of the byte code as it
   * is performed by the java interpreter).
   *
   * @return Class object representing the parsed class file
   * @throws  IOException
   * @throws  ClassFormatException
   */  
  public JavaClass parse() throws IOException, ClassFormatException
  {
    /******************* Read headers ********************************/
    // Check magic tag of class file
    readID();

    // Get compiler version
    readVersion();

    /******************* Read constant pool and related **************/
    // Read constant pool entries
    readConstantPool();
	
    // Get class information
    readClassInfo();

    // Get interface information, i.e., implemented interfaces
    readInterfaces();

    /******************* Read class fields and methods ***************/ 
    // Read class fields, i.e., the variables of the class
    readFields();

    // Read class methods, i.e., the functions in the class
    readMethods();

    // Read class attributes
    readAttributes();

    // Check for unknown variables
    //Unknown[] u = Unknown.getUnknownAttributes();
    //for(int i=0; i < u.length; i++)
    //  System.err.println("WARNING: " + u[i]);

    // Everything should have been read now
    //      if(file.available() > 0) {
    //        int bytes = file.available();
    //        byte[] buf = new byte[bytes];
    //        file.read(buf);
    
    //        if(!(is_zip && (buf.length == 1))) {
    //  	System.err.println("WARNING: Trailing garbage at end of " + file_name);
    //  	System.err.println(bytes + " extra bytes: " + Utility.toHexString(buf));
    //        }
    //      }

    // Read everything of interest, so close the file
    file.close();
    if(zip != null)
      zip.close();

    // Return the information we have gathered in a new object
    return new JavaClass(class_name_index, superclass_name_index, 
			 file_name, major, minor, access_flags,
			 constant_pool, interfaces, fields,
			 methods, attributes, is_zip? JavaClass.ZIP : JavaClass.FILE);
  }

  /***
   * Read information about the attributes of the class.
   * @throws  IOException
   * @throws  ClassFormatException
   */
  private final void readAttributes() throws IOException, ClassFormatException
  {
    int attributes_count;

    attributes_count = file.readUnsignedShort();
    attributes       = new Attribute[attributes_count];

    for(int i=0; i < attributes_count; i++)
      attributes[i] = Attribute.readAttribute(file, constant_pool);
  }

  /***
   * Read information about the class and its super class.
   * @throws  IOException
   * @throws  ClassFormatException
   */
  private final void readClassInfo() throws IOException, ClassFormatException
  {
    access_flags = file.readUnsignedShort();

    /* Interfaces are implicitely abstract, the flag should be set
     * according to the JVM specification.
     */
    if((access_flags & Constants.ACC_INTERFACE) != 0)
      access_flags |= Constants.ACC_ABSTRACT;

    if(((access_flags & Constants.ACC_ABSTRACT) != 0) && 
       ((access_flags & Constants.ACC_FINAL)    != 0 ))
      throw new ClassFormatException("Class can't be both final and abstract");

    class_name_index      = file.readUnsignedShort();
    superclass_name_index = file.readUnsignedShort();
  }    
  /***
   * Read constant pool entries.
   * @throws  IOException
   * @throws  ClassFormatException
   */
  private final void readConstantPool() throws IOException, ClassFormatException
  {
    constant_pool = new ConstantPool(file);
  }    

  /***
   * Read information about the fields of the class, i.e., its variables.
   * @throws  IOException
   * @throws  ClassFormatException
   */
  private final void readFields() throws IOException, ClassFormatException
  {
    int fields_count;

    fields_count = file.readUnsignedShort();
    fields       = new Field[fields_count];

    for(int i=0; i < fields_count; i++)
      fields[i] = new Field(file, constant_pool);
  }    

  /********************* Private utility methods **********************/

  /***
   * Check whether the header of the file is ok.
   * Of course, this has to be the first action on successive file reads.
   * @throws  IOException
   * @throws  ClassFormatException
   */
  private final void readID() throws IOException, ClassFormatException
  {
    int magic = 0xCAFEBABE;

    if(file.readInt() != magic)
      throw new ClassFormatException(file_name + " is not a Java .class file");
  }    
  /***
   * Read information about the interfaces implemented by this class.
   * @throws  IOException
   * @throws  ClassFormatException
   */
  private final void readInterfaces() throws IOException, ClassFormatException
  {
    int interfaces_count;

    interfaces_count = file.readUnsignedShort();
    interfaces       = new int[interfaces_count];

    for(int i=0; i < interfaces_count; i++)
      interfaces[i] = file.readUnsignedShort();
  }     
  /***
   * Read information about the methods of the class.
   * @throws  IOException
   * @throws  ClassFormatException
   */
  private final void readMethods() throws IOException, ClassFormatException
  {
    int methods_count;

    methods_count = file.readUnsignedShort();
    methods       = new Method[methods_count];

    for(int i=0; i < methods_count; i++)
      methods[i] = new Method(file, constant_pool);
  }      
  /***
   * Read major and minor version of compiler which created the file.
   * @throws  IOException
   * @throws  ClassFormatException
   */
  private final void readVersion() throws IOException, ClassFormatException
  {
    minor = file.readUnsignedShort();
    major = file.readUnsignedShort();
  }    
}

The scripts haven't been written for performance. I just wanted to keep my effort to the minimum! I have used Regular Expressions extensively. One of the tricky parts was to identify that "mailto:markus.dahm@berlin.de" occurring  in a comment as part of the comment and not to highlight it as a String literal and at the same time identify /* hello */ occurring within a string literal as a part of the string literal and not highlight it as a comment! [Did I say that right?! :)] The solution was quite straight forward though. Instead of searching first for comments and then for string literals, I had to create a regular expression which matched both!

One optimization which I haven't figured out how to implement in JavaScript is to download only jscc.js by default and dowload jsccJAVA.js if ONLY if there is atleast one pre element with the className as jsccJAVA. Any pointers and suggestions in this direction are appreciated!

There are quite a few limitations of this implementation. Character literals with unicode escapes are not supported. IE converts the HTML tags inside javadocs to upper case. The HTML file is no longer XHTML compliant. Also, I have tested it only on FireFox[0.8+] and IE [5.5+].

If you want to use the scripts, add the following lines to the <head>...</head> part of your html and paste your java code between <pre class="jsccJAVA"></pre>

<script type="text/javascript" src="http://www.me.umn.edu/~shivane/jscc/jscc.js" />
<script type="text/javascript" src="http://www.me.umn.edu/~shivane/jscc/jsccJAVA.js" />
<link rel="stylesheet" type="text/css" href="http://www.me.umn.edu/~shivane/jscc/jscc.css" />

  ¶ 12:34 PM | (8) comments  | trackbacks