Emails to Android Developer list

Hi Taras,

I'm trying to run Apache Tuscany on Android as part of my Google
Summer of Code project: Incubating and Android in Delft 1. I've run
all the (modified) source code through retrotranslator but still get
"native method not implemented" errors that I believe are related to
the lack of annotations support on the current Android SDK.

To test Apache Tuscany I'm running a modified calculator project
(setup as an Android project) called calculator-android. I have
imported the Apache Tuscany SCA modules and modified code for Android
into an eclipse workspace. Then I'm running retrotranslator from the
command line as shown below. I'm keeping the target code at java 1.5
but get the annotation related errors when running the calculator-
android as an Android Application. The same thing happens when I try
to convert the code to java 1.4.

$ java -jar /../retrotranslator-transformer-1.2.6.jar -target 1.5 - 
stripannot -embed support -reflection safe -classpath /../ 
retrotranslator-android-1.2.6.jar -srcdir /../workspace/ -verbose 

Any advice you may have will be greatly appreciated. There's more
details on what I'm currently doing in 2 in case you can help.
Thanks in advance.

1 http://cwiki.apache.org/confluence/display/TUSCANYWIKI/Allow+Google+A...
2 http://cwiki.apache.org/confluence/display/TUSCANYWIKI/30may2008

Hi Oscar,

Retrotranslator does it's job by modifying compiled classes in the following
way:

1. Embedding into each class its metadata (including annotations) as one big
character string.
2. Replacing some method invocations, so java.lang.Class.isAnnotationPresent
is being replaced with
net.sf.retrotranslator.android.main.java.lang.Class.isAnnotationPresent etc.
So when you call getAnnotations() on a particular class the code from
retrotranslator-android-1.2.6.jar will parse the corresponding metadata and
return the result.

AFAIK when you run your Android project from Eclipse it both compiles your
sources into classes and the classes into a dex file. So Retrotranslator has
no chance to modify classes in between unless you fix the ADT plugin. The
workaround is not to put your sources into Eclipse and to process it with
javac, Retrotranslator and jar manually. After that you may add the jar to
Eclipse.

If you execute the following line you should obtain mytuscany-android.jar
that is one-third bigger then the original mytuscany.jar and does not
contain references to Java 5 reflection methods.

java -jar retrotranslator-transformer-1.2.6.jar -srcjar mytuscany.jar 
-destjar mytuscany-android.jar  -target 1.5 -reflection safe -stripannot 
-classpath retrotranslator-android-1.2.6.jar 

So the line java.lang.Class.isAnnotationPresent(Class.java:1131) in the
stack trace means that the application is based on non-translated binaries.
Maybe you had successfully processed your classes but Eclipse overwrote them
after that or Eclipse had already created the dex file before you executed
Retrotranslator.

If you're using Eclipse the most elegant way would be to add the translation
step to the ADT plugin source code, but otherwise don't include Tuscany
sources into the project, only include a jar processed by Retrotranslator
and retrotranslator-android-1.2.6.jar.

Cheers,
Taras.

Hi Taras,

Thank you so much for the detailed response.

I think you're right that when I run the Android project from Eclipse it both compiles the sources into classes and the classes into a dex file. I also agree that the most elegant way would be to add the translation step to the ADT plugin source code, thus enabling users to run Retrotranslator on the sources from within Eclipse. In some cases, such as mine, this is preferred over manually translating libraries that contain annotations and then adding them to the the main project (for instance, to simplify debugging).

There is a workaround to prevent classes that have already been translated by Retrotranslator to be recompiled from source. The workaround is to disable the Java Builders from within each project's properties. I tried this and no longer receive the Annotation error:

java.lang.Class.isAnnotationPresent(Class.java:1131)

However, I'm getting the error shown below (and listed on the full stack trace in 1). I took care to include retrotranslator-android-1.2.6.jar as an external library to all Tuscany projects, including calculator-android, as you pointed out. Additionally, I confirmed through the timestamps on class files and dex file that retrotranslated classes are not being recompiled and that the dex file was generated a few minutes after running retrotranslator.

java.lang.NoClassDefFoundError: net.sf.retrotranslator.android.main.java.lang._Class

I've tried this running retrotranslator as shown below, setting the target to java 1.5 at first and then to 1.4.

$ java -jar /../Retrotranslator-1.2.6-bin/retrotranslator-transformer-1.2.6.jar -srcdir /../workspace -target 1.5 -reflection safe -stripannot -classpath /../Retrotranslator-1.2.6-bin/retrotranslator-android-1.2.6.jar -verbose

What could be going wrong? Any suggestions would be greatly appreciated.

1 http://cwiki.apache.org/confluence/display/TUSCANYWIKI/14Jun2008

Hi Oscar,

Unfortunately I cannot open the TUSCANY wiki at the moment but it looks like the contents of retrotranslator-android-1.2.6.jar hasn't been included into the dex file. Try to explicitly refer to it in your code, for instance, call
net.sf.retrotranslator.android.main.java.lang._Class.cast(String.class,
"test");

And you needn't use the 1.4 target - only using the 1.5 one makes sense for Android.

Cheers,
Taras.

I forgot to mention that you can use the embed option making inclusion
of retrotranslator-android-1.2.6.jar into the compile path redundant.

So you can add "-embed <a single dot or any package name>":

java -jar ../Retrotranslator-1.2.6-bin/retrotranslator-transformer-1.2.6.jar
-srcdir ../workspace -target 1.5 -reflection safe -stripannot -embed .
-classpath ../Retrotranslator-1.2.6-bin/retrotranslator-android-1.2.6.jar
-verbose

Hi Taras,

After following your suggestion and adding retrotranslator-android-1.2.6.jar as an external library (as opposed to an external user library) the retrotranslator related errors disappeared. I'm now receiving errors about java.rmi.Remote 1, which from what I could find is not supported in Android. I just thought I'd mention it in case you have any suggestions

I'm glad Retrotrotranslator works for you Oscar. If you just want to get rid of java.rmi.Remote you may replace it with java.lang.Cloneable using the folowing Retrotranslator option:

-backport java.rmi.Remote:java.lang.Cloneable

But in case you have some important code using the java.rmi package you may try to take sources of java.rmi from the Apache Harmony.
Then move the sources into another package, so you'll have, for example, mypack.java.rmi, compile and jar them. After that you may use it with Retrotranslator:

-backport mypack -classpath mypack.jar

But this approach works only if the sources don't employ native code

Hi Taras,

Thanks for your help. I tried the -backport option on the previously retrotranslated classes and received some errors that were resolved by including backport-util-concurrent-3.1.jar. After that I was getting some errors related to getAnnotation 1 so I tried building all the projects from scratch and running retrotranslator again together with the -backport option. Now I'm getting UnsupportedOperationException errors 2. Any advice you can give me will be greatly appreciated.

1 http://cwiki.apache.org/confluence/display/TUSCANYWIKI/30Jun2008
2 http://cwiki.apache.org/confluence/display/TUSCANYWIKI/2Jul2008

Hi Oscar,

It looks like Retrotranslator haven't translated the ConversationProcessor class if you see the following stack trace:

java.lang.UnsupportedOperationException: native method not implemented
  at java.lang.Class.getDeclaredAnnotations(Native Method)
  at java.lang.Class.getAnnotations(Class.java:204)
  at java.lang.Class.getAnnotation(Class.java:187)
  at org.apache.tuscany.sca.implementation.java.introspect.impl.ConversationProcessor.visitClass(ConversationProcessor.java:49)

Check the size of the translated class, it should become one third bigger.

If you need backport-util-concurrent-3.1.jar you have probably run Retrotranslator without specifying "-target 1.5". Always use "-target 1.5" and "-classpath retrotranslator-android-1.2.6.jar" even if repeatedly translating classes. However I recommend to translate code in one pass.

Cheers,

Taras

Hi Taras,

I think you're right, but I find weird since I have retrotranslated all classes using the backport and target options. This is how I'm running retrotranslator now:

$ java -jar ../Retrotranslator-1.2.6-bin/retrotranslator-transformer-1.2.6.jar -srcdir srcdir -backport java.rmi.Remote:java.lang.Cloneable -target 1.5 -reflection safe -stripannot -embed . -classpath ../Retrotranslator-1.2.6-bin/retrotranslator-android-1.2.6.jar -verbose

Just to double check I rebuilt my workspace and ran retrotranslator again as shown above. I don't get any errors when running retrotranslator 1 2 but still get the same errors when running the calculator-android project. Also, I checked the sizes of the translated classes and found that they are not becoming much bigger 3 4. Do you think I'm missing something?

Thank you so much for all your help.

1 http://androidindelft.googlepages.com/tuscany_android_retr_output.txt
2 http://androidindelft.googlepages.com/tuscany_rev_retr_output.txt
3 http://androidindelft.googlepages.com/5Jul_retrotranslated.jpg
4 http://androidindelft.googlepages.com/5Jul_NOTtranslated.jpg

Hi Oscar,

The command line looks OK, but the embed option cannot be used there
since your srcdir does not point to the root package of your classes.
So don't use this option but add retrotranslator-android-1.2.6.jar to
your project instead. If the problem persists point srcdir exectly to
the root package (e.g. -srcdir
tuscany_android/mobile-android/calculator-android/bin)

BTW, the size of your classes must change significantly. I have
translated QName.class from Xalan-J and its size increased from 2827
to 4083 bytes.

Regards,
Taras

Hi Taras,

I was missing the point on the embed option. Thanks for the tip!

Today, I tested the newest version of retrotranslator (1.2.7) and did not get the RMI errors as before or the getAnnotation errors that indicated code wasn't being retrotranslated (appropriately). Instead I got a NoClassDefFoundError for java.beans.Introspector. I worked around this issue by commenting out the java.beans.Introspector import and related code. I am now getting NoClassDefFoundError on javax.jws.WebService 1.

I found that this class is not included in the Android package list and was wondering if maybe you had any suggestions. Is this something that could be backported as well?

Thank you very much for all your suggestions, they've been really helpful!

PS. I checked out the class sizes and found that classes that rely heavily on annotations are increasing 1/3 in size (from 8k to 12k for example), as you adviced.

1 http://cwiki.apache.org/confluence/display/TUSCANYWIKI/7Jul2008+-+Error+Stack+trace

best,
-oscar