Google Summer of Code 2008

Allow Google Android applications to easily consume business services - Oscar Castañeda

Sub-title: Incubating an Android in Delft.

Project Details

This wiki page will be used to display the current status of this project. This page will be updated frequently with the current updates of the project. A separate thread in the Tuscany-dev list will be used for discussions about the project.

Time line and the schedule of the project

 Project Issues

 Blocking Issues

 Limitations found when porting SCA to Android platform

Project Log

26March2008 - Submitted Apache CLA

7April2008 - Submitted GSoc Application

21April2008 - Got Accepted.

24April2008 - Started communicating with mentor.

29April2008 - Submitted proof of enrollment and foreign certification form to Google. Received acceptance of submission to perform research for CS course in Internet Technologies based on GSoc project.

1May2008 - Reviewed mentor's posts to Android Developer list. Started reading "Hello, Android" by Ed Burnette.

2May2008 - Read and followed Getting Started with Tuscany using the Command Line and Getting Started with Tuscany using a Tuscany Distribution In Eclipse . Experienced some issues . Created Google Notebook and shared with mentor for play by play summary.

10May2008 - Reviewed and followed steps in email from Adriano Running Tuscany/SCA in Google Android mobile platform . Sent follow-up with questions to dev-list. Read about licensing in URL posted by Simon in response to Adriano .

14May2008 - Followed steps in follow-up email from Adriano . Was able to build entire workspace from instructions. Now running calculator-Android as an "Android application". Not sure if exceptions received in Android emulator are what would be expected. Sent email to dev-list asking for opinions.

20May2008 - Followed steps in follow-up email from Adriano . Now getting a different error than expected . Sent email to dev-list asking for help. Also tested retrotranslator; with Ant still need to include entire tuscany project (similar to Eclipse setup), manually it's not working with a jar created from JavaRuntimeModuleActivator (nothing is getting translated), and there's a maven plugin that would offer a third option. Will continue testing.

25May2008 - End of community bonding period. Sent email to dev-list with most recent update about the project.

26May2008 - Official start of Google Summer of Code 2008!

27May2008 - Succesfully ran retrotranslator on some classes. Imported those projects but now facing build errors. Eclipse is not specific on what the errors are, but every single module has an error reported in the errors section. Will continue looking into it tomorrow. Started testing Luciano's suggestions as pointed out by Adriano for a version without annotations.

Update: Fixed build errors by adjusting the build path.

28May2008 - Ran retrotranslator on Eclipse workspace. However, I suspect that as classes from calculator-android are only available on the Android emulator they were not processed and hence are now resulting in errors. I am now getting an NPE and some errors on CalculatorServiceImpl.java. Will continue testing, including Luciano's suggestions.

30May2008 - Found that the errors only show up after including retrotranslator-android-1.2.6.jar as a library on calculator-android, details: here.

Update: Continued testing retrotranslator by giving it 1.4 as the java target version. Experienced exact same issues. Tested with untranslated code and also experienced same issues. This led me to think that something was not being translated.

2Jun2008 - Found that the project wasn't being rebuilt after uncommenting code. Forced a build of all the projects and then ran retrotranslator and received different results, details: here. Sent email to Tuscany-dev about the progress made thus far.

4Jun2008 - Researched errors in stack trace on Android Developers list. Found a suggestion relating to introspection on Android Issue 268 . Found more information relating to retrotranslator on Issue 29 and related discussions. Contacted Shai Almog (involved in the report of Issue 29) to ask about his experience with retrotranslator (he had reported problems with retrotranslator but then resolved them). Shai suggested to contact Taras Puchko, the creator of retrotranslator. Taras adviced to use retrotranslator on JAR's instead of class files, especially because of the ordering of the Android build process. His reply on the Android Developers list can be found here.

Received reply from Adriano suggesting to first start with a simple scenario to build on. Sent follow-up asking about introspection and about projects that aren't being imported. Received reply suggesting to not go into introspection yet and to avoid testing both introspection and retrotranslator at the same time. Will continue project focusing on retrotranslator for now.

5Jun2008 - Generated JAR from tuscany modules (including those modified by Adriano for Android). Ran retrotranslator on JAR and obtained a (retro)translated JAR to use in calculator-android project. However, now getting errors when converting to dex format.

6Jun2008 - Testing a different approach - instead of creating one big JAR file I copied all the JAR files contained in the individual modules to a separate folder and then ran retrotranslator on each JAR. The jars are found inside the target folder of each module and are named as shown below:

tuscany-<NAME>-incubating-SNAPSHOT.jar

From this I found that only the SCA modules have these JAR files inside their respective target folders, whereas the Android modules don't. So I kept the Android modules and added the translated JAR's as libraries where needed to have the workspace build with no problems. Sent email to dev-list about this and plans for coming up with a "lightweight mobile SCA core/runtime" from reduced JAR's.

9Jun2008 - The hard drive on my computer died

10Jun2008 - Restored from backup, setup working environment. Back online

11Jun2008 - Running into problem while doing clean install. Sent email to dev-list and posted trace.

12Jun2008 - Finally able to build workspace and get back to point in which to run retrotranslator on workspace as srcdir. After running retrotranslator on workspace, deleted and re-imported calculator-android. Confirmed that no changes are being made to classes through the timestamps on class files in ../workspace/calculator-android/bin/calculator. Now getting different errors in Android logs and emulator. Will continue testing with calculator-android and simpler scenario as suggested by Adriano.

16Jun2008 - Disabled "Java Builder" from all projects as suggested by Adriano. Added retrotranslator-android-1.2.6.jar as a library to all projects. Re-ran retrotranslator on all classes and then generated dex file. Confirmed that classes are not being recompiled by Eclipse and that the dex file is generated after running retrotranslator (guaranteeing that Android emulator uses translated classes). Getting errors related to retrotranslator. Started creating a simpler scenario for testing.

17Jun2008 - Re-tested retrotranslator on "build-free" projects, using java 1.5 and then 1.4 as target versions for translation. However, experiencing the exact same issues in both cases. Sent email to Android Developer list hoping to get a response from Taras Puchko, creator of retrotranslator. Reading up on SCA to see how it uses Annotations, continuing work on simpler scenario.

Update Received response from Taras.

18Jun2008 - Followed suggestions from Taras; used "embed" option within Retrotranslator, but still was receiving the same errors. After several tests (Removing Java builders, adding retrotranslator jar to every project, recompiling) I found that the problem was that retrotranslator-android-1.2.6.jar was being added as an external user library. I think the ADT plugin doesn't include external user libraries during conversion to dex format and creation of the Android package. After including the retrotranslator-android-1.2.6.jar directly as an external JAR the retrotranslator related errors disappeared. However, I am now getting errors related to the java.rmi.Remote library. From what I found, Android does not support java.rmi.

Update Sent follow-up to Taras including question about RMI.

19Jun2008 - Received follow-up response from Taras. Re-created working environment in preparation to create JIRA issue ticket about Retrotranslator changes.

24Jun2008 - Created patch and discussed with mentor. Will recreate workspace with the option to copy projects into workspace turned OFF, so as to reduce the size of patches created.

25Jun2008 - Recreated workspace with option to copy projects into workspace turned OFF. Generated patch and updated retrotranslator-readme file. Re-tested calculator-android with new workspace and received different errors on the Android emulator. Created JIRA ticket. Updated Blocking issues page.

30Jun2008 - Tested removing tuscany-binding-rmi from calculator-android (as I couldn't find host-android), but did not see significant changes. Tested suggestion from Taras to backport the RMI dependencies, again using retrotranslator. This produced ClassNotFoundException errors that were resolved by including the backport-util-concurrent-3.1.jar as an external JAR. However, this resulted in further errors on the Android emulator.

2Jul2008 - Re-tested suggestion from Taras by building all projects from the start and running retrotranslator, including the backport option. The errors received were different this time. Sent email to Android Developers list asking for suggestions.

3Jul2008 - Received response from Taras. Rebuilt workspace and ran retrotranslator again, switching the backport option before and after the other options. Still received the same errors when running the calculator-android project.

5Jul2008 - Rebuilt workspace and tested backport option only on specific module projects. Sent follow-up to Android Developers list asking about the backport option and noting that classes are not growing by 1/3 as expected.

7Jul2008 - Started testing newest version of retrotranslator after receiving tip from mentor (thanks Adriano!). Followed suggestion from Taras to leave out the embed option when retrotranslating on a non-individual-module basis. Again received errors related to Beans API. Commented java.beans.Introspector import and related code from (org.apache.tuscany.sca.implementation.java.introspect.impl) to work around the Netbeans issue. Now experiencing errors related to javax.jws.WebService. Error Stack trace, here.

Found that classes that rely heavily on annotations, like JavaRuntimeModuleActivator.class, do grow about 1/3 in size as Taras adviced.
Sent follow-up email to Taras asking about javax.jws.WebService class.

Update Backported javax.jws.Webservice succesfully as ClassNotFoundException errors were no longer shown when running calculator-android. However, backport of java.rmi.Remote did not have the same effect. Tried retrotranslating mobile-android modules separately to make sure that overlapping retrotranslator runs weren't the cause of the issue, received the same results.

8Jul2008 - Adriano commited some modifications done on the android sandbox:

• the code that uses the Java RMI API was commented
• the code that uses the JWS API was also commented
• the code that uses the Introspector.decapitalize() (Java Beans API) method was replaced by the logic this method provides
• It was also throwing a NPE when SCA tries to load the TransformationExtensionPoint, so I loaded the DefaultTransformationExtensionPoint programatically

After these modifications some modules were added to the android sandbox (core-databinding, interface-java and inteface-java-jws). Currently receiving
a binding related exception when running calculator-android.

9Jul2008 - Started testing tool to automate modifications to Java Builder properties of individual module projects in Eclipse. Found that tool used by Adriano doesn't work on Mac OS X, even after enabling "full keyboard access", because Eclipse checkboxes are different to native checkboxes.

Ran into problems when building revision 674723 uploaded by Adriano. The problem has been experienced before, and results in not being able to download the artifact from any repository. Replicated working environment on Windows and Linux and received the same problems when building with Maven.

10Jul2008 - Continued testing suggestions from Taras while I find how to resolve the Maven issue. After backporting java.rmi.Remote, adding geronimo-ws-metadata_2.0_spec-1.1.2.jar to the calculator-android as a library, and retrotranslating the android sandbox and SCA modules I am finally getting to the point in which there are no errors related to ClassNotFoundException, but instead are related to NPE. The most recent stack trace can be found here.

Update Partially resolved issue with the help of Adriano. Now debugging as I can't import projects into Eclipse, even after a doing

mvn clean install -fn

11Jul2008 - Reviewed current progress and sent mid-term progress email

12Jul2008 - Resolved problems reported on 9Jul2008 with the help of Adriano. The issue was resolved by placing the missing files on the 1.4-SNAPSHOT directory on .m2/repository

16Jul2008 - Started looking into calculator2 as suggested by Luciano. However, when checking out revision 673092 I get dependency errors related to those that were reported on 9Jul2008. The errors are about the following missing files (which mvn can't download):

1) org.apache.tuscany.sca:tuscany-runtime2:jar:1.4-SNAPSHOT
2) org.apache.tuscany.sca:tuscany-scdl4j:jar:1.4-SNAPSHOT
3) org.apache.tuscany.sca:tuscany-api:jar:1.4-SNAPSHOT

Once I get these files in place the plan is to retrotranslate the associated jars and then try to run calculator2 on Android making some of the adjustments Adriano did for calculator-android.

Update Received feedback from Luciano confirming the idea of calculator2 and further suggesting to remove a lot of bindings and
implementations from the runtime, as a first approach. Additionally, Luciano fixed the issue with the missing jar's and I was able to bulid revision 673092. Will continue with testing of calculator2.

18Jul2008 - Created new workspace with the reduced modules for calculator2. This workspace was created by only importing the reduced list of modules and removing dependencies from each project in eclipse.

19Jul2008 - Signed the petition for an updated SDK. Adriano also signed this petition.

20Jul2008 - Continued testing mojo retrotranslator plugin for maven. During the tests I found that mojo doesn't support all of the options that are available for retrotranslator (and that are being used for the Android retrotranslation). Sent an email to the Mojo project and was asked to open a JIRA ticket so that the issue can be dealt with. Created the ticket and replied to Mojo project. Sent email to Adriano to let him know that the maven integration will not be possible because of these missing features in the plugin.

22Jul2008 - Sent a progress update on calculator2 for Android to the Tuscany dev list. Adriano suggested to open up a new thread as we are now on a different stage of the project.

23Jul2008 - Started new thread as recommended by Adriano. Created new instructions for reproducing the workspace.

25Jul2008 - Started debugging code based on recommendations of things to check for. Looked in more detail to see whether the composite is really being read. Also found that there is an exception after processReadPhase where the read problems are arising. I have the suspicion that class files are not being accessed because Android uses a different (dex) mechanism. Sent email update in new working thread.

26Jul2008 - Luciano confirmed that the class reading should be ok, as we just ask the classLoaders to load it inside ClassReferenceModelResolver. He recommended to check how the CompositeProcessor is viewing the contents of dex://calculator.android/raw/calculator.composite. Started checking this.

28Jul2008 - Found that the composite is indeed being read, not sure still if completely and correctly though. At this point I have found that the error is coming from createInvoker() in org.apache.tuscany.sca.binding.sca.impl. The getInvoker() method is returning null instead of the result of a "new SCABindingInvoker(chain)". The result is that invoker is null and thus returns the error. According to Simon, this points to a non-existent service, or one that the runtime thinks does not exist. My thinking is that maybe the assembly isn't being properly populated.

30Jul2008 - Debugged the issue with Adriano through chat. Still the same problems are arising and haven't made much progress finding out why.

2Aug2008 - Further debugging showed that the calculator.composite is indeed being read appropriately, eliminating the possibility of a parsing related issue. I verified this by seeing the composite being correctly populated while debugging. To accomplish this I compared the reduced Android SCA to revision 643746 without the Android sandbox code and with the normal calculator sample. This was suggested by Adriano as a way to approach testing and debugging for Android.

5Aug2008 - To further simplify the debugging runs I narrowed down the calculator operations to only the "add" operation. I set a breakpoint in RuntimeSCAReferenceBindingProvider on the getInvoker() method, which is where the error is ocurring. I found that the problem is that the service, as shown below, is being assigned a value of null.

RuntimeComponentService service = (RuntimeComponentService)target.getContract();

In order to proceed, service must have a value other than null. Instead, getInvoker() method returns a null invoker, which results in the ServiceUnavailableException.

8Aug2008 - Continuing the approach of testing against a baseline scenario, I decided to run retrotranslator on SCA revision 643746. First, I verified that the revision works as expected. In other words, I downloaded and bulit the revision and ran the non-Android calculator sample. Then, I ran retrotranslator with the same options as those for the Android scenario. The result, I found, was pretty similar to the Android scenario.

10Aug2008 - Sent progress update to developer list through the new thread that was opened. Continued debugging and searching for alternate solutions.

11Aug2008 - Simon suggested that not using annotations can be an option. This had been suggested before by Luciano as well. Continued from previous efforts, completely removed all annotations but still getting errors when running the code on the emulator.

13Aug2008 - Discussed the issue with Adriano, received pointers about what to change. After trying out the changes I still get similar errors. I think that annotations are still being accessed somewhere.

14Aug2008 - Received suggestion from Mike to turn off introspection and instead supply the equivalent information as <componentType/> side files. I tested the suggestion but still, because annotations are being accessed somewhere the emulator returns errors.

15Aug2008 - Discussed with Adriano, he confirmed that because annotations are still being read the solution of using a side file does not resolve the problems.

18Aug2008 - Official 'pencils-down' date. Google released updated SDK (0.9).