You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 3 Next »

In the flex-utilities repo, there is an app called MD5Checker.  It is used to periodically verify the MD5 checksums of some of the dependency downloads for Flex such as the AIR SDK, playerglobal.swc, and Google Closure Library.  The providers of these downloads do not appear to have posted MD5s, but we've found that the Installer frequently has problems downloading these artifacts correctly, so we've taken to computing the MD5s ourselves and posting the results in flex.apache.org/installer/sdk-installer-config-4.0.xml.

In hindsight, it probably would have been better for the CI server to compute MD5s and leave them in files on the CI server, but the installer scripts are hardwired to look on flex.apache.org, so we'll probably live with this scheme forever.

There is a CI server job on apacheflexbuilds.cloudapp.net:8080 that computes the MD5s every hour.  Instead of downloading each very large AIR SDK, it checks the eTag http header and only downloads and recomputes the MD5 if the eTag has changed.

If a file's MD5 changes, the CI server send a build failure to the dev@ mailing list.  We originally thought MD5s would not change very often, but Adobe frequently updates beta SDK and for some reason occasionally alters the contents of existing older SDKs.

To update, check the email.  It will tell you which node(s) in sdk-installer-config-4.0 to change, but be careful.  There are a lot of similar entries in the file.

In the flex-utilities repo, there is a build.xml in the MD5Checker folder.  It builds and runs MD5Checker, but also can commit the updated file to the web-site SVN if you supply the path to your SVN working copy for flex.apache.org.

To execute the update, create a local.properties file that specifies svn.site=<path to web-site working copy>.  Then the build script will copy the modified sdk-installer-config-4.0.xml to the working copy and commit it to the web-site SVN.  It will prompt for apache.username which you can also supply in local.properties and it will prompt for password.  The password will not be obfuscated so be careful who is watching your screen.  You will still need to use ASF CMS to push the updated file to the production web server so future MD5Checker runs and our Installer customers will see the changes.

The CI Server has limited memory, and the AIR SDKs have grown in size such that if an AIR SDK changes, the CI Server will hang while computing the new MD5.  You can tell the MD5Checker job is hung because the progress bar in the Jenkins UI will be red.  If you see this, manually cancel the Jenkins job so the other jobs can run, then run MD5Checker locally and update the sdk-installer-config-4.0.xml file as described above.

You may also notice that some times there are several "build failed" emails followed by a "build fixed" then later, more "build failed" emails.  I'm not quite sure why this happens, but my theory is that things like the AIR SDK is published on a CDN and CDNs are essentially mirrors behind the same URL.  It takes time for these files to propagate to all mirrors, and during that time, different runs of MD5Checker will pull the new version and fail and sometimes the old version a succeed.  I've found that after a day or two, we get pretty consistent failures and updating sdk-installer-config-4.0 as described above is warranted.

Another occasional failure scenario is that the "build failed" email shows that the MD5 checksum is the same, but the cacheID (which is the eTag HTTP Header) is different.  I don't know why that sometimes is the case, but if you see more and more "build failed" messages like that, the remedy is to copy the cacheID from the "new version" in the email to the cacheID2 in the sdk-installer-config-4.0.xml file and update the file in SVN and on the web-site as described above.

In some future, MD5Checker and/or the build.xml could be improved to auto-update the sdk-installer-config-4.0.xml automatically, without human intervention, but that would imply leaving SVN credentials on the CI Server and I am not willing to put my credentials on that machine.  Also, I haven't found a way to "hit the publish button" in ASF CMS through Ant.  I think it is possible if someone wants to try.

 

  • No labels