You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

Falcon is a new compiler for Flex applications. It will eventually replace the legacy compiler that has been part of the Flex SDK.

Current Status

Falcon has completed a legal review at Adobe and is awaiting signature by an Adobe VP, after which it will be donated to Apache.

The ActionScript side of Falcon is shipping quality and is the basis for Flash Builder 4.7. The rest of Falcon (MXML, CSS, FXG, PROPERTIES) is pre-alpha quality and not ready for production use. However, Falcon already supports enough of MXML to compile a correctly-running version of Checkinapp, the test app that is part of the SDK. THe main areas where Falcon needs polish are databinding, states, MXML-specific error reporting, and ASDoc generation.

After donation, Gordon Smith of Adobe, who has worked on Falcon from the beginning of the Falcon project, is on loan to Apache Flex for one day a week to improve Falcon's MXML support, with the goal of making it a suitable replacement for the legacy compiler.

Everyone can contribute to Falcon's development by using it to compile their Flex applications and reporting any bugs you find in Jira. Reducing a bug to a simple test case will make it infinitely more likely to be fixed.

Initial development efforts:

  • Run the source code for all Apache Flex SWCs through Falcon to see if it reports spurious problems. Falcon is pickier than the legacy compiler, so we will need to update the source code in some cases for Falcon.
  • Get various test apps besides Checkinapp compiling and running.
  • Get Mustella tests compiling with Falcon and passing.

Falcon's Original Goals

  • Reduced compilation time for both full and incremental compiles, especially for multi-project workspaces.
  • Reduced memory consumption, especially for multi-project workspaces.
  • Usefulness as a code-intelligence engine and incremental compiler for an integrated development environment; not just a command-line compiler.
  • Modular design that is easy to understand and to evolve.
  • Generation of beter bytecode for better runtime performance.

Source Code

The Falcon trunk will soon be available in Apache's Subversion repository at https://svn.apache.org/repos/asf/incubator/flex/falcon/trunk. You can download it with any Subversion client.

Building Falcon

Falcon is written in Java 1.6. Its build scripts use Ant 1.7. Eclipse projects for development use Eclipse 4.2 (Juno).

Information about Ant targets will be available by the time of donation.

Testing Falcon

Test suites for Falcon are not yet ready for donation, because they require more extensive legal review. They will be donated later.

Javadoc

Falcon has extensive Javadoc, including a short overview and package-level documentation that provides a general orientation to various subsystems. The Javadoc is not currently available online. To build it, download the source code and do

ant javadoc

in the compiler directory within Falcon's trunk. The generated HTML is inside generated/javadoc within Falcon's trunk. Open index.html in any browser and start by clicking on the Description link at the top to see the overview.

History

Falcon began as a project at Adobe in the spring of 2010, in response to the Flex community's concerns about long compilation times with the legacy compiler. There were several months of discussions about improving the old compiler versus starting over with Falcon. The main reasons for starting over were:

1. The compiler team at Adobe wanted a compiler that could do double-duty as the code-intelligence engine for Flash Builder, rather than being something completely separate that Flash Builder called just to create a SWF or SWC. This approach would avoid duplicate data structures and duplicate algorithms, and would enable live error highlighting based on accurate semantic analysis. The team ended up taking Flash Builder's code intelligence engine, improving it, and adding semantic analysis and code generation to it. The resulting Falcon compiler doesn't have dependencies on Flash Builder though... it's the other way around.

2. The legacy compiler wasn't designed for compilation on multiple threads. The team wanted a compiler that was designed with concurrency in mind from the beginning, as a primary way to get better – and scalable – performance.

3. There was an awkward relationship between the legacy asc (which essentially understands only how to compile a single .as file) and the legacy mxmlc (which layers on top multi-file compilation and support for .mxml, .css, .properties, and .fxg), based on their historical development by two separate teams at Adobe. The team wanted a single compiler, designed by a single team, that had a unified architecture, unified data structures, and multi-file / multi-language support from the bottom up.

In the fall of 2011, a strategic shift at Adobe resulted a refocus on ActionScript and a defocus on Flex and MXML. Flex was donated to Apache, with a promise to finish the ActionScript side of Falcon and donate it by the end of 2012. Non-ActionScript support was left intact, at pre-alpha quality.

In the summer of 2012, Falcon was ready for donation to Apache.

Architecture

Falcon is designed from the beginning to support compiling multiple targets in multiple projects in a workspace, as exist in an IDE like Flash Builder. By contrast, the legacy asc was designed to compile a single AS file, and then the legacy mxmlc was built by another team on top of that.

Falcon uses multiple threads to compile multiple files at the same time. The more cores you have, the faster it goes. The threading model is implemented using Java's Future class.

Critical data structures such as the symbol table (which stores information about which classes are known, what methods they have, etc.) are shared across the entire workspace, to minimize memory usage.

Critical data structures are maintained in memory to support both compilation and IDE code intelligence in an efficient and consistent way. For example: If you open an ActionScript file in Flash Builder 4.7, Falcon builds a syntax tree and symbol table for it to support intelligent editing. Compiling the file requires just one additional code generation step. By contrast, in Flash Builder 4.6, which uses the legacy compiler in the SDK, the IDE builds its own parse trees and symbol tables to support editing, and then when you compile the compiler in the SDK builds another set of parse trees and symbol tables. This is slow and a waste of memory.

Falcon was designed from the beginning for incremental compilation of a complete workspace, so it tracks dependencies between compilation units in various projects. If you edit a file, it knows which other files it should recompile.

It can compile an app against a library project without creating the SWC for that library on disk.

Falcon understands .as, .mxml, .fxg, .css, and .properties files.

The parse trees for AS consist of nodes from about 100 classes, such as LiteralNode, BinaryOperatorPlusNode, FunctionNode, and ClassNode. The parse trees for MXML consist of nodes from about 50 classes, such as MXMLDocumentNode, MXMLInstanceNode, MXMLScriptNode, etc. The ActionScript-y parts of MXML are represented by AS nodes that are children of MXML nodes.

The symbol table consists of objects representing the things your code defines, such as ClassDefinition, FunctionDefinition, and VariableDefinition. These are stored in scope objects which form a hierarchy reflecting the lexical blocks of the source code.

Falcon uses third-party grammars like JFlex, ANTLR, and JBurg to generate various lexers, parsers, and code generators.

The lexers for AS and MXML are written in JFlex .lex files. The lexer for CSS, and the parsers for AS, MXML, and CSS, are written in ANTLR .g files. The code generator is written in JBurg .jbg files. JBurg is a Java implementation of a Bottom-Up Rewrite Generator (BURG) which generates Java code for a Bottom-Up-Rewrite Machine (BURM). The BURM uses patterns and rules to find subtrees within in an abstract syntax tree that it can "reduce" to an intermediate form, typically a sequence of bytecode instructions. The process continues until the entire tree is reduced to ABC, which goes into a DoABC tag of a SWF file.

The typical data flow for an AS file is

.as file --> tokens --> abstract syntax tree --> file scope
                                             --> byte code and compilation problems

and for an MXML file is

.mxml file --> tokens --> XML DOM --> file scope
                                  --> abstract syntax tree -> byte code and compilation problems

To maximize performance, MXML is compiled directly to ABC, not to ActionScript source code or an ActionScript parse tree.

Falcon tracks where in the source every token/node/definition came from, to support operations like go-to-definition in Flash Builder.

Falcon has the general concept of a "problem", where a problem can end up being considered an error, a warning, or ignored. Problems are discovered by the semantic checker, which is part of the same phase of compilation as code generation. (This minimizes the number of times that the syntax tree must be traversed.) Falcon provides live problem highlighting to Flash Builder. When you pause during editing a file, the file is compiled and the problems show up as red squiggly underlines. Falcon tries to continue after almost all problems, so that you don't necessarily have to fix every problem before you can get a SWF to run.

Falcon includes libraries for reading and writing SWFs and SWCs files, and a library for reading and writing ActionScript byte code (ABC), at the level of individual instructions.

Falcon includes transcoders for the various types of embedded assets, which read, for example, a PNG file and convert it to the form required inside a SWF. It does NOT support font trancoding, because this relies on proprietary Adobe technology which Adobe has not donated to Apache. Instead, you must use a separate fontswf tool to convert your font files to SWF files and then embed the fonts from them.

FAQs

Please add questions here and people who contribute to Falcon will be happy to try to answer them.

Q: What is a lexer?

A: A lexer turns source code into tokens.

Q: What is a parser?

A: A parser turns tokens into an abstract syntax tree.

Q: What is a code generator?

A: A code generator turns an abstract syntax tree into bytecode.

Q: What is a semantic checker?

A: A semantic checker finds problems in the source code.

Q: What is an example of how Falcon generates better bytecode than the legacy compiler?

Falcon does both constant propogation and inlining.

  • No labels