Page History

...

1. Skim this document to get a feeling for IR
: http://llvm.org/docs/LangRef.html

...

• hdfs-avro-scanner.cc, CodegenMaterializeTuple
  • understand how we type the tuple* based on the tuple layout

3. Review the Adding a builtin function to Impala documentation for the relatively easy-to-understand UDF use case.

4. Cross-compiled functions (TODO)  

I do not recommend plowing through the LLVM tutorial. It has some useful info, but is mostly useless for our purposes.

...

• Run a single impalad instance if you're not already. It's generally easy to manage and reason about. (start-impala-cluster.py -s 1)
• Check that the query executes successfully with codegen disabled. If not, it's probably a bug in cross-compiled code. 
• Inspect the generated IR
  • Every LLVM Value* object (including Function*) has a dump() method that will print human-readable IR to stderr.
    • This is useful for printing instructions, e.g. when LLVM complains about a calling a function with the wrong args, dump the offending instruction.
  • If you'd like to dump an object to a string instead of stderr, use LlvmCodegen::Print().
  • The -dump_ir impalad process flag will make the impalad call dump() on every generated function (see LlvmCodegen::FinalizeFunction()). When using start-impala-cluster.py, use the flags "--impalad_args -dump_ir". Output goes to the INFO log file in $IMPALA_HOME/logs/cluster/impalad.<something>.INFO.<timestamp>. Look for "Dump of Function" in the log.
  • The -opt_module_dir process flag takes an already-created directory and will write every optimized module to that directory (there's a similar -unopt_module_dir flag for the unoptimized module, but usually -dump_ir is sufficient). However, the unoptimized IR is usually more useful (especially since the optimized module does not include any unused functions, including inlined functions, usually leaving you with one or two giant unreadable functions).
  • The -asm_module_dir is similar to -opt_module_dir, except it outputs assembly for generated functions.
• By default we don't build a debug version of LLVM. If you want, you can manually build a debug version to link against so you can introspect LLVM functions in gdb. I generally find this not very helpful though, it's faster and more effective to add liberal dump() calls.

...

• Never use IRBuilder::CreateAlloca() directly. Instead, use LlvmCodegen::CreateEntryBlockAlloca(). Creating alloca's in the middle of functions will yield strange results (e.g. bad optimizations, blowing the stack).
• When you create a new cross-compiled file (i.e. a *-ir.cc file), you need to add it to codegen/impala-ir.cc to include it in the IR module.
• After generating an IR function, you should call OptimizeFunctionWithExprs() or FinalizeFunction() on it, as well as AddFunctionToJit() (TODO: expand on this)

Code Overview

The planner divides a query into fragments, the BE runs multiple fragment instances. Each instance is represented by FragmentInstance in fragment-instance-state.cc. The Open() method checks if code generation should be done, and if so, triggers the codegen work by calling the CodeGen() method on on the set of exec nodes (operators), which will call CodeGen() on expressions (for those nodes that have expressions.)

Consider an expression in the SELECT clause. Impala is row-based, with each row represented by TupleRow (tuple.h). TupleRow::MaterializeExprs() iterates over the expressions to be used to materialize a row, which will include either a slot reference or a scalar expr (ScalarExpr). For each, GetValue() is called, which is either a code generated function or an interpreted function. This function itself can be code generated as the MaterializeExprs function.

Useful APIs and references

...