THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
Feature | Available in Release | Comments |
|---|---|---|
Describe Schema | 0.1 |
|
Explain Plan | 0.1 |
|
Add log4j to Pig Latin | 0.1 |
|
Parameterized Queries | 0.1 |
|
Streaming | 0.1 |
|
Documentation | 0.2 | Docs are never really done of course, but Pig now has a setup document, tutorial, Pig Latin users and reference guides, a cookbook, a UDF writers guide, and API javadocs. |
Early error detection and failure | 0.2 | When this was originally added to the !ProposedRoadMap it referred to being able to do type checking and other basic semantic checks. |
Remove automatic string encoding | 0.2 |
|
Add ORDER BY DESC | 0.2 |
|
Add LIMIT | 0.2 |
|
Add support for NULL values | 0.2 |
|
Types beyond String | 0.2 |
|
Multiquery support | 0.3 |
|
Add skewed join | 0.4 |
|
Add merge join | 0.4 |
|
Add Zebra as contrib project | 0.4 |
|
Support Hadoop 0.20 | 0.5 |
|
Improved Sampling | 0.6 | There is still room for improvement for order by sampling |
Change bags to spill after reaching fixed size | 0.6 | Also created bag backed by Hadoop iterator for single UDF cases |
Add Accumulator interface for UDFs | 0.6 |
|
Switch local mode to Hadoop local mode | 0.6 |
|
Outer join for default, fragment-replicate, skewed | 0.6 |
|
Make configuration available to UDFs | 0.6 |
|
Load Store Redesign | 0.7 |
|
Pig Mix 2.0 | 0.7 |
|
Rewrite Logical Optimizer | 0.8 |
|
Cleanup of javadocs | not yet released 0.8 |
|
UDFs in scripting languages | 0.8 |
|
Ability to specify a custom partitioner | 0.8 |
|
Pig usage stats collection | 0.8 |
|
Make Pig available via Maven | 0.8 |
|
Standard UDFs Pig Should Provide | 0.8 |
|
Add Scalars To Pig Latin | 0.8 |
|
Run Map Reduce Jobs Directly From Pig | 0.8 |
|
...
This covers work that is currently being done. For each entry the main JIRA for the work is referenced.
Feature | JIRA | Comments | ||
|---|---|---|---|---|
Make Illustrate Work |
| |||
Better Parser and Scanner Technology | PIG-1618 0.9 |
| ||
Clarify Pig Latin Semantics | many 0.9 |
| ||
Extending Pig to Include Branching, Looping, and Functions | TuringCompletePig |
| 0.9 |
|
Typed maps | 0.9 |
|
Work in Progress
This covers work that is currently being done. For each entry the main JIRA for the work is referenced.
Feature | JIRA | Comments | Typed maps | ||
|---|---|---|---|---|---|
Move Piggybank out of github | https://github.com/wilbur/Piggybank | Currently Pig hosts Piggybank (our repository of user contributed UDFs) as part of our contrib. This is not ideal for a couple of reasons. One, it means those who wish to share their UDFs have to go through the rigor of the patch process. Two, since contrib is tied to releases of the main product, there is no way for users to share functions for older versions or quickly disseminate their new functions. If Piggybank were instead more similar to CPAN than users could upload their own packages with little assistance from Pig committers and specify what versions of Pig the function is for. This could be done via hosting site such as github. | |||
Nested cross/foreach |
| ||||
Boolean and Timedate type |
| ||||
Move parameter substitution/grunt to Antlr |
|
| |||
Make Pig work with hadoop 23 |
|
Proposed Future Work
Work that the Pig project proposes to do in the future is further broken into three categories:
...