THIS IS A TEST INSTANCE. ALL YOUR CHANGES WILL BE LOST!!!!
...
What defines this AIP as "done"?
- airflow.io is available
AFS interface should get one of the standard "citizens" of the Provider Package features.
Existing provider packages must still be supported and previous provider packages not knowing about AFS should be also usable with newer Airflow versions
filesystems:
- airflow.providers.microsoft.azure.fs.adls
- DAGs can work with AFS
- Initial implementation supports LocalFilesystem, S3, GCS, ADLS
These implementations must be hosted within the respective provider packages.
- LocalFileSystem→ Okay to put this in core with the interface definitions
- S3→AWS
- GCS→Google
- ADLS→Azure
- Exception could be for generic filesystem that don't have a real provider (say for example a ZipFileSystem)
- Out of scope: XCom, DAG Processor
...