Status

Motivation

This is part of AIP-1, which aims to run Airflow in a multi-tenant way. Today, Airflow doesn’t have built-in support for multi-tenant environments. Airflow provides Role-Based Access Control (RBAC), but the access control is not fine-grained and allows various Airflow components and users to access/modify resources across the environment without restriction. This document describes a proposal to introduce a tenant model paving a way to running Airflow in a multi-tenant fashion.

Note: The resource access control is out of scope for this AIP. It will be done in a separate AIP(s). The purpose of this AIP is to introduce a new “tenant” entity that will be utilized in the future AIPs related to multi tenancy.

Considerations

Current state

The current access management uses a role based access control (RBAC) to authorize users to access specific pages/resources. Here is the database diagram.

To better understand it, here is an example. This is only a simplified version of an Airflow configuration.

• The user John is associated to the roles Admin and Op
• The role Admin has the permission “can read on Users”
• The role Op has the permission “can read on Variables”
• As a result, John can read the variables and the users

Proposal

The proposal is to introduce a new entity called "tenant". Users can belong to one or multiple tenants. The purpose of a tenant is to be self isolated from each other, therefore, users in a tenant cannot have access to users and roles from another tenant (unless they have Admin permissions). Note: the resource access control is out of scope for this AIP. It will be done in a separate AIP(s).

A new Airflow configuration is introduced to whether activate this new entity "tenant":

• [core] enable_tenant

When enabling this option, the multi tenancy experience will be enabled, as such Admins will be able to manage tenants and to assign users to them. On users side, they will be then restricted to their tenant scope(s). In other words, if a user has "List users" permissions, when enabling this option, this user will be able to list only users which belongs to the same tenants he belongs to as opposed to all users when the option was turned off.

Since some database changes are needed to introduce this new tenant entity (see next section), and by default tenants will be disabled, we need the database to be compatible with when [core] enable_tenant is True or False. TODO: need to explain global default tenant here

Database

To better understand it, here is an example:

• The role Admin is associated to the tenants HR and Marketing which means the role Admin can be associated with users in the tenants HR and Marketing
• The user John belongs to the tenant HR with the role Admin
• The user John belongs to the tenant Marketing with the role Admin
• The role Admin has the permission “can read on Users”
• As a result, John can read the users of the tenant HR and Marketing

---

• The role Op is associated to the tenant Marketing which means the role Op can be associated with users in the tenant Marketing
• The user Bob belongs to the tenant Marketing with the role Op
• The role Op has the permission “can read on Variables”
• As a result, Bob can read the variables of the tenant Marketing

Resource management changes

In order to manage the new entity tenant, some modifications have to be made across the UI, the Rest API and the Airflow CLI. Airflow CLI changes will be described in its own section at the end of this section.

Tenant menu access

List tenants

UI

Rest API

Input

GET /tenants

Output

{
  "tenants": [
    {
      "name": "string"
    }
  ],
  "total_entries": 0
}

Errors

Create tenant

UI

Rest API

Input

POST /tenants

Output

{
  "name": "string"
}

Errors

Get tenant

UI

Rest API

Input

GET /tenants/{tenant_name}

Output

{
  "name": "string"
}

Errors

Edit tenant

UI

Rest API

Input

PATCH /tenants/{tenant_name}

Output

{
  "name": "string"
}

Errors

Delete tenant

UI

Rest API

Input

DELETE /tenants/{tenant_name}

Output

No output

Errors

List roles

For each role, the list of associated tenants needs to be added.

Only roles which are associated to the same tenant(s) as the logged in user need to be returned.

Open question: What if Admin is associated to HR and Marketing tenants and the logged in user has permission to list roles in the HR tenant. Should HR be the only tenant returned as part of Admin's associated tenants information?

UI

Rest API

Only the output has to be updated. The field "tenants"needs to be added.

Output

{
  "roles": [
    {
      "name": "string",
      "actions": [
        {
          "action": {
            "name": "string"
          },
          "resource": {
            "name": "string"
          }
        }
      ],
      "tenants": [
        {
          "name": "string",
        }
      ]
    }
  ],
  "total_entries": 0
}

Create role

The list of associated tenants needs to be added as mandatory field.

UI

Rest API

Input

POST /roles

{
  "name": "string",
  "actions": [
    {
      "action": {
        "name": "string"
      },
      "resource": {
        "name": "string"
      }
    }
  ],
  "tenants": [
    {
      "name": "string",
    }
  ]
}

Output

{
  "name": "string",
  "actions": [
    {
      "action": {
        "name": "string"
      },
      "resource": {
        "name": "string"
      }
    }
  ],
  "tenants": [
    {
      "name": "string",
    }
  ]
}

Errors

Unchanged

Get role

The list of associated tenants needs to be added.

Only if the role is associated to the same tenant(s) as the logged in user can be returned.

Open question: What if Admin is associated to HR and Marketing tenants and the logged in user has permission to read roles in the HR tenant. Should HR be the only tenant returned as part of Admin's associated tenants information?

UI

Rest API

Only the output has to be updated. The field "tenants"needs to be added.

Output

{
  "name": "string",
  "actions": [
    {
      "action": {
        "name": "string"
      },
      "resource": {
        "name": "string"
      }
    }
  ],
  "tenants": [
    {
      "name": "string",
    }
  ]
}

Edit role

The list of associated tenants needs to be added.

Only if the role is associated to the same tenant(s) as the logged in user can be modified.

Open question: What if Admin is associated to HR and Marketing tenants and the logged in user has permission to edit roles in the HR tenant. Should the update request be applied to only tenants the logged in user has permissions to? In other words, in this example, if the logged in user edit the role Admin and remove HR from the tenants list, it will send an empty list as tenants associated to the role but as consequence will just remove HR from the association (hence keeping Marketing as tenant associated to the role Admin)?

UI

Rest API

Input

PATCH /roles/{role_name}

{
  "name": "string",
  "actions": [
    {
      "action": {
        "name": "string"
      },
      "resource": {
        "name": "string"
      }
    }
  ],
  "tenants": [
    {
      "name": "string",
    }
  ]
}

Output

{
  "name": "string",
  "actions": [
    {
      "action": {
        "name": "string"
      },
      "resource": {
        "name": "string"
      }
    }
  ],
  "tenants": [
    {
      "name": "string",
    }
  ]
}

Errors

Unchanged

List users

For each user, the associated role needs to be updated to the associated “Role in Tenant”.

Only users which belong to the same tenant(s) as the logged in user need to be returned.

Open question: What if John belongs to HR and Marketing tenants and the logged in user has permission to list users in the HR tenant. Should HR be the only tenant returned as part of John’s "role in tenant" information?

UI

Rest API

Only the output has to be updated. The field "roles" needs to be replaced by "tenant_roles".

Output

{
  "users": [
    {
      "first_name": "string",
      "last_name": "string",
      "username": "string",
      "email": "string",
      "active": true,
      "last_login": "string",
      "login_count": 0,
      "failed_login_count": 0,
      "tenant_roles": [
        {
          "role": {
            "name": "string"
          },
          "tenant": {
            "name": "string"
          }
        }
      ],
      "created_on": "string",
      "changed_on": "string"
    }
  ],
  "total_entries": 0
}

Create user

The associated role needs to be updated to the associated “Role in Tenant”.

UI

Rest API

Input

POST /users

{
  "first_name": "string",
  "last_name": "string",
  "username": "string",
  "email": "string",
  "tenant_roles": [
    {
      "role": {
        "name": "string"
      },
      "tenant": {
        "name": "string"
      }
    }
  ]
}

Output

{
  "first_name": "string",
  "last_name": "string",
  "username": "string",
  "email": "string",
  "active": true,
  "last_login": "string",
  "login_count": 0,
  "failed_login_count": 0,
  "tenant_roles": [
    {
      "role": {
        "name": "string"
      },
      "tenant": {
        "name": "string"
      }
    }
  ],
  "created_on": "string",
  "changed_on": "string"
}

Errors

Unchanged

Get user

The associated role needs to be updated to the associated “Role in Tenant”.

Only if the user belongs to the same tenant(s) as the logged in user need can be returned.

Open question: What if John belongs to HR and Marketing tenants and the logged in user has permission to read users in the HR tenant. Should HR be the only tenant returned as part of John’s "role in tenant" information?

UI

Rest API

Only the output has to be updated. The field "roles" needs to be replaced by "tenant_roles".

Output

{
  "first_name": "string",
  "last_name": "string",
  "username": "string",
  "email": "string",
  "active": true,
  "last_login": "string",
  "login_count": 0,
  "failed_login_count": 0,
  "tenant_roles": [
    {
      "role": {
        "name": "string"
      },
      "tenant": {
        "name": "string"
      }
    }
  ],
  "created_on": "string",
  "changed_on": "string"
}

Edit user

The associated role needs to be updated to the associated “Role in Tenant”.

Only if the user belongs to the same tenant(s) as the logged in user need can be modified.

Open question: What if John belongs to HR and Marketing tenants and the logged in user has permission to edit users in the HR tenant. Should the update request be applied to only tenants the logged in user has permissions to? In other words, in this example, if the logged in user edit the user John and remove "Admin in HR" from the list, it will send an empty list as "role in tenant" associated to the user but as consequence will just remove "Admin in HR" from the association (hence keeping "Admin in Marketing" as "role in tenant" associated to the user John)?

UI

Rest API

Input

PATCH /users/{username}

{
  "first_name": "string",
  "last_name": "string",
  "username": "string",
  "email": "string",
  "tenant_roles": [
    {
      "role": {
        "name": "string"
      },
      "tenant": {
        "name": "string"
      }
    }
  ]
}

Output

{
  "first_name": "string",
  "last_name": "string",
  "username": "string",
  "email": "string",
  "active": true,
  "last_login": "string",
  "login_count": 0,
  "failed_login_count": 0,
  "tenant_roles": [
    {
      "role": {
        "name": "string"
      },
      "tenant": {
        "name": "string"
      }
    }
  ],
  "created_on": "string",
  "changed_on": "string"
}

Errors

Unchanged

Airflow CLI

Tenants

A new sub-command needs to be created: "tenants". Below are the commands associated to this new sub-command.

airflow tenants [-h] COMMAND ...

Roles

Users

Permissions

The following permissions need to be added in order to grant/deny access to tenants management.

Default configuration

A new Airflow configuration is introduced to whether activate this new entity "tenants":

• [core] enable_tenants

When enabling this option, if there is no tenant already configured, a default tenant configuration will be created. This initial configuration needs to be created so that the transition between being non multi tenancy and being tenancy is as seamless as possible.

1. A tenant named Default needs to be created
2. All existing roles needs to be associated with the tenant Default
3. For each user, assign the existing associated roles to the tenant Default
4. Grant all permissions defined above to the role Admin
5. Create a new role named Tenant admin (we can revisit the name) which has all the permissions besides the ones related to Tenant (listed above). Associate this role to the tenant Default as well. This role is not needed but it is worth creating it since we want to encourage users to have clear separation between tenants. The role Admin should be restrained to a few individuals.

POC

A POC will be created in order to verify that the new tenant entity and modified relationship using existing FlaskApp Builder actually works. This POC will be done before AIP voting.

Other considerations

Why is it needed?

This change introduce this new entity "tenant" which allows administrators/operators to group users in different groups.

Which users are affected by the change?

Only those that set [core]enable_tenants=True

How are users affected by the change? (e.g. DB upgrade required?)

• DB upgrade is required
• UI experience is different: some new pages and some existing pages are modified
• Rest API: some non backward compatible changes
• Airflow CLI: some non backward compatible changes

What defines this AIP as "done"?

Users can be organized in tenants by administrators/operators.