Status
Page properties | |||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Motivation
This is part of AIP-1, which aims to run Airflow in a multi-tenant way. Today, Airflow doesn’t have built-in support for multi-tenant environments. Airflow provides Role-Based Access Control (RBAC), but the access control is not fine-grained and allows various Airflow components and users to access/modify resources across the environment without restriction. This document describes a proposal to introduce a tenant model paving a way to running Airflow in a multi-tenant fashion.
Note: The resource access control is out of scope for this AIP. It will be done in a separate AIP(s). The purpose of this AIP is to introduce a new “tenant” entity that will be utilized in the future AIPs related to multi tenancy.
Considerations
Current state
The current access management uses a role based access control (RBAC) to authorize users to access specific pages/resources. Here is the database diagram.
To better understand it, here is an example. This is only a simplified version of an Airflow configuration.
- The user John is associated to the roles Admin and Op
- The role Admin has the permission “can read on Users”
- The role Op has the permission “can read on Variables”
- As a result, John can read the variables and the users
Proposal
Database
To better understand it, here is an example:
- The role Admin is associated to the tenants HR and Marketing which means the role Admin can be associated with users in the tenants HR and Marketing
- The user John belongs to the tenant HR with the role Admin
- The user John belongs to the tenant Marketing with the role Admin
- The role Admin has the permission “can read on Users”
- As a result, John can read the users of the tenant HR and Marketing
---
- The role Op is associated to the tenant Marketing which means the role Op can be associated with users in the tenant Marketing
- The user Bob belongs to the tenant Marketing with the role Op
- The role Op has the permission “can read on Variables”
- As a result, Bob can read the variables of the tenant Marketing
Resource management changes
In order to manage the new entity tenant, some modifications have to be made across the UI, the Rest API and the Airflow CLI. Airflow CLI changes will be described in its own section at the end of this section.
Tenant menu access
List tenants
UI
Rest API
Input
GET /tenants
Query parameters | |||
---|---|---|---|
Name | Type | Default | Description |
limit | integer | 100 | The numbers of items to return. |
offset | integer | 0 | The number of items to skip before starting to collect the result set. |
order_by | string | The name of the field to order the results by. Prefix a field name with |
Output
Code Block | ||
---|---|---|
| ||
{ "tenants": [ { "name": "string" } ], "total_entries": 0 } |
Errors
Code | Description | Schema |
---|---|---|
401 | Request not authenticated due to missing, invalid, authentication info. | Similar to get_roles API |
403 | Client does not have sufficient permission. | Similar to get_roles API |
Create tenant
UI
Rest API
Input
POST /tenants
Body schema | ||
---|---|---|
Name | Type | Description |
name | string | The tenant name |
Output
Code Block | ||
---|---|---|
| ||
{ "name": "string" } |
Errors
Code | Description | Schema |
---|---|---|
400 | Client specified an invalid argument. | Similar to post_role API |
401 | Request not authenticated due to missing, invalid, authentication info. | Similar to post_role API |
403 | Client does not have sufficient permission. | Similar to post_role API |
Get tenant
UI
Rest API
Input
GET /tenants/{tenant_name}
Path parameters | ||
---|---|---|
Name | Type | Description |
tenant_name | string | The tenant name |
Output
Code Block | ||
---|---|---|
| ||
{ "name": "string" } |
Errors
Code | Description | Schema |
---|---|---|
401 | Request not authenticated due to missing, invalid, authentication info. | Similar to get_role API |
403 | Client does not have sufficient permission. | Similar to get_role API |
404 | A specified resource is not found. | Similar to get_role API |
Edit tenant
UI
Rest API
Input
PATCH /tenants/{tenant_name}
Path parameters | ||
---|---|---|
Name | Type | Description |
tenant_name | string | The tenant name |
Body schema | ||
name | string | The tenant name |
Output
Code Block | ||
---|---|---|
| ||
{ "name": "string" } |
Errors
Code | Description | Schema |
---|---|---|
400 | Client specified an invalid argument. | Similar to patch_role API |
401 | Request not authenticated due to missing, invalid, authentication info. | Similar to patch_role API |
403 | Client does not have sufficient permission. | Similar to patch_role API |
404 | A specified resource is not found. | Similar to patch_role API |
Delete tenant
UI
Rest API
Input
DELETE /tenants/{tenant_name}
Path parameters | ||
---|---|---|
Name | Type | Description |
tenant_name | string | The tenant name |
Output
No output
Errors
Code | Description | Schema |
---|---|---|
400 | Client specified an invalid argument. | Similar to delete_role API |
401 | Request not authenticated due to missing, invalid, authentication info. | Similar to delete_role API |
403 | Client does not have sufficient permission. | Similar to delete_role API |
404 | A specified resource is not found. | Similar to delete_role API |
List roles
For each role, the list of associated tenants needs to be added.
Only roles which are associated to the same tenant(s) as the logged in user need to be returned.
Open question: What if Admin is associated to HR and Marketing tenants and the logged in user has permission to list roles in the HR tenant. Should HR be the only tenant returned as part of Admin's associated tenants information?
UI
Rest API
Only the output has to be updated. The field "tenants"needs to be added.
Output
Code Block | ||
---|---|---|
| ||
{ "roles": [ { "name": "string", "actions": [ { "action": { "name": "string" }, "resource": { "name": "string" } } ], "tenants": [ { "name": "string", } ] } ], "total_entries": 0 } |
Create role
The list of associated tenants needs to be added as mandatory field.
UI
Rest API
Input
POST /roles
Body schema | |||
---|---|---|---|
Name | Type | Description | Comments |
tenants | array of objects | List of tenants associated to the role | To be added |
Code Block | ||||
---|---|---|---|---|
| ||||
{ "name": "string", "actions": [ { "action": { "name": "string" }, "resource": { "name": "string" } } ], "tenants": [ { "name": "string", } ] } |
Output
Code Block | ||
---|---|---|
| ||
{ "name": "string", "actions": [ { "action": { "name": "string" }, "resource": { "name": "string" } } ], "tenants": [ { "name": "string", } ] } |
Errors
Unchanged
Get role
The list of associated tenants needs to be added.
Only if the role is associated to the same tenant(s) as the logged in user can be returned.
Open question: What if Admin is associated to HR and Marketing tenants and the logged in user has permission to read roles in the HR tenant. Should HR be the only tenant returned as part of Admin's associated tenants information?
UI
Rest API
Only the output has to be updated. The field "tenants"needs to be added.
Output
Code Block | ||
---|---|---|
| ||
{ "name": "string", "actions": [ { "action": { "name": "string" }, "resource": { "name": "string" } } ], "tenants": [ { "name": "string", } ] } |
Edit role
The list of associated tenants needs to be added.
Only if the role is associated to the same tenant(s) as the logged in user can be modified.
Open question: What if Admin is associated to HR and Marketing tenants and the logged in user has permission to edit roles in the HR tenant. Should the update request be applied to only tenants the logged in user has permissions to? In other words, in this example, if the logged in user edit the role Admin and remove HR from the tenants list, it will send an empty list as tenants associated to the role but as consequence will just remove HR from the association (hence keeping Marketing as tenant associated to the role Admin)?
UI
Rest API
Input
PATCH /roles/{role_name}
Query parameters | |||
---|---|---|---|
Name | Type | Description | Comments |
update_mask | Array of strings | The fields to update on the resource. If absent or empty, all modifiable fields are updated. A comma-separated list of fully qualified names of fields. | Need to handle the new field "tenants" |
Body schema | |||
Name | Type | Description | Comments |
tenants | array of objects | List of tenants associated to the role | To be added |
Code Block | ||||
---|---|---|---|---|
| ||||
{ "name": "string", "actions": [ { "action": { "name": "string" }, "resource": { "name": "string" } } ], "tenants": [ { "name": "string", } ] } |
Output
Code Block | ||
---|---|---|
| ||
{ "name": "string", "actions": [ { "action": { "name": "string" }, "resource": { "name": "string" } } ], "tenants": [ { "name": "string", } ] } |
Errors
Unchanged
List users
For each user, the associated role needs to be updated to the associated “Role in Tenant”.
Only users which belong to the same tenant(s) as the logged in user need to be returned.
Open question: What if John belongs to HR and Marketing tenants and the logged in user has permission to list users in the HR tenant. Should HR be the only tenant returned as part of John’s "role in tenant" information?
UI
Rest API
Only the output has to be updated. The field "roles" needs to be replaced by "tenant_roles".
Output
Code Block | ||
---|---|---|
| ||
{ "users": [ { "first_name": "string", "last_name": "string", "username": "string", "email": "string", "active": true, "last_login": "string", "login_count": 0, "failed_login_count": 0, "tenant_roles": [ { "role": { "name": "string" }, "tenant": { "name": "string" } } ], "created_on": "string", "changed_on": "string" } ], "total_entries": 0 } |
Create user
The associated role needs to be updated to the associated “Role in Tenant”.
UI
Rest API
Input
POST /users
Body schema | |||
---|---|---|---|
Name | Type | Description | Comments |
roles | Array of objects | User roles | To be removed |
tenant_roles | Array of objects | User tenant roles | To be added |
Code Block | ||||
---|---|---|---|---|
| ||||
{ "first_name": "string", "last_name": "string", "username": "string", "email": "string", "tenant_roles": [ { "role": { "name": "string" }, "tenant": { "name": "string" } } ] } |
Output
Code Block | ||
---|---|---|
| ||
{ "first_name": "string", "last_name": "string", "username": "string", "email": "string", "active": true, "last_login": "string", "login_count": 0, "failed_login_count": 0, "tenant_roles": [ { "role": { "name": "string" }, "tenant": { "name": "string" } } ], "created_on": "string", "changed_on": "string" } |
Errors
Unchanged
Get user
The associated role needs to be updated to the associated “Role in Tenant”.
Only if the user belongs to the same tenant(s) as the logged in user need can be returned.
Open question: What if John belongs to HR and Marketing tenants and the logged in user has permission to read users in the HR tenant. Should HR be the only tenant returned as part of John’s "role in tenant" information?
UI
Rest API
Only the output has to be updated. The field "roles" needs to be replaced by "tenant_roles".
Output
Code Block | ||
---|---|---|
| ||
{ "first_name": "string", "last_name": "string", "username": "string", "email": "string", "active": true, "last_login": "string", "login_count": 0, "failed_login_count": 0, "tenant_roles": [ { "role": { "name": "string" }, "tenant": { "name": "string" } } ], "created_on": "string", "changed_on": "string" } |
Edit user
The associated role needs to be updated to the associated “Role in Tenant”.
Only if the user belongs to the same tenant(s) as the logged in user need can be modified.
Open question: What if John belongs to HR and Marketing tenants and the logged in user has permission to edit users in the HR tenant. Should the update request be applied to only tenants the logged in user has permissions to? In other words, in this example, if the logged in user edit the user John and remove "Admin in HR" from the list, it will send an empty list as "role in tenant" associated to the user but as consequence will just remove "Admin in HR" from the association (hence keeping "Admin in Marketing" as "role in tenant" associated to the user John)?
UI
Rest API
Input
PATCH /users/{username}
Query parameters | |||
---|---|---|---|
Name | Type | Description | Comments |
update_mask | Array of strings | The fields to update on the resource. If absent or empty, all modifiable fields are updated. A comma-separated list of fully qualified names of fields. | Need to handle the new field "tenant_roles" |
Body schema | |||
Name | Type | Description | Comments |
roles | Array of objects | User roles | To be removed |
tenant_roles | Array of objects | User tenant roles | To be added |
Code Block | ||||
---|---|---|---|---|
| ||||
{ "first_name": "string", "last_name": "string", "username": "string", "email": "string", "tenant_roles": [ { "role": { "name": "string" }, "tenant": { "name": "string" } } ] } |
Output
Code Block | ||
---|---|---|
| ||
{ "first_name": "string", "last_name": "string", "username": "string", "email": "string", "active": true, "last_login": "string", "login_count": 0, "failed_login_count": 0, "tenant_roles": [ { "role": { "name": "string" }, "tenant": { "name": "string" } } ], "created_on": "string", "changed_on": "string" } |
Errors
Unchanged
Airflow CLI
Tenants
A new sub-command needs to be created: "tenants". Below are the commands associated to this new sub-command.
airflow tenants [-h] COMMAND ...
Command | Description | Arguments |
---|---|---|
create | Create tenant |
|
delete | Delete tenant |
|
export | Export tenants from db to JSON file | Similar to export roles |
import | Import tenants from JSON file to db | Similar to import roles |
list | List tenants |
|
Roles
Command | Description | Arguments | Comments |
---|---|---|---|
add-tenant | Associate a tenant to the role |
| To be added |
del-tenant | Dissociate a tenant from the role |
| To be added |
create |
| Need to add --tenant as mandatory argument. To keep it simple, associate only one tenant at creation. To add more, the user can use "add-tenant". | |
export | Need to export tenants | ||
import | Need to import tenants | ||
list | Need to return tenants |
Users
Command | Description | Arguments | Comments |
---|---|---|---|
add-role | To be removed | ||
remove-role | To be removed | ||
add-role-tenant | Add a role associated to a tenant to a user |
| To be added |
remove-role-tenant | Remove a role associated to a tenant to a user |
| To be added |
create |
| Need to add the argument --tenant | |
export | Need to export role tenants | ||
import | Need to import role tenants | ||
list | Need to return role tenants |
Permissions
The following permissions need to be added in order to grant/deny access to tenants management.
Action | Resource |
can_create | Tenant |
can_delete | Tenant |
can_edit | Tenant |
can_read | Tenant |
menu_access | List Tenants |
Default configuration
A new Airflow configuration is introduced to whether activate this new entity "tenants":
[core] enable_tenants
When enabling this option, if there is no tenant already configured, a default tenant configuration will be created. This initial configuration needs to be created so that the transition between being non multi tenancy and being tenancy is as seamless as possible.
- A tenant named Default needs to be created
- All existing roles needs to be associated with the tenant Default
- For each user, assign the existing associated roles to the tenant Default
- Grant all permissions defined above to the role Admin
- Create a new role named Tenant admin (we can revisit the name) which has all the permissions besides the ones related to Tenant (listed above). Associate this role to the tenant Default as well. This role is not needed but it is worth creating it since we want to encourage users to have clear separation between tenants. The role Admin should be restrained to a few individuals.
POC
A POC will be created in order to verify that the new tenant entity and modified relationship using existing FlaskApp Builder actually works. This POC will be done before AIP voting.
Other considerations
Why is it needed?
This change introduce this new entity "tenant" which allows administrators/operators to group users in different groups.
Which users are affected by the change?
Only those that set [core]enable_tenants=True
How are users affected by the change? (e.g. DB upgrade required?)
- DB upgrade is required
- UI experience is different: some new pages and some existing pages are modified
- Rest API: some non backward compatible changes
- Airflow CLI: some non backward compatible changes
What defines this AIP as "done"?
Users can be organized in tenants by administrators/operators.