Related JIRAs

Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.

Goal

Currently, the Admin API allows management and access to topologies, descriptors, and provider configurations but does not provide access to the Service Definitions for the proxied services.

The goal is to expand the Admin API to include service definitions and allow for customization through the API: we should be able to create, read, update and delete service definitions including both service and rewrite XML files. Changes made via the new API should target topology file redeployment that is referencing the changed service definition.

How to fetch service definitions

A service definition in Knox consists of two following major parts:

  • service descriptor: identifies and describes the service with name/role/version. This is specified in a file named service.xml  under GATEYAY_DATA_DIR/services/$service/$version folder.
  • rules: specifies the list of rewrite rules - read from rewrite.xml - belong to the given service/role/version. This is optional, in case there are no rewrite rules for the given service you will not see any XML element/JSON data with the name 'rules'.

All of the below fetch operations will provide you a result with the following structure:

  • as XML
    <serviceDefinitions>
        <serviceDefinition>
            <service name="..." role="..." version="...">
                ...
            </service>
            <rules>
                ...
            </rules>
        </serviceDefinition>
        <serviceDefinition>
            ...
        <serviceDefinition>
        ...
    </serviceDefinitions>
  • as JSON
{
   "serviceDefinitions" : {
      "serviceDefinition" : [ {
         "service" : {
            "name" : "...",
            "role" : "...",
            "version" : "...",
             ...
         },
         "rules" : {
             ...
         }
      }, {
         "service" : {
         },
         "rules" : {
      } ]
   }
}


To read an existing service definition or a list of service definitions you should send a GET request to one of the following REST API endpoints:

Fetching all service definitions

https://{gateway-host}:{gateway-port}/{gateway-path}/admin/api/v1/servicedefinitions

Sample:

$ curl -ku admin:*** -H "Accept: application/json" -X GET 'https://localhost:8443/gateway/admin/api/v1/servicedefinitions/'

{
   "serviceDefinitions" : {
      "serviceDefinition" : [ {
         "service" : {
            "name" : "ambari",
            "role" : "AMBARI",
            "version" : "0.2.2.0",
            "routes" : {
               "route" : [ {
                  "path" : "/ambari/api/v1/**",
                  "rewrite" : [ {
                     "apply" : "AMBARI/ambari/api/outbound",
                     "to" : "response.body"
                  }, {
                     "apply" : "AMBARI/ambari/api/inbound",
                     "to" : "request.body"
                  } ]
               }, {
                  "path" : "/ambari/api/v1/persist/*?*",
                  "rewrite" : [ {
                     "apply" : "AMBARI/ambari/api/inbound",
                     "to" : "request.body"
                  } ]
               } ]
            }
         },
         "rules" : {
            "rule" : {
               "dir" : "IN",
               "name" : "AMBARI/ambari/inbound",
               "pattern" : "*://*:*/**/ambari/api/v1/{**}?{**}",
               "rewrite" : {
                  "template" : "{$serviceUrl[AMBARI]}/api/v1/{**}?{**}"
               }
            },
            "rule" : {
               "dir" : "OUT",
               "name" : "AMBARI/ambari/href/outbound",
               "match" : {
                  "pattern" : "*://*:*/api/{**}?{**}"
               },
               "rewrite" : {
                  "template" : "{$frontend[url]}/ambari/api/{**}?{**}"
               }
            },
            "rule" : {
               "dir" : "OUT",
               "name" : "AMBARI/ambari/context_path/outbound",
               "match" : {
                  "pattern" : "/{**}"
               },
               "rewrite" : {
                  "template" : "{$frontend[path]}/ambari/{**}"
               }
            },
            "filter" : {
               "name" : "AMBARI/ambari/api/outbound",
               "content" : {
                  "asType" : "application/json",
                  "type" : "text/plain",
                  "apply" : {
                     "path" : "$.**.href",
                     "rule" : "AMBARI/ambari/href/outbound"
                  },
                  "apply" : {
                     "path" : "$.**.href",
                     "rule" : "AMBARI/ambari/href/outbound"
                  },
                  "apply" : {
                     "path" : "$.**.context_path",
                     "rule" : "AMBARI/ambari/context_path/outbound"
                  }
               }
            },
            "filter" : {
               "name" : "AMBARI/ambari/api/inbound",
               "content" : {
                  "asType" : "application/octet-stream",
                  "type" : "application/x-www-form-urlencoded"
               }
            }
         }
      }, {
         "service" : {
            "name" : "ambari",
            "role" : "AMBARI",
            "version" : "2.2.0",
            "dispatch" : {
               "classname" : "org.apache.knox.gateway.dispatch.PassAllHeadersNoEncodingDispatch",
               "use-two-way-ssl" : false,
               "param" : [ ]
            },
       ...
      }, {
         "service" : {
            "name" : "zeppelinws",
            "role" : "ZEPPELINWS",
            "version" : "0.8.1",
            "routes" : {
               "route" : [ {
                  "path" : "/zeppelin/ws",
                  "rewrite" : [ {
                     "apply" : "ZEPPELINWS/zeppelin/ws/inbound",
                     "to" : "request.url"
                  } ]
               }, {
                  "path" : "/zeppelin/ws**",
                  "rewrite" : [ {
                     "apply" : "ZEPPELINWS/zeppelin/inbound",
                     "to" : "request.url"
                  } ]
               } ]
            }
         },
         "rules" : {
            "rule" : {
               "dir" : "IN",
               "name" : "ZEPPELINWS/zeppelin/ws/inbound",
               "pattern" : "*://*:*/**/ws",
               "rewrite" : {
                  "template" : "{$serviceUrl[ZEPPELINWS]}/ws"
               }
            },
            "rule" : {
               "dir" : "IN",
               "name" : "ZEPPELINWS/zeppelin/inbound",
               "pattern" : "*://*:*/**/ws{**}",
               "rewrite" : {
                  "template" : "{$serviceUrl[ZEPPELINWS]}/ws{**}"
               }
            }
         }
      } ]
   }

Fetching all service definitions for a particular service

https://{gateway-host}:{gateway-port}/{gateway-path}/admin/api/v1/servicedefinitions/{serviceName}

Sample:

$ curl -ku admin:*** -H "Accept: application/json" -X GET 'https://localhost:8443/gateway/admin/api/v1/servicedefinitions/atlas-api'
{
   "serviceDefinitions" : {
      "serviceDefinition" : [ {
         "service" : {
            "name" : "atlas-api",
            "role" : "ATLAS-API",
            "version" : "0.1.2.0",
            "dispatch" : {
               "classname" : "org.apache.knox.gateway.dispatch.DefaultDispatch",
               "ha-classname" : "org.apache.knox.gateway.ha.dispatch.AtlasApiTrustedProxyHaDispatch",
               "use-two-way-ssl" : false,
               "param" : [ ]
            },
            "routes" : {
               "route" : [ {
                  "path" : "/atlas/api/**"
               } ]
            }
         },
         "rules" : {
            "rule" : {
               "dir" : "OUT",
               "name" : "ATLAS-API/atlas/outbound",
               "pattern" : "hdfs://{host}/{path=**}?{**}",
               "rewrite" : {
                  "template" : "hdfs://{host}/{path=**}?{**}"
               }
            },
            "rule" : {
               "dir" : "IN",
               "name" : "ATLAS-API/atlas/inbound",
               "pattern" : "*://*:*/**/atlas/api/{path=**}?{**}",
               "rewrite" : {
                  "template" : "{$serviceUrl[ATLAS-API]}/api/{path=**}?{**}"
               }
            }
         }
      }, {
         "service" : {
            "name" : "atlas-api",
            "role" : "ATLAS-API",
            "version" : "2.0.0",
            "dispatch" : {
               "classname" : "org.apache.knox.gateway.dispatch.DefaultDispatch",
               "ha-classname" : "org.apache.knox.gateway.ha.dispatch.AtlasApiTrustedProxyHaDispatch",
               "use-two-way-ssl" : false,
               "param" : [ ]
            },
            "routes" : {
               "route" : [ {
                  "path" : "/atlas/api/**"
               } ]
            }
         },
         "rules" : {
            "rule" : {
               "dir" : "OUT",
               "name" : "ATLAS-API/atlas/outbound",
               "pattern" : "hdfs://{host}/{path=**}?{**}",
               "rewrite" : {
                  "template" : "hdfs://{host}/{path=**}?{**}"
               }
            },
            "rule" : {
               "dir" : "IN",
               "name" : "ATLAS-API/atlas/inbound",
               "pattern" : "*://*:*/**/atlas/api/{path=**}?{**}",
               "rewrite" : {
                  "template" : "{$serviceUrl[ATLAS-API]}/api/{path=**}?{**}"
               }
            }
         }
      }, {
         "service" : {
            "name" : "atlas-api",
            "role" : "ATLAS-API",
            "version" : "0.8.0",
            "dispatch" : {
               "classname" : "org.apache.knox.gateway.dispatch.PassAllHeadersDispatch",
               "ha-classname" : "org.apache.knox.gateway.ha.dispatch.AtlasApiHaDispatch",
               "use-two-way-ssl" : false,
               "param" : [ ]
            },
            "policies" : {
               "policy" : [ {
                  "role" : "webappsec"
               }, {
                  "name" : "Anonymous",
                  "role" : "authentication"
               }, {
                  "role" : "rewrite"
               }, {
                  "role" : "authorization"
               } ]
            },
            "routes" : {
               "route" : [ {
                  "path" : "/atlas/api/**"
               } ]
            }
         },
         "rules" : {
            "rule" : {
               "dir" : "OUT",
               "name" : "ATLAS-API/atlas/outbound",
               "pattern" : "hdfs://{host}/{path=**}?{**}",
               "rewrite" : {
                  "template" : "hdfs://{host}/{path=**}?{**}"
               }
            },
            "rule" : {
               "dir" : "IN",
               "name" : "ATLAS-API/atlas/inbound",
               "pattern" : "*://*:*/**/atlas/api/{path=**}?{**}",
               "rewrite" : {
                  "template" : "{$serviceUrl[ATLAS-API]}/api/{path=**}?{**}"
               }
            }
         }
      } ]
   }

Fetching all service definitions for a particular service and role

https://{gateway-host}:{gateway-port}/{gateway-path}/admin/api/v1/servicedefinitions/{serviceName}/{roleName}

Sample: curl -ku admin:*** -H "Accept: application/json" -X GET 'https://localhost:8443/gateway/admin/api/v1/servicedefinitions/atlas-api/atlas-api' generates the same output as above for the atlas-api service since it only has atlas-api role with different versions

Fetching a service definition for a particular service, role, and version

https://{gateway-host}:{gateway-port}/{gateway-path}/admin/api/v1/servicedefinitions/{serviceName}/{roleName}/{version}

Sample:

$ curl -ku admin:*** -H "Accept: application/json" -X GET 'https://localhost:8443/gateway/admin/api/v1/servicedefinitions/atlas-api/atlas-api/2.0.0'
{
   "serviceDefinitions" : {
      "serviceDefinition" : [ {
         "service" : {
            "name" : "atlas-api",
            "role" : "ATLAS-API",
            "version" : "2.0.0",
            "dispatch" : {
               "classname" : "org.apache.knox.gateway.dispatch.DefaultDispatch",
               "ha-classname" : "org.apache.knox.gateway.ha.dispatch.AtlasApiTrustedProxyHaDispatch",
               "use-two-way-ssl" : false,
               "param" : [ ]
            },
            "routes" : {
               "route" : [ {
                  "path" : "/atlas/api/**"
               } ]
            }
         },
         "rules" : {
            "rule" : {
               "dir" : "OUT",
               "name" : "ATLAS-API/atlas/outbound",
               "pattern" : "hdfs://{host}/{path=**}?{**}",
               "rewrite" : {
                  "template" : "hdfs://{host}/{path=**}?{**}"
               }
            },
            "rule" : {
               "dir" : "IN",
               "name" : "ATLAS-API/atlas/inbound",
               "pattern" : "*://*:*/**/atlas/api/{path=**}?{**}",
               "rewrite" : {
                  "template" : "{$serviceUrl[ATLAS-API]}/api/{path=**}?{**}"
               }
            }
         }
      } ]
   }
}


Important note: in case you are fetching a non-existing service definition you will receive an empty list of service definitions along with an HTTP response with 200 response code. For instance:


$ curl -iku admin:*** -H "Accept: application/json" -X GET 'https://localhost:8443/gateway/admin/api/v1/servicedefinitions/nonExistingService'
HTTP/1.1 200 OK
Date: Tue, 19 Nov 2019 12:56:52 GMT
Set-Cookie: KNOXSESSIONID=node014vzonbq60zjn19nnzpveqsiw316.node0; Path=/gateway/admin; Secure; HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/admin; Max-Age=0; Expires=Mon, 18-Nov-2019 12:56:52 GMT
Content-Type: application/json
Content-Length: 68
Server: Jetty(9.4.22.v20191022)

{
   "serviceDefinitions" : {
      "serviceDefinition" : [ ]
   }
}

How to add a new service definition

The  https://{gateway-host}:{gateway-port}/{gateway-path}/admin/api/v1/servicedefinitions endpoint accepts POST requests to create a brand new service definition. In case the service definition you are trying to create exists already an HTTP response with 500 error code will be returned.

Sample:

$ cat testServiceDefinitionPayload.xml 
<?xml version="1.0" encoding="UTF-8"?>
<serviceDefinition>
    <service name="mySampleService" role="MYSAMPLEROLE" version="2.0.0">
        <dispatch classname="org.apache.knox.gateway.dispatch.PassAllHeadersNoEncodingDispatch" use-two-way-ssl="false"/>
        <policies>
            <policy role="webappsec"/>
            <policy name="Anonymous" role="authentication"/>
        </policies>
        <routes>
            <route path="/mysamplerole">
                <rewrite apply="mySampleService/mysamplerole/inbound/root" to="request.url"/>
                <rewrite apply="mySampleService/mysamplerole/outbound/mainpage" to="response.body"/>
            </route>
            <route path="/mysamplerole/**">
                <rewrite apply="mySampleService/mysamplerole/inbound/path" to="request.url"/>
            </route>
            <route path="/mysamplerole/views/SMARTSENSE/**/assets/hstapp-*.js">
                <rewrite apply="mySampleService/mysamplerole/outbound/apiendpoint" to="response.body"/>
            </route>
        </routes>
    </service>
    <rules>
        <rule dir="IN" name="mySampleService/mysamplerole/inbound/root" pattern="*://*:*/**/mysamplerole/">
            <rewrite template="{$serviceUrl[mySampleService]}/"/>
        </rule>
        <rule dir="IN" name="mySampleService/mysamplerole/inbound/path" pattern="*://*:*/**/mysamplerole/{**}">
            <rewrite template="{$serviceUrl[mySampleService]}/{**}"/>
        </rule>
        <rule dir="OUT" name="mySampleService/mysamplerole/outbound/sitepath">
            <rewrite template="{$frontend[path]}/mysamplerole"/>
        </rule>
        <rule dir="OUT" name="mySampleService/mysamplerole/outbound/websocket">
            <rewrite template="{$frontend[path]}/mysamplerolews/api/stomp/v1"/>
        </rule>

        <filter name="mySampleService/mysamplerole/outbound/apiendpoint">
            <content type="*/x-javascript">
                <apply path="/api/v1" rule="mySampleService/mysamplerole/outbound/extrapath"/>
            </content>
            <content type="application/javascript">
                <apply path="/api/v1" rule="mySampleService/mysamplerole/outbound/extrapath"/>
            </content>
        </filter>
        <filter name="mySampleService/mysamplerole/outbound/apiendpoint/html">
            <content type="text/html">
                <apply path="/api/v1" rule="mySampleService/mysamplerole/outbound/extrapath"/>
            </content>
        </filter>
        <filter name="mySampleService/mysamplerole/outbound/mainpage">
            <content type="*/html">
                <apply path="stylesheets/vendor.css" rule="mySampleService/mysamplerole/outbound/vendorcss" />
                <apply path="stylesheets/app.css" rule="mySampleService/mysamplerole/outbound/appcss" />
                <apply path="javascripts/vendor.js" rule="mySampleService/mysamplerole/outbound/vendorjs" />
                <apply path="javascripts/app.js" rule="mySampleService/mysamplerole/outbound/appjs" />
                <apply path="/img/logo.png" rule="mySampleService/mysamplerole/outbound/favicon"/>
                <apply path="/licenses/NOTICE.txt" rule="mySampleService/mysamplerole/outbound/notice"/>
            </content>
        </filter>
    </rules>
</serviceDefinition>


$ curl -H "Content-Type: application/xml" -iku admin:*** -d "@testServiceDefinitionPayload.xml" -X POST 'https://localhost:8443/gateway/admin/api/v1/servicedefinitions'

HTTP/1.1 100 Continue

HTTP/1.1 201 Created
Date: Tue, 19 Nov 2019 13:01:29 GMT
Set-Cookie: KNOXSESSIONID=node0cyn7v4caqbpk1wsou1qmdq1qb17.node0; Path=/gateway/admin; Secure; HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/admin; Max-Age=0; Expires=Mon, 18-Nov-2019 13:01:29 GMT
Location: https://localhost:8443/gateway/admin/api/v1/servicedefinitions/mySampleService/MYSAMPLEROLE/2.0.0
Content-Length: 0
Server: Jetty(9.4.22.v20191022)

$ curl -H "Content-Type: application/xml" -iku admin:*** -d "@testServiceDefinitionPayload.xml" -X POST 'https://localhost:8443/gateway/admin/api/v1/servicedefinitions'
HTTP/1.1 100 Continue

HTTP/1.1 500 Internal Server Error
Date: Tue, 19 Nov 2019 13:01:36 GMT
Set-Cookie: KNOXSESSIONID=node0i9vn469u6vjkinkf8wh9hrk418.node0; Path=/gateway/admin; Secure; HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/admin; Max-Age=0; Expires=Mon, 18-Nov-2019 13:01:36 GMT
Content-Type: application/json
Content-Length: 111
Server: Jetty(9.4.22.v20191022)

{ "CREATION_ERROR": "The requested service definition (mySampleService, MYSAMPLEROLE, 2.0.0) already exists!" }

How to update existing service definitions

In order to update the existing service definition(s), you need to use the same endpoint as you used above for service definition creation but you should issue a PUT request.

Sample (please note that I kept the previously added mySampleService, MYSAMPLEROLE, 2.0.0 for testing purposes):

$ curl -H "Content-Type: application/xml" -iku admin:*** -d "@testServiceDefinitionPayload.xml" -X PUT 'https://localhost:8443/gateway/admin/api/v1/servicedefinitions'
HTTP/1.1 100 Continue

HTTP/1.1 201 Created
Date: Tue, 19 Nov 2019 13:08:20 GMT
Set-Cookie: KNOXSESSIONID=node0h67f7dq41ydl8ci18t41vld319.node0; Path=/gateway/admin; Secure; HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/admin; Max-Age=0; Expires=Mon, 18-Nov-2019 13:08:20 GMT
Location: https://localhost:8443/gateway/admin/api/v1/servicedefinitions/mySampleService/MYSAMPLEROLE/2.0.0
Content-Length: 0
Server: Jetty(9.4.22.v20191022)

How to delete an existing service definition

The  https://{gateway-host}:{gateway-port}/{gateway-path}/admin/api/v1/servicedefinitions/{service}/{role}/{version} endpoint accepts DELETE requests to remove an existing service definition. In case the service definition you are trying to delete does not exist an HTTP response with 500 error code will be returned.

Sample:

$ $ curl -H "Content-Type: application/xml" -iku admin:*** -X DELETE 'https://localhost:8443/gateway/admin/api/v1/servicedefinitions/mysampleservice/mysamplerole/2.0.0'
HTTP/1.1 200 OK
Date: Tue, 19 Nov 2019 13:23:15 GMT
Set-Cookie: KNOXSESSIONID=node015yf04uunrpyjocl05uz4or6k22.node0; Path=/gateway/admin; Secure; HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/admin; Max-Age=0; Expires=Mon, 18-Nov-2019 13:23:15 GMT
Location: https://localhost:8443/gateway/admin/api/v1/servicedefinitions/mysampleservice/mysamplerole/2.0.0
Content-Length: 0
Server: Jetty(9.4.22.v20191022)

$ curl -H "Content-Type: application/xml" -iku admin:*** -X DELETE 'https://localhost:8443/gateway/admin/api/v1/servicedefinitions/mysampleservice/mysamplerole/2.0.0'
HTTP/1.1 500 Internal Server Error
Date: Tue, 19 Nov 2019 13:23:22 GMT
Set-Cookie: KNOXSESSIONID=node017x14a6arppex10ucm0ooygohp23.node0; Path=/gateway/admin; Secure; HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Set-Cookie: rememberMe=deleteMe; Path=/gateway/admin; Max-Age=0; Expires=Mon, 18-Nov-2019 13:23:22 GMT
Content-Type: application/json
Content-Length: 116
Server: Jetty(9.4.22.v20191022)

{ "DELETION_ERROR": "There is no service definition with the given attributes: mysampleservice,mysamplerole,2.0.0" }

Service Definition Management in the Admin UI

The Admin UI has also been modified to support the previously introduced service definition management features


  • No labels