July 7^th 2017 OpenWhisk on Kubernetes meeting notes:

Attendees: Dan Lavine, Mark Peek, Kavitha Devara, Matt Rutkowski

Notes:

• 9:10 - Start

  • DL: Starting, agenda is to go over what work has happened over the last 2 weeks since the previous meeting.

    • Generated a number of Github Issues from Proposals

    • Start to remove Ansible deployments from the configuration image bottom up. Start with postdeploy.yml and work towards setup.yml

• 9:12

  • DL: Talk about how Nginx is pull out of the configuration Image.

    • Point out documentation/discuss how Kube Secrets and ConfigMap now need to be generated with certs, etc to deploy Nginx

• 9:14

  • KD: (Question) How does Kubenretes get properties from ConfigMaps and start the Nginx Job?

  • DL: Walk though Kubernetes yaml file for Nginx and README.

    • Show from the README how Kube resouces are created

    • Look in the Nginx yaml for where those properties are imported at

    • Also general discussion on what ConfigMaps can be used for: environment variables, files, JSON blobs, etc.

• 9:22

  • DL: Start talking about changes for the Invoker.

    • There is a PR with all the changes to the Invoker with environment variable properties set rather than use Consul.

  • DL: Part of last week was spent setting up local environment so that he can develop on core OpenWhisk code and run tests.

    • Specifically changes to Invoker so you do not need to provide a Consul environment variable.

    • Took a couple days to setup environment to test all code. Lost on how to setup the environment so that you can actually run through tests.

  • DL: Note that after this work there is a PR which actually removes all of Consul from OpenWhisk.

    • Until that code is pulled in, Dan said he has a custom Invoker image which has the same changes as the Pending PR. Just does not need to supply a Consul environment variable. Once the PR is merged and a new OpenWhisk Invoker Docker Image is generated, we will switch to using that image. For now we can use Dan's image to get over this as a blocker.

• 9:25

  • DL: There are many properties that need to be set on the Invoker.

    • Going to get annoying very quickly to manage all of the environment files by hand.

    • Idea suggested is to use Ansible to generate an “env_file” for Docker and convert that file into Kubernetes Secrets/ConfigMap.

    • Then when deploying OpenWhisk in todays form, you could use “docker run -env_file=...”

  • KD: (Question) What are env_files and how can we use them today?

  • DL:

    • Goes and takes a look at the current PR open for removing Consul.

    • Notes that there are currently a large number of new environment variables that need to be configured when deploying some components (looking at Invoker).

    • All of the environment variables in the deployment process can be built into a file(s) to use a multiple places

• 9:33

  • MP: Raises the concern that Dan would like to use Ansible to generate these env_files for Docker and Kubernetes. But One of the goals is to get rid of Ansible as part of the Deployment process.

    • Also raises the question if this is incremental or will Ansible also be a required as part of the deployment process.

  • DL: Agrees, that it does seem odd to want to use Ansible, but notes that it would be a place to start for a way of managing deployment process.

  • MP, DL: Agree that is isn't required at all, just a nice to have so a user does not need to manage all of the environment variables by hand. (Potential future stuff to think about)

• 9:36: Discussion topic is over, general quiestions

  • KD: (Question) Is there is a test suite for OpenWhisk rather than running simple “hello world”?

  • DL: Unsure if there is a real integration test suite for Openwhisk

    • Notes that the current tests in OpenWhisk repo already require a running OpenWhisk to fully run since they do some integration level testing.

  • DL: Considering writing some docs for running tests and developing for OpenWhisk

    • Unsure what he actually wants to do for this since a number of repos are hard to test for and cannot be run localy. Some repos require additional setup.

    • Testing might be a bigger issue

  • KD: (Question) So the basic test suite checks for services and endpoints?

  • DL: He thinks that the current OpenWhisk's “gradlew test” basically runs through those scenarios

• 9:40:

  • KD: (Question) She notices that there currently for Kubernetes there are a bunch of services that get started right away. Does this take the place of Consul on Kubernets or are we suppose to use Consul to discover these?

  • DL: No, it is kind of weird. Consul is only used as a key-value store. Currently because OpenWhisk is deployed via Ansible Components already Know the IPs or everything just talk via Kafka.

  • KD: (Question) So can KubeDNS just substitue for Consul or do they provide different functionalities:

  • DL: It could if DNS is ues, but it is a bit confusing.

    • Thinks Consul does round robin requests

    • Kube requires a CNI to be deployed and it is up to the CNI to actually manage DNS requests.

  • KD: (Question) Just wondering what it means to remove Consul and what it does for service discovery or something else too?

  • DL: No, it only is used as the key-value store, and only for configuration when a process first starts. Doesn't think it is used beyond that.

  • DL: The reason I have a bunch of services, was because I still relied on Ansible with the configuration image. Ansible would go and do a bunch of post deployment configurations and there needed to be some way of reaching those containers. Currently we are trying to build that into the Docker Images or processes themselves.

• 9:46

  • DL: I don't think there is a need for Invokers to have a service since nobody talkes to them directly.

  • MR: On that last statement there is the new thread about different pools for different types of requests. Long running requests that bypass Kafka directly.

  • DL: That is fine, we can bring back the service if need be. Currently everything gets its own unique “IP” address by getting a unique DNS entry assigned to it.

  • MR: The larger challenge is that there are different Invoker pools and they need to be split. Initially these are different pools and how do we predetermine these?

  • DL: That is OK. Can just copy the current yaml thats fine when deploying the Invoker, change the name and necessary environment variables to deploy them however you want.