Attendees: Matt, Martin, Peter, Michele Sciabarra, Dave G., Daisy, Gorkem, Christian, Vincent, Tyson, Olivier, Markus, Himvanth, Priti, Ruedigermaass, Sandeep, Chethan, Carlos, Dragos
Notes:
  • Martin Henke is moderating today
  • last meeting was 2018-01-31
Introductions of new attendees
  • Michele - lost connection during intro. agenda time
  • Ruedigermaass - new member of ICF team
  • Gorkem - Red Hat, dev. experience, IDE, tools, etc. contrib. to devtools repo. for a few week, traveling recently, but able to attend this call for first time.
  • Martin - new OW team member in Germany, along with others from IBM ICF team
Open comments on status/updates in a few areas:
  • Main/core OpenWhisk (Carlos reviewing PRs since last mtg. ~2 weeks)
    • https://github.com/apache/incubator-openwhisk/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Amerged
    • Markus:
      • from recent commits (view0
      • new LB in, will talk about in a few mins
      • Splunk backed log store from Tyson
      • PureConfig movement continues, all new PRs should do all new configs here 
        • goal get rid of whisk config
      • Triggers API change, not generating activations Rules that are disabled, dev list detailed changes
      • Some perf. improvements from different parties
    • Martin:
      • Log reduction is in, as a default
  • Release process (Vincent)
    • Vincent: 
      • update: (shares screen)
      • Created new docs “Release Mgmt. Tutorial” in incubator-openwhisk-release repo.
        • describes Manual and Automated modes (steps, with details)
      • All scripts that automate are in Bash and ready to “try out”
      • Has demos of publishing to release repos at Apache, more work needed, but works for a few repos. now
        • Mac and Ubuntu have been tested by Vincent so far
      • Tools folder setup to install (local) dependencies (for release manager to automate)
      •  config.json will list all repos and commit hashes the automation will use
        • along with candidate version #s
      • Run scripts that will…
        • download source code (using config.json list/hashes)
          • by default under HOME_DIR, but can configure param. to alter destination dir.
      • Run “Make” (source code only in this first stage of release)
        • So far build uses gradle on 2 repos, other repos, need work to automate others, but can get general idea from the 3 repos. working now
      • Sign release artifacts (using PGP keys) per Apache policy
        • this is most recent work this week
        • Generate key to sign artifacts and generate checksum (SHA512, MD5, ASC signature file)
        • Upload public key others can use to verify
      • Also, there is a “clean” script to clean up artifacts (if needed) if something goes wrong and for now for test purposes
      • Send out vote to community and follow Apache policies
      • If stage release approved other scripts can move tags artifacts to official release
      • Of course, this was a demo of manual steps; goal is to automate fully through Travis CI
        • triggered off a CI job, via PR submission to release repo.
        • The PR should have the config.json file with the correct repos/hash values
        • Push-based build will stage
        • Tag-based will publish once vote completes
    • Carlos: between steps 8 and 9 there would be 2 “votes” since we are Incubator
      • So our mentors, if part of IPMC, then we can reuse their votes for 2nd vote
      • We should also provide list of where/how to announce (e.g., apart from “Dev” list too)
    • Vincent: also provided a verify script to test checksums as well 
    • Carlos: this is good for rel. mgrs. to verify before they vote
    • Daisy:
      • Important to audit releases using Apache Licensing policy
      • Apache Creadur, release auditing toolset, RAT, Tentacles, whisker (supports different audit pieces
      • Today will talk about RAT header auditing (a java application that scans TAR of release)
      • Implemented Gradle plug-in if projects support gradle on 1 repo (wskdeploy) so far to test this
      • Exclusions are possible, diff. Apache projects 
    • Matt
      • RAT does not work with all types of file extensions and does not support ASF approved “mini” header, will have to discuss where we want exclusions, specifically test dirs and samples.
      • We made decisions last year to exclude “test” samples as JS files in samples/tests are not desirable, esp. for perf. tests
      • Could not release these files perhaps or treat separately
      • Shows CWIKI with table that tracks release automation progress, to be moved io release repo. as pseudo release “readiness board"
    • Markus:
      • LB changes: no fancy slides
        • PR merged today. Addresses many issues old LB  had all summarized in PR
        • avoids queueing that was occurring
        • address inconsistent “state” between multiple controllers
        • New approach is “sharding”, spec. opted for horizontal sharding (vs. vertical sharding)
          • explained diff. bw/ horz. and vert. sharding
        • Goal to make sure we do not have unused (invoker) resources and maintain use of warm containers
        • of course, horz. sharding is limited in scalability
        • eventually will have to move to vertical sharding to overcome limitations
        • used Akka cluster to have all LB be aware of each other, but only used to manage “slow data” (e.g., # of invokers avail. in system). 
        • Old LB had other bottlenecks, every invocation request we wre performing checks, new LB fixes these checks
        • New LB, the state is consistent even locally, slots to invokers impl. with semaphores (low level of concurrency); means if there is any avail. resources, we will use them
          • works under burst as well
        • We used Tyson’s SPI to test new implementation of LB (thanks Tyson), testing now to see if it is ready "for production”
        • Tyson had use cases on concurrent actions which should work with the invoker logic (as this change does not affect that)
        • Please let me know on dev list questions or issues; working on perf. #s we are seeing
    • Carlos: is it a feature toggle?
    • Markus: it is an SPI, load new class (default is still old class). Will have dev list discussion to adopt new LB if people want.
    • Carlos: If you have diagrams to post somewhere? 
    • Markus: will try/agree
  • CLI
    • Priti made changes to enable Annotations on Rules (was avail. server-side, but go-client did not support)
  • Kubernetes:
  • Mesos:
    • (via Dragos): Adobe is working hard to get their OW based platform ready for the Adobe Summit (MARCH 25–29, LAS VEGAS).
  • API Gateway
  • Catalog/Packages/Samples
    • None today
  • Tooling/Utilities
Topics / agenda items
Confirm moderator for next call
  • Vadim volunteers. for Feb. 28th meeting
  • No labels