Versions Compared

Old Version 7

changes.mady.by.user Meghna Baijal

Saved on

compared with

New Version Current

changes.mady.by.user Qing Lan

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Release Retrospective to correct Apache MXNet source licenses in post MXNet 1.0 release.References:

A. Problem Description 

Links to email threads and github issues that record the existing issues during various voting periods - 

B. Apache Licensing Policies

C. Open Pull Requests to fix these Issues

D. Open Issues/Questions/Doubts/Concerns (Help Requested!)

Question 1: What all can be excluded from RAT checks? Currently the following are being excluded from the RAT checks. Is this ok ?



Name of File/Folder Excluded from RAT CheckReason for IgnoringConcerns
1

These File Types:  *.css ; *.txt; \..* ; *.ipynb ; *.html ; *.json ; *.svg; *.names; *.csv



2

Submodules - 3rdparty/*


None
3
R-package
Not a part of MXNet ReleaseNone
4

Ignore all Dockerfiles - docker/*; Dockerfile* ; docker_multiarch/*

Dockerfiles cant have license
5 __init__.py files These files contain no text None
6theme.conf A header might affect website. Only one script here without a license -
docs/_static/mxnet-theme/theme.conf
7 This file - src/operator/nn/pool.hIt was decided that this file should not have an Apache License and it was removed here - PR 9170None. But can be verified again.
8This file - src/operator/special_functions-inl.hIt was decided that this file should not have an Apache License and it was removed here - PR 9170None. But can be verified again.
9example/rcnn/rcnn/cython/*This is licensed under MIT but RAT doesnt pick that. ASF should not be addedShould MIT license text be added explicitly?
10This Dataset - example/gluon/tree_lstm/dataset.cPickleThis is a datasetNone. But can be verified again.
11CODEOWNERStext file


Question 2: There are still 2 files with unknown licenses as per Apache RAT - SEE TABLE BELOW for this LIst

    1. This list assumes above excluded folders and formats are acceptable. 

      S.NoFile NameComments
      1. CODEOWNERSMoved to excludes
      2.appveyor.ymlAdded ASF header - Open PR
      3.readthedocs.ymlDoes this need an Apache License ?
      4.snap.pythonCan I add Apache License here?
      5.snapcraft.yamlCan I add Apache License here?
      6.python/mxnet/cython/base.pyiFile format --?
      7.tests/ci_build/pylintrcAdded ASF header - Open PR


Question 3: Issue 23 and Issue 37 in table below

This Folder needs to be reviewed - src/operator/contrib/ctc_include

 

E. More Details about the Issues & their Status

NoCategoryProblemSource FilesSuggested byLeadsComments
1.
Source 
tree
* Move the various git submodules into third-party/ or similar so it's
simpler to see what is Apache original source when we review a release.

submodules
Henri Yandell
  
Haibin Lin

2.

Copyright

*

Deal

with

the

Copyright

statements

in

perl-package

(copyright

holder

has

approved

doing

 

this)

perl-package

Henri Yandell

Henri Yandell
 
Fixed by Henri in this PR -
https://github.com/apache/incubator-mxnet/pull/9191
3.

Formatting

*

Lots

of

whitespace

on

end

of

NOTICE

NOTICE

Henri Yandell

Meghna
PR in Review 
 Baijal
PR Merged
https://github.com/apache/incubator-mxnet/pull/9459
4.
Comments
* Comment added to CODEOWNERS to explain the file so we don't cause
community problems
CODEOWNERS
Henri Yandell
 
Steffen Rochel
5.
Licenses

LICENSE

*

There

was

a

suggestion

to

simplify

the

LICENSE

to

not

explicitly

list

which

packages

are

under

each

license.

Something

to

consider.

LICENSE

Henri Yandell

        

Meghna Baijal

If this is done, it would resolve points 8-11 and 13-19 of this wiki
PR - https://github.com/apache/incubator-mxnet/pull/9484

LICENSE
* Update the Paths to license files once submodules are moved
LICENSE
Meghna Baijal
Haibin Lin

Automated Check
* Update the license_header.py script instead of manual exclusions
tools/license_header.py
Meghna BaijalMeghna Baijalmakefiles,









ISSUES IN SPECIFIC FILES
 




6.
Specific 
Files
* 
 
 Resolve 
 
 License 
 
Header - 
 
 if 
 
 you 
 
 follow 
 
 the 
 
 URL 
 
 mentioned 
 
 in 
 
 the 
 
 file 
 
 it 
 
 is 
 
 unclear 
 
if 
 
 the 
 
 code 
 
 came 
 
 form 
 
 that 
 
 site 
 
 or 
 
 was 
 
 written 
 
 for 
 
 the 
 
 project 
 
 by 
 
 the 
 
 author 
 
of 
 
 that 
 
 site.
example/image-classification/predict-cpp/image-classification-predict.cc

Justin McClean
  


7.
Specific 
Files
* 
 
 Resolve 
 
 License 
 
Header - 
 
 This 
 
 file 
 
 has 
 
 two 
 
 license 
 
 headers
tools/license_header.py
Justin McClean
Meghna Baijal
Possibly RAT got confused 
by the matching string. Does not need a fix.
8.
Specific 
Files
* 
 
 Check 
 License 
LICENSE - 
 
 Listed 
 
 as 
 
 Apache 
 
 License 
 
 in 
 
 the 
 
 LICENSE 
 
 but 
 
 is 
 
 it 
 
 MIT 
 
 license?
nnvm/tvm/HalideIR/LICENSE
  
Justin McClean
Meghna BaijalAdded to MIT license in LICENSE
 
9.
Specific 
Files
* 
 
 Check 
 License 
LICENSE - 
 
 Listed 
 
 as 
 
 Apache 
 
 License 
 
 in 
 
 the 
 
 LICENSE 
 
 but 
 
 is 
 
 it 
 
 MIT 
 
 license?example/rcnn/rcnn/cython/gpu_nms.pyx
  
Justin McClean
Meghna Baijal
Files licenses under different licenses, added to all in the top level LICENSE -
/incubator-mxnet/example/rcnn/LICENSE
 
10.
Specific 
Files
* 
 
 Check 
 License - Listed as Apache License in the LICENSE but is it MIT 
LICENSE - Listed as Apache License in the LICENSE but is it MIT license?example/rcnn/rcnn/cython/gpu_nms.pyx
Justin McClean
Meghna Baijal
Files licenses under different licenses, added to all in the top level LICENSE -
/incubator-mxnet/example/rcnn/LICENSE
11.
Specific 
Files
* Check LICENSE - Listed as Apache License in the LICENSE but is it MIT license?example/rcnn/rcnn/cython/
gpu
cpu_nms.pyx
Justin McClean
Meghna Baijal
Files licenses under different licenses, added to all in the top level LICENSE -
/incubator-mxnet/example/rcnn/LICENSE
12.
Specific 
Files
* Missing License Header - is it Apache License?
perl-package/AI-NNVMCAPI/Makefile.PL
Justin McClean


13.
Specific 
Files
* Add to LICENSE - BSD license
src/operator/contrib/ctc_include/contrib/moderngpu/include/mgpudevice.cuh
Justin McClean

Wrongly Named in the LICENSE file as Warp-CTC, corrected
14.
Specific 
Files
* Add to LICENSE - A dozen files here are BSD
src/operator/contrib/ctc_include/contrib/moderngpu/include/device/
Justin McClean

Wrongly Named in the LICENSE file as Warp-CTC, corrected
15.
Specific 
Files
* Add to LICENSE - BSD license
nnvm/tvm/dmlc-core/cmake/Modules/FindCrypto.cmake
Justin McClean
Meghna BaijalRepeat again for the same file??
16.
Specific 
Files
* Add to LICENSE - BSD license
example/ssd/dataset/pycocotools/coco.py
Justin McClean

Already there in LICENSE file as MS coco
17.
Specific 
Files
* Add to LICENSE - 6 files are BSD licensed
example/rcnn/rcnn/pycocotools
Justin McClean

Already there in LICENSE 
18.
Specific 
Files
* Add to LICENSE - BSD license
dmlc-core/cmake/Modules/FindCrypto.cmake
Justin McClean

Added to LICENSE as BSD
19
   11
.
Specific 
Files
* 
 Check License - Listed as Apache License in the LICENSE but is it MIT license?example/rcnn/rcnn/cython/cpu_nms.pyx   12.                                         
Add to LICENSE - BSD license
cub/experimental/spmv_compare.cu
Justin McClean

CUB exists as BSD
20.
Specific 
Files
* Incorrect License Header - Has ASF header but is it BSD
prepare_mkl.sh
Justin McClean

Does not say BSD, add?
21.
Specific 
Files
* Incorrect License Header - Has ASF header but is it BSD
src/operator/nn/im2col.h
src/operator/nn/im2col.cuh
Justin McClean

Created PR, in review 
https://github.com/apache/incubator-mxnet/pull/9462
22.
Specific 
Files
* Incorrect License Header - Has ASF header but is it BSD
src/operator/contrib/nn/deformable_im2col.h
src/operator/contrib/nn/deformable_im2col.cuh
Justin McClean

First was fixed for 1.0.0 here - https://github.com/apache/incubator-mxnet/pull/9170
Second in PR review - https://github.com/apache/incubator-mxnet/pull/9462
 23.
Specific Files* Incorrect License Header - Has ASF header but is it BSD
src/operator/contrib/ctc_include/contrib/moderngpu/include/mgpuenums.h
Justin McClean

License seems to be correct. No ASF header.
24. 
Specific 
Files
* Incorrect License Header - Has ASF header but is it BSD
example/ssd/dataset/pycocotools/coco.py
Justin McClean

Removed ASF header in PR
ADD BSD License text??
25. 
Specific 
Files
* Incorrect License Header - Has ASF header but is it MIT
example/rcnn/rcnn/cython/setup.py
Justin McClean

Removed ASF Header in PR
Add MIT license Text?
26.
Specific 
Files
* Incorrect License Header - Has ASF header but is it MIT
example/rcnn/rcnn/cython/nms_kernel.cu
Justin McClean

Removed ASF Header in PR
Add MIT license Text?
27.
Specific 
Files
* Incorrect License Header - Has ASF header but is it MIT
example/rcnn/rcnn/cython/setup.py
Justin McClean

 Repeat

Specific Files
Resolve License: should this file get an apache license? (RAT thinks so but why does the script skip it?)
src/operator/special_functions-inl.h












APACHE RAT CHECK FAILURES




28.
RAT Failure
* Fix Submodules - RAT detected almost 2000 files with unknown licenses in submodules
Submodules (nnvm, dlpack, 3rdparty, ps-lite, mshadow) and R-package


Decision needed on how to handle submodule licenses
29.
RAT Failure
* Check Docs - RAT detected almost 200 files with unknown licenses in the Docs directory
/docs


Excluded for now
30.
RAT Failure
* Fix dockerfiles without license headers
/docker

Meghna Baijal
Added a top level License.md to this folder
31.
RAT Failure
* Fix docker_multiarch - unknown license header
/docker_multiarch

Meghna BaijalAdded a top level License.md to this folder
37.
RAT Failure
* Fix ctc_include
src/operator/contrib/ctc_include







































F. Static Linked Dependencies in Release
This section lists all dependencies used in the releases of pip and maven. Frank Liu pointed out some of them are using BSD3, GPL and customized License which should be placed in the official releases.
Currently, the licenses were covered when we created the static-linked so file which can be found in this PR.

current stable versionbinary release for ubuntu 16.04mxnet release 1.4Build dependenciesLicenseLicense Name
OpenBLAS0.2.200.2.180.3.2
https://github.com/xianyi/OpenBLAS/blob/develop/LICENSEBSD3
OpenCV4.0.02.4.9.13.4.2eigen, libpng, libjpg-turbo, libtiff, zlibhttps://github.com/opencv/opencv/blob/master/LICENSEBSD3
zlib1.2.111.2.81.2.6
https://github.com/LuaDist/zlib/blob/master/README
libjpeg-turbo2.0.11.5.3-621.5.90 (2.0 beta 1)nasmhttps://github.com/libjpeg-turbo/libjpeg-turbo/blob/master/LICENSE.md
libpng1.6.361.2.541.6.34zlibhttps://github.com/glennrp/libpng/blob/libpng16/LICENSE
libtiffv4.0.104-0-64-0-9
https://gitlab.com/libtiff/libtiff/blob/master/COPYRIGHT
libssl-dev1.1.1a1.0.2g1.0.2l
https://github.com/openssl/openssl/blob/master/LICENSEApache2
libcurl7.62.07.47.07.61.0libtool, automakehttps://curl.haxx.se/docs/copyright.html
eigen3.3.53.2.923.3.4
https://github.com/eigenteam/eigen-git-mirror/blob/master/COPYING.BSDBSD
protobuf3.6.12.6.13.5.1
https://github.com/protocolbuffers/protobuf/blob/master/LICENSE
cityhash1.1.1
1.1.1
https://github.com/google/cityhash/blob/master/COPYING
zmq4.2.54.1.44.2.2
https://github.com/zeromq/libzmq/blob/master/COPYINGGPL
lz4v1.8.3r131r130
https://github.com/lz4/lz4/blob/dev/LICENSE

OpenCV depends on following libraries, and can build them on the fly:
  • OpenBLAS
  • zlib (ver 1.2.6)
  • libjpeg-turbo (ver 1.5.90)
  • libpng (ver 1.6.34)
  • libtiff (ver 42 - 4.0.9)
  • eigen (ver 3.3.4)

ps-lite dependencies (USE_DIST_KVSTORE):
  • zeromq - 4.1.4
  • protobuf 2.5.0
  • lz4
  • cityhash
USE_S3
  • libcurl
  • libssl-dev