Versions Compared

Old Version 9

changes.mady.by.user Meghna Baijal

Saved on

compared with

New Version Current

changes.mady.by.user Qing Lan

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Release Retrospective to correct Apache MXNet source licenses in post MXNet 1.0 release.References:

A. Problem Description 

Links to email threads and github issues that record the existing issues during various voting periods - 

B. Apache Licensing Policies

C. Open Pull Requests to fix these Issues

D. Open Issues/Questions/Doubts/Concerns (Help Requested!)

Question 1: What all can be excluded from RAT checks? Currently the following are being excluded from the RAT checks. Is this ok ?



Name of File/Folder Excluded from RAT CheckReason for IgnoringConcerns
1

These File Types:  *.css ; *.txt; \..* ; *.ipynb ; *.html ; *.json ; *.svg; *.names; *.csv



2

Submodules - 3rdparty/*


None
3
R-package
Not a part of MXNet ReleaseNone
4

Ignore all Dockerfiles - docker/*; Dockerfile* ; docker_multiarch/*

Dockerfiles cant have license
5 __init__.py files These files contain no text None
6theme.conf A header might affect website. Only one script here without a license -
docs/_static/mxnet-theme/theme.conf
7 This file - src/operator/nn/pool.hIt was decided that this file should not have an Apache License and it was removed here - PR 9170None. But can be verified again.
8This file - src/operator/special_functions-inl.hIt was decided that this file should not have an Apache License and it was removed here - PR 9170None. But can be verified again.
9example/rcnn/rcnn/cython/*This is licensed under MIT but RAT doesnt pick that. ASF should not be addedShould MIT license text be added explicitly?
10This Dataset - example/gluon/tree_lstm/dataset.cPickleThis is a datasetNone. But can be verified again.
11CODEOWNERStext file


Question 2: There are still 2 files with unknown licenses as per Apache RAT - SEE TABLE BELOW for this LIst

    1. This list assumes above excluded folders and formats are acceptable. 

      S.NoFile NameComments
      1. CODEOWNERSMoved to excludes
      2.appveyor.ymlAdded ASF header - Open PR
      3.readthedocs.ymlDoes this need an Apache License ?
      4.snap.pythonCan I add Apache License here?
      5.snapcraft.yamlCan I add Apache License here?
      6.python/mxnet/cython/base.pyiFile format --?
      7.tests/ci_build/pylintrcAdded ASF header - Open PR


Question 3: Issue 23 and Issue 37 in table below

This Folder needs to be reviewed - src/operator/contrib/ctc_include

 

E. More Details about the Issues & their Status

NoCategoryProblemSource FilesSuggested byLeadsComments
1.
Source 
tree
* Move the various git submodules into third-party/ or similar so it's
simpler to see what is Apache original source when we review a release.

submodules
Henri Yandell
 
Haibin Lin
 

2.

Copyright

*

Deal

with

the

Copyright

statements

in

perl-package

(copyright

holder

has

approved

doing

this)

perl-package

Henri Yandell

Henri
 
Yandell
 
Fixed by Henri in this PR -
https://github.com/apache/incubator-mxnet/pull/9191
3.

Formatting

*

Lots

of

whitespace

on

end

of

NOTICE

NOTICE

Henri

Yandell

Meghna Baijal
PR
in Review 
Merged
https://github.com/apache/incubator-mxnet/pull/9459
4.
Comments
* Comment added to CODEOWNERS to explain the file so we don't cause
community problems
CODEOWNERS
Henri Yandell
 
Steffen Rochel
5.
Licenses

LICENSE

*

There

was

a

suggestion

to

simplify

the

LICENSE

to

not

explicitly

list

which

packages

are

under

each

license.

Something

to

consider.

LICENSE

Henri

Yandell        

Yandell

Meghna Baijal

If this is done, it would resolve points 8-11 and 13-19 of this wiki
PR - https://github.com/apache/incubator-mxnet/pull/9484

LICENSE
* Update the Paths to license files once submodules are moved
LICENSE
Meghna Baijal
Haibin Lin

Automated Check
* Update the license_header.py script instead of manual exclusions
tools/license_header.py
Meghna BaijalMeghna Baijalmakefiles,









ISSUES IN SPECIFIC FILES
 




6.
Specific 
Files
* 
 
 Resolve 
 
 License 
 
 Header 
 
 - 
 
 if 
 
 you 
 
 follow 
 
 the 
 
 URL 
 
 mentioned 
 
 in 
 
 the 
 
 file 
 
 it 
 
 is 
 
 unclear 
 
if 
 
 the 
 
 code 
 
 came 
 
 form 
 
 that 
 
 site 
 
 or 
 
 was 
 
 written 
 
 for 
 
 the 
 
 project 
 
 by 
 
 the 
 
 author 
 
of 
 
 that 
 
 site.
example/image-classification/predict-cpp/image-classification-predict.cc

Justin McClean
  


7.
Specific 
Files
* 
 
 Resolve 
 
 License 
 
 Header 
 
 - 
 
 This 
 
 file 
 
 has 
 
 two 
 
 license 
 
 headers
tools/license_header.py
Justin McClean
Meghna Baijal
Possibly RAT got confused 
by the matching string. Does not need a fix.
8.
Specific 
Files
* 
 
 Check 
 
 LICENSE 
 
 - 
 
 Listed 
 
 as 
 
 Apache 
 
 License 
 
 in 
 
 the 
 
 LICENSE 
 
 but 
 
 is 
 
 it 
 
 MIT 
 
 license?
nnvm/tvm/HalideIR/LICENSE
  
Justin McClean
Meghna BaijalAdded to MIT license in LICENSE
 
9.
Specific 
Files
* 
 
 Check 
 
 LICENSE 
 
 - 
 
 Listed 
 
 as 
 
 Apache 
 
 License 
 
 in 
 
 the 
 
 LICENSE 
 
 but 
 
 is 
 
 it 
 
 MIT 
 
 license?example/rcnn/rcnn/cython/gpu_nms.pyx
  
Justin McClean
Meghna Baijal
Files licenses under different licenses, added to all in the top level LICENSE -
/incubator-mxnet/example/rcnn/LICENSE
 
10.
Specific 
Files
* 
 
 Check 
 
 LICENSE 
 
 - 
 
 Listed 
 
 as 
 
 Apache 
 
 License 
 
 in 
 
 the 
 
 LICENSE 
 
 but 
 
 is 
 
 it 
 
 MIT 
 
 license?example/rcnn/rcnn/cython/gpu_nms.pyx
Justin McClean
Meghna Baijal
Files licenses under different licenses, added to all in the top level LICENSE -
/incubator-mxnet/example/rcnn/LICENSE
   
11.
Specific 
Files
* 
 
 Check 
 
 LICENSE 
 
 - 
 
 Listed 
 
 as 
 
 Apache 
 
 License 
 
 in 
 
 the 
 
 LICENSE 
 
 but 
 
 is 
 
 it 
 
 MIT 
 
 license?example/rcnn/rcnn/cython/cpu_nms.pyx
Justin McClean
Meghna Baijal
Files licenses under different licenses, added to all in the top level LICENSE -
/incubator-mxnet/example/rcnn/LICENSE
   
12.
Specific 
Files
* 
 
 Missing 
 
 License 
 
 Header 
 
 - 
 
 is 
 
 it 
 
 Apache 
 
 License?
perl-package/AI-NNVMCAPI/Makefile.PL
   
Justin McClean


13.
Specific 
Files
* 
 
 Add 
 
 to 
 
 LICENSE 
 
 - 
 
 BSD 
 
 license
src/operator/contrib/ctc_include/contrib/moderngpu/include/mgpudevice.cuh
  
Justin McClean

Wrongly Named in the LICENSE file as Warp-CTC, corrected
 
14.
Specific 
Files
* 
 
 Add 
 
 to 
 
 LICENSE 
 
 - 
 
 A 
 
 dozen 
 
 files 
 
 here 
 
 are 
 
 BSD
src/operator/contrib/ctc_include/contrib/moderngpu/include/device/
  
Justin McClean

Wrongly Named in the LICENSE file as Warp-CTC, corrected
 
15.
Specific 
Files
* 
 
 Add 
 
 to 
 
 LICENSE 
 
 - 
 
 BSD 
 
 license
nnvm/tvm/dmlc-core/cmake/Modules/FindCrypto.cmake
  
Justin McClean
Meghna BaijalRepeat again for the same file??
 
16.
Specific 
Files
* 
 
 Add 
 
 to 
 
 LICENSE 
 
 - 
 
 BSD 
 
 license
example/ssd/dataset/pycocotools/coco.py
  
Justin McClean

Already there in LICENSE file as MS coco
 
17.
Specific 
Files
* 
 
 Add 
 
 to 
 
 LICENSE 
 
 - 
 
 6 
 
 files 
 
 are 
 
 BSD 
 
 licensed
example/rcnn/rcnn/pycocotools
   
Justin McClean

Already there in LICENSE 
18.
Specific 
Files
* 
 
 Add 
 
 to 
 
 LICENSE 
 
 - 
 
 BSD 
 
 license
dmlc-core/cmake/Modules/FindCrypto.cmake
   
Justin McClean

Added to LICENSE as BSD
19.
Specific 
Files
* 
 
 Add 
 
 to 
 
 LICENSE 
 
 - 
 
 BSD 
 
 license
cub/experimental/spmv_compare.cu
  
Justin McClean

CUB exists as BSD
 
20.
Specific 
 
Files
* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
 
 BSD
prepare_mkl.sh
   
Justin McClean

Does not say BSD, add?
21.
Specific 
 
Files
* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
 
 BSD
src/operator/nn/im2col.h
src/operator/nn/im2col.
c
cuh
  
Justin McClean

Created PR, in review 
https://github.com/apache/incubator-mxnet/pull/9462
 
22.
Specific 
 
Files
* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
 
 BSD
src/operator/contrib/nn/deformable_im2col.h
src/operator/contrib/nn/deformable_im2col.
c
cuh
  
Justin McClean

First was fixed for 1.0.0 here - https://github.com/apache/incubator-mxnet/pull/9170
Second in PR review - https://github.com/apache/incubator-mxnet/pull/9462
 
 23.
Specific 
 
 Files* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
 
 BSD
src/operator/contrib/ctc_include/contrib/moderngpu/include/mgpuenums.h
  
Justin McClean

License seems to be correct. No ASF header.
 
24. 
Specific 
 
Files
* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
 
 BSD
example/ssd/dataset/pycocotools/coco.py
  
Justin McClean

Removed ASF header in PR
ADD BSD License text??
Yes, looks like BSD license
25. 
Specific 
 
Files
* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
 
 MIT
example/rcnn/rcnn/cython/setup.py
  
Justin McClean

Removed ASF Header in PR
Add MIT license Text?
Yes, looks like MIT license
26.
Specific 
 
Files
* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
 
 MIT
example/rcnn/rcnn/cython/nms_kernel.cu
  
Justin McClean

Removed ASF Header in PR
Add MIT license Text?
 Yes, looks like MIT license
27.
Specific 
 
Files
* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
 
 MIT
example/rcnn/rcnn/cython/setup.py
   Yes, looks like MIT license                                                                                                         
Justin McClean

 Repeat

Specific Files
Resolve License: should this file get an apache license? (RAT thinks so but why does the script skip it?)
src/operator/special_functions-inl.h












APACHE RAT CHECK FAILURES




28.
RAT Failure
* Fix Submodules - RAT detected almost 2000 files with unknown licenses in submodules
Submodules (nnvm, dlpack, 3rdparty, ps-lite, mshadow) and R-package


Decision needed on how to handle submodule licenses
29.
RAT Failure
* Check Docs - RAT detected almost 200 files with unknown licenses in the Docs directory
/docs


Excluded for now
30.
RAT Failure
* Fix dockerfiles without license headers
/docker

Meghna Baijal
Added a top level License.md to this folder
31.
RAT Failure
* Fix docker_multiarch - unknown license header
/docker_multiarch

Meghna BaijalAdded a top level License.md to this folder
37.
RAT Failure
* Fix ctc_include
src/operator/contrib/ctc_include







































F. Static Linked Dependencies in Release
This section lists all dependencies used in the releases of pip and maven. Frank Liu pointed out some of them are using BSD3, GPL and customized License which should be placed in the official releases.
Currently, the licenses were covered when we created the static-linked so file which can be found in this PR.

current stable versionbinary release for ubuntu 16.04mxnet release 1.4Build dependenciesLicenseLicense Name
OpenBLAS0.2.200.2.180.3.2
https://github.com/xianyi/OpenBLAS/blob/develop/LICENSEBSD3
OpenCV4.0.02.4.9.13.4.2eigen, libpng, libjpg-turbo, libtiff, zlibhttps://github.com/opencv/opencv/blob/master/LICENSEBSD3
zlib1.2.111.2.81.2.6
https://github.com/LuaDist/zlib/blob/master/README
libjpeg-turbo2.0.11.5.3-621.5.90 (2.0 beta 1)nasmhttps://github.com/libjpeg-turbo/libjpeg-turbo/blob/master/LICENSE.md
libpng1.6.361.2.541.6.34zlibhttps://github.com/glennrp/libpng/blob/libpng16/LICENSE
libtiffv4.0.104-0-64-0-9
https://gitlab.com/libtiff/libtiff/blob/master/COPYRIGHT
libssl-dev1.1.1a1.0.2g1.0.2l
https://github.com/openssl/openssl/blob/master/LICENSEApache2
libcurl7.62.07.47.07.61.0libtool, automakehttps://curl.haxx.se/docs/copyright.html
eigen3.3.53.2.923.3.4
https://github.com/eigenteam/eigen-git-mirror/blob/master/COPYING.BSDBSD
protobuf3.6.12.6.13.5.1
https://github.com/protocolbuffers/protobuf/blob/master/LICENSE
cityhash1.1.1
1.1.1
https://github.com/google/cityhash/blob/master/COPYING
zmq4.2.54.1.44.2.2
https://github.com/zeromq/libzmq/blob/master/COPYINGGPL
lz4v1.8.3r131r130
https://github.com/lz4/lz4/blob/dev/LICENSE

OpenCV depends on following libraries, and can build them on the fly:
  • OpenBLAS
  • zlib (ver 1.2.6)
  • libjpeg-turbo (ver 1.5.90)
  • libpng (ver 1.6.34)
  • libtiff (ver 42 - 4.0.9)
  • eigen (ver 3.3.4)

ps-lite dependencies (USE_DIST_KVSTORE):
  • zeromq - 4.1.4
  • protobuf 2.5.0
  • lz4
  • cityhash
USE_S3
  • libcurl
  • libssl-dev