Versions Compared

Old Version 10

changes.mady.by.user Meghna Baijal

Saved on

compared with

New Version Current

changes.mady.by.user Qing Lan

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Release Retrospective to correct Apache MXNet source licenses in post MXNet 1.0 release.References:

A. Problem Description 

Links to email threads and github issues that record the existing issues during various voting periods - 

B. Apache Licensing Policies

C. Open Pull Requests to fix these Issues

D. Open Issues/Questions/Doubts/Concerns (Help Requested!)

Question 1: What all can be excluded from RAT checks? Currently the following are being excluded from the RAT checks. Is this ok ?



Name of File/Folder Excluded from RAT CheckReason for IgnoringConcerns
1

These File Types:  *.css ; *.txt; \..* ; *.ipynb ; *.html ; *.json ; *.svg; *.names; *.csv



2

Submodules - 3rdparty/*


None
3
R-package
Not a part of MXNet ReleaseNone
4

Ignore all Dockerfiles - docker/*; Dockerfile* ; docker_multiarch/*

Dockerfiles cant have license
5 __init__.py files These files contain no text None
6theme.conf A header might affect website. Only one script here without a license -
docs/_static/mxnet-theme/theme.conf
7 This file - src/operator/nn/pool.hIt was decided that this file should not have an Apache License and it was removed here - PR 9170None. But can be verified again.
8This file - src/operator/special_functions-inl.hIt was decided that this file should not have an Apache License and it was removed here - PR 9170None. But can be verified again.
9example/rcnn/rcnn/cython/*This is licensed under MIT but RAT doesnt pick that. ASF should not be addedShould MIT license text be added explicitly?
10This Dataset - example/gluon/tree_lstm/dataset.cPickleThis is a datasetNone. But can be verified again.
11CODEOWNERStext file


Question 2: There are still 2 files with unknown licenses as per Apache RAT - SEE TABLE BELOW for this LIst

    1. This list assumes above excluded folders and formats are acceptable. 

      S.NoFile NameComments
      1. CODEOWNERSMoved to excludes
      2.appveyor.ymlAdded ASF header - Open PR
      3.readthedocs.ymlDoes this need an Apache License ?
      4.snap.pythonCan I add Apache License here?
      5.snapcraft.yamlCan I add Apache License here?
      6.python/mxnet/cython/base.pyiFile format --?
      7.tests/ci_build/pylintrcAdded ASF header - Open PR


Question 3: Issue 23 and Issue 37 in table below

This Folder needs to be reviewed - src/operator/contrib/ctc_include

 

E. More Details about the Issues & their Status

NoCategoryProblemSource FilesSuggested byLeadsComments
1.
Source 
tree
* Move the various git submodules into third-party/ or similar so it's
simpler to see what is Apache original source when we review a release.

submodules
Henri Yandell
 
Haibin Lin
 

2.

Copyright

*

Deal

with

the

Copyright

statements

in

perl-package

(copyright

holder

has

approved

doing

this)

perl-package

Henri

 

Yandell

Henri Yandell
 
Fixed by Henri in this PR -
https://github.com/apache/incubator-mxnet/pull/9191
3.

Formatting

*

Lots

of

whitespace

on

end

of

NOTICE

NOTICE

Henri

Yandell

Meghna Baijal
PR
in Review 
Merged
https://github.com/apache/incubator-mxnet/pull/9459
4.
Comments
* Comment added to CODEOWNERS to explain the file so we don't cause
community problems
CODEOWNERS
Henri Yandell
 
Steffen Rochel
5.

LICENSE

*

There

was

a

suggestion

to

simplify

the

LICENSE

to

not

explicitly

list

which

packages

are

under

each

license.

Something

to

consider.

LICENSE

Henri

Yandell

  

Meghna Baijal

If this is done, it would resolve points 8-11 and 13-19 of this wiki
PR - https://github.com/apache/incubator-mxnet/pull/9484
          ISSUES IN SPECIFIC FILES     

LICENSE
* Update the Paths to license files once submodules are moved
LICENSE
Meghna Baijal
  
Haibin Lin

Automated Check
* Update the license_header.py script instead of manual exclusions
tools/license_header.py
Meghna BaijalMeghna Baijalmakefiles,









ISSUES IN SPECIFIC FILES 




6.
Specific 
Files
* 
 
 Resolve 
 
 License 
 
 Header 
 
 - 
 
 if 
 
 you 
 
 follow 
 
 the 
 
 URL 
 
 mentioned 
 
 in 
 
 the 
 
 file 
 
 it 
 
 is 
 
 unclear 
 
if 
 
 the 
 
 code 
 
 came 
 
 form 
 
 that 
 
 site 
 
 or 
 
 was 
 
 written 
 
 for 
 
 the 
 
 project 
 
 by 
 
 the 
 
 author 
 
of 
 
 that 
 
 site.
example/image-classification/predict-cpp/image-classification-predict.cc

Justin McClean
 
 


7.
Specific 
Files
* 
 
 Resolve 
 
 License 
 
 Header 
 
 - 
 
 This 
 
 file 
 
 has 
 
 two 
 
 license 
 
 headers
tools/license_header.py
Justin McClean
Meghna Baijal
Possibly RAT got confused 
by the matching string. Does not need a fix.
8.
Specific 
Files
* 
 
 Check 
 
 LICENSE 
 
 - 
 
 Listed 
 
 as 
 
 Apache 
 
 License 
 
 in 
 
 the 
 
 LICENSE 
 
 but 
 
 is 
 
 it 
 
 MIT 
 
 license?
nnvm/tvm/HalideIR/LICENSE
Justin McClean
 
Meghna BaijalAdded to MIT license in LICENSE
 
9.
Specific 
Files
* 
 
 Check 
 
 LICENSE 
 
 - 
 
 Listed 
 
 as 
 
 Apache 
 
 License 
 
 in 
 
 the 
 
 LICENSE 
 
 but 
 
 is 
 
 it 
 
 MIT 
 
 license?example/rcnn/rcnn/cython/gpu_nms.pyx
Justin McClean
 
Meghna Baijal
Files licenses under different licenses, added to all in the top level LICENSE -
/incubator-mxnet/example/rcnn/LICENSE
 
10.
Specific 
Files
* 
 
 Check 
 
 LICENSE 
 
 - 
 
 Listed 
 
 as 
 
 Apache 
 
 License 
 
 in 
 
 the 
 
 LICENSE 
 
 but 
 
 is 
 
 it 
 
 MIT 
 
 license?example/rcnn/rcnn/cython/gpu_nms.pyx
Justin McClean
 
Meghna Baijal
Files licenses under different licenses, added to all in the top level LICENSE -
/incubator-mxnet/example/rcnn/LICENSE
 
11.
Specific 
Files
* 
 
 Check 
 
 LICENSE 
 
 - 
 
 Listed 
 
 as 
 
 Apache 
 
 License 
 
 in 
 
 the 
 
 LICENSE 
 
 but 
 
 is 
 
 it 
 
 MIT 
 
 license?example/rcnn/rcnn/cython/cpu_nms.pyx
Justin McClean
 
Meghna Baijal
Files licenses under different licenses, added to all in the top level LICENSE -
/incubator-mxnet/example/rcnn/LICENSE
 
12.
Specific 
Files
* 
 
 Missing 
 
 License 
 
 Header 
 
 - 
 
 is 
 
 it 
 
 Apache 
 
 License?
perl-package/AI-NNVMCAPI/Makefile.PL
Justin 
 
 McClean
  


13.
Specific 
Files
* 
 
 Add 
 
 to 
 
 LICENSE 
 
 - 
 
 BSD 
 
 license
src/operator/contrib/ctc_include/contrib/moderngpu/include/mgpudevice.cuh
Justin McClean
  

Wrongly Named in the LICENSE file as Warp-CTC, corrected
14.
Specific 
Files
* 
 
 Add 
 
 to 
 
 LICENSE 
 
 - 
 
 A 
 
 dozen 
 
 files 
 
 here 
 
 are 
  
 BSD
src/operator/contrib/ctc_include/contrib/moderngpu/include/device/
Justin McClean
 

Wrongly Named in the LICENSE file as Warp-CTC, corrected
15.
Specific 
Files
* 
 
 Add 
 
 to 
 
 LICENSE 
 
 - 
 
 BSD 
 
 license
nnvm/tvm/dmlc-core/cmake/Modules/FindCrypto.cmake
Justin McClean
 
Meghna BaijalRepeat again for the same file??
 
16.
Specific 
Files
* 
 
 Add 
 
 to 
 
 LICENSE 
 
 - 
 
 BSD 
  
 license
example/ssd/dataset/pycocotools/coco.py
Justin McClean
 

Already there in LICENSE file as MS coco
17.
Specific 
Files
* 
 
 Add 
 
 to 
 
 LICENSE 
 
 - 
 
 6 
 
 files 
 
 are 
 
 BSD 
  
 licensed
example/rcnn/rcnn/pycocotools
Justin McClean
 

Already there in LICENSE 
18.
Specific 
Files
* 
 
 Add 
 
 to 
 
 LICENSE 
 
 - 
 
 BSD 
  
 license
dmlc-core/cmake/Modules/FindCrypto.cmake
Justin McClean
 

Added to LICENSE as BSD
19.
Specific 
Files
* 
 
 Add 
 
 to 
 
 LICENSE 
 
 - 
 
 BSD 
 Cub
 license
cub/experimental/spmv_compare.cu
Justin McClean
 

CUB 
is added to LICENSE 
but is that a BSD license?
exists as BSD
20.
Specific 
 
Files
* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
 
 BSD
prepare_mkl.sh
Justin McClean
  

Does not say BSD, add?
21.
Specific 
 
Files
* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
 
 BSD
src/operator/nn/im2col.h
src/operator/nn/im2col.
c 
cuh
Justin McClean
 

Created PR, in review 
https://github.com/apache/incubator-mxnet/pull/9462
22.
Specific 
 
Files
* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
 
 BSD
src/operator/contrib/nn/deformable_im2col.h
src/operator/contrib/nn/deformable_im2col.
c 
cuh
Justin McClean
 

First was fixed for 1.0.0 here - https://github.com/apache/incubator-mxnet/pull/9170
Second in PR review - https://github.com/apache/incubator-mxnet/pull/9462
 23.
Specific 
 
 Files* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
 
 BSD
src/operator/contrib/ctc_include/contrib/moderngpu/include/mgpuenums.h
Justin McClean
  

License seems to be correct. No ASF header.
24. 
Specific 
 
Files
* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
 Yes, looks like BSD license
 BSD
example/ssd/dataset/pycocotools/coco.py
Justin McClean
 

Removed ASF header in PR
ADD BSD License text??
25. 
Specific 
 
Files
* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
 Yes, looks like 
 MIT
example/rcnn/rcnn/cython/setup.py
Justin McClean
 

Removed ASF Header in PR
Add MIT license Text?
26.
Specific 
 
Files
* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
  Yes, looks like MIT license
 MIT
example/rcnn/rcnn/cython/nms_kernel.cu
Justin McClean
 

Removed ASF Header in PR
Add MIT license Text?
27.
Specific 
 
Files
* 
 
 Incorrect 
 
 License 
 
 Header 
 
 - 
 
 Has 
 
 ASF 
 
 header 
 
 but 
 
 is 
 
 it 
          APACHE RAT CHECK FAILURES    28.
RAT Failure
* Fix Submodules - RAT detected almost 2000 files with unknown licenses in submodules
Submodules (nnvm, dlpack, 3rdparty, ps-lite, mshadow) and R-package
  
 MIT
example/rcnn/rcnn/cython/setup.py
Justin McClean
  Yes, looks like MIT license

 Repeat

Specific Files
Resolve License: should this file get an apache license? (RAT thinks so but why does the script skip it?)
src/operator/special_functions-inl.h












APACHE RAT CHECK FAILURES




28.
RAT Failure
* Fix Submodules - RAT detected almost 2000 files with unknown licenses in submodules
Submodules (nnvm, dlpack, 3rdparty, ps-lite, mshadow) and R-package


 
Decision needed on how to handle submodule licenses
29.
RAT Failure
* Check Docs - RAT detected almost 200 files with unknown licenses in the Docs directory
/docs
  


Excluded for now
30.
RAT 
 
 Failure
 
* 
 
 Fix 
 
 dockerfiles 
 
 without 
 
 license 
 
 headers
/docker
 

Meghna
 Baijal
Added a top level License.md to this folder
 
31.
RAT 
 
 Failure
* 
 
 Fix 
 
 docker_multiarch 
 
 - 
 
 unknown 
 
 license 
 
 header
/docker_multiarch
   

Meghna BaijalAdded a top level License.md to this folder
37
32
.
RAT Failure
* Fix 
scala-package - unknown license
/scala-package
   33.
RAT Failure
* Fix tools
/tools
   34.
RAT Failure
* Fix tests
/tests   35.
RAT Failure
* Fix examples
/examples   36.
RAT Failure
* Check file types that can be ignored - .txt; .js, .md, .json, .yml, *.*, 
/    
RAT Failure
      
RAT Failure
 
     
RAT Failure
 
     
RAT Failure
 
    
ctc_include
src/operator/contrib/ctc_include







































F. Static Linked Dependencies in Release
This section lists all dependencies used in the releases of pip and maven. Frank Liu pointed out some of them are using BSD3, GPL and customized License which should be placed in the official releases.
Currently, the licenses were covered when we created the static-linked so file which can be found in this PR.

current stable versionbinary release for ubuntu 16.04mxnet release 1.4Build dependenciesLicenseLicense Name
OpenBLAS0.2.200.2.180.3.2
https://github.com/xianyi/OpenBLAS/blob/develop/LICENSEBSD3
OpenCV4.0.02.4.9.13.4.2eigen, libpng, libjpg-turbo, libtiff, zlibhttps://github.com/opencv/opencv/blob/master/LICENSEBSD3
zlib1.2.111.2.81.2.6
https://github.com/LuaDist/zlib/blob/master/README
libjpeg-turbo2.0.11.5.3-621.5.90 (2.0 beta 1)nasmhttps://github.com/libjpeg-turbo/libjpeg-turbo/blob/master/LICENSE.md
libpng1.6.361.2.541.6.34zlibhttps://github.com/glennrp/libpng/blob/libpng16/LICENSE
libtiffv4.0.104-0-64-0-9
https://gitlab.com/libtiff/libtiff/blob/master/COPYRIGHT
libssl-dev1.1.1a1.0.2g1.0.2l
https://github.com/openssl/openssl/blob/master/LICENSEApache2
libcurl7.62.07.47.07.61.0libtool, automakehttps://curl.haxx.se/docs/copyright.html
eigen3.3.53.2.923.3.4
https://github.com/eigenteam/eigen-git-mirror/blob/master/COPYING.BSDBSD
protobuf3.6.12.6.13.5.1
https://github.com/protocolbuffers/protobuf/blob/master/LICENSE
cityhash1.1.1
1.1.1
https://github.com/google/cityhash/blob/master/COPYING
zmq4.2.54.1.44.2.2
https://github.com/zeromq/libzmq/blob/master/COPYINGGPL
lz4v1.8.3r131r130
https://github.com/lz4/lz4/blob/dev/LICENSE

OpenCV depends on following libraries, and can build them on the fly:
  • OpenBLAS
  • zlib (ver 1.2.6)
  • libjpeg-turbo (ver 1.5.90)
  • libpng (ver 1.6.34)
  • libtiff (ver 42 - 4.0.9)
  • eigen (ver 3.3.4)

ps-lite dependencies (USE_DIST_KVSTORE):
  • zeromq - 4.1.4
  • protobuf 2.5.0
  • lz4
  • cityhash
USE_S3
  • libcurl
  • libssl-dev