DUE TO SPAM, SIGN-UP IS DISABLED. Goto Selfserve wiki signup and request an account.
...
Attempt to run distributed training with FP16 lead to failure.
PR-8904: Fix multiprocessing too many open files
- Problem: The data-loader has a file-descriptor leak when the number of files being opened exceeds the ulimit of the user session.
- Workaround: Increase the ulimit of the user session or at the OS level using any of the following steps:
- $ ulimit -n unlimited (to make the fd limit as unlimited), OR
- $ ulimit -n 60000 (to make the fd limit as 60,000 files assuming you have around but less than 60,000 files to be opened) OR
- $ echo 65535 > /proc/sys/fs/file-max (to increase the system-wide file limit) along with the ulimit command.
- Further details: https://docs.oracle.com/cd/E19623-01/820-6168/file-descriptor-requirements.html
How to build MXNet
Please follow the instructions at https://mxnet.incubator.apache.org/install/index.html
...