error when run tools/train.py

[IAS-01]

[Warning] Your mxnet does not support ProposalTarget [Warning] Your mxnet does not support DecodeBBox [Warning] Your mxnet does not support BBoxNorm [Warning] Your mxnet does not support FocalLoss Traceback (most recent call last): File "tools/train.py", line 412, in <module> train_net(parse_args()) File "tools/train.py", line 407, in parse_args config = importlib.import_module(args.config.replace('.py', '').replace('/', '.')) File "/mmdetection3d/RangeDet/config/rangedet/rangedet_wo_aug_4_18e.py", line 22, in <module> from rangedet.symbol.head.builder import RangeRCNN as Detector File "/mmdetection3d/RangeDet/rangedet/symbol/head/builder.py", line 4, in <module> from operator_py import \ File "/mmdetection3d/RangeDet/operator_py/batch_rotated_iou.py", line 3, in <module> from mxnet.ndarray.contrib import RotatedIOU ImportError: cannot import name 'RotatedIOU' from 'mxnet.ndarray.contrib' (/usr/local/lib/python3.7/dist-packages/mxnet/ndarray/contrib.py)

the link to download MxNet is invalid, can you provide a new link?

this is provided by the SimpleDet, but it is invalid
https://1dv.aflat.top/mxnet_cu101-1.6.0b20191214-py2.py3-none-manylinux1_x86_64.whl

[Abyssaledge]

Sorry for the inconvenience. This error is raised because our customized MXNet operators in operator_cxx/contrib is not built.
First you need to clone MXNet from its repo and build it from the source. This is because the release version of MXNet is 1.8.0, which does not support numpy-style API. If you are not familiar with the whole installation process, please be patient and wait for a couple of days. We will provide detailed instructions after finishing this period of CVPR. A little bit busy now.

[IAS-01]

Sorry for the inconvenience. This error is raised because our customized MXNet operators in operator_cxx/contrib is not built. First you need to clone MXNet from its repo and build it from the source. This is because the release version of MXNet is 1.8.0, which does not support numpy-style API. If you are not familiar with the whole installation process, please be patient and wait for a couple of days. We will provide detailed instructions after finishing this period of CVPR. A little bit busy now.

Thank you very much, can you provide a compiled Mxnet whl package, if it is convenient

[Abyssaledge]

We have not tried to make the wheel file before. This might be a solution, but we need time to check. I suggest you follow our upcoming instructions if you are still stuck at this moment and not in a hurry.

[Treemann]

Hi @Abyssaledge ,
I prepare all the dependencies and try with mxnet-2.0 and error occurs.
from mxnet import metric ImportError: cannot import name 'metric'
Because in mxnet-2.0 metrics is under mxnet.gloun, I believe your code is developed under version<2.
Then I install mxnet-1.9.x and install rangedet by "pip install -v -e ." and there is no error in the log, but I notice that your self-defined operators seems not be compiled and installed. Is there anything else to do for the installation

[Warning] Your mxnet does not support ProposalTarget
[Warning] Your mxnet does not support DecodeBBox
[Warning] Your mxnet does not support BBoxNorm
[Warning] Your mxnet does not support FocalLoss
.......
    from mxnet.ndarray.contrib import RotatedIOU
ImportError: cannot import name 'RotatedIOU'

@IAS-01 Have you solve the problem ?

[Abyssaledge]

@Treemann Could you please provide the detailed information how you install MXNet and customized operators?

[Treemann]

@Abyssaledge , I follow the instruction of mxnet

# https://mxnet.apache.org/versions/1.8.0/get_started/build_from_source.html
git clone -b v1.9.x https://github.com/apache/incubator-mxnet.git --recursive
cd incubator-mxnet
cp config/linux_gpu.cmake config.cmake
mkdir build; cd build
cmake ..
cmake --build .
cd ..
python3 -m pip install -e ./python

Then following your instruction:
pip install -v -e .

[Abyssaledge]

@Treemann
Thanks for information and sorry for this inconvenience. The error about metric import is weird, because we develope RangeDet with MXNet 2.0 and we never meet this error. We will check this in couple of days and answer you as soon as we figure it out.

[Treemann]

@Abyssaledge FYI
For trying mxnet-2.0, I used mxnet-2.0.0a0 with pip installation since there is a released pip version, I didn't build mxnet-2.0 from source code.

[Abyssaledge]

@Treemann Hi, I found that the API of mxnet2.0 (in the master branch) has changed. You can pull the newest MXNet from the master branch and checkout to 1496c91871b9d81d6a18785bdc8a1c3450bedbca commit and build it from source, where we conduct our experiments and from mxnet import metric works fine.
As for the customized operators, you should modify the mx_home to your MXNet directory and simply run make in operator_cxx/contrib.
BTW, we have updated the train.py and test.py to import the customized operator.

[Treemann]

Thanks for your reply.
I'll try following your instruction and give some feedback.

[Treemann]

Hi @Abyssaledge, I followed your instruction to install all the dependencies (mxnet, openmpi, etc.), the aforementioned error disappeared, but some other error occurred.
Waiting for your update about the docs on environment setup.

[Warning] Your mxnet does not support ProposalTarget
[Warning] Your mxnet does not support DecodeBBox
[Warning] Your mxnet does not support BBoxNorm
[Warning] Your mxnet does not support FocalLoss
Please using new data mean and var
-------------------- meta_baseline_bias --------------------
*** An error occurred in MPI_Init_thread
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
......
  ompi_mpi_init: ompi_rte_init failed
  --> Returned "(null)" (-43) instead of "Success" (0)

[Abyssaledge]

Can you successfully run the code with single GPU training? @Treemann

[Treemann]

@Abyssaledge , I find the problem is related to horovod.
After I change the cuda and nccl version, horovod should be recompiled & reinstalled.
Now the training goes well. Thanks!