o h@svddlZddlZddlmZddlmZmZddlZgdZ dZ ddZ ddZ d d Z d d Zd eejeejfddfddZde ddfddZdde ddfddd eejdeeejeejfdededeeejjdeeejddfddZ d!d eejdeddfddZ  d"d eejdeejddfddZe ddfd eejdeejdeddfdd ZdS)#N)Sequence)OptionalUnion) all_reducereduce broadcast all_gatherreduce_scattercCstttjds tddSt}|D]%}|jrdS|s!dS|js'dS| }||vr2dS| |qdS)N_nccl_all_reducez)PyTorch is not compiled with NCCL supportFT) hasattrtorch_Cwarningswarnset is_sparse is_contiguousis_cuda get_deviceadd)tensorsdevicestensordevicerc/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/torch/cuda/nccl.py is_availables   rcCsTtj}|d?}|d?d@}|d@}tjd}|dkr$|||fS||||fS)a Returns the version of the NCCL. This function returns a tuple containing the major, minor, and patch version numbers of the NCCL. The suffix is also included in the tuple if a version suffix exists. Returns: tuple: The version information of the NCCL. izutf-8)r r _nccl_version_nccl_version_suffixdecode)vermajorminorpatchsuffixrrrversion$s    r(cCs tjSN)r r _nccl_unique_idrrrr unique_id9s r+cCstj|||Sr))r r _nccl_init_rank) num_ranksuidrankrrr init_rank=sr0inputsreturncCs&t|tjjr t|tjrtddS)Nz(Inputs should be a collection of tensors) isinstance collectionsabc Containerr Tensor TypeError)r1rrr_check_sequence_typeAs r9cCs4t||dur |}t|tj|||||dSr))r9r r r r1outputsopstreamscommsrrrrHs r)r;outputrootr<r=r;cCst||dur|durtdtjdtdd||}n$t|tjs7t|tj j r7tjdtdd||}n |dur?||n|}tj ||||||dS)Nz'output' and 'outputs' can not be both specified. 'outputs' is deprecated in favor of 'output', taking in a single output tensor. The signature of reduce is: reduce(inputs, output=None, root=0, op=SUM, streams=None, comms=None).z`nccl.reduce` with an output tensor list is deprecated. Please specify a single output tensor with argument 'output' instead instead.) stacklevelz\nccl.reduce with an output tensor list is deprecated. Please specify a single output tensor.) r9 ValueErrorrr FutureWarningr3r r7r4r5rr _nccl_reduce)r1r?r@r<r=r>r;_outputrrrrRs.   rcCst|tj||||dSr))r9r r _nccl_broadcast)r1r@r=r>rrrr}srcCs&t|t|tj||||dSr))r9r r _nccl_all_gather)r1r;r=r>rrrrsrcCs(t|t|tj|||||dSr))r9r r _nccl_reduce_scatterr:rrrr sr )rNN)NN)r4rcollections.abcrtypingrr torch.cudar __all__SUMrr(r+r0r7r9rintcudaStreamrrrr rrrrs "   ,