o h"@s|ddlmZddlZddlmZddlZgdZedddZedd dZ d'd(ddZ d'd)ddZ d*d!d"Z d+d%d&Z dS),) annotationsN)TypeVar)fuse_conv_bn_evalfuse_conv_bn_weightsfuse_linear_bn_evalfuse_linear_bn_weightsConvTztorch.nn.modules.conv._ConvNd)boundLinearTztorch.nn.LinearFconvbn%torch.nn.modules.batchnorm._BatchNorm transposeboolreturnc Csf|js|jr Jdt|}|jdur|jdusJt|j|j|j|j|j|j|j|\|_|_|S)a+Fuse a convolutional module and a BatchNorm module into a single, new convolutional module. Args: conv (torch.nn.modules.conv._ConvNd): A convolutional module. bn (torch.nn.modules.batchnorm._BatchNorm): A BatchNorm module. transpose (bool, optional): If True, transpose the convolutional weight. Defaults to False. Returns: torch.nn.modules.conv._ConvNd: The fused convolutional module. .. note:: Both ``conv`` and ``bn`` must be in eval mode, and ``bn`` must have its running buffers computed. Fusion only for eval!N) trainingcopydeepcopy running_mean running_varrweightbiaseps)r r r fused_convri/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/torch/nn/utils/fusion.pyrs   rconv_w torch.Tensorconv_btorch.Tensor | Nonebn_rmbn_rvbn_epsfloatbn_wbn_b-tuple[torch.nn.Parameter, torch.nn.Parameter]cCs|j}|dur |jn|} |durt|}|durt|}|dur't|}t||} |r?ddgdgt|jd} nddgdgt|jd} ||| | j|d} ||| ||j| d} tj | |j tj | |j fS)aFuse convolutional module parameters and BatchNorm module parameters into new convolutional module parameters. Args: conv_w (torch.Tensor): Convolutional weight. conv_b (Optional[torch.Tensor]): Convolutional bias. bn_rm (torch.Tensor): BatchNorm running mean. bn_rv (torch.Tensor): BatchNorm running variance. bn_eps (float): BatchNorm epsilon. bn_w (Optional[torch.Tensor]): BatchNorm weight. bn_b (Optional[torch.Tensor]): BatchNorm bias. transpose (bool, optional): If True, transpose the conv weight. Defaults to False. Returns: Tuple[torch.nn.Parameter, torch.nn.Parameter]: Fused convolutional weight and bias. Ndtype) r,torch zeros_like ones_likersqrtlenshapereshapetonn Parameter requires_grad)rrr!r"r#r%r&rconv_weight_dtypeconv_bias_dtype bn_var_rsqrtr2 fused_conv_w fused_conv_brrrr8s*   rlinearcCs|js|jr Jdt|} |j|jks|jdksJd|jdur)|jdus+Jt|j|j |j|j|j |j|j \|_|_ |S)aFuse a linear module and a BatchNorm module into a single, new linear module. Args: linear (torch.nn.Linear): A Linear module. bn (torch.nn.modules.batchnorm._BatchNorm): A BatchNorm module. Returns: torch.nn.Linear: The fused linear module. .. note:: Both ``linear`` and ``bn`` must be in eval mode, and ``bn`` must have its running buffers computed. rr(zGTo fuse, linear.out_features == bn.num_features or bn.num_features == 1N) rrr out_features num_featuresrrrrrr)r=r fused_linearrrrrms"    rlinear_wlinear_bc Cs|j}|dur |jn|}|durt|}|t||} || dj|d} ||| |j|d} tj| |jtj| |jfS)a2Fuse linear module parameters and BatchNorm module parameters into new linear module parameters. Args: linear_w (torch.Tensor): Linear weight. linear_b (Optional[torch.Tensor]): Linear bias. bn_rm (torch.Tensor): BatchNorm running mean. bn_rv (torch.Tensor): BatchNorm running variance. bn_eps (float): BatchNorm epsilon. bn_w (torch.Tensor): BatchNorm weight. bn_b (torch.Tensor): BatchNorm bias. Returns: Tuple[torch.nn.Parameter, torch.nn.Parameter]: Fused linear weight and bias. Nr)r+) r,r-r.r0 unsqueezer4r5r6r7) rArBr!r"r#r%r&linear_weight_dtypelinear_bias_dtypebn_scalefused_wfused_brrrrs r)F)r rr r rrrr)rrrr r!rr"rr#r$r%r r&r rrrr')r=r r r rr )rArrBr r!rr"rr#r$r%rr&rrr') __future__rrtypingrr-__all__rr rrrrrrrrs     , 5/