o h@s"ddlmZddlmmZddlmZddlm Z m Z m Z m Z m Z mZmZmZmZddlmZddlmZmZmZgdZGd d d eZGd d d eZGd ddeZGdddeZGdddeZGdddeZGdddeZGdddeZ GdddeZ!Gddde!Z"Gddde!Z#Gdd d e!Z$Gd!d"d"eZ%Gd#d$d$eZ&Gd%d&d&eZ'Gd'd(d(e'Z(Gd)d*d*e'Z)Gd+d,d,e'Z*Gd-d.d.eZ+Gd/d0d0e+Z,Gd1d2d2e+Z-Gd3d4d4e+Z.Gd5d6d6eZ/Gd7d8d8e/Z0Gd9d:d:e/Z1Gd;d<dt||_|dur|n||_||_||_||_||_dSN)super__init__r'r(r)r*r+r,)selfr'r(r)r*r+r, __class__l/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/torch/nn/modules/pooling.pyr19  z_MaxPoolNd.__init__cCdjdi|jS)Nzikernel_size={kernel_size}, stride={stride}, padding={padding}, dilation={dilation}, ceil_mode={ceil_mode}r5format__dict__r2r5r5r6 extra_reprJ z_MaxPoolNd.extra_repr)Nrr FF) __name__ __module__ __qualname__ __constants__bool__annotations__r rr1strr= __classcell__r5r5r3r6r&-s2 r&c@@eZdZUdZeed<eed<eed<eed<defddZd S) raApplies a 1D max pooling over an input signal composed of several input planes. In the simplest case, the output value of the layer with input size :math:`(N, C, L)` and output :math:`(N, C, L_{out})` can be precisely described as: .. math:: out(N_i, C_j, k) = \max_{m=0, \ldots, \text{kernel\_size} - 1} input(N_i, C_j, stride \times k + m) If :attr:`padding` is non-zero, then the input is implicitly padded with negative infinity on both sides for :attr:`padding` number of points. :attr:`dilation` is the stride between the elements within the sliding window. This `link`_ has a nice visualization of the pooling parameters. Note: When ceil_mode=True, sliding windows are allowed to go off-bounds if they start within the left padding or the input. Sliding windows that would start in the right padded region are ignored. Args: kernel_size: The size of the sliding window, must be > 0. stride: The stride of the sliding window, must be > 0. Default value is :attr:`kernel_size`. padding: Implicit negative infinity padding to be added on both sides, must be >= 0 and <= kernel_size / 2. dilation: The stride between elements within a sliding window, must be > 0. return_indices: If ``True``, will return the argmax along with the max values. Useful for :class:`torch.nn.MaxUnpool1d` later ceil_mode: If ``True``, will use `ceil` instead of `floor` to compute the output shape. This ensures that every element in the input tensor is covered by a sliding window. Shape: - Input: :math:`(N, C, L_{in})` or :math:`(C, L_{in})`. - Output: :math:`(N, C, L_{out})` or :math:`(C, L_{out})`, where .. math:: L_{out} = \left\lfloor \frac{L_{in} + 2 \times \text{padding} - \text{dilation} \times (\text{kernel\_size} - 1) - 1}{\text{stride}} + 1\right\rfloor Examples:: >>> # pool of size=3, stride=2 >>> m = nn.MaxPool1d(3, stride=2) >>> input = torch.randn(20, 16, 50) >>> output = m(input) .. _link: https://github.com/vdumoulin/conv_arithmetic/blob/master/README.md r'r(r)r*inputc C$tj||j|j|j|j|j|jdSN)r,r+)F max_pool1dr'r(r)r*r,r+r2rHr5r5r6forwardzMaxPool1d.forwardNr?r@rA__doc__rrDrrNr5r5r5r6rQs .rc@rG) ra Applies a 2D max pooling over an input signal composed of several input planes. In the simplest case, the output value of the layer with input size :math:`(N, C, H, W)`, output :math:`(N, C, H_{out}, W_{out})` and :attr:`kernel_size` :math:`(kH, kW)` can be precisely described as: .. math:: \begin{aligned} out(N_i, C_j, h, w) ={} & \max_{m=0, \ldots, kH-1} \max_{n=0, \ldots, kW-1} \\ & \text{input}(N_i, C_j, \text{stride[0]} \times h + m, \text{stride[1]} \times w + n) \end{aligned} If :attr:`padding` is non-zero, then the input is implicitly padded with negative infinity on both sides for :attr:`padding` number of points. :attr:`dilation` controls the spacing between the kernel points. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. Note: When ceil_mode=True, sliding windows are allowed to go off-bounds if they start within the left padding or the input. Sliding windows that would start in the right padded region are ignored. The parameters :attr:`kernel_size`, :attr:`stride`, :attr:`padding`, :attr:`dilation` can either be: - a single ``int`` -- in which case the same value is used for the height and width dimension - a ``tuple`` of two ints -- in which case, the first `int` is used for the height dimension, and the second `int` for the width dimension Args: kernel_size: the size of the window to take a max over stride: the stride of the window. Default value is :attr:`kernel_size` padding: Implicit negative infinity padding to be added on both sides dilation: a parameter that controls the stride of elements in the window return_indices: if ``True``, will return the max indices along with the outputs. Useful for :class:`torch.nn.MaxUnpool2d` later ceil_mode: when True, will use `ceil` instead of `floor` to compute the output shape Shape: - Input: :math:`(N, C, H_{in}, W_{in})` or :math:`(C, H_{in}, W_{in})` - Output: :math:`(N, C, H_{out}, W_{out})` or :math:`(C, H_{out}, W_{out})`, where .. math:: H_{out} = \left\lfloor\frac{H_{in} + 2 * \text{padding[0]} - \text{dilation[0]} \times (\text{kernel\_size[0]} - 1) - 1}{\text{stride[0]}} + 1\right\rfloor .. math:: W_{out} = \left\lfloor\frac{W_{in} + 2 * \text{padding[1]} - \text{dilation[1]} \times (\text{kernel\_size[1]} - 1) - 1}{\text{stride[1]}} + 1\right\rfloor Examples:: >>> # pool of square window of size=3, stride=2 >>> m = nn.MaxPool2d(3, stride=2) >>> # pool of non-square window >>> m = nn.MaxPool2d((3, 2), stride=(2, 1)) >>> input = torch.randn(20, 16, 50, 32) >>> output = m(input) .. _link: https://github.com/vdumoulin/conv_arithmetic/blob/master/README.md r'r(r)r*rHc CrIrJ)rK max_pool2dr'r(r)r*r,r+rMr5r5r6rNrOzMaxPool2d.forwardNr?r@rArQrrDrrNr5r5r5r6rs =rc@rG) ra Applies a 3D max pooling over an input signal composed of several input planes. In the simplest case, the output value of the layer with input size :math:`(N, C, D, H, W)`, output :math:`(N, C, D_{out}, H_{out}, W_{out})` and :attr:`kernel_size` :math:`(kD, kH, kW)` can be precisely described as: .. math:: \begin{aligned} \text{out}(N_i, C_j, d, h, w) ={} & \max_{k=0, \ldots, kD-1} \max_{m=0, \ldots, kH-1} \max_{n=0, \ldots, kW-1} \\ & \text{input}(N_i, C_j, \text{stride[0]} \times d + k, \text{stride[1]} \times h + m, \text{stride[2]} \times w + n) \end{aligned} If :attr:`padding` is non-zero, then the input is implicitly padded with negative infinity on both sides for :attr:`padding` number of points. :attr:`dilation` controls the spacing between the kernel points. It is harder to describe, but this `link`_ has a nice visualization of what :attr:`dilation` does. Note: When ceil_mode=True, sliding windows are allowed to go off-bounds if they start within the left padding or the input. Sliding windows that would start in the right padded region are ignored. The parameters :attr:`kernel_size`, :attr:`stride`, :attr:`padding`, :attr:`dilation` can either be: - a single ``int`` -- in which case the same value is used for the depth, height and width dimension - a ``tuple`` of three ints -- in which case, the first `int` is used for the depth dimension, the second `int` for the height dimension and the third `int` for the width dimension Args: kernel_size: the size of the window to take a max over stride: the stride of the window. Default value is :attr:`kernel_size` padding: Implicit negative infinity padding to be added on all three sides dilation: a parameter that controls the stride of elements in the window return_indices: if ``True``, will return the max indices along with the outputs. Useful for :class:`torch.nn.MaxUnpool3d` later ceil_mode: when True, will use `ceil` instead of `floor` to compute the output shape Shape: - Input: :math:`(N, C, D_{in}, H_{in}, W_{in})` or :math:`(C, D_{in}, H_{in}, W_{in})`. - Output: :math:`(N, C, D_{out}, H_{out}, W_{out})` or :math:`(C, D_{out}, H_{out}, W_{out})`, where .. math:: D_{out} = \left\lfloor\frac{D_{in} + 2 \times \text{padding}[0] - \text{dilation}[0] \times (\text{kernel\_size}[0] - 1) - 1}{\text{stride}[0]} + 1\right\rfloor .. math:: H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[1] - \text{dilation}[1] \times (\text{kernel\_size}[1] - 1) - 1}{\text{stride}[1]} + 1\right\rfloor .. math:: W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[2] - \text{dilation}[2] \times (\text{kernel\_size}[2] - 1) - 1}{\text{stride}[2]} + 1\right\rfloor Examples:: >>> # pool of square window of size=3, stride=2 >>> m = nn.MaxPool3d(3, stride=2) >>> # pool of non-square window >>> m = nn.MaxPool3d((3, 2, 2), stride=(2, 1, 2)) >>> input = torch.randn(20, 16, 50, 44, 31) >>> output = m(input) .. _link: https://github.com/vdumoulin/conv_arithmetic/blob/master/README.md r'r(r)r*rHc CrIrJ)rK max_pool3dr'r(r)r*r,r+rMr5r5r6rN'rOzMaxPool3d.forwardNr?r@rArQr rDrrNr5r5r5r6rs Arc@seZdZdefddZdS) _MaxUnpoolNdr-cCd|jd|jd|jSNz kernel_size=z , stride=z , padding=r'r(r)r<r5r5r6r=4z_MaxUnpoolNd.extra_reprN)r?r@rArEr=r5r5r5r6rV3srVc |eZdZUdZeed<eed<eed<  ddedeededdffdd Z dd ed ed ee e defd dZ Z S)ra Computes a partial inverse of :class:`MaxPool1d`. :class:`MaxPool1d` is not fully invertible, since the non-maximal values are lost. :class:`MaxUnpool1d` takes in as input the output of :class:`MaxPool1d` including the indices of the maximal values and computes a partial inverse in which all non-maximal values are set to zero. Note: This operation may behave nondeterministically when the input indices has repeat values. See https://github.com/pytorch/pytorch/issues/80827 and :doc:`/notes/randomness` for more information. .. note:: :class:`MaxPool1d` can map several input sizes to the same output sizes. Hence, the inversion process can get ambiguous. To accommodate this, you can provide the needed output size as an additional argument :attr:`output_size` in the forward call. See the Inputs and Example below. Args: kernel_size (int or tuple): Size of the max pooling window. stride (int or tuple): Stride of the max pooling window. It is set to :attr:`kernel_size` by default. padding (int or tuple): Padding that was added to the input Inputs: - `input`: the input Tensor to invert - `indices`: the indices given out by :class:`~torch.nn.MaxPool1d` - `output_size` (optional): the targeted output size Shape: - Input: :math:`(N, C, H_{in})` or :math:`(C, H_{in})`. - Output: :math:`(N, C, H_{out})` or :math:`(C, H_{out})`, where .. math:: H_{out} = (H_{in} - 1) \times \text{stride}[0] - 2 \times \text{padding}[0] + \text{kernel\_size}[0] or as given by :attr:`output_size` in the call operator Example:: >>> # xdoctest: +IGNORE_WANT("do other tests modify the global state?") >>> pool = nn.MaxPool1d(2, stride=2, return_indices=True) >>> unpool = nn.MaxUnpool1d(2, stride=2) >>> input = torch.tensor([[[1., 2, 3, 4, 5, 6, 7, 8]]]) >>> output, indices = pool(input) >>> unpool(output, indices) tensor([[[ 0., 2., 0., 4., 0., 6., 0., 8.]]]) >>> # Example showcasing the use of output_size >>> input = torch.tensor([[[1., 2, 3, 4, 5, 6, 7, 8, 9]]]) >>> output, indices = pool(input) >>> unpool(output, indices, output_size=input.size()) tensor([[[ 0., 2., 0., 4., 0., 6., 0., 8., 0.]]]) >>> unpool(output, indices) tensor([[[ 0., 2., 0., 4., 0., 6., 0., 8.]]]) r'r(r)Nrr-c8tt||_t|dur|n||_t||_dSr/)r0r1rr'r(r)r2r'r(r)r3r5r6r1w  zMaxUnpool1d.__init__rHindices output_sizecCt|||j|j|j|Sr/)rK max_unpool1dr'r(r)r2rHr_r`r5r5r6rNzMaxUnpool1d.forwardNrr/) r?r@rArQrrDrr1rlistintrNrFr5r5r3r6r8s4 :  rc r[)ra Computes a partial inverse of :class:`MaxPool2d`. :class:`MaxPool2d` is not fully invertible, since the non-maximal values are lost. :class:`MaxUnpool2d` takes in as input the output of :class:`MaxPool2d` including the indices of the maximal values and computes a partial inverse in which all non-maximal values are set to zero. Note: This operation may behave nondeterministically when the input indices has repeat values. See https://github.com/pytorch/pytorch/issues/80827 and :doc:`/notes/randomness` for more information. .. note:: :class:`MaxPool2d` can map several input sizes to the same output sizes. Hence, the inversion process can get ambiguous. To accommodate this, you can provide the needed output size as an additional argument :attr:`output_size` in the forward call. See the Inputs and Example below. Args: kernel_size (int or tuple): Size of the max pooling window. stride (int or tuple): Stride of the max pooling window. It is set to :attr:`kernel_size` by default. padding (int or tuple): Padding that was added to the input Inputs: - `input`: the input Tensor to invert - `indices`: the indices given out by :class:`~torch.nn.MaxPool2d` - `output_size` (optional): the targeted output size Shape: - Input: :math:`(N, C, H_{in}, W_{in})` or :math:`(C, H_{in}, W_{in})`. - Output: :math:`(N, C, H_{out}, W_{out})` or :math:`(C, H_{out}, W_{out})`, where .. math:: H_{out} = (H_{in} - 1) \times \text{stride[0]} - 2 \times \text{padding[0]} + \text{kernel\_size[0]} .. math:: W_{out} = (W_{in} - 1) \times \text{stride[1]} - 2 \times \text{padding[1]} + \text{kernel\_size[1]} or as given by :attr:`output_size` in the call operator Example:: >>> pool = nn.MaxPool2d(2, stride=2, return_indices=True) >>> unpool = nn.MaxUnpool2d(2, stride=2) >>> input = torch.tensor([[[[ 1., 2., 3., 4.], [ 5., 6., 7., 8.], [ 9., 10., 11., 12.], [13., 14., 15., 16.]]]]) >>> output, indices = pool(input) >>> unpool(output, indices) tensor([[[[ 0., 0., 0., 0.], [ 0., 6., 0., 8.], [ 0., 0., 0., 0.], [ 0., 14., 0., 16.]]]]) >>> # Now using output_size to resolve an ambiguous size for the inverse >>> input = torch.tensor([[[[ 1., 2., 3., 4., 5.], [ 6., 7., 8., 9., 10.], [11., 12., 13., 14., 15.], [16., 17., 18., 19., 20.]]]]) >>> output, indices = pool(input) >>> # This call will not work without specifying output_size >>> unpool(output, indices, output_size=input.size()) tensor([[[[ 0., 0., 0., 0., 0.], [ 0., 7., 0., 9., 0.], [ 0., 0., 0., 0., 0.], [ 0., 17., 0., 19., 0.]]]]) r'r(r)Nrr-cr\r/)r0r1rr'r(r)r]r3r5r6r1r^zMaxUnpool2d.__init__rHr_r`cCrar/)rK max_unpool2dr'r(r)rcr5r5r6rNrdzMaxUnpool2d.forwardrer/) r?r@rArQrrDrr1rrfrgrNrFr5r5r3r6rs4 G  rc r[)ra Computes a partial inverse of :class:`MaxPool3d`. :class:`MaxPool3d` is not fully invertible, since the non-maximal values are lost. :class:`MaxUnpool3d` takes in as input the output of :class:`MaxPool3d` including the indices of the maximal values and computes a partial inverse in which all non-maximal values are set to zero. Note: This operation may behave nondeterministically when the input indices has repeat values. See https://github.com/pytorch/pytorch/issues/80827 and :doc:`/notes/randomness` for more information. .. note:: :class:`MaxPool3d` can map several input sizes to the same output sizes. Hence, the inversion process can get ambiguous. To accommodate this, you can provide the needed output size as an additional argument :attr:`output_size` in the forward call. See the Inputs section below. Args: kernel_size (int or tuple): Size of the max pooling window. stride (int or tuple): Stride of the max pooling window. It is set to :attr:`kernel_size` by default. padding (int or tuple): Padding that was added to the input Inputs: - `input`: the input Tensor to invert - `indices`: the indices given out by :class:`~torch.nn.MaxPool3d` - `output_size` (optional): the targeted output size Shape: - Input: :math:`(N, C, D_{in}, H_{in}, W_{in})` or :math:`(C, D_{in}, H_{in}, W_{in})`. - Output: :math:`(N, C, D_{out}, H_{out}, W_{out})` or :math:`(C, D_{out}, H_{out}, W_{out})`, where .. math:: D_{out} = (D_{in} - 1) \times \text{stride[0]} - 2 \times \text{padding[0]} + \text{kernel\_size[0]} .. math:: H_{out} = (H_{in} - 1) \times \text{stride[1]} - 2 \times \text{padding[1]} + \text{kernel\_size[1]} .. math:: W_{out} = (W_{in} - 1) \times \text{stride[2]} - 2 \times \text{padding[2]} + \text{kernel\_size[2]} or as given by :attr:`output_size` in the call operator Example:: >>> # pool of square window of size=3, stride=2 >>> pool = nn.MaxPool3d(3, stride=2, return_indices=True) >>> unpool = nn.MaxUnpool3d(3, stride=2) >>> output, indices = pool(torch.randn(20, 16, 51, 33, 15)) >>> unpooled_output = unpool(output, indices) >>> unpooled_output.size() torch.Size([20, 16, 51, 33, 15]) r'r(r)Nrr-cr\r/)r0r1rr'r(r)r]r3r5r6r1$r^zMaxUnpool3d.__init__rHr_r`cCrar/)rK max_unpool3dr'r(r)rcr5r5r6rN/rdzMaxUnpool3d.forwardrer/) r?r@rArQr rDrr1rrfrgrNrFr5r5r3r6rs4 6  rc@s"eZdZgdZdefddZdS) _AvgPoolNd)r'r(r)r,count_include_padr-cCrWrXrYr<r5r5r6r=@rZz_AvgPoolNd.extra_reprN)r?r@rArBrEr=r5r5r5r6rj7srjc seZdZUdZeed<eed<eed<eed<eed<   ddededededed df fd d Zded efddZ Z S)ra2Applies a 1D average pooling over an input signal composed of several input planes. In the simplest case, the output value of the layer with input size :math:`(N, C, L)`, output :math:`(N, C, L_{out})` and :attr:`kernel_size` :math:`k` can be precisely described as: .. math:: \text{out}(N_i, C_j, l) = \frac{1}{k} \sum_{m=0}^{k-1} \text{input}(N_i, C_j, \text{stride} \times l + m) If :attr:`padding` is non-zero, then the input is implicitly zero-padded on both sides for :attr:`padding` number of points. Note: When ceil_mode=True, sliding windows are allowed to go off-bounds if they start within the left padding or the input. Sliding windows that would start in the right padded region are ignored. The parameters :attr:`kernel_size`, :attr:`stride`, :attr:`padding` can each be an ``int`` or a one-element tuple. Args: kernel_size: the size of the window stride: the stride of the window. Default value is :attr:`kernel_size` padding: implicit zero padding to be added on both sides ceil_mode: when True, will use `ceil` instead of `floor` to compute the output shape count_include_pad: when True, will include the zero-padding in the averaging calculation Shape: - Input: :math:`(N, C, L_{in})` or :math:`(C, L_{in})`. - Output: :math:`(N, C, L_{out})` or :math:`(C, L_{out})`, where .. math:: L_{out} = \left\lfloor \frac{L_{in} + 2 \times \text{padding} - \text{kernel\_size}}{\text{stride}} + 1\right\rfloor Per the note above, if ``ceil_mode`` is True and :math:`(L_{out} - 1) \times \text{stride} \geq L_{in} + \text{padding}`, we skip the last window as it would start in the right padded region, resulting in :math:`L_{out}` being reduced by one. Examples:: >>> # pool with window of size=3, stride=2 >>> m = nn.AvgPool1d(3, stride=2) >>> m(torch.tensor([[[1., 2, 3, 4, 5, 6, 7]]])) tensor([[[2., 4., 6.]]]) r'r(r)r,rkNrFTr-csDtt||_t|dur|n||_t||_||_||_dSr/)r0r1rr'r(r)r,rk)r2r'r(r)r,rkr3r5r6r1{s    zAvgPool1d.__init__rHcCst||j|j|j|j|jSr/)rK avg_pool1dr'r(r)r,rkrMr5r5r6rNszAvgPool1d.forward)NrFT) r?r@rArQrrDrCr1rrNrFr5r5r3r6rDs2 0rcseZdZUdZgdZeed<eed<eed<eed<eed<  ddedeedededed ee d dffdd Z de d e fddZ Z S)ra* Applies a 2D average pooling over an input signal composed of several input planes. In the simplest case, the output value of the layer with input size :math:`(N, C, H, W)`, output :math:`(N, C, H_{out}, W_{out})` and :attr:`kernel_size` :math:`(kH, kW)` can be precisely described as: .. math:: out(N_i, C_j, h, w) = \frac{1}{kH * kW} \sum_{m=0}^{kH-1} \sum_{n=0}^{kW-1} input(N_i, C_j, stride[0] \times h + m, stride[1] \times w + n) If :attr:`padding` is non-zero, then the input is implicitly zero-padded on both sides for :attr:`padding` number of points. Note: When ceil_mode=True, sliding windows are allowed to go off-bounds if they start within the left padding or the input. Sliding windows that would start in the right padded region are ignored. The parameters :attr:`kernel_size`, :attr:`stride`, :attr:`padding` can either be: - a single ``int`` -- in which case the same value is used for the height and width dimension - a ``tuple`` of two ints -- in which case, the first `int` is used for the height dimension, and the second `int` for the width dimension Args: kernel_size: the size of the window stride: the stride of the window. Default value is :attr:`kernel_size` padding: implicit zero padding to be added on both sides ceil_mode: when True, will use `ceil` instead of `floor` to compute the output shape count_include_pad: when True, will include the zero-padding in the averaging calculation divisor_override: if specified, it will be used as divisor, otherwise size of the pooling region will be used. Shape: - Input: :math:`(N, C, H_{in}, W_{in})` or :math:`(C, H_{in}, W_{in})`. - Output: :math:`(N, C, H_{out}, W_{out})` or :math:`(C, H_{out}, W_{out})`, where .. math:: H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[0] - \text{kernel\_size}[0]}{\text{stride}[0]} + 1\right\rfloor .. math:: W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[1] - \text{kernel\_size}[1]}{\text{stride}[1]} + 1\right\rfloor Per the note above, if ``ceil_mode`` is True and :math:`(H_{out} - 1)\times \text{stride}[0]\geq H_{in} + \text{padding}[0]`, we skip the last window as it would start in the bottom padded region, resulting in :math:`H_{out}` being reduced by one. The same applies for :math:`W_{out}`. Examples:: >>> # pool of square window of size=3, stride=2 >>> m = nn.AvgPool2d(3, stride=2) >>> # pool of non-square window >>> m = nn.AvgPool2d((3, 2), stride=(2, 1)) >>> input = torch.randn(20, 16, 50, 32) >>> output = m(input) r'r(r)r,rkdivisor_overrider'r(r)r,rkNrFTrnr-cr.r/r0r1r'r(r)r,rkrnr2r'r(r)r,rkrnr3r5r6r1r7zAvgPool2d.__init__rHc C"t||j|j|j|j|j|jSr/)rK avg_pool2dr'r(r)r,rkrnrMr5r5r6rNzAvgPool2d.forwardNrFTN)r?r@rArQrBrrDrCrrgr1rrNrFr5r5r3r6rs: = rcseZdZUdZgdZeed<eed<eed<eed<eed<  ddedeedededed ee d dffdd Z de d e fddZ fddZ ZS)rai Applies a 3D average pooling over an input signal composed of several input planes. In the simplest case, the output value of the layer with input size :math:`(N, C, D, H, W)`, output :math:`(N, C, D_{out}, H_{out}, W_{out})` and :attr:`kernel_size` :math:`(kD, kH, kW)` can be precisely described as: .. math:: \begin{aligned} \text{out}(N_i, C_j, d, h, w) ={} & \sum_{k=0}^{kD-1} \sum_{m=0}^{kH-1} \sum_{n=0}^{kW-1} \\ & \frac{\text{input}(N_i, C_j, \text{stride}[0] \times d + k, \text{stride}[1] \times h + m, \text{stride}[2] \times w + n)} {kD \times kH \times kW} \end{aligned} If :attr:`padding` is non-zero, then the input is implicitly zero-padded on all three sides for :attr:`padding` number of points. Note: When ceil_mode=True, sliding windows are allowed to go off-bounds if they start within the left padding or the input. Sliding windows that would start in the right padded region are ignored. The parameters :attr:`kernel_size`, :attr:`stride` can either be: - a single ``int`` -- in which case the same value is used for the depth, height and width dimension - a ``tuple`` of three ints -- in which case, the first `int` is used for the depth dimension, the second `int` for the height dimension and the third `int` for the width dimension Args: kernel_size: the size of the window stride: the stride of the window. Default value is :attr:`kernel_size` padding: implicit zero padding to be added on all three sides ceil_mode: when True, will use `ceil` instead of `floor` to compute the output shape count_include_pad: when True, will include the zero-padding in the averaging calculation divisor_override: if specified, it will be used as divisor, otherwise :attr:`kernel_size` will be used Shape: - Input: :math:`(N, C, D_{in}, H_{in}, W_{in})` or :math:`(C, D_{in}, H_{in}, W_{in})`. - Output: :math:`(N, C, D_{out}, H_{out}, W_{out})` or :math:`(C, D_{out}, H_{out}, W_{out})`, where .. math:: D_{out} = \left\lfloor\frac{D_{in} + 2 \times \text{padding}[0] - \text{kernel\_size}[0]}{\text{stride}[0]} + 1\right\rfloor .. math:: H_{out} = \left\lfloor\frac{H_{in} + 2 \times \text{padding}[1] - \text{kernel\_size}[1]}{\text{stride}[1]} + 1\right\rfloor .. math:: W_{out} = \left\lfloor\frac{W_{in} + 2 \times \text{padding}[2] - \text{kernel\_size}[2]}{\text{stride}[2]} + 1\right\rfloor Per the note above, if ``ceil_mode`` is True and :math:`(D_{out} - 1)\times \text{stride}[0]\geq D_{in} + \text{padding}[0]`, we skip the last window as it would start in the padded region, resulting in :math:`D_{out}` being reduced by one. The same applies for :math:`W_{out}` and :math:`H_{out}`. Examples:: >>> # pool of square window of size=3, stride=2 >>> m = nn.AvgPool3d(3, stride=2) >>> # pool of non-square window >>> m = nn.AvgPool3d((3, 2, 2), stride=(2, 1, 2)) >>> input = torch.randn(20, 16, 50, 44, 31) >>> output = m(input) rmr'r(r)r,rkNrFTrnr-cr.r/rorpr3r5r6r1Sr7zAvgPool3d.__init__rHc Crqr/)rK avg_pool3dr'r(r)r,rkrnrMr5r5r6rNdrszAvgPool3d.forwardcs:t||jdd|jdd|jdddS)Nr)rr,FrkT)r0 __setstate__r; setdefault)r2dr3r5r6rvos zAvgPool3d.__setstate__rt)r?r@rArQrBr rDrCrrgr1rrNrvrFr5r5r3r6rs< D  rc eZdZUdZgdZeed<eed<eed<eed<    ddede ede eded df fd d Z d e fd dZ Z S)raApplies a 2D fractional max pooling over an input signal composed of several input planes. Fractional MaxPooling is described in detail in the paper `Fractional MaxPooling`_ by Ben Graham The max-pooling operation is applied in :math:`kH \times kW` regions by a stochastic step size determined by the target output size. The number of output features is equal to the number of input planes. .. note:: Exactly one of ``output_size`` or ``output_ratio`` must be defined. Args: kernel_size: the size of the window to take a max over. Can be a single number k (for a square kernel of k x k) or a tuple `(kh, kw)` output_size: the target output size of the image of the form `oH x oW`. Can be a tuple `(oH, oW)` or a single number oH for a square image `oH x oH`. Note that we must have :math:`kH + oH - 1 <= H_{in}` and :math:`kW + oW - 1 <= W_{in}` output_ratio: If one wants to have an output size as a ratio of the input size, this option can be given. This has to be a number or tuple in the range (0, 1). Note that we must have :math:`kH + (output\_ratio\_H * H_{in}) - 1 <= H_{in}` and :math:`kW + (output\_ratio\_W * W_{in}) - 1 <= W_{in}` return_indices: if ``True``, will return the indices along with the outputs. Useful to pass to :meth:`nn.MaxUnpool2d`. Default: ``False`` Shape: - Input: :math:`(N, C, H_{in}, W_{in})` or :math:`(C, H_{in}, W_{in})`. - Output: :math:`(N, C, H_{out}, W_{out})` or :math:`(C, H_{out}, W_{out})`, where :math:`(H_{out}, W_{out})=\text{output\_size}` or :math:`(H_{out}, W_{out})=\text{output\_ratio} \times (H_{in}, W_{in})`. Examples: >>> # pool of square window of size=3, and target output size 13x12 >>> m = nn.FractionalMaxPool2d(3, output_size=(13, 12)) >>> # pool of square window and target output size being half of input image size >>> m = nn.FractionalMaxPool2d(3, output_ratio=(0.5, 0.5)) >>> input = torch.randn(20, 16, 50, 32) >>> output = m(input) .. _Fractional MaxPooling: https://arxiv.org/abs/1412.6071 r'r+r` output_ratior'r+r`r{NFr-cstt||_||_|d||durt|nd|_|dur&t|nd|_|dur5|dur5td|durA|durAtd|jdurkd|jdkrSdkrcnnd|jdkrbdksmntd|ddSdS)N_random_sampleszQFractionalMaxPool2d requires specifying either an output size, or a pooling ratio9only one of output_size and output_ratio may be specifiedrr *output_ratio must be between 0 and 1 (got )) r0r1rr'r+register_bufferr`r{ ValueErrorr2r'r`r{r+r|r3r5r6r1s*    : zFractionalMaxPool2d.__init__rHcC tj||j|j|j|j|jdSN)r|)rKfractional_max_pool2dr'r`r{r+r|rMr5r5r6rNzFractionalMaxPool2d.forwardNNFN)r?r@rArQrBrrDrCrrr1rrNrFr5r5r3r6rvs. )rc ry)ra0Applies a 3D fractional max pooling over an input signal composed of several input planes. Fractional MaxPooling is described in detail in the paper `Fractional MaxPooling`_ by Ben Graham The max-pooling operation is applied in :math:`kT \times kH \times kW` regions by a stochastic step size determined by the target output size. The number of output features is equal to the number of input planes. .. note:: Exactly one of ``output_size`` or ``output_ratio`` must be defined. Args: kernel_size: the size of the window to take a max over. Can be a single number k (for a square kernel of k x k x k) or a tuple `(kt x kh x kw)` output_size: the target output size of the image of the form `oT x oH x oW`. Can be a tuple `(oT, oH, oW)` or a single number oH for a square image `oH x oH x oH` output_ratio: If one wants to have an output size as a ratio of the input size, this option can be given. This has to be a number or tuple in the range (0, 1) return_indices: if ``True``, will return the indices along with the outputs. Useful to pass to :meth:`nn.MaxUnpool3d`. Default: ``False`` Shape: - Input: :math:`(N, C, T_{in}, H_{in}, W_{in})` or :math:`(C, T_{in}, H_{in}, W_{in})`. - Output: :math:`(N, C, T_{out}, H_{out}, W_{out})` or :math:`(C, T_{out}, H_{out}, W_{out})`, where :math:`(T_{out}, H_{out}, W_{out})=\text{output\_size}` or :math:`(T_{out}, H_{out}, W_{out})=\text{output\_ratio} \times (T_{in}, H_{in}, W_{in})` Examples: >>> # pool of cubic window of size=3, and target output size 13x12x11 >>> m = nn.FractionalMaxPool3d(3, output_size=(13, 12, 11)) >>> # pool of cubic window and target output size being half of input size >>> m = nn.FractionalMaxPool3d(3, output_ratio=(0.5, 0.5, 0.5)) >>> input = torch.randn(20, 16, 50, 32, 16) >>> output = m(input) .. _Fractional MaxPooling: https://arxiv.org/abs/1412.6071 rzr'r+r`r{NFr-cstt||_||_|d||durt|nd|_|dur&t|nd|_|dur5|dur5td|durA|durAtd|jdurzd|jdkrSdkrrnnd|jdkrbdkrrnnd|jdkrqdks|ntd|ddSdS) Nr|zQFractionalMaxPool3d requires specifying either an output size, or a pooling ratior}rr r~r) r0r1rr'r+rr`r{rrr3r5r6r1s.     zFractionalMaxPool3d.__init__rHcCrr)rKfractional_max_pool3dr'r`r{r+r|rMr5r5r6rNrzFractionalMaxPool3d.forwardr)r?r@rArQrBr rDrCrrr1rrNrFr5r5r3r6rs. &!rc sdeZdZUgdZeed<eed<  d dededeededdf fd d Z de fd d Z Z S) _LPPoolNd) norm_typer'r(r,rr,NFr'r(r-cs&t||_||_||_||_dSr/)r0r1rr'r(r,)r2rr'r(r,r3r5r6r1.s  z_LPPoolNd.__init__cCr8)NzXnorm_type={norm_type}, kernel_size={kernel_size}, stride={stride}, ceil_mode={ceil_mode}r5r9r<r5r5r6r=;r>z_LPPoolNd.extra_repr)NF) r?r@rArBfloatrDrCr rr1rEr=rFr5r5r3r6r(s$  rc@4eZdZUdZeed<eed<dedefddZdS) raApplies a 1D power-average pooling over an input signal composed of several input planes. On each window, the function computed is: .. math:: f(X) = \sqrt[p]{\sum_{x \in X} x^{p}} - At p = :math:`\infty`, one gets Max Pooling - At p = 1, one gets Sum Pooling (which is proportional to Average Pooling) .. note:: If the sum to the power of `p` is zero, the gradient of this function is not defined. This implementation will set the gradient to zero in this case. Args: kernel_size: a single int, the size of the window stride: a single int, the stride of the window. Default value is :attr:`kernel_size` ceil_mode: when True, will use `ceil` instead of `floor` to compute the output shape Shape: - Input: :math:`(N, C, L_{in})` or :math:`(C, L_{in})`. - Output: :math:`(N, C, L_{out})` or :math:`(C, L_{out})`, where .. math:: L_{out} = \left\lfloor\frac{L_{in} - \text{kernel\_size}}{\text{stride}} + 1\right\rfloor Examples:: >>> # power-2 pool of window of length 3, with stride 2. >>> m = nn.LPPool1d(2, 3, stride=2) >>> input = torch.randn(20, 16, 50) >>> output = m(input) r'r(rHr-cCt|t|j|j|j|jSr/)rK lp_pool1drrr'r(r,rMr5r5r6rNfzLPPool1d.forwardNrPr5r5r5r6rBs  rc@r) ra$Applies a 2D power-average pooling over an input signal composed of several input planes. On each window, the function computed is: .. math:: f(X) = \sqrt[p]{\sum_{x \in X} x^{p}} - At p = :math:`\infty`, one gets Max Pooling - At p = 1, one gets Sum Pooling (which is proportional to average pooling) The parameters :attr:`kernel_size`, :attr:`stride` can either be: - a single ``int`` -- in which case the same value is used for the height and width dimension - a ``tuple`` of two ints -- in which case, the first `int` is used for the height dimension, and the second `int` for the width dimension .. note:: If the sum to the power of `p` is zero, the gradient of this function is not defined. This implementation will set the gradient to zero in this case. Args: kernel_size: the size of the window stride: the stride of the window. Default value is :attr:`kernel_size` ceil_mode: when True, will use `ceil` instead of `floor` to compute the output shape Shape: - Input: :math:`(N, C, H_{in}, W_{in})` or :math:`(C, H_{in}, W_{in})`. - Output: :math:`(N, C, H_{out}, W_{out})` or :math:`(C, H_{out}, W_{out})`, where .. math:: H_{out} = \left\lfloor\frac{H_{in} - \text{kernel\_size}[0]}{\text{stride}[0]} + 1\right\rfloor .. math:: W_{out} = \left\lfloor\frac{W_{in} - \text{kernel\_size}[1]}{\text{stride}[1]} + 1\right\rfloor Examples:: >>> # power-2 pool of square window of size=3, stride=2 >>> m = nn.LPPool2d(2, 3, stride=2) >>> # pool of non-square window of power 1.2 >>> m = nn.LPPool2d(1.2, (3, 2), stride=(2, 1)) >>> input = torch.randn(20, 16, 50, 32) >>> output = m(input) r'r(rHr-cCrr/)rK lp_pool2drrr'r(r,rMr5r5r6rNrzLPPool2d.forwardNrSr5r5r5r6rls -rc@r) raApplies a 3D power-average pooling over an input signal composed of several input planes. On each window, the function computed is: .. math:: f(X) = \sqrt[p]{\sum_{x \in X} x^{p}} - At p = :math:`\infty`, one gets Max Pooling - At p = 1, one gets Sum Pooling (which is proportional to average pooling) The parameters :attr:`kernel_size`, :attr:`stride` can either be: - a single ``int`` -- in which case the same value is used for the height, width and depth dimension - a ``tuple`` of three ints -- in which case, the first `int` is used for the depth dimension, the second `int` for the height dimension and the third `int` for the width dimension .. note:: If the sum to the power of `p` is zero, the gradient of this function is not defined. This implementation will set the gradient to zero in this case. Args: kernel_size: the size of the window stride: the stride of the window. Default value is :attr:`kernel_size` ceil_mode: when True, will use `ceil` instead of `floor` to compute the output shape Shape: - Input: :math:`(N, C, D_{in}, H_{in}, W_{in})` or :math:`(C, D_{in}, H_{in}, W_{in})`. - Output: :math:`(N, C, D_{out}, H_{out}, W_{out})` or :math:`(C, D_{out}, H_{out}, W_{out})`, where .. math:: D_{out} = \left\lfloor\frac{D_{in} - \text{kernel\_size}[0]}{\text{stride}[0]} + 1\right\rfloor .. math:: H_{out} = \left\lfloor\frac{H_{in} - \text{kernel\_size}[1]}{\text{stride}[1]} + 1\right\rfloor .. math:: W_{out} = \left\lfloor\frac{W_{in} - \text{kernel\_size}[2]}{\text{stride}[2]} + 1\right\rfloor Examples:: >>> # power-2 pool of square window of size=3, stride=2 >>> m = nn.LPPool3d(2, 3, stride=2) >>> # pool of non-square window of power 1.2 >>> m = nn.LPPool3d(1.2, (3, 2, 2), stride=(2, 1, 2)) >>> input = torch.randn(20, 16, 50, 44, 31) >>> output = m(input) r'r(rHr-cCrr/)rK lp_pool3drrr'r(r,rMr5r5r6rNrzLPPool3d.forwardNrUr5r5r5r6rs 1rcsNeZdZUddgZeed< d dededdffdd Zdefdd Z Z S) _AdaptiveMaxPoolNdr`r+Fr-Ncst||_||_dSr/)r0r1r`r+)r2r`r+r3r5r6r1s  z_AdaptiveMaxPoolNd.__init__cC d|jSNz output_size=r`r<r5r5r6r= z_AdaptiveMaxPoolNd.extra_repr)F) r?r@rArBrCrDr r1rEr=rFr5r5r3r6rs rc@(eZdZUdZeed<defddZdS)r a8Applies a 1D adaptive max pooling over an input signal composed of several input planes. The output size is :math:`L_{out}`, for any input size. The number of output features is equal to the number of input planes. Args: output_size: the target output size :math:`L_{out}`. return_indices: if ``True``, will return the indices along with the outputs. Useful to pass to nn.MaxUnpool1d. Default: ``False`` Shape: - Input: :math:`(N, C, L_{in})` or :math:`(C, L_{in})`. - Output: :math:`(N, C, L_{out})` or :math:`(C, L_{out})`, where :math:`L_{out}=\text{output\_size}`. Examples: >>> # target output size of 5 >>> m = nn.AdaptiveMaxPool1d(5) >>> input = torch.randn(1, 64, 8) >>> output = m(input) r`rHcCt||j|jSr/)rKadaptive_max_pool1dr`r+rMr5r5r6rN zAdaptiveMaxPool1d.forwardNrPr5r5r5r6r s r c@r)r!aEApplies a 2D adaptive max pooling over an input signal composed of several input planes. The output is of size :math:`H_{out} \times W_{out}`, for any input size. The number of output features is equal to the number of input planes. Args: output_size: the target output size of the image of the form :math:`H_{out} \times W_{out}`. Can be a tuple :math:`(H_{out}, W_{out})` or a single :math:`H_{out}` for a square image :math:`H_{out} \times H_{out}`. :math:`H_{out}` and :math:`W_{out}` can be either a ``int``, or ``None`` which means the size will be the same as that of the input. return_indices: if ``True``, will return the indices along with the outputs. Useful to pass to nn.MaxUnpool2d. Default: ``False`` Shape: - Input: :math:`(N, C, H_{in}, W_{in})` or :math:`(C, H_{in}, W_{in})`. - Output: :math:`(N, C, H_{out}, W_{out})` or :math:`(C, H_{out}, W_{out})`, where :math:`(H_{out}, W_{out})=\text{output\_size}`. Examples: >>> # target output size of 5x7 >>> m = nn.AdaptiveMaxPool2d((5, 7)) >>> input = torch.randn(1, 64, 8, 9) >>> output = m(input) >>> # target output size of 7x7 (square) >>> m = nn.AdaptiveMaxPool2d(7) >>> input = torch.randn(1, 64, 10, 9) >>> output = m(input) >>> # target output size of 10x7 >>> m = nn.AdaptiveMaxPool2d((None, 7)) >>> input = torch.randn(1, 64, 10, 9) >>> output = m(input) r`rHcCrr/)rKadaptive_max_pool2dr`r+rMr5r5r6rN5rzAdaptiveMaxPool2d.forwardNr?r@rArQrrDrrNr5r5r5r6r!s #r!c@r)r"aApplies a 3D adaptive max pooling over an input signal composed of several input planes. The output is of size :math:`D_{out} \times H_{out} \times W_{out}`, for any input size. The number of output features is equal to the number of input planes. Args: output_size: the target output size of the image of the form :math:`D_{out} \times H_{out} \times W_{out}`. Can be a tuple :math:`(D_{out}, H_{out}, W_{out})` or a single :math:`D_{out}` for a cube :math:`D_{out} \times D_{out} \times D_{out}`. :math:`D_{out}`, :math:`H_{out}` and :math:`W_{out}` can be either a ``int``, or ``None`` which means the size will be the same as that of the input. return_indices: if ``True``, will return the indices along with the outputs. Useful to pass to nn.MaxUnpool3d. Default: ``False`` Shape: - Input: :math:`(N, C, D_{in}, H_{in}, W_{in})` or :math:`(C, D_{in}, H_{in}, W_{in})`. - Output: :math:`(N, C, D_{out}, H_{out}, W_{out})` or :math:`(C, D_{out}, H_{out}, W_{out})`, where :math:`(D_{out}, H_{out}, W_{out})=\text{output\_size}`. Examples: >>> # target output size of 5x7x9 >>> m = nn.AdaptiveMaxPool3d((5, 7, 9)) >>> input = torch.randn(1, 64, 8, 9, 10) >>> output = m(input) >>> # target output size of 7x7x7 (cube) >>> m = nn.AdaptiveMaxPool3d(7) >>> input = torch.randn(1, 64, 10, 9, 8) >>> output = m(input) >>> # target output size of 7x9x8 >>> m = nn.AdaptiveMaxPool3d((7, None, None)) >>> input = torch.randn(1, 64, 10, 9, 8) >>> output = m(input) r`rHcCrr/)rKadaptive_max_pool3dr`r+rMr5r5r6rN`rzAdaptiveMaxPool3d.forwardNr?r@rArQr rDrrNr5r5r5r6r"9s $r"cs:eZdZdgZdeddffdd ZdefddZZS)_AdaptiveAvgPoolNdr`r-Ncst||_dSr/)r0r1r`)r2r`r3r5r6r1gs  z_AdaptiveAvgPoolNd.__init__cCrrrr<r5r5r6r=krz_AdaptiveAvgPoolNd.extra_repr) r?r@rArBr r1rEr=rFr5r5r3r6rdsrc@,eZdZUdZeed<dedefddZdS)r#aApplies a 1D adaptive average pooling over an input signal composed of several input planes. The output size is :math:`L_{out}`, for any input size. The number of output features is equal to the number of input planes. Args: output_size: the target output size :math:`L_{out}`. Shape: - Input: :math:`(N, C, L_{in})` or :math:`(C, L_{in})`. - Output: :math:`(N, C, L_{out})` or :math:`(C, L_{out})`, where :math:`L_{out}=\text{output\_size}`. Examples: >>> # target output size of 5 >>> m = nn.AdaptiveAvgPool1d(5) >>> input = torch.randn(1, 64, 8) >>> output = m(input) r`rHr-cCt||jSr/)rKadaptive_avg_pool1dr`rMr5r5r6rNzAdaptiveAvgPool1d.forwardNrPr5r5r5r6r#os r#c@r)r$aApplies a 2D adaptive average pooling over an input signal composed of several input planes. The output is of size H x W, for any input size. The number of output features is equal to the number of input planes. Args: output_size: the target output size of the image of the form H x W. Can be a tuple (H, W) or a single H for a square image H x H. H and W can be either a ``int``, or ``None`` which means the size will be the same as that of the input. Shape: - Input: :math:`(N, C, H_{in}, W_{in})` or :math:`(C, H_{in}, W_{in})`. - Output: :math:`(N, C, S_{0}, S_{1})` or :math:`(C, S_{0}, S_{1})`, where :math:`S=\text{output\_size}`. Examples: >>> # target output size of 5x7 >>> m = nn.AdaptiveAvgPool2d((5, 7)) >>> input = torch.randn(1, 64, 8, 9) >>> output = m(input) >>> # target output size of 7x7 (square) >>> m = nn.AdaptiveAvgPool2d(7) >>> input = torch.randn(1, 64, 10, 9) >>> output = m(input) >>> # target output size of 10x7 >>> m = nn.AdaptiveAvgPool2d((None, 7)) >>> input = torch.randn(1, 64, 10, 9) >>> output = m(input) r`rHr-cCrr/)rKadaptive_avg_pool2dr`rMr5r5r6rNrzAdaptiveAvgPool2d.forwardNrr5r5r5r6r$  r$c@r)r%a(Applies a 3D adaptive average pooling over an input signal composed of several input planes. The output is of size D x H x W, for any input size. The number of output features is equal to the number of input planes. Args: output_size: the target output size of the form D x H x W. Can be a tuple (D, H, W) or a single number D for a cube D x D x D. D, H and W can be either a ``int``, or ``None`` which means the size will be the same as that of the input. Shape: - Input: :math:`(N, C, D_{in}, H_{in}, W_{in})` or :math:`(C, D_{in}, H_{in}, W_{in})`. - Output: :math:`(N, C, S_{0}, S_{1}, S_{2})` or :math:`(C, S_{0}, S_{1}, S_{2})`, where :math:`S=\text{output\_size}`. Examples: >>> # target output size of 5x7x9 >>> m = nn.AdaptiveAvgPool3d((5, 7, 9)) >>> input = torch.randn(1, 64, 8, 9, 10) >>> output = m(input) >>> # target output size of 7x7x7 (cube) >>> m = nn.AdaptiveAvgPool3d(7) >>> input = torch.randn(1, 64, 10, 9, 8) >>> output = m(input) >>> # target output size of 7x9x8 >>> m = nn.AdaptiveAvgPool3d((7, None, None)) >>> input = torch.randn(1, 64, 10, 9, 8) >>> output = m(input) r`rHr-cCrr/)rKadaptive_avg_pool3dr`rMr5r5r6rNrzAdaptiveAvgPool3d.forwardNrr5r5r5r6r%rr%)3typingrtorch.nn.functionalnn functionalrKtorchrtorch.nn.common_typesrrrrrr r r r modulerutilsrrr__all__r&rrrrVrrrrjrrrrrrrrrrr r!r"rr#r$r%r5r5r5r6sB  , $@OSR_N QjwYY*7;*+ '