o  h)@sddlmZmZmZerddlmZer)ddlZddlmZddl mm Z e e ZdZdejdejfdd Zejd ejd ejdejfd d ZGdddejZ     dddZ    dddZdS))is_accelerate_availableis_torch_availablelogging)init_empty_weightsNquantized_weightsreturnc Cs|j}|dtdt}t|dkr|f}n |g|ddR}|d7}tj||jtjd}|tj}tt|d|d}t |D]#}||}t|||d} |d| |||| d|>O<qD|S)a Packs a tensor of quantized weights into a compact format using 2 bits per value. Parameters: ----------- quantized_weights : torch.Tensor A tensor containing ternary quantized weights with values in {-1, 0, 1}. These values are adjusted to {0, 1, 2} before being packed. Returns: -------- torch.Tensor A packed tensor where each element stores 4 quantized values (each using 2 bits) in an 8-bit format. rNdevicedtyper) shapeVALUES_PER_ITEMlentorchzerosr uint8tominrange) roriginal_shaperow_dimpacked_tensor_shapepackedunpackeditistartendr t/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/transformers/integrations/bitnet.py pack_weightss   *r"rr c Cs|j}t|dkr|dt}|f}n|dt}|g|ddR}tj||jtjd}ttD] }||d}||d}dd|>} || @d|?|||<q1||dS)u5 Unpacks a tensor of quantized weights that were stored in a packed format using 2 bits per value. Parameters: ----------- packed : torch.Tensor A tensor containing packed weights where each element represents 4 quantized values (using 2 bits per value). dtype : torch.dtype The dtype of the returned Tensor Returns: -------- torch.Tensor A tensor of unpacked weights, where each value is converted from its packed 2-bit representation. Example: -------- packed = torch.tensor([[0b10100001, 0b00011000], [0b10010000, 0b00001010]], dtype=torch.uint8) # Unpack the values unpacked = unpack_weights(packed) # Resulting unpacked tensor print(unpacked) # Output: tensor([[ 0, -1], [-1, 1], [-1, 1], [-1, 1], [ 1, 0], [ 0, -1], [ 1, -1], [ 1, -1]]) Explanation of the example: --------------------------- Let's take the first value for example 0b10100001, we we will only focus on the first column, because every element is unpacked across the first dimension - First 2 bits: `01` → 0 at [0][0] - Second 2 bits: `00` → -1 at [0][2] - Third 2 bits: `10` → 1 at [0][4] - Fourth 2 bits: `10` → 1 at [0][6] the second value of the same row (0b10010000) will give the values for [0][1], [0][3], [0][5], [0][7] We subtract 1 because during the packing process, it's easier to work with values like 0, 1, and 2. To make this possible, we add 1 to the original ternary weights (which are typically -1, 0, and 1) when packing them. When unpacking, we reverse this by subtracting 1 to restore the original ternary values. r rNr r) rrrrrr rrr) rr packed_shapeoriginal_row_dimunpacked_shaperrrrmaskr r r!unpack_weights9s1       r(csReZdZddededeffdd Zejddd Zejd d Z d d Z Z S) BitLinearN in_features out_featuresbiascst||_||_||_|dtj|t|ftj |d|dtj d||d|r;|dtj|||ddSd|_ dS)Nweight)r r weight_scaler r,) super__init__r r*r+register_bufferrrrronesr,)selfr*r+r,r r  __class__r r!r0s.   zBitLinear.__init__cCsbd|d }d|dd}||jdddjjdd}||||}|tj|fS)aX Activation function : Performs symmetric, per-token quantization on the input activations. Parameters: ----------- x : torch.Tensor Input activations to be quantized. num_bits : int, optional (default=8) Number of bits to use for quantization, determining the quantization range. Returns: -------- result : torch.Tensor Quantized activation tensor, with values mapped to an `int8` range. scale : torch.Tensor The per-channel scaling factors used to quantize the tensor. rr T)dimkeepdimgh㈵>)r)absmaxvaluesclamproundrrint8)r3inputnum_bitsQnQpscaleresultr r r!activation_quants  zBitLinear.activation_quantcCs|||}|S)Nr )r3r@ input_scaler.outr r r!post_quant_processs zBitLinear.post_quant_processcCsl|j}t||jd}||\}}t||j|}|||j|}|j dur4||j dd |7}|S)N)r r r7) r-r(r rFFlinearrrIr.r,view expand_as)r3r@ww_quant input_quantrGyr r r!forwards zBitLinear.forward)NN)r6) __name__ __module__ __qualname__intboolr0rcompilerFrIrR __classcell__r r r4r!r)~s  r)Fc sdurg|D]t\}}durg|tfdd|Dsdt5t|tjrU||vrU|j}|j} t || |j du|j j |j j d|j|<d}|j|dWdn1s_wYtt|dkryt||||d\} }d q ||fS) z Private method that wraps the recursion for module replacement. Returns the converted model and a boolean that indicates if the conversion has been successfull or not. Nc3s|] }|dvVqdS).N)join).0keycurrent_key_namer r! sz._replace_with_bitnet_linear..)r*r+r,r r TFr)modules_to_not_convertr_quantization_confighas_been_replacedr7)named_childrenappendanyr isinstancennLinearr*r+r)r,r-r r _modulesrequires_grad_rlistchildren_replace_with_bitnet_linearpop) modelrar_rbrc pre_quantizednamemoduler*r+_r r^r!rns@   rncCs`|durdgn|}|r|jdur||jtt|}t|||||d\}}|s.td|S)af A helper function to replace all `torch.nn.Linear` modules by `BitLinear158` modules`. The function will be run recursively and replace all `torch.nn.Linear` modules except for the `lm_head` that should be kept as a `torch.nn.Linear` module. The replacement is done under `init_empty_weights` context manager so no CPU/GPU memory is required to run this function. Each weight will be quantized along the channel. Parameters: model (`torch.nn.Module`): Input model or `torch.nn.Module` as the function is run recursively. modules_to_not_convert (`List[`str`]`, *optional*, defaults to `["lm_head"]`): Names of the modules to not convert in `EetqLinear`. In practice we keep the `lm_head` in full precision for numerical stability reasons. current_key_name (`List[`str`]`, *optional*): An array to track the current key of the recursion. This is used to check whether the current key (part of it) is not in the list of modules to not convert (for instances modules that are offloaded to `cpu` or `disk`). Nlm_head)rqzYou are loading your model using bitnet but no linear modules were found in your model. Please double check your model architecture, or submit an issue on github if you think this is a bug.)raextendrlsetrnloggerwarning)rprar_rbrqrcr r r!replace_with_bitnet_linears    rz)NNNFF)NNNF)utilsrrr acceleraterrtorch.nnrhtorch.nn.functional functionalrJ get_loggerrSrxrTensorr"rXr r(Moduler)rnrzr r r r!s0   &DE 5