o  h@s>dZddlmZddlmZddlmZ    dddZdS) z9VPTQ (Vector Post-Training Quantization) integration fileN)init_empty_weights) VQuantLinearFcCsf|sdgn|}|D]\}}|durg}||d|}|j}|j} t|tjr||vr|| vs9|d|vr| |dpF||dd} t B|j } |j } t | | | d| d| d| d| d | d | d | d | d dd|j dud|j|<d}|j|dWdn1swYtt|dkrt|||||d\} }|dq ||fS)ax Public method that recursively replaces the Linear layers of the given model with VPTQ quantized layers. `accelerate` is needed to use this method. Returns the converted model and a boolean that indicates if the conversion has been successfull or not. Args: model (`torch.nn.Module`): The model to convert, can be any `torch.nn.Module` instance. quantization_config (`VptqConfig`): The quantization config object that contains the quantization parameters. modules_to_not_convert (`List[`str`]`, *optional*, defaults to `["lm_head"]`): Names of the modules to not convert in `VQuantLinear`. In practice we keep the `lm_head` in full precision for numerical stability reasons. current_key_name (`list`, *optional*): A list that contains the current key name. This is used for recursion and should not be passed by the user. has_been_replaced (`bool`, *optional*): A boolean that indicates if the conversion has been successful or not. This is used for recursion and should not be passed by the user. lm_headN. vector_lens num_centroidsnum_res_centroids group_num group_size outlier_sizeindices_as_float enable_norm enable_permTF) rrr r r r r rris_indice_packedenable_proxy_errorbiasr)quantization_configmodules_to_not_convertcurrent_key_namehas_been_replaced)named_childrenappendjoinshared_layer_configconfig_for_layers isinstancennLineargetr in_features out_featuresrr_modulesrequires_grad_lenlistchildrenreplace_with_vptq_linearpop)modelrrrrnamemodule layer_namerr layer_paramsr r!_r/r/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/transformers/integrations/vptq.pyr's^     r')NNNF)__doc__torch.nnr acceleratervptqrr'r/r/r/r0s