o  h~, @sddlmZmZddlZddlZddlmZddl m Z ddl m Z e eZdeejejfdeefd d Zd+d ejd eed eedejfddZd,ddZ d-deefddZd.ddZdejdejfddZd/dejded eddfd!d"Zd#d$Zd%d&Zd'd(Zd)d*Z dS)0)OptionalUnionN) BatchFeature) BatchEncoding)loggingtensorreturncsTt|tjr t|jSt||jtdkrS|j}fddt|DS)z Deal with dynamic shape in tensorflow cleanly. Args: tensor (`tf.Tensor` or `np.ndarray`): The tensor we want the shape of. Returns: `List[int]`: The shape of the tensor as a list. Ncs$g|]\}}|dur|n|qSN).0isdynamicr i/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/transformers/tf_utils.py 0$zshape_list..) isinstancenpndarraylistshapetf TensorShapeas_list enumerate)rstaticr rr shape_lists   rlogitsaxisnamecCstjj|d||dS)a Stable wrapper that returns the same output as `tf.nn.softmax`, but that works reliably with XLA on CPU. It is meant as a workaround for the [following issue](https://github.com/tensorflow/tensorflow/issues/55682), and will be removed after it gets fixed. The arguments and outputs are the same as `tf.nn.softmax`, and relies on the fact that `softmax(x) = softmax(x + c)` (see https://ogunlao.github.io/2020/04/26/you_dont_really_know_softmax.html). Args: logits (`tf.Tensor`): Must be one of the following types: half, float32, float64. axis (`int`, *optional*): The dimension softmax would be performed on. The default is -1 which indicates the last dimension. name (`str`, *optional*): A name for the operation. Returns: `tf.Tensor`: A Tensor. Has the same type and shape as logits. g& .>rr r!)rnnsoftmaxr"r r rstable_softmax3sr%h㈵>c Cs|jjdks|jjdkst|tstdtjj||gdd\}}|dkr@dg|jj}t||||<t ||}t ||}tjj ||||||d}|S)NrzOOnly 1D weight and bias tensors are supported for now, with only a single axis.T)axeskeepdimsr')offsetscalevariance_epsilon) rrankrintNotImplementedErrorrr#momentsrreshapebatch_normalization) inputsweightbiasepsilonr meanvarianceroutputsr r rfunctional_layernormKs""  r:Fr+c Cs|dkrtd|r|durtd|r1tjt|dt|dftjd}tjjj|dd}|durP|jj s=|jj rPt |dkt d|jt d |j}t d ||}|durit t|d |jd }||9}|duru||7}tj|}||S) zDTF equivalent for torch's nn.functional.scaled_dot_product_attentionr;zDropout is not supported in this implementation - file an issue with Transformers and ping @Rocketknight1 if you need it for a port!Nz?You cannot specify an attn_mask and is_causal at the same time!dtyper)kg@z...qd, ...kd -> ...qkr'g) ValueErrorronesrint32 experimentalnumpytrilr> is_integeris_boolwherecasteinsumr#r$) querykeyvalue attn_mask dropout_p is_causalr+rprobsr r rscaled_dot_product_attentionks& (& rRcCs|dkr ||jj7}|dkr||jj7}||kr|St|}tj|||d}tj|d||g||ddgdd}t||S)Nrrr )rr-rmath reduce_prodconcatr1)input start_dimend_dimin_shape flattened_dim out_shaper r rflattens   * r]encoder_attention_maskcCst|tjs t|}|jjdkr|dddddddf}|jjdkr1|ddddddf}td|j||jj}|S)z Invert an attention mask (e.g., switches 0. and 1.). Args: encoder_attention_mask (`torch.Tensor`): An attention mask. Returns: `tf.Tensor`: The inverted attention mask. Nr) rrTensorconvert_to_tensorrr-rIr>min)r^encoder_extended_attention_maskr r rinvert_attention_masks   re input_ids embed_dim tensor_namec Cs>tjj|tj||jdd|dtj|d|dddS)a `tf.gather`, on which TF embedding layers are based, won't check positive out of bound indices on GPU, returning zeros instead. This function adds a check against that dangerous silent behavior. Args: tensor (`tf.Tensor`): The tensor of indices to check. embed_dim (`int`): The embedding dimension. tensor_name (`str`, *optional*): The name of the tensor to use in the error message. r=zThe maximum value of z (z>) must be smaller than the embedding layer's input dimension (z9). The likely cause is some problem at tokenization time.)messageN)r debugging assert_lessrIr>rT reduce_max)rrgrhr r rcheck_embeddings_within_boundss  rmc sdfdd|D}|rtdd|t|}d}t||}tfdd|DrD|d7}t||}tfdd|Ds/|dkr\t|D] \}}||jd ||f<qLd S||j|<d S) aSaves attributes (data) of the specified name into the HDF5 group. This method deals with an inherent problem of HDF5 file which is not able to store data larger than HDF5_OBJECT_HEADER_LIMIT bytes. Args: group: A pointer to a HDF5 group. name: A name of the attributes to save. data: Attributes data to store. Raises: RuntimeError: If any single attribute is too large to be saved. Copied from Keras to Transformers to avoid versioning issues. icsg|] }t|kr|qSr )lenr xHDF5_OBJECT_HEADER_LIMITr rrsz1save_attributes_to_hdf5_group..zSThe following attributes cannot be saved to HDF5 file because they are larger than z bytes: rc3s|]}|jkVqdSr )nbytesrorqr r sz0save_attributes_to_hdf5_group..%s%dN) RuntimeErrorrasarray array_splitanyrattrs) groupr!databad_attributesdata_npy num_chunks chunked_datachunk_id chunk_datar rqrsave_attributes_to_hdf5_groups,   rcCs|||jvrdd|j|D}|Sg}d}d||f|jvr<|dd|jd||fD|d7}d||f|jvs|S)aLoads attributes of the specified name from the HDF5 group. This method deals with an inherent problem of HDF5 file which is not able to store data larger than HDF5_OBJECT_HEADER_LIMIT bytes. Args: group: A pointer to a HDF5 group. name: A name of the attributes to load. Returns: data: Attributes data. Copied from Keras to Transformers to avoid versioning issues. cS$g|]}t|dr|dn|qSdecodeutf8hasattrrr nr r rrrz3load_attributes_from_hdf5_group..rrucSrrrrr r rr rr)rzextend)r{r!r|rr r rload_attributes_from_hdf5_groups  rcCsdd}tj||S)zwExpands 1-dimensional `Tensor`s into 2-dimensional `Tensor`s. Copied from Keras to here to avoid versioning issues.cSs*t|tjr|jjdkrtj|ddS|S)Nrr'rS)rrrarr- expand_dims)tr r r_expand_single_1d_tensorsz+expand_1d.._expand_single_1d_tensor)rnest map_structure)r|rr r r expand_1dsrcOsh|rt|dttfrt|}t|d|d<||fSd|vr0t|dttfr0t|d|d<||fS)Nrrp)rrrrdict)argskwargsr r rconvert_batch_encodingsr)NN)r&r')Nr;FN)rr')rf)!typingrrrDr tensorflowrfeature_extraction_utilsrtokenization_utils_baserutilsr get_logger__name__loggerrarrr.rstrr%r:floatrRr]rermrrrrr r r rs*    "( ! /