o  h@svddlZddlZddlZddlmZddlmZmZmZm Z ddl Z ddl Z ddl m Zddl Zddl mZddlmZmZmZddlmZmZdd lmZdd lmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%e&e'Z(e j)j*Gd d d eZ+e j)j*Gd ddeZ,e j)j*GdddeZ-e j)j*GdddZ.e j)j*GdddZ/e j)j*GdddZ0GdddZ1dS)N)partial)AnyDictOptionalUnion)lax) FLAX_MODEL_FOR_CAUSAL_LM_MAPPING+FLAX_MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING#FLAX_MODEL_FOR_VISION_2_SEQ_MAPPING) ModelOutputlogging)GenerationConfig) !FlaxForcedBOSTokenLogitsProcessor!FlaxForcedEOSTokenLogitsProcessorFlaxForceTokensLogitsProcessorFlaxLogitsProcessorListFlaxMinLengthLogitsProcessor FlaxNoRepeatNGramLogitsProcessor(FlaxSuppressTokensAtBeginLogitsProcessor!FlaxSuppressTokensLogitsProcessorFlaxTemperatureLogitsWarperFlaxTopKLogitsWarperFlaxTopPLogitsWarperc@$eZdZUdZdZeejed<dS)FlaxGreedySearchOutputz Flax Base class for outputs of decoder-only generation models using greedy search. Args: sequences (`jnp.ndarray` of shape `(batch_size, max_length)`): The generated sequences. N sequences __name__ __module__ __qualname____doc__rrjnpndarray__annotations__r&r&v/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/transformers/generation/flax_utils.pyr7  rc@r)FlaxSampleOutputz Flax Base class for outputs of decoder-only generation models using sampling. Args: sequences (`jnp.ndarray` of shape `(batch_size, max_length)`): The generated sequences. Nrrr&r&r&r'r)Er(r)c@s6eZdZUdZdZeejed<dZ eejed<dS)FlaxBeamSearchOutputaT Flax Base class for outputs of decoder-only generation models using greedy search. Args: sequences (`jnp.ndarray` of shape `(batch_size, max_length)`): The generated sequences. scores (`jnp.ndarray` of shape `(batch_size,)`): The scores (log probabilities) of the generated sequences. Nrscores) rr r!r"rrr#r$r%r+r&r&r&r'r*Ss  r*c@sHeZdZUejed<ejed<ejed<ejed<eeejfed<dS) GreedyStatecur_lenr running_tokenis_sent_finished model_kwargsNrr r!r#r$r%rstrr&r&r&r'r,ds    r,c@sReZdZUejed<ejed<ejed<ejed<ejed<eeejfed<dS) SampleStater-rr.r/prng_keyr0Nr1r&r&r&r'r3ms     r3c@s\eZdZUejed<ejed<ejed<ejed<ejed<ejed<eeejfed<dS) BeamSearchStater-running_sequencesrunning_scoresrr+r/r0Nr1r&r&r&r'r5ws      r5c@seZdZdZddZeddZddZ   d9d ed e ed e ed e e e e j fd e j f ddZ d:d e ed e ed efddZeddZddZddZd e e effddZ     d;de j de ede e j dede e e e j fd e ef d!d"Zded efd#d$Zded%ed e ed efd&d'Zd(ed)ed efd*d+Z       dddd,e ed-e ed.e ed4e ed5e eee fd e edede e e e j fd6e ed e e e e j ffd7d8Z dS)?FlaxGenerationMixina A class containing all functions for auto-regressive text generation, to be used as a mixin in [`FlaxPreTrainedModel`]. The class exposes [`~generation.FlaxGenerationMixin.generate`], which can be used for: - *greedy decoding* by calling [`~generation.FlaxGenerationMixin._greedy_search`] if `num_beams=1` and `do_sample=False` - *multinomial sampling* by calling [`~generation.FlaxGenerationMixin._sample`] if `num_beams=1` and `do_sample=True` - *beam-search decoding* by calling [`~generation.FlaxGenerationMixin._beam_search`] if `num_beams>1` and `do_sample=False` You do not need to call any of the above methods directly. Pass custom parameter values to 'generate' instead. To learn more about decoding strategies refer to the [text generation strategies guide](../generation_strategies). cOstd)NzbA model class needs to define a `prepare_inputs_for_generation` method in order to use `generate`.)NotImplementedError)selfargskwargsr&r&r'prepare_inputs_for_generationsz1FlaxGenerationMixin.prepare_inputs_for_generationcCs |}||r||}||s|S)zc Run generation in untraced mode. This should only be used for debugging purposes. r&)cond_fnbody_fn init_statestater&r&r'_run_loop_in_debugs z&FlaxGenerationMixin._run_loop_in_debugcCs2dd|D}|j|f|dd||d<|S)NcSs*i|]\}}|ds|ds||qS)decoder_ cross_attn) startswith).0argumentvaluer&r&r' szVFlaxGenerationMixin._prepare_encoder_decoder_kwargs_for_generation..T)params return_dictencoder_outputs)itemsencode)r: input_idsrJr0encoder_kwargsr&r&r'._prepare_encoder_decoder_kwargs_for_generations zBFlaxGenerationMixin._prepare_encoder_decoder_kwargs_for_generationN batch_sizedecoder_start_token_id bos_token_idr0returncCsR|durd|vr|d}|dur|S|||}tj|ddddj|ddS)Ndecoder_input_idsi4dtyperraxis)pop_get_decoder_start_token_idr#arrayreshaperepeat)r:rRrSrTr0rVr&r&r')_prepare_decoder_input_ids_for_generations    z=FlaxGenerationMixin._prepare_decoder_input_ids_for_generationcCs|dur|n|jj}|dur|n|jj}|dur|St|jdr3t|jjdr3|jjjdur3|jjjS|dur9|St|jdrRt|jjdrR|jjjdurR|jjjStd)NdecoderrSrTz\`decoder_start_token_id` or `bos_token_id` has to be defined for encoder-decoder generation.)generation_configrSrThasattrconfigrc ValueError)r:rSrTr&r&r'r^s0      z/FlaxGenerationMixin._get_decoder_start_token_idcCs0t|dddf|jd|f|jddS)Nrr)r# broadcast_toshape)tensor num_beamsr&r&r'_expand_to_num_beamss0z(FlaxGenerationMixin._expand_to_num_beamscCs|S)z This function can be overwritten in the specific modeling_flax_.py classes to allow for custom beam search behavior. Note that the only model that overwrites this method is [`~transformes.FlaxMarianMTModel`]. r&)r:logitsr&r&r'_adapt_logits_for_beam_searchsz1FlaxGenerationMixin._adapt_logits_for_beam_searchcCsx|s:tttg}t}|D]}|jt|jdd}|dur$||j qd|j j d}|r6|d|7}t |dS)z Confirms that the model class is compatible with generation. If not, raises an exception that points to the right class to use. N)defaultzThe current model class (zQ) is not compatible with `.generate()`, as it doesn't have a language model head.z2 Please use one of the following classes instead: ) can_generater r r setgettyperfaddr __class__ TypeError)r:generate_compatible_mappingsgenerate_compatible_classes model_mappingsupported_modelsexception_messager&r&r'_validate_model_classs" z)FlaxGenerationMixin._validate_model_classcCsg}tt|jj}d|vsd|vr|tt|jjO}|D]\}}|dur3||vr3||q"|r>td|ddS)zXValidates model kwargs for generation. Generate argument typos will also be caught here.r<r0Nz8The following `model_kwargs` are not used by the model: zG (note: typos in the generate arguments will also show up in this list)) rqinspect signaturer= parameters__call__rMappendrg)r:r0unused_model_args model_argskeyrHr&r&r'_validate_model_kwargss  z*FlaxGenerationMixin._validate_model_kwargsTrOrdr4tracerJlogits_processorc Ks||dur+|jjr(|jjt|jkr(t|j}||jkr(t d||_|j}t |}|j d&i|} | | |durE|nt}|durN|ntjd}|jdury|jdury| ddurjtd|j} t| trv| d} | |_|jdur|jjrtd|jjs|s|jdurt|dddf|jkdkrtd|jd} |jjr| d dur|||| } |j | |j|j!| d }|jd} |d duo|j"du} | r|j#dur|j"d krt d |j"dt$nE|j#dur| s|j"durtd|j#d|j"d|j#| |_"n!|j"tj"kr6|j"| |_"t%|jdd}|dur6t&|j"||_"|j'durP|j'|j"krPtd|j'd|j"d| |j"kro|jjr]dnd}td|d| d|j"d|j(|| |d}|j)s|j*dkr|j+||j"|j|j|||| dS|j)r|j*dkr|j,|d}|j-||j"|j|j|||||| d S|j)s|j*dkr|j.||j*d!}d | vr|j.| d d"|j*d!| d d"<d#D]}|| vr|j.| ||j*d!| |<q|j/||j"|j|j|j0|j1||||j2| d$ St3d%)'a Generates sequences of token ids for models with a language modeling head. Parameters: input_ids (`jnp.ndarray` of shape `(batch_size, sequence_length)`): The sequence used as a prompt for the generation. generation_config (`~generation.GenerationConfig`, *optional*): The generation configuration to be used as base parametrization for the generation call. `**kwargs` passed to generate matching the attributes of `generation_config` will override them. If `generation_config` is not provided, the default will be used, which had the following loading priority: 1) from the `generation_config.json` model file, if it exists; 2) from the model configuration. Please note that unspecified parameters will inherit [`~generation.GenerationConfig`]'s default values, whose documentation should be checked to parameterize generation. trace (`bool`, *optional*, defaults to `True`): Whether to trace generation. Setting `trace=False` should only be used for debugging and will lead to a considerably slower runtime. params (`Dict[str, jnp.ndarray]`, *optional*): Optionally the model parameters can be passed. Can be useful for parallelized generation. logits_processor (`FlaxLogitsProcessorList `, *optional*): Custom logits processors that complement the default logits processors built from arguments and generation config. If a logit processor is passed that is already created with the arguments or a generation config an error is thrown. This feature is intended for advanced users. kwargs (`Dict[str, Any]`, *optional*): Ad hoc parametrization of `generate_config` and/or additional model-specific kwargs that will be forwarded to the `forward` function of the model. If the model is an encoder-decoder model, encoder specific kwargs should not be prefixed and decoder specific kwargs should be prefixed with *decoder_*. Return: [`~utils.ModelOutput`]. NaSYou have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use and modify the model generation configuration (see https://huggingface.co/docs/transformers/generation_strategies#default-text-generation-configuration )rattention_maskzThe attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.zJ`decoder_start_token_id` has to be defined for encoder-decoder generation.rZzA decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer.rL)rSrTr0 max_lengthz0Using the model-agnostic default `max_length` (=zx) to control the generation length. recommend setting `max_new_tokens` to control the maximum length of the generation.zBoth `max_new_tokens` (=z) and `max_length`(=z) seem to have been set. `max_new_tokens` will take precedence. Please refer to the documentation for more information. (https://huggingface.co/docs/transformers/main/en/main_classes/text_generation)max_position_embeddingsz3Unfeasable length constraints: the minimum length (z%) is larger than the maximum length ()rVrOzInput length of z is z, but `max_length` is set to zW. This can lead to unexpected behavior. You should consider increasing`max_new_tokens`.)rdinput_ids_seq_lengthrr)rrrJr0)rd) logits_warperrrrJr0)rklast_hidden_staterdecoder_attention_mask)length_penaltyearly_stoppingrrrJnum_return_sequencesr0z,`Beam sampling is currently not implemented.r&)4r|rd_from_model_config_original_object_hashhashrfrom_model_configrfwarningswarncopydeepcopyupdaterrjaxrandomPRNGKey pad_token_id eos_token_idrrloggerwarning isinstancelistrSis_encoder_decoderrgr#sumrirQrbrTrmax_new_tokens UserWarninggetattrmin min_length_get_logits_processor do_samplerk_greedy_search_get_logits_warper_samplerl _beam_searchrrrr9)r:rOrdr4rrJrr<new_generation_configr0rrRrhas_default_max_lengthrinput_ids_stringrkwargr&r&r'generates *                      zFlaxGenerationMixin.generatecCs~t}|jdur|jdkr|t|j|jdur)|jdkr)|t|jdd|jdur=|jdkr=|t|jdd|S)z This class returns a [`FlaxLogitsProcessorList`] list object that contains all relevant [`FlaxLogitsWarper`] instances used for multinomial sampling. Ng?rr)top_kmin_tokens_to_keep)top_pr)r temperaturerrrrrr)r:rdwarpersr&r&r'rsz&FlaxGenerationMixin._get_logits_warperrcsRt}|jdur|jdur|jdkr|t|j|j|jdur)|t|j|jdur8|t|j |j|j durE|t |j |j dury}dksU|jdurW|n|d}|j durpt|j dkrp||j dd7}|t|j ||j durfdd|j D}|t||jdur|jdkr|t|j|||}|S)z This class returns a [`FlaxLogitsProcessorList`] list object that contains all relevant [`FlaxLogitsProcessor`] instances used to modify the scores of the language model head. NrZrrcs$g|]}|dd|dgqS)rrr&)rFirr&r' sz=FlaxGenerationMixin._get_logits_processor..)rrrrrforced_bos_token_idrforced_eos_token_idrrsuppress_tokensrbegin_suppress_tokensforced_decoder_idslenrrno_repeat_ngram_sizer_merge_criteria_processor_list)r:rdrr processors begin_indexrr&rr'rsF             z)FlaxGenerationMixin._get_logits_processor default_list custom_listcCs~t|dkr|S|D]-}|D](}t|t|ur6d}td|dt|d|d|d|d|d |d qq |||S) Nrzlogits processorz A custom z of type z with values zP has been passed to `generate`, but it has already been created with the values z. z has been created by passing the corresponding arguments to generate or by the model's config default values. If you just want to change the default values of zL consider passing them as arguments to `generate` instead of using a custom .)rrsrgextend)r:rrrocustom object_typer&r&r'r's*  z2FlaxGenerationMixin._merge_criteria_processor_listrrrc sPdurnjjdurnjjdurnjj|j\} } tjdur-tjnddtjtjdt| } tj| ftjd} t | |d} tj | ftj d} j jr`jnj|fi|}t| | || |d} fdd}fdd}|jddkr|| } |s||| } nt ||| } t| jd S) NrXrrr-rr.r/r0c(|jk}t|j}t||}|Szstate termination condition fn.r-r#allr/ logical_orrAhas_reached_max_lengthall_sequence_finishedfinish_generationrr&r'greedy_search_cond_fnh   zAFlaxGenerationMixin._greedy_search..greedy_search_cond_fncs|jfdi|j}|jdddf}|j||j}tj|dd}||j|j}|j|kB}|dddf}t |j|d|jf} ||j}t |jd||||dS)state update fn.rJNrZr[rrr) r.r0rmrr-r#argmaxr/rdynamic_update_sliceupdate_inputs_for_generationr,)rA model_outputsrm next_tokennext_is_sent_finishednext_sequencesnext_model_kwargs)rrmodelrrJr:r&r'greedy_search_body_fnos zAFlaxGenerationMixin._greedy_search..greedy_search_body_fnrr)rdrrrrir#r_int32fullrrzerosbool_rfrdecoder=r,rB while_looprr)r:rOrrrrrrJr0rRr-rr/rArrr&)rrrrrrJr:r'r<s6     z"FlaxGenerationMixin._greedy_searchrc sldurnjjdurnjjdurnjj|dur$|ntjd}|j\} } tj dur9tj nddtj tj dt | } tj | ftj d} t | |d} tj| ftjd}jjrljnj|fi| } t| | |||| d}fdd}fdd}|jd d kr||}|s|||}nt |||}t|jd S) NrrXr)r-rr.r/r4r0crrrrrr&r'sample_search_cond_fnrz:FlaxGenerationMixin._sample..sample_search_cond_fnc stj|j\}}|jfdi|j}|jdddf}|j||j}|||j}tjj ||dd}||j |j }|j |kB}|dddf}t |j|d|jf} ||j}t|jd|||||dS)rrJNrZr[rr)r-rr.r/r0r4)rrsplitr4r.r0rmrr- categoricalr/rrrr3) rAr4 prng_key_nextrrmrrrr)rrrrrrJr:r&r'sample_search_body_fns&z:FlaxGenerationMixin._sample..sample_search_body_fnrr)rdrrrrrrrir#r_rrrrrrrfrrr=r3rBrr)r)r:rOrrrr4rrrrJr0rRr-rr/rArrr&)rrrrrrrJr:r'rs:    zFlaxGenerationMixin._samplerrrc sdddd dddurn jj|dur|n jj}dur&n jjdur0n jjdur:n jj| durD| n jj} |j\ } tj durXtj nddtj |tj d}t | } |jd tj f|tj d} tj f|tj d}t | |d }tj ftjd}tt d gt d g d d g}t ft d } jjr jn d| vr҈| dd| dd<dD]}|| vr| || |<qԈ j|fi| } t| ||| ||| d}fdd}d f dd }t||jd d|}|s, |||}nt |||}tj|jd d}t|ddddf|j|j} t|dddf|j |j!}| ddd| ddf} |ddd| f}t"| |dS)z This beam search function is heavily inspired by Flax's official example: https://github.com/google/flax/blob/main/examples/wmt/decode.py cSs8|jdkr|S||jd|jdf|jddS)z8Flattens the first two dimensions of a non-scalar array.rrrNndimr`rirjr&r&r'flatten_beam_dims *z:FlaxGenerationMixin._beam_search..flatten_beam_dimcSs*|jdkr|S|||f|jddS)zFUnflattens the first, flat batch*beam dimension of a non-scalar array.rrNr)rjrRrkr&r&r'unflatten_beam_dim s z.unflatten_beam_dimcs:tt|||||ffdd}tj||S)z^ Gathers the beam slices indexed by beam_indices into new beam array. cs|jdkr|S|fS)Nr)rr batch_indices beam_indicesr&r' gather_fns  zIFlaxGenerationMixin._beam_search..gather_beams..gather_fn)r#r`aranger tree_utiltree_map)nestedrrR new_num_beamsrr&rr' gather_beamss z6FlaxGenerationMixin._beam_search..gather_beamsNrXrZ)rrrcrrLrr)r-r6r7rr+r/r0cs|jk}dkrdkr|jddddf}n|jddddf|j}t|jtj|jdddtd}t ||k}t |jdu@}||@|@S)z+beam search state termination condition fn.neverrNrTr\keepdimsr) r-r7r#wherer/rr+npr_anyr)rAnot_max_length_yetbest_running_scoreworst_finished_scoreimprovement_still_possiblestill_open_beam)decoder_prompt_lenrrrr&r'beam_search_cond_fn_s    z=FlaxGenerationMixin._beam_search..beam_search_cond_fnc st|jdd|j|f |f}|fd i|j} |jdddf }tj fdd|j } |}tj |}|j||j} | }|t j|jdd}|jd}| |f}d }tj||d \} } | |} |j| |} t j| |dd} t| | dd|jf}|dddd|jfk}| |td }tj| d d }||g| \}}| |jd } t |jjdd d |jd u@}||B}| |td 7} t j|j|gd d}t j|j| gd d}t j|j|gd d}tj| d d }|||g| \}}}| | }|| }tjfdd||d< ||j}t|jd ||||||dS)zbeam search state update fn.rrJNrZcs |SNr&r)rRrkrr&r's zOFlaxGenerationMixin._beam_search..beam_search_body_fn..rr[)krrTrcs|Srr&)x)rr&r'rspast_key_values)r-r7r6r+rr/r0)r dynamic_slicer6r-r0rmrrrrrnnn log_softmaxr# expand_dimsr7rir`rrr r_rhr/r concatenaterr+rr5) rAinput_ids_length input_tokenrrmcache log_probs vocab_size beams_to_keeptopk_log_probs topk_indicestopk_beam_indicestopk_running_sequencestopk_idstopk_sequencesdid_topk_just_finishedrunning_topk_log_probsnext_topk_indicesnext_running_sequencesnext_running_scoresbeams_in_batch_are_full add_penaltymerged_sequences merged_scoresmerged_is_sent_finishedtopk_merged_indicesr next_scoresrnext_running_indices next_cacher) rRrrrrrrrrrkrJr:rr&r'beam_search_body_fn}s~          z=FlaxGenerationMixin._beam_search..beam_search_body_fn)rr[)rr+)r)#rdrrrrrrrir#r_rrrrrrtiler onesrfrrr=r5rrBrr r/r rr6r+r7r*)r:rOrrrrrrrrJrr0r-rr6r/r7r+rrArr8 none_finishedr&)rRrrrrrrrrrrkrJr:rr'rsh   ,   &w  z FlaxGenerationMixin._beam_search)NNN)NN)NNTNN)NNNNTNN) NNNNNNTNN) NNNNNNTNNN)!rr r!r"r= staticmethodrBrQintrrr2r#r$rbr^rlrnr|rrrboolrrrrrrrfloatrrr&r&r&r'r8s8       V 4  Y   d    r8)2rr}r functoolsrtypingrrrrflaxr jax.numpynumpyr#r r models.autor r r utilsr r configuration_utilsrflax_logits_processrrrrrrrrrrr get_loggerrrstruct dataclassrr)r*r,r3r5r8r&r&r&r's8    4