o  hp@sddlmZmZddlmZddlmZmZmZm Z m Z ddl Z ddl Z ddlmZddlmZmZd Zd ZGd d d eZGd ddeZGdddeZGdddZdS))ABCabstractmethod)UserDict)DictListOptionalTupleUnionN)add_start_docstrings) ConstraintConstraintListStatea Args: input_ids (`torch.LongTensor` of shape `(batch_size * num_beams, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using any class inheriting from [`PreTrainedTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) next_scores (`torch.FloatTensor` of shape `(batch_size, 2 * num_beams)`): Current scores of the top `2 * num_beams` non-finished beam hypotheses. next_tokens (`torch.LongTensor` of shape `(batch_size, 2 * num_beams)`): `input_ids` of the tokens corresponding to the top `2 * num_beams` non-finished beam hypotheses. next_indices (`torch.LongTensor` of shape `(batch_size, 2 * num_beams)`): Beam indices indicating to which beam hypothesis the `next_tokens` correspond. pad_token_id (`int`, *optional*): The id of the *padding* token. eos_token_id (`Union[int, List[int]]`, *optional*): The id of the *end-of-sequence* token. Optionally, use a list to set multiple *end-of-sequence* tokens. beam_indices (`torch.LongTensor`, *optional*): Beam indices indicating to which beam hypothesis each token correspond. group_index (`int`, *optional*): The index of the group of beams. Used with [`~PreTrainedModel.group_beam_search`]. Return: `UserDict`: A dictionary composed of the fields as defined above: - **next_beam_scores** (`torch.FloatTensor` of shape `(batch_size * num_beams)`) -- Updated scores of all non-finished beams. - **next_beam_tokens** (`torch.FloatTensor` of shape `(batch_size * num_beams)`) -- Next tokens to be added to the non-finished beam_hypotheses. - **next_beam_indices** (`torch.FloatTensor` of shape `(batch_size * num_beams)`) -- Beam indices indicating to which beam the next tokens shall be added. a Args: input_ids (`torch.LongTensor` of shape `(batch_size * num_beams, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using any class inheriting from [`PreTrainedTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) final_beam_scores (`torch.FloatTensor` of shape `(batch_size * num_beams)`): The final scores of all non-finished beams. final_beam_tokens (`torch.FloatTensor` of shape `(batch_size * num_beams)`): The last tokens to be added to the non-finished beam_hypotheses. final_beam_indices (`torch.FloatTensor` of shape `(batch_size * num_beams)`): The beam indices indicating to which beam the `final_beam_tokens` shall be added. pad_token_id (`int`, *optional*): The id of the *padding* token. eos_token_id (`Union[int, List[int]]`, *optional*): The id of the *end-of-sequence* token. Optionally, use a list to set multiple *end-of-sequence* tokens. Return: `torch.LongTensor` of shape `(batch_size * num_return_sequences, sequence_length)`: The generated sequences. The second dimension (sequence_length) is either equal to `max_length` or shorter if all batches finished early due to the `eos_token_id`. c@seZdZdZeeedejdej dejdejde ej f ddZ eee dejdej dejdejd edejf d d Zd S) BeamScorerz Abstract base class for all beam scorers that are used for [`~PreTrainedModel.beam_search`] and [`~PreTrainedModel.beam_sample`]. input_ids next_scores next_tokens next_indicesreturncKtdNzThis is an abstract method.NotImplementedError)selfrrrrkwargsrw/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/transformers/generation/beam_search.pyprocessas zBeamScorer.process max_lengthcKrrr)rrrrrrrrrrfinalizems zBeamScorer.finalizeN)__name__ __module__ __qualname____doc__rr PROCESS_INPUTS_DOCSTRINGtorch LongTensor FloatTensorrTensorrFINALIZE_INPUTS_DOCSTRINGintrrrrrr[s< rc@sleZdZdZ     d$dededejd eed ee e e fd eed eed eefddZ e de fddZ     d%dejdejdejdejdee eejfdee eeeejfdeejdeedeedee ejffddZ    d&dejdejd ejd!ejd edee eejfdee eeeejfdeejdeedeejfd"d#ZdS)'BeamSearchScorera [`BeamScorer`] implementing standard beam search decoding. Adapted in part from [Facebook's XLM beam search code](https://github.com/facebookresearch/XLM/blob/9e6f6814d17be4fe5b15f2e6c43eb2b2d76daeb4/src/model/transformer.py#L529). Reference for the diverse beam search algorithm and implementation [Ashwin Kalyan's DBS implementation](https://github.com/ashwinkalyan/dbs/blob/master/dbs/beam_utils.lua) Args: batch_size (`int`): Batch Size of `input_ids` for which standard beam search decoding is run in parallel. num_beams (`int`): Number of beams for beam search. device (`torch.device`): Defines the device type (*e.g.*, `"cpu"` or `"cuda"`) on which this instance of `BeamSearchScorer` will be allocated. length_penalty (`float`, *optional*, defaults to 1.0): Exponential penalty to the length that is used with beam-based generation. It is applied as an exponent to the sequence length, which in turn is used to divide the score of the sequence. Since the score is the log likelihood of the sequence (i.e. negative), `length_penalty` > 0.0 promotes longer sequences, while `length_penalty` < 0.0 encourages shorter sequences. do_early_stopping (`bool` or `str`, *optional*, defaults to `False`): Controls the stopping condition for beam-based methods, like beam-search. It accepts the following values: `True`, where the generation stops as soon as there are `num_beams` complete candidates; `False`, where an heuristic is applied and the generation stops when is it very unlikely to find better candidates; `"never"`, where the beam search procedure only stops when there cannot be better candidates (canonical beam search algorithm). num_beam_hyps_to_keep (`int`, *optional*, defaults to 1): The number of beam hypotheses that shall be returned upon calling [`~transformers.BeamSearchScorer.finalize`]. num_beam_groups (`int`, *optional*, defaults to 1): Number of groups to divide `num_beams` into in order to ensure diversity among different groups of beams. See [this paper](https://arxiv.org/pdf/1610.02424.pdf) for more details. max_length (`int`, *optional*): The maximum length of the sequence to be generated. ?Fr N batch_size num_beamsdevicelength_penaltydo_early_stoppingnum_beam_hyps_to_keepnum_beam_groupsrc s|_|_|_|_|_|_jj_d_fddt|jD_ t j ddt|jDt j jd_ t|trJ|dkrRtd|dt|tra||ksa||d krltd |d |d dS) NFc"g|] }tjjjdqS)r.r0early_stoppingr)BeamHypotheses group_sizer0r1.0_rrrr z-BeamSearchScorer.__init__..cSg|]}dqSFrr9rrrr=dtyper/r A`num_beams` has to be an integer strictly greater than 1, but is G. For `num_beams` == 1, one should make use of `greedy_search` instead.r`num_beam_groups` has to be an integer smaller or equal than `num_beams` and `num_beams` has to be divisible by `num_beam_groups`, but is  with `num_beams` being .)r.r/r0r1r2r3r8_is_initrange _beam_hypsr%tensorbool_done isinstancer* ValueError) rr-r.r/r0r1r2r3rrr<r__init__s6     zBeamSearchScorer.__init__rcC |jSNrNallrrrris_done zBeamSearchScorer.is_donerrrrr pad_token_id eos_token_id beam_indices group_indexdecoder_prompt_lenc  Cs|jdd} t|j|j} | |jd|jks<|jdkr-td|jdd|jdtd|jdd|jd|j} tj| |jf|j | d } tj| |jf|j | d }tj| |jf|j | d }|durzt |tj szt |t ru|g}t |}t| D]}||j|}|j|r|jt|j|krtd |jd |dus|durtd d| |ddf<|||ddf<d||ddf<q~d}tt||||||D]f\}\}}}||j|}|dur||vr||jk}|rq|dur||}||f}nd}|j|j||||| | d n|| ||f<||||f<||||f<|d7}||jkr:nq||jkrWtd|jd||d|d||d |j|pl|j|||| | |j|<q~t| d|d|ddS)Nr rA group beam size of 0 is used as the input, but a group beam size of is expected by the beam scorer.A beam size of * is used as the input, but a beam size of rB#Batch can only be done if at least  beams have been generatedMGenerated beams >= num_beams -> eos_token_id and pad_token have to be definedr[ generated_lenAt most tokens in can be equal to `eos_token_id: `. Make sure  are corrected.next_beam_scoresnext_beam_tokensnext_beam_indices)shapelenrKr3r8rPr/r%zerosrCrOr(r*rLrJrNr. enumeratezipitemaddclonerWmaxrview)rrrrrrYrZr[r\r]cur_lenr-r/rorprq batch_idxbatch_group_idxbeam_idxbeam_token_rank next_token next_score next_indexbatch_beam_idx&is_beam_token_worse_than_top_num_beams beam_indexrrrrs                 zBeamSearchScorer.processfinal_beam_scoresfinal_beam_tokensfinal_beam_indicesc &Cst|j|j} |durt|tjst|tr|g}t|}t|jD]=\} } |j | r.q$t |j D]-} | |j | }|| }||}|durN||nd}|j d| }| j||||dq3q$|| |j}g}g}tj| |j|jtjd}t | D]V}|j||j|d|j}dd|D}t|ddd }t |jD]0}|}|d }|d}|d }t|||j||<||||||||j|<qq| d} |durt| |n| }!|| |j|!}"t|d kr |d dur || |j|!}#nd}#| | kr'|dur"td |"||#dur1|#dtt||D]4\}\}$}%|$|"|d||f<|#durZt|%|#|dt|%f<|||!krk|d |"|||f<q8t|"||#d S)Nr^rgr/rCr cSsg|] }|jD]}|qqSr)beams)r:beam_hypbeamrrrr=ksz-BeamSearchScorer.finalize..cS|dSNrrxrrrlz+BeamSearchScorer.finalize..keyrr `pad_token_id` has to be defined sequencessequence_scoresr[)rsrKr3rOr%r(r*rLrurNrJr8rwrrrxnewr2rtr/float32sortedpopappendrzminrPfill_rvr)&rrrrrrrYrZr[r]r-r~rindex_per_groupr final_score final_tokensrrh sent_lengthsbest best_indices best_scoresibeam_hyps_in_batchcandidate_beams sorted_hypsjbest_hyp_tuple best_scorebest_hyp best_indexsent_lengths_max sent_max_lendecodedindiceshypobest_idxrrrr@sv              zBeamSearchScorer.finalizer,Fr r N)NNNrrNNNr)r r!r"r#r*r%r/rfloatr rMstrrQpropertyrWr&r'r(rrrrrrrrrr+{s+ 1    p   r+c@seZdZdZ     d1dededeed ejd e e d e e e e fd e ed e ede efddZede fddZddZddZ    d2dejdejdejdejdejde e eejfde e eeeejfd e ejd!e edeejfd"d#Z d3d$edejd%ejd&ejd'ejd(ejd)e fd*d+Z    d2dejd,ejd-ejd.ejdede e eejfde e eeeejfd e ejd!e edeejfd/d0ZdS)4ConstrainedBeamSearchScorera [`BeamScorer`] implementing constrained beam search decoding. Args: batch_size (`int`): Batch Size of `input_ids` for which standard beam search decoding is run in parallel. num_beams (`int`): Number of beams for beam search. constraints (`List[Constraint]`): A list of positive constraints represented as `Constraint` objects that must be fulfilled in the generation output. For more information, the documentation of [`Constraint`] should be read. device (`torch.device`): Defines the device type (*e.g.*, `"cpu"` or `"cuda"`) on which this instance of `BeamSearchScorer` will be allocated. length_penalty (`float`, *optional*, defaults to 1.0): Exponential penalty to the length that is used with beam-based generation. It is applied as an exponent to the sequence length, which in turn is used to divide the score of the sequence. Since the score is the log likelihood of the sequence (i.e. negative), `length_penalty` > 0.0 promotes longer sequences, while `length_penalty` < 0.0 encourages shorter sequences. do_early_stopping (`bool` or `str`, *optional*, defaults to `False`): Controls the stopping condition for beam-based methods, like beam-search. It accepts the following values: `True`, where the generation stops as soon as there are `num_beams` complete candidates; `False`, where an heuristic is applied and the generation stops when is it very unlikely to find better candidates; `"never"`, where the beam search procedure only stops when there cannot be better candidates (canonical beam search algorithm). num_beam_hyps_to_keep (`int`, *optional*, defaults to 1): The number of beam hypotheses that shall be returned upon calling [`~transformers.BeamSearchScorer.finalize`]. num_beam_groups (`int`, *optional*, defaults to 1): Number of groups to divide `num_beams` into in order to ensure diversity among different groups of beams. See [this paper](https://arxiv.org/pdf/1610.02424.pdf) for more details. max_length (`int`, *optional*): The maximum length of the sequence to be generated. r,Fr Nr-r. constraintsr/r0r1r2r3rc s|_|_|_|_|_|_jj_|_d_fddt |D_ t j ddt |Dt j jd_t|trG|dkrOtd|dt|tr^||ks^||d kritd |d |d dS) NFcr4r5)r7r.r0r1r9r<rrr=r>z8ConstrainedBeamSearchScorer.__init__..cSr?r@rr9rrrr=rArBr rDrErrFrGrH)r.r/r0r1r2r3r8rrIrJrKr%rLrMrNrOr*rP) rr-r.rr/r0r1r2r3rrr<rrQs4  $  z$ConstrainedBeamSearchScorer.__init__rcCrRrSrTrVrrrrWrXz#ConstrainedBeamSearchScorer.is_donecsfddt|DS)Ncs g|] }tddjDqS)cSg|]}|qSr)copy)r: constraintrrrr=zQConstrainedBeamSearchScorer.make_constraint_states...)rrr9rVrrr=s zFConstrainedBeamSearchScorer.make_constraint_states..)rJ)rnrrVrmake_constraint_statessz2ConstrainedBeamSearchScorer.make_constraint_statescCs|dd}|||jS)Nr r)rreset completed)rsequence new_staterrrcheck_completes_constraintss z7ConstrainedBeamSearchScorer.check_completes_constraintsrrrrrscores_for_all_vocabrYrZr[r]c  Cs<|jdd} t|j} | |jd|jks9|jdkr*td|jdd|jdtd|jdd|jd|j} tj| |jf|j | d } tj| |jf|j | d }tj| |jf|j | d }|d urwt |tj swt |t rr|g}t |}t|jD]\}}|j|r|jt|krtd |jd |d us|d urtd d| |d d f<|||d d f<d||d d f<q|d}tt||||||D]o\}\}}}||j|}|d ur||vr||jk}|rq|||}|r|d ur||}||f}nd }|j||||| | dn|| ||f<||||f<||||f<|d7}||jkr:nq||||| |||||\}}}|| |<|||<|||<||jkrvtd|jd||d|d||d |j|p|||| | |j|<q|t| d|d|ddS)al Args: input_ids (`torch.LongTensor` of shape `(batch_size * num_beams, sequence_length)`): Indices of input sequence tokens in the vocabulary. Indices can be obtained using any class inheriting from [`PreTrainedTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) next_scores (`torch.FloatTensor` of shape `(batch_size, 2 * num_beams)`): Current scores of the top `2 * num_beams` non-finished beam hypotheses. next_tokens (`torch.LongTensor` of shape `(batch_size, 2 * num_beams)`): `input_ids` of the tokens corresponding to the top `2 * num_beams` non-finished beam hypotheses. next_indices (`torch.LongTensor` of shape `(batch_size, 2 * num_beams)`): Beam indices indicating to which beam hypothesis the `next_tokens` correspond. scores_for_all_vocab (`torch.FloatTensor` of shape `(batch_size * num_beams, sequence_length)`): The scores of all tokens in the vocabulary for each of the beam hypotheses. pad_token_id (`int`, *optional*): The id of the *padding* token. eos_token_id (`Union[int, List[int]]`, *optional*): The id of the *end-of-sequence* token. Optionally, use a list to set multiple *end-of-sequence* tokens. beam_indices (`torch.LongTensor`, *optional*): Beam indices indicating to which beam hypothesis each token correspond. decoder_prompt_len (`int`, *optional*): The length of prompt that is included in the input to decoder. Return: `UserDict`: A dictionary composed of the fields as defined above: - **next_beam_scores** (`torch.FloatTensor` of shape `(batch_size * num_beams)`) -- Updated scores of all non-finished beams. - **next_beam_tokens** (`torch.FloatTensor` of shape `(batch_size * num_beams)`) -- Next tokens to be added to the non-finished beam_hypotheses. - **next_beam_indices** (`torch.FloatTensor` of shape `(batch_size * num_beams)`) -- Beam indices indicating to which beam the next tokens shall be added. r^r rr_r`rarbrcrBNrdrerfrgrirjrkrlrmrn)rrrsrKr8r3rPr/r%rtrCrOr(r*rLrurNr.rvrwrtolistrxrystep_sentence_constraintrWrzrr{)rrrrrrrYrZr[r]r|r-r/rorprqr}rrrrrrrrcompletes_constraintr new_scores new_tokens new_indicesrrrrs4                 z#ConstrainedBeamSearchScorer.processr} vocab_scoressent_beam_scoressent_beam_tokenssent_beam_indices push_progressc+Csb|d}|j} ||} ||} |||d|} } || | }|| | }tj|||dfdd}|ggggd}t|D]\}}| |}|||| |}|||j stj | tj | d}|D]L}|j dd}||t||dfd}||d vr|d ||d | ||d ||d ||||d |qqqE|rt||d\}}t||dfd}| |}|}||||d vr|d ||d ||d ||d ||d |qEt|d dkrtj |d | d}t|d | }t|d | }| |d }t||fd} t||fd}!tj dd|D| d}"|"d|!}#|#jddj}$|"|$}%d}&|%d}'g}(|%D]})|)|'kr~|&d7}&nd}&|)}'|(|&qrt tj|(dd}*|$|*d|}$|!|$}| |$}t||f|$}|||fS)Nrr r^)dim)new_seqs new_statesrrrrBT)statefulrrrrr)r/cSrr)get_bank)r:onerrrr=rzHConstrainedBeamSearchScorer.step_sentence_constraint..d) descending mergesort)kind)sizer/rr%cat unsqueezerrurrrLadvancelongrrxrtakerzrsstacktosortrnpargsort)+rr}rrrrrrorig_lenr/topk_contraint_statesadvance_constraint_statessidxeidxthis_batch_input_idsthis_batch_token_scoresfull_hypotheses track_newseq_idxpre_seq topk_state advance_stateadvance_tokens advance_tokenr advance_seq new_score new_tokenrrr all_states all_tokens all_scores all_bankszippedr sorted_bankscountercur_bank incrementsbank rearrangersrrrrs              z4ConstrainedBeamSearchScorer.step_sentence_constraintrrrc &CsZt|j} |durt|tjst|tr|g}t|}t|jD]\} } |j| r+q!g} t |j D];}| |j |}|| }||}| | }|rm|durV||nd}|jd| }| j||||d| |q2t| |jkrt |j D]/}|| vr| |j |}|| }||}|jd| }| j|||dt| |jkrnqzq!|| |j}g}g}tj| |j|jtjd}t|jD]C\}} t| jddd}t |jD]0}|}|d}|d }|d }t|||j||<||||||||j|<qq| d } |durt| |n| }!|| |j|!}"t|dkrB|ddurB|| |j|!}#nd}#| | kr_|durZtd |"||#duri|#dtt||D]4\}\}$}%|$|"|d||f<|#durt|%|#|dt|%f<|||!kr|d|"|||f<qpt|"||#d S) Nr^rg)rhrcSrrrrrrrrjrz6ConstrainedBeamSearchScorer.finalize..rrr r rr)rsrKrOr%r(r*rLrurNrJr.rwrrrrrxrr2rrtr/rrrrrzrrPrrvr)&rrrrrrrYrZr[r]r-r}r ids_collectbeam_idrrrrrrhrrrrrrrrrrrrrrrrrrrrr-s              z$ConstrainedBeamSearchScorer.finalizerrr@)r r!r"r#r*rr r%r/rrr rMrrQrrWrrr&r'r(rrrrrrrrrs*  -    (    rc @seZdZddedededeefddZdd Z  dd e j d ed ee j d eefddZ ddededeedefddZ dS)r7Nr.r0r6rcCsJ||_||_||_||_g|_d|_t|jts!|jdur#tddSdS)z7 Initialize n-best list of hypotheses. geANzWhen `do_early_stopping` is set to a string, `max_length` must be defined. Ensure it is passed to the BeamScorer class instance at initialization time.) r0r6rr.r worst_scorerOrMrP)rr.r0r6rrrrrQszBeamHypotheses.__init__cCs t|jS)z3 Number of hypotheses in the list. )rsrrVrrr__len__s zBeamHypotheses.__len__hyp sum_logprobsr[rhcCs|dur |||j}n ||jd|j}t||jks"||jkrX|j|||ft||jkrOtddt|jD}|j|dd=|dd|_dSt ||j|_dSdS)z3 Add a new hypothesis to the list. Nr^cSsg|] \}\}}}||fqSrr)r:idxsr;rrrr=sz&BeamHypotheses.add..rr ) r0rrrsr.r rrrrur)rr r r[rhscoresorted_next_scoresrrrrxs zBeamHypotheses.addrbest_sum_logprobsr|r]rcCst||jkr dS|jdurdS|jdur%||||j}|j|k}|S|jdkr>|j|kr3td||j||j}n ||||j}|j|k}|S)z If there are enough hypotheses and that none of the hypotheses being generated can become better than the worst one in the heap, then we are done with this sentence. FTgz3max_length is not larger than decoder prompt length)rsr.r6r0r rrP)rrr|r]highest_attainable_scoreretrrrrWs       zBeamHypotheses.is_donerS)NN)r) r r!r"r*rrMrrQr r%r&rxrWrrrrr7s   $r7)abcrr collectionsrtypingrrrrr numpyrr%utilsr beam_constraintsr rr$r)rr+rr7rrrrs&  $ *