o  ha@sddlZddlZddlZddlmZddlmZmZmZm Z ddl m Z m Z e r9ddl Z ddlmZddlmZerFddlZdd lmZGd d d ejZGd d d Zee ddGddde ZdS)N)Dict) ModelOutputadd_end_docstringsis_tf_availableis_torch_available)Pipelinebuild_pipeline_init_args)!MODEL_FOR_CAUSAL_LM_MAPPING_NAMES) KeyDataset)$TF_MODEL_FOR_CAUSAL_LM_MAPPING_NAMESc@seZdZdZdZdZdS) ReturnTyperrrN)__name__ __module__ __qualname__TENSORSNEW_TEXT FULL_TEXTrrz/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/transformers/pipelines/text_generation.pyrsrc@seZdZdZdefddZdS)Chata This class is intended to just be used internally in this pipeline and not exposed to users. We convert chats to this format because the rest of the pipeline code tends to assume that lists of messages are actually a batch of samples rather than messages in the same conversation.messagescCs,|D]}d|vr d|vstdq||_dS)NrolecontentzQWhen passing chat dicts as input, each dict must have a 'role' and 'content' key.) ValueErrorr)selfrmessagerrr__init__!s  z Chat.__init__N)rrr__doc__rrrrrrrsrT) has_tokenizercseZdZdZdZfddZ           dddZfdd Zfd d Z      dd dZ ddZ e j ddfddZ ZS)TextGenerationPipelinea Language generation pipeline using any `ModelWithLMHead`. This pipeline predicts the words that will follow a specified text prompt. When the underlying model is a conversational model, it can also accept one or more chats, in which case the pipeline will operate in chat mode and will continue the chat(s) by adding its response(s). Each chat takes the form of a list of dicts, where each dict contains "role" and "content" keys. Examples: ```python >>> from transformers import pipeline >>> generator = pipeline(model="openai-community/gpt2") >>> generator("I can't believe you did such a ", do_sample=False) [{'generated_text': "I can't believe you did such a icky thing to me. I'm so sorry. I'm so sorry. I'm so sorry. I'm so sorry. I'm so sorry. I'm so sorry. I'm so sorry. I"}] >>> # These parameters will return suggestions, and only the newly created text making it easier for prompting suggestions. >>> outputs = generator("My tart needs some", num_return_sequences=4, return_full_text=False) ``` ```python >>> from transformers import pipeline >>> generator = pipeline(model="HuggingFaceH4/zephyr-7b-beta") >>> # Zephyr-beta is a conversational model, so let's pass it a chat instead of a single string >>> generator([{"role": "user", "content": "What is the capital of France? Answer in one word."}], do_sample=False, max_new_tokens=2) [{'generated_text': [{'role': 'user', 'content': 'What is the capital of France? Answer in one word.'}, {'role': 'assistant', 'content': 'Paris'}]}] ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial). You can pass text generation parameters to this pipeline to control stopping criteria, decoding strategy, and more. Learn more about text generation parameters in [Text generation strategies](../generation_strategies) and [Text generation](text_generation). This language generation pipeline can currently be loaded from [`pipeline`] using the following task identifier: `"text-generation"`. The models that this pipeline can use are models that have been trained with an autoregressive language modeling objective. See the list of available [text completion models](https://huggingface.co/models?filter=text-generation) and the list of [conversational models](https://huggingface.co/models?other=conversational) on [huggingface.co/models]. a In 1991, the remains of Russian Tsar Nicholas II and his family (except for Alexei and Maria) are discovered. The voice of Nicholas's young son, Tsarevich Alexei Nikolaevich, narrates the remainder of the story. 1883 Western Siberia, a young Grigori Rasputin is asked by his father and a group of men to perform magic. Rasputin has a vision and denounces one of the men as a horse thief. Although his father initially slaps him for making such an accusation, Rasputin watches as the man is chased outside and beaten. Twenty years later, Rasputin sees a vision of the Virgin Mary, prompting him to become a priest. Rasputin quickly becomes famous, with people, even a bishop, begging for his blessing. cstj|i|||jdkrtntd|jvrVd}|jdur$|j}|dur2|jj j dvr2|j }|durX|j dd|i|j \}}}i|j||_i|j ||_ dSdSdS)Ntfprefix)XLNetLMHeadModelTransfoXLLMHeadModelTFXLNetLMHeadModelTFTransfoXLLMHeadModelr)superrcheck_model_type frameworkr r _preprocess_paramsr#model __class__r XL_PREFIX_sanitize_parameters_forward_params)rargskwargsr#preprocess_paramsforward_params_r-rrrbs   zTextGenerationPipeline.__init__Nc Ksi} d}d| vr| d}| d<d| vr| d| d<| dur$| | d<| dur0| | d<| | d<|dur8|| d<|rM|j|d||jd}|djd | d <|dur`|d vr\t|d || d <| durh| | d<| | | }i}|dur|dur|durtd|durtd|rtjntj}|dur|dur|durtdtj }|dur||d<|dur||d<| dur| |d<|dur|jj |dd}|| d<|j dur|j |d<|j dur|j|d<|j |d<| ||fS)NFadd_special_tokenspadding truncation max_lengthr#)r8r7return_tensors input_ids prefix_length>holezT is not a valid value for `handle_long_generation` parameter expected [None, 'hole']handle_long_generationcontinue_final_messagez;`return_text` is mutually exclusive with `return_full_text`z>`return_full_text` is mutually exclusive with `return_tensors`z9`return_text` is mutually exclusive with `return_tensors` return_typeclean_up_tokenization_spaces)r7 eos_token_idassistant_model tokenizerassistant_tokenizer) poprFr*shaperupdaterrrrencoderErG)rreturn_full_textr; return_textrBrCr#r@ stop_sequencer9r:rAgenerate_kwargsr3r7 prefix_inputsr4postprocess_paramsstop_sequence_idsrrrr/}sl        z+TextGenerationPipeline._sanitize_parameterscs.|jjjdvr|dditj|i|S)z. Parse arguments and tokenize )r%add_space_before_punct_symbolT)r,r-rrJr(_parse_and_tokenize)rr1r2r6rrrTsz*TextGenerationPipeline._parse_and_tokenizec st|tr tttjtfntttjfrmt|tjr-t|\}}dd|Dt |}}n|d}t|ttt frmt|t rJt j t |fi|Sdd|D}t|tjrat j |fi|St j t|fi|St j |fi|S)a( Complete the prompt(s) given as inputs. Args: text_inputs (`str`, `List[str]`, List[Dict[str, str]], or `List[List[Dict[str, str]]]`): One or several prompts (or one list of prompts) to complete. If strings or a list of string are passed, this pipeline will continue each prompt. Alternatively, a "chat", in the form of a list of dicts with "role" and "content" keys, can be passed, or a list of such chats. When chats are passed, the model's chat template will be used to format them before passing them to the model. return_tensors (`bool`, *optional*, defaults to `False`): Returns the tensors of predictions (as token indices) in the outputs. If set to `True`, the decoded text is not returned. return_text (`bool`, *optional*): Returns the decoded texts in the outputs. return_full_text (`bool`, *optional*, defaults to `True`): If set to `False` only added text is returned, otherwise the full text is returned. Cannot be specified at the same time as `return_text`. clean_up_tokenization_spaces (`bool`, *optional*, defaults to `True`): Whether or not to clean up the potential extra spaces in the text output. continue_final_message( `bool`, *optional*): This indicates that you want the model to continue the last message in the input chat rather than starting a new one, allowing you to "prefill" its response. By default this is `True` when the final message in the input chat has the `assistant` role and `False` otherwise, but you can manually override that behaviour by setting this flag. prefix (`str`, *optional*): Prefix added to prompt. handle_long_generation (`str`, *optional*): By default, this pipelines does not handle long generation (ones that exceed in one form or the other the model maximum length). There is no perfect way to address this (more info :https://github.com/huggingface/transformers/issues/14033#issuecomment-948385227). This provides common strategies to work around that problem depending on your use case. - `None` : default strategy where nothing in particular happens - `"hole"`: Truncates left of input, and leaves a gap wide enough to let generation happen (might truncate a lot of the prompt and not suitable when generation exceed the model capacity) generate_kwargs (`dict`, *optional*): Additional keyword arguments to pass along to the generate method of the model (see the generate method corresponding to your framework [here](./text_generation)). Return: A list or a list of lists of `dict`: Returns one of the following dictionaries (cannot return a combination of both `generated_text` and `generated_token_ids`): - **generated_text** (`str`, present when `return_text=True`) -- The generated text. - **generated_token_ids** (`torch.Tensor` or `tf.Tensor`, present when `return_tensors=True`) -- The token ids of the generated text. css|]}|VqdSNr).0xrrr sz2TextGenerationPipeline.__call__..rcss|]}t|VqdSrU)r)rVchatrrrrXs) isinstancerlisttupletypes GeneratorTyper itertoolsteenextdictr(__call__r)r text_inputsr2r5 first_itemchatsr6rrrcs$/    zTextGenerationPipeline.__call__c KsZ||||d} dd| D} t|tr;| dd|dur(|jdddk}|jj|jf| |d|jd | } n|j||fd |ji| } || d <|d kr| d jd} d| vra| d} n| d|j j | } | dkrst d| | |jj kr|jj | }|dkrt d| d dd| df| d <d| vr| ddd| df| d<| S)N)r7r9r8r:cSsi|] \}}|dur||qSrUr)rVkeyvaluerrr 4z5TextGenerationPipeline.preprocess..r7r=r assistantT)add_generation_promptrA return_dictr;r; prompt_textr?r<max_new_tokensr:rz0We cannot infer how many new tokens are expectedziWe cannot use `hole` to handle this generation the number of desired tokens exceeds the models max lengthattention_mask)itemsrZrrHrrFapply_chat_templater*rIgetgeneration_configr:rmodel_max_length)rror#r@r7r9r8r:rArOtokenizer_kwargsinputscur_len new_tokens keep_lengthrrr preprocess!sN     z!TextGenerationPipeline.preprocesscKs|d}|dd}|jddkrd}d}d}n|jd}|d}|dd}|dkrmd|vp:d|vo:|djdu}|sP|d pE|jj|d <|d |7<d |vp^d|vo^|djdu} | smd |vrm|d |7<d|vrv|j|d<|jjd||d |} t | t r1| j } d d| D} | jd} |j dkr| D]>\}}t |tjr|jd| kr|j|| |g|jddR| |<t |trt|d| krt|dd}|| |<qnS|j dkr0| D]C\}}t |tjr|jd| krt||| |g|jddR| |<t |tr/t|d| kr/t|dd}|| |<qn| } i} | jd} |j dkrR| j|| |g| jddR} n|j dkrjt| || |g| jddR} | ||d}|| |S)Nr<rqrrror>rprur:min_new_tokens min_length)r<rqcSsi|] \}}|dkr||qS) sequencesr)rVkvrrrrjrkz3TextGenerationPipeline._forward..ptr")generated_sequencer<ror)rtrIrHrprur:r}r,generaterZrrrrr*torchTensorreshaper\lenstackswapaxesr"rJ)r model_inputsrOr<rqin_bror>has_max_new_tokenshas_min_new_tokensoutputr other_outputsout_brhri model_outputsrrr_forward_sx           & (  $ $ zTextGenerationPipeline._forwardTcCs|dd}|d}|d}|}g}|di} i} | rk|jdkrF| D]\} } t| tjrD| jdt |krD| | | <q)n%|jdkrk| D]\} } t| t jrj| jdt |krj| | | <qOt |D]\} }|t j kr}d|i}n|t jt jhvr|jj|d |d }|durd}n t |jj|dd |d }||d}|t jkrt|tr||}n;t|tr|dur|jd d d k}|rt|jdd |jd d |jd d|dg}n t|jd |dg}d|i}| D] \}}|| ||<q||qo|S)Nrrr<roadditional_outputsrr"generated_token_idsT)skip_special_tokensrCr=rrlr)rrgenerated_text)numpytolistrtr*rrrZrrrIrr" enumeraterrrrrFdecodestrrrr[append)rrrBrCrArr<rorecordsr splitted_keysrridxsequencerecordtext prompt_lengthall_textrhvaluesrrr postprocesssp              z"TextGenerationPipeline.postprocess) NNNNNNNNNNN)rgNNNNNN)rrrrr.rr/rTrcr|rrrr __classcell__rrr6rr!(s>.  T I >Lr!)enumr_r]typingrutilsrrrrbaser r rmodels.auto.modeling_autor pt_utilsr tensorflowr"models.auto.modeling_tf_autor Enumrrr!rrrrs"