o  h @sdZddlZddlZddlZddlZddlZddlZddlZddlm Z ddlm Z m Z m Z m Z mZmZmZddlZddlZddlmZddlmZddlmZmZmZmZmZmZmZeredd lm Z dd l!m"Z"m#Z#m$Z$m%Z%m&Z&dd l'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3e34e5Z6e/e e7j8Z9d d ddZ:ej;dkrejGdddeddZ?GdddeddZ@GdddeddZAGddde=e>e?e@eAddZBGdd d eddZCGd!d"d"eddZDGd#d$d$eDeCddZEGd%d&d&e=e>e?e@eAeEZFGd'd(d(e)ZGd)d*ZHe.eGjIeG_IeGjIjdur@eGjIjjJd+d,d-d.eGjI_dSdS)/z8 Processing saving/loading class for common processors. N)Path)AnyCallableDictListOptional TypedDictUnion) load_audio)custom_object_save)ChannelDimension ImageInput VideoInputis_valid_imageis_vision_available load_image load_video)PILImageResampling)PaddingStrategyPreTokenizedInputPreTrainedTokenizerBase TextInputTruncationStrategy) PROCESSOR_NAMEPushToHubMixin TensorTypeadd_model_info_to_auto_map"add_model_info_to_custom_pipelines cached_file copy_funcdirect_transformers_import download_urlis_offline_mode is_remote_urlloggingrFeatureExtractionMixinImageProcessingMixin) AutoTokenizerAutoFeatureExtractorAutoImageProcessor) c@s.eZdZUdZeeeeeeeefe d<eeeeeeefe d<eeeeeeeefe d<ee e d<ee e e fe d<ee e e fe d<eee d<eee d <ee e d <eee d <ee e d <ee e d <ee e d<ee e d<ee e d<ee e d<ee e d<ee e d<dS) TextKwargsa Keyword arguments for text processing. For extended documentation, check out tokenization_utils_base methods and docstrings associated. Attributes: add_special_tokens (`bool`, *optional*) Whether or not to add special tokens when encoding the sequences. padding (`bool`, `str` or [`~utils.PaddingStrategy`], *optional*) Activates and controls padding. truncation (`bool`, `str` or [`~tokenization_utils_base.TruncationStrategy`], *optional*): Activates and controls truncation. max_length (`int`, *optional*): Controls the maximum length to use by one of the truncation/padding parameters. stride (`int`, *optional*): If set, the overflowing tokens will contain some tokens from the end of the truncated sequence. is_split_into_words (`bool`, *optional*): Whether or not the input is already pre-tokenized. pad_to_multiple_of (`int`, *optional*): If set, will pad the sequence to a multiple of the provided value. return_token_type_ids (`bool`, *optional*): Whether to return token type IDs. return_attention_mask (`bool`, *optional*): Whether to return the attention mask. return_overflowing_tokens (`bool`, *optional*): Whether or not to return overflowing token sequences. return_special_tokens_mask (`bool`, *optional*): Whether or not to return special tokens mask information. return_offsets_mapping (`bool`, *optional*): Whether or not to return `(char_start, char_end)` for each token. return_length (`bool`, *optional*): Whether or not to return the lengths of the encoded inputs. verbose (`bool`, *optional*): Whether or not to print more information and warnings. padding_side (`str`, *optional*): The side on which padding will be applied. text_pair text_targettext_pair_targetadd_special_tokenspadding truncation max_lengthstrideis_split_into_wordspad_to_multiple_ofreturn_token_type_idsreturn_attention_maskreturn_overflowing_tokensreturn_special_tokens_maskreturn_offsets_mapping return_lengthverbose padding_sideN)__name__ __module__ __qualname____doc__rr rrlist__annotations__boolstrrrintrIrIq/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/transformers/processing_utils.pyr-Xs(  %             r-F)totalc@seZdZUdZeeed<eeee fed<ee ed<eeee fed<ee de fed<eeed<ee ed <eeed <ee e e e fed <ee e e e fed <eeed <eeee fed<eeed<ee ed<ee ee fed<eeed<dS) ImagesKwargsa Keyword arguments for image processing. For extended documentation, check the appropriate ImageProcessor class methods and docstrings. Attributes: do_resize (`bool`, *optional*): Whether to resize the image. size (`Dict[str, int]`, *optional*): Resize the shorter side of the input to `size["shortest_edge"]`. size_divisor (`int`, *optional*): The size by which to make sure both the height and width can be divided. crop_size (`Dict[str, int]`, *optional*): Desired output size when applying center-cropping. resample (`PILImageResampling`, *optional*): Resampling filter to use if resizing the image. do_rescale (`bool`, *optional*): Whether to rescale the image by the specified scale `rescale_factor`. rescale_factor (`int` or `float`, *optional*): Scale factor to use if rescaling the image. do_normalize (`bool`, *optional*): Whether to normalize the image. image_mean (`float` or `List[float]`, *optional*): Mean to use if normalizing the image. image_std (`float` or `List[float]`, *optional*): Standard deviation to use if normalizing the image. do_pad (`bool`, *optional*): Whether to pad the image to the `(max_height, max_width)` of the images in the batch. pad_size (`Dict[str, int]`, *optional*): The size `{"height": int, "width" int}` to pad the images to. do_center_crop (`bool`, *optional*): Whether to center crop the image. data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the output image. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. device (`str`, *optional*): The device to use for processing (e.g. "cpu", "cuda"), only relevant for fast image processing. do_resizesize size_divisor crop_sizerresample do_rescalerescale_factor do_normalize image_mean image_stddo_padpad_sizedo_center_crop data_formatinput_data_formatdeviceN)r@rArBrCrrFrEdictrGrHr floatrDr rIrIrIrJrLs$  '       rLc@seZdZUdZeeed<eeee fed<ee ed<eded<eeed<ee ed<eeed <ee e e e fed <ee e e e fed <eeed <eeed <ee ed<ee ee fed<dS) VideosKwargsa Keyword arguments for video processing. Attributes: do_resize (`bool`): Whether to resize the image. size (`Dict[str, int]`, *optional*): Resize the shorter side of the input to `size["shortest_edge"]`. size_divisor (`int`, *optional*): The size by which to make sure both the height and width can be divided. resample (`PILImageResampling`, *optional*): Resampling filter to use if resizing the image. do_rescale (`bool`, *optional*): Whether to rescale the image by the specified scale `rescale_factor`. rescale_factor (`int` or `float`, *optional*): Scale factor to use if rescaling the image. do_normalize (`bool`, *optional*): Whether to normalize the image. image_mean (`float` or `List[float]`, *optional*): Mean to use if normalizing the image. image_std (`float` or `List[float]`, *optional*): Standard deviation to use if normalizing the image. do_pad (`bool`, *optional*): Whether to pad the image to the `(max_height, max_width)` of the images in the batch. do_center_crop (`bool`, *optional*): Whether to center crop the image. data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the output image. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. rMrNrOrrQrRrSrTrUrVrWrYrZr[N)r@rArBrCrrFrEr]rGrHr^r rDr rIrIrIrJr_s          r_c@seZdZUdZeeed<eedee edeee fed<eee e e fed<eeed<ee ed<eeed<ee ed <d S) AudioKwargsa Keyword arguments for audio processing. Attributes: sampling_rate (`int`, *optional*): The sampling rate at which the `raw_speech` input was sampled. raw_speech (`np.ndarray`, `List[float]`, `List[np.ndarray]`, `List[List[float]]`): The sequence or batch of sequences to be padded. Each sequence can be a numpy array, a list of float values, a list of numpy arrays or a list of list of float values. Must be mono channel audio, not stereo, i.e. single float per timestep. padding (`bool`, `str` or [`~utils.PaddingStrategy`], *optional*): Select a strategy to pad the returned sequences (according to the model's padding side and padding index) among: - `True` or `'longest'`: Pad to the longest sequence in the batch (or no padding if only a single sequence if provided). - `'max_length'`: Pad to a maximum length specified with the argument `max_length` or to the maximum acceptable input length for the model if that argument is not provided. - `False` or `'do_not_pad'` max_length (`int`, *optional*): Maximum length of the returned list and optionally padding length (see above). truncation (`bool`, *optional*): Activates truncation to cut input sequences longer than *max_length* to *max_length*. pad_to_multiple_of (`int`, *optional*): If set, will pad the sequence to a multiple of the provided value. return_attention_mask (`bool`, *optional*): Whether or not [`~ASTFeatureExtractor.__call__`] should return `attention_mask`. sampling_ratez np.ndarray raw_speechr2r4r3r7r9N) r@rArBrCrrHrEr rDr^rFrGrrIrIrIrJr`s  (   r`c@s"eZdZUeeeefed<dS) CommonKwargsreturn_tensorsN)r@rArBrr rGrrErIrIrIrJrc#s rcc@sleZdZUdZiejZeed<iejZeed<ie jZ e ed<ie jZ e ed<ie jZe ed<dS)ProcessingKwargsa' Base class for kwargs passing to processors. A model should have its own `ModelProcessorKwargs` class that inherits from `ProcessingKwargs` to provide: 1) Additional typed keys and that this model requires to process inputs. 2) Default values for existing keys under a `_defaults` attribute. New keys have to be defined as follows to ensure type hinting is done correctly. ```python # adding a new image kwarg for this model class ModelImagesKwargs(ImagesKwargs, total=False): new_image_kwarg: Optional[bool] class ModelProcessorKwargs(ProcessingKwargs, total=False): images_kwargs: ModelImagesKwargs _defaults = { "images_kwargs: { "new_image_kwarg": False, } "text_kwargs": { "padding": "max_length", }, } ``` For Python 3.8 compatibility, when inheriting from this class and overriding one of the kwargs, you need to manually update the __annotations__ dictionary. This can be done as follows: ```python class CustomProcessorKwargs(ProcessingKwargs, total=False): images_kwargs: CustomImagesKwargs CustomProcessorKwargs.__annotations__["images_kwargs"] = CustomImagesKwargs # python 3.8 compatibility ```python common_kwargs text_kwargs images_kwargs videos_kwargs audio_kwargsN)r@rArBrCrcrErfr-rgrLrhr_rir`rjrIrIrIrJre's" %    rec@sreZdZUdZdZeeeed<dZ eeee e fed<dZ ee ed<dZ ee ed<dZee ed<dS) TokenizerChatTemplateKwargsaU Keyword arguments for tokenizer's `apply_chat_template`, when it is called from within a processor. tools (`List[Dict]`, *optional*): A list of tools (callable functions) that will be accessible to the model. If the template does not support function calling, this argument will have no effect. Each tool should be passed as a JSON Schema, giving the name, description and argument types for the tool. See our [chat templating guide](https://huggingface.co/docs/transformers/main/en/chat_templating#automated-function-conversion-for-tool-use) for more information. documents (`List[Dict[str, str]]`, *optional*): A list of dicts representing documents that will be accessible to the model if it is performing RAG (retrieval-augmented generation). If the template does not support RAG, this argument will have no effect. We recommend that each document should be a dict containing "title" and "text" keys. Please see the RAG section of the [chat templating guide](https://huggingface.co/docs/transformers/main/en/chat_templating#arguments-for-RAG) for examples of passing documents with chat templates. add_generation_prompt (bool, *optional*): If this is set, a prompt with the token(s) that indicate the start of an assistant message will be appended to the formatted output. This is useful when you want to generate a response from the model. Note that this argument will be passed to the chat template, and so it must be supported in the template for this argument to have any effect. continue_final_message (bool, *optional*): If this is set, the chat will be formatted so that the final message in the chat is open-ended, without any EOS tokens. The model will continue this message rather than starting a new one. This allows you to "prefill" part of the model's response for it. Cannot be used at the same time as `add_generation_prompt`. return_assistant_tokens_mask (`bool`, defaults to `False`): Whether to return a mask of the assistant generated tokens. For tokens generated by the assistant, the mask will contain 1. For user and system tokens, the mask will contain 0. This functionality is only available for chat templates that support it via the `{% generation %}` keyword. Ntools documentsFadd_generation_promptcontinue_final_messagereturn_assistant_tokens_mask)r@rArBrCrlrrDr]rErmrGrnrFrorprIrIrIrJrk^s rkc@sreZdZUdZdZeeed<dZee ed<dZ eeed<dZ eeed<dZ ee ed <d Zeeed <dS) ChatTemplateLoadKwargsaZ Keyword arguments used to load multimodal data in processor chat templates. num_frames (`int`, *optional*): Number of frames to sample uniformly. If not passed, the whole video is loaded. video_load_backend (`str`, *optional*, defaults to `"pyav"`): The backend to use when loading the video which will be used only when there are videos in the conversation. Can be any of ["decord", "pyav", "opencv", "torchvision"]. Defaults to "pyav" because it is the only backend that supports all types of sources to load from. video_fps (`int`, *optional*): Number of frames to sample per second. Should be passed only when `num_frames=None`. If not specified and `num_frames==None`, all frames are sampled. sample_indices_fn (`Callable`, *optional*): A callable function that will return indices at which the video should be sampled. If the video has to be loaded using by a different sampling technique than provided by `num_frames` or `fps` arguments, one should provide their own `sample_indices_fn`. If not provided, simple uniformt sampling with fps is performed, otherwise `sample_indices_fn` has priority over other args. The function expects at input the all args along with all kwargs passed to `load_video` and should output valid indices at which the video should be sampled. For example: def sample_indices_fn(num_frames, fps, metadata, **kwargs): # add you sampling logic here ... return np.linspace(start_idx, end_idx, num_frames, dtype=int) N num_framespyavvideo_load_backend video_fpsi>rasample_indices_fnFload_audio_from_video)r@rArBrCrrrrHrErtrGrurarvrrwrFrIrIrIrJrqs rqc@s2eZdZUdZdZeeed<dZeeed<dS)ProcessorChatTemplateKwargsa: Keyword arguments for processor's `apply_chat_template`. tokenize (`bool`, *optional*, defaults to `False`): Whether to tokenize the output or not. return_dict (`bool`, defaults to `False`): Whether to return a dictionary with named outputs. Has no effect if tokenize is `False`. Ftokenize return_dictN) r@rArBrCryrrFrErzrIrIrIrJrxs  rxc@s eZdZdS)AllKwargsForChatTemplateN)r@rArBrIrIrIrJr{sr{c@sZeZdZUdZddgZdgZgZeee d<dZ dZ dZ gZ eee d<dd Zd eeeffd d Zd efd dZdeeejffddZddZdDdefddZedeeejfd eeeefeeefffddZedeeeffddZ dEdedeed eeeffd d!Z e     "dFdeeejfd#eeeejfd$ed%ed&eeeefd'ef d(d)Z!edGd+d,Z"ed-d.Z#e$d/d0Z%e&d1d2Z'e$d3d4Z(d5d6Z)d7e*e*e+eefd8e*e,d9e*e-d:e*e*e+ee.fd;e/e0f de/e2d efd?d@Z3dHdBdCZ4dS)IProcessorMixinza This is a mixin used to provide saving/loading functionality for all processor classes. feature_extractor tokenizer chat_templateoptional_call_argsN valid_kwargsc sPjD] }t|||dq|D]}|jvr!td|dqt|jD]\}}||vr8td|d|||<q(t|tjkr]tdtjddjdt|d| D]D\}}t |d }t ||}t |trtfd d |D}n|}t ||std t|jd |d|dt||qadS)NzUnexpected keyword argument .z!Got multiple values for argument zThis processor requires z arguments: z, z. Got z arguments instead._classc3s"|] }|dur|VqdSNget_possibly_dynamic_module.0nselfrIrJ s z*ProcessorMixin.__init__..z Received a z for argument z, but a z was expected.)optional_attributessetattrpop attributes TypeErrorziplen ValueErrorjoinitemsgetattrAUTO_TO_BASE_CLASS_MAPPINGget isinstancetuplertyper@) rargskwargsoptional_attributekeyargattribute_name class_name proper_classrIrrJ__init__s8       zProcessorMixin.__init__returncstj}tj}|jfddDdg7fdd|D}jj |d<d|vr5|d=d|vr<|d=d |vrC|d =d |vrJ|d =d d|D}|S) z Serializes this instance to a Python dictionary. Returns: `Dict[str, Any]`: Dictionary of all the attributes that make up this processor instance. csg|] }|jjvr|qSrI) __class__r)rxrrIrJ z*ProcessorMixin.to_dict..auto_mapcsi|] \}}|vr||qSrIrIrkv) attrs_to_saverIrJ sz*ProcessorMixin.to_dict..processor_classr~image_processorr}rcSs,i|]\}}t|ts|jjdks||qS)BeamSearchDecoderCTC)rrrr@rrIrIrJrs ) copydeepcopy__dict__inspect signaturer parametersrrr@)routputsigrI)rrrJto_dicts&    zProcessorMixin.to_dictcCs|}tj|ddddS)z Serializes this instance to a JSON string. Returns: `str`: String containing all the attributes that make up this feature_extractor instance in JSON format. Tindent sort_keys )rjsondumps)r dictionaryrIrIrJto_json_stringszProcessorMixin.to_json_stringjson_file_pathcCsBt|ddd}||WddS1swYdS)z Save this instance to a JSON file. Args: json_file_path (`str` or `os.PathLike`): Path to the JSON file in which this processor instance's parameters will be saved. wutf-8encodingN)openwriter)rrwriterrIrIrJ to_json_file!s"zProcessorMixin.to_json_filecs:fddjD}d|}jjd|dS)Nc s&g|]}d|dtt|qS)z- z: )reprr)rnamerrIrJr-s&z+ProcessorMixin.__repr__..rz: z )rrrr@r)rattributes_reprrIrrJ__repr__,s zProcessorMixin.__repr__F push_to_hubc s|dd}|dur tdt|dddurtd||d<tj|dd|rJ|dd}|d |tj j d }j |fi|} |}j durlfd d jD}d d |D} | t|| djD]} t| } t| dr| jj| |qoj durjD]} t| } t| tr| jd=qtj |t} tj |d} tj |d}}jdur|ddrt| ddd}|jWdn1swYt !d| n2t"j#djidddd}t|ddd }||Wdn 1swYt !d|t$|%dhkr5&| t !d| |rEj'|||||ddt$|%dhkrQgS| gS) a Saves the attributes of this processor (feature extractor, tokenizer...) in the specified directory so that it can be reloaded using the [`~ProcessorMixin.from_pretrained`] method. This class method is simply calling [`~feature_extraction_utils.FeatureExtractionMixin.save_pretrained`] and [`~tokenization_utils_base.PreTrainedTokenizerBase.save_pretrained`]. Please refer to the docstrings of the methods above for more information. Args: save_directory (`str` or `os.PathLike`): Directory where the feature extractor JSON file and the tokenizer files will be saved (directory will be created if it does not exist). push_to_hub (`bool`, *optional*, defaults to `False`): Whether or not to push your model to the Hugging Face model hub after saving it. You can specify the repository you want to push to with `repo_id` (will default to the name of `save_directory` in your namespace). kwargs (`Dict[str, Any]`, *optional*): Additional key word arguments passed along to the [`~utils.PushToHubMixin.push_to_hub`] method. use_auth_tokenNrThe `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.tokenV`token` and `use_auth_token` are both specified. Please set only the argument `token`.T)exist_okcommit_messagerepo_idcsg|]}t|qSrI)r)rrrrIrJr`z2ProcessorMixin.save_pretrained..cSs g|] }t|tr |jn|qSrI)rr init_kwargs)rarIrIrJras )config_set_processor_classrchat_template.jinjachat_template.jsonsave_raw_chat_templateFrrrzchat template saved in rrrrrzprocessor saved in )rr)(rwarningswarn FutureWarningrrosmakedirssplitpathsep _create_repo_get_files_timestamps _auto_classrappendr rhasattrrrr@save_pretrainedrrrrrrrrrloggerinforrsetkeysr_upload_modified_files)rsave_directoryrrrrrfiles_timestampsattrsconfigsr attributeoutput_processor_fileoutput_raw_chat_template_fileoutput_chat_template_fileprocessor_dictrchat_template_json_stringrIrrJr1s~                 zProcessorMixin.save_pretrainedpretrained_model_name_or_pathcKs|dd}|dd}|dd}|dd}|dd}|dd}|d d} |d d } |d d} |d d} d| d} | durI| | d<trU|sUtdd}t|}tj|}tj|rltj|t }tj |r{|}d}d}d}ndt |r|}t |}d}d}nUt }d}d}z2t ||||||||| | | dd }t ||||||||| | | dd }t ||||||||| | | dd }Wntytytd|d|dt dw|durt|dd }|}Wdn1swY||d<n+|dur.t|dd }|}Wdn 1swYt|d}||d<|durEi}d|vrAd|di}||fSz"t|dd }|}Wdn 1s\wYt|}Wntjyxtd|dw|rtd|n td|d |d|vr|ddurtd!d|vr|d|d<|sd"|vrt|d"||d"<d#|vrt|d#||d#<||fS)$a From a `pretrained_model_name_or_path`, resolve to a dictionary of parameters, to be used for instantiating a processor of type [`~processing_utils.ProcessingMixin`] using `from_args_and_dict`. Parameters: pretrained_model_name_or_path (`str` or `os.PathLike`): The identifier of the pre-trained checkpoint from which we want the dictionary of parameters. subfolder (`str`, *optional*, defaults to `""`): In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can specify the folder name here. Returns: `Tuple[Dict, Dict]`: The dictionary(ies) that will be used to instantiate the processor object. cache_dirNforce_downloadFresume_downloadproxiesrlocal_files_onlyrevision subfolder_from_pipeline _from_auto processor) file_typefrom_auto_classusing_pipelinez+Offline mode: forcing local_files_only=TrueTrr) rrrrrr user_agentrr%_raise_exceptions_for_missing_entrieszCan't load processor for 'z'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'z2' is the correct path to a directory containing a z filerrrz"It looks like the config file at 'z' is not a valid JSON file.zloading configuration file z from cache at zChat templates should be in a 'chat_template.jinja' file but found key='chat_template' in the processor's config. Make sure to move your template to its own file.rcustom_pipelines)rr#rrrGrrisdirrrisfiler$r"rOSError ExceptionrreadrloadsJSONDecodeError warning_oncerr)clsrrrrrrrrrr from_pipelinerris_localprocessor_fileresolved_processor_fileresolved_chat_template_fileresolved_raw_chat_template_filechat_template_fileraw_chat_template_filereaderrtextrrIrIrJget_processor_dicts                             z!ProcessorMixin.get_processor_dictrcKs|}|dd}d|vr|d=d|vr|d=|j||jd}||i|}t|D]}t||r=t||||q-||t d||rQ||fS|S)a Instantiates a type of [`~processing_utils.ProcessingMixin`] from a Python dictionary of parameters. Args: processor_dict (`Dict[str, Any]`): Dictionary that will be used to instantiate the processor object. Such a dictionary can be retrieved from a pretrained checkpoint by leveraging the [`~processing_utils.ProcessingMixin.to_dict`] method. kwargs (`Dict[str, Any]`): Additional parameters from which to initialize the processor object. Returns: [`~processing_utils.ProcessingMixin`]: The processor object instantiated from those parameters. return_unused_kwargsFrr)processor_configrz Processor ) rrvalidate_init_kwargsrrrrrupdaterr)rrrrr' unused_kwargsr rrIrIrJfrom_args_and_dictOs"   z!ProcessorMixin.from_args_and_dictModelProcessorKwargstokenizer_init_kwargsc s.iiiiid}iiiiidhd}t}D]2}|j|i|<|j|jD]}||vrJt|j|r@t|j|n||} | ||<q.q| t|t|} |D]N}|j|jD]C}||vr|| |d} | dkr|| vrt d|d|dn ||vr||d} nd} t | t r| dkr| |||<||qeq[tfdd|Dr|D] \}} |vr| D]\} }| |vr|||| <|| qqn+|D](}||vr||jd jvr|||d |<q||vrtd |d q|D] }|| |d q|S) a Method to merge dictionaries of kwargs cleanly separated by modality within a Processor instance. The order of operations is as follows: 1) kwargs passed as before have highest priority to preserve BC. ```python high_priority_kwargs = {"crop_size" = {"height": 222, "width": 222}, "padding" = "max_length"} processor(..., **high_priority_kwargs) ``` 2) kwargs passed as modality-specific kwargs have second priority. This is the recommended API. ```python processor(..., text_kwargs={"padding": "max_length"}, images_kwargs={"crop_size": {"height": 222, "width": 222}}}) ``` 3) kwargs passed during instantiation of a modality processor have fourth priority. ```python tokenizer = tokenizer_class(..., {"padding": "max_length"}) image_processor = image_processor_class(...) processor(tokenizer, image_processor) # will pass max_length unless overridden by kwargs at call ``` 4) defaults kwargs specified at processor level have lowest priority. ```python class MyProcessingKwargs(ProcessingKwargs, CommonKwargs, TextKwargs, ImagesKwargs, total=False): _defaults = { "text_kwargs": { "padding": "max_length", "max_length": 64, }, } ``` Args: ModelProcessorKwargs (`ProcessingKwargs`): Typed dictionary of kwargs specifically required by the model passed. tokenizer_init_kwargs (`Dict`, *optional*): Dictionary of kwargs the tokenizer was instantiated with and need to take precedence over defaults. Returns: output_kwargs (`Dict`): Dictionary of per-modality kwargs to be passed to each modality-specific processor. )rgrhrjrirf>r%audioimagesvideos __empty__zKeyword argument z+ was passed two times: in a dictionary for z and as a **kwarg.c3s|]}|vVqdSrrI)rrdefault_kwargsrIrJrsz/ProcessorMixin._merge_kwargs..rfzKeyword argument `zA` is not a valid argument for this processor and will be ignored.)r _defaultsrrrErrr~rr*rrrrGaddanyrrr)rr-r.r output_kwargspossible_modality_keywords used_keysmodality modality_keyvaluenon_modality_kwargs kwarg_valuesubdictsubkeysubvaluerrIr3rJ _merge_kwargszs/           zProcessorMixin._merge_kwargsmainrrrrrc  s||d<||d<||d<||d<|dd}|dur*tdt|dur(td|}|dur2||d <|j|fi|} |j|fi|\}fd d |D|j | fi|S) a[ Instantiate a processor associated with a pretrained model. This class method is simply calling the feature extractor [`~feature_extraction_utils.FeatureExtractionMixin.from_pretrained`], image processor [`~image_processing_utils.ImageProcessingMixin`] and the tokenizer [`~tokenization_utils_base.PreTrainedTokenizer.from_pretrained`] methods. Please refer to the docstrings of the methods above for more information. Args: pretrained_model_name_or_path (`str` or `os.PathLike`): This can be either: - a string, the *model id* of a pretrained feature_extractor hosted inside a model repo on huggingface.co. - a path to a *directory* containing a feature extractor file saved using the [`~SequenceFeatureExtractor.save_pretrained`] method, e.g., `./my_model_directory/`. - a path or url to a saved feature extractor JSON *file*, e.g., `./my_model_directory/preprocessor_config.json`. **kwargs Additional keyword arguments passed along to both [`~feature_extraction_utils.FeatureExtractionMixin.from_pretrained`] and [`~tokenization_utils_base.PreTrainedTokenizer.from_pretrained`]. rrrrrNrrrcs"i|] \}}|vr||qSrI)rrrrIrJr9s"z2ProcessorMixin.from_pretrained..) rrrrr_get_arguments_from_pretrainedr&r*rr,) rrrrrrrrrrrIrErJfrom_pretraineds*' zProcessorMixin.from_pretrained AutoProcessorcCsDt|ts|j}ddlmm}t||st|d||_dS)a Register this class with a given auto class. This should only be used for custom feature extractors as the ones in the library are already mapped with `AutoProcessor`. This API is experimental and may have some slight breaking changes in the next releases. Args: auto_class (`str` or `type`, *optional*, defaults to `"AutoProcessor"`): The auto class to register this new feature extractor with. rNz is not a valid auto class.) rrGr@transformers.models.automodelsautorrr)r auto_class auto_modulerIrIrJregister_for_auto_class<s   z&ProcessorMixin.register_for_auto_classc  sg}jD]W}t|d}t|trKtfdd|D}|dkr3|dd}|dur2tdn|dd}|rF|d durF|d }n |d }n|}||j |fi|q|S) a Identify and instantiate the subcomponents of Processor classes, like image processors and tokenizers. This method uses the Processor attributes like `tokenizer_class` to figure out what class those subcomponents should be. Note that any subcomponents must either be library classes that are accessible in the `transformers` root, or they must be custom code that has been registered with the relevant autoclass, via methods like `AutoTokenizer.register()`. If neither of these conditions are fulfilled, this method will be unable to find the relevant subcomponent class and will raise an error. rc3s&|]}|dur |ndVqdSrrrrrIrJrds$z@ProcessorMixin._get_arguments_from_pretrained..ruse_fastNaCUsing a slow image processor as `use_fast` is unset and a slow processor was saved with this model. `use_fast=True` will be the default behavior in v4.52, even if the model was saved with a slow processor. This will result in minor differences in outputs. You'll still be able to use a slow processor with `use_fast=False`.Tr r) rrrrrrrrrrG) rrrrrrclassesrPattribute_classrIrOrJrFVs&       z-ProcessorMixin._get_arguments_from_pretrainedcCstt|r tt|Stjtjtjg}|D]5}|jD]-}t|t r9|D]}|dur7|j |kr7|Sq$q|durH|j |krH|Sqqt d|d)NzCould not find module z in `transformers`. If this is a custom class, it should be registered using the relevant `AutoClass.register()` function so that other functions can find it!) rtransformers_modulerIMAGE_PROCESSOR_MAPPINGTOKENIZER_MAPPINGFEATURE_EXTRACTOR_MAPPING_extra_contentvaluesrrr@r) module_namelookup_locationslookup_location custom_classcustom_subclassrIrIrJrzs*     z*ProcessorMixin.get_possibly_dynamic_modulecCst||jd}t|ddS)Nrmodel_input_names)rr)rfirst_attributerIrIrJr^s z ProcessorMixin.model_input_namescs6}i}t|t|}|rfdd|D}|S)Ncsi|]}||qSrIrI)rrr(rIrJrrz7ProcessorMixin.validate_init_kwargs..)rr)r(rkwargs_from_configr+ unused_keysrIr`rJr)s z#ProcessorMixin.validate_init_kwargscGsht|r tdt|t|jkr)tdt|jdd|jdt|dddt||jDS) a Matches optional positional arguments to their corresponding names in `optional_call_args` in the processor class in the order they are passed to the processor call. Note that this should only be used in the `__call__` method of the processors with special arguments. Special arguments are arguments that aren't `text`, `images`, `audio`, nor `videos` but also aren't passed to the tokenizer, image processor, etc. Examples of such processors are: - `CLIPSegProcessor` - `LayoutLMv2Processor` - `OwlViTProcessor` Also note that passing by position to the processor call is now deprecated and will be disallowed in future versions. We only have this for backward compatibility. Example: Suppose that the processor class has `optional_call_args = ["arg_name_1", "arg_name_2"]`. And we define the call method as: ```python def __call__( self, text: str, images: Optional[ImageInput] = None, *arg, audio=None, videos=None, ) ``` Then, if we call the processor as: ```python images = [...] processor("What is common in these images?", images, arg_value_1, arg_value_2) ``` Then, this method will return: ```python { "arg_name_1": arg_value_1, "arg_name_2": arg_value_2, } ``` which we could then pass as kwargs to `self._merge_kwargs` zPassing positional arguments to the processor call is now deprecated and will be disallowed in v4.47. Please pass all arguments as keyword arguments.zExpected *at most* zK optional positional arguments in processor callwhich will be matched with  z+ in the order they are passed.However, got z positional arguments instead.Please pass all arguments as keyword arguments instead (e.g. `processor(arg_name_1=..., arg_name_2=...))`.cSsi|]\}}||qSrIrI)r arg_valuearg_namerIrIrJrrzJProcessorMixin.prepare_and_validate_optional_call_args..)rrrrrrr)rrrIrIrJ'prepare_and_validate_optional_call_argss, z6ProcessorMixin.prepare_and_validate_optional_call_args conversation batch_images batch_videosbatch_video_metadatamm_load_kwargscKs|S)a) Used within `apply_chat_template` when a model has a special way to process conversation history. For example, video models might want to specify in the prompt the duration of video or which frame indices at which timestamps were sampled. This information cannot be accessed before the video is loaded. For most models it is a no-op, and must be overridden by model processors which require special processing. Args: conversation (`List[Dict, str, str]`): The conversation to process. Always comes in batched format. batch_images (`List[List[ImageInput]]`): Batch of images that were loaded from url/path defined in the conversation. The images are ordered in the same way as in the conversation. Comes in nested list format, one list of `PIL` images per batch. batch_videos (`List[List[ImageInput]]`): Batch of videos that were loaded from url/path defined in the conversation. The videos are ordered in the samm way as in the conversation. Comes in nested list format, one list of 4D video arrays per batch. batch_video_metadata (`List[List[Dict[[str, any]]]]`): Batch of metadata returned from loading videos. That includes video fps, duration and total number of framer in original video. Metadata are ordered in the same way as `batch_videos`. Comes in nested list format, one list of 4D video arrays per batch. rI)rrgrhrirjrkrIrIrJ#_process_messages_for_chat_templates z2ProcessorMixin._process_messages_for_chat_templaterc KsD|dur|jdur |j}ntdi}tjD]}tt|d}|||}|||<qi}tjD]} tt| d}|| |}||| <q2t|t t frat|dt t fs\t |ddrad} |} nd} |g} |dd} |dd} | rHgg}}g}g}| D]}gg}}g}|D]}d d |dD}d d |dD}d d |D}d d |D}|D] }| t |q|ds|D] }| t||ddqn|D] }| t||ddq|D]C}t|t t frt|dtrdd |D}t|}d}tdnt||d|d|d|dd\}}| || |qq|r-| ||r:| || |q|j| f|||d|} |jj| f|ddd|}| s]|d}| r| rg|dn|}|jjdur|||jjr|d|d<|d||r|nd|r|nd|r|ndd|}| r|S|dS|S)a Similar to the `apply_chat_template` method on tokenizers, this method applies a Jinja template to input conversations to turn them into a single tokenizable string. The input is expected to be in the following format, where each message content is a list consisting of text and optionally image or video inputs. One can also provide an image, video, URL or local path which will be used to form `pixel_values` when `return_dict=True`. If not provided, one will get only the formatted text, optionally tokenized text. conversation = [ { "role": "user", "content": [ {"type": "image", "image": "https://www.ilankelman.org/stopsigns/australia.jpg"}, {"type": "text", "text": "Please describe this image in detail."}, ], }, ] Args: conversation (`Union[List[Dict, [str, str]], List[List[Dict[str, str]]]]`): The conversation to format. chat_template (`Optional[str]`, *optional*): The Jinja template to use for formatting the conversation. If not provided, the tokenizer's chat template is used. NzNo chat template is set for this processor. Please either set the `chat_template` attribute, or provide a chat template as an argument. See https://huggingface.co/docs/transformers/main/en/chat_templating for more information.rcontentTFryrzcSsg|] }|ddvr|qS)r)imagevideorI)rrmrIrIrJrHrz6ProcessorMixin.apply_chat_template..cS2g|]}dD]}||vr|ddkr||qqS))r/urlrrr/rI)rrmrrIrIrJrIcSrp))rnrqrbase64rrnrIr vision_inforrIrIrJrOrrcSrp))rorqrrrorIrtrIrIrJrUrrrwra)racSsg|] }tt|jqSrI)nparrayrT)r image_fnamerIrIrJrirzWhen loading the video from list of images, we cannot infer metadata such as `fps` or `duration`. If your model uses this metadata during processing, please load the whole video and let the model sample frames instead.rrrurtrv)rrfpsbackendrv)rhrirj)rryrzr1)r%r0r1r/ input_idsrI)rrrkrErrrrqrrDrrrrr rGrvstackrwarningrrlr~apply_chat_template bos_token startswith) rrgrrtokenizer_template_kwargs tokenizer_key default_valuer=rk mm_load_key is_batched conversationsryrzrhri batch_audiosrjr0r1video_metadatamessagevisuals audio_fnames image_fnames video_fnamesfnamerometadataprompt single_promptoutrIrIrJrs                      z"ProcessorMixin.apply_chat_templateTcKs|jj|fd|i|S)a Post-process the output of a vlm to decode the text. Args: generated_outputs (`torch.Tensor` or `np.ndarray`): The output of the model `generate` function. The output is expected to be a tensor of shape `(batch_size, sequence_length)` or `(sequence_length,)`. skip_special_tokens (`bool`, *optional*, defaults to `True`): Whether or not to remove special tokens in the output. Argument passed to the tokenizer's `batch_decode` method. **kwargs: Additional arguments to be passed to the tokenizer's `batch_decode method`. Returns: `List[str]`: The decoded text. skip_special_tokens)r~ batch_decode)rgenerated_outputsrrrIrIrJpost_process_image_text_to_textsz.ProcessorMixin.post_process_image_text_to_text)Fr)NFFNrD)rH)T)5r@rArBrCrrrrDrGrEfeature_extractor_classtokenizer_classrrrr]rrrr rPathLikerrrFr classmethodrr&r,rerrCrGrNrF staticmethodrpropertyr^r)rfrrrrr7Unpackrqrlr{rrrIrIrIrJr|s &(  l  2-    ?  #   : %& 5r|csdtfddfddfdddd }||}|}||}|}|r2|r2||fS|d ur8|sB|d ur>|sB|rK|rKtd ||fStd ) a For backward compatibility: reverse the order of `images` and `text` inputs if they are swapped. This method should only be called for processors where `images` and `text` have been swapped for uniformization purposes. Note that this method assumes that two `None` inputs are valid inputs. If this is not the case, it should be handled in the processor's `__call__` method before calling this method. rcSst|to |dS)Nhttp)rrGr)valrIrIrJis_urlsz1_validate_images_text_input_order..is_urlcsBt|ttfr|D] }|sdSq dSt|s|sdSdS)NFT)rrDrr)imgsimg)$_is_valid_images_input_for_processorrrIrJrszO_validate_images_text_input_order.._is_valid_images_input_for_processorcsDt|trdSt|ttfr t|dkrdS|D]}|SdS)NTrF)rrGrDrr)tt_s)"_is_valid_text_input_for_processorrIrJrs   zM_validate_images_text_input_order.._is_valid_text_input_for_processorcSs||p|duSrrI)input validatorrIrIrJ _is_validsz4_validate_images_text_input_order.._is_validNzYou may have used the wrong order for inputs. `images` should be passed before `text`. The `images` and `text` inputs will be swapped. This behavior will be deprecated in transformers v4.47.zGInvalid input type. Check that `images` and/or `text` are valid inputs.)rFrrr)r0r%rimages_is_validimages_is_text text_is_validtext_is_imagesrI)rrrrJ!_validate_images_text_input_orders     rr rHzprocessor files)object object_class object_files)KrCrrrrsystypingrpathlibrrrrrrrr numpyrvtyping_extensions audio_utilsr dynamic_module_utilsr image_utilsr rrrrrrrtokenization_utils_baserrrrrutilsrrrrrrr r!r"r#r$r% get_loggerr@r__file__parentrSr version_inforr-rLr_r`rcrerkrqrxr{r|rrformatrIrIrIrJsp  $  $ 8  ::0'7'!   :