o  h@sddlZddlmZddlmZmZddlZddlmZm Z m Z m Z m Z ddl mZmZe r7ddlZddlZe r@ddlmZe eZeed d Gd d d eZd dZdS)N)BytesIO)ListUnion)add_end_docstringsis_av_availableis_torch_availableloggingrequires_backends)Pipelinebuild_pipeline_init_args),MODEL_FOR_VIDEO_CLASSIFICATION_MAPPING_NAMEST)has_image_processorcsfeZdZdZfddZdddZddeeeefffdd Z dd d Z d dZ dddZ Z S)VideoClassificationPipelinea Video classification pipeline using any `AutoModelForVideoClassification`. This pipeline predicts the class of a video. This video classification pipeline can currently be loaded from [`pipeline`] using the following task identifier: `"video-classification"`. See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=video-classification). cs*tj|i|t|d|tdS)Nav)super__init__r check_model_typer)selfargskwargs __class__/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/transformers/pipelines/video_classification.pyr6s z$VideoClassificationPipeline.__init__NcCsti}|dur ||d<|dur||d<i}|dur||d<|dur1|dvr,td|d||d<nd|d<|i|fS) Nframe_sampling_rate num_framestop_k)softmaxsigmoidnonez'Invalid value for `function_to_apply`: z2. Valid options are ['softmax', 'sigmoid', 'none']function_to_applyr) ValueError)rrrrr"preprocess_paramspostprocess_paramsrrr_sanitize_parameters;s    z0VideoClassificationPipeline._sanitize_parametersinputsc sBd|vrtdt|d}|durtdtj|fi|S)a Assign labels to the video(s) passed as inputs. Args: inputs (`str`, `List[str]`): The pipeline handles three types of videos: - A string containing a http link pointing to a video - A string containing a local path to a video The pipeline accepts either a single video or a batch of videos, which must then be passed as a string. Videos in a batch must all be in the same format: all as http links or all as local paths. top_k (`int`, *optional*, defaults to 5): The number of top labels that will be returned by the pipeline. If the provided number is higher than the number of labels available in the model configuration, it will default to the number of labels. num_frames (`int`, *optional*, defaults to `self.model.config.num_frames`): The number of frames sampled from the video to run the classification on. If not provided, will default to the number of frames specified in the model configuration. frame_sampling_rate (`int`, *optional*, defaults to 1): The sampling rate used to select frames from the video. If not provided, will default to 1, i.e. every frame will be used. function_to_apply(`str`, *optional*, defaults to "softmax"): The function to apply to the model output. By default, the pipeline will apply the softmax function to the output of the model. Valid options: ["softmax", "sigmoid", "none"]. Note that passing Python's built-in `None` will default to "softmax", so you need to pass the string "none" to disable any post-processing. Return: A list of dictionaries or a list of list of dictionaries containing result. If the input is a single video, will return a list of `top_k` dictionaries, if the input is a list of several videos, will return a list of list of `top_k` dictionaries corresponding to the videos. The dictionaries contain the following keys: - **label** (`str`) -- The label identified by the model. - **score** (`int`) -- The score attributed by the model for that label. videoszuThe `videos` argument has been renamed to `inputs`. In version 5 of Transformers, `videos` will no longer be acceptedNzICannot call the video-classification pipeline without an inputs argument!)warningswarn FutureWarningpopr#r__call__)rr'rrrrr-Ps' z$VideoClassificationPipeline.__call__r c Cs|dur |jjj}|ds|drtt|j}t |}d}||d}t j |||t j d}t ||}t|}|j||jd}|jdkrN||j}|S)Nzhttp://zhttps://rr )numdtype)return_tensorspt)modelconfigr startswithrrequestsgetcontentropennplinspaceint64read_video_pyavlistimage_processor frameworkto torch_dtype) rvideorr container start_idxend_idxindices model_inputsrrr preprocesss      z&VideoClassificationPipeline.preprocesscCs|jdi|}|S)Nr)r2)rrG model_outputsrrr_forwardsz$VideoClassificationPipeline._forwardrcs|jjjkr jjj}jdkr7|dkr|jdd}n|dkr*|jd}n|jd}||\}}ntdj| }| }fddt ||DS) Nr1rrr zUnsupported framework: cs$g|]\}}|jjj|dqS))scorelabel)r2r3id2label).0rM_idrrr s$z;VideoClassificationPipeline.postprocess..) r2r3 num_labelsr?logitsrr topkr#tolistzip)rrIrr"probsscoresidsrrRr postprocesss   z'VideoClassificationPipeline.postprocess)NNNN)N)Nr )rKr)__name__ __module__ __qualname____doc__rr&rstrrr-rHrJr\ __classcell__rrrrr)s   1rcCsrg}|d|d}|d}t|jddD]\}}||kr!n||kr.||vr.||qtdd|DS)NrrL)rBcSsg|]}|jddqS)rgb24)format) to_ndarray)rPxrrrrSsz#read_video_pyav..)seek enumeratedecodeappendr9stack)rCrFframes start_index end_indexiframerrrr<s  r<)r)iortypingrrr5utilsrrrr r baser r rnumpyr9models.auto.modeling_autor get_loggerr]loggerrr<rrrrs