o  hK@s>ddlZddlmZddlmZddlmZddlmZm Z m Z m Z m Z m Z ddlZddlmZmZddlmZdd lmZmZmZmZd d lmZerSdd lmZer\dd lmZerpddl m!Z!ddl"m#Z#ddl$m%Z%e&e'Z(edZ)defddZ*  d3de ddddede+dede dde,d e e e,e e,ffd!d"Z- d4de d#dd$dede+dede dd e e e,e e,ffd%d&Z.  d3de dde d'dede+dede dde,d e e e,e e,ffd(d)Z/ d4dede dd*e d'd+ed,e e,d-e0de dfd.d/Z1de d'd0e e,d e e2e e,ffd1d2Z3dS)5N) signature)chain)Path) TYPE_CHECKINGIterableListOptionalTupleUnion)Versionparse)PreTrainedTokenizerBase) TensorTypeis_tf_availableis_torch_availablelogging) OnnxConfig)PreTrainedModel)TFPreTrainedModel)FeatureExtractionMixin)ProcessorMixin)PreTrainedTokenizerz1.4.0minimum_versioncCsPzddl}t|j}|tkrtd|jd|dWdSty'tdw)z Check onnxruntime is installed and if the installed version match is recent enough Raises: ImportError: If onnxruntime is not installed or too old version is found rNz*We found an older version of onnxruntime (z&) but we require onnxruntime to be >= zp to enable all the conversions options. Please update onnxruntime by running `pip install --upgrade onnxruntime`zonnxruntime doesn't seem to be currently installed. Please install the onnxruntime by running `pip install onnxruntime` and relaunch the conversion.) onnxruntimer __version__ORT_QUANTIZE_MINIMUM_VERSION ImportError)rr ort_versionr m/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/transformers/onnx/convert.pycheck_onnxruntime_requirements5s   r"cpu preprocessor)rrrmodelrconfigopsetoutput tokenizerrdevicereturnc s:t|tr |dur td|durtdttd|}tt |t rddl ddl m }tdjd|j_||jdurstd t|jd |jD]\}} td |d | t|j|| q\|j|tjd } j dkrɈjr|i} | D]1\} } t| trtfdd| D| | <qt| t rfdd| D| | <q| | | <q| } t!|| "\}}t#|j$"}|std|%||| f|&t#|j'"|t(t)|j'|j$d|d|*Wd||fS1swY||fS)a Export a PyTorch model to an ONNX Intermediate Representation (IR) Args: preprocessor: ([`PreTrainedTokenizer`], [`FeatureExtractionMixin`] or [`ProcessorMixin`]): The preprocessor used for encoding the data. model ([`PreTrainedModel`]): The model to export. config ([`~onnx.config.OnnxConfig`]): The ONNX configuration associated with the exported model. opset (`int`): The version of the ONNX operator set to use. output (`Path`): Directory to store the exported ONNX model. device (`str`, *optional*, defaults to `cpu`): The device on which the ONNX model will be exported. Either `cpu` or `cuda`. Returns: `Tuple[List[str], List[str]]`: A tuple with an ordered list of the model's inputs, and the named inputs from the ONNX configuration. NKYou cannot provide both a tokenizer and a preprocessor to export the model.tThe `tokenizer` argument is deprecated and will be removed in version 5 of Transformers. Use `preprocessor` instead.ROverwriting the `preprocessor` argument with `tokenizer` to generate dummy inputs.r)exportzUsing framework PyTorch: T Overriding  configuration item(s) -  ->  frameworkcudac3*|]}t|jr|ndVqdSN isinstanceTensorto.0xr*torchr r! s z!export_pytorch..cs$g|]}tfdd|DqS)c3r7r8r9r=r@r r!rBs(z,export_pytorch...)tuple)r>tr@r r! sz"export_pytorch..z%Model and config inputs doesn't match)f input_names output_names dynamic_axesdo_constant_folding opset_version)+r:r ValueErrorwarningswarn FutureWarningloggerinfo issubclasstyperrA torch.onnxr/rno_gradr& return_dictevalvalues_overridelenitemssetattrgenerate_dummy_inputsrPYTORCHr*r6 is_availabler<r rCr$ensure_model_and_config_inputs_matchkeyslistoutputs patch_opsas_posixinputsdictr restore_ops)r$r%r&r'r(r)r* onnx_exportoverride_config_keyoverride_config_value model_inputsmodel_inputs_devicekv inputs_matchmatched_inputs onnx_outputsr r@r!export_pytorchRsr            55rr)rrrcs2ddl}ddlddl}t|tr|durtd|dur*tdtt d|}d|j _ |j dur[t dt|j d|j D]\}} t d |d | t|j || qD|j|tjd } t|| \} } t|j} fd d | D}|jj|||d\}}||||| | fS)a Export a TensorFlow model to an ONNX Intermediate Representation (IR) Args: preprocessor: ([`PreTrainedTokenizer`] or [`FeatureExtractionMixin`]): The preprocessor used for encoding the data. model ([`TFPreTrainedModel`]): The model to export. config ([`~onnx.config.OnnxConfig`]): The ONNX configuration associated with the exported model. opset (`int`): The version of the ONNX operator set to use. output (`Path`): Directory to store the exported ONNX model. Returns: `Tuple[List[str], List[str]]`: A tuple with an ordered list of the model's inputs, and the named inputs from the ONNX configuration. rNzIYou cannot provide both a tokenizer and preprocessor to export the model.r-r.Tr0r1r2r3r4cs*g|]\}}jdg|j|j|dqS)N)dtypename) TensorSpecndimrs)r>keytensortfr r!rEsz%export_tensorflow..)r')onnx tensorflowtf2onnxr:rrLrMrNrOrPrQr&rVrXrYrZr[r\r TENSORFLOWr_r`rarbconvert from_kerassaverdrg)r$r%r&r'r(r)r{r}rirjrkrorprqinput_signature onnx_model_r ryr!export_tensorflows8   r)rrc Csts ts tdtrt|tr|dkrtdt|tr'|dur'td|dur8t dt t d|}trQdd l m}|jsQt d |jd |trftt|trft|||||||d Strztt|tr|t||||||d SdSdS)a Export a Pytorch or TensorFlow model to an ONNX Intermediate Representation (IR) Args: preprocessor: ([`PreTrainedTokenizer`], [`FeatureExtractionMixin`] or [`ProcessorMixin`]): The preprocessor used for encoding the data. model ([`PreTrainedModel`] or [`TFPreTrainedModel`]): The model to export. config ([`~onnx.config.OnnxConfig`]): The ONNX configuration associated with the exported model. opset (`int`): The version of the ONNX operator set to use. output (`Path`): Directory to store the exported ONNX model. device (`str`, *optional*, defaults to `cpu`): The device on which the ONNX model will be exported. Either `cpu` or `cuda`. Only PyTorch is supported for export on CUDA devices. Returns: `Tuple[List[str], List[str]]`: A tuple with an ordered list of the model's inputs, and the named inputs from the ONNX configuration. zrCannot convert because neither PyTorch nor TensorFlow are not installed. Please install torch or tensorflow first.r6z1`tf2onnx` does not support export on CUDA device.Nr,r-r.r )get_torch_versionz@Unsupported PyTorch version for this model. Minimum required is z, got: )r)r*)r))rrrr:r RuntimeErrorrrLrMrNrOrPrQutilsris_torch_support_availablewarningtorch_onnx_minimum_versionrRrSrrrr)r$r%r&r'r(r)r*rr r r!r/s8    r/reference_modelronnx_named_outputsatolc CsFddlm}m}tdt|tr|durtd|dur+t dt td|}t rFt t |trF|j||jd|jdtjd} n|j||jd|jdtjd} |} ||| d gd } t rrt t |trr|d |d$i| } i} | D]!\}}|d krd }t|ttfr|||}| |q|| |<q|| }i}|D]$\}}t|ttfr|||}|dd|Dq|||<q| ||}t| t|}}|!|std|d|td|"|td|dt#||D]\}}t r#t t |tr#| |$}n| |}td|d|j%|j%ksStd|j%d|j%td|j%d|j%dtd|j%d|j%t&j'|||dst&(t&j)|||d}td|dtd t&*t&+||d!||d"||td#|dq dS)%Nr)InferenceSessionSessionOptionszValidating ONNX model...zUYou cannot provide both a tokenizer and a preprocessor to validate the model outputs.r-r.r) batch_size seq_lengthr5CPUExecutionProvider) providersr#past_key_valuespresentcSsi|] \}}||qSr )numpy)r> tensor_name pt_tensorr r r! sz*validate_model_outputs..z -[x] ONNX model output names z do not match reference model zGOutputs doesn't match between reference model and ONNX exported model: u7 -[✓] ONNX model output names match reference model ()z! - Validating ONNX Model output "z":z -[x] shape z doesn't match zQOutputs shape doesn't match between reference model and ONNX exported model: Got z (reference) and z (ONNX)u -[✓] z matches )rz& -[x] values not close enough (atol: znOutputs values doesn't match between reference model and ONNX exported model: Got max absolute difference of: z for z vs u! -[✓] all values close (atol: r ),rrrrPrQr:rrLrMrNrOrrRrSrr\default_fixed_batchdefault_fixed_sequencerr]r~rdr<rZrarC"flatten_output_collection_propertyupdate!generate_dummy_inputs_onnxruntimerrunsetr`issubset differencezipdetachshapenpallclose logical_notiscloseamaxabs)r&r$rrrrr)rrreference_model_inputsoptionssession ref_outputsref_outputs_dictrtvalue"reference_model_inputs_onnxruntime onnx_inputsrqref_outputs_setonnx_outputs_set ort_value ref_value bad_indicesr r r!validate_model_outputs=s           rrkcsttrtt|trt|jj}nt|jj}t|}t| }| |}| |fdd| D}||fS)z> :param model_inputs: :param config_inputs: :return: csg|]}|vr|qSr r )r> parametermatching_inputsr r!rEsz8ensure_model_and_config_inputs_match..) rrRrSrrforward parameterscallrr`r intersection)r%rkforward_parametersmodel_inputs_setforward_inputs_setis_okordered_inputsr rr!r_s    r_)Nr#r8)4rMinspectr itertoolsrpathlibrtypingrrrrr r rrpackaging.versionr r tokenization_utils_baserrrrrrr&rmodeling_utilsrmodeling_tf_utilsrfeature_extraction_utilsrprocessing_utilsrtokenization_utilsr get_logger__name__rPrr"intstrrrrr/floatrboolr_r r r r!s            # m H I {