o  h @sRddlZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl Z ddlZddlZddlZddlZddlZddlZddlmZmZddlmZmZmZmZddlmZmZddlmZmZddl m!Z!ddl"m#Z#ddl$m%Z%m&Z&m'Z'm(Z(dd lm)Z)dd l*m+Z+ddl,Z-ddl.Z.ddl/Z/dd l-m0Z0dd l1m2Z2dd l3m4Z4ddl3mZ5ddl6m7Z7m8Z8m9Z9m:Z:m;Z;mm?Z?ddl@mAZAmBZBmCZCmDZDmEZEmFZFmGZGmHZHmIZImJZJmKZKmLZLmMZMmNZNmOZOmPZPmQZQmRZRmSZSmTZTmUZUmVZVmWZWmXZXmYZYmZZZm[Z[m\Z\m]Z]m^Z^m_Z_m`Z`maZambZbmcZcmdZdmeZemfZfmgZgmhZhmiZimjZjmkZkmlZlmmZmmnZnmoZompZpmqZqmrZrmsZsmtZtmuZumvZvmwZwmxZxmyZymzZzm{Z{m|Z|m}Z}m~Z~mZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZeCrddlmZmZddlmZeurddlmZmZmZmZmZmZmZddlmZddlmZddlmZneZeZdZdZdZdZdZdZerddlZej2jduZej2jduZndZdZe5e¡ZÐdd d!ZĐdd"d#Zed$dd%Zed&dd%Zed'dd%Zed(d)d%Zed*dd%Zed+dd%Zd,d-Zd.d/Zd0d1Zd2d3Zd4d5Zd6d7Zd8d9Zd:d;Zdd?Zd@dAZdBdCZdDdEZdFdGZdHdIZdJdKZdLdMZeAfdNefdOdPZeBfdNefdQdRZߐddNefdTdUZdVdWZdXdYZdZd[Zd\d]Zd^d_Zd`daZdbdcZdddeZdfdgZdhdiZdjefdkdlZdmdnZdodpZdqdrZdsdtZdudvZdwdxZdydzZd{d|Zd}d~ZddZddZddZddZddZddZddZddZddZddZddZddZddZddZddZddZddZddZddZddZddZ ddZ ddZ ddZ ddZ ddZddZddZddZddZerddlZde jvre jdZz eeZWn eyZzededeedZ[wwde jvrMe jdZedkrejs֐edededkresededÝedkresededŝedkresededǝedkr&es&ededɝz eeZWnZeyLZz edʐeedZ[wwejrXdZn3eːrcercdĐZn(eːrnerndƐZneːryerydȐZneːrerdZndːZndZerddlZeVrddl Z e !Z"ndZ"dd̈́Z#ddτZ$ddфZ%ddӄZ&ddՄZ'ddאe(fddلZ)ddۄZ*dd݄Z+dd߄Z,ddZ-ddZ.ddZ/ddZ0ddZ1ddZ2ddZ3ddZ4ddZ5ddZ6ddZ7ddZ8ddZ9ddZ:ddZ;ddZ<ddZ=ddZ>ddZ?ddZ@dd ZAd d ZBd d ZCddZDddZEddZFddZGddZHddZIddZJddZKddZLd d!ZMd"d#ZNd$d%ZOd&d'ZPd(d)ZQd*d+ZRd,d-ZSd.d/ZTd0d1ZUd2d3ZVd4d5ZWd6d7ZXd8d9ZYd:d;ZZd<d=Z[dŐd>d?Z\d@e4dAe]fdBdCZ^dDe_e`eݐe(fdAee`eݐe(fddffdEdFZadGdHZbdIdJZcdKdLZddMdNZedOdPZfGdQdRdRZgGdSdTdTegZhGdUdVdVegZiGdWdXdXZjejkdYdZZlGd[d\d\Zmejkd]e(ee jnfdAedfd^d_ZoGd`dadaejpZqdbdcZrejkdddeZsiZtdfdgZudhdiZvddlwZwGdjdkdkZxdldmZydȐdAexfdndoZzdɐdAexfdqdrZ{dsdtZ|dudvZ}dʐdxdyZ~dzd{Zd|d}ZGd~ddeZdĐde_efddZGdddZdːde]de'e(de'efddZd̐de]de'e(fddZddZd͐ddZddZ ddZGdddejZGdddeZdeݐde`ee&ffddZer ejjejejdZejjdddZejjdddddZnddiZddiZdddiZerLejjed<ejjed<erkejjed<ejjed<ejjed<erejjed<ejjed<ejjed<erejjed<ejjed<ejjed<erejjed<ejjed<ejjed<deݐde]fddZdefddZdefddZe rde jv re jdZe#e sedee je je\ZZejezededZWne y]Zz edeedZ[wweeZzejZWne yZzededZ[wwde jv reek rdededZed7ZeeeZde`ee&fdefddZeedeedeedddZedĐdefddZee(edfe(e]dffZedAefddZGddÄdeee%fZdS(N)UserDict defaultdict) GeneratorIterableIteratorMapping)MISSINGfields)cachewraps)StringIO)Path)AnyCallableOptionalUnion)mockpatch) delete_repoversion)Trainer)logging)is_clearml_availableis_optuna_availableis_ray_availableis_sigopt_availableis_swanlab_availableis_tensorboard_availableis_wandb_available)is_deepspeed_available)aACCELERATE_MIN_VERSIONGGUF_MIN_VERSIONis_accelerate_availableis_apex_availableis_apollo_torch_availableis_aqlm_availableis_auto_awq_availableis_auto_gptq_availableis_av_availableis_bitsandbytes_available'is_bitsandbytes_multi_backend_availableis_bs4_availableis_compressed_tensors_availableis_cv2_availableis_cython_availableis_detectron2_availableis_eetq_availableis_essentia_availableis_faiss_availableis_fbgemm_gpu_availableis_flash_attn_2_availableis_flax_availableis_flute_availableis_fsdp_availableis_ftfy_availableis_g2p_en_availableis_galore_torch_availableis_gguf_availableis_gptqmodel_availableis_grokadamw_availableis_hadamard_availableis_hqq_availableis_ipex_availableis_jieba_availableis_jinja_availableis_jumanpp_availableis_keras_nlp_availableis_levenshtein_availableis_librosa_availableis_liger_kernel_availableis_lomo_availableis_natten_availableis_nltk_availableis_onnx_availableis_optimum_availableis_optimum_quanto_availableis_pandas_availableis_peft_availableis_phonemizer_availableis_pretty_midi_availableis_pyctcdecode_availableis_pytesseract_availableis_pytest_available!is_pytorch_quantization_availableis_quark_availableis_rjieba_availableis_sacremoses_availableis_safetensors_availableis_schedulefree_availableis_scipy_availableis_sentencepiece_availableis_seqio_availableis_soundfile_availableis_spacy_availableis_spqr_availableis_sudachi_availableis_sudachi_projection_available#is_tensorflow_probability_availableis_tensorflow_text_availableis_tf2onnx_availableis_tf_availableis_tiktoken_availableis_timm_availableis_tokenizers_availableis_torch_available!is_torch_bf16_available_on_deviceis_torch_bf16_cpu_availableis_torch_bf16_gpu_availableis_torch_deterministic!is_torch_fp16_available_on_deviceis_torch_greater_or_equalis_torch_hpu_availableis_torch_mlu_availableis_torch_neuroncore_availableis_torch_npu_availableis_torch_sdpa_availableis_torch_tensorrt_fx_availableis_torch_tf32_availableis_torch_xla_availableis_torch_xpu_availableis_torchao_availableis_torchaudio_availableis_torchdynamo_availableis_torchvision_availableis_vision_availableis_vptq_available strtobool)AcceleratorState PartialState)is_fp8_available)Module _get_checker_get_continue_on_failure _get_runner _is_mocked_patch_unwrap_mock_awareget_optionflags)skip) import_path) DoctestItemzjulien-c/bert-xsmall-dummyzjulien-c/dummy-unknownzjulien-c/dummy-diff-tokenizer__DUMMY_TRANSFORMERS_USER__zhttps://hub-ci.huggingface.co%hf_94wBhPGp6KrrTH3KDchhKpRxZwd6dmHWLLFcCVztj|}Wn ty|}Y|Swzt|}W|Sty*td|dw)NIf set, z must be yes or no.)osenvironKeyErrorr ValueErrorkeydefaultvalue_valuern/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/transformers/testing_utils.pyparse_flag_from_envs   rcCr)Nrz must be a int.)rrrintrrrrrparse_int_from_envs   rRUN_SLOW)rRUN_CUSTOM_TOKENIZERSHUGGINGFACE_CO_STAGINGRUN_PIPELINE_TESTSTRUN_AGENT_TESTSRUN_THIRD_PARTY_DEVICE_TESTScCFts tjdd|Szddl}Wn ty|YSw|j|S)z Decorator marking a test as a staging test. Those tests will run using the staging environment of huggingface.co instead of the real model hub. ztest is staging testreasonrN) _run_stagingunittestrpytest ImportErrormarkis_staging_test test_caserrrrrs  rcCr)z Decorator marking a test as a pipeline test. If RUN_PIPELINE_TESTS is set to a falsy value, those tests will be skipped. ztest is pipeline testrrN)_run_pipeline_testsrrrrris_pipeline_testrrrrr s  rcCr)z| Decorator marking a test as an agent test. If RUN_TOOL_TESTS is set to a falsy value, those tests will be skipped. ztest is an agent testrrN)_run_agent_testsrrrrr is_agent_testrrrrrs  rcCttd|S)z Decorator marking a test as slow. Slow tests are skipped by default. Set the RUN_SLOW environment variable to a truthy value to run them. z test is slow)r skipUnless_run_slow_testsrrrrslow)rcCstjdd|S)z Decorator marking a test as too slow. Slow tests are skipped while they're in the process of being fixed. No test should stay tagged as "tooslow" as these will not be tested by the CI. ztest is too slowr)rrrrrrtooslow3rcstfdd}|S)Nc s:z|i|WSty}ztd|d}~ww)Nz)Test skipped due to NotImplementedError: )NotImplementedErrorrSkipTest)argskwargse test_funcrrwrapper?s z(skip_if_not_implemented..wrapper functoolsr )rrrrrskip_if_not_implemented>srcCs>t|D]}|drt||}t|rt||t|q|S)zP Class decorator to apply @skip_if_not_implemented to all test methods. test_)dir startswithgetattrcallablesetattrr)cls attr_nameattrrrrapply_skip_if_not_implementedIs   rcCr)z Decorator marking a test for a custom tokenizer. Custom tokenizers require additional dependencies, and are skipped by default. Set the RUN_CUSTOM_TOKENIZERS environment variable to a truthy value to run them. ztest of custom tokenizers)rr_run_custom_tokenizersrrrrcustom_tokenizersUrrcCttd|S)z} Decorator marking a test that requires BeautifulSoup4. These tests are skipped when BeautifulSoup4 isn't installed. ztest requires BeautifulSoup4)rrr.rrrr require_bs4_rcCr)z Decorator marking a test that requires GaLore. These tests are skipped when GaLore isn't installed. https://github.com/jiaweizzhao/GaLore ztest requires GaLore)rrr=rrrrrequire_galore_torchfrcCr)z Decorator marking a test that requires GaLore. These tests are skipped when APOLLO isn't installed. https://github.com/zhuhanqing/APOLLO ztest requires APOLLO)rrr'rrrrrequire_apollo_torchnrrcCr)z Decorator marking a test that requires LOMO. These tests are skipped when LOMO-optim isn't installed. https://github.com/OpenLMLab/LOMO ztest requires LOMO)rrrKrrrr require_lomovrrcCr)zs Decorator marking a test that requires GrokAdamW. These tests are skipped when GrokAdamW isn't installed. ztest requires GrokAdamW)rrr@rrrrrequire_grokadamw~rrcCr)z Decorator marking a test that requires schedulefree. These tests are skipped when schedulefree isn't installed. https://github.com/facebookresearch/schedule_free ztest requires schedulefree)rrr]rrrrrequire_schedulefreerrcCr)zs Decorator marking a test that requires OpenCV. These tests are skipped when OpenCV isn't installed. ztest requires OpenCV)rrr0rrrr require_cv2rcCr)z} Decorator marking a test that requires Levenshtein. These tests are skipped when Levenshtein isn't installed. ztest requires Levenshtein)rrrHrrrrrequire_levenshteinrrcCr)zo Decorator marking a test that requires NLTK. These tests are skipped when NLTK isn't installed. ztest requires NLTK)rrrMrrrr require_nltkrr min_versioncCtt|d||S)zu Decorator marking a test that requires accelerate. These tests are skipped when accelerate isn't installed. z$test requires accelerate version >= )rrr%rrrrrrequire_accelerates rcCr)zj Decorator marking a test that requires ggguf. These tests are skipped when gguf isn't installed. ztest requires gguf version >= )rrr>rrrr require_ggufr1.12.0cCr)zi Decorator marking a test that requires fsdp. These tests are skipped when fsdp isn't installed. ztest requires torch version >= )rrr:rrrr require_fsdprrcCr)zt Decorator marking a test that requires g2p_en. These tests are skipped when SentencePiece isn't installed. ztest requires g2p_en)rrr<rrrrrequire_g2p_enrrcCr)zw Decorator marking a test that requires safetensors. These tests are skipped when safetensors isn't installed. ztest requires safetensors)rrr\rrrrrequire_safetensorsrrcCr)zm Decorator marking a test that requires rjieba. These tests are skipped when rjieba isn't installed. ztest requires rjieba)rrrZrrrrrequire_rjiebarrcCr)zk Decorator marking a test that requires jieba. These tests are skipped when jieba isn't installed. ztest requires jieba)rrrDrrrr require_jiebarrcCr)zk Decorator marking a test that requires jinja. These tests are skipped when jinja isn't installed. ztest requires jinja)rrrErrrr require_jinjarrcCr)Nztest requires tf2onnx)rrrhrrrrrequire_tf2onnxrcCr)Nztest requires ONNX)rrrNrrrr require_onnxrrcCr)zo Decorator marking a test that requires Timm. These tests are skipped when Timm isn't installed. ztest requires Timm)rrrkrrrr require_timmrrcCr)zs Decorator marking a test that requires NATTEN. These tests are skipped when NATTEN isn't installed. ztest requires natten)rrrLrrrrrequire_nattenrrcCr)zu Decorator marking a test that requires PyTorch. These tests are skipped when PyTorch isn't installed. test requires PyTorch)rrrmrrrr require_torchrrrcfdd}|S)z Decorator marking a test that requires PyTorch version >= `version`. These tests are skipped when PyTorch version is less than `version`. csttd|S)Nz!test requires PyTorch version >= )rrrsrrrr decoratorsz1require_torch_greater_or_equal..decoratorr)rrrrrrequire_torch_greater_or_equals rcCr)z Decorator marking a test that requires Flash Attention. These tests are skipped when Flash Attention isn't installed. ztest requires Flash Attention)rrr7rrrrrequire_flash_attnrrcCr)z Decorator marking a test that requires PyTorch's SDPA. These tests are skipped when requirements are not met (torch version). ztest requires PyTorch SDPA)rrrxrrrrrequire_torch_sdpa(srcs$tdtfdd}|S)zZ A decorator that loads the HF token for tests that require to load gated models. HF_HUB_READ_TOKENcsTdur#tdd|i|WdS1swYdS|i|S)Nz(huggingface_hub.utils._headers.get_token) return_valuerrrfntokenrr_inner7s  $z"require_read_token.._inner)rgetenvr )rrrrrrequire_read_token1s rcCr)zo Decorator marking a test that requires PEFT. These tests are skipped when PEFT isn't installed. ztest requires PEFT)rrrRrrrr require_peftBrrcCr)z} Decorator marking a test that requires Torchvision. These tests are skipped when Torchvision isn't installed. ztest requires Torchvision)rrrrrrrrequire_torchvisionLrrcCttptd|S)z Decorator marking a test that requires PyTorch or TensorFlow. These tests are skipped when neither PyTorch not TensorFlow is installed. z#test requires PyTorch or TensorFlow)rrrmrirrrrrequire_torch_or_tfVsr cCr)z Decorator marking a test that requires Intel Extension for PyTorch. These tests are skipped when Intel Extension for PyTorch isn't installed or it does not match current PyTorch version. ztest requires Intel Extension for PyTorch to be installed and match current PyTorch version, see https://github.com/intel/intel-extension-for-pytorch)rrrCrrrr#require_intel_extension_for_pytorchbs r cCr)z Decorator marking a test that requires TensorFlow probability. These tests are skipped when TensorFlow probability isn't installed. z$test requires TensorFlow probability)rrrfrrrrrequire_tensorflow_probabilityqs r cCr)zu Decorator marking a test that requires torchaudio. These tests are skipped when torchaudio isn't installed. ztest requires torchaudio)rrr~rrrrrequire_torchaudio}rr cCr)zu Decorator marking a test that requires TensorFlow. These tests are skipped when TensorFlow isn't installed. ztest requires TensorFlow)rrrirrrr require_tfrr cCr)zv Decorator marking a test that requires JAX & Flax. These tests are skipped when one / both are not installed ztest requires JAX & Flax)rrr8rrrr require_flaxrrcCr){ Decorator marking a test that requires SentencePiece. These tests are skipped when SentencePiece isn't installed. ztest requires SentencePiece)rrr_rrrrrequire_sentencepiecerrcCr)zu Decorator marking a test that requires Sacremoses. These tests are skipped when Sacremoses isn't installed. ztest requires Sacremoses)rrr[rrrrrequire_sacremosesrrcCr)rztest requires Seqio)rrr`rrrr require_seqiorrcCr)zs Decorator marking a test that requires Scipy. These tests are skipped when SentencePiece isn't installed. ztest requires Scipy)rrr^rrrr require_scipyrrcCr)u Decorator marking a test that requires 🤗 Tokenizers. These tests are skipped when 🤗 Tokenizers isn't installed. ztest requires tokenizers)rrrlrrrrrequire_tokenizersrrcCr)z Decorator marking a test that requires tensorflow_text. These tests are skipped when tensroflow_text isn't installed. ztest requires tensorflow_text)rrrgrrrrrequire_tensorflow_textrrcCr)zs Decorator marking a test that requires keras_nlp. These tests are skipped when keras_nlp isn't installed. ztest requires keras_nlp)rrrGrrrrrequire_keras_nlprrcCr)zm Decorator marking a test that requires pandas. These tests are skipped when pandas isn't installed. ztest requires pandas)rrrQrrrrrequire_pandasrrcCr)zw Decorator marking a test that requires PyTesseract. These tests are skipped when PyTesseract isn't installed. ztest requires PyTesseract)rrrVrrrrrequire_pytesseractrrcCr)z Decorator marking a test that requires PyTorch Quantization Toolkit. These tests are skipped when PyTorch Quantization Toolkit isn't installed. z*test requires PyTorch Quantization Toolkit)rrrXrrrrrequire_pytorch_quantizations rcCr)z Decorator marking a test that requires the vision dependencies. These tests are skipped when torchaudio isn't installed. ztest requires vision)rrrrrrrrequire_visionrrcCr)zi Decorator marking a test that requires ftfy. These tests are skipped when ftfy isn't installed. ztest requires ftfy)rrr;rrrr require_ftfyrrcCr)zk Decorator marking a test that requires SpaCy. These tests are skipped when SpaCy isn't installed. ztest requires spacy)rrrbrrrr require_spacyrrcCs8ts tjdd|Sddl}t|jdkd|S)a Decorator marking a test that requires a multi-GPU CUDA setup (in PyTorch). These tests are skipped on a machine without multiple CUDA GPUs. To run *only* the multi_gpu tests, assuming all test names contain multi_gpu: $ pytest -sv ./tests -k "multi_gpu" rrrNrz test requires multiple CUDA GPUsrmrrtorchrcuda device_countrrrrrrequire_torch_multi_gpusr"cCs.ts tjdd|Stttdkd|S)a( Decorator marking a test that requires a multi-accelerator (in PyTorch). These tests are skipped on a machine without multiple accelerators. To run *only* the multi_accelerator tests, assuming all test names contain multi_accelerator: $ pytest -sv ./tests -k "multi_accelerator" rrrz#test requires multiple acceleratorsrmrrrbackend_device_count torch_devicerrrrrequire_torch_multi_accelerators r&cC8ts tjdd|Sddl}t|jdkd|S)zO Decorator marking a test that requires 0 or 1 GPU setup (in PyTorch). rrrNztest requires 0 or 1 GPUrr!rrrrequire_torch_non_multi_gpur)cC.ts tjdd|Stttdkd|S)zW Decorator marking a test that requires 0 or 1 accelerator setup (in PyTorch). rrr(z test requires 0 or 1 acceleratorr#rrrr#require_torch_non_multi_acceleratorsr,cCr')zT Decorator marking a test that requires 0 or 1 or 2 GPU setup (in PyTorch). rrrNztest requires 0 or 1 or 2 GPUsrr!rrrrequire_torch_up_to_2_gpus%r*r.cCr+)z\ Decorator marking a test that requires 0 or 1 or 2 accelerator setup (in PyTorch). rrr-z&test requires 0 or 1 or 2 acceleratorsr#rrrr"require_torch_up_to_2_accelerators1s r/cCr)zG Decorator marking a test that requires TorchXLA (in PyTorch). ztest requires TorchXLA)rrr{rrrrrequire_torch_xla=rr0cCsttddd|S)zI Decorator marking a test that requires NeuronCore (in PyTorch). F) check_devicez test requires PyTorch NeuronCore)rrrvrrrrrequire_torch_neuroncoreDsr2cCr)zB Decorator marking a test that requires NPU (in PyTorch). test requires PyTorch NPU)rrrwrrrrrequire_torch_npuMrr4cC0ts tjdd|Sttjdkd|S)a Decorator marking a test that requires a multi-NPU setup (in PyTorch). These tests are skipped on a machine without multiple NPUs. To run *only* the multi_npu tests, assuming all test names contain multi_npu: $ pytest -sv ./tests -k "multi_npu" r3rrztest requires multiple NPUs)rwrrrrnpur rrrrrequire_torch_multi_npuTr7cCttdkd|S)zB Decorator marking a test that should be skipped for HPU. hpuztest requires a non-HPUrrr%rrrrrequire_non_hpuar<cCr)aN Decorator marking a test that requires XPU (in PyTorch). These tests are skipped when XPU backend is not available. XPU backend might be available either via stock PyTorch (>=2.4) or via Intel Extension for PyTorch. In the latter case, if IPEX is installed, its version must match match current PyTorch version. ztest requires XPU device)rrr|rrrrrequire_torch_xpuhsr>cCr9)zB Decorator marking a test that should be skipped for XPU. xpuztest requires a non-XPUr;rrrrrequire_non_xpusr=r@cCr5)a Decorator marking a test that requires a multi-XPU setup (in PyTorch). These tests are skipped on a machine without multiple XPUs. To run *only* the multi_xpu tests, assuming all test names contain multi_xpu: $ pytest -sv ./tests -k "multi_xpu" ztest requires PyTorch XPUrrztest requires multiple XPUs)r|rrrrr?r rrrrrequire_torch_multi_xpuzr8rAcCr5)a Decorator marking a test that requires a multi-HPU setup (in PyTorch). These tests are skipped on a machine without multiple HPUs. To run *only* the multi_hpu tests, assuming all test names contain multi_hpu: $ pytest -sv ./tests -k "multi_hpu" ztest requires PyTorch HPUrrztest requires multiple HPUs)rtrrrrr:r rrrrrequire_torch_multi_hpur8rBTRANSFORMERS_TEST_BACKENDz.Failed to import `TRANSFORMERS_TEST_BACKEND` 'zf'! This should be the name of an installed module. The original error (look up to see its traceback): TRANSFORMERS_TEST_DEVICErzTRANSFORMERS_TEST_DEVICE=zH, but CUDA is unavailable. Please double-check your testing environment.r?zG, but XPU is unavailable. Please double-check your testing environment.r6zG, but NPU is unavailable. Please double-check your testing environment.mluzG, but MLU is unavailable. Please double-check your testing environment.r:zG, but HPU is unavailable. Please double-check your testing environment.zUUnknown testing device specified by environment variable `TRANSFORMERS_TEST_DEVICE`: cpucCr)z2Decorator marking a test that requires TorchDynamoztest requires TorchDynamo)rrrrrrrrequire_torchdynamorGcCr)z.Decorator marking a test that requires torchaoztest requires torchao)rrr}rrrrrequire_torchaorHrIcr)NcsBtotttjdjtk}t|dd|S)Ntorchaoz4Test requires torchao with the version greater than .)r}rparse importlibmetadata base_versionrr)rcorrect_torchao_versiontorchao_versionrrrs  z;require_torchao_version_greater_or_equal..decoratorr)rRrrrQr(require_torchao_version_greater_or_equals rScCr)z8Decorator marking a test that requires Torch-TensorRT FXztest requires Torch-TensorRT FX)rrryrrrrrequire_torch_tensorrt_fxrHrTcCsttdkd|S)z8Decorator marking a test that requires CUDA and PyTorch.rztest requires CUDAr;rrrrrequire_torch_gpusrUmemorycCsJtdkrtjd|dd|Sttjdjd|kd|d|S)zXDecorator marking a test that requires a CUDA GPU with more than `memory` GiB of memory.rz(test requires a CUDA GPU with more than z GiB of memoryrri@z#test requires a GPU with more than )r%rrrrrget_device_properties total_memory)rrWrrrrequire_torch_large_gpus rZcCstrtr|St|S)zl Decorator marking a test that requires a GPU if bitsandbytes multi-backend feature is not enabled. )r,r-rUrrrr2require_torch_gpu_if_bnb_not_multi_backend_enableds r[cCsttduo tdkd|S)zMDecorator marking a test that requires an accessible accelerator and PyTorch.NrFztest requires acceleratorr;rrrrrequire_torch_acceleratorsr\cCtttd|S)zBDecorator marking a test that requires a device that supports fp16z&test requires device with fp16 support)rrrrr%rrrrrequire_torch_fp16 r^cCttotd|S)z7Decorator marking a test that requires supports for fp8ztest requires fp8 support)rrr%rrrrr require_fp8sracCr])zBDecorator marking a test that requires a device that supports bf16z&test requires device with bf16 support)rrrnr%rrrrrequire_torch_bf16#r_rbcCr)zbDecorator marking a test that requires torch>=1.10, using Ampere GPU or newer arch with cuda>=11.0zItest requires torch>=1.10, using Ampere GPU or newer arch with cuda>=11.0)rrrprrrrrequire_torch_bf16_gpu* rccCr)z>Decorator marking a test that requires torch>=1.10, using CPU.z$test requires torch>=1.10, using CPU)rrrorrrrrequire_torch_bf16_cpu2rdrecCstr ttd|S|S)Nz3test requires torch to use deterministic algorithms)r|rrrqrrrrrequire_deterministic_for_xpu:s  rfcCr)z[Decorator marking a test that requires Ampere or a newer GPU arch, cuda>=11 and torch>=1.7.zAtest requires Ampere or a newer GPU arch, cuda>=11 and torch>=1.7)rrrzrrrrrequire_torch_tf32Cs rgcCr)z2Decorator marking a test that requires detectron2.ztest requires `detectron2`)rrr2rrrrrequire_detectron2JrHrhcCr)z-Decorator marking a test that requires faiss.ztest requires `faiss`)rrr5rrrr require_faissOrHricCr)zs Decorator marking a test that requires optuna. These tests are skipped when optuna isn't installed. ztest requires optuna)rrrrrrrrequire_optunaTrrjcCr)zw Decorator marking a test that requires Ray/tune. These tests are skipped when Ray/tune isn't installed. ztest requires Ray/tune)rrrrrrr require_ray^rrkcCr)zs Decorator marking a test that requires SigOpt. These tests are skipped when SigOpt isn't installed. ztest requires SigOpt)rrrrrrrrequire_sigopthrrlcCr)zu Decorator marking a test that requires swanlab. These tests are skipped when swanlab isn't installed. ztest requires swanlab)rrrrrrrrequire_swanlabrrrmcCr)zq Decorator marking a test that requires wandb. These tests are skipped when wandb isn't installed. ztest requires wandb)rrr!rrrr require_wandb|rrncCr)zp Decorator marking a test requires clearml. These tests are skipped when clearml isn't installed. ztest requires clearml)rrrrrrrrequire_clearmlrrocCr)zx Decorator marking a test that requires soundfile These tests are skipped when soundfile isn't installed. ztest requires soundfile)rrrarrrrrequire_soundfilerrpcCr)z: Decorator marking a test that requires deepspeed ztest requires deepspeed)rrr"rrrrrequire_deepspeedrrqcCr)z5 Decorator marking a test that requires apex ztest requires apex)rrr&rrrr require_apexrrrcCr)z5 Decorator marking a test that requires aqlm ztest requires aqlm)rrr(rrrr require_aqlmrrscCr)z5 Decorator marking a test that requires vptq ztest requires vptq)rrrrrrr require_vptqrrtcCr)z5 Decorator marking a test that requires spqr ztest requires spqr)rrrcrrrr require_spqrrruc Cs\t}|r&zddl}Wnty%}zdt|vrd}WYd}~nd}~wwt|d|S)z5 Decorator marking a test that requires eetq rNshard_checkpointFztest requires eetq)r3eetqrstrrr)reetq_availablerwexcrrr require_eetqs  r{cCr)z3 Decorator marking a test that requires av ztest requires av)rrr+rrrr require_avrr|cCsJtrtrz ddl}|j|WSty|YSwtjdd|S)z Decorator marking a test that requires the bitsandbytes library. Will be skipped when the library or its hard dependency torch is not installed. rNz$test requires bitsandbytes and torchr)r,rmrr bitsandbytesrrrrrrrrequire_bitsandbytess  r~cCr)z* Decorator for optimum dependency ztest requires optimum)rrrOrrrrrequire_optimumrrcCsttdS)z0 Decorator for `tensorboard` dependency ztest requires tensorboard)rrr rrrrrequire_tensorboardsrcCr)z, Decorator for auto_gptq dependency z$test requires gptqmodel or auto-gptq)rrr?r*rrrr require_gptq  rcCr)z& Decorator for hqq dependency ztest requires hqq)rrrBrrrr require_hqqrrcCr)z+ Decorator for auto_awq dependency ztest requires autoawq)rrr)rrrrrequire_auto_awqrrcCr)z) Decorator for quanto dependency ztest requires optimum-quanto)rrrPrrrrrequire_optimum_quanto rrcCr)z5 Decorator for compressed_tensors dependency z test requires compressed_tensors)rrr/rrrrrequire_compressed_tensorsrrcCr)z- Decorator for fbgemm_gpu dependency ztest requires fbgemm-gpu)rrr6rrrrrequire_fbgemm_gpurrcCr)z( Decorator for quark dependency ztest requires quark)rrrYrrrr require_quarkrrcCr`)zC Decorator marking a test that requires higgs and hadamard z/test requires flute and fast_hadamard_transform)rrr9rArrrrrequire_flute_hadamard%rrcCr)z; Decorator marking a test that requires phonemizer ztest requires phonemizer)rrrSrrrrrequire_phonemizer.rrcCr)z< Decorator marking a test that requires pyctcdecode ztest requires pyctcdecode)rrrUrrrrrequire_pyctcdecode5rrcCr)z8 Decorator marking a test that requires librosa ztest requires librosa)rrrIrrrrrequire_librosa<rrcCr)z= Decorator marking a test that requires liger_kernel ztest requires liger_kernel)rrrJrrrrrequire_liger_kernelCrrcCr)z9 Decorator marking a test that requires essentia ztest requires essentia)rrr4rrrrrequire_essentiaJrrcCr)z< Decorator marking a test that requires pretty_midi ztest requires pretty_midi)rrrTrrrrrequire_pretty_midiQrrcCst|duSN)shutilwhich)cmdrrr cmd_existsXrcCsttdd|S)z@ Decorator marking a test that requires `/usr/bin/time` /usr/bin/timeztest requires /usr/bin/time)rrrrrrrrequire_usr_bin_time\r=rcCr)z8 Decorator marking a test that requires sudachi ztest requires sudachi)rrrdrrrrrequire_sudachicrrcCr)zC Decorator marking a test that requires sudachi_projection z/test requires sudachi which supports projection)rrrerrrrrequire_sudachi_projectionjs rcCr)8 Decorator marking a test that requires jumanpp ztest requires jumanpp)rrrFrrrrrequire_jumanppsrrcCr)rztest requires cython)rrr1rrrrrequire_cythonzrrcCr)zq Decorator marking a test that requires TikToken. These tests are skipped when TikToken isn't installed. ztest requires TikToken)rrrjrrrrrequire_tiktokenrrcCsPtr ddl}|jStrddl}t|jdSt r&ddl }|SdS)z^ Return the number of available gpus (regardless of whether torch, tf or jax is used) rNGPU) rmrrr ri tensorflowlenconfiglist_physical_devicesr8jax)rtfrrrr get_gpu_counts rcCsZtdd}tjtj|}|ds"tj|}|dr|r+tj||S|S)a Args: append_path: optional path to append to the tests dir path Return: The full path to the `tests` dir, so that the tests can be invoked from anywhere. Optionally `append_path` is joined after the `tests` dir the former is provided. rtests)inspectstackrpathabspathdirnameendswithjoin) append_pathcaller__file__ tests_dirrrr get_tests_dirs    rtrainerreturncCs,|j}|}|j|||jd}|d}|S)N)r dataloadertotal_train_batch_sizer)rget_train_dataloaderset_initial_training_valuesper_device_train_batch_size)r training_argstrain_dataloaderinitial_training_valuessteps_per_epochrrrget_steps_per_epochsrside_effect_valuesccs|EdH |dVq)z Function that returns side effects for the _evaluate method. Used when we're unsure of exactly how many times _evaluate will be called. NTr)rrrrevaluate_side_effect_factorys   rcCstdd|dtjS)Nz^.*\rr)resubM)bufrrrapply_print_resetsrcCs:t|}||}|dksJd|d|dS)Nrzexpecting to find z in output: f)rlowerfind)outwhatout_pr match_strrrrassert_screenouts  rcCs&d}gd}|jjj|vrd}t|jdst|jdrd}t|jdr+|dur+||j_t|jdrWd|jjvrW|durWt|jj|j_t|jjt rRd|jjd<nd|jj_t|jdrd|jj vr|durt|jj |j_ t|jj t r~d|jj d<nd|jj _t|jdr|jj|j_ dSdS) Nr)ZambaModelTesterZamba2ModelTesterRwkvModelTesterAriaVisionText2TextModelTesterGPTNeoModelTesterDPTModelTester out_features out_indicesnum_hidden_layers vision_config text_confignum_hidden_groups) model_tester __class____name__hasattrrrcopydeepcopy isinstancedictrr)rtarget_num_hidden_layersexceptional_classesrrr$set_model_tester_for_less_flaky_tests6       rcCsZgd}|D]}t||dqgd}|D]}t||r*|D] }tt|||dqqdS)N) rms_norm_epslayer_norm_epsnorm_eps norm_epsilonlayer_norm_epsilonbatch_norm_eps?)rr text_encoder audio_encoderdecoder)rrr)r target_attrs target_attrattrsrrrrset_config_for_less_flaky_tests rcCsjd}gd}tr/t|tjjr1|D]}t|j|r.|D] }t ||r-t ||dq qdSdSdS)N) LayerNorm GroupNorm BatchNormRMSNorm BatchNorm2d BatchNorm1dBitGroupNormActivationWeightStandardizedConv2d)epsepsilonvariance_epsilonr) rmrrnnrmodulestyperrrr)model target_namesrmodulerrrrset_model_for_less_flaky_test*s    rc@s2eZdZdZd ddZddZddZd d Zd S) CaptureStdav Context manager to capture: - stdout: replay it, clean it up and make it available via `obj.out` - stderr: replay it and make it available via `obj.err` Args: out (`bool`, *optional*, defaults to `True`): Whether to capture stdout or not. err (`bool`, *optional*, defaults to `True`): Whether to capture stderr or not. replay (`bool`, *optional*, defaults to `True`): Whether to replay or not. By default each captured stream gets replayed back on context's exit, so that one can see what the test was doing. If this is a not wanted behavior and the captured data shouldn't be replayed, pass `replay=False` to disable this feature. Examples: ```python # to capture stdout only with auto-replay with CaptureStdout() as cs: print("Secret message") assert "message" in cs.out # to capture stderr only with auto-replay import sys with CaptureStderr() as cs: print("Warning: ", file=sys.stderr) assert "Warning" in cs.err # to capture both streams with auto-replay with CaptureStd() as cs: print("Secret message") print("Warning: ", file=sys.stderr) assert "message" in cs.out assert "Warning" in cs.err # to capture just one of the streams, and not the other, with auto-replay with CaptureStd(err=False) as cs: print("Secret message") assert "message" in cs.out # but best use the stream-specific subclasses # to capture without auto-replay with CaptureStd(replay=False) as cs: print("Secret message") assert "message" in cs.out ```TcCsL||_|r t|_d|_nd|_d|_|rt|_d|_dSd|_d|_dS)Nz=error: CaptureStd context is unfinished yet, called too earlyznot capturing stdoutznot capturing stderr)replayr out_bufrerr_buferr)selfrr rrrr__init__ps  zCaptureStd.__init__cCs0|jr tj|_|jt_|jrtj|_|jt_|Sr)rsysstdoutout_oldr stderrerr_oldr rrr __enter__szCaptureStd.__enter__cGsl|jr|jt_|j}|jrtj|t||_|j r4|j t_ |j }|jr/tj |||_ dSdSr) rrr rgetvaluerwriterrr rrr )r rzcapturedrrr__exit__s      zCaptureStd.__exit__cCs8d}|jr|d|jd7}|jr|d|jd7}|S)Nrzstdout:  stderr: )rrr r )r msgrrr__repr__s zCaptureStd.__repr__N)TTTr __module__ __qualname____doc__r rrrrrrrr?s  0 rc"eZdZdZdfdd ZZS) CaptureStdoutz+Same as CaptureStd but captures only stdoutTctjd|ddS)NF)r rsuperr r rrrrr rzCaptureStdout.__init__Trrrrr  __classcell__rrr&rr!r!cr ) CaptureStderrz+Same as CaptureStd but captures only stderrTcr")NF)rrr#r%r&rrr rzCaptureStderr.__init__r'r(rrr&rr+r*r+c@s0eZdZdZddZddZddZdd Zd S) CaptureLoggera: Context manager to capture `logging` streams Args: logger: 'logging` logger object Returns: The captured output is available via `self.out` Example: ```python >>> from transformers import logging >>> from transformers.testing_utils import CaptureLogger >>> msg = "Testing 1, 2, 3" >>> logging.set_verbosity_info() >>> logger = logging.get_logger("transformers.models.bart.tokenization_bart") >>> with CaptureLogger(logger) as cl: ... logger.info(msg) >>> assert cl.out, msg + " " ``` cCs&||_t|_t|j|_d|_dS)Nr)loggerr ior StreamHandlershr)r r-rrrr s zCaptureLogger.__init__cCs|j|j|Sr)r- addHandlerr0rrrrrszCaptureLogger.__enter__cGs|j|j|j|_dSr)r- removeHandlerr0r.rr)r rzrrrrszCaptureLogger.__exit__cCsd|jdS)Nz captured: r)rrrrrrrzCaptureLogger.__repr__Nrrrrrr,s  r,c cs8t}zt|dVWt|dSt|w)ae This is a context manager to temporarily change transformers modules logging level to the desired value and have it restored to the original setting at the end of the scope. Example: ```python with LoggingLevel(logging.INFO): AutoModel.from_pretrained("openai-community/gpt2") # calls logger.info() several times ``` N)transformers_logging get_verbosity set_verbosity)level orig_levelrrr LoggingLevels   r8c@s@eZdZdZd deedeeddfddZdd Zd d ZdS) TemporaryHubRepoaCreate a temporary Hub repository and return its `RepoUrl` object. This is similar to `tempfile.TemporaryDirectory` and can be used as a context manager. For example: with TemporaryHubRepo(token=self._token) as temp_repo: ... Upon exiting the context, the repository and everything contained in it are removed. Example: ```python with TemporaryHubRepo(token=self._token) as temp_repo: model.push_to_hub(tmp_repo.repo_id, token=self._token) ``` N namespacerrcCsf||_t"}t|j}|dur|d|}tj||jd|_WddS1s,wYdS)N/)r)rtempfileTemporaryDirectoryr namehuggingface_hub create_reporepo_url)r r:rtmp_dirrepo_idrrrr  s  "zTemporaryHubRepo.__init__cC|jSr)rArrrrrszTemporaryHubRepo.__enter__cCst|jj|jdddS)NT)rCr missing_ok)rrArCr)r rzrtbrrrrszTemporaryHubRepo.__exit__NN) rrrrrrxr rrrrrrr9s   r9rc csBt|}ztjd|dVWtj|dStj|w)z Temporary add given path to `sys.path`. Usage : ```python with ExtendSysPath("/path/to/dir"): mymodule = importlib.import_module("mymodule") ``` rN)rfspathr rinsertremove)rrrr ExtendSysPaths  rKc@seZdZdZddZeddZeddZedd Zed d Z ed d Z eddZ eddZ eddZ eddZeddZeddZeddZddZd%dd Zd!d"Zd#d$ZdS)& TestCasePlusa This class extends *unittest.TestCase* with additional features. Feature 1: A set of fully resolved important file and dir path accessors. In tests often we need to know where things are relative to the current test file, and it's not trivial since the test could be invoked from more than one directory or could reside in sub-directories with different depths. This class solves this problem by sorting out all the basic paths and provides easy accessors to them: - `pathlib` objects (all fully resolved): - `test_file_path` - the current test file path (=`__file__`) - `test_file_dir` - the directory containing the current test file - `tests_dir` - the directory of the `tests` test suite - `examples_dir` - the directory of the `examples` test suite - `repo_root_dir` - the directory of the repository - `src_dir` - the directory of `src` (i.e. where the `transformers` sub-dir resides) - stringified paths---same as above but these return paths as strings, rather than `pathlib` objects: - `test_file_path_str` - `test_file_dir_str` - `tests_dir_str` - `examples_dir_str` - `repo_root_dir_str` - `src_dir_str` Feature 2: Flexible auto-removable temporary dirs which are guaranteed to get removed at the end of test. 1. Create a unique temporary dir: ```python def test_whatever(self): tmp_dir = self.get_auto_remove_tmp_dir() ``` `tmp_dir` will contain the path to the created temporary dir. It will be automatically removed at the end of the test. 2. Create a temporary dir of my choice, ensure it's empty before the test starts and don't empty it after the test. ```python def test_whatever(self): tmp_dir = self.get_auto_remove_tmp_dir("./xxx") ``` This is useful for debug when you want to monitor a specific directory and want to make sure the previous tests didn't leave any data in there. 3. You can override the first two options by directly overriding the `before` and `after` args, leading to the following behavior: `before=True`: the temporary dir will always be cleared at the beginning of the test. `before=False`: if the temporary dir already existed, any existing files will remain there. `after=True`: the temporary dir will always be deleted at the end of the test. `after=False`: the temporary dir will always be left intact at the end of the test. Note 1: In order to run the equivalent of `rm -r` safely, only subdirs of the project repository checkout are allowed if an explicit `tmp_dir` is used, so that by mistake no `/tmp` or similar important part of the filesystem will get nuked. i.e. please always pass paths that start with `./` Note 2: Each test can register multiple temporary dirs and they all will get auto-removed, unless requested otherwise. Feature 3: Get a copy of the `os.environ` object that sets up `PYTHONPATH` specific to the current test suite. This is useful for invoking external programs from the test suite - e.g. distributed training. ```python def test_whatever(self): env = self.get_env() ```cCsg|_t|j|_t|j}|jd|_dD]}|j|}|d r.|d r.nq|r5||_ nt d|j|j d|_ |j d|_ |j d|_dS)Nr)rr(r-srcrz+can't figure out the root of the repo from examples)teardown_tmp_dirsrgetfiler_test_file_pathr resolveparents_test_file_diris_dir_repo_root_dirr _tests_dir _examples_dir_src_dir)r ruprBrrrsetUp}s    zTestCasePlus.setUpcCrDr)rQrrrrtest_file_pathzTestCasePlus.test_file_pathcC t|jSr)rxrQrrrrtest_file_path_str zTestCasePlus.test_file_path_strcCrDr)rTrrrr test_file_dirr]zTestCasePlus.test_file_dircCr^r)rxrTrrrrtest_file_dir_strr`zTestCasePlus.test_file_dir_strcCrDr)rWrrrrrr]zTestCasePlus.tests_dircCr^r)rxrWrrrr tests_dir_strr`zTestCasePlus.tests_dir_strcCrDr)rXrrrr examples_dirr]zTestCasePlus.examples_dircCr^r)rxrXrrrrexamples_dir_strr`zTestCasePlus.examples_dir_strcCrDr)rVrrrr repo_root_dirr]zTestCasePlus.repo_root_dircCr^r)rxrVrrrrrepo_root_dir_strr`zTestCasePlus.repo_root_dir_strcCrDr)rYrrrrsrc_dirr]zTestCasePlus.src_dircCr^r)rxrYrrrr src_dir_strr`zTestCasePlus.src_dir_strcCsZtj}|jg}d|jvr||jn||j||ddd ||d<|S)a Return a copy of the `os.environ` object that sets up `PYTHONPATH` correctly, depending on the test suite it's invoked from. This is useful for invoking external programs from the test suite - e.g. distributed training. It always inserts `./src` first, then `./tests` or `./examples` depending on the test suite type and finally the preset `PYTHONPATH` if any (all full resolved paths). z /examples PYTHONPATHr:) rrrrirbappendrercgetr)r envpathsrrrget_envs   zTestCasePlus.get_envNcCs|dur:|dur d}|durd}t|}|ds#td|d|dur2|r2tj|dd|jdddn|dur@d}|durFd}t }|durT|j ||S) a Args: tmp_dir (`string`, *optional*): if `None`: - a unique temporary path will be created - sets `before=True` if `before` is `None` - sets `after=True` if `after` is `None` else: - `tmp_dir` will be created - sets `before=True` if `before` is `None` - sets `after=False` if `after` is `None` before (`bool`, *optional*): If `True` and the `tmp_dir` already exists, make sure to empty it right away if `False` and the `tmp_dir` already exists, any existing files will remain there. after (`bool`, *optional*): If `True`, delete the `tmp_dir` at the end of the test if `False`, leave the `tmp_dir` and its contents intact at the end of the test. Returns: tmp_dir(`string`): either the same value as passed via *tmp_dir* or the path to the auto-selected tmp dir NTFz./zI`tmp_dir` can only be a relative path, i.e. `./some/path`, but received `` ignore_errorsrSexist_ok) r rRrrexistsrrmtreemkdirr<mkdtemprOrl)r rBbeforeafterrrrrget_auto_remove_tmp_dirs*    z$TestCasePlus.get_auto_remove_tmp_dircCs|tdstdtd|d}t}t||dWdn1s'wYt|jdd d d d }|S) a Runs the passed python one liner (just the code) and returns how much max cpu memory was used to run the program. Args: one_liner_str (`string`): a python one liner code that gets passed to `python -c` Returns: max cpu memory bytes used to run the program. This value is likely to vary slightly from run to run. Requirements: this helper needs `/usr/bin/time` to be installed (`apt install time`) Example: ``` one_liner_str = 'from transformers import AutoModel; AutoModel.from_pretrained("google-t5/t5-large")' max_rss = self.python_one_liner_max_rss(one_liner_str) ``` rz:/usr/bin/time is required, install with `apt install time`z/usr/bin/time -f %M python -c ''rnNrrri) rrshlexsplitrexecute_subprocess_asyncrprr replace)r  one_liner_strrcsmax_rssrrrpython_one_liner_max_rsss z%TestCasePlus.python_one_liner_max_rsscCsb|jD] }tj|ddqg|_tr-ttttj D] }d|vr,tj |=q"dSdS)NTrr ACCELERATE) rOrrwr%r _reset_staterlistrrkeys)r rkrrrtearDown:s zTestCasePlus.tearDown)NNN)rrrrr[propertyr\r_rarbrrcrdrerfrgrhrirpr|rrrrrrrL.s>N             D !rLcKstjtj|S)z this is a convenience wrapper, that allows this :: @mockenv(RUN_SLOW=True, USE_TF=False) def test_something(): run_slow = os.getenv("RUN_SLOW", False) use_tf = os.getenv("USE_TF", False) )rrrrr)rrrrmockenvIrrc /stj|pi}|p g}t|t|Bt@}fdd|D}tfdd|D}z"|fdd|DdVW|fdd|DdS|fdd|Dw) aM Temporarily updates the `os.environ` dictionary in-place. Similar to mockenv The `os.environ` dictionary is updated in-place so that the modification is sure to work in all situations. Args: remove: Environment variables to remove. update: Dictionary of environment variables and values to add/update. csi|]}||qSrr.0rr~rr gz#mockenv_context..c3s|] }|vr|VqdSrrrr~rr isz"mockenv_context..csg|]}|dqSrpoprr~rr mz#mockenv_context..Ncsg|]}|qSrrrr~rrrqr)rrsetr frozensetupdate)rJrstomped update_after remove_afterrr~rmockenv_contextUs     rcCs.d}|tvr|j|dddddt|<dSdS)z This function is to be called from `conftest.py` via `pytest_addoption` wrapper that has to be defined there. It allows loading both `conftest.py` files at once without causing a failure due to adding the same `pytest` option. z--make-reportsstoreFzSgenerate report files. The value of this option is used as a prefix to report names)actionrhelprN)pytest_opt_registered addoption)parseroptionrrrpytest_addoption_sharedzs rc s`ddlm}t|s d}|j}|}|jj}|j}d|tj dddfddd D}g}|j D]} | D] } t | d rH| | q.) durationserrors failures_longfailures_short failures_linepassesstats summary_shortwarningsdurationcSrDr)rxrrrsz.pytest_terminal_summary_main..)rreverserwg?zslowest durations z durations < z secs were omittedz02.2fzs z<8 rNc Ssp|d}|s dS|dd|D]$}||}|jd|dddtdd|jd tjtjB}|j |qdS) Nfailed=zFAILURES SHORT STACK_T)redboldz.*_ _ _ (_ ){10,}_ _ rr) getreports write_sep_getfailureheadlinerr longreprtextrS_twline)trreportsreprlongreprrrrsummary_failures_shorts   z.summary_failures_shortautorrrrrrwPpsxXEfrr)_pytest.configrrrget_terminal_writerrtbstyle reportcharsr rxrvaluesrrlsortopenr enumeraterwhennodeidrsummary_failuressummary_errorssummary_warningsshort_test_summary summary_stats)ridrr orig_writer orig_tbstyleorig_reportchars report_filesdlistreplistrf durations_minirrrrpytest_terminal_summary_mains       (                rc@seZdZddZdS) _RunOutputcCs||_||_||_dSr) returncoderr)r rrrrrrr  s z_RunOutput.__init__N)rrrr rrrrr s rcs& |IdH}|r||ndSqr)readline)streamcallbackrrrr _read_stream s rc s|r tdd|tj|dg|ddR|tjjtjj|dIdH}ggd fdd tjtt|j fd d tt|j fd d g|d IdHt |IdHS)Nz Running: rrr)stdinrrrnrcs2|d}||st|||ddSdS)Nutf-8)file)decoderstriprlprint)rsinkpipelabel)quietrrtee@ s  z_stream_subprocess..teec|tjddS)Nzstdout:r)r rl)rrrrrI z$_stream_subprocess..cr)Nzstderr:r)r rr)r rrrrJ rtimeout)r) rrasynciocreate_subprocess_exec subprocessPIPEwait create_taskrrrr)rrnrrrechopr)r rrrr_stream_subprocess( s.  rc Cs~t}|t||||||d}d|}|jdkr/d|j} td|d|jd| |js=|js=td|d|S) N)rnrrrrrrrr}z' failed with returncode z, The combined stderr from workers follows: z' produced no output.) rget_event_looprun_until_completerrrr RuntimeErrorr) rrnrrrrloopresultcmd_strrrrrrQ s    rcCs*tjdd}tdd|dtj}t|S)z Returns an int value of worker's numerical id under `pytest-xdist`'s concurrent workers `pytest -n N` regime, or 0 if `-n 1` or `pytest-xdist` isn't being used. PYTEST_XDIST_WORKERgw0z^gwrr)rrrmrrrr)workerrrrpytest_xdist_worker_idg srcCsd}t}||S)z Returns a port number that can be fed to `torch.distributed.launch`'s `--master_port` argument. Under `pytest-xdist` it adds a delta number based on a worker id so that concurrent tests don't try to use the same port at once. ifdd|DSt|t t |j frI|S|durO|St r_t|t jr_t|Strot|rot|St|tryt|St||j|j|jfrt|Stdt|) z Simplifies an object by rounding float numbers, and downcasting tensors/numpy arrays to get simple equality test within tests. rNcg|]}t|qSrnested_simplifyritemdecimalsrrr rz#nested_simplify..crrrrrrrr rcs"i|] \}}t|t|qSrr)rrvrrrr s"z#nested_simplify..zNot supported: )numpyrrtuplendarrayrtolistritemsrxrint64rmrTensorrir is_tensorfloatroundint32float32float16r Exceptionr)objrnprrrr} s,       rcCst|]}|}t|dkr|ddksJn9t|dks J|ddks*J|ddD]}t|dt|d}|dksFJq0|ddksYJWddSWddS1sdwYdS) Nrrz{}r-{rr(})r readlinesrstriplstrip) file_pathrlinesr left_indentrrr"check_json_file_has_correct_format s   "r6cCst|tjjr |S||fSr)r collectionsabcrrrrr to_2tuple sr9c@s eZdZdS)SubprocessCallExceptionN)rrrrrrrr: sr:commandc Csrztj|tjd}|rt|dr|d}|WSWdStjy8}ztdd|d|j|d}~ww)z Runs `command` with `subprocess.check_output` and will potentially return the `stdout`. Will also properly capture if an error occurred while running `command` )rrrz Command `rz$` failed with the following error: N) r check_outputSTDOUTrrCalledProcessErrorr:routput)r; return_stdoutr?rrrr run_command s  rAc@sFeZdZdZddZd ddZdedefd d Ze defd d Z dS)RequestCountera Helper class that will count all requests made online. Might not be robust if urllib3 changes its logging format but should be good enough for us. Usage: ```py with RequestCounter() as counter: _ = AutoTokenizer.from_pretrained("hf-internal-testing/tiny-random-bert") assert counter["GET"] == 0 assert counter["HEAD"] == 1 assert counter.total_calls == 1 ``` csVtt_t_g_fdd}tjt j j d|t j j j d_ j _S)Ncsfdd}|S)Ncsjt|i|Sr) _extra_inforl threading get_identr)funcr rrwrap szHRequestCounter.__enter__..patched_with_thread_info..wrapr)rFrGr)rFrpatched_with_thread_info sz:RequestCounter.__enter__..patched_with_thread_infodebug) side_effect)rr_counterrDrE _thread_idrCrobjecturllib3connectionpoollogrIpatcherstartr)r rHrrrr s    zRequestCounter.__enter__rNcOst|jjt|jks Jt|j|jjD]2\}}||jkrq|jddkr'q|jd|jdd}dD]}||vrF|j|d7<nq5q|j dS)Nri3rr) HEADGETPOSTPUTDELETECONNECTOPTIONSTRACEPATCH) rrcall_args_listrCziprLrrKrQstop)r rr thread_idcallrPmethodrrrr s zRequestCounter.__exit__rcCs |j|Sr)rK)r rrrr __getitem__  zRequestCounter.__getitem__cCst|jSr)sumrKrrrrr total_calls szRequestCounter.total_callsrN) rrrrrrrxrrbrrerrrrrB s rB max_attemptswait_before_retry descriptioncsfdd}|S)a To decorate flaky tests. They will be retried on failures. Please note that our push tests use `pytest-rerunfailures`, which prompts the CI to rerun certain types of failed tests. More specifically, if the test exception contains any substring in `FLAKY_TEST_FAILURE_PATTERNS` (in `.circleci/create_circleci_config.py`), it will be rerun. If you find a recurrent pattern of failures, expand `FLAKY_TEST_FAILURE_PATTERNS` in our CI configuration instead of using `is_flaky`. Args: max_attempts (`int`, *optional*, defaults to 5): The maximum number of attempts to retry the flaky test. wait_before_retry (`float`, *optional*): If provided, will wait that number of seconds before retrying the test. description (`str`, *optional*): A string to describe the situation (what / where / why is flaky, link to GH issue/PR comments, errors, etc.) ctfdd}|S)Nc sd}|krAz|i|WSty<}z"td|d|dddur.t|d7}WYd}~nd}~ww|ks|i|S)NrTest failed with  at try r;rK)r+r-errortimesleeprr retry_countr rh test_func_refrirrr s  z,is_flaky..decorator..wrapperrrtrrhrirtrr szis_flaky..decoratorr)rhrirjrrrvris_flaky srxr(csfdd}|S)a To decorate tests that download from the Hub. They can fail due to a variety of network issues such as timeouts, connection resets, etc. Args: max_attempts (`int`, *optional*, defaults to 5): The maximum number of attempts to retry the flaky test. wait_before_retry (`float`, *optional*, defaults to 2): If provided, will wait that number of seconds before retrying the test. crk)Nc sd}|krPz|i|WStjjtjjtjjtjjtjjfyK}z"td|d|dddur=t |d7}WYd}~nd}~ww|ks|i|S)Nrrlrmr;z8 as it couldn't connect to the specified Hub repository.) requests exceptionsConnectionErrorTimeout ReadTimeout HTTPErrorRequestExceptionr-rnrorprqrsrrr8 s*  z-hub_retry..decorator..wrapperrrurvrwrr7 szhub_retry..decoratorr)rhrirrrvr hub_retry+ s rcCsddl}|jd|S)a Decorator marking a test with order(1). When pytest-order plugin is installed, tests marked with this decorator are garanteed to run first. This is especially useful in some test settings like on a Gaudi instance where a Gaudi device can only be used by a single process at a time. So we make sure all tests that run in a subprocess are launched first, to avoid device allocation conflicts. rNr)rrorderrrrr run_firstU s rc Cs|dur ttjdd}d}t|}|d}|d}|j||d|j ||||fd}| z |j|d} | Wnt yZ} z| || WYd} ~ nd} ~ ww|j|d| ddurq|| ddSdS) a To run a test in a subprocess. In particular, this can avoid (GPU) memory issue. Args: test_case (`unittest.TestCase`): The test that will run `target_func`. target_func (`Callable`): The function implementing the actual testing logic. inputs (`dict`, *optional*, defaults to `None`): The inputs that will be passed to `target_func` through an (input) queue. timeout (`int`, *optional*, defaults to `None`): The timeout (in seconds) that will be passed to the input and output queues. If not specified, the env. variable `PYTEST_TIMEOUT` will be checked. If still `None`, its value will be set to `600`. NPYTEST_TIMEOUTiXspawnrr)targetrrn)rrrrmmultiprocessing get_contextQueue JoinableQueueputProcessrR task_doner+ terminatefailr) r target_funcinputsr start_methohdctx input_queue output_queueprocessresultsrrrrrun_test_in_subprocessc s*       rcs$ddltfdd}|S)z To decorate a test to run in a subprocess using the `subprocess` module. This could avoid potential GPU memory issues (GPU OOM or a test that causes many subsequential failing with `CUDA error: device-side assert triggered`). rNc stdddkr|i|dSdtjdddd}zkddl}|tj}d|d<d|d<d |vrxt|d j j }t |D]"\}}||d j vrf|d d d}d j d g|||<qDt jd dg|}dd|D}n t jd d|g}tj||dddWdStjy}zg|j} | d} d| dvrd} | d dD]9} | dr| tdd} d| dd d} q| dr| drd| vrnt| dkr| d| 7} qd| } | g| } d| } j| ddd}~ww)N_INSIDE_SUB_PROCESS1rPYTEST_CURRENT_TESTrrtrueCI pytestconfigz::r__file__z-mrcSsg|]}|dvr|qS))z --no-summaryr)rrrrrr rz>run_test_using_subprocess..wrapper..T)rncheckcapture_outputrz= test session starts =rzFAILED z - rz failed in z (subprocess) F)pytrace)rrrrrmrrrrinvocation_paramsrr __globals__r  executablerrunr>rrrrrr) rrtestrrnr;idxrrexception_messager4textrrFrrrr sN         z*run_test_using_subprocess..wrapper)rrr )rFrrrrrun_test_using_subprocess s1rcCsd}tj||tjd}d}t|D])\}}d|vr&d|vr&tdd|||<d|vs.d |vr:td |r:|r:d }nqd }|sDd |}|S) aPrepare a docstring or a `.md` file to be run by doctest. The argument `string` would be the whole file content if it is a `.md` file. For a python file, it would be one of its docstring. In each case, it may contain multiple python code examples. If `skip_cuda_tests` is `True` and a cuda stuff is detective (with a heuristic), this method will return an empty string so no doctest will be run for `string`. z&(```(?:python|py)\s*\n\s*>>> )(.*?```))flagsFz load_dataset(z# doctest: +IGNORE_RESULTz(>>> .*load_dataset\(.*)z\1 # doctest: +IGNORE_RESULTz>>>z...zcuda|to\(0\)|device=0Tr)rrDOTALLrrsearchr)stringskip_cuda_testscodeblock_pattern codeblocks is_cuda_foundr codeblockmodified_stringrrrpreprocess_string s$  rcsReZdZUdZedejejBZe e j ddZ e ed<d fdd ZZS) HfDocTestParsera Overwrites the DocTestParser from doctest to properly parse the codeblocks that are formatted with black. This means that there are no extra lines at the end of our snippets. The `# doctest: +IGNORE_RESULT` marker is also added anywhere a `load_dataset` call is made as a print would otherwise fail the corresponding line. Tests involving cuda are skipped base on a naive pattern that should be updated if it is not enough. a # Source consists of a PS1 line followed by zero or more PS2 lines. (?P (?:^(?P [ ]*) >>> .*) # PS1 line (?:\n [ ]* \.\.\. .*)*) # PS2 lines \n? # Want consists of any non-blank lines that do not start with PS1. (?P (?:(?![ ]*$) # Not a blank line (?![ ]*>>>) # Not a line starting with PS1 # !!!!!!!!!!! HF Specific !!!!!!!!!!! (?:(?!```).)* # Match any character except '`' until a '```' is found (this is specific to HF because black removes the last line) # !!!!!!!!!!! HF Specific !!!!!!!!!!! (?:\n|$) # Match a new line or end of string )*) SKIP_CUDA_DOCTESTFrcst||j}t||S)z Overwrites the `parse` method to incorporate a skip for CUDA tests, and remove logs and dataset prints before calling `super().parse` )rrr$rL)r rr>r&rrrL s zHfDocTestParser.parse)r)rrrrrcompile MULTILINEVERBOSE _EXAMPLE_REboolrrrmr__annotations__rLr)rrr&rr s  rc@s"eZdZdZdeefddZdS)HfDoctestModulez Overwrites the `DoctestModule` of the pytest package to make sure the HFDocTestParser is used when discovering tests. rccsGdddtj}|jjdkr"|jjj|j|jd|jjd}n)zt |j|jj|jdd}Wnt yJ|j drGt d|jnYnw|t d }t|}td |tt|jd }|||jD]}|jrxtj||j||d VqhdS) Ncs.eZdZdZfddZdfdd ZZS) z7HfDoctestModule.collect..MockAwareDocTestFinderzA hackish doctest finder that overrides stdlib internals to fix a stdlib bug. https://github.com/pytest-dev/pytest/issues/3456 https://bugs.python.org/issue25532 cs8t|tr t|d|}t|drt|}t||S)a3Doctest code does not take into account `@property`, this is a hackish way to fix it. https://bugs.python.org/issue17446 Wrapped Doctests will need to be unwrapped so the correct line number is returned. This will be reported upstream. #8796 fget __wrapped__)rrrrrunwrapr$ _find_lineno)r r, source_linesr&rrr, s    zDHfDoctestModule.collect..MockAwareDocTestFinder._find_linenorNc sPt|rdStt|||||||WddS1s!wYdSr)rrr$_find)r rr,r>rrglobsseenr&rrr@ s"z=HfDoctestModule.collect..MockAwareDocTestFinder._findrf)rrrrrrr)rrr&rMockAwareDocTestFinder& s rz conftest.py importmode)rootpath)rootmodedoctest_ignore_import_errorszunable to import module %r)rF)verbose optionflagscheckercontinue_on_failure)r>runnerdtest)doctest DocTestFinderrr>r pluginmanager_importconftest getoptionrrrrrrrrrrrrrNr from_parent)r rrfinderrrrrrrcollect% sB #     zHfDoctestModule.collectN)rrrrrrrrrrrr srdevicedispatch_tablecOs<||vr |d|i|S||}|durdS||i|S)Nrr)rrrrrrrr_device_agnostic_dispatchk s r)rrFrcCdSNrrrrrrr rcCr)Nrrrrrrr rrcCrrrrrrrr rxlaseedcCs t|t|Sr)rBACKEND_MANUAL_SEED)rrrrrbackend_manual_seed s rcC t|tSr)rBACKEND_EMPTY_CACHErrrrbackend_empty_cache rcrcCrr)rBACKEND_DEVICE_COUNTrrrrr$ rcr$TRANSFORMERS_TEST_DEVICE_SPECzISpecified path to device spec file is not a file or not found. Received 'z.pyz;Provided device spec file was not a Python file! Received 'z.Device spec file did not contain `DEVICE_NAME`zBMismatch between environment variable `TRANSFORMERS_TEST_DEVICE` 'z' and device found in spec 'z' zNEither unset `TRANSFORMERS_TEST_DEVICE` or ensure it matches device spec name.device_fn_dictattribute_namec Cs`z tt|}||t<WdSty/}zd|vr$td|dtd|WYd}~dSd}~ww)Nrrqz` not found in 'z)' and no default fallback function found.)rdevice_spec_moduler%AttributeErrordevice_spec_path)rrspec_fnrrrrupdate_mapping_from_spec s rMANUAL_SEED_FNEMPTY_CACHE_FNDEVICE_COUNT_FNcsg}g}ddt|Dtfdd|D}t|D]}|jtur0|j|vr0||jq|D] }|vr>||q3|sC|rkdg}|rP|d||rZ|d||rd|d|td |dS) NcSsh|]}|jqSr)r>)rfieldrrr rz6compare_pipeline_output_to_hub_spec..csg|]}|vr|qSrr)rrall_field_namesrrr rz7compare_pipeline_output_to_hub_spec..z(Pipeline output does not match Hub spec!zMatching keys: z*Missing required keys in pipeline output: z2Keys in pipeline output that are not in Hub spec: r) r sortedrrrr>rlrr)r?hub_spec missing_keysunexpected_keys matching_keysr output_keyrnrrr#compare_pipeline_output_to_hub_spec s,   rcCs"|rtt|tjdSr)gcrrr_dynamoreset)r gc_collectrrrcleanup srcCs:tstrddl}|j\}}trd|fSd|fStdfS)z, Get environment device properties. rNrocmr)IS_CUDA_SYSTEMIS_ROCM_SYSTEMrrget_device_capabilityr%)rmajorrrrrrX srXc@sfeZdZdefddZededefddZededede fdd Z ddedefd d Z d dZ dS) ExpectationsrcCs |tS)zX Find best matching expectation based on environment device properties. )find_expectationrXrrrrget_expectation! s zExpectations.get_expectationrcCstdd|DS)Ncss|]}|duVqdSrr)rrrrrr) sz*Expectations.is_default..)allrrrr is_default' rHzExpectations.is_defaultothercCsp|\}}|\}}d}||kr|dO}n |dvr|dvr|dO}||kr+|dur+|dO}t|r4|dO}t|S)a Returns score indicating how similar two instances of the `Properties` tuple are. Points are calculated using bits, but documented as int. Rules are as follows: * Matching `type` gives 8 points. * Semi-matching `type`, for example cuda and rocm, gives 4 points. * Matching `major` (compute capability major version) gives 2 points. * Default expectation (if present) gives 1 points. r)rrNr(r)r rr)rr device_typer other_device_type other_majorscorerrrr+ s   zExpectations.scorerGcs@t|jfddd\}}t|dkrtd|S)zU Find best matching expectation based on provided device properties. cst|dSr)r rrrrrrK sz/Expectations.find_expectation..rrz"No matching expectation found for )maxdatar"r rr)r r result_keyr rrrr G szExpectations.find_expectationcCs|jSr)rrrrrrR szExpectations.__repr__N)rG) rrrrr staticmethodDevicePropertiesrrrrr rrrrrr  s r )Fr)r)rV)NNNFF)NNrFT)r-)rgNN)rgr(rG(r7 contextlibrrrrrMrrrrrrrrr r<rDrorrrcollections.abcrrrr dataclassesrr r r r.r pathlibr typingrrrrr unittest.mockrhuggingface_hub.utilsr?ryrNr packagingr transformersrr3 integrationsrrrrrr r!integrations.deepspeedr"utilsr#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r2r3r4r5r6r7r8r9r:r;r<r=r>r?r@rArBrCrDrErFrGrHrIrJrKrLrMrNrOrPrQrRrSrTrUrVrWrXrYrZr[r\r]r^r_r`rarbrcrdrerfrgrhrirjrkrlrmrnrorprqrrrsrtrurvrwrxryrzr{r|r}r~rrrrraccelerate.staterraccelerate.utils.importsr_pytest.doctestrrrrrrr_pytest.outcomesr_pytest.pathlibrrrrMSMALL_MODEL_IDENTIFIERDUMMY_UNKNOWN_IDENTIFIERDUMMY_DIFF_TOKENIZER_IDENTIFIERUSERENDPOINT_STAGINGTOKENrhipr rr get_loggerrr-rrrrrrr_run_third_party_device_testsrrrrrrrrrrrrrrrrrrxrrrrrrrrrrrrrrrrrrrr r r r r rrrrrrrrrrrrrrr"r&r)r,r.r/r0r2r4r7r<r>r@rArBrbackend import_modulerModuleNotFoundErrorrr% is_availablerrr rrrdefault_backend jax_devicerGrIrSrTrUr&rZr[r\r^rarbrcrerfrgrhrirjrkrlrmrnrorprqrrrsrtrur{r|r~rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr!r+r,contextmanagerr8r9PathLikerKTestCaserLrrrrrrrrrrrrrr6r9r+r:rArBrxrrrrr DocTestParserrrr manual_seedr empty_cacherr rr:rEr6r?rrr$ris_filerrrealpathdevice_spec_dirrlindex import_namer DEVICE_NAME device_namerrrrrrrrXr rrrrs        $ e $                                                                      .k+ . !  )   29()*+< .&L        &    & "&