o  h-@s(ddlZddlZddlZddlZddlmZddlmZddlm Z ddl m Z e r?ddl Z ddl Z ddl mZddlmZdd l mZeeZe jZd d Zed Zd edefddZddZddZddZddZ defddZ!e ddddZ"e dded"deefd d!Z#dS)#N)contextmanager)Optional)export)is_torch_available)nn)PreTrainedModel)loggingcCs trtjs dStjdkS)z7Return True if rank=0 or we aren't running distributed.Tr)_torch_distributed_availabletorch distributedis_initializedget_rankrrv/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/transformers/model_debugging_utils.py _is_rank_zero,srzobject at 0x[0-9A-Fa-f]+x_strreturncCs td|S)z Replace memory addresses in an object's repr with a stable placeholder so that beautiful JSON diffs won't be ruined by ephemeral addresses. zobject at 0xXXXXXXXX)MEMORY_ADDRESS_REGEXsub)rrrr_sanitize_repr_for_diff6s rcCstr dt|jSdS)z@Return a stable string representation for a DTensor-like object.zDTensor (rank0) -> zDTensor(non-rank0))rrepr _local_tensor)xrrr _dtensor_repr>srcCst|ttfrdd|DSt|trdd|DSt|dr3t|jjt|jj t t|dSt|t j rIt|jt|j t t|dSt t|S)a Recursively build a JSON-serializable Python structure from `value`. Tensors and DTensors become sanitized repr strings. Lists/tuples/dicts are recursed into. All memory addresses are replaced with a stable placeholder. Args: value: Any Python object, often including torch Tensors, lists, dicts, etc. Returns: A nested Python structure (list, dict, or sanitized string) that is safe to json.dump. cSsg|]}t|qSr _serialize_io).0vrrr Ssz!_serialize_io..cSsi|] \}}|t|qSrr)rkrrrr Vsz!_serialize_io..r)shapedtypevalue) isinstancelisttupledictitemshasattrrrr"r#rr Tensor)r$rrrrEs         rcCs4|dr|dd|dD]}t|qdSdS)Nchildrenoutputs)getpopprune_outputs_if_children)nodechildrrrr0is    r0c Cs|r,ztj|ddtj||jd}Wnty+}z td|d|d}~ww|jd}td|t |d}t |j t j |j |dd WddS1sXwYdS) NF)exist_okz_debug_tree.jsonz"Unexpected or existing debug_path=z. zWriting model trace at w)indent)osmakedirspathjoin_debugger_module_dump_name Exception ValueErrorloggerinfoopenr0 _call_treejsondump) debug_pathmodel output_patheoutfilerrrlog_model_debug_tracers   "rIrDc sddgd_g__fdd}D]\}}|dkr!q||d|qjtfdd}|_fdd }|d d g}|D]} t| d} | rkt | t j t frk| |dSqQdS) N module_pathinputsr-r,cs,jtfdd}|_dS)Ncstr ||dfddDtdgd}j|t|i|}Wdn1s6wYtrqtddDdkrPd|d<nt||d<j}|d sd|d jrqjd d ||S) Nargskwargscs&i|]}t|dkr||qS)r)lenrr  dict_inputsrrr!s&zY_attach_debugger_logic..wrap_forward..wrapped_forward..rJcss|]}dVqdS)rNr)r_rrr szX_attach_debugger_logic..wrap_forward..wrapped_forward..rr-r,) rr_debugger_model_call_stackappendr inference_modesumnamed_childrenr/)inpskwsr1outfinished) full_pathrEmodule orig_forwardrRrwrapped_forwards,       zE_attach_debugger_logic..wrap_forward..wrapped_forward)forward functoolswraps)rar`rcrE)r`rarbr wrap_forwards z,_attach_debugger_logic..wrap_forward.cstrdt||ddgd}j||i|}trSjrSt||d<j}|djd<|djd<|djd<fddtjD|S) Nz (top-level)rMrJr-rLr,cs$g|]}j|sj|dqSN)rAr/rQrgrrrs$zG_attach_debugger_logic..top_wrapped_forward..)rrrWrXr/rAr&keys)r\r]top_noder^r_) class_namerEreal_top_forwardrrtop_wrapped_forwards      z3_attach_debugger_logic..top_wrapped_forwardcsZtr jr j}|djd<|djd<|djd<tr+tddSdS)NrLr-r,rDrE)rrWr/rArI)rTrLr-r_rqrr final_hooks  z*_attach_debugger_logic..final_hooklanguage_modelrE) rArWr; named_modulesrdrerfregister_forward_hookgetattrr%rModuler) rErnrDrhname submodulerprrpossible_model_calls model_callthis_model_callr)rnrDrEror_attach_debugger_logics, $   r})r )backendscs*jtjfdd}|_S)a # Model addition debugger - a model adder tracer This decorator is a power user tool intended for model adders. It tracks all forward calls within a model forward and logs a slice of each input and output on a nested Json. To note, this decorator enforces `torch.inference_mode()`. ## Usage add decorator to your model class ```python from ...modeling_utils import model_addition_debugger @model_addition_debugger class MyModel(nn.Module) # Can inherit from PreTrainedModel too # ... nothing else changes ``` Then, in a separate script (example is for Llava) ```python import torch from PIL import Image import requests from transformers import LlavaProcessor, LlavaForConditionalGeneration torch.random.manual_seed(673) # load pretrained model and processor model_id = "llava-hf/llava-1.5-7b-hf" processor = LlavaProcessor.from_pretrained(model_id) model = LlavaForConditionalGeneration.from_pretrained(model_id, low_cpu_mem_usage=True) # create random image input random_image = Image.fromarray(torch.randint(0, 256, (224, 224, 3), dtype=torch.uint8).numpy()) # prompt prompt = "Describe this image." # process inputs inputs = processor(text=prompt, images=random_image, return_tensors="pt") # call forward method (not .generate!) with torch.no_grad(): output = model.forward(**inputs) ``` cs&|g|Ri|t|jdSrk)r}__name__)selfrNrOcls orig_initrr wrapped_initsz-model_addition_debugger..wrapped_init)__init__rerf)rrrrrmodel_addition_debuggers . rccs"t||jj|z|VWdSw)a # Model addition debugger - context manager for model adders This context manager is a power user tool intended for model adders. It tracks all forward calls within a model forward and logs a slice of each input and output on a nested Json. To note, this context manager enforces `torch.inference_mode()`. ## Usage add the context manager to a model to debug ```python import torch from PIL import Image import requests from transformers import LlavaProcessor, LlavaForConditionalGeneration torch.random.manual_seed(673) # load pretrained model and processor model_id = "llava-hf/llava-1.5-7b-hf" processor = LlavaProcessor.from_pretrained(model_id) model = LlavaForConditionalGeneration.from_pretrained(model_id, low_cpu_mem_usage=True) # create random image input random_image = Image.fromarray(torch.randint(0, 256, (224, 224, 3), dtype=torch.uint8).numpy()) # prompt prompt = "Describe this image." # process inputs inputs = processor(text=prompt, images=random_image, return_tensors="pt") # call forward method (not .generate!) with model_addition_debugger_context(model): output = model.forward(**inputs) ``` N)r} __class__r)rErDrrrmodel_addition_debugger_contexts (rrk)$rerBr7re contextlibrtypingrtransformers.utils.import_utilsrutilsrr torch.distributed.tensorrmodeling_utilsrr get_loggerrr>r is_availabler rcompilerstrrrrr0rIr}rrrrrrs:          $ c 8