o
h @ s~ d Z ddlmZ ddlmZmZ e r ddlZddlmZ e e
Z edde fdd Z
edd
d de fd d
ZdddZdS )a
Since, https://github.com/huggingface/transformers/pull/36963, loading is always performed with models on meta
device. But since the `init_empty_weights` and `find_tied_parameters` functions are from accelerate, and accelerate is
somewhat still a soft dependency, we copy the functions here to be used natively in Transformers.
The `init_empty_weights` and `init_on_device` functions were copied from `accelerate.big_modeling.py`, and the
`find_tied_parameters` was copied from `accelerate.utils.modeling.py`
)contextmanager )is_torch_availableloggingNFinclude_buffersc c s@ t td| d }|V W d dS 1 sw Y dS )a
A context manager under which models are initialized with all parameters on the meta device, therefore creating an
empty model. Useful when just initializing the model would blow the available RAM.
Args:
include_buffers (`bool`, *optional*):
Whether or not to also put all buffers on the meta device while initializing.
Example:
```python
import torch.nn as nn
from accelerate import init_empty_weights
# Initialize a model with 100 billions parameters in no time and without using any RAM.
with init_empty_weights():
tst = nn.Sequential(*[nn.Linear(10000, 10000) for _ in range(1000)])
```
Any model created under this context manager has no weights. As such you can't do something like
`model.to(some_device)` with it. To load weights inside your empty model, see [`load_checkpoint_and_dispatch`].
Make sure to overwrite the default device_map param for [`load_checkpoint_and_dispatch`], otherwise dispatch is not
called.
meta)r N)init_on_devicetorchdevice)r f r x/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/transformers/integrations/accelerate.pyinit_empty_weights$ s "r r
z torch.devicec # s8 |r dV W d dS 1 sw Y dS t jj|r%t jj fdd}d fdd }|r>dd d D }ni } fd
d }z;|t j_|rQ|t j_| D ]
}tt||tt| qUdV W t j_|rqt j_| D ]
\}}tt|| qudS t j_|rt j_| D ]
\}}tt|| qw )
a)
A context manager under which models are initialized with all parameters on the specified device.
Args:
device (`torch.device`):
Device to initialize all parameters on.
include_buffers (`bool`, *optional*):
Whether or not to also put all buffers on the meta device while initializing.
Example:
```python
import torch.nn as nn
from accelerate import init_on_device
with init_on_device(device=torch.device("cuda")):
tst = nn.Linear(100, 100) # on `cuda` device
```
Nc sb | || |d ur/t | j| }| j| j}|j|d< || j| fi || j|< d S d S )N
requires_grad)type _parameters__dict__r to)modulenameparam param_clskwargs)r
old_register_parameterr r
register_empty_parameterd s
&z0init_on_device..register_empty_parameterTc s6 | |||d |d ur| j | | j |< d S d S )N)
persistent)_buffersr )r r bufferr )r
old_register_bufferr r
register_empty_bufferl s z-init_on_device..register_empty_bufferc S s i | ]}|t t|qS r )getattrr ).0torch_function_namer r r
s s
z"init_on_device..)emptyzerosonesfullc s fdd}|S )Nc s |d< | i |S )Nr
r )argsr )r
fnr r
wrapper{ s zAinit_on_device..patch_tensor_constructor..wrapperr )r) r* )r
)r) r
patch_tensor_constructorz s z0init_on_device..patch_tensor_constructor)T) nnModuleregister_parameterregister_bufferkeyssetattrr r items)r
r r r tensor_constructors_to_patchr+ r" old_torch_functionr )r
r r r
r F sL
r model nn.Modulec
K s t | jdd}t | jdd}t| t| }i }|D ]"}|| }| D ]\}} | |u rA||vr:g ||< || | q*q dd | D S )a
Find the tied parameters in a given model.
The signature accepts keyword arguments, but they are for the recursive part of this function and you should ignore
them.
Args:
model (`torch.nn.Module`): The model to inspect.
Returns:
List[List[str]]: A list of lists of parameter names being all tied together.
Example:
```py
>>> from collections import OrderedDict
>>> import torch.nn as nn
>>> model = nn.Sequential(OrderedDict([("linear1", nn.Linear(4, 4)), ("linear2", nn.Linear(4, 4))]))
>>> model.linear2.weight = model.linear1.weight
>>> find_tied_parameters(model)
[['linear1.weight', 'linear2.weight']]
```
F)remove_duplicateTc S s& g | ]\}}t |gtt| qS r )sortedlistset)r! weighttiedr r r
s & z(find_tied_parameters..)dictnamed_parametersr: r0 r2 append)
r5 r all_named_parameters no_duplicate_named_parameterstied_param_namestied_param_groupstied_param_name
tied_param
param_namer r r r
find_tied_parameters s rH )F)r5 r6 )__doc__
contextlibr utilsr r r torch.nnr,
get_logger__name__loggerboolr r rH r r r r
s
!I