o
hy- @ s d dl mZ d dlZddlmZmZmZmZ ddl m
Z
m Z m Z m
Z
e r/d dlZddlmZ e r6d dlZeeZee
dd d
G d d d e ZdS )
)DictN )add_end_docstringsis_tf_availableis_torch_availablelogging )
GenericTensorPipelinePipelineExceptionbuild_pipeline_init_args)stable_softmaxT)
has_tokenizeraO
top_k (`int`, *optional*, defaults to 5):
The number of predictions to return.
targets (`str` or `List[str]`, *optional*):
When passed, the model will limit the scores to the passed targets instead of looking up in the whole
vocab. If the provided targets are not in the model vocab, they will be tokenized and the first resulting
token will be used (with a warning, and that might be slower).
tokenizer_kwargs (`dict`, *optional*):
Additional dictionary of keyword arguments passed along to the tokenizer.c s e Zd ZdZdedejfddZdedejfddZdefd d
Z dde
e ef fd d
Z dd Z
dddZdddZd ddZ fddZ ZS ) FillMaskPipelinea
Masked language modeling prediction pipeline using any `ModelWithLMHead`. See the [masked language modeling
examples](../task_summary#masked-language-modeling) for more information.
Example:
```python
>>> from transformers import pipeline
>>> fill_masker = pipeline(model="google-bert/bert-base-uncased")
>>> fill_masker("This is a simple [MASK].")
[{'score': 0.042, 'token': 3291, 'token_str': 'problem', 'sequence': 'this is a simple problem.'}, {'score': 0.031, 'token': 3160, 'token_str': 'question', 'sequence': 'this is a simple question.'}, {'score': 0.03, 'token': 8522, 'token_str': 'equation', 'sequence': 'this is a simple equation.'}, {'score': 0.027, 'token': 2028, 'token_str': 'one', 'sequence': 'this is a simple one.'}, {'score': 0.024, 'token': 3627, 'token_str': 'rule', 'sequence': 'this is a simple rule.'}]
```
Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial)
This mask filling pipeline can currently be loaded from [`pipeline`] using the following task identifier:
`"fill-mask"`.
The models that this pipeline can use are models that have been trained with a masked language modeling objective,
which includes the bi-directional models in the library. See the up-to-date list of available models on
[huggingface.co/models](https://huggingface.co/models?filter=fill-mask).
This pipeline only works for inputs with exactly one token masked. Experimental: We added support for multiple
masks. The returned values are raw model output, and correspond to disjoint probabilities where one might expect
joint probabilities (See [discussion](https://github.com/huggingface/transformers/pull/10222)).
This pipeline now supports tokenizer_kwargs. For example try:
```python
>>> from transformers import pipeline
>>> fill_masker = pipeline(model="google-bert/bert-base-uncased")
>>> tokenizer_kwargs = {"truncation": True}
>>> fill_masker(
... "This is a simple [MASK]. " + "...with a large amount of repeated text appended. " * 100,
... tokenizer_kwargs=tokenizer_kwargs,
... )
```
input_idsreturnc C sP | j dkrt|| jjk }|S | j dkr$tj|| jjkdd}|S td)NtfptFas_tuplezUnsupported framework) frameworkr where tokenizer
mask_token_idnumpytorchnonzero
ValueError)selfr masked_index r t/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/transformers/pipelines/fill_mask.pyget_masked_indexW s
z!FillMaskPipeline.get_masked_indexc C s> | |}t|j}|dk r td| jjd| jj dd S )Nr fill-maskzNo mask_token (z) found on the input) r" npprodshaper modelbase_model_prefixr
mask_token)r r r numelr r r! _ensure_exactly_one_mask_token` s
z/FillMaskPipeline._ensure_exactly_one_mask_token model_inputsc C sF t |tr|D ] }| |d d qd S |d D ]}| | qd S )Nr r )
isinstancelistr+ )r r, model_inputr r r r! ensure_exactly_one_mask_tokenj s
z.FillMaskPipeline.ensure_exactly_one_mask_tokenNc K s> |d u r| j }|d u r
i }| j|fd|i|}| | |S )Nreturn_tensors)r r r0 )r inputsr1 tokenizer_kwargspreprocess_parametersr, r r r!
preprocessr s
zFillMaskPipeline.preprocessc C s | j di |}|d |d< |S )Nr r )r' )r r,
model_outputsr r r! _forward~ s zFillMaskPipeline._forward c C s |d ur|j d |k r|j d }|d d }|d }| jdkrot|| jjk d d df }| }|d|d d f }t|dd}|d ur[tt |d|
dd}t |d}tj j
||d} | j | j }
} n.tj|| jjkd d
d}|d|d d f }|jdd }|d ur|d |f }||\}
} g } |
j d dk}
tt|
| D ]Q\}\}}g }t||D ]>\}}| }|d ur|| }|||| < |t|| jjk }| jj||
d
}||| j|g|d}|| q| | q|
r
| d S | S )Nr r logitsr )axisr )kFr )dim.)skip_special_tokens)scoretoken token_strsequence) r& r r r r r r r
gather_ndsqueezereshape expand_dimsmathtop_kvaluesindicesr r softmaxtopk enumerateziptolistcopyr$ pad_token_iddecodeappend)r r6 rH
target_idsr outputsr r9 probsrL rI predictionsresult single_maski_values _predictionsrowvptokensrB propositionr r r! postprocess sJ
" " z FillMaskPipeline.postprocessc C s t |tr|g}z| j }W n ty i }Y nw g }|D ]B}||d }|d u r\| j|ddddddd }t|dkrHtd| d q |d }td| d | j | d
|
| q t t |}t|dkrrt
d t|}|S ) NFr T)add_special_tokensreturn_attention_maskreturn_token_type_ids
max_length
truncationr r z The specified target token `zd` does not exist in the model vocabulary. We cannot replace it with anything meaningful, ignoring itz:` does not exist in the model vocabulary. Replacing with `z`.z1At least one target must be provided when passed.)r- strr get_vocab Exceptiongetlenloggerwarningconvert_ids_to_tokensrS r. setr r$ array)r targetsrH vocabrT targetid_r r r r! get_target_ids sL
z FillMaskPipeline.get_target_idsc C sj i }|d ur
||d< i }|d ur| ||}||d< |d ur"||d< | jjd u r0td| jjd|i |fS )Nr3 rT rH r# z-The tokenizer does not define a `mask_token`.)rv r r r r' r( )r rH rr r3 preprocess_paramspostprocess_paramsrT r r r! _sanitize_parameters s
z%FillMaskPipeline._sanitize_parametersc s6 t j|fi |}t|trt|dkr|d S |S )a
Fill the masked token in the text(s) given as inputs.
Args:
inputs (`str` or `List[str]`):
One or several texts (or one list of prompts) with masked tokens.
targets (`str` or `List[str]`, *optional*):
When passed, the model will limit the scores to the passed targets instead of looking up in the whole
vocab. If the provided targets are not in the model vocab, they will be tokenized and the first
resulting token will be used (with a warning, and that might be slower).
top_k (`int`, *optional*):
When passed, overrides the number of predictions to return.
Return:
A list or a list of list of `dict`: Each result comes as list of dictionaries with the following keys:
- **sequence** (`str`) -- The corresponding input with the mask token prediction.
- **score** (`float`) -- The corresponding probability.
- **token** (`int`) -- The predicted token id (to replace the masked one).
- **token_str** (`str`) -- The predicted token (to replace the masked one).
r r )super__call__r- r. rl )r r2 kwargsrU __class__r r! r{ s zFillMaskPipeline.__call__)NN)r8 N)N)NNN)__name__
__module__ __qualname____doc__r r$ ndarrayr" r+ r0 r rh r5 r7 rb rv ry r{
__classcell__r r r} r! r s
4
7
)r )typingr r r$ utilsr r r r baser r
r r
tensorflowr tf_utilsr
r
get_loggerr rm r r r r r! s