o  h%@sddlmZddlmZddlmZmZmZddlm Z ddl m Z ddl m Z ddlmZdd lmZeeZGd d d eeZifd eeeefd eeeffddZejejiZGdddZGdddeZGdddeZdddZdddZdS))deepcopy)Enum)DictListOptional)InferenceClient)_deprecate_method) AutoTokenizer)Pipeline)loggingc@s,eZdZdZdZdZdZdZeddZ dS) MessageRoleuser assistantsystemz tool-callz tool-responsecCsdd|DS)NcSsg|]}|jqS)value).0rrrr/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/transformers/agents/llm_engine.py )sz%MessageRole.roles..r)clsrrrroles'szMessageRole.rolesN) __name__ __module__ __qualname__USER ASSISTANTSYSTEM TOOL_CALL TOOL_RESPONSE classmethodrrrrrr sr message_listrole_conversionscCsg}t|}|D]U}t|ddhkstd|d}|tvr/td|dtd||vr9|||d<t|dkrX|d|ddkrX|ddd |d7<q||q|S) z Subsequent messages with the same role will be concatenated to a single message. Args: message_list (`List[Dict[str, str]]`): List of chat messages. rolecontentz6Message should contain only 'role' and 'content' keys!zIncorrect role z, only z are supported for now.rz ======= )rsetkeys ValueErrorr rlenappend)r"r#final_message_listmessager$rrrget_clean_message_list,s    r.c @seZdZedddddeefddZdd Z dd ee eefd eeed eefd dZ dd ee eefd eeed eedefddZ dS)HfEnginez4.51.0zzSwitch to smolagents instead, with the same functionalities and similar API (https://huggingface.co/docs/smolagents/index))versionr-Nmodel_idc Csd|_d|_|durd}td|dz t||_WdStyC}ztd|d|dtd|_WYd}~dSd}~ww)Nz#HuggingFaceTB/SmolLM2-1.7B-Instructz)Using default model for token counting: ''z#Failed to load tokenizer for model z: z$. Loading default tokenizer instead.)last_input_token_countlast_output_token_countloggerwarningr from_pretrained tokenizer Exception)selfr1errr__init__MszHfEngine.__init__cCs|j|jdS)N)input_token_countoutput_token_count)r3r4)r:rrrget_token_counts]szHfEngine.get_token_countsmessagesstop_sequencesgrammarcCstN)NotImplementedError)r:r@rArBrrrgeneratecszHfEngine.generatereturncCst|ts td|durg}||||}t|jj|dd|_t|j||_ |D]}|t| d|krB|dt| }q,|S)a9Process the input messages and return the model's response. This method sends a list of messages to the Hugging Face Inference API, optionally with stop sequences and grammar customization. Parameters: messages (`List[Dict[str, str]]`): A list of message dictionaries to be processed. Each dictionary should have the structure `{"role": "user/system", "content": "message content"}`. stop_sequences (`List[str]`, *optional*): A list of strings that will stop the generation if encountered in the model's output. grammar (`str`, *optional*): The grammar or formatting structure to use in the model's response. Returns: `str`: The text content of the model's response. Example: ```python >>> engine = HfApiEngine( ... model="meta-llama/Meta-Llama-3.1-8B-Instruct", ... token="your_hf_token_here", ... max_tokens=2000 ... ) >>> messages = [{"role": "user", "content": "Explain quantum mechanics in simple terms."}] >>> response = engine(messages, stop_sequences=["END"]) >>> print(response) "Quantum mechanics is the branch of physics that studies..." ``` zIMessages should be a list of dictionaries with 'role' and 'content' keys.NT)tokenize) isinstancerr)rEr*r8apply_chat_templater3encoder4)r:r@rArBresponsestop_seqrrr__call__hs zHfEngine.__call__rCNN) rrrrrstrr<r?rrrErMrrrrr/Ls2    r/c s~eZdZdZ    ddedeedeed eeffd d Z dd ee eefd eeedeedefddZ Z S) HfApiEnginea7A class to interact with Hugging Face's Inference API for language model interaction. This engine allows you to communicate with Hugging Face's models using the Inference API. It can be used in both serverless mode or with a dedicated endpoint, supporting features like stop sequences and grammar customization. Parameters: model (`str`, *optional*, defaults to `"meta-llama/Meta-Llama-3.1-8B-Instruct"`): The Hugging Face model ID to be used for inference. This can be a path or model identifier from the Hugging Face model hub. token (`str`, *optional*): Token used by the Hugging Face API for authentication. If not provided, the class will use the token stored in the Hugging Face CLI configuration. max_tokens (`int`, *optional*, defaults to 1500): The maximum number of tokens allowed in the output. timeout (`int`, *optional*, defaults to 120): Timeout for the API request, in seconds. Raises: ValueError: If the model name is not provided. %meta-llama/Meta-Llama-3.1-8B-InstructNxmodeltoken max_tokenstimeoutcs0tj|d||_t|j||d|_||_dS)N)r1)rUrW)superr<rTrclientrV)r:rTrUrVrW __class__rrr<s zHfApiEngine.__init__r@rArBrFcCsRt|td}|dur|jj|||j|d}n |jj|||jd}|jdjj}|S)Nr#)stoprVresponse_format)r]rVr)r.llama_role_conversionsrYchat_completionrVchoicesr-r%)r:r@rArBrKrrrrEs  zHfApiEngine.generate)rQNrRrSrN) rrr__doc__rOrintr<rrrE __classcell__rrrZrrPs2  rPc sneZdZdZddedeeffdd Z   ddee eefd eeed eed e d ef d dZ Z S)TransformersEnginezBThis engine uses a pre-initialized local text-generation pipeline.Npipeliner1cst|||_dSrC)rXr<rf)r:rfr1rZrrr<s  zTransformersEngine.__init__rRr@rArB max_lengthrFcCsXt|td}|durt|dkr|}nd}|j||||jjd}|dddd}|S)Nr\r) stop_stringsrgr8generated_textr&r%)r.r_r*rfr8)r:r@rArBrgrhoutputrKrrrrEs zTransformersEngine.generaterC)NNrR) rrrrbr rrOr<rrrcrErdrrrZrres" reregexzhThought: .+?\nAction:\n\{\n\s{4}"action":\s"[^"\n]+",\n\s{4}"action_input":\s"[^"\n]+"\n\}\n)typerzCThought: .+?\nCode:\n```(?:py|python)?\n(?:.|\s)+?\n```N) copyrenumrtypingrrrhuggingface_hubr"huggingface_hub.utils._deprecationrr pipelines.baser utilsr get_loggerrr5rOr r.r rr_r/rPreDEFAULT_JSONAGENT_REGEX_GRAMMARDEFAULT_CODEAGENT_REGEX_GRAMMARrrrrs,        * J3#