o  h[ @s0ddlmZmZddlmZmZmZddlmZm Z m Z ddl m Z ddl mZz&ddlmZmZmZdd lmZdd lmZdd lmZdd lmZd ZWneefyaeZddZdZYnwe dZ defddZ!GdddeZ"GdddeZ#GdddeZ$GdddeZ%GdddeZ&dS) )ArgumentParser Namespace)AnyListOptional)Pipelineget_supported_taskspipeline)logging)BaseTransformersCLICommand)BodyFastAPI HTTPException)APIRoute) BaseModel) JSONResponse)runTcOsdS)N)xyrrq/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/transformers/commands/serving.pyr"srFztransformers-cli/servingargscCs:t|j|jr |jnd|j|j|jd}t||j|j|j S)z~ Factory function used to instantiate serving server from provided command line arguments. Returns: ServeCommand N)taskmodelconfig tokenizerdevice) r rrrrr ServeCommandhostportworkers)rnlprrrserve_command_factory+sr$c@eZdZUdZeed<dS)ServeModelInfoResultz" Expose model information infosN)__name__ __module__ __qualname____doc__dict__annotations__rrrrr&;  r&c@s.eZdZUdZeeed<eeeed<dS)ServeTokenizeResultz Tokenize result model tokens tokens_idsN) r(r)r*r+rstrr-rintrrrrr/Cs  r/c@r%)ServeDeTokenizeResultz! DeTokenize result model textN)r(r)r*r+r2r-rrrrr4Lr.r4c@r%)ServeForwardResultz Forward result model outputN)r(r)r*r+rr-rrrrr6Tr.r6c@seZdZedefddZdedededefdd Z d d Z d d Z e ddde dddfdede fddZe ddde ddde dddfdeede de fddZe dddfddZdS)rparsercCs|jddd}|jdttdd|jdtdd d |jd td d d |jdtddd |jdtdd|jdtdd|jdtdd|jdtddd |jtddS)z Register this command to argparse so it's available for the transformer-cli Args: parser: Root parser to register command-specific arguments servezFCLI tool to run inference requests through REST and GraphQL endpoints.)helpz--taskzThe task to run the pipeline on)typechoicesr:z--host localhostz$Interface the server will listen on.)r;defaultr:z--porti"z Port the serving will listen to.z --workersr zNumber of http workersz--modelz%Model's name or path to stored model.)r;r:z--configz,Model's config name or path to stored model.z --tokenizerzTokenizer name to use.z--devicezSIndicate the device to run onto, -1 indicates CPU, >= 0 indicates GPU (default: -1))funcN) add_parser add_argumentr2r r3 set_defaultsr$)r8 serve_parserrrrregister_subcommand]s,z ServeCommand.register_subcommandr r r!r"c Cs||_||_||_||_tstdtd|d|tt d|j t t dgdt d|j tt dgdt d |jtt dgdt d |jtt dgdgd d |_dS) NzUsing serve command requires FastAPI and uvicorn. Please install transformers with [serving]: pip install "transformers[serving]". Or install FastAPI and uvicorn separately.zServing model over :/GET)response_modelresponse_classmethodsz /tokenizePOSTz /detokenizez/forwardiX)routestimeout) _pipeliner r!r"_serve_dependencies_installed RuntimeErrorloggerinforr model_infor&rtokenizer/ detokenizer4forwardr6_app)selfr r r!r"rrr__init__|sR zServeCommand.__init__cCst|j|j|j|jddS)N)r r!r")rrXr r!r"rYrrrrszServeCommand.runcCstt|jjjdS)N)r')r&varsrOrrr[rrrrTszServeCommand.model_infoNT)embedF text_input return_idsc Csjz|jj|}|r|jj|}t||dWSt|dWSty4}z tddt|ddd}~ww)z Tokenize the provided input and eventually returns corresponding tokens id: - **text_input**: String to tokenize - **return_ids**: Boolean flags indicating if the tokens have to be converted to their integer mapping. )r0r1)r0rerror status_codedetailN)rOrrUconvert_tokens_to_idsr/ Exceptionrr2)rYr^r_ tokens_txtr1errrrUs zServeCommand.tokenizer1skip_special_tokenscleanup_tokenization_spacesc CsPz|jj|||}td|dWSty'}z tddt|ddd}~ww)a: Detokenize the provided tokens ids to readable text: - **tokens_ids**: List of tokens ids - **skip_special_tokens**: Flag indicating to not try to decode special tokens - **cleanup_tokenization_spaces**: Flag indicating to remove all leading/trailing spaces and intermediate ones. ra)rr5r`rbrdN)rOrdecoder4rhrr2)rYr1rkrl decoded_strrjrrrrVs zServeCommand.detokenizec s\t|dkr tggdSz ||}t|dWSty-}z tddt|id}~ww)zF **inputs**: **attention_mask**: **tokens_type_ids**: r)r7 attention)r7r`rcN)lenr6rOrhrr2)rYinputsr7rjrrrrWs    zServeCommand.forward)r(r)r* staticmethodrrErr2r3rZrrTrboolrUrrVrWrrrrr\s$1(    rN)'argparserrtypingrrr pipelinesrr r utilsr rar fastapirrrfastapi.routingrpydanticrstarlette.responsesruvicornrrP ImportErrorAttributeErrorobject get_loggerrRr$r&r/r4r6rrrrrs0