S^(h%ZddlmZddlmZddlmZmZmZddlm Z ddl m Z ddl m Z ddlmZdd lmZej$eZGd d eeZifd eeeefd eeeffdZej0ej2iZGddZGddeZGddeZdddZdddZy))deepcopy)Enum)DictListOptional)InferenceClient)_deprecate_method) AutoTokenizer)Pipeline)loggingc0eZdZdZdZdZdZdZedZ y) MessageRoleuser assistantsystemz tool-callz tool-responsec@|Dcgc]}|jc}Scc}wN)value)clsrs \/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/transformers/agents/llm_engine.pyroleszMessageRole.roles's!$%A%%%sN) __name__ __module__ __qualname__USER ASSISTANTSYSTEM TOOL_CALL TOOL_RESPONSE classmethodrrrr s- DI FI#M&&r$r message_listrole_conversionscg}t|}|D]}t|jddhk(s td|d}|tj vr$td|dtj d||vr|||d<t |dkDr%|d|ddk(r|ddxxd |dzz cc<|j||S) z Subsequent messages with the same role will be concatenated to a single message. Args: message_list (`List[Dict[str, str]]`): List of chat messages. rolecontentz6Message should contain only 'role' and 'content' keys!zIncorrect role z, only z are supported for now.rz ======= )rsetkeys ValueErrorrrlenappend)r%r&final_message_listmessager(s rget_clean_message_listr2,sL)L/7<<>"vy&99UV Vv {((* *tfGKPQS>TU[>\+\ r "9 -AS1S S -  % %g ./ r$c eZdZeddddeefdZdZ ddee eefd eeed eefd Z ddee eefd eeed eed efd Z y)HfEnginez4.51.0zzSwitch to smolagents instead, with the same functionalities and similar API (https://huggingface.co/docs/smolagents/index))versionr1Nmodel_idc(d|_d|_|d}tjd|d t j ||_y#t$r@}tjd|d|dt j d|_Yd}~yd}~wwxYw)Nz#HuggingFaceTB/SmolLM2-1.7B-Instructz)Using default model for token counting: ''z#Failed to load tokenizer for model z: z$. Loading default tokenizer instead.)last_input_token_countlast_output_token_countloggerwarningr from_pretrained tokenizer Exception)selfr6es r__init__zHfEngine.__init__Ms '+#'+$  ">  r$messagesstop_sequencesgrammarctr)NotImplementedError)r@rGrHrIs rgeneratezHfEngine.generatecs "!r$returncdt|ts td|g}|j|||}t |j j |d|_t |j j||_ |D]$}|t | d|k(s|dt | }&|S)a9Process the input messages and return the model's response. This method sends a list of messages to the Hugging Face Inference API, optionally with stop sequences and grammar customization. Parameters: messages (`List[Dict[str, str]]`): A list of message dictionaries to be processed. Each dictionary should have the structure `{"role": "user/system", "content": "message content"}`. stop_sequences (`List[str]`, *optional*): A list of strings that will stop the generation if encountered in the model's output. grammar (`str`, *optional*): The grammar or formatting structure to use in the model's response. Returns: `str`: The text content of the model's response. Example: ```python >>> engine = HfApiEngine( ... model="meta-llama/Meta-Llama-3.1-8B-Instruct", ... token="your_hf_token_here", ... max_tokens=2000 ... ) >>> messages = [{"role": "user", "content": "Explain quantum mechanics in simple terms."}] >>> response = engine(messages, stop_sequences=["END"]) >>> print(response) "Quantum mechanics is the branch of physics that studies..." ``` zIMessages should be a list of dictionaries with 'role' and 'content' keys.NT)tokenize) isinstancerr-rLr.r>apply_chat_templater9encoder:)r@rGrHrIresponsestop_seqs r__call__zHfEngine.__call__hs>(D)hi i  !N==>7C&)$..*L*LX`d*L*e&f#'*4>>+@+@+J'K$' 6HX()X5#$4s8}n5 6r$rNN) rrrr rstrrBrFrrrLrUr#r$rr4r4LsM b# b  b tx"T#s(^,">FtCy>Q"cklocp" tx+T#s(^,+>FtCy>Q+cklocp+ +r$r4c eZdZdZ d dedeedeedeeffd Z d dee eefdeeed eed efd Z xZ S) HfApiEnginea7A class to interact with Hugging Face's Inference API for language model interaction. This engine allows you to communicate with Hugging Face's models using the Inference API. It can be used in both serverless mode or with a dedicated endpoint, supporting features like stop sequences and grammar customization. Parameters: model (`str`, *optional*, defaults to `"meta-llama/Meta-Llama-3.1-8B-Instruct"`): The Hugging Face model ID to be used for inference. This can be a path or model identifier from the Hugging Face model hub. token (`str`, *optional*): Token used by the Hugging Face API for authentication. If not provided, the class will use the token stored in the Hugging Face CLI configuration. max_tokens (`int`, *optional*, defaults to 1500): The maximum number of tokens allowed in the output. timeout (`int`, *optional*, defaults to 120): Timeout for the API request, in seconds. Raises: ValueError: If the model name is not provided. modeltoken max_tokenstimeoutc|t||||_t|j|||_||_y)N)r6)r[r])superrBrZrclientr\)r@rZr[r\r] __class__s rrBzHfApiEngine.__init__s7 %( %djjwO $r$rGrHrIrMct|t}|*|jj|||j|}n(|jj|||j}|j dj j}|S)Nr&)stopr\response_format)rdr\r)r2llama_role_conversionsr`chat_completionr\choicesr1r))r@rGrHrIrSs rrLzHfApiEngine.generates*(E[\  {{22~$//[b3H{{228.]a]l]l2mH##A&..66r$)z%meta-llama/Meta-Llama-3.1-8B-InstructNxrV) rrr__doc__rWrintrBrrrL __classcell__ras@rrYrYs,=#$(!$ % %} %SM % # %txT#s(^,>FtCy>Qcklocp r$rYc zeZdZdZd dedeeffd Z d dee eefdeeedeede d ef d Z xZ S) TransformersEnginezBThis engine uses a pre-initialized local text-generation pipeline.pipeliner6c2t||||_yr)r_rBrq)r@rqr6ras rrBzTransformersEngine.__init__s "  r$rGrHrI max_lengthrMct|t}|t|dkDr|}nd}|j||||jj}|dddd}|S)Nrcr) stop_stringsrsr>generated_textr*r))r2rfr.rqr>)r@rGrHrIrsruoutputrSs rrLzTransformersEngine.generatesx*(E[\  %#n*=*A)LL %!mm--  !9-.r29=r$r)NNri) rrrrkr rrWrBrrrlrLrmrns@rrprpswL!!Xc]!/3!% tCH~&!c+#    r$rpregexzhThought: .+?\nAction:\n\{\n\s{4}"action":\s"[^"\n]+",\n\s{4}"action_input":\s"[^"\n]+"\n\}\n)typerzCThought: .+?\nCode:\n```(?:py|python)?\n(?:.|\s)+?\n```N) copyrenumrtypingrrrhuggingface_hubr"huggingface_hub.utils._deprecationr r pipelines.baser utilsr get_loggerrr;rWrr2r!rrfr4rYrpDEFAULT_JSONAGENT_REGEX_GRAMMARDEFAULT_CODEAGENT_REGEX_GRAMMARr#r$rrs"''+@%   H % &#t &ced38n)=QUVY[^V^Q_8{// GGT0(0fF F#  W#r$