T^(hKddlZddlmZddlmZddlmZddlmZm Z m Z m Z m Z m Z ddlZddlmZmZddlmZdd lmZmZmZmZd d lmZerdd lmZerdd lmZerddl m!Z!ddl"m#Z#ddl$m%Z%ejLe'Z(edZ)defdZ* d,de ddddede+dede dde,de e e,e e,ffdZ- d-de d dd!dede+dede dde e e,e e,ffd"Z. d,de dde d#dede+dede dde,de e e,e e,ffd$Z/ d-dede dd%e d#d&ed'e e,d(e0de dfd)Z1de d#d*e e,de e2e e,ffd+Z3y).N) signature)chain)Path) TYPE_CHECKINGIterableListOptionalTupleUnion)Versionparse)PreTrainedTokenizerBase) TensorTypeis_tf_availableis_torch_availablelogging) OnnxConfig)PreTrainedModel)TFPreTrainedModel)FeatureExtractionMixin)ProcessorMixin)PreTrainedTokenizerz1.4.0minimum_versionc ddl}t|j}|tkrt d|jd|dy#t$r t dwxYw)z Check onnxruntime is installed and if the installed version match is recent enough Raises: ImportError: If onnxruntime is not installed or too old version is found rNz*We found an older version of onnxruntime (z&) but we require onnxruntime to be >= zp to enable all the conversions options. Please update onnxruntime by running `pip install --upgrade onnxruntime`zonnxruntime doesn't seem to be currently installed. Please install the onnxruntime by running `pip install onnxruntime` and relaunch the conversion.) onnxruntimer __version__ORT_QUANTIZE_MINIMUM_VERSION ImportError)rr ort_versions W/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/transformers/onnx/convert.pycheck_onnxruntime_requirementsr#5s K334  5 5<[=T=TAA preprocessor)rrrmodelrconfigopsetoutput tokenizerrdevicereturnct|tr | td|1tjdt t jd|}tt|trddl ddl m }t jdjj5d|j _|j%|j&t jd t)|j&d |j&j+D]7\}} t jd |d | t-|j || 9|j/|t0j2 } j5jdk(rj6j9r|j;i} | j+D]y\} } t| t<rt?fd| D| | </t| t@r'| Dcgc]}t?fd|Dc}| | <f| j;| | <{| } tC|| jE\}}tG|jHjE}|s td|jK||| f|jMtG|jNjE|tQtS|jNj+|jHj+d||jUddd||fSfScc}w#1swYfSxYw)a Export a PyTorch model to an ONNX Intermediate Representation (IR) Args: preprocessor: ([`PreTrainedTokenizer`], [`FeatureExtractionMixin`] or [`ProcessorMixin`]): The preprocessor used for encoding the data. model ([`PreTrainedModel`]): The model to export. config ([`~onnx.config.OnnxConfig`]): The ONNX configuration associated with the exported model. opset (`int`): The version of the ONNX operator set to use. output (`Path`): Directory to store the exported ONNX model. device (`str`, *optional*, defaults to `cpu`): The device on which the ONNX model will be exported. Either `cpu` or `cuda`. Returns: `Tuple[List[str], List[str]]`: A tuple with an ordered list of the model's inputs, and the named inputs from the ONNX configuration. NKYou cannot provide both a tokenizer and a preprocessor to export the model.tThe `tokenizer` argument is deprecated and will be removed in version 5 of Transformers. Use `preprocessor` instead.ROverwriting the `preprocessor` argument with `tokenizer` to generate dummy inputs.r)exportzUsing framework PyTorch: T Overriding  configuration item(s) -  ->  frameworkcudac3pK|]-}t|jr|jnd/ywN isinstanceTensorto.0xr*torchs r" z!export_pytorch..s07VWJq%,,,GADDLTQ736c3pK|]-}t|jr|jnd/ywr9r:r>s r"rBz!export_pytorch..s-!c\]*Q 2M!$$v,SW"W!crCz%Model and config inputs doesn't match)f input_names output_names dynamic_axesdo_constant_folding opset_version)+r;r ValueErrorwarningswarn FutureWarningloggerinfo issubclasstyperrA torch.onnxr0rno_gradr& return_dictevalvalues_overridelenitemssetattrgenerate_dummy_inputsrPYTORCHr*r7 is_availabler=r tupler$ensure_model_and_config_inputs_matchkeyslistoutputs patch_opsas_posixinputsdictr restore_ops)r$r%r&r'r(r)r* onnx_exportoverride_config_keyoverride_config_value model_inputsmodel_inputs_devicekvt inputs_matchmatched_inputs onnx_outputsrAs ` @r"export_pytorchrsRs>, 78Y=Rfgg  '   hi $u+/4 /0A0A/BCD ]]_3 !'+ELL $ JJL%%1 k#f.D.D*E)FF\]^BHBXBXB^B^B`V>')>KK$':&;4@U?V WXELL*=?TUV "77 PZPbPb7cL\\&)F{{f$)@)@)B &(#(..0 >DAq!!U+167[\72+A.$At,mn2hiE!cab!cc2+A.23f+A. > 3 +OPUWcWhWhWj+k (L. 3 3 56L !HII     //# !3!3!56)!% (;(;(=v~~?S?S?U"VW$(#     g3 !j < ''>< '';213 !j < ''s )E/MM4DMMM$)rrrc ddl}ddl}ddl}t|tr | t d|1t jdttjd|}d|j_ |jtjdt|jd|jjD]7\} } tjd | d | t!|j| | 9|j#|t$j& } t)|| j+\} } t-|j.j+}| jDcgc]1\}}|j1dg|j2z|j4| 3}}}|j6j9||| \}}|j;||j=|j?| |fScc}}w)a Export a TensorFlow model to an ONNX Intermediate Representation (IR) Args: preprocessor: ([`PreTrainedTokenizer`] or [`FeatureExtractionMixin`]): The preprocessor used for encoding the data. model ([`TFPreTrainedModel`]): The model to export. config ([`~onnx.config.OnnxConfig`]): The ONNX configuration associated with the exported model. opset (`int`): The version of the ONNX operator set to use. output (`Path`): Directory to store the exported ONNX model. Returns: `Tuple[List[str], List[str]]`: A tuple with an ordered list of the model's inputs, and the named inputs from the ONNX configuration. rNzIYou cannot provide both a tokenizer and preprocessor to export the model.r.r/Tr1r2r3r4r5)dtypename)r') onnx tensorflowtf2onnxr;rrKrLrMrNrOrPr&rUrWrXrYrZr[r TENSORFLOWr_r`rarb TensorSpecndimruconvert from_kerassaverdrg)r$r%r&r'r(r)rwtfryrirjrkrprqrrkeytensorinput_signature onnx_model_s r"export_tensorflowrs6, 78Y=Rdee  '   hi #ELL) k#f&<&<"=!>>TUV:@:P:P:V:V:X N 6 !6 KK$2348M7NO P ELL"57L M N // H]H]/^L#G|O`O`Ob#c L.++-.L^j]o]o]qNYcSY tfv{{*&,,S IOOO..uoU.SMJIIj&//+,  < ''s.6G<)rrc tsts tdtr t|tr|dk(r t dt|t r | td|1tjdttjd|}tr  O$5 8  Z/@AfPVFVNOO, 78Y=Rfgg  '   hi -00 NNRSYStStRuv*,-/   4; HlE65&T]flmm  z$u+7HI ufeVW`aa J reference_modelronnx_named_outputsatolc  ddlm}m}tj dt |t r | td|1tjdttj d|}trWtt|tr>|j||j dz|j"dzt$j&} n=|j||j dz|j"dzt$j(} |} ||j+| d g } tr*tt|tr|j-d |d"i| } i} | j/D]K\}}|d k(rd }t |t0t2fr$|j5||}| j7|G|| |<M|j9| }i}|j/D]\}}t |t0t2frT|j5||}|j7|j/Dcic]\}}||j;c}}p|j;||<| j=||}t?| jAt?|}}|jC|s8tj d|d|td|jE|tj d|dtG||D]\}}tr;tt|tr"| |jIj;}n| |j;}tj d|d|jJ|jJk(sUtj d|jJd|jJtd|jJd|jJdtj d|jJd|jJtMjN|||stMjPtMjR|||}tj d|dtdtMjTtMjV||z d||d ||tj d!|dycc}}w)#Nr)InferenceSessionSessionOptionszValidating ONNX model...zUYou cannot provide both a tokenizer and a preprocessor to validate the model outputs.r.r/r) batch_size seq_lengthr6CPUExecutionProvider) providerscpupast_key_valuespresentz -[x] ONNX model output names z do not match reference model zGOutputs doesn't match between reference model and ONNX exported model: u7 -[✓] ONNX model output names match reference model ()z! - Validating ONNX Model output "z":z -[x] shape z doesn't match zQOutputs shape doesn't match between reference model and ONNX exported model: Got z (reference) and z (ONNX)u -[✓] z matches )rz& -[x] values not close enough (atol: znOutputs values doesn't match between reference model and ONNX exported model: Got max absolute difference of: z for z vs u! -[✓] all values close (atol: ),rrrrOrPr;rrKrLrMrNrrQrRrr[default_fixed_batchdefault_fixed_sequencerr\rzrdr=rYrar^"flatten_output_collection_propertyupdate!generate_dummy_inputs_onnxruntimenumpyrunsetr`issubset differencezipdetachshapenpallclose logical_notiscloseamaxabs)r&r$rrrrr)rrreference_model_inputsoptionssession ref_outputsref_outputs_dictrvvalue"reference_model_inputs_onnxruntime onnx_inputs tensor_name pt_tensorrrref_outputs_setonnx_outputs_set ort_value ref_value bad_indicess r"validate_model_outputsr=s= KK*+, 78Y=Rpqq  '   hi  4+@/ R!'!=!= 11A544q8 (( ">" "(!=!= 11A544q8 ++ ">" Gz224gJ`IabG 4+@/ R5!!;$:;K#((* + e $ $D edE] +==dEJE  # #E *%* T " +*0)Q)QRh)i&K9??A. e edE] +==dEJE   ]b]h]h]jkCY;PY Y__-> >k l % K  .;;1;?L),,<,A,A,C(DcJ\F]%O  $ $_ 5 -.>-??]^m]n o  U**?;< >   NO_N``abc1<@Gi  JtO/Do$V(.557==?I(.446I 8bAB)//1 KK/)//):/)//IZ[ \ ''88IR  KK+ioo%6i ?PQ R{{9id;..Iyt)TUK KKB4&J K33577266)iBW;X3Y2ZZ_[)*$y/E.FH  KK=dV1E F7G- ls8S& rkctr9tt|tr t |j j }nt |jj }t|}t|j}|j|}|j|}|jDcgc] }||vs| }}||fScc}w)z> :param model_inputs: :param config_inputs: :return: ) rrQrRrrforward parameterscallrr`r intersection) r%rkforward_parametersmodel_inputs_setforward_inputs_setis_okmatching_inputs parameterordered_inputss r"r_r_s 4; H&u}}5@@&uzz2==<(/4467  % %&8 9E)556FGO1C1H1H1JkIi[jNjikNk .  ls ; CC)Nrr9)4rLinspectr itertoolsrpathlibrtypingrrrr r r rrpackaging.versionr r tokenization_utils_baserrrrrrr&rmodeling_utilsrmodeling_tf_utilsrfeature_extraction_utilsrprocessing_utilsrtokenization_utilsr get_logger__name__rOrr#intstrrsrr0floatrboolr_rrr"rsHH,= 05A18   H % %W~ G F26d(YZd( d( d(  d(  d( -. d( d( 49d3i  d(Z26 ?(GH?( ?( ?(  ?(  ?( -. ?( 49d3i  ?(P26?bYZ?b 7 8?b ?b  ?b  ?b -. ?b ?b 49d3i  ?bR26xG xGYZxGABxG xG S xG  xG-.xGv! 7 8!HPQT ! 4c?!r