T^(hIddlZddlZddlZddlmZmZddlmZddlm Z m Z m Z m Z m Z mZmZmZmZmZddlZddlmZddlmZmZmZmZdd lmZmZmZe rdd lm Z dd l!m"Z"dd l#m$Z$dd l%m&Z&erddl'm(Z(ejRe*Z+dZ,dZ-ej\GddZ/GddeZ0Gdde0eZ1Gdde1Z2y)N)ABCabstractmethod) OrderedDict) TYPE_CHECKINGAnyCallableDictIterableListMappingOptionalTupleUnion)version) TensorTypeis_torch_availableis_vision_availablelogging)ParameterFormat compute_effective_axis_dimension"compute_serialized_parameters_size)PretrainedConfigFeatureExtractionMixinImageProcessingMixinPreTrainedTokenizerBase)Image lcXeZdZUdZeed<eed<eed<dZe eed<dZ e eed<y) PatchingSpeca Data class that holds patching specifications. Args: o: Module / object where the op to patch is located name: Name of the op to monkey patch custom_op: Custom op that patches the original op orig_op: Original op that is being patched op_wrapper: Wrapper (optional) that wraps both the original and custom ops. It is useful for ops that are class or static methods for instance. oname custom_opNorig_op op_wrapper) __name__ __module__ __qualname____doc__r__annotations__strrr(r r)V/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/transformers/onnx/config.pyr$r$/s7  F I"&GXh &%)J")r1r$c8eZdZdZdZdZdZejdZ e dddd ie d ddd ie dddd ie ddd ddd ddd d e dddd ie dddd ie dd diie ddd ddd d e ddd ddd de ddddddie dddd ie dd diie dddd ie dddd ie dddd idZ dAddde de efdZedBddde ddfdZeedee eee fffdZedee eee fffdZedeee effdZedefd Zedefd!Zedefd"Zedefd#Zedefd$Z ede!fd%Z"e#d&ede!fd'Z$ dCd(ed)ed*ed+efd,Z% dDd(ed-ed.ed/efd0Z& dEd1e'd2d(ed3ed4ed5e!d6ee(d)ed+ed*ed-ed.ed/ed7ed8dee effd9Z)d:ee efdee effd;Z*d<Z+d=Z,ed>e d?e-ede.e effd@Z/y)F OnnxConfigzv Base class for ONNX exportable model describing metadata on how to export the model through the ONNX format. rz1.8logitsbatchsequencerrlast_hidden_state)r7 pred_boxes pred_masksr)r7r<) start_logits end_logits num_labelsheightwidth)rrrdecoder_sequence)z causal-lmdefaultzimage-classificationzimage-segmentationz masked-imz masked-lmmultiple-choicezobject-detectionzquestion-answeringzsemantic-segmentationz seq2seq-lmzsequence-classificationztoken-classificationz vision2seq-lmz speech2seq-lmNconfigrtaskpatching_specscl||_||jvr(t|d|jj||_g|_||ngD]`}|}|j 5tj|t|j|j}|j j|by)Nz+ is not a supported task, supported tasks: )r() _config_tasks_to_common_outputs ValueErrorkeysrH_patching_specsr( dataclassesreplacegetattrr%r&append)selfrGrHrIspec final_specs r2__init__zOnnxConfig.__init__os t44 4&CDDaDaDfDfDhCij  !&4&@Nb 4DJ||#(00wtvvtyy?YZ  ' ' 3  4r1returnc|||S)z Instantiate a OnnxConfig for a specific model Args: config: The model's configuration to use when exporting to ONNX Returns: OnnxConfig for this model )rHr0clsrGrHs r2from_model_configzOnnxConfig.from_model_configs6%%r1ct)z Mapping containing the axis definition of the input tensors to provide to the model Returns: For each input: its name associated to the axes symbolic name and the axis position within the tensor )NotImplementedErrorrTs r2inputszOnnxConfig.inputss "##r1c^|j|j}tj|S)z Mapping containing the axis definition of the output tensors to provide to the model Returns: For each output: its name associated to the axes symbolic name and the axis position within the tensor )rLrHcopydeepcopy)rTcommon_outputss r2outputszOnnxConfig.outputss'66tyyA}}^,,r1c8t|jdrddiSy)z Dictionary of keys to override in the model's config before exporting Returns: Dictionary with the keys (and their corresponding values) to override use_cacheFN)hasattrrKr_s r2values_overridezOnnxConfig.values_overrides  4<< -' 'r1c"tjS)zp The default batch size to use if no other indication Returns: Integer > 0 )r4default_fixed_batchr_s r2default_batch_sizezOnnxConfig.default_batch_sizes---r1c"tjS)zu The default sequence length to use if no other indication Returns: Integer > 0 )r4default_fixed_sequencer_s r2default_sequence_lengthz"OnnxConfig.default_sequence_lengths000r1c"tjS)zw The default number of choices to use if no other indication Returns: Integer > 0 )r4default_fixed_num_choicesr_s r2default_num_choiceszOnnxConfig.default_num_choicess333r1ctS)z{ Which onnx opset to use when exporting the model Returns: Integer ONNX Opset version )DEFAULT_ONNX_OPSETr_s r2default_onnx_opsetzOnnxConfig.default_onnx_opsets "!r1cy)z What absolute tolerance value to use during model conversion validation. Returns: Float absolute tolerance value. gh㈵>r0r_s r2atol_for_validationzOnnxConfig.atol_for_validationsr1crtr-ddlm}tj||j k\Sy)z The minimum PyTorch version required to export the model. Returns: `bool`: Whether the installed version of PyTorch is compatible with the model. r)get_torch_versionF)rtransformers.utilsryrparsetorch_onnx_minimum_version)rTrys r2is_torch_support_availablez%OnnxConfig.is_torch_support_availables.   <==!2!459X9XX Xr1num_parameterscDt|tjtk\S)a Flag indicating if the model requires using external data format Args: num_parameters: Number of parameter on the model Returns: True if model.num_parameters() * size_of(float32) >= 2Gb False otherwise )rrFloatEXTERNAL_DATA_FORMAT_SIZE_LIMIT)r~s r2use_external_data_formatz#OnnxConfig.use_external_data_formats! /~?T?T U. / r1 batch_size num_channels image_height image_widthcg}t|D]h}tjj|||dz}|j t j |jdjdj|S)Nuint8RGB) rangenprandomrandrSr! fromarrayastypeconvert)rTrrrrimages_datas r2_generate_dummy_imagesz!OnnxConfig._generate_dummy_imagessjz" PA99>>, \JSPD MM%//$++g*>?GGN O P r1 sampling_rate time_duration frequencyc g}t|D]e}tjd|t||zd}|j dtj dtj z|z|zzg|S)NrF)endpointg?r)rrlinspaceintrSsinpi)rTrrrr audio_datarts r2_generate_dummy_audioz OnnxConfig._generate_dummy_audio sv z" GA A}c--2O.P[`aA   cBFF1ruu9y+@1+D$EE F  Gr1 preprocessor)r rr seq_length num_choicesis_pair framework tokenizerr cddlm}ddlm}ddlm}t ||r | td| 1tjdttjd| }t ||rAt|tjd }|j!|}t|tj"| }|j$$t'|j$dkDr |j$nd }d j)|g|zg|z}|j*d k(rt|tj,d }||z}||| }|j/D]2\}}t1dt'||Dcgc] }||||z c}||<4t3|j5|St3|||St ||r|j6ddk7r2td|j8j:d|j6dt|tj}|j=||| |}t3|||St ||rT|j6ddk(rBt|tj}|j=||| |}t3|||St ||rT|j6ddk(rBt|tj}|j?|| | | }t3|||Stdcc}w)am Generate inputs to provide to the ONNX exporter for the specific framework Args: preprocessor: ([`PreTrainedTokenizerBase`], [`FeatureExtractionMixin`], or [`ImageProcessingMixin`]): The preprocessor associated with this model configuration. batch_size (`int`, *optional*, defaults to -1): The batch size to export the model for (-1 means dynamic axis). num_choices (`int`, *optional*, defaults to -1): The number of candidate answers provided for multiple choice task (-1 means dynamic axis). seq_length (`int`, *optional*, defaults to -1): The sequence length to export the model for (-1 means dynamic axis). is_pair (`bool`, *optional*, defaults to `False`): Indicate if the input is a pair (sentence 1, sentence 2) framework (`TensorType`, *optional*, defaults to `None`): The framework (PyTorch or TensorFlow) that the tokenizer will generate tensors for. num_channels (`int`, *optional*, defaults to 3): The number of channels of the generated images. image_width (`int`, *optional*, defaults to 40): The width of the generated images. image_height (`int`, *optional*, defaults to 40): The height of the generated images. sampling_rate (`int`, *optional* defaults to 22050) The sampling rate for audio data generation. time_duration (`float`, *optional* defaults to 5.0) Total seconds of sampling for audio data generation. frequency (`int`, *optional* defaults to 220) The desired natural frequency of generated audio. Returns: Mapping[str, Tensor] holding the kwargs to provide to the model's forward function rrrrzPYou cannot provide both a tokenizer and a preprocessor to generate dummy inputs.ztThe `tokenizer` argument is deprecated and will be removed in version 5 of Transformers. Use `preprocessor` instead.zROverwriting the `preprocessor` argument with `tokenizer` to generate dummy inputs.r)fixed_dimensionnum_token_to_add0 rF) text_pair) tensor_type)return_tensors pixel_valuesz*The `preprocessor` is an image processor (zC) and expects `model_input_names[0]` to be "pixel_values", but got )r)rrinput_featuresz\Unable to generate dummy inputs for the model. Please provide a tokenizer or a preprocessor.) feature_extraction_utilsrimage_processing_utilsrtokenization_utils_baser isinstancerMwarningswarn FutureWarningloggerwarningrr4rknum_special_tokens_to_addrn unk_tokenlenjoinrHrqitemsrdictconvert_to_tensorsmodel_input_names __class__r*rr)rTrrrrrrrrrrrrrrrr token_to_add input_token dummy_inputtokenized_inputkvis r2generate_dummy_inputsz OnnxConfig.generate_dummy_inputss ` FAE l$; <AVop p  MM+  NNo p$L l$; <9J,J,J]^J(AA'JL9J,M,M`lJ !**63|?U?U;VYZ;Z&&  88[M2Z?@:MKyy--?1U1Uhi *K7 ".{k"R+113iDAqJOPQSVWXSY[fJg)hQ!AK*@)hOA&iO>>9>UVV [KL L  &: ;--a0NB @AWAWA`A`@abMMYMkMklmMnLoq :*V`VtVtuJ55j,P\^ijK K RS S  &< =,B`B`abBcguBu9*V`VtVtuJ55j,P\^ijK K RS S |%; <A_A_`aAbfvAv:*V`VtVtuJ44ZP]_hiK [KL Ln 5*is(L reference_model_inputsc|S)a Generate inputs for ONNX Runtime using the reference model inputs. Override this to run inference with seq2seq models which have the encoder and decoder exported as separate ONNX files. Args: reference_model_inputs ([`Mapping[str, Tensor]`): Reference inputs for the model. Returns: `Mapping[str, Tensor]`: The mapping holding the kwargs to provide to the model's forward function r0)rTrs r2!generate_dummy_inputs_onnxruntimez,OnnxConfig.generate_dummy_inputs_onnxruntimes &%r1c|jD]V}|j |jn|j|j}t|j|j |XyN)rOr)r'setattrr%r&)rTrUr's r2 patch_opszOnnxConfig.patch_opssO(( 2D*.//*AtW[WeWeGfI DFFDIIy 1 2r1c|jD]V}|j |jn|j|j}t|j|j |Xyr)rOr)r(rr%r&)rTrUr(s r2 restore_opszOnnxConfig.restore_opssO(( 0D&*oo&=dll4??SWS_S_C`G DFFDIIw / 0r1r&fieldcddlm}t|j|Dcic] \}}|d||c}}Scc}}w)a Flatten any potential nested structure expanding the name of the field with the index of the element within the structure. Args: name: The name of the nested structure field: The structure to, potentially, be flattened Returns: (Dict[str, Any]): Outputs with flattened structure and key mapping this new structure. r)chain.) itertoolsr enumerate from_iterable)r[r&rridxitems r2"flatten_output_collection_propertyz-OnnxConfig.flatten_output_collection_propertys? $7@ATATUZA[7\])#t4&#%]]]s:)rENrE)rrC(r)r"V@) rrFNrCrrrrrN)0r*r+r,r-rkrnrqrr{r|rrLr/r r$rW classmethodr\propertyrr rr`rer rrirlrorrrufloatrwboolr} staticmethodrrrrrrrrrr r rr0r1r2r4r4Ds !!.u!5 (J,G!HI 3Z5PQR +X7z7R,S T)%*5")j9")j9  !(J,G!HI (J,G!HI&1g,'?@'%*5")j9  *$+ ;")j9  "-hG Yafm8n-o!p!8=O-P"QR#.1g,/G#H +X7z7R,S T$hG 0K%LM$hG 0K%LM? D4144Z^_kZl4  &'9 & &Ua & &$WS#X%6 67$$-gc3h&7!78-- '#s(*;!<  .C..1114S44"C""U D      "fh14HK_bmp  25 NS fi  *.""9=vghvv v  v  vJ'vvvvvvv56v c vp &PSUXPXHY &^efiknfn^o &2 0 ^c^(3-^TXY\^aYaTb^^r1r4c|eZdZ ddddedeeedeffd Ze ddddeddfdZ e de ee e effffd Ze dee eeffd Ze de fd Ze de fd Z dd dde de dedeede eeff fd Z dde ee e effdedefdZdZdedeedeeefffd ZxZS) OnnxConfigWithPastrGrrHrIuse_pastc8t||||||_y)N)rHrI)superrWr)rTrGrHrIrrs r2rWzOnnxConfigWithPast.__init__s  d>J  r1rXc|||dS)z Instantiate a OnnxConfig with `use_past` attribute set to True Args: config: The underlying model's config to use when exporting to ONNX Returns: OnnxConfig with `.use_past = True` T)rHrr0rZs r2 with_pastzOnnxConfigWithPast.with_pasts6t44r1cZt|}|jr|j|d|S)Nre direction)rrerfill_with_past_key_values_)rTrdrs r2rezOnnxConfigWithPast.outputss, ==  + +Ni + Pr1cLt|jdrd|jiSy)Nrg)rhrKrr_s r2riz"OnnxConfigWithPast.values_overrides" 4<< -/ /r1cpt|jds td|jjS)z The number of layers attribute retrieved from the model config. Override this for model configs where the number of layers attribute is not called `num_layers`. num_layerszcould not find the number of layers attribute in the model configuration, override the num_layers property of the model OnnxConfig to solve this)rhrKAttributeErrorrr_s r2rzOnnxConfigWithPast.num_layerss7 t||\2 B ||&&&r1cpt|jds td|jjS)z The number of attention heads attribute retrieved from the model config. Override this for model configs where the number of attention heads attribute is not called `num_attention_heads`. num_attention_headszcould not find the number of attention heads attribute in the model configuration, override the num_attention_heads property of the model OnnxConfig to solve this)rhrKrrr_s r2rz&OnnxConfigWithPast.num_attention_headss8 t||%:; V ||///r1rr rrrrc0t||||||}|jrts t dddl}|dj \}} | dz} ||j| |jj|jzf} d|vr<|dj} |j|d|j|| | gd |d<g|d <t|jD]6} |d j|j!| |j!| f8|S) NrrrrACannot generate dummy past_keys inputs without PyTorch installed.r input_idsrattention_mask)dtyper)dimpast_key_values)rrrrrMtorchshaperrK hidden_sizercatonesrrrSzeros)rTrrrrr common_inputsrr8seqlenpast_key_values_lengthr mask_dtyperrs r2rz(OnnxConfigWithPast.generate_dummy_inputss@5 *W`i6  ==%' !dee)+6<C]3^ as$/ 0$=DIc9d!TF!A3f"56=DIc9d!TF!A3f"56  er1c<|d||d|d<|d||d|d<y)Nrrrrrr0rTflattened_outputr&rrs r2_flatten_past_key_values_z,OnnxConfigWithPast._flatten_past_key_values_?s:01!D63%t,-23A$D63%v./r1r&rci}|dvr)t|D]\}}|j|||||St| ||}|S)N)rr)rrrr)rTr&rrrrrs r2rz5OnnxConfigWithPast.flatten_output_collection_propertyCsd 1 1#E* OQ../?sAN O   %wI$PUV r1)rENFrrrFN)F)r*r+r,r/r listr$rrWrrrr rrerrirrrrrrr r r __classcell__rs@r2rrs7; !"!!!l!34 !  ! 51 5 5Ma 5 5gc3h&7!78'#s(*;!<  'C ' ' 0S 0 0*. (,(( (  ( J' ( c (Vqve!(gc3h.?)?!@eMPeime27 s 8C= UYZ]_bZbUc  r1rceZdZedeeeeeffffd Zedeeffd Z edeeffd Z dde ddeded e d e e deeeff fd Zd eeeeeffd efdZdZxZS)OnnxSeq2SeqConfigWithPastrXctt| }|jD]4\}}d|vrdnd}|jD]\}}d|vr|||<|||<6|jr|j |d|S)Nencoderencoder_sequencerDr9rer)rrrerrr)rTrdr& axes_names sequence_nameaxis_idxrs r2rez!OnnxSeq2SeqConfigWithPast.outputsOs14@ . 4 4 6 0 D*2;t2C.I[M","2"2"4 0$%+8Jx(,0Jx(  0 0 ==  + +Ni + Pr1c  t|}||f}|S#t$rgt|jdrEt|jdr/|jj |jj f}Y|StdwxYw)Nencoder_layersdecoder_layerszcould not find the number of encoder and decoder layers attributes in the model configuration, override the num_layers property of the model OnnxConfig to solve this)rrrrhrKr)r*)rTrrs r2rz$OnnxSeq2SeqConfigWithPast.num_layers`s +J$j1J t||%5674<?GDLLZsDt'+||'K'KT\\MqMq&r##" %  r+rr rrrrc tt| |||||}|js|nd}tt| |||||}|j D cic] \} } d| |  }} } t d i||} |jrt s tdddl} | djd} | djd}| djd}|j\}}| |||jj|zf}| ||dz|jj|zf}g| d <|j\}}t||}t|||z }||kDrd nd }t!|D]V}| d j#| j%|| j%|| j%|| j%|fX|d k(r|n|}t!||D]6}| d j#| j%|| j%|f8| Scc} } w) Nrrdecoder_rrrdecoder_input_idsrCrr#decoderr0)rrrrrrrrMrrrrKrrminmaxrrSr)rTrrrrrencoder_inputsdecoder_seq_lengthdecoder_inputsr&tensorr rr8encoder_seq_lengthnum_encoder_attention_headsnum_decoder_attention_heads encoder_shape decoder_shapenum_encoder_layersnum_decoder_layersmin_num_layersmax_num_layersremaining_side_namerrrs r2rz/OnnxSeq2SeqConfigWithPast.generate_dummy_inputsso14N *W`iO  04}}Z!14N *9KU\hqO IWH\H\H^_ fHTF+V3__@~@@ ==%' !dee!+.44Q7E!.{!;!A!A!!D !./B!C!I!I!!L GKG_G_ D ')D+" ((,GG M+"Q& ((,GG M02M+ ,59__ 2  2 !35GHN !35GH>YN/ADV/V)\e >* /077 M2 M2 M2 M2  &9I%EM=E>>: b/077U9KU[[Y^M_8`a be`sH r rc |dvrtd|d|dk(rdnd}|j\}}t||}t|||z }||kDrdnd}d } |dk(rd nd } t |D]:} d | d ||d| d<d | d ||d| d<d | d ||d| d<d | d ||d| d<<t ||D] } |dk(rd | d } nd | d } | ||d| d|d<"y)Nrrrr`rrr#r2past_encoder_sequencepast_decoder_sequencez past_decoder_sequence + sequencer8rr .decoder.key.decoder.value .encoder.key.encoder.valuer)rMrr3r4r) rTr rr&r>r?r@rArBr$rDr axes_infos r2rz4OnnxSeq2SeqConfigWithPast.fill_with_past_key_values_ss 1 1ST]S^^hij j$-$9 y26../1CD/1CD~U+=@R+RiXa26?86K2Qs~& _A?FK[;\ as,7 8AHM]=^ as.9 :?FK[;\ as,7 8AHM]=^ as.9 :  _ ~~6 SA"i/ ',<= ',<= IR as!,?+@E F  Sr1ct|d||d|d<|d||d|d<|d||d|d<|d||d|d <y) NrrrFrrGrrHrCrIr0rs r2rz3OnnxSeq2SeqConfigWithPast._flatten_past_key_values_sr89!D63%|45:;A$D63%~6789!D63%|45:;A$D63%~67r1r)r*r+r,rr r/rrerrrr rrrrrrrrs@r2r!r!Ns gc3h&7!78  E#J   #U3Z # #$*. C56CC C  C J' C c CJSGCQTVYQYIZDZ<[ShkS8?r1r!)3rbrPrabcrr collectionsrtypingrrrr r r r r rrnumpyr packagingrutilsrrrrrrrconfiguration_utilsrrrrrrr PILr! get_loggerr*rrtr dataclassr$r4rr!r0r1r2rVs ##fffPPhh6A=A   H %#9 ***(t^t^n P SP fW? 2W?r1