S^(h~, ddlmZmZddlZddlZddlmZddl m Z ddl m Z e jeZdeej ej"fdeefd Zdd ej d eed eedej fd ZddZ ddeefdZddZdej dej fdZddej dededdfdZdZdZdZdZ y) )OptionalUnionN) BatchFeature) BatchEncoding)loggingtensorreturnc\t|tjrt|jSt j|}|jt j dk(r|S|jj}t|Dcgc]\}}|||n|c}}Scc}}w)z Deal with dynamic shape in tensorflow cleanly. Args: tensor (`tf.Tensor` or `np.ndarray`): The tensor we want the shape of. Returns: `List[int]`: The shape of the tensor as a list. N) isinstancenpndarraylistshapetf TensorShapeas_list enumerate)r dynamicstaticiss S/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/transformers/tf_utils.py shape_listrs&"**%FLL!!hhvG ||r~~d++ \\ ! ! #F7@7H Itq!!)GAJ * II IsB(logitsaxisnamecLtjj|dz||S)a Stable wrapper that returns the same output as `tf.nn.softmax`, but that works reliably with XLA on CPU. It is meant as a workaround for the [following issue](https://github.com/tensorflow/tensorflow/issues/55682), and will be removed after it gets fixed. The arguments and outputs are the same as `tf.nn.softmax`, and relies on the fact that `softmax(x) = softmax(x + c)` (see https://ogunlao.github.io/2020/04/26/you_dont_really_know_softmax.html). Args: logits (`tf.Tensor`): Must be one of the following types: half, float32, float64. axis (`int`, *optional*): The dimension softmax would be performed on. The default is -1 which indicates the last dimension. name (`str`, *optional*): A name for the operation. Returns: `tf.Tensor`: A Tensor. Has the same type and shape as logits. g& .>rrr)rnnsoftmaxrs rstable_softmaxr"3s!* 55== Dt= DDc|jjdk7s)|jjdk7st|ts t dt j j||gd\}}|dk7rWdg|jjz}t||||<t j||}t j||}t j j||||||}|S)NrzOOnly 1D weight and bias tensors are supported for now, with only a single axis.T)axeskeepdims)offsetscalevariance_epsilon) rrankr intNotImplementedErrorrr momentsrreshapebatch_normalization) inputsweightbiasepsilonrmeanvarianceroutputss rfunctional_layernormr8Ks ||AA!5Zc=R!"sttUU]]6]FND( rzfll''' (.d FE*zz$&ee''  (G Nr#r)cL|dk7r td|r | td|r~tjtj|dtj|dftj}tj j j|d}||jjs|jjrVtj|dkDtjd|jtjd|j}tjd ||}|9tjtj|d |jd z}||z}|||z }tjj|}||zS) zDTF equivalent for torch's nn.functional.scaled_dot_product_attentionzDropout is not supported in this implementation - file an issue with Transformers and ping @Rocketknight1 if you need it for a port!z?You cannot specify an attn_mask and is_causal at the same time!dtyper)kg@z...qd, ...kd -> ...qkr'g) ValueErrorronesrint32 experimentalnumpytrilr= is_integeris_boolwherecasteinsumr r!) querykeyvalue attn_mask dropout_p is_causalr)rprobss rscaled_dot_product_attentionrQksRC S  Y*Z[[GGRXXe_R0"((3-2CDBHHU OO))..yA.> )//"<"< @W@WHHY]BGGC,ErwwwX]XcXcGde YY.s ;F } b)6<<8D@ eOF) EEMM& !E 5=r#cp|dkr||jjz }|dkr||jjz }||k(r|Stj|}tjj |||dz}tj |d||g||dzdgd}tj ||S)Nrrr)rr+rmath reduce_prodconcatr/)input start_dimend_dimin_shape flattened_dim out_shapes rflattenr]s{5;;###1}U[[%%% G xxHGG''Wq[(IJM 8JY/-(7UV;=BYZabcI ::eY ''r#encoder_attention_maskctt|tjstj|}|jj dk(r|dddddddf}|jj dk(r |ddddddf}tj d|jz |jjz}|S)z Invert an attention mask (e.g., switches 0. and 1.). Args: encoder_attention_mask (`torch.Tensor`): An attention mask. Returns: `tf.Tensor`: The inverted attention mask. Nr) r rTensorconvert_to_tensorrr+rHr=min)r^encoder_extended_attention_masks rinvert_attention_maskrfs ,bii 8!#!5!56L!M##((A-*@D!Q*O'##((A-*@D$PQAQ*R' )//03RR'--11'2# +*r# embed_dim tensor_namec tjj|tj||jd|dtj j |d|dy)a `tf.gather`, on which TF embedding layers are based, won't check positive out of bound indices on GPU, returning zeros instead. This function adds a check against that dangerous silent behavior. Args: tensor (`tf.Tensor`): The tensor of indices to check. embed_dim (`int`): The embedding dimension. tensor_name (`str`, *optional*): The name of the tensor to use in the error message. r<zThe maximum value of z (z>) must be smaller than the embedding layer's input dimension (z9). The likely cause is some problem at tokenization time.)messageN)r debugging assert_lessrHr=rT reduce_max)r rgrhs rcheck_embeddings_within_boundsrnsaLL  .#K=2773E3Ef3M2NO((1{2k m r#c d |Dcgc]}t| kDs|}}|rtd d|tj|}d}tj||}t fd|Dr0|dz }tj||}t fd|Dr0|dkDr(t |D]\}} | |jd||fz<y||j|<ycc}w)aSaves attributes (data) of the specified name into the HDF5 group. This method deals with an inherent problem of HDF5 file which is not able to store data larger than HDF5_OBJECT_HEADER_LIMIT bytes. Args: group: A pointer to a HDF5 group. name: A name of the attributes to save. data: Attributes data to store. Raises: RuntimeError: If any single attribute is too large to be saved. Copied from Keras to Transformers to avoid versioning issues. izSThe following attributes cannot be saved to HDF5 file because they are larger than z bytes: rc3<K|]}|jkDyw)N)nbytes).0xHDF5_OBJECT_HEADER_LIMITs r z0save_attributes_to_hdf5_group..s Haahh11 Hs%s%dN)len RuntimeErrorr asarray array_splitanyrattrs) grouprdatarsbad_attributesdata_npy num_chunks chunked_datachunk_id chunk_datarts @rsave_attributes_to_hdf5_grouprs  %"&KAQ2J)JaKNK $$<#=>$% '  zz$HJ>>(J7L  H< H Ha ~~h ;   H< H HA~$-l$; @ Hj5?EKK$!11 2 @! D1Ls C#C#c ||jvr;|j|Dcgc]!}t|dr|jdn|#}}|Sg}d}d||fz|jvrg|j|jd||fzDcgc]!}t|dr|jdn|#c}|dz }d||fz|jvrg|Scc}wcc}w)aLoads attributes of the specified name from the HDF5 group. This method deals with an inherent problem of HDF5 file which is not able to store data larger than HDF5_OBJECT_HEADER_LIMIT bytes. Args: group: A pointer to a HDF5 group. name: A name of the attributes to load. Returns: data: Attributes data. Copied from Keras to Transformers to avoid versioning issues. decodeutf8rrvr)r|hasattrrextend)r}rnr~rs rload_attributes_from_hdf5_grouprs u{{INUYIZ[AGAx$8 a?[[ Kh''5;;6 KKJO++V\`dfn_oVoJpqQWQ%9&!q@q  MH h''5;;6 K\ rs &C &CcHd}tjj||S)zwExpands 1-dimensional `Tensor`s into 2-dimensional `Tensor`s. Copied from Keras to here to avoid versioning issues.ct|tjr0|jjdk(rtj |dS|S)Nrr'rS)r rrbrr+ expand_dims)ts r_expand_single_1d_tensorz+expand_1d.._expand_single_1d_tensors5 a # (9>>!"- -r#)rnest map_structure)r~rs r expand_1drs! 77 !94 @@r#c|r9t|dttfr t|}t |d|d<||fSd|vr*t|dttfrt |d|d<||fS)Nrrs)r rrrdict)argskwargss rconvert_batch_encodingrst 47]L$ABDztAw-Q < :fSkM<3PQ6#;'s <r#)NN)gh㈵>r')Nr:FN)rr') input_ids)!typingrrrCr tensorflowrfeature_extraction_utilsrtokenization_utils_baserutilsr get_logger__name__loggerrbrrr,rstrr"r8floatrQr]rfrnrrrrr#rrs#22   H %JuRYY 23JS J.E299EHSMEQT EacajajE0BaeNVW\o6($+"))+ +8299SVim(,!^8 Ar#