T^(h @dZddlZddlmZddlmZmZmZmZddl Z ddl Z ddl m Z ddl m Z ddlmZmZmZmZdd lmZdd lmZmZdd lmZdd lmZmZdd lmZddlm Z m!Z!m"Z"m#Z#m$Z$ddl%m&Z&erddlm'Z'e#jPe)Z*dZ+eGdde Z,eGdde Z-eGdde Z.Gdde j^Z0Gdde j^Z1Gdde j^Z2Gdd e j^Z3Gd!d"e j^Z4Gd#d$e j^Z5d%Z6dId&Z7Gd'd(e j^Z8d)e jrd*e:d+e jrfd,Z;Gd-d.e j^Z<Gd/d0e<Z=Gd1d2e<Z>ed3Z?Gd4d5e j^Z@Gd6d7e j^ZAGd8d9e j^ZBGd:d;e j^ZCGd<d=e j^ZDGd>d?e j^ZEGd@dAe j^ZFGdBdCeZGdDZHdEZIe!dFeHGdGdHeGZJdHdCgZKy)JzPyTorch Mimi model.N) dataclass)ListOptionalTupleUnion)nn)ACT2FN)Cache DynamicCacheSlidingWindowCache StaticCache)AttentionMaskConverter)!flash_attn_supports_top_left_maskis_flash_attn_available)BaseModelOutputWithPast)ROPE_INIT_FUNCTIONSdynamic_rope_update)PreTrainedModel) ModelOutputadd_start_docstrings%add_start_docstrings_to_model_forwardloggingreplace_return_docstrings) MimiConfig)_flash_attention_forwardrceZdZUdZdZeejed<dZ eejed<dZ ee e eejfed<dZee e eejfed<y) MimiOutputa Args: audio_codes (`torch.LongTensor` of shape `(batch_size, num_quantizers, codes_length)`, *optional*): Discret code embeddings computed using `model.encode`. audio_values (`torch.FloatTensor` of shape `(batch_size, sequence_length)`, *optional*) Decoded audio values, obtained using the decoder part of Mimi. encoder_past_key_values (`Cache`, *optional*): Pre-computed hidden-states (key and values in the self-attention blocks) that can be used to speed up sequential decoding of the encoder transformer. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`. The model will output the same cache format that is fed as input. If `past_key_values` are used, the user can optionally input only the last `audio_values` or `audio_codes (those that don't have their past key value states given to this model). decoder_past_key_values (`Cache`, *optional*): Pre-computed hidden-states (key and values in the self-attention blocks) that can be used to speed up sequential decoding of the decoder transformer. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`. The model will output the same cache format that is fed as input. If `past_key_values` are used, the user can optionally input only the last `audio_values` or `audio_codes (those that don't have their past key value states given to this model). N audio_codes audio_valuesencoder_past_key_valuesdecoder_past_key_values)__name__ __module__ __qualname____doc__r rtorch LongTensor__annotations__r! FloatTensorr"rr rr#d/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/transformers/models/mimi/modeling_mimi.pyrr4s}0/3K%**+204L(5,,-4OSXeE48I8I3J,J&KLSOSXeE48I8I3J,J&KLSr-rcreZdZUdZdZeejed<dZ ee e e ejfed<y)MimiEncoderOutputaY Args: audio_codes (`torch.LongTensor` of shape `(batch_size, num_quantizers, codes_length)`, *optional*): Discret code embeddings computed using `model.encode`. encoder_past_key_values (`Cache`, *optional*): Pre-computed hidden-states (key and values in the self-attention blocks) that can be used to speed up sequential decoding of the encoder transformer. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`. The model will output the same cache format that is fed as input. If `past_key_values` are used, the user can optionally input only the last `audio_values` or `audio_codes (those that don't have their past key value states given to this model). Nr r")r$r%r&r'r rr(r)r*r"rr rr+r,r-r.r0r0TsC /3K%**+2OSXeE48I8I3J,J&KLSr-r0creZdZUdZdZeejed<dZ ee e e ejfed<y)MimiDecoderOutputaU Args: audio_values (`torch.FloatTensor` of shape `(batch_size, segment_length)`, *optional*): Decoded audio values, obtained using the decoder part of Mimi. decoder_past_key_values (`Cache`, *optional*): Pre-computed hidden-states (key and values in the self-attention blocks) that can be used to speed up sequential decoding of the decoder transformer. This typically consists in the `past_key_values` returned by the model at a previous stage of decoding, when `use_cache=True` or `config.use_cache=True`. The model will output the same cache format that is fed as input. If `past_key_values` are used, the user can optionally input only the last `audio_values` or `audio_codes (those that don't have their past key value states given to this model). Nr!r#) r$r%r&r'r!rr(r+r*r#rr rr,r-r.r2r2hsC 15L(5,,-4OSXeE48I8I3J,J&KLSr-r2ceZdZdZ ddededededededeffd Zd Zd Zd e jd e jfdZ e dd e jde eefdedefdZdZxZS) MimiConv1dz;Conv1d with asymmetric or causal padding and normalization. in_channels out_channels kernel_sizestridedilationgroupsbiasc t ||j|_| |jn||_|dkDr$|dkDrt j d|d|d|dtj||||||| |_ |jjd}tj|jjdtj}|jjd}tj|dz |zdztj}|j!d |d |j!d |d |j!d ||z d |j"dz|_|j"|j$z |_y)NrzNMimiConv1d has been initialized with stride > 1 and dilation > 1 (kernel_size=z stride=z , dilation=).)r9r:r;rdtyper8F persistentr7 padding_total)super__init__use_causal_convcausalpad_modeloggerwarningrConv1dconvr7r(tensorr8int64r9register_bufferrB padding_right padding_left) selfconfigr5r6r7r8r9r:rHr; __class__s r.rEzMimiConv1d.__init__so ,, +3+;  A:(Q, NN!!, XfX[ RTV  II {FXV\cg  ii++A. dii..q1E99%%a(llK!Ox#?!#C5;;W  Xv%@ ]KEJ _kF.BuU"//14 ..1C1CCr-ctjj}ttjjdr$tjjj}||j yN weight_normrutilsrWhasattrparametrizationsrLrRrWs r.apply_weight_normzMimiConv1d.apply_weight_normFhh** 288,,m <((33??KDIIr-cVtjj|jyNrrYremove_weight_normrLrRs r.rbzMimiConv1d.remove_weight_norm ##DII.r- hidden_statesreturnc>|jd}||jz |jz|jz dz}t j |j tjdz }||jz|jz|jz }||z S)zSee `pad_for_conv1d`.r)shaper7rBr8r(ceiltorN)rRrelengthn_frames ideal_lengths r._get_extra_padding_for_conv1dz(MimiConv1d._get_extra_padding_for_conv1ds $$R(T---0B0BBdkkQTUU::h'**5;;7!;$++-0@0@@4CUCUU f$$r-paddingsmodevaluecl|jd}|\}}|dk(s"tjj||||St ||}d}||kr*||z dz}tjj|d|f}tjj||||} | jd|z } | dd| fS)zTiny wrapper around torch.nn.functional.pad, just to allow for reflect padding on small input. If this is the case, we insert extra 0 padding to the right before the reflection happens. rhreflectrr.N)rir functionalpadmax) rerprqrrrlrQrPmax_pad extra_padpaddedends r._pad1dzMimiConv1d._pad1ds $$R(&.# my ==$$]HdEJ JlM2 W &(1,IMM--ma^LM""=(D%Hll2*c4C4i  r-c&|j|}|jr+|j||j|f|j}n7|j||j |j |zf|j}|j|}|S)N)rq)rorGr|rBrHrQrPrL)rRre extra_paddings r.forwardzMimiConv1d.forwards::=I ;; KK 8J8JM7ZaeananKoM KK 1 143E3E 3UV]a]j]j(M -0 r-)rrrNT)zero)r$r%r&r'intboolrEr]rbr(Tensorro staticmethodrstrfloatr|r __classcell__rTs@r.r4r4|sE(D(D (D  (D  (D(D(D(DT/ %|| %  %!ell!eCHo!S!bg!!$ r-r4c ReZdZdZ d dededededef fd ZdZd Zd ZxZ S) MimiConvTranspose1dzDConvTranspose1d with asymmetric or causal padding and normalization.r5r6r7r8r:ct ||j|_|j|_t j |||||||_|js|jdk(s td|jjd}|jjd}||z }|jr(tj||jz|_ n |dz|_ ||jz |_y)N)r:r;?zB`trim_right_ratio` != 1.0 only makes sense for causal convolutionsrrC)rDrErFrGtrim_right_ratiorConvTranspose1drL ValueErrorr7r8mathrjrPrQ) rRrSr5r6r7r8r:r;rBrTs r.rEzMimiConvTranspose1d.__init__s ,, & 7 7&&{L+v^dkop  t44;ab bii++A. !!!$#f, ;;"&=4;P;P+P!QD "/!!3D )D,>,>>r-ctjj}ttjjdr$tjjj}||j yrVrXr\s r.r]z%MimiConvTranspose1d.apply_weight_norm r^r-cVtjj|jyr`rarcs r.rbz&MimiConvTranspose1d.remove_weight_normrdr-c|j|}|jd|jz }|d|j|f}|S)Nrh.)rLrirPrQ)rRrer{s r.rzMimiConvTranspose1d.forwardsM -0 !!"%(:(::%c4+<+ *v/B/B/N#0044[&BUBUBYBYZ`BabDN&DN"("@"@$*$B$B! /?+/+<+)rrexpandrirkr isinstancerrr(autocast transposecatcosrsinr?) rRr position_idsinv_freq_expandedposition_ids_expandedrfreqsembrrs r.rzMimiRotaryEmbedding.forwardsV!MM$4-8>>@GG HZHZ[\H]_acdehhijiqiqr ,QaZ 8 > > @'1!((--'E!((--[`J`ahhmmfk ^^ U C 5&,,.1F1L1L1NNYYZ[]^_E))UEN3C'')d444C'')d444C  5 vvAGGv$cff177f&;;;  5 5s BF%%F.r`) r$r%r&rrEr(no_gradrrrrs@r.rrns3/z/"U]]_ <