S^(hePdZddlZddlZddlmZddlmZmZmZm Z ddl Z ddl Z ddl m Z ddl mZddlmZdd lmZmZdd lmZmZdd lmZmZmZmZdd lmZmZdd lm Z m!Z!ddl"m#Z#m$Z$m%Z%m&Z&m'Z'ddl(m)Z)m*Z*m+Z+e&jXe-Z.dZ/de j`de j`fdZ1de j`de j`fdZ2dZ3dCdZ4 dDdZ5eGdde#Z6eGdde#Z7Gdde jpZ9Gd d!e jpZ:Gd"d#e jpZ;Gd$d%e jpZ<Gd&d'e jpZ=Gd(d)e jpZ>Gd*d+e jpZ?Gd,d-e jpZ@Gd.d/e jpZAGd0d1eZBd2ZCd3ZDd4ZEGd5d6eBZFGd7d8eBZGe$d9eCGd:d;eBZHe$deBeZIe$d?eCGd@dAeBeZJgdBZKy)EzPyTorch CLVP model.N) dataclass)DictOptionalTupleUnion)nn)CrossEntropyLoss)ACT2FN)GenerationConfigGenerationMixin)_prepare_4d_attention_mask!_prepare_4d_causal_attention_mask)BaseModelOutput)BaseModelOutputWithPastAndCrossAttentionsBaseModelOutputWithPooling!CausalLMOutputWithCrossAttentions)PreTrainedModelSequenceSummary)Conv1Disin_mps_friendly) ModelOutputadd_start_docstrings%add_start_docstrings_to_model_forwardloggingreplace_return_docstrings) ClvpConfigClvpDecoderConfigClvpEncoderConfigzsusnato/clvp_devlogitsreturnctjj|tjt ||j S)Ndevice)r functional cross_entropytorcharangelenr%)r!s d/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/transformers/models/clvp/modeling_clvp.pycontrastive_lossr,;s/ == & &vu||CKPVP]P]/^ __ similaritycZt|}t|j}||zdz S)Ng@)r,t)r. caption_loss speech_losss r+ clvp_lossr3@s,#J/L":<<>2K ; &# --r-c|dd|jddzf}|d|jddzdf}tj| |fdS)z*Rotates half the hidden dims of the input..Ndim)shaper(cat)xx1x2s r+ rotate_halfr>GsZ 3"!''"+"" " #B 3 q " " #B 99rc2YB ''r-c||j|}||j|}||zt||zz}||zt||zz}||zt||zz} ||| fS)anApplies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`): The position indices of the tokens corresponding to the query and key tensors. For example, this can be used to pass offsetted position ids when working with a KV-cache. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. ) unsqueezer>) qkvcossin position_ids unsqueeze_dimq_embedk_embedv_embeds r+apply_rotary_pos_embrKNs* l  % %m 4C l  % %m 4C3w;q>C/0G3w;q>C/0G3w;q>C/0G GW $$r-cT|r\tjjj|d|}|,tjjj|ddn|}|}|rBtj|j d|j ddzf|j |j}t|D]\}} t| |jrktj| |k(dj} tj| d| tj|g|j| | dg||<tjjj| d|||<|,tjjj|ddn|}||fS) z This method adds extra bos and eos tokens to input_ids and accordingly modifies the attention_mask which is used in `ClvpConditioningEncoder` and the generation loop of the `ClvpModelForConditionalGeneration`. )rrvalueNrrdtyper%r$)rr)r(rr&padzerosr9rPr% enumeratersumwheremin concatenatetensor) input_idsattention_mask pad_token_id bos_token_id eos_token_id add_bos_token add_eos_tokenmodified_input_idsi each_input_idposs r+_pad_extra_bos_eos_tokensrdksHH''++Iv\+R HVHbEHH   # #NF! # Dhv #"[[ __Q !3a!7 8 XaXhXh !*) 4 k A}  =AACkk-<"?@CGGI(-(9(9"4C(%,, ~iN^N^*_anorosatu)"1% ).(;(;(?(? v]i(?(j"1% kIWHbEHH   # #NF! # Dhv  ~ --r-ceZdZUdZdZeejed<dZ eejed<dZ eejed<dZ ee ejed<dZ ee ejed<y)ClvpEncoderOutputak Base class for CLVP encoder's outputs that contains a pooling of the last hidden states as well as a projection output (a linear layer on top of the pooled output). Args: embeds (`torch.FloatTensor` of shape `(batch_size, output_dim)`, *optional*, returned when model is initialized with `with_projection=True`): The embeddings obtained by applying the projection layer to the pooler_output. last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): The hidden state of the last layer of the model. pooler_output (`torch.FloatTensor` of shape `(batch_size, hidden_size)`): Pooled output of the `last_hidden_state`. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the optional initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. Nembedslast_hidden_state pooler_output hidden_states attentions)__name__ __module__ __qualname____doc__rgrr( FloatTensor__annotations__rhrirjrrkr-r+rfrfs}*+/FHU&& '.59x 1 12915M8E--.58