T^(h"tdZddlmZmZmZddlmZddlmZeje Z GddeZ dgZ y) zProphetNet model configuration)CallableOptionalUnion)PretrainedConfig)loggingc6eZdZdZdZdgZddiZ d#deedee e e fdee d ee d ee d ee dee d ee d ee dee deedeedee deedee dee dee dee dee dee dee deedee dee dee dee f4fd Zed e fd!Zej"d"ZxZS)$ProphetNetConfiga This is the configuration class to store the configuration of a [`ProphetNetModel`]. It is used to instantiate a ProphetNet model according to the specified arguments, defining the model architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of the ProphetNet [microsoft/prophetnet-large-uncased](https://huggingface.co/microsoft/prophetnet-large-uncased) architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: activation_dropout (`float`, *optional*, defaults to 0.1): The dropout ratio for activations inside the fully connected layer. activation_function (`str` or `function`, *optional*, defaults to `"gelu"`): The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`, `"relu"`, `"silu"` and `"gelu_new"` are supported. vocab_size (`int`, *optional*, defaults to 30522): Vocabulary size of the ProphetNET model. Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling [`ProphetNetModel`]. hidden_size (`int`, *optional*, defaults to 1024): Dimensionality of the layers and the pooler layer. encoder_ffn_dim (`int`, *optional*, defaults to 4096): Dimensionality of the "intermediate" (often named feed-forward) layer in decoder. num_encoder_layers (`int`, *optional*, defaults to 12): Number of encoder layers. num_encoder_attention_heads (`int`, *optional*, defaults to 16): Number of attention heads for each attention layer in the Transformer encoder. decoder_ffn_dim (`int`, *optional*, defaults to 4096): Dimensionality of the `intermediate` (often named feed-forward) layer in decoder. num_decoder_layers (`int`, *optional*, defaults to 12): Number of decoder layers. num_decoder_attention_heads (`int`, *optional*, defaults to 16): Number of attention heads for each attention layer in the Transformer decoder. attention_dropout (`float`, *optional*, defaults to 0.1): The dropout ratio for the attention probabilities. dropout (`float`, *optional*, defaults to 0.1): The dropout probability for all fully connected layers in the embeddings, encoder, and pooler. max_position_embeddings (`int`, *optional*, defaults to 512): The maximum sequence length that this model might ever be used with. Typically set this to something large just in case (e.g., 512 or 1024 or 2048). init_std (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. add_cross_attention (`bool`, *optional*, defaults to `True`): Whether cross-attention layers should be added to the model. is_encoder_decoder (`bool`, *optional*, defaults to `True`): Whether this is an encoder/decoder model. pad_token_id (`int`, *optional*, defaults to 1) Padding token id. bos_token_id (`int`, *optional*, defaults to 0) Beginning of stream token id. eos_token_id (`int`, *optional*, defaults to 2) End of stream token id. ngram (`int`, *optional*, defaults to 2) Number of future tokens to predict. Set to 1 to be same as traditional Language model to predict next first token. num_buckets (`int`, *optional*, defaults to 32) The number of buckets to use for each attention layer. This is for relative position calculation. See the [T5 paper](see https://arxiv.org/abs/1910.10683) for more details. relative_max_distance (`int`, *optional*, defaults to 128) Relative distances greater than this number will be put into the last same bucket. This is for relative position calculation. See the [T5 paper](see https://arxiv.org/abs/1910.10683) for more details. disable_ngram_loss (`bool`, *optional*, defaults to `False`): Whether be trained predicting only the next first token. eps (`float`, *optional*, defaults to 0.0): Controls the `epsilon` parameter value for label smoothing in the loss calculation. If set to 0, no label smoothing is performed. use_cache (`bool`, *optional*, defaults to `True`): Whether or not the model should return the last key/values attentions (not used by all models). prophetnetpast_key_valuesnum_attention_headsnum_encoder_attention_headsactivation_dropoutactivation_function vocab_size hidden_sizeencoder_ffn_dimnum_encoder_layersdecoder_ffn_dimnum_decoder_layersnum_decoder_attention_headsattention_dropoutdropoutmax_position_embeddingsinit_stdis_encoder_decoderadd_cross_attentiondecoder_start_token_idngram num_bucketsrelative_max_distancedisable_ngram_losseps use_cache pad_token_id bos_token_id eos_token_idc J||_||_||_||_||_||_| |_| |_| |_||_ ||_ ||_ ||_ ||_ ||_||_| |_||_| |_||_t)|Td||||||d|y)N)r%r&r'rrr)rrrrrrrrrrrrr r!r"r#rrrr$super__init__)selfrrrrrrrrrrrrrrrrrrr r!r"r#r$r%r&r'kwargs __class__s u/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/transformers/models/prophetnet/configuration_prophetnet.pyr+zProphetNetConfig.__init__fs<%&."4+F(."4+F('>$  #6  &%:""4"3"4 "  %%%1 3#9   returnc4|j|jzS)N)rr)r,s r/num_hidden_layersz"ProphetNetConfig.num_hidden_layerss&&)@)@@@r0ctd)NzyThis model does not support the setting of `num_hidden_layers`. Please set `num_encoder_layers` and `num_decoder_layers`.)NotImplementedError)r,values r/r3z"ProphetNetConfig.num_hidden_layerss! %  r0)皙?gelui:wi r9r:r;r7r7ig{Gz?TTr FgTrr<)__name__ __module__ __qualname____doc__ model_typekeys_to_ignore_at_inference attribute_maprfloatrstrrintboolr+propertyr3setter __classcell__)r.s@r/r r sDCJJ#4"5<M /2>D$)%))-,.57)-,.57-0#&14$(-1.201 %'/2-2"$(&'&'&'7@ $UO@ &eCM&:;@ SM @ c] @ "# @ %SM@ &.c]@ "#@ %SM@ &.c]@ $E?@ %@ "*#@ 5/@ %TN!@ "&d^#@ $!) %@ &}'@ (c])@ * (}+@ ,%TN-@ .e_/@ 0D>1@ 2sm3@ 4sm5@ 6sm7@ DA3AA  r0r N)rCtypingrrrconfiguration_utilsrutilsr get_loggerr@loggerr __all__r)r0r/rTsD%,,3   H %W 'W t  r0