l^(huO\ddlmZddlZddlZddlmZddlmZmZddl Z ddl m Z ddl m Z ddlmZmZmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZmZddlm Z ddl!m"Z"ddl#m$Z$m%Z%ddl&m'Z'ddl(m)Z)m*Z*e)r ddl+m,Z,m-Z-m.Z.ej^e0Z1Gdde'Z2y)) annotationsN)partial)AnyCallable)parse)nn)EvalPredictionPreTrainedTokenizerBaseTrainerCallback) __version__) DataCollator) WandbCallback CrossEncoder)CrossEncoderDataCollator)BinaryCrossEntropyLossCrossEntropyLoss)CrossEncoderModelCardCallback)CrossEncoderTrainingArguments)SentenceEvaluatorSequentialEvaluator)SentenceTransformerTrainer)is_datasets_availableis_training_available)Dataset DatasetDictIterableDatasetceZdZdZ d d fd Zd dZ d dZd dZd fd ZddZ xZ S)CrossEncoderTraineru CrossEncoderTrainer is a simple but feature-complete training and eval loop for PyTorch based on the 🤗 Transformers :class:`~transformers.Trainer`. This trainer integrates support for various :class:`transformers.TrainerCallback` subclasses, such as: - :class:`~transformers.integrations.WandbCallback` to automatically log training metrics to W&B if `wandb` is installed - :class:`~transformers.integrations.TensorBoardCallback` to log training metrics to TensorBoard if `tensorboard` is accessible. - :class:`~transformers.integrations.CodeCarbonCallback` to track the carbon emissions of your model during training if `codecarbon` is installed. - Note: These carbon emissions will be included in your automatically generated model card. See the Transformers `Callbacks `_ documentation for more information on the integrated callbacks and how to write your own callbacks. Args: model (:class:`~sentence_transformers.SentenceTransformer`, *optional*): The model to train, evaluate or use for predictions. If not provided, a `model_init` must be passed. args (:class:`~sentence_transformers.training_args.SentenceTransformerTrainingArguments`, *optional*): The arguments to tweak for training. Will default to a basic instance of :class:`~sentence_transformers.training_args.SentenceTransformerTrainingArguments` with the `output_dir` set to a directory named *tmp_trainer* in the current directory if not provided. train_dataset (Union[:class:`datasets.Dataset`, :class:`datasets.DatasetDict`, :class:`datasets.IterableDataset`, Dict[str, :class:`datasets.Dataset`]], *optional*): The dataset to use for training. Must have a format accepted by your loss function, see `Training Overview > Dataset Format <../../../docs/sentence_transformer/training_overview.html#dataset-format>`_. eval_dataset (Union[:class:`datasets.Dataset`, :class:`datasets.DatasetDict`, :class:`datasets.IterableDataset`, Dict[str, :class:`datasets.Dataset`]], *optional*): The dataset to use for evaluation. Must have a format accepted by your loss function, see `Training Overview > Dataset Format <../../../docs/sentence_transformer/training_overview.html#dataset-format>`_. loss (Optional[Union[:class:`torch.nn.Module`, Dict[str, :class:`torch.nn.Module`], Callable[[:class:`~sentence_transformers.SentenceTransformer`], :class:`torch.nn.Module`], Dict[str, Callable[[:class:`~sentence_transformers.SentenceTransformer`]]]], *optional*): The loss function to use for training. Can either be a loss class instance, a dictionary mapping dataset names to loss class instances, a function that returns a loss class instance given a model, or a dictionary mapping dataset names to functions that return a loss class instance given a model. In practice, the latter two are primarily used for hyper-parameter optimization. Will default to :class:`~sentence_transformers.losses.CoSENTLoss` if no ``loss`` is provided. evaluator (Union[:class:`~sentence_transformers.evaluation.SentenceEvaluator`, List[:class:`~sentence_transformers.evaluation.SentenceEvaluator`]], *optional*): The evaluator instance for useful evaluation metrics during training. You can use an ``evaluator`` with or without an ``eval_dataset``, and vice versa. Generally, the metrics that an ``evaluator`` returns are more useful than the loss value returned from the ``eval_dataset``. A list of evaluators will be wrapped in a :class:`~sentence_transformers.evaluation.SequentialEvaluator` to run them sequentially. callbacks (List of [:class:`transformers.TrainerCallback`], *optional*): A list of callbacks to customize the training loop. Will add those to the list of default callbacks detailed in [here](callback). If you want to remove one of the default callbacks used, use the [`Trainer.remove_callback`] method. optimizers (`Tuple[:class:`torch.optim.Optimizer`, :class:`torch.optim.lr_scheduler.LambdaLR`]`, *optional*, defaults to `(None, None)`): A tuple containing the optimizer and the scheduler to use. Will default to an instance of :class:`torch.optim.AdamW` on your model and a scheduler given by :func:`transformers.get_linear_schedule_with_warmup` controlled by `args`. Important attributes: - **model** -- Always points to the core model. If using a transformers model, it will be a [`PreTrainedModel`] subclass. - **model_wrapped** -- Always points to the most external model in case one or more other modules wrap the original model. This is the model that should be used for the forward pass. For example, under `DeepSpeed`, the inner model is wrapped in `DeepSpeed` and then again in `torch.nn.DistributedDataParallel`. If the inner model hasn't been wrapped, then `self.model_wrapped` is the same as `self.model`. - **is_model_parallel** -- Whether or not a model has been switched to a model parallel mode (different from data parallelism, this means some of the model layers are split on different GPUs). - **place_model_on_device** -- Whether or not to automatically place the model on the device - it will be set to `False` if model parallel or deepspeed is used, or if the default `TrainingArguments.place_model_on_device` is overridden to return `False` . - **is_in_train** -- Whether or not a model is currently running `train` (e.g. when `evaluate` is called while in `train`) c j ts td|(d}tjd|dt |}nt |ts t d|%| | |_|j}n)td| tjd| |_| tjd t d j}|jr;|jjs%|jj|j|&t |jt r |j}|t#t%|d d d }t'ddg||gD]U\}}t |t(s2t |t*s't-d|j/DsHt d|dt |t*rt |t0s t1|}t |t*rt |t0s t1|}|jrdn|||||||nd| | | | | d }t3t4t3dk\r||d<n||d<|*|(|j6dk7rt d|j6dt9t:|zd+i||j>dk(rd|_d |_ i|_!|||t-|jDjFDcgc]}t |tHc}r tJjLjOdd|n|jPjRdk(r+tjdtU|jP}n*tjd tW|jP}t |t*r|jYDcic]\}}||j[||c}}|_.t'ddg||gD]\}}| t |t*st d!|d"t_|jat_|jaz x}s^t d#|d$tc|d%te|dk(rd&nd'd(|d) n|j[|||_.|t |tfs ti|}||_5|jl#|jo||jpd*|_6|j>#|jo||jpd*|_|js|ycc}wcc}}w),NzTo train a CrossEncoder model, you need to install the `accelerate` and `datasets` modules. You can do so with the `train` extra: pip install -U "sentence-transformers[train]" tmp_trainerz=No `CrossEncoderTrainingArguments` passed, using `output_dir=z`.) output_dirzQPlease use `CrossEncoderTrainingArguments` imported from `sentence_transformers`.z<`Trainer` requires either a `model` or `model_init` argumentz`Trainer` requires either a `model` or `model_init` argument, but not both. `model_init` will overwrite your model when calling the `train` method.z`compute_metrics` is currently not compatible with the CrossEncoderTrainer. Please use the `evaluator` argument instead for detailed evaluation metrics, or the `eval_dataset` argument for the evaluation loss.unusedTpt)padding truncationreturn_tensors) tokenize_fntrainevalc3<K|]}t|tyw)N) isinstancer).0ds i/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/sentence_transformers/cross_encoder/trainer.py z/CrossEncoderTrainer.__init__..s1kUV*Q2P1kszACrossEncoderTrainer does not support an IterableDataset for the `zg_dataset`. Please convert the dataset to a `Dataset` or `DatasetDict` before passing it to the trainer.dummy) modelargs data_collator train_dataset eval_dataset model_initcompute_metrics callbacks optimizerspreprocess_logits_for_metricsz4.46.0processing_class tokenizernoz%You have set `args.eval_strategy` to z, but you didn't provide an `eval_dataset` or an `evaluator`. Either provide an `eval_dataset` or an `evaluator` to `CrossEncoderTrainer`, or set `args.eval_strategy='no'` to skip evaluation. WANDB_PROJECTzsentence-transformerszLNo `loss` passed, using `losses.BinaryCrossEntropyLoss` as a default option.zFNo `loss` passed, using `losses.CrossEntropyLoss` as a default option.z,If the provided `loss` is a dict, then the `z"_dataset` must be a `DatasetDict`.z:If the provided `loss` is a dict, then all keys from the `z;_dataset` dictionary must occur in `loss` also. Currently, z occursz in `z_dataset` but not in `loss`.) dataset_name):r RuntimeErrorloggerinforr, ValueErrorr7call_model_initwarningto_dict hub_model_idmodel_card_datamodel_id set_model_idr=r rrziprdictanyvaluesr parse_versiontransformers_version eval_strategysuperr__init__r6can_return_loss_prompt_length_mappingcallback_handlerr9rosenviron setdefaultr2 num_labelsrritems prepare_losslosssetkeyssortedlenrr evaluatorr5(maybe_add_prompts_or_dataset_name_columnpromptsadd_model_card_callback)selfr2r3r5r6rbrgr4r=r7r8r9r:r;r"default_args_dictrCdataset super_kwargscallbackloss_fnmissing __class__s r/rXzCrossEncoderTrainer.__init__es9(%&@  <&J KKWXbWccef g0JGDD"?@pq q =%",,,."#abb%M)DO  & NN' :XNVVX   U%:%:%C%C  ! ! . .t/@/@ A  EOO=T!UI  4#It]abM&)'6):]LP\@])^ % g?!'40$F|nTvw"',,.1C 4DDD7D$TUaTbc&&,Wo%6fCLTUDUS[]<^^cdpcqrNO ))$6DI  I?P)Q+I6I"    )!%!N!Nt||'"O"D     ( $ M Mdll!N!D  $$%67Sets 'T*8T/ct|}|j||j|j|j|j |j |y)ah Add a callback responsible for automatically tracking data required for the automatic model card generation This method is called in the ``__init__`` method of the :class:`~sentence_transformers.trainer.SentenceTransformerTrainer` class. Args: default_args_dict (Dict[str, Any]): A dictionary of the default training arguments, so we can determine which arguments have been changed for the model card. .. note:: This method can be overriden by subclassing the trainer to remove/customize this callback in custom uses cases )r2trainerN)r add_callback on_init_endr3statecontrolr2)rkrlmodel_card_callbacks r/rjz+CrossEncoderTrainer.add_model_card_callbacksL <>s4yI ZZ ZZ%%  13TKM $ #DJ0DJJ    !% #DJ0DJJ sAB<C < CC &C1cy)ac Return whether the prompt length should be passed to the model's forward method. This is never the case for CrossEncoder models, as the prompt length is not used in the forward method, unlike with Sentence Transformers models, where it may be relevant to mask out the prompt tokens in the embedding pooling step. FrD)rks r/_include_prompt_lengthz*CrossEncoderTrainer._include_prompt_lengthOsrz) NNNNNNNNNNN)NNN)r2zCrossEncoder | Noner3rr51Dataset | DatasetDict | dict[str, Dataset] | Noner6rrbznn.Module | dict[str, nn.Module] | Callable[[CrossEncoder], torch.nn.Module] | dict[str, Callable[[CrossEncoder], torch.nn.Module]] | Nonergz2SentenceEvaluator | list[SentenceEvaluator] | Noner4zDataCollator | Noner=z)PreTrainedTokenizerBase | Callable | Noner7z!Callable[[], CrossEncoder] | Noner8z'Callable[[EvalPrediction], dict] | Noner9zlist[TrainerCallback] | Noner:z?tuple[torch.optim.Optimizer, torch.optim.lr_scheduler.LambdaLR]r;z;Callable[[torch.Tensor, torch.Tensor], torch.Tensor] | NonereturnNone)rlzdict[str, Any]rr)rzdict[str, torch.Tensor | Any]rz9tuple[list[dict[str, torch.Tensor]], torch.Tensor | None])rstrrr)rr)rbool) __name__ __module__ __qualname____doc__rXrjrrrr __classcell__)rrs@r/rrsCN&*.2KOJN HL-1?C8 1&rzr)3 __future__rloggingr\ functoolsrtypingrrtorchpackaging.versionrrTr transformersr r r r rUtransformers.data.data_collatorr transformers.integrationsrrr1sentence_transformers.cross_encoder.data_collatorr*sentence_transformers.cross_encoder.lossesrr.sentence_transformers.cross_encoder.model_cardr1sentence_transformers.cross_encoder.training_argsr sentence_transformers.evaluationrrsentence_transformers.trainerrsentence_transformers.utilrrdatasetsrrr getLoggerrrFrrDrzr/rsq"  4QQ<83<V_X[SDS>>   8 $x4xrz