l^(hbddlmZddlmZmZddlmZddlmZGddejZ y)) annotations)Tensornn) CrossEncoder)fullnamec`eZdZejdf dfd ZddZdZxZS)BinaryCrossEntropyLossNc t|||_||_||_t j dd|i||_t|jts8t|jjdt|jd|jjdk7r9t|jjd|jjdy) a! Computes the Binary Cross Entropy Loss for a CrossEncoder model. This loss is used to train a model to predict a high logit for positive pairs and a low logit for negative pairs. The model should be initialized with ``num_labels = 1`` (a.k.a. the default) to predict one class. It has been used to train many of the strong `CrossEncoder MS MARCO Reranker models `_. Args: model (:class:`~sentence_transformers.cross_encoder.CrossEncoder`): A CrossEncoder model to be trained. activation_fn (:class:`~torch.nn.Module`): Activation function applied to the logits before computing the loss. Defaults to :class:`~torch.nn.Identity`. pos_weight (Tensor, optional): A weight of positive examples. Must be a :class:`torch.Tensor` like ``torch.tensor(4)`` for a weight of 4. Defaults to None. **kwargs: Additional keyword arguments passed to the underlying :class:`torch.nn.BCEWithLogitsLoss`. References: - :class:`torch.nn.BCEWithLogitsLoss` - `Cross Encoder > Training Examples > Semantic Textual Similarity <../../../examples/cross_encoder/training/sts/README.html>`_ - `Cross Encoder > Training Examples > Quora Duplicate Questions <../../../examples/cross_encoder/training/quora_duplicate_questions/README.html>`_ - `Cross Encoder > Training Examples > MS MARCO <../../../examples/cross_encoder/training/ms_marco/README.html>`_ - `Cross Encoder > Training Examples > Rerankers <../../../examples/cross_encoder/training/rerankers/README.html>`_ Requirements: 1. Your model must be initialized with `num_labels = 1` (a.k.a. the default) to predict one class. Inputs: +-------------------------------------------------+----------------------------------------+-------------------------------+ | Texts | Labels | Number of Model Output Labels | +=================================================+========================================+===============================+ | (anchor, positive/negative) pairs | 1 if positive, 0 if negative | 1 | +-------------------------------------------------+----------------------------------------+-------------------------------+ | (sentence_A, sentence_B) pairs | float similarity score between 0 and 1 | 1 | +-------------------------------------------------+----------------------------------------+-------------------------------+ Recommendations: - Use :class:`~sentence_transformers.util.mine_hard_negatives` with ``output_format="labeled-pair"`` to convert question-answer pairs to the ``(anchor, positive/negative) pairs`` format with labels as 1 or 0, using hard negatives. Example: :: from sentence_transformers.cross_encoder import CrossEncoder, CrossEncoderTrainer, losses from datasets import Dataset model = CrossEncoder("microsoft/mpnet-base") train_dataset = Dataset.from_dict({ "query": ["What are pandas?", "What are pandas?"], "response": ["Pandas are a kind of bear.", "Pandas are a kind of fish."], "label": [1, 0], }) loss = losses.BinaryCrossEntropyLoss(model) trainer = CrossEncoderTrainer( model=model, train_dataset=train_dataset, loss=loss, ) trainer.train() pos_weightz? expects a model of type CrossEncoder, but got a model of type .z; expects a model with 1 output label, but got a model with z output labels.N)super__init__model activation_fnr rBCEWithLogitsLossbce_with_logits_loss isinstancer ValueError __class____name__type num_labels)selfrrr kwargsrs /var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/sentence_transformers/cross_encoder/losses/BinaryCrossEntropyLoss.pyrzBinaryCrossEntropyLoss.__init__ sB  *$$&$8$8$YJ$YRX$Y!$**l3>>**+,++/ +;*>**+,((, (=(='>oO  &ct|dk7rtdt|dtt|d|d}|jj |ddd}|j |jj|jd i|djd }|j|}|j||j}|S) Nz\BinaryCrossEntropyLoss expects a dataset with two non-label columns, but got a dataset with z columns.rr Tpt)padding truncationreturn_tensorsr) lenrlistzipr tokenizertodeviceviewrrfloat)rinputslabelspairstokenslogitslosss rforwardzBinaryCrossEntropyLoss.forward]s v;! norsyozn{|EF SF1I./%%  &   $**##$%f%a(--b1##F+((@ rct|j|j|jdS|jjdS)N)rr )rrr item)rs rget_config_dictz&BinaryCrossEntropyLoss.get_config_dictpsD%d&8&89-1__-D$//  JN//J^J^J`  r)rrrz nn.Moduler z Tensor | NonereturnNone)r.zlist[list[str]]r/rr8r) r __module__ __qualname__rIdentityrr4r7 __classcell__)rs@rr r sN$/2;;=$( QQ!Q" Q  Qf& rr N) __future__rtorchrr0sentence_transformers.cross_encoder.CrossEncoderrsentence_transformers.utilrModuler rrrrCs#"I/k RYYk r