S^(hP4ddlZddlZddlmZddlmZddlmZmZm Z ddl Z ddl Z ddlmZmZddlmZddlmZmZdd lmZdd lmZej4eZGd d ej:j<ZGd dej:j<Z y)N)Path)sleep)CallableOptionalUnion) Repository create_repo)parse)IntervalStrategyPreTrainedTokenizerBase)TrainingSummary)kerasceZdZdZ ddedeejje jeje e fdeeedeeedeededed ee ffd Zedd Zd Zdd ZxZS)KerasMetricCallbacka[ Callback to compute metrics at the end of every epoch. Unlike normal Keras metrics, these do not need to be compilable by TF. It is particularly useful for common NLP metrics like BLEU and ROUGE that require string operations or generation loops that cannot be compiled. Predictions (or generations) will be computed on the `eval_dataset` before being passed to the `metric_fn` in `np.ndarray` format. The `metric_fn` should compute metrics and return a dict mapping metric names to metric values. We provide an example of a suitable metric_fn that computes ROUGE scores for a summarization model below. Note that this example skips some post-processing for readability and simplicity, and should probably not be used as-is! ```py from datasets import load_metric rouge_metric = load_metric("rouge") def rouge_fn(predictions, labels): decoded_predictions = tokenizer.batch_decode(predictions, skip_special_tokens=True) decoded_labels = tokenizer.batch_decode(labels, skip_special_tokens=True) result = rouge_metric.compute(predictions=decoded_predictions, references=decoded_labels) return {key: value.mid.fmeasure * 100 for key, value in result.items()} ``` The above function will return a dict containing values which will be logged like any other Keras metric: ``` {'rouge1': 37.4199, 'rouge2': 13.9768, 'rougeL': 34.361, 'rougeLsum': 35.0781 ``` Args: metric_fn (`Callable`): Metric function provided by the user. It will be called with two arguments - `predictions` and `labels`. These contain the model's outputs and matching labels from the dataset. It should return a dict mapping metric names to numerical values. eval_dataset (`tf.data.Dataset` or `dict` or `tuple` or `np.ndarray` or `tf.Tensor`): Validation data to be used to generate predictions for the `metric_fn`. output_cols (`List[str], *optional*): A list of columns to be retained from the model output as the predictions. Defaults to all. label_cols ('`List[str]`, *optional*'): A list of columns to be retained from the input dataset as the labels. Will be autodetected if this is not supplied. batch_size (`int`, *optional*): Batch size. Only used when the data is not a pre-batched `tf.data.Dataset`. predict_with_generate (`bool`, *optional*, defaults to `False`): Whether we should use `model.generate()` to get outputs for the model. use_xla_generation (`bool`, *optional*, defaults to `False`): If we're generating, whether to compile model generation with XLA. This can massively increase the speed of generation (up to 100X speedup) but will require a new XLA compilation for each input shape. When using XLA generation, it's a good idea to pad your inputs to the same size, or to use the `pad_to_multiple_of` argument in your `tokenizer` or `DataCollator`, which will reduce the number of unique input shapes and save a lot of compilation time. This option has no effect is `predict_with_generate` is `False`. generate_kwargs (`dict`, *optional*): Keyword arguments to pass to `model.generate()` when generating. Has no effect if `predict_with_generate` is `False`. metric_fn eval_dataset output_cols label_cols batch_sizepredict_with_generateuse_xla_generationgenerate_kwargsc t |||_||_t |t j jsG| tdt j jj|j|d}||_ ||_ ||_ t |jtr(t!|jdk(r|j\} } n|j} d} |)|D]} | | vstd| d||_d|_ns| d|_d|_nbd| vr%dg|_d|_t'j(d n9d | vr*d | vr&d d g|_d|_t'j(d n td t+t j,t+dkrt'j(d||_|in||_d|_y)NzwWhen passing data to KerasMetricCallback that is not a pre-batched tf.data.Dataset the batch_size argument must be set.F)drop_remainderzLabel z? is in label_cols but could not be found in the dataset inputs!TlabelszTNo label_cols specified for KerasMetricCallback, assuming you want the 'labels' key.start_positions end_positionsznNo label_cols specified for KerasMetricCallback, assuming you want the start_positions and end_positions keys.zMCould not autodetect label_cols for KerasMetricCallback, please specify them!z2.7zHTF versions less than 2.7 may encounter issues with KerasMetricCallback!)super__init__rr isinstancetfdataDataset ValueErrorfrom_tensor_slicesbatchrrr element_spectuplelenruse_keras_labelloggingwarningr __version__rrgeneration_function) selfrrrrrrrr input_spec label_speclabel __class__s Z/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/transformers/keras_callbacks.pyr!zKerasMetricCallback.__init__Ns "$,8! ; 77??==lKQQR\mrQsL(%:"& l// 7C @Y@Y%> "J %22JJ  !# v *$veW4s%tuu v)DO#(D  ##DO#'D  #'jDO#(D OOr s * ,J1N0/BDO#(D OO:  lm m  5< / OOf g"4%4%z;KerasMetricCallback._concatenate_batches..s.&bQVu{{1~9I9I!9L'L&bs,/)axisr) fill_valuer:) ndimallnp concatenatemaxr:sum full_likelistr+)r< padding_indexr(max_len num_samplesoutputis` r6_concatenate_batchesz(KerasMetricCallback._concatenate_batchess  1:??a 3&bZa&b#b>>'2 27;%u{{1~;<w?e5;;q>?@  AJ=g8NQUV]^_V`VfVfghgiVjQk8k   E;@F1q3u:~%'7Q'77 8 UOA  <?s D.Dct|dtrki}|djD]'}|j|Dcgc]}|| c}||<)t |dk(rt |j d}|St|dt st|dtrDg}t|D]"}|j|j|$t |dk(r|d}|St|dtjr|j|}|St|dtjr/|j|Dcgc]}|jc}}|Stdt!|ddcc}wcc}w)Nrr zCouldn't handle batch of type !)r"dictkeysrMr+rGvaluesr*zipappendrBndarrayr#Tensornumpy TypeErrortype)r1inputsoutputskeyr( input_listtensors r6"_postprocess_predictions_or_labelsz6KerasMetricCallback._postprocess_predictions_or_labelss} fQi &Gay~~' [#88RX9Y%*9YZ  [7|q w~~/03q 4 (Jvay%,HG!6l F t88DE F7|q !!* q 2:: .//7G  q 299 -//f0UF0UVG.generation_functions+.4::..vmnmX\XlXlmmr7T) jit_compilerg)rglosszConfused by labels of type zSmetric_fn should return a dict mapping metric names to values but instead returned ) hasattrrhgetattrrarrcrdrr0r#functionrr"r*rPgetrirpredict_on_batchritemsrTr,rrWrGrVrXrYr_rupdate)r1epochlogs ignore_keysrdr0prediction_list label_listr(rgeneration_inputsrg predictionsr\valarray all_preds all_labels metric_outputs` r6 on_epoch_endz KerasMetricCallback.on_epoch_endst 4::x (!$**"3"35RTVWKK  % %tzz9-'$**:L:LN_2`"&**"4"4"D"D")$**6G"U&&4+C+C+Kn,.;;7JX\+]( &&) &E%' % v))eT*(-o(>%%*YY/?%FN(-%%)N**"&":":;L]k":"lK"5$**"5"5)#:H#LPL`L`#K#jj99%@ k40#'{"3K''3HLHXHX&YsK,<'<&Y &Y6A5F5F5H')1cCWbflemWmLmCH' '  " "; /''=A__Mc#uSz//11MMFD)?E||~Ne#u{{},NNFD)Z-F5;? -.efsetu  M"='Z' NN$(7;&*#' "/#t)$"/S"223"/SM "/ 34 "/ sm "/C="/"/H# 6Sr7r)!r-rpathlibrtimertypingrrrrWrB tensorflowr#huggingface_hubrr packaging.versionr r r modelcardrmodeling_tf_utilsr getLoggerrlogger callbacksCallbackrrrr7r6rss ,,3#7&$   8 $u#%//22u#pQS00QSr7