T^(h'{ dZddlmZmZmZmZmZddlZddl m Z m Z m Z ddl mZmZddlmZmZmZmZmZmZmZmZmZmZddlmZmZmZmZm Z erddl!Z!er ejDe#Z$d e%d eee%e%ffd Z& dd ejNd eeee%e%fdeed ejNfdZ(Gdde Z)dgZ*y)z!Image processor class for SegGPT.)DictListOptionalTupleUnionN)BaseImageProcessor BatchFeature get_size_dict)resizeto_channel_dimension_format) IMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STDChannelDimension ImageInputPILImageResamplinginfer_channel_dimension_formatis_scaled_imagemake_list_of_imagesto_numpy_array valid_images) TensorTypeis_torch_availableis_vision_availableloggingrequires_backends num_labelsreturnct|dzdz}d|z}dg}t|D]F}||dzz}||dzz|z}||z}d||zz }d||zz } d||zz } |j|| | fH|S)NgUUUUUU?)rrr)intrangeappend) rbasemargin color_listlocation num_seq_r num_seq_g num_seq_bRGBs p/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/transformers/models/seggpt/image_processing_seggpt.py build_paletter22s ze$ % )D D[FJ*% %a' a'D0 tO )f$ $ )f$ $ )f$ $1a)$ % maskpalette data_formatc||ntj}||j\}}tjd||ftj }tj |}|D]}||}||k(jtj } tj| d} | tj|z} tj| dd} || jtj z }tj|ddjtj }ntj|ddd}t||S)Nr)dtype)axisrr#)N.)rFIRSTshapenpzerosuint8uniqueastype expand_dimsarraymoveaxiscliprepeatr ) r4r5r6heightwidthrgb_maskclasses_in_mask class_idx rgb_value class_maskclass_rgb_masks r1 mask_to_rgbrOFs"-!8+>N>T>TK 88Q.bhh?))D/( 8I *I)+33BHH=J *DY&/&;AU,r3rct|S)a_Build a palette to map the prompt mask from a single channel to a 3 channel RGB. Args: num_labels (`int`): Number of classes in the segmentation task (excluding the background). Returns: `List[Tuple[int, int]]`: Palette to map the prompt mask from a single channel to a 3 channel RGB. )r2)r`rs r1 get_palettez SegGptImageProcessor.get_palettesZ((r3imager5r6ct|||S)aConverts a segmentation map to RGB format. Args: image (`np.ndarray`): Segmentation map with dimensions (height, width) where pixel values represent the class index. palette (`List[Tuple[int, int]]`, *optional*, defaults to `None`): Palette to use to convert the mask to RGB format. If unset, the mask is duplicated across the channel dimension. data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. Returns: `np.ndarray`: The mask in RGB format. )r5r6)rO)r`rer5r6s r1rOz SegGptImageProcessor.mask_to_rgbs.5'{KKr3input_data_formatc t|}d|vsd|vrtd|j|d|df}t|f||||d|S)a Resize an image to `(size["height"], size["width"])`. Args: image (`np.ndarray`): Image to resize. size (`Dict[str, int]`): Dictionary in the format `{"height": int, "width": int}` specifying the size of the output image. resample (`PILImageResampling`, *optional*, defaults to `PILImageResampling.BICUBIC`): `PILImageResampling` filter to use when resizing the image e.g. `PILImageResampling.BICUBIC`. data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. Returns: `np.ndarray`: The resized image. rGrHzFThe `size` dictionary must contain the keys `height` and `width`. Got )rTrUr6rg)r ValueErrorkeysr )r`rerTrUr6rgra output_sizes r1r zSegGptImageProcessor.resizesyFT" 4 7$#6efjfofofqerst tH~tG}5   #/     r3imagesc z||n |j}||n |j}||n |j}| | n |j} ||n |j}||n |j }||n |j }| | n |j} ||n |j}t|}t|| rdnd}t|s td|r | td|r | td|r|| td|Dcgc] }t|}}|r#t|d rtj!d | | st#|d } | rT| |j%| nd}|Dcgc]$}|j'||t(j* &}}t(j*} |r"|Dcgc]}|j-|||| }}|r!|Dcgc]}|j/||| }}|r"|Dcgc]}|j1||| | }}|Dcgc]}t3|| | }}|Scc}wcc}wcc}wcc}wcc}wcc}w)a Preprocess an image or batch of images. Args: images (`ImageInput`): Image to _preprocess. Expects a single or batch of images with pixel values ranging from 0 to 255. If passing in images with pixel values between 0 and 1, set `do_rescale=False`. do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the image. size (`Dict[str, int]`, *optional*, defaults to `self.size`): Dictionary in the format `{"height": h, "width": w}` specifying the size of the output image after resizing. resample (`PILImageResampling` filter, *optional*, defaults to `self.resample`): `PILImageResampling` filter to use if resizing the image e.g. `PILImageResampling.BICUBIC`. Only has an effect if `do_resize` is set to `True`. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the image values between [0 - 1]. rescale_factor (`float`, *optional*, defaults to `self.rescale_factor`): Rescale factor to rescale the image by if `do_rescale` is set to `True`. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the image. image_mean (`float` or `List[float]`, *optional*, defaults to `self.image_mean`): Image mean to use if `do_normalize` is set to `True`. image_std (`float` or `List[float]`, *optional*, defaults to `self.image_std`): Image standard deviation to use if `do_normalize` is set to `True`. return_tensors (`str` or `TensorType`, *optional*): The type of tensors to return. Can be one of: - Unset: Return a list of `np.ndarray`. - `TensorType.TENSORFLOW` or `'tf'`: Return a batch of type `tf.Tensor`. - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `torch.Tensor`. - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`. - `TensorType.JAX` or `'jax'`: Return a batch of type `jax.numpy.ndarray`. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - Unset: Use the channel dimension format of the input image. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. do_convert_rgb (`bool`, *optional*, defaults to `self.do_convert_rgb`): Whether to convert the prompt mask to RGB format. If `num_labels` is specified, a palette will be built to map the prompt mask from a single channel to a 3 channel RGB. If unset, the prompt mask is duplicated across the channel dimension. Must be set to `False` if the prompt mask is already in RGB format. num_labels: (`int`, *optional*): Number of classes in the segmentation task (excluding the background). If specified, a palette will be built, assuming that class_idx 0 is the background, to map the prompt mask from a single class_idx channel to a 3 channel RGB. Not specifying this will result in the prompt mask either being passed through as is if it is already in RGB format or being duplicated across the channel dimension. Nr"r)expected_ndimszkInvalid image type. Must be of type PIL.Image.Image, numpy.ndarray, torch.Tensor, tf.Tensor or jax.ndarray.z,Size must be specified if do_resize is True.z7Rescale factor must be specified if do_rescale is True.z=Image mean and std must be specified if do_normalize is True.rzIt looks like you are trying to rescale already rescaled images. If the input images have pixel values between 0 and 1, set `do_rescale=False` to avoid rescaling them again.)rer5r6)rerTrUrg)rescalerg)remeanstdrg)input_channel_dim)rSrVrXr[rUrWrYrZrTr rrrirrlogger warning_oncerrdrOrr;r rescale normalizer )r`rlrSrTrUrVrWrXrYrZr6rgr[rra size_dictrer5s r1_preprocess_stepz%SegGptImageProcessor._preprocess_stepsL"+!6IDNN #-#9Zt '3'?|TEVEV +9+E4K^K^'38+9+E4K^K^#-#9Zt !*!6IDNN 'tTYY!$' %VAUVWF#:  KL L .0VW W Z/93D\] ]6<vay I  6@6Ld&&z2RVGqwgl  ugK[KaKa bF!1 6 6  $ %i(^o pF $ 5Rc dF $U^opF ou ej '{N_ `   U=     s$6H)H$H)H.$H3H8 prompt_images prompt_masksreturn_tensorsc ftd|||fDr tdi}|%|j|fd|||||| | | d||d |}||d<|%|j|fd|||||| | | d||d |}||d<|3|j|f||tj||| | | | | ||d |}||d<t || S) a Preprocess an image or batch of images. Args: images (`ImageInput`): Image to _preprocess. Expects a single or batch of images with pixel values ranging from 0 to 255. If passing in images with pixel values between 0 and 1, set `do_rescale=False`. prompt_images (`ImageInput`): Prompt image to _preprocess. Expects a single or batch of images with pixel values ranging from 0 to 255. If passing in images with pixel values between 0 and 1, set `do_rescale=False`. prompt_masks (`ImageInput`): Prompt mask from prompt image to _preprocess that specify prompt_masks value in the preprocessed output. Can either be in the format of segmentation maps (no channels) or RGB images. If in the format of RGB images, `do_convert_rgb` should be set to `False`. If in the format of segmentation maps, `num_labels` specifying `num_labels` is recommended to build a palette to map the prompt mask from a single channel to a 3 channel RGB. If `num_labels` is not specified, the prompt mask will be duplicated across the channel dimension. do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the image. size (`Dict[str, int]`, *optional*, defaults to `self.size`): Dictionary in the format `{"height": h, "width": w}` specifying the size of the output image after resizing. resample (`PILImageResampling` filter, *optional*, defaults to `self.resample`): `PILImageResampling` filter to use if resizing the image e.g. `PILImageResampling.BICUBIC`. Only has an effect if `do_resize` is set to `True`. Doesn't apply to prompt mask as it is resized using nearest. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the image values between [0 - 1]. rescale_factor (`float`, *optional*, defaults to `self.rescale_factor`): Rescale factor to rescale the image by if `do_rescale` is set to `True`. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the image. image_mean (`float` or `List[float]`, *optional*, defaults to `self.image_mean`): Image mean to use if `do_normalize` is set to `True`. image_std (`float` or `List[float]`, *optional*, defaults to `self.image_std`): Image standard deviation to use if `do_normalize` is set to `True`. do_convert_rgb (`bool`, *optional*, defaults to `self.do_convert_rgb`): Whether to convert the prompt mask to RGB format. If `num_labels` is specified, a palette will be built to map the prompt mask from a single channel to a 3 channel RGB. If unset, the prompt mask is duplicated across the channel dimension. Must be set to `False` if the prompt mask is already in RGB format. num_labels: (`int`, *optional*): Number of classes in the segmentation task (excluding the background). If specified, a palette will be built, assuming that class_idx 0 is the background, to map the prompt mask from a plain segmentation map with no channels to a 3 channel RGB. Not specifying this will result in the prompt mask either being passed through as is if it is already in RGB format (if `do_convert_rgb` is false) or being duplicated across the channel dimension. return_tensors (`str` or `TensorType`, *optional*): The type of tensors to return. Can be one of: - Unset: Return a list of `np.ndarray`. - `TensorType.TENSORFLOW` or `'tf'`: Return a batch of type `tf.Tensor`. - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `torch.Tensor`. - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`. - `TensorType.JAX` or `'jax'`: Return a batch of type `jax.numpy.ndarray`. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - Unset: Use the channel dimension format of the input image. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. c3$K|]}|du yw)Nr]).0vs r1 z2SegGptImageProcessor.preprocess..sHQqDyHszFAt least one of images, prompt_images, prompt_masks must be specified.F) is_maskrSrTrUrVrWrXrYrZr[r6rgrRprompt_pixel_values) rSrTrUrVrWrXrYrZr[rr6rgrz)data tensor_type)allrirxrNEARESTr )r`rlryrzrSrTrUrVrWrXrYrZr[rr{r6rgrars r1 preprocesszSegGptImageProcessor.preprocesssMh H6=,"GH Hef f  *T**#!%-)%#$'"3F"$*D  $1D11#!%-)%#$'"3M"+8D& '  #0400#+33%-)%#-%'"3L"$0D >BBr3 target_sizesct|dg|j}|dddd|jddzdddf}tj|j j |j}tj|jj |j}|jdddd|z|z}|jdddd}tj|dzdd}g}d}||j|nd} | ftj| j |jtj}|j\} } } } |jdd|dz| }t|D]\} } |Atjj j#| j%d|| d d} |z| j\}}}| jdddj||d|}||z }tj&|d}tj(|d }|j+d }n | j-d j/}|j1||S) a Converts the output of [`SegGptImageSegmentationOutput`] into segmentation maps. Only supports PyTorch. Args: outputs ([`SegGptImageSegmentationOutput`]): Raw outputs of the model. target_sizes (`List[Tuple[int, int]]`, *optional*): List of length (batch_size), where each list item (`Tuple[int, int]`) corresponds to the requested final size (height, width) of each prediction. If left to None, predictions will not be resized. num_labels (`int`, *optional*): Number of classes in the segmentation task (excluding the background). If specified, a palette will be built, assuming that class_idx 0 is the background, to map prediction masks from RGB values to class indices. This value should be the same used when preprocessing inputs. Returns: semantic_segmentation: `List[torch.Tensor]` of length `batch_size`, where each item is a semantic segmentation map of shape (height, width) corresponding to the target_sizes entry (if `target_sizes` is specified). Each entry of each `torch.Tensor` correspond to a semantic class id. torchNr"rrr r#)devicer8nearest)rTmoder9)dim)r pred_masksr<rtensorrZtorrYpermuterErdfloatview enumeratenn functional interpolate unsqueezepowsumargminrpr$r&)r`outputsrrmasksrqrpsemantic_segmentationpalette_tensorr5_ num_channelsidxr4channelsrGrHdistpreds r1"post_process_semantic_segmentationz7SegGptImageProcessor.post_process_semantic_segmentationsQ, $ *""aEKKNa/1145ll4>>*--ell;||DOO,// = aAq)C/$6 aAq) 53;3/ "2<2H$"":.d  "\\'255U\\QVQ\Q\5]N$)KK !A|Q+00AzA~|TN"5) /IC'xx**66NN1%%c*"7  %*.**'&%||Aq!,11&%HMn,yyq)yy2.{{r{*yyQy'++- ! ( ( .) /,%$r3NN)__name__ __module__ __qualname____doc__model_input_namesrBICUBICboolrrstrr$rrrr_rrdr=ndarrayrrOr r;rrxrrr __classcell__)rbs@r1rQrQbsB(()-'9'A'A,3!:>9=#--tCH~&-% -  - c5j) --U5$u+#567-E%e"456-- -4 )c )d5c?.C )48>B LzzL$uS#X/0LeC)9$9:; L  L<(:'A'A>BDH . zz. 38n. % . eC)9$9:; . $E#/?*?$@A .  . f%)#'+%)*.'+:>9=4D4J4JDH)-$(OOD>O38n O % O TN O!OtnOU5$u+#567OE%e"456O3 001O$E#/?*?$@AO!OSMOf(,.2-1$(#'+%)*.'+:>9=)-$(;?4D4J4JDH#UC$UC +UCz* UC D> UC 38n UC%UCTNUC!UCtnUCU5$u+#567UCE%e"456UC!UCSMUC!sJ!78UC 3 001!UC"$E#/?*?$@A#UCpjnI%%-d5c?.C%DI%YabeYfI%r3rQr)+rtypingrrrrrnumpyr=image_processing_utilsr r r image_transformsr r image_utilsrrrrrrrrrrutilsrrrrrr get_loggerrrsr$r2rrOrQ__all__r]r3r1rs(55UUC   ed   H % cd5c?&;*rv> **>'U38_(=>>T\]mTn>ZZ>8E%-E%P " "r3