T^(hx9|dZddlmZddlmZmZmZddlZddlZddlm Z ddl m Z ddl m Z mZmZmZmZdd lmZd d lmZej,eZd ZeGd de ZGdde ZdZdZddZGdde j>Z Gdde j>Z!edeGddeZ"ddgZ#y)zPyTorch VitPose model.) dataclass)OptionalTupleUnionN)nn)PreTrainedModel) ModelOutputadd_start_docstrings%add_start_docstrings_to_model_forwardloggingreplace_return_docstrings) load_backbone) VitPoseConfigrceZdZUdZdZeejed<dZ eejed<dZ ee ejdfed<dZ ee ejdfed<y)VitPoseEstimatorOutputaQ Class for outputs of pose estimation models. Args: loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Loss is not supported at this moment. See https://github.com/ViTAE-Transformer/ViTPose/tree/main/mmpose/models/losses for further detail. heatmaps (`torch.FloatTensor` of shape `(batch_size, num_keypoints, height, width)`): Heatmaps as predicted by the model. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, + one for the output of each stage) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states (also called feature maps) of the model at the output of each stage. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, patch_size, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. Nlossheatmaps. hidden_states attentions) __name__ __module__ __qualname____doc__rrtorch FloatTensor__annotations__rrrrj/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/transformers/models/vitpose/modeling_vitpose.pyrr*sq()-D(5$$ %,,0Hhu(()0=AM8E%"3"3S"89:A:>Ju00#567>r rcxeZdZdZeZdZdZdZde e je je jfddfdZy) VitPosePreTrainedModelz An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. vit pixel_valuesTmodulereturnNct|tjtjfrtjj |j jjtjd|jjj|j j|j _|j%|jjjyyt|tj rJ|jjj|j jj#dyy)zInitialize the weightsg)meanstdNg?) isinstancerLinearConv2dinit trunc_normal_weightdatatorfloat32configinitializer_rangedtypebiaszero_ LayerNormfill_)selfr&s r! _init_weightsz$VitPosePreTrainedModel._init_weightsQs fryy"))4 5"$!6!6 ""%%emm43DKKDaDa"7"b$$% MM {{&   &&('  - KK   " " $ MM   $ $S ).r )rrrrr config_classbase_model_prefixmain_input_namesupports_gradient_checkpointingrrr,r-r9r<rr r!r#r#FsK !L$O&*# *E"))RYY *L$M *RV *r r#aI This model is a PyTorch [torch.nn.Module](https://pytorch.org/docs/stable/nn.html#torch.nn.Module) subclass. Use it as a regular PyTorch Module and refer to the PyTorch documentation for all matter related to general usage and behavior. Parameters: config ([`VitPoseConfig`]): Model configuration class with all the parameters of the model. Initializing with a config file does not load the weights associated with the model, only the configuration. Check out the [`~PreTrainedModel.from_pretrained`] method to load the model weights. a4 Args: pixel_values (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`): Pixel values. Pixel values can be obtained using [`VitPoseImageProcessor`]. See [`VitPoseImageProcessor.__call__`] for details. dataset_index (`torch.Tensor` of shape `(batch_size,)`): Index to use in the Mixture-of-Experts (MoE) blocks of the backbone. This corresponds to the dataset index used during training, e.g. For the single dataset index 0 refers to the corresponding dataset. For the multiple datasets index 0 refers to dataset A (e.g. MPII) and index 1 refers to dataset B (e.g. CrowdPose). flip_pairs (`torch.tensor`, *optional*): Whether to mirror pairs of keypoints (for example, left ear -- right ear). output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. c|dvr td|jdk7r td|j\}}}}d}|dk(rd}|dddddd f |dddddd f<|j|d |||}|j }|j D])\} } |dd| d f|dd| d f<|dd| d f|dd| d f<+|j||||f}|j d }|S) aFlip the flipped heatmaps back to the original form. Args: output_flipped (`torch.tensor` of shape `(batch_size, num_keypoints, height, width)`): The output heatmaps obtained from the flipped images. flip_pairs (`torch.Tensor` of shape `(num_keypoints, 2)`): Pairs of keypoints which are mirrored (for example, left ear -- right ear). target_type (`str`, *optional*, defaults to `"gaussian-heatmap"`): Target type to use. Can be gaussian-heatmap or combined-target. gaussian-heatmap: Classification target with gaussian distribution. combined-target: The combination of classification target (response map) and regression target (offset map). Paper ref: Huang et al. The Devil is in the Details: Delving into Unbiased Data Processing for Human Pose Estimation (CVPR 2020). Returns: torch.Tensor: heatmaps that flipped back to the original image )gaussian-heatmapcombined-targetz9target_type should be gaussian-heatmap or combined-targetzCoutput_flipped should be [batch_size, num_keypoints, height, width]rrCrN.) ValueErrorndimshapereshapeclonetolistflip) output_flipped flip_pairs target_type batch_size num_keypointsheightwidthchannelsoutput_flipped_backleftrights r! flip_backrXs8"AATUUa^__/=/C/C,J vuH''(6q!$Q$|(D'Dq!$Q$|$#++JHfeTN(..0"((*J e,:1eS=,IAtSL)-;AtSL-IAucM*J.55z=RXZ_6`a-2226 r c|eZdZdZdfd Zddej deej dej fdZxZ S) VitPoseSimpleDecoderz Simple decoding head consisting of a ReLU activation, 4x upsampling and a 3x3 convolution, turning the feature maps into heatmaps. r'ct|tj|_tj |j dd|_tj|jj|jddd|_ y)NbilinearF) scale_factormode align_cornersrr kernel_sizestridepadding) super__init__rReLU activationUpsampler] upsamplingr-backbone_config hidden_size num_labelsconvr;r4 __class__s r!rezVitPoseSimpleDecoder.__init__se '')++63F3FZglmII  " " . .0A0AqYZde  r hidden_staterNc|j|}|j|}|j|}| t||}|SN)rgrirmrXr;rprNrs r!forwardzVitPoseSimpleDecoder.forwardsC|4 |4 99\*  ! :6Hr )r'Nrr) rrrrrerTensorrrt __classcell__ros@r!rZrZs;   ELL hu||>T `e`l`l r rZcheZdZdZdeffd ZddejdeejfdZ xZ S)VitPoseClassicDecoderz Classic decoding head consisting of a 2 deconvolutional blocks, followed by a 1x1 convolution layer, turning the feature maps into heatmaps. r4ct|tj|jj ddddd|_tjd|_tj|_ tjdddddd|_ tjd|_ tj|_ tjd|jddd|_y) NrDrF)rarbrcr7rr`)rdrerConvTranspose2drjrkdeconv1 BatchNorm2d batchnorm1rfrelu1deconv2 batchnorm2relu2r-rlrmrns r!rezVitPoseClassicDecoder.__init__s ))  " " . .1VW^c  ..-WWY ))#s!UV]bc ..-WWY IIc6#4#4!AWXY r rprNc|j|}|j|}|j|}|j|}|j |}|j |}|j |}| t||}|Srr)r~rrrrrrmrXrss r!rtzVitPoseClassicDecoder.forwards{||L1 |4 zz,/ ||L1 |4 zz,/ 99\*  ! :6Hr rr) rrrrrrerrurrtrvrws@r!ryrys6 Z} ZELLhu||>Tr ryz5The VitPose model with a pose estimation head on top.c eZdZdeddffd Zeeeee  dde jde e jde e jd e e jd e e d e e d e e deeeffd ZxZS)VitPoseForPoseEstimationr4r'Nct||t||_t |jj ds t dt |jj ds t dt |jj ds t d|jr t|n t||_ |jy)Nrkz0The backbone should have a hidden_size attribute image_sizez0The backbone should have an image_size attribute patch_sizez/The backbone should have a patch_size attribute) rdrerbackbonehasattrr4rFuse_simple_decoderrZryhead post_initrns r!rez!VitPoseForPoseEstimation.__init__s  %f- t}}++];OP Pt}}++\:OP Pt}}++\:NO O4:4M4M(0ShioSp  r ) output_typer=r% dataset_indexrNlabelsoutput_attentionsoutput_hidden_states return_dictc|||n|jj}||n|jj}||n|jj}d}| t d|j j |||||} |r| jdn| dd} | jd} |jjjd|jjjdz} |jjjd|jjjdz} | jdddj| d| | j} |j| |}|s|r |f| ddz}n |f| ddz}||f|zS|St!||| j"| j$ S) aT Returns: Examples: ```python >>> from transformers import AutoImageProcessor, VitPoseForPoseEstimation >>> import torch >>> from PIL import Image >>> import requests >>> processor = AutoImageProcessor.from_pretrained("usyd-community/vitpose-base-simple") >>> model = VitPoseForPoseEstimation.from_pretrained("usyd-community/vitpose-base-simple") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> boxes = [[[412.8, 157.61, 53.05, 138.01], [384.43, 172.21, 15.12, 35.74]]] >>> inputs = processor(image, boxes=boxes, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**inputs) >>> heatmaps = outputs.heatmaps ```NzTraining is not yet supported)rrrrrErrr|)rN)rrrr)r4use_return_dictrrNotImplementedErrorrforward_with_filtered_kwargs feature_mapsrHrjrrpermuterI contiguousrrrr)r;r%rrNrrrrroutputssequence_outputrP patch_height patch_widthroutputs r!rtz VitPoseForPoseEstimation.forwardsH&1% K 'tnK d^K  u,, -K aEK r r)rB)$r dataclassesrtypingrrrrtorch.utils.checkpointrmodeling_utilsr utilsr r r r rutils.backbone_utilsrconfiguration_vitposer get_loggerrloggerrrr#VITPOSE_START_DOCSTRINGrrXModulerZryr__all__rr r!rs!)) -20   H %" ?[? ?6*_*4 2%P2996#BII#L;` 5` ` F $%? @r