ehdZddlmZddlmZmZddlZddlZddl m Z m Z ddl m Z ddlmZddlmZdd lmZdd lmZdd lmZmZdd lmZdd lmZddlmZmZddl m!cm"Z#ddl$m%Z%ddl&m'Z'ddl(m)Z)ddl*m+Z+m,Z,m-Z-ddl.m/Z/ddl0m1Z1erddl2m3Z3m4Z4ddl5m6Z6m7Z7m8Z8m9Z9ddl:m;Z;GddZd!dZ?d"dZ@y)#z] Provide user facing operators for doing the split part of the split-apply-combine paradigm. ) annotations) TYPE_CHECKINGfinalN)using_copy_on_writewarn_copy_on_write)lib)OutOfBoundsDatetime)InvalidIndexError)cache_readonly)find_stack_level) is_list_like is_scalar)CategoricalDtype) algorithms) CategoricalExtensionArray) DataFrame)ops)recode_for_groupby)CategoricalIndexIndex MultiIndex)Series) pprint_thing)HashableIterator) ArrayLikeAxisNDFrameTnpt)NDFramec^eZdZUdZded<ded<ded<ded<dZd ed <fd Zd d d ejd df ddZ d ddZ dd d ddZ e e ddZe e dZe e dZe e dZe e dZe ddZxZS) Groupera A Grouper allows the user to specify a groupby instruction for an object. This specification will select a column via the key parameter, or if the level and/or axis parameters are given, a level of the index of the target object. If `axis` and/or `level` are passed as keywords to both `Grouper` and `groupby`, the values passed to `Grouper` take precedence. Parameters ---------- key : str, defaults to None Groupby key, which selects the grouping column of the target. level : name/number, defaults to None The level for the target index. freq : str / frequency object, defaults to None This will groupby the specified frequency if the target selection (via key or level) is a datetime-like object. For full specification of available frequencies, please see `here `_. axis : str, int, defaults to 0 Number/name of the axis. sort : bool, default to False Whether to sort the resulting labels. closed : {'left' or 'right'} Closed end of interval. Only when `freq` parameter is passed. label : {'left' or 'right'} Interval boundary to use for labeling. Only when `freq` parameter is passed. convention : {'start', 'end', 'e', 's'} If grouper is PeriodIndex and `freq` parameter is passed. origin : Timestamp or str, default 'start_day' The timestamp on which to adjust the grouping. The timezone of origin must match the timezone of the index. If string, must be one of the following: - 'epoch': `origin` is 1970-01-01 - 'start': `origin` is the first value of the timeseries - 'start_day': `origin` is the first day at midnight of the timeseries - 'end': `origin` is the last value of the timeseries - 'end_day': `origin` is the ceiling midnight of the last day .. versionadded:: 1.3.0 offset : Timedelta or str, default is None An offset timedelta added to the origin. dropna : bool, default True If True, and if group keys contain NA values, NA values together with row/column will be dropped. If False, NA values will also be treated as the key in groups. Returns ------- Grouper or pandas.api.typing.TimeGrouper A TimeGrouper is returned if ``freq`` is not ``None``. Otherwise, a Grouper is returned. Examples -------- ``df.groupby(pd.Grouper(key="Animal"))`` is equivalent to ``df.groupby('Animal')`` >>> df = pd.DataFrame( ... { ... "Animal": ["Falcon", "Parrot", "Falcon", "Falcon", "Parrot"], ... "Speed": [100, 5, 200, 300, 15], ... } ... ) >>> df Animal Speed 0 Falcon 100 1 Parrot 5 2 Falcon 200 3 Falcon 300 4 Parrot 15 >>> df.groupby(pd.Grouper(key="Animal")).mean() Speed Animal Falcon 200.0 Parrot 10.0 Specify a resample operation on the column 'Publish date' >>> df = pd.DataFrame( ... { ... "Publish date": [ ... pd.Timestamp("2000-01-02"), ... pd.Timestamp("2000-01-02"), ... pd.Timestamp("2000-01-09"), ... pd.Timestamp("2000-01-16") ... ], ... "ID": [0, 1, 2, 3], ... "Price": [10, 20, 30, 40] ... } ... ) >>> df Publish date ID Price 0 2000-01-02 0 10 1 2000-01-02 1 20 2 2000-01-09 2 30 3 2000-01-16 3 40 >>> df.groupby(pd.Grouper(key="Publish date", freq="1W")).mean() ID Price Publish date 2000-01-02 0.5 15.0 2000-01-09 2.0 30.0 2000-01-16 3.0 40.0 If you want to adjust the start of the bins based on a fixed timestamp: >>> start, end = '2000-10-01 23:30:00', '2000-10-02 00:30:00' >>> rng = pd.date_range(start, end, freq='7min') >>> ts = pd.Series(np.arange(len(rng)) * 3, index=rng) >>> ts 2000-10-01 23:30:00 0 2000-10-01 23:37:00 3 2000-10-01 23:44:00 6 2000-10-01 23:51:00 9 2000-10-01 23:58:00 12 2000-10-02 00:05:00 15 2000-10-02 00:12:00 18 2000-10-02 00:19:00 21 2000-10-02 00:26:00 24 Freq: 7min, dtype: int64 >>> ts.groupby(pd.Grouper(freq='17min')).sum() 2000-10-01 23:14:00 0 2000-10-01 23:31:00 9 2000-10-01 23:48:00 21 2000-10-02 00:05:00 54 2000-10-02 00:22:00 24 Freq: 17min, dtype: int64 >>> ts.groupby(pd.Grouper(freq='17min', origin='epoch')).sum() 2000-10-01 23:18:00 0 2000-10-01 23:35:00 18 2000-10-01 23:52:00 27 2000-10-02 00:09:00 39 2000-10-02 00:26:00 24 Freq: 17min, dtype: int64 >>> ts.groupby(pd.Grouper(freq='17min', origin='2000-01-01')).sum() 2000-10-01 23:24:00 3 2000-10-01 23:41:00 15 2000-10-01 23:58:00 45 2000-10-02 00:15:00 45 Freq: 17min, dtype: int64 If you want to adjust the start of the bins with an `offset` Timedelta, the two following lines are equivalent: >>> ts.groupby(pd.Grouper(freq='17min', origin='start')).sum() 2000-10-01 23:30:00 9 2000-10-01 23:47:00 21 2000-10-02 00:04:00 54 2000-10-02 00:21:00 24 Freq: 17min, dtype: int64 >>> ts.groupby(pd.Grouper(freq='17min', offset='23h30min')).sum() 2000-10-01 23:30:00 9 2000-10-01 23:47:00 21 2000-10-02 00:04:00 54 2000-10-02 00:21:00 24 Freq: 17min, dtype: int64 To replace the use of the deprecated `base` argument, you can now use `offset`, in this example it is equivalent to have `base=2`: >>> ts.groupby(pd.Grouper(freq='17min', offset='2min')).sum() 2000-10-01 23:16:00 0 2000-10-01 23:33:00 9 2000-10-01 23:50:00 36 2000-10-02 00:07:00 39 2000-10-02 00:24:00 24 Freq: 17min, dtype: int64 boolsortdropna Index | None _gpr_index_grouper)keylevelfreqaxisr%r&ztuple[str, ...] _attributescT|jdddlm}|}t||S)Nr,r) TimeGrouper)getpandas.core.resampler0super__new__)clsargskwargsr0 __class__s Y/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/pandas/core/groupby/grouper.pyr4zGrouper.__new__s) ::f  ) 8Cws##NFTcvt|tur9|tjur%t j dt tnd}|tjurd}||_||_ ||_ ||_ ||_ ||_ d|_d|_d|_d|_d|_d|_d|_y)Nz~Grouper axis keyword is deprecated and will be removed in a future version. To group on axis=1, use obj.T.groupby(...) instead stacklevelr)typer#r no_defaultwarningswarn FutureWarningr r*r+r,r-r%r&_grouper_deprecated_indexer_deprecated_obj_deprecatedr(binnerr)_indexer)selfr*r+r,r-r%r&s r9__init__zGrouper.__init__s : 3>>) "/1  3>> !D     #' @D #  59 r:c |j|\}}}t||jg|j|j|j ||j \}}}||_||fS)z Parameters ---------- obj : Series or DataFrame validate : bool, default True if True, validate the grouper Returns ------- a tuple of grouper, obj (possibly sorted) )r-r+r%validater&) _set_grouper get_grouperr*r-r+r%r&rC)rHobjrK_groupers r9 _get_grouperzGrouper._get_grouper,si%%c* Q%  XXJ**;; C$+ |r:) gpr_indexc|J|j|j td|j||_|j|_|j|j}t |dd|k(rt|tr|jJ|j Q|j j}|jj|}|j|j}n|jj|j}n||jvrtd|dt|||}n|j|j }|jv|j}t|t"r;|j%|}t|j'||j(|}n|d|j*fvrtd|d d} |j,s|r^|j.sR|j0jd d x} |_|j| }|j| |j }||_||_||| fS)a given an object and the specifications, setup the internal grouper for this particular specification Parameters ---------- obj : Series or DataFrame sort : bool, default False whether the resulting grouper should be sorted gpr_index : Index or None, default None Returns ------- NDFrame Index np.ndarray[np.intp] | None Nz2The Grouper cannot specify both a key and a level!namezThe grouper name z is not foundrTrz The level z is not valid mergesortfirst)kind na_positionr-)r*r+ ValueErrorr)rDrGgetattr isinstancerargsorttakeindex _info_axisKeyErrorr _get_axisr-r_get_level_number_get_level_valuesnamesrTr%is_monotonic_increasingarrayrEr() rHrNr%rRr*reverse_indexer unsorted_axaxr+indexers r9rLzGrouper._set_grouperKs*( 88 DJJ$:QR R == %DM 44DM 88 ((Cy&$/36:c6;R }}000==,&*mm&;&;&=O"&--"4"4_"EK$))#))4B++CII6Bcnn,"%6se=#IJJ3s8#.tyy)Bzz% b*-007Er33E:%QBQL0(:eWM)JKK04 IIr'A'A241A1A g2B2 Gd.!B((7(3C #Br:ctjt|jdtt |j }| td|S)NzS.ax is deprecated and will be removed in a future version. Use Resampler.ax insteadr<z1_set_grouper must be called before ax is accessed)r@rAr>__name__rBr r(r[)rHr`s r9rkz Grouper.axsT  Dz""#$7 7 ')   =PQ Q r:ctjt|jdtt |j S)Nz^.indexer is deprecated and will be removed in a future version. Use Resampler.indexer instead.r<)r@rAr>rnrBr rDrHs r9rlzGrouper.indexersC  Dz""#$B B ')  '''r:ctjt|jdtt |j S)NzX.obj is deprecated and will be removed in a future version. Use GroupBy.indexer instead.r<)r@rAr>rnrBr rErps r9rNz Grouper.objsC  Dz""#$@ @ ')  ###r:ctjt|jdtt |j S)Nz\.grouper is deprecated and will be removed in a future version. Use GroupBy.grouper instead.r<)r@rAr>rnrBr rCrps r9rPzGrouper.groupersC  Dz""#$@ @ ')  '''r:ctjt|jdtt |j jS)NzZ.groups is deprecated and will be removed in a future version. Use GroupBy.groups instead.r<)r@rAr>rnrBr rCgroupsrps r9rtzGrouper.groupssG  Dz""#$? ? ')  ''...r:cfdjD}dj|}tj}|d|dS)Nc 3jK|]*}t||dtt|,yw)N=)r\repr).0 attr_namerHs r9 z#Grouper.__repr__..s> tY'3k4i 89: ; s03z, ())r.joinr>rn)rH attrs_listattrscls_names` r9__repr__zGrouper.__repr__sJ !--  *%:&&1UG1%%r:)r-zAxis | lib.NoDefaultr%r$r&r$returnNone)T)rNrrKr$rz tuple[ops.BaseGrouper, NDFrameT])F)rNrr%r$rRr'rz3tuple[NDFrameT, Index, npt.NDArray[np.intp] | None]rrrstr)rn __module__ __qualname____doc____annotations__r.r4rr?rIrQrLrpropertyrkrlrNrPrtr __classcell__)r8s@r9r#r#Bssrh J L#UKU$  %(^^%: # %:  %:%: %:P/3'+ )@+0R NRR R #'R ?KR <R h       ( (   $  $  ( (  / / & &r:r#ceZdZUdZdZded<ded<ded<d ed < d dd Zdd Zdd Ze d dZ e d!dZ e d"dZ e d#dZe d$dZe d%dZe d&dZe d&dZe d'dZe d'dZe d'dZe d'dZe d(dZe d)dZy)*Groupingah Holds the grouping information for a single key Parameters ---------- index : Index grouper : obj : DataFrame or Series name : Label level : observed : bool, default False If we are a Categorical, use the observed values in_axis : if the Grouping is a column in self.obj and hence among Groupby.exclusions list dropna : bool, default True Whether to drop NA groups. uniques : Array-like, optional When specified, will be used for unique values. Enables including empty groups in the result for a BinGrouper. Must not contain duplicates. Attributes ------- indices : dict Mapping of {group -> index_list} codes : ndarray Group codes group_index : Index or None unique groups groups : dict Mapping of {group -> label_list} Nz$npt.NDArray[np.signedinteger] | None_codeszCategorical | None _all_grouperr' _orig_catsr_indexc ||_||_t||} d|_d|_||_||_||_||_||_ ||_ | |_ |j} | ?t|tr|j| } n|} | | } n^| } | j!| } nIt| t"r|jJ| j%|jd\}}||_t|t&j(r|} n|j*dj,}t/||j0j2} nt| t4t.t6t8j:fst=| dddk7r#t?tA| }tCd|d|j!| } tE| d rtG| tG|k(stI| }d |}tK|t| t8j:r9| jLjNd vrbt5| jQ} | |_ytt=| d dtRr&| jT|_tW| ||\} |_| |_y) NFrKrrUndim Grouper for '' not 1-dimensional__len__z9Grouper result violates len(labels) == len(data) result: mMdtype),r+ _orig_grouper_convert_grouperrrr_sortrN _observedin_axis_dropna_uniques_ilevelr]rget_level_valuesmapr#rQr BinGrouper groupingsgrouping_vectorr result_indexrTrrnpndarrayr\rr>r[hasattrlenrAssertionErrorrrXto_numpyr categoriesr)rHr`rPrNr+r%observedrr&uniquesrilevel index_levelmapper newgroupernewobjngtgrpererrmsgs r9rIzGrouping.__init__sf $*5':   !     %,#44V< # &"-("-//&"9 1 88' ''!0!=!=dhhQV!=!W JDH*cnn5#-  ))!,<<"'1H1H1M1M"N fe^RZZH 2a7_-. =3F!GHH#ii8O3(CJ6$_5**/2%V,, orzz 2$$))T1 #)"9"B"B"D /$?AQ R-88DO1Cx2 .OT. /r:c"d|jdS)Nz Grouping(r}rUrps r9rzGrouping.__repr__xs499+Q''r:c,t|jSN)iterindicesrps r9__iter__zGrouping.__iter__{sDLL!!r:cPt|jdd}t|tS)Nr)r\rr]r)rHrs r9_passed_categoricalzGrouping._passed_categorical~s$,,gt<%!122r:c|j}||jj|St|jt t fr|jjSt|jtjr |jjjSt|jt r|jjSyr) rrrfr]rrrrTrr BaseGrouperr)rHrs r9rTz Grouping.names  ;;$$V, , d((5&/ :%%** * ,,coo >''4499 9 ,,e 4'',, ,r:c|j}|yt|tsD|j}||jvrt d|d|jj |S|S)zS If necessary, converted index level name to index level position. NzLevel z not in index)r+r]intrrfrr`)rHr+r`s r9rzGrouping._ilevelsb   =%%KKEEKK'$veWM%BCC;;$$U+ + r:c,t|jSr)r _group_indexrps r9ngroupszGrouping.ngroupss4$$%%r:ct|jtjr|jjSt |j}|j Sr)r]rrrrr_reverse_indexer)rHvaluess r9rzGrouping.indicessI d**COO <''// /T112&&((r:c |jdS)Nr)_codes_and_uniquesrps r9codeszGrouping.codess&&q))r:c|j|jjS|jr|jjS|j dS)v Analogous to result_index, but holding an ArrayLike to ensure we can retain ExtensionDtypes. r)r _result_index_valuesrrrrps r9_group_arraylikezGrouping._group_arraylikesN    (%%-- -  % %$$,, ,&&q))r:cbtjdtt|jS)rzOgroup_arraylike is deprecated and will be removed in a future version of pandascategoryr=)r@rArBr rrps r9group_arraylikezGrouping.group_arraylikes,   "')  $$$r:c|j;|j}t|tsJ|j}|j |S|jSr)rrr]rrset_categories)rH group_idxcatss r9rzGrouping._result_indexsR    ())Ii)9: ::??D++D1 1   r:cbtjdtt|jS)NzLresult_index is deprecated and will be removed in a future version of pandasr)r@rArBr rrps r9rzGrouping.result_indexs*  "')  !!!r:c|j\}}|js7|jr*t|tsJ|j r^|t |k(jrBt jtj|jdg|jd}nt |dkDr|j}|jdkj}|j|dkretj |jd|}tj"|j|d}t j||jd}t%j&||j(S)NFrrrU)rrrr]rrrany from_codesrappendrrrargmaxr nunique_intsinsertr _with_inferrT)rHrrcatna_idx na_unique_idx new_codess r9rzGrouping._group_indexs 00w|| 8 8g{3 33zzuG 499;%00IIgmmbT2G4F4FQVUa**))a-//199V$q($.$;$;CIIgv>!44T__E'> ! ,,coo >((33E**77??Gg~]] &d22t}}MCIIEmmGg~(11$$4::t||NE7g~r:ctj|j|jd}|jj |S)NFr)rrrrrgroupby)rHrs r9rtzGrouping.groupsHs5%%djj$2C2CeT{{""4((r:)NNNTFFTN)r`rrNzNDFrame | Noner%r$rr$rr$r&r$rzArrayLike | Nonerrr)rrrr$)rr)rz int | None)rr)rz$dict[Hashable, npt.NDArray[np.intp]])rznpt.NDArray[np.signedinteger])rrr)rz/tuple[npt.NDArray[np.signedinteger], ArrayLike])rzdict[Hashable, np.ndarray])rnrrrrrrIrrr rrTrrrrrrrrrrrrrtr:r9rrs@48F 07$$ M "$(g/g/ g/  g/g/g/g/"g/ g/R("33"  &&))** * * % % ! !""::,!!<<|))r:rc  j|}|t|tr?t|rt |dk(r|d}|t |r|j |}d}nt|r1t |} | dk(r|d}n| dk(r tdtdt|tr>j|j|k7r5td|dj||dkDs|dkr td d}|}t|trH|jd \} |j | tfS| t|jhfSt|tj r |tfSt|t"s|g} d } n|} t | t |k(} t%d | D} t%d | D}t%d| D}| sk|si|sg| re|ctt&rt)fd| D}n&tt*sJt)fd| D}|st-j.| g} t|t0t"fr|dgt |z} |}n|gt | z}g}t3}dfd }dfd }t5| |D],\}}||rd}|j7|jn||rj8dk7rR|vrN|rj;||d||}}}|j8dk7rtd|d|j7|naj=||rd |d}}}nGt?|t|tr*|j|j7|jd}nd }t|t@stA|||||||n|}|jC|/t |dk(rt r tdt |dk(rI|jCtAtEgdtGjHgtFjJtj ||||} | t|fS)a Create and return a BaseGrouper, which is an internal mapping of how to create the grouper indexers. This may be composed of multiple Grouping objects, indicating multiple groupers Groupers are ultimately index mappings. They can originate as: index mappings, keys to columns, functions, or Groupers Groupers enable local references to axis,level,sort, while the passed in axis, level, and sort are 'global'. This routine tries to figure out what the passing in references are and then creates a Grouping for each one, combined into a BaseGrouper. If observed & we have a categorical grouper, only show the observed values. If validate, then check for key/level overlaps. NrrzNo group keys passed!z*multiple levels only valid with MultiIndexz level name z is not the name of the rz2level > 0 or level < -1 only valid with MultiIndexFrc3VK|]!}t|xst|t#ywr)callabler]dictrygs r9r{zget_grouper..s#Hax{9jD&99Hs')c3HK|]}t|ttfywr)r]r#rrs r9r{zget_grouper..sHaz!gx%89Hs "c 3zK|]3}t|ttttt j f5ywr)r]listtuplerrrrrs r9r{zget_grouper..s+DE 1tUFE2::>?s9;c3nK|],}|jvxs|jjv.ywr)columnsr`rfryrrNs r9r{zget_grouper..s4'=>S[[ 8A$88's25c3NK|]}|jjvywr)r`rfr s r9r{zget_grouper..s&JqCIIOO';&Js"%ct|s2jdk(ryjd} |j|yy#tt t f$rYywxYw)NrFrT)_is_label_likeraxesget_locrb TypeErrorr )r*itemsrNs r9 is_in_axiszget_grouper..is_in_axissZc"xx1}HHRLE  c"  i):;  sAAAct|dsyts trW |j}t|tr6t|tr&|jj|jdSy ||juS#tt t tf$rYywxYw#tt t tf$rYywxYw)NrTFr) rrrrTrb IndexErrorr r r]r_mgrreferences_same_values)gprobj_gpr_columnrNs r9 is_in_objzget_grouper..is_in_objssF#  $6$8 !$SXX#v&:nf+Mxx66"'' #chh-' 'j*;=PQ  *&79LM  s#B ;B+ B('B(+CCTrZrr)rNr+r%rrr&r)r)r%r&r)&rcr]rr rrrr[rrT_get_axis_namer#rQr* frozensetrrrrrallrcomasarray_tuplesafersetzipaddr_check_label_or_level_ambiguity_is_level_referencerbrrrrrhintp)rNr*r-r+r%rrKr& group_axisnlevelsrPkeysmatch_axis_length any_callable any_groupers any_arraylikeall_in_columns_indexlevelsr exclusionsrrrrrTpings` r9rMrMNsz@t$J  j* -E"s5zQa{y/ 11%8 E"e*a<!!HE\$%<==$%QRR%%==&++u4$%eW-""%"4"4T":!;=ebj !UVVEC#w''e'<  77?IK, ,Iswwi0#5 5 C )IK$$ c4 u!IZ8H4HHLH4HHLIMM   M c9 %#&'BF'$ c6* **#&&JT&J#J #))$/0D%%' ;6CJ&D3t9$ "I #J  4$'+ U S>G NN388 $ _xx1}77$7G%)3Cs88q=%}TF:M%NOOt$((4(8&+S$sm# W %#''*= NN377 #GGc8, !   W+Z 9~s3x011 9~%%"8"((2RWW:UVWooj)$vNG Ij)3 ..r:cTt|ttfxs|duxr t|Sr)r]rrr)vals r9rr8s% cC< ( PS_-O3Pr:ct|tr |jSt|trB|jj |r |j S|j|j St|tr |j St|tttttjfrOt|t|k7r t!dt|ttfrt#j$|}|S|S)Nz$Grouper and axis must be same length)r]rr1rr`equalsrreindexrrrrrrrrr[rr)r-rPs r9rr<s'4 {{ GV $ ==   %?? "??4(00 0 GZ ( GdE5+rzzJ K w<3t9 $CD D ge} -++G4Gr:)NrNTFTT)rNrr-rr%r$rr$rKr$r&r$rz5tuple[ops.BaseGrouper, frozenset[Hashable], NDFrameT]r)r-r)Ar __future__rtypingrrr@numpyrpandas._configrr pandas._libsrpandas._libs.tslibsr pandas.errorsr pandas.util._decoratorsr pandas.util._exceptionsr pandas.core.dtypes.commonr rpandas.core.dtypes.dtypesr pandas.corerpandas.core.arraysrrpandas.core.commoncorecommonrpandas.core.framerpandas.core.groupbyrpandas.core.groupby.categoricalrpandas.core.indexes.apirrrpandas.core.seriesrpandas.io.formats.printingrcollections.abcrrpandas._typingrrrr pandas.core.genericr!r#rrMrrrr:r9rOs# 3+247"! '#> &3 ,c&c&L b)b)b)N  g/ g/ g/  g/  g/g/ g/;g/TQr: