Q^(hD TdZddlZddlmZmZddlmZddlmZddl m Z m Z m Z m Z mZmZmZddlmZmZmZmZd d lmZd d lmZ dd lmZdd lmZddlmZdZ edZ"de de fdZ#dZ$GddeZ%d.dedee"fdZ&e# d.dede e'dee"de e'fdZ(e#d/de'de)de)fd Z* d.d!e ede e'dee"de fd"Z+ d.dede e'dee"de e'fd#Z,d/de'de)de)fd$Z-ded%e e'de'fd&Z.d'e'de e'fd(Z/d)e$d*e$d+jaZ1d,ede fd-Z2y#e!$rdZ YwxYw)0asContains command to delete some revisions from the HF cache directory. Usage: huggingface-cli delete-cache huggingface-cli delete-cache --disable-tui huggingface-cli delete-cache --dir ~/.cache/huggingface/hub huggingface-cli delete-cache --sort=size NOTE: This command is based on `InquirerPy` to build the multiselect menu in the terminal. This dependency has to be installed with `pip install huggingface_hub[cli]`. Since we want to avoid as much as possible cross-platform issues, I chose a library that is built on top of `python-prompt-toolkit` which seems to be a reference in terminal GUI (actively maintained on both Unix and Windows, 7.9k stars). For the moment, the TUI feature is in beta. See: - https://github.com/kazhala/InquirerPy - https://inquirerpy.readthedocs.io/en/latest/ - https://github.com/prompt-toolkit/python-prompt-toolkit Other solutions could have been: - `simple_term_menu`: would be good as well for our use case but some issues suggest that Windows is less supported. See: https://github.com/IngoMeyer441/simple-term-menu - `PyInquirer`: very similar to `InquirerPy` but older and not maintained anymore. In particular, no support of Python3.10. See: https://github.com/CITGuru/PyInquirer - `pick` (or `pickpack`): easy to use and flexible but built on top of Python's standard library `curses` that is specific to Unix (not implemented on Windows). See https://github.com/wong2/pick and https://github.com/anafvana/pickpack. - `inquirer`: lot of traction (700 stars) but explicitly states "experimental support of Windows". Not built on top of `python-prompt-toolkit`. See https://github.com/magmax/python-inquirer TODO: add support for `huggingface-cli delete-cache aaaaaa bbbbbb cccccc (...)` ? TODO: add "--keep-last" arg to delete revisions that are not on `main` ref TODO: add "--filter" arg to filter repositories by name ? TODO: add "--limit" arg to limit to X repos ? TODO: add "-y" arg for immediate deletion ? See discussions in https://github.com/huggingface/huggingface_hub/issues/1025. N) Namespace_SubParsersActionwraps)mkstemp)AnyCallableIterableListLiteralOptionalUnion)CachedRepoInfoCachedRevisionInfo HFCacheInfoscan_cache_dir)BaseHuggingfaceCLICommand)ANSI)inquirer)Choice) SeparatorTF alphabetical lastUpdatedlastUsedsizefnreturnc.tfd}|S)z4Decorator to flag methods that require `InquirerPy`.c6ts td|i|S)NzThe `delete-cache` command requires extra dependencies to work with the TUI. Please run `pip install huggingface_hub[cli]` to install them. Otherwise, disable TUI using the `--disable-tui` flag.)_inquirer_py_available ImportError)argskwargsrs c/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/huggingface_hub/commands/delete_cache.py_innerz#require_inquirer_py.._innerVs+%Q  4"6""r)rr(s` r'require_inquirer_pyr*Rs" 2Y## Mr)CANCEL_DELETIONc8eZdZedefdZdeddfdZdZy)DeleteCacheCommandparserc|jdd}|jdtdd|jddd |jd d gd d|jty)Nz delete-cachez*Delete revisions from the cache directory.)helpz--dirzEcache directory (optional). Default to the default HuggingFace cache.)typedefaultr0z --disable-tui store_truezrDisable Terminal User Interface (TUI) mode. Useful if your platform/terminal doesn't support the multiselect menu.)actionr0z--sort?rzSort repositories by the specified criteria. Options: 'alphabetical' (A-Z), 'lastUpdated' (newest first), 'lastUsed' (most recent first), 'size' (largest first).)nargschoicesr0)func) add_parser add_argumentstr set_defaultsr-)r.delete_cache_parsers r'register_subcommandz&DeleteCacheCommand.register_subcommandis$//Eq/r(( X )  (( K )  (( G* )  ((.@(Ar)r%r Ncj|j|_|j|_|j|_yN)dir cache_dir disable_tuisortsort_by)selfr%s r'__init__zDeleteCacheCommand.__init__s&(,!%!1!126)) r)c 2t|j}|jrt|g|j}nt |g|j}t |dkDrt|vrt||dz}|jr t|}n t|}|rp|j|}td|jtdt |jdt |jd|j dy td y ) z/Run `delete-cache` command with or without TUI.) preselectedrErz Confirm deletion ?zStart deletion.zDone. Deleted z repo(s) and z revision(s) for a total of .Nz"Deletion is cancelled. Do nothing.)rrBrC_manual_review_no_tuirE_manual_review_tuilen_CANCEL_DELETION_STR_get_expectations_str_ask_for_confirmation_no_tui_ask_for_confirmation_tuidelete_revisionsprintexecuterepos snapshotsexpected_freed_size_str)rF hf_cache_infoselected_hashesconfirm_message confirmedstrategys r'runzDeleteCacheCommand.runs't~~6    3Mr[_[g[ghO0BX\XdXdeO  ! #(rrGr]r)r'r-r-hs8 B$5 B BDz(_get_repo_sorting_key..s@#C%%@srr) repo_typerepo_idlowermax revisions last_accessed size_on_diskrcrEs r'_get_repo_sorting_keyrss.  2 2 455 M !@@@@@ J """" F !!!! --r)rXrIctj||}tjd|ddt |Dcgc],}t |t s|js!|j.c}ddd fd }jd jd |i jScc}w#t$rgcYSwxYw) zzAsk the user for a manual review of the revisions to delete. Displays a multi-select menu in the terminal (TUI). )rUrIrEzSelect revisions to delete:FdrYzWPress to select, to validate and to quit without modification.ct|dS)Nz revision(s) selected.)rM)results r'z$_manual_review_tui..sc&k]2H#Ir))messager7cycleheight instructionlong_instruction transformerctjjDcgc] }|ds |dc}_ycc}w)Nenabledvaluerv)rOcontent_controlr7 _instruction)_choicecheckboxrXs r'_update_expectationsz0_manual_review_tui.._update_expectationss?!6 ;C;S;S;[;[q_efo_pVG_q! qs A A toggler8)r N) _get_tui_choices_from_scanrUrrrO isinstancerrrkb_func_lookupappendrTKeyboardInterrupt)rXrIrEr7crrs` @r'rLrLs)!!G   -* .5]Av9NSTS\S\QWW] sIH&  H%,,f6J-KL!!-^.  s#B1 B1B1!B66 CCrzr2cLtj||jS)z$Ask for confirmation using Inquirer.r2)rconfirmrT)rzr2s r'rQrQs    GW 5 = = ??r)rUcfg}|jttddt|fd}|D]}|jt d|j j d|jd|jd |jd t|jtD]z}|jt|j|jd d d djt|jxsdd|j|j|v||S)aBuild a list of choices from the scanned repos. Args: repos (*Iterable[`CachedRepoInfo`]*): List of scanned repos on which we want to delete revisions. preselected (*List[`str`]*): List of revision hashes that will be preselected. sort_by (*Optional[SortingOption_T]*): Sorting direction. Choices: "alphabetical", "lastUpdated", "lastUsed", "size". Return: The list of choices to pass to `inquirer.checkbox`. z=None of the following (if selected, nothing will be deleted).F)namerct|Sr@rsrrs r'ryz,_get_tui_choices_from_scan..s2Gg2Vr)key   (, used )Nz: , (detached) # modified )rrrNsortedrrk capitalizerlsize_on_disk_strlast_accessed_strro_revision_sorting_order commit_hashjoinrefslast_modified_str)rUrIrEr7 sorted_reposrcrevisions ` r'rrs>$/1G NN P %%VWL T^^..014<<.4CXCXBYZ//03  t~~3JK H NN((#//345 IIfX]]&;<L MN%%-%?%?$@B%00K?  . Nr)c 0td\}}tj|g}t|jfd}|D]}|j d|j jd|jd|jd|jd t|jtD]h}|j |j|vrd nd d |jd djt|jxsdd|j jt#|d5} | j%t&| j%dj|ddddt)j*|d} t-djd| j/j1dD t3|} t5t7|| dzdrn'tj8|t| S#1swYxYw)zAsk the user for a manual review of the revisions to delete. Used when TUI is disabled. Manual review happens in a separate tmp file that the user can manually edit. z.txt)suffixct|Sr@rrrs r'ryz'_manual_review_no_tui..Gs@UVZ\c@dr)rz # rrrr#z z # Refs: rrrwrNa* TUI is disabled. In order to select which revisions you want to delete, please edit the following file using the text editor of your choice. Instructions for manual editing are located at the beginning of the file. Edit the file, save it and confirm to continue. File to edit: z c3<K|]}|jywr@)strip)rhlines r'rjz(_manual_review_no_tui..esNTDJJLNsz Continue ?Fr)roscloserrUrrkrrlrrrorrrrropenwrite"_MANUAL_REVIEW_NO_TUI_INSTRUCTIONSrboldrSrsplit_read_manual_review_tmp_filerPrOremove) rXrIrEfdtmp_pathlinesrrcrf instructionsrYs ` r'rKrK7s&)LBHHRL E---3deL 4>>,,./qbAVAV@WX++,A / t~~3JK H LL!--<2#FG(()*IIfX]]34D EF../ 1    h " 23  % !"  99X&'( L $))N\-?-?-A-G-G-MN NO 6x@ ' !- AM Q   IIh / ""3""s /6H  Hcd}d}d}||z|fz}||rdndz} t|j}||k(r|S||vry||vrytd|9) z'Ask for confirmation using pure-python.)yyes1)nno0rz (Y/n) z (y/N) TFzInvalid input. Must be one of )inputrmrS)rzr2YESNODEFAULTALL full_messageanswers r'rPrPus{ C BG (gZ C7i BL |$**, W N S= R< .se45 r)rYcht|vry|j|}t|d|jdS)zFormat a string to display to the user how much space would be saved. Example: ``` >>> _get_expectations_str(hf_cache_info, selected_hashes) '7 revisions selected counting for 4.3G.' ``` zNothing will be deleted.z! revisions selected counting for rJ)rNrRrMrW)rXrYr\s r'rOrOsC.)-}--?H/" ##DXEeEeDffg hhr)rct|5}|j}dddjdDcgc]}|j}}|Dcgc]}|j dr|}}|Dcgc]$}|jddj&}}|Dcgc]}t |dkDs|c}S#1swYxYwcc}wcc}wcc}wcc}w)aRead the manually reviewed instruction file and return a list of revision hash. Example: ```txt # This is the tmp file content ### # Commented out line 123456789 # revision hash # Something else # a_newer_hash # 2 days ago an_older_hash # 3 days ago ``` ```py >>> _read_manual_review_tmp_file(tmp_path) ['123456789', 'an_older_hash'] ``` Nrrr)rreadrr startswithrM)rrcontentrrselected_linesrYhashs r'rrs* h1&&('.mmD&9 :dTZZ\ :E :(-ItDOOC4HdINI?MMdtzz#q)//1MOM- >TD A D >> ;JN ?s.CC C,C6)C%C9CC a # INSTRUCTIONS # ------------ # This is a temporary file created by running `huggingface-cli delete-cache` with the # `--disable-tui` option. It contains a set of revisions that can be deleted from your # local cache directory. # # Please manually review the revisions you want to delete: # - Revision hashes can be commented out with '#'. # - Only non-commented revisions in this file will be deleted. # - Revision hashes that are removed from this file are ignored as well. # - If `aa` line is uncommented, the all cache deletion is cancelled and # no changes will be applied. # # Once you've manually reviewed this file, please confirm deletion in the terminal. This # file will be automatically removed once done. # ------------ # KILL SWITCH # ------------ # Un-comment following line to completely cancel the deletion process # z, # ------------ # REVISIONS # ------------ rc|jSr@rf)rs r'rrs  ! !!r)r@)T)3__doc__rargparserr functoolsrtempfilertypingrr r r r r rutilsrrrrrr _cli_utilsr InquirerPyrInquirerPy.base.controlrInquirerPy.separatorrr#r$SortingOption_Tr*rNr-rsr;rLboolrQrrKrPrOrrrrrbr)r'rs7*X 1JJJSS'##..!KLH&)K42K4\ . ./9R .*.00c0o &0 #Y 00f@s@T@T@@*.7 N #7c7o &7 7z*.;#;#c;#o &;# #Y ;#|6#666$ i itCy iUX i"?3"?49"?J *  !+&4 EG5#:"&8"S"Y #"#sDD'&D'