f^(h]xddlZddlmZddlmZmZmZddlZddlZddl m Z ddl m Z ddl mZddlmZdd lmZmZmZmZmZmZmZgd Zej4eZe d Gd deZe d GddeZe d GddeZ eGddZ!GddZ"y)N) dataclass)AnyCallableOptional) compatibility)map_arg) ShapeProp) split_by_tags)CALLABLE_NODE_OPSFxNetAccFusionsFinderNamesNodeListNodeSetTensorOrTensorsTensors)FxNetMinimizerBadModuleErrorFxNetMinimizerRunFuncError!FxNetMinimizerResultMismatchErrorF)is_backward_compatibleceZdZdZy)rz9 Raised if failed to split out a minimize module N__name__ __module__ __qualname____doc__Z/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/torch/fx/passes/net_min_base.pyrr!rrceZdZdZy)rz@ Raised if error occurs during run_a or run_b functions Nrrrrrr(r rrceZdZdZy)rzJ Raised if comparing function thinks the results are mismatching. Nrrrrrr/r rrc^eZdZUdZdZeed<dZeed<dZ eed<dZ eed<dZ eed<d Z y ) _MinimizerSettingBasea Args: `accumulate_error`: Instead of using a's input for both converted module to verify , use the previous outputs of each converted module as input to accumulate the errors. `traverse_method`: "sequential" or "binary" or "accumulate" Determine the way of traverse the nodes in FX module. `find_all`: Minimizer will go through the entire model and return all problematic nodes. `return_intermediate`: If true, when using `run_nodes()` function to run the model, intermediate results of all the ops will be returned as output. `all_outputs`: If true, when using `_run_and_compare()` function, all the output nodes in the subgraph will be used for comparison. Faccumulate_error sequentialtraverse_methodfind_allreturn_intermediate all_outputscdd}t|jD]\}}|d|d|dz }|S)NzFX Minimizer Settings:  :  )varsitems)self settings_strkvs r__str__z_MinimizerSettingBase.__str__PsG1 J$$& ,DAq b2aSO +L ,rN) rrrrr%bool__annotations__r'strr(r)r*r5rrrr$r$6sB$#d"'OS'Hd %%Krr$ceZdZdZ d6dej j dedee e e ge e e ffdedeeeej j egdfdeeeeegdff d Zd7d Z d8d ej j d eded e fdZ d8d ej j d eded e fdZde de dej j fdZdej j ded e eeffdZdefdZded e ej j effdZ d8dej j dede defdZd ed!ed"ed efd#Zded efd$Zded efd%Zded!ed"ed&e d ef d'Z ded&ee d efd(Z!ded efd)Z"ded efd*Z#d ed!ed"ed efd+Z$d ed,e%d efd-Z&d.eed/eed efd0Z'd6d.eed/eefd1Z(d2e%efd3Z)d4Z* d9d.eed/eed,ee%d&ee d ef d5Z+y):_MinimizerBasea This class is used to automatically find problematic nodes in a model. It takes a FX graphmodule and generate some submodules while traverse the graph. Then two functions `run_a` and `run_b` will be used to run the same submodule and a function `compare_fn` will be used to compare the results. Currently we provides two ways to traverse the graph and generate submodules. 1. Sequential traversal: this will traverse the graph node by node and generate one submodule with one sigle node. 2. Binary searching: this will do a binary search style traversal on the graph. For internal Users, a guide can be found here https://fb.quip.com/HDtuAgiKGfkP. Nmodule sample_input compare_fnsettingsmodule_exporter exclusion_fnc t|tjjsJ||_||_||_||_||_||_ i|_ i|_ i|_ g|_ d|_|jjj Dchc]}|j"t$vs|}}|j't)|j||_|jjj Dcgc]}|j"dk(s|j, } }t/| t/|j k(sJt1| D])\} } || |j| <|| |j| <+ycc}wcc}w)Nr placeholder) isinstancetorchfx GraphModuler;r<r=r?r>r@ a_outputs b_outputsresultsreports iterationgraphnodesopr run_shape_propr fusionsnamelen enumerate) r1r;r<r=r>r?r@nodecallable_nodes placeholdersirQs r__init__z_MinimizerBase.__init__hsy&%(("6"6777 ($.  (*,*,(* )+  "[[..44 CT8TD   I,T[[.IK #'++"3"3"9"9 TWW =UDII  < C(9(9$::::!. 3GAt#/?DNN4 #/?DNN4  3  sF.FF F returnc\t|jj|jy)z Helper function to run shape propagation on module. Can be overridden by subclasses for custom shape propagation logic. N)r r; propagater<)r1s rrOz_MinimizerBase.run_shape_props# ) $++(($*;*;z0_MinimizerBase._store_outputs..s  dgg6ID srN) nextrLrMrCargsrDrENoderGrQrHrS)r1rgrhri output_noderWargs r_store_outputsz_MinimizerBase._store_outputss  &__22  k&&q)588== 97?DNN;++A.33 47?DNN;++A.33 4$K$4$4Q$78 73+3A;sxx(+3A;sxx( 7r main_module submod_pathc g g}t||}|jjDcgc]}|jdk(s|j }}t ||j jkrD|D]>} j|j ||j|j|@n}|jjrtd|ddtjjdt f fd }|j#|} ||j$| j' }|jjs fS |fScc}w)al Try get submodule inputs from stored outputs. If not found then use torch_glow.get_submod_inputs to get the inputs. If accumulate_error is False, use a_input for run_a() and run_b() otherwise use a_input for run_a and b_input for run_b. Args: main_module: Top-levlel fx module. submod_path: Path to the submodule we want to run and compare results. Returns: a_input: List of tensor(s) that will be used by run_a() as submodule inputs. b_input: List of tensor(s) that will be used by run_b() as submodule inputs. rBz)Can't find previous stored outputs named !r1r]c |yNr)r1r]a_inputs r get_inputsz5_MinimizerBase._get_submod_inputs..get_inputss r)getattrrLrMrNrQsetrGkeysappendrHr>r%printrDnnModulerregister_forward_pre_hookr<remove) r1rurvb_inputrirTrVrQr|handler{s @r_get_submod_inputsz!_MinimizerBase._get_submod_inputss5$K5 "+//"7"7 477m;SDII   |  3 3 5 5$ 5t~~d34t~~d34 5}}--A,qQR ! !# ! 88DF ** + MMOG}}--G# #; s EEselected_nodesc|jjjD]L}|jtvr||vrd|_"t d|jDrd|_Fd|_Ny)ag Tag selected nodes with tag "minimize". Nodes with the same tags will be split to the same submodule afterwards. Args: selected_nodes: Nodes that we want to minimize. We will tag those nodes with "minimize", all preceding nodes with "main_0" and all following nodes with "main_1". minimizec3\K|]$}|jtvr|jdv&yw)>main_1rN)rNr tag)rmns rrnz,_MinimizerBase._tag_nodes..s/44,,//s*,rmain_0N)r;rLrMrNr ranyall_input_nodes)r1rrTs r _tag_nodesz_MinimizerBase._tag_nodes smKK%%++ $Dww//~%%-- $# $rrMc|j|t|jgd}d}|jD]\}}d|vr |dk(r|}t d||dk(rt d|||fS)ak Split self.module so that one submodule consists of `nodes` and only `nodes`. Args: nodes: Nodes that we want to include in the minimize submodule. Returns: split_module (torch.fx.GraphModule): the module after split. submodule_name (str): the name of the submodule that consists of `nodes`. )rrrrz0Expected only one minimize submodule with nodes z,Minimize submodule was not found with nodes )rr r;named_childrenr)r1rM split_modulesubmodule_name child_name_s r_build_submodulez_MinimizerBase._build_submodule"s %T[[2RS !)88: MJ+#!+2FugN  R .>ugF ^++rr submod_name output_namesct||}|j||\}}t|jdk(r"|jj gd|_|j|dk\r|n|j dz }|j d|r|j jsg} |jjD]L} | jdk(r|jj| | j|vs<| j | N|jjt| dk(r| dn t| |jj|j!|jjD])} | jdk(st#| j$d} + |j'|||} |j)|||} |j+| | ||}| Dcgc] }t1|}}|j3| | |\}}||j4 <|j d ||s|j d | |j6rYt9| tr| d } |j7||t1| dd z|j7||t1| dd zt;d | y#t,$r-}|j d|d|t/d|d|d}~wwxYwcc}w)a Run the submodule in `split_module` that has name `submod_name` using `self.run_a` and `self.run_b` and compare their results. Args: split_module: Main module that contains the minimize submodule. submod_name: Name of the minimize submodule. output_names: Names of the node we want to output. If None, we will use the original output. rr zRun and compare ...rlc|jSrz)rQ)xs rz1_MinimizerBase._run_and_compare..us !&&rzException raised when running r-NzNumerical accuracy = zResult mismatch for _cpu_acc)r}rrRrJrrKr>r*rLrMrN erase_noderQrltuplelint recompilerrprcrfrt Exceptionrr8r=rIr?rCr)r1rrrr^rir{rreport output_nodesrT result_keyrgrhenamesr4numeric_result bool_results r_run_and_comparez_MinimizerBase._run_and_compareHs"L+6 22<M t||  ! LL   #DNJ!OjRSAST +,  9 9%'L!-- .77h&OO..t499 , ''-  . OO " "#&|#4#9 Qu\?R  OO "    !OO)) BDww("$TYY0@A  B zz)WjAHzz)WjAH   (I >$  %/0SV0E0&*ooh%&P# #1 Z  -n-=>? MM0 = >##j%0!+BJ$$ 1 &/ $$ 1 &/ 46J:,4WX X#  MM:;-r!M N,0 RsC  1s39K 4L L(K<<L all_nodes start_idxend_idxct}|||}g}|jA|j|||t|dk(r dg}|jj ||S|dj }|dj }|xj dz c_|jj ||j d|j |j d|d|d|dz d|d t| t|} |j| \} } |j| | |g|j d |j|tS#ttf$rt|dk(r*|j d | d |j|| cYS|j d |j|t|dz} |j|||| z}t|dk7r|jjs|cYS|j||| z|}t|dk(r%|j d||j||cYSwxYw)z9 Recursive binary search implementation. rAll nodes are excluded by userrr zBinary search iteration From node index : to &. Size of the interested node list is No discrepancy found.zcThis is the last node in the sub-module. Search in the current branch is successful with culprit = .MProceed to split and lower the halves of the current sub-module individually.zaFurther split and lowering found no errors. Unable to minimize the submodule with list of nodes: )r~r@rRrJrrQrKrr print_reportrr_binary_search_implr>r() r1rrrculpritsrMrfirst_node_nameoutput_node_name cur_nodesrrmids rrz"_MinimizerBase._binary_search_implsm  E#Ig6    (   eY 85zQ:; ##F+(-- 9>> ! F# 00@AB yk?*;4! }AN^M_`336u:, @ !Z & (,(=(=i(H %L+  ! !, >N=O PD MM1 2   f %5LE+,MN 5zQ QQZP[[\^!!&)  MM+    f %e*/C// 9iRUoVH8}!$--*@*@// 9s?GTH8}! LLQ7T!!&)O= s!(EAIA,I A IIc:|j|dt|S)z7 Binary search on `nodes` for culprit. r)rrR)r1rMs r_binary_traversez_MinimizerBase._binary_traverses''q#e*==rct}|D]q}g}|jj||xjdz c_|jd|jd|jd|jt j d|j|g}|jl|j|ddt|dk(rK|jd|j|j||jjs|cS|h}||jvr|j|} |j|\}}|j|||jg|j|t|S#t$rV|j!||jd ||j||jjs|cYcSYt"$rV|j%||jd ||j||jjs|cYcSY2wxYw) zX Traverse `nodes` one by one and determine if any of them is a culprit. r zSequential traverse iteration rz Visit node: zVisit node: %srrzUser exclusion : "Found culprit from numeric error: Found culprit from run error: )r~rJrrKrQ_LOGGERinfor@rRrr>r(rPrrraddrupdate) r1rMrrTr node_listrrrs r_sequential_traversez#_MinimizerBase._sequential_traverses E' $D "F LL   ' NNa N MM:4>>:J!L M MML 4 5 LL)499 5#'&I  ,!!)R4y>Q&MM$5dii["AB%%f-==11' "&It||# LL.  $,0,A,A),L) k%%lK$))M!!&)7' $R5 $ T" B4&IJ!!&)}}--#O.- $ * >tfEF!!&)}}--#O.  $s ! F# )$..)9:   fGDDUV"9+QuY/?/D/D.ET'RSTYZaTbTgTgShhjk*?*;4?OP"#3"456w7JQ7N6OP   &* < X(,(=(=i(H %L+  ! !kN+;Z @ MM8QuSz>OqQ C CN$+]53C3H3H2IK!!&)(6y1}IIMI MM+    f %0037G^00 3WWo23MN  MM+O+ threshold 2nd pass: search for start_idx by finding the first node in culprit block where Numerical accuracy (start_idx, end_idx) < threshold Form minimum block by (start_idx - 1, end_idx) rrzBlock search from rz2**************************************************r TFz(Start searching for last node in culpritz Finish Pass 1zFind end_idx = rz)Start searching for first node in culpritz Finish Pass 2zFind start_idx = z'Finish searching, found minimum block (,)) r~rQrrJrRrrrr) r1rMrrrrlast_node_reportrrrun_bothfirst_node_report result_reports r_block_traversez_MinimizerBase._block_traverses E(--r00AnEUVW) ,- e*q.)14u ~  # #$N O   . ///y'4PG  # # OG9AeGn>Q>Q=R"ST    . / >!L M    / 011a'A+& 7EI  $ $X . LL   1 2  $ $#' {!E)4D4I4I3JK    / 0 i'A+675eI6F5GqwHXXY Z  M* -(rct}|j|j|ddt|dk(r dg}|jj ||S|dj }|dj }d|d|g}t|} |j |\}}|j|||g|j||S#ttf$r*|j d||j||cYSwxYw)zL run user defined `nodes` and determine if it is a culprit. rrrzDefined graph from rFound culprit ) r~r@rRrJrrQrrrrr) r1rMrrrrrrrs r_defined_traversez _MinimizerBase._defined_traverses E    (   eR , u:?67F LL   'O(-- 9>>''8=MN=O P   f %  23MN  MMN9+6 7   f %O s 9C6C?>C?ct}t}|jjr td|S|D]}g}|jj ||xj dz c_|j d|j d|j||j}|t|tr|d}|t|ts Jd||j d| |j|\}}|j|||g|j||S#tt f$r=|j||j d||j||cYcSwxYw) Nz9'Find All' mode is not supported in accumulate traversal.r zAccumulate traverse iteration rrzminimize: node_name: z Add node: r)r~r>r(rrJrrKrrQrCrr8rrrrr) r1rMr nodes_to_runrTr node_namerrs r_accumulate_traversez#_MinimizerBase._accumulate_traverseswE #  == ! ! M NO D "F LL   ' NNa N MM:4>>:J!L M   T " I$Iu)E%aL (Z3. 3&yk2 3 MMJyk2 3 ,0,A,A,,O) k%%lK)M!!&)) 6 67QR  T" tf56!!&)  s39D00AE<;E<c t}|||}t|}|j|j|||t|}n4|D]/}||jvs|j|j|1g}|jj ||xj dz c_|j d|j d|j d|d|dz dt| |j|\} } |j| | g|j d|j|tS#t$r;|j||j d||j||cYSt$r;|j||j d ||j||cYSwxYw) ? Skip certain nodes in graph based on settings r z Nodes block rrrrrrr) r~r@rPrrJrrKrRrrrrr) r1rrrrrMrrTrrrs r_skip_traverse_implz"_MinimizerBase._skip_traverse_impls  E#Ig6 Z    (   eY 8E I 94<<'$$T\\$%78 9 F# ! dnn%5Q78 ykgk];336u:, @  (,(=(=i(H %L+  ! !, R @ MM1 2   f %5L1  OOI & MM>ykJ K   f %O)  OOI & MM:9+F G   f %O  s2'EAG AG  G  skip_nodescd}t|}d}t}||kr_||}|j|vr||kDr|j|||}|dz}n#||dz k(r||kr|j|||dz}|dz }||kr_|S)rrr )rRr~rQr)r1rrr num_nodesidxrrTs r_skip_traversez_MinimizerBase._skip_traverse's  N 5IoS>DyyJ&?#77 9cRH!G  A %)s*:33Iy#PQ'R 1HCIorstartendcg}|du}|jjjD]K}|jtvr|j |k(rd}|r|j ||j |k(sJ|S|S)z Collect nodes in the model that between nodes with name of `start` and `end`. These two nodes are also included. NT)r;rLrMrNr rQr)r1rrrMadd_noderTs r_collect_nodesz_MinimizerBase._collect_nodes;s| D=KK%%++ Dww//yyE! T"yyC   rc|j||}t|}|D]/}||jvs|j|j|1g}|jj r|Dcgc]}|j }} |j|\}}|j|||ycc}w#ttf$r} t| Yd} ~ yd} ~ wwxYw)a] Run part of the model from `start` node to `end` node. If `start` is None then we start from the beginning of the model. If `end` is None then we stop at the end of the model. Args: start: The name of the node which is the first node of the submodule we want to run. If set to None, then we'll start with the first node of the model. end: The name of the node which is the last node of the submodule we want to run. If set to None, we'll end with the last node of the model. N) rr~rPrr>r)rQrrrrr) r1rrrMrrTrrrrs r run_nodesz_MinimizerBase.run_nodesRs##E3/J  5Dt||#  d!34 5 == , ,278$DII8L8 (,(=(=i(H %L+  ! !, \ J 9 ' -   !HH  s.B,'B11C CCrctt|D]'}|dkDrtd||zt||)y)Nrz . )rangerRr)r1rrWs rrz_MinimizerBase.print_reportts>s6{# !A1uefQi'(fQi  !rcH|jD]}|j|yrz)rJr)r1rs r print_reportsz_MinimizerBase.print_reports{s#ll &F   f % &rct|jt|jj|j ||}|jj dk(r|j |S|jj dk(r|j|S|jj dk(r|j|S|jj dk(r| td|j||S|jj dk(r|j|S|jj dk(r|j||Std|jj d ) az Minimizing the model from node with name `start` to node with name `end` base on self.settings. Find culprits that causes FxNetMinimizerRunFuncError or FxNetMinimizerResultMismatchError errors. Args: start: The name of the node where we want to start minimizing. If set to None, then we'll start with the first node of the model. end: The name of the node where we want to terminate minimizing. If set to None, we'll end with the last node of the model. skip_nodes: The names of nodes where we want to skip during minimizing. It'll create subgraphs without these skip nodes under the hood. Only applicable in mode "skip". find_last_node: True if only last_node of a culprits is needed in mode "block". False if only the first_node of a culprits is needed. Only applicable in mode "block". Returns: nodes: A list of nodes that causes FxNetMinimizerRunFuncError or FxNetMinimizerResultMismatchError errors during minimizing. r&binary accumulateskipz<'skip_nodes' can't be None when 'traverse_method' is 'skip'.definedblockzUnknown traverse method rx) rr>r;rLrr'rrrrarrr)r1rrrrrMs rrz_MinimizerBase.minimizesC: dmm dkk ##E3/ == ( (L 8,,U3 3 == ( (H 4((/ / == ( (L 8,,U3 3 == ( (F 2!"R&&uj9 9 == ( (I 5))%0 0 == ( (G 3''~> >5dmm6S6S5TTUVWWr)NN)rYN)r)NNNN),rrrrrDrErFrrrrrfloatr6r$rr8rintrXrOrcrfrtrrrrrrrrrrrrrlistrrrrrrrrrr:r:Ys 0 GK43$$4343 ou 5uUD[7I I 43(43" guxx33S94? @ 43x3(A2 w 2 h$$2$,g$,%8L8Lc8Q2R$,V SYhh**SYSY SY  SYjB!B.1Bh>7> /(/w/beXeX*-eX8;eXMQeX eXN99/7~9 9vxG6%(%w%N)!).1)rs!** 1!!&  '  H %e,9- e,- e, -   D^ X^ Xr