[^(hb-ddlmZddlZddlZddlZddlmZmZddlZddl m Z er ddl Z ddl mZejeZ d d dZdZ d d dZy) ) annotationsN)IO TYPE_CHECKING) _type_utils)FileLikecddl}|3tjj|jj n(tjj |j}|<|j|jk7r|j|j}|j}||_ |j|_ |jj|j|jj |_|d|j%j'd}|j)D]6\} } |j*j-} | | _t1| | _8t4j6j9||} t4j6j;| rt5j<| t4j6j?| } t4j6j;| st5j@| tC| d5}|jE|jGdjIddd|S#1swY|SxYw)aCreate a TensorProto with external data from a PyTorch tensor. The external data is saved to os.path.join(basepath, location). Args: tensor: Tensor to be saved. name: Name of the tensor (i.e., initializer name in ONNX graph). location: Relative location of the external data file (e.g., "/tmp/initializers/weight_0" when model is "/tmp/model_name.onnx"). basepath: Base path of the external data file (e.g., "/tmp/external_data" while model must be in "/tmp"). Reference for ONNX's external data format: How to load? https://github.com/onnx/onnx/blob/5dac81ac0707bdf88f56c35c0a5e8855d3534673/onnx/external_data_helper.py#L187 How to save? https://github.com/onnx/onnx/blob/5dac81ac0707bdf88f56c35c0a5e8855d3534673/onnx/external_data_helper.py#L43 How to set ONNX fields? https://github.com/onnx/onnx/blob/5dac81ac0707bdf88f56c35c0a5e8855d3534673/onnx/external_data_helper.py#L88 rN)locationoffsetlengthxbT)force)%onnxjit_type_utils JitScalarTypefrom_onnx_type tensor_type elem_type from_dtypedtypeto TensorProtoname onnx_type data_typedimsextendshapeEXTERNAL data_locationuntyped_storagenbytesitems external_dataaddkeystrvalueospathjoinexistsremovedirnamemakedirsopenwritenumpytobytes)tensorrr basepathdtype_overrider scalar_type tensor_protokey_value_pairskventryexternal_data_file_pathexternal_data_dir_path data_files c/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/torch/onnx/_internal/fx/serialization.py'_create_tensor_proto_with_external_datar@s6  % $$33  & & 0 0  ) ) 4 4V\\ B !k&7&7&9V\\&I;,,./#4##%LL(224LV\\*!%!1!1!:!:L ((*113O  %%'1**..0 !f  !ggll8X> ww~~-. )* WW__-DE 77>>0 1 *+ %t ,<   4 088:; <  < s ,0I&&I0cddlm}i}||dd5}|jD]$}|j|j ||<& ddd|S#1swY|SxYw)Nr) safe_openptcpu) frameworkdevice) safetensorsrBkeys get_tensorrD)safetensors_filerBtensorsfr9s r?$_convert_safetensors_to_torch_formatrMish&G #tE B/a /Aa,,.GAJ // N/ Ns 8AA!c ddl}i}t|jjD cic]\}} | j|} }} |jj D chc]} | j} } |D]} t | tr| }nit | ttjfr0tj| jdr t| }n tj| dd}|j5D]\}}|r|j7d d }|| vr| j9|n>| D]9}|j|s|j|s&|}| j9|ntj:j=||}|jj D cic]} | j| j>}} || vr||| |<tA|||||jC|d}|jjjE|ttG|j5d }|jID]}|jj|=|jJ|tj:j=||ycc} }wcc} w#t t"f$r}dt|vs@t | t$j&t(fr~| j+rn| j-r^t.j1dt | t$j&t(fr| j3dtj| d }n|Yd}~|d}~wwxYwcc} w) aLoad PyTorch tensors from files and add to "onnx_model" as external initializers. Output files: ONNX model file path: ONNX initializer folder: os.path.join(basepath, initializer_location) After running this function, you can do ort_sess = onnxruntime.InferenceSession(os.path.join(basepath, model_location)) to execute the model. Arguments: basepath: Base path of the ONNX external data file (e.g., "/path/to/large_model/"). model_location: Relative location of the ONNX model file. E.g., "model.onnx" so that the model file is saved to "/model.onnx". initializer_location: Relative location of the ONNX initializer folder. E.g., "initializers" so that the initializers are saved to "/initializers/". Note: When initializers are >2GB, must be the same as `model_location`. torch_state_dicts: Dictionaries or files which contain PyTorch tensors to be saved as ONNX initializers. For non-dict arguments, `torch.load` will be used to load them from file-like objects. onnx_model: ONNX model to be saved with external initializers. If an input name matches a tensor loaded from "torch_state_dicts", the tensor will be saved as that input's external initializer. rename_initializer: Replaces "." by "_" for all ONNX initializer names. Not needed by the official torch.onnx.dynamo_export. This is a hack for supporting `FXSymbolicTracer` tracer with fake tensor mode. In short, `FXSymbolicTracer` lifts FX parameters (self.linear_weight) as inputs (`def forward(self, linear_weight)`) and therefore, `.` cannot be used. rNz .safetensorsrDT) map_locationmmapz+mmap can only be used with files saved withzFailed to load the checkpoint with memory-map enabled, retrying without memory-map.Consider updating the checkpoint with mmap by using torch.save() on PyTorch version >= 1.6.)rO._)reverse)&r enumerategraph initializerrinput isinstancedictr&r(PathLikefspathendswithrMtorchload RuntimeError ValueErrorioIOBaserreadableseekablelogwarningseekr"replacer,r)r*typer@popappendsortedrHsave)r4model_locationinitializer_locationtorch_state_dicts onnx_modelrename_initializerrinitializers_to_be_deletedidxr9existing_initializersrWonnx_input_namesel state_dicterr3onnx_input_namerelative_tensor_file_pathmodel_input_typesr7s r?save_model_with_external_datar}vs&N!#"+J,<,<,H,H"IQ 1;0@0@0F0FGu GGM> b$ J"sBKK01biim6L6L7B"E  "'BU!NJ"',,.. >LD&! ||C-'' ''-'7O&//59W /(//@ )+ 5I4(P %:D9I9I9O9O PA P  P,,JN*+@+FGB)!%%dD1 L    ( ( / / =]. >?M>^"&)//14@"*..0.    ( ( -.DIIj"'',,x@AuH %j1 DAN"2 27KKMKKM z&b299b/:GGAJ%*ZZ%G # b!Qs*J!J'J,)M6,M3;B-M..M3)N) r3z torch.Tensorrr&r r&r4r&r5zonnx.TypeProto | Nonereturnzonnx.TensorProto)F)r4r&rnr&ror&rpztuple[dict | FileLike, ...]rqzonnx.ModelProtorrboolr~None) __future__rraloggingr(typingrrr] torch.onnxrrr torch.typesr getLogger__name__rer@rMr}r?rs"  $ 4$g!-1 Q Q QQ Q * Q  Qh & % DBDBDBDB3 DB  DB  DB DBr