Q(h. ^dZddlZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z ddl mZddlmZddlmZddlmZddlZddlZdd lmZmZmZdd lmZdd lmZmZmZmZdd l m!Z!m"Z"m#Z#m$Z$m%Z%edd l&m'Z'ddl(m)Z) ddl*Z* ddl,m-Z- ddl.Z.ddl/m0Z0m1Z1m2Z2m3Z3m4Z4ddl5m6Z6m7Z7m8Z8m9Z9m:Z:mZm;Z;mZ^e"d?e6e"d@e6dAZ_dBZ`edCZadDZbe"d0gdEdFZce"d/eEdGZdedHZeeee"dIgdJe"dKddLgdMZfe"d/eEdNZgee"d/eEdOZhee"d/eie?dPhz e"dQdRdSgdTZjee"d/eDdUZkee"d/e6dVZldWZme"d/e6e"dXdgdYfdZgd[fgd\Znee"d/eEd]Zod^Zpee"d/eDd_Zqd`ZrdaZsdbZtee"d/eEejjdcddeZvGdfdge3ZwdhZxe"d/eCdiZyee"d/eEe"d0gdjdkZzdlZ{dmZ|e$edudnddoZ}dpZ~eEddZejedqDcgc]}dr|z c}ee"d/ee"d&e;ejs q5Lr1c |dzSNr0r<s r2squarerCns a4Kr1ceZdZdZdZy)MyExceptionWithFinickyInitz,An exception class with non trivial __init__cyr.r0)selfabcds r2__init__z#MyExceptionWithFinickyInit.__init__us r1N__name__ __module__ __qualname____doc__rLr0r1r2rErErs 6 r1rEFc<|dk(r|rtddddt|S)NrHrIrJrK)rE ValueError)r<custom_exceptions r2exception_raiserrVys4Av  'sCc :  Hr1c8tjdt)Ng?)timerKeyboardInterruptrBs r2interrupt_raiserrZsJJt r1c|dz|z|zS)zNA module-level function so that it can be spawn with multiprocessing. rAr0)r<r=zs r2fr]s a4!8a<r1cBttjdSNr)typer get_active_backendr0r1r2_active_backend_typerbs ++-a0 11r1cHt||dtdDS)Nn_jobsr9c3FK|]}tt|ywr.r#rC.0is r2 z parallel_func..s : :!r!range) inner_n_jobsr9s r2 parallel_funcrqs) 98< 9:$)!H: r1c"tdkDsJyr_)r"r0r1r2test_cpu_countrss ;??r1c"tdkDsJyr_r$r0r1r2test_effective_n_jobsrvs   !! !r1contextzbackend_n_jobs, expected_n_jobs)rmrmreNr)z positive-intz negative-intNone)idsc|d|5td|k(sJ dddtddk(sJy#1swYxYw)Nr*ryrru)rwbackend_n_jobsexpected_n_jobss r2test_effective_n_jobs_NonersQ ^ 4@ t,???@ 4 (A -- - @@s 8Ar9re)rrArxverbose)rA dctdDcgc] }t|c}t|||dtdDk(sJycc}w)Nrer9rc3FK|]}tt|ywr.rgrir<s r2rkz'test_simple_parallel..s+Qogfoa+rlrorCr!)r9rerr<s r2test_simple_parallelrsP %Qx (!F1I (-Hw-+%(+-, ,, , (sA rAc d}tjd}Gddt}|||d}|t|Dcgc]}t t |c}t }|jD]=}|j|x} s| j\} } |j| | z ?t|dk(sJycc}w)Nrz(Done\s+\d+ out of \d+ \|)ceZdZgZdZy)/test_parallel_pretty_print..ParallelLogc:|jj|yr.)messagesappend)rGmsgs r2_printz6test_parallel_pretty_print..ParallelLog._prints MM  %r1N)rNrOrPrrr0r1r2 ParallelLogrs  &r1ri'rr) recompiler!ror#r]setrsearchspanaddlen) r9ren_taskspatternrexecutorrjlensmessagesrHrIs r2test_parallel_pretty_printrsGjj67G&h& &'5IH U7^ 4jgajm 45 5D$$w' '1 '668DAq HHQUO t9>> 5sC c.|jtjddtjd5}t d|dt d D}|gd k(sJ dddDcgc]5}d t|jvrt|jts|7}}|d vst|tr)d }|Dcgc]}|t|jvs|}}t|dk(sJy#1swYxYwcc}wcc}w)Nname!some_new_name_for_the_main_thread)targetrvalueTrecordrArdc3FK|]}tt|ywr.rgrs r2rkz6test_main_thread_renamed_no_warning..s 6 #$OGFOA 6 rlrm)rrzworker timeout)Nr(z3multi-threaded, use of fork() may lead to deadlocksr) setattrr*current_threadwarningscatch_warningsr!rostrr isinstanceDeprecationWarningrr)r9 monkeypatchwarninforesultsw message_parts r2#test_main_thread_renamed_no_warningrs" '') 1   -$5(!W56 (-a6  )### $ 3qyy> 1199&89 H++z'0M 'N!K|]}d|jdvyw)zbacked parallel loops cannotrN)args)rieachs r2rkz)_assert_warning_nested.. s#'CG.$))A,>'rF)rr simplefilterrqrallrr)r9rpexpectedrrwarnings_are_correctwarnings_have_the_right_lengths r2_assert_warning_nestedrs   -Bh'gLAB$,,a ,H, #&'KS'$ '6H "3x=A;M +(J,J J|)BB-s#BB+B(z%parent_backend,child_backend,expected))r)r(T)r)r)F)r(r(T)r(r)T)r*r(T)r*r)Tctd|fdtdDtd|fdtdD}|dk(rt|sJyt|sJy)NrArdc3NK|]}ttddyw)rFr9rprNr#rri_ child_backends r2rkz0test_nested_parallel_warnings..(s4/  (&'!E  /"%rc3NK|]}ttdyw)rArNr)rirrrs r2rkz0test_nested_parallel_warnings..0s45  (&'!H  5rr*)r!roanyr)parent_backendrrress `` r2test_nested_parallel_warningsrsq/HA~./q / 5(!^ 45q 5 C$3xx3xxr1)r)r(r*cdg}d}tj||f}|j|j|dsJy)NFctjd5}tddtdDdddt t t |dk(|d<y#1swY/xYw)NTrrAryc3FK|]}ttdyw)皙?Nr#rrirs r2rkzPtest_background_thread_parallelism..background_thread..FsEq~wu~c2Erlrr)rrr!roprintr)is_run_parallelrs r2background_threadz=test_background_thread_parallelism..background_threadDsc  $ $D 1 FX HA EE!HE E F c(m ]a/ F Fs #A((A1)rrr)r*Threadstartjoin)r9rrts r2"test_background_thread_parallelismr?sHgO0  18JKAGGIFFH 1  r1cJtd|dtdDy)NrArdc3FK|]}ttdyw{Gz?Nrgrs r2rkznested_loop..Qs'P!(='Prlrnr8s r2 nested_looprPs'HAw''PuQx'PPr1rrcPtd|fdtdDy)NrArdc3HK|]}ttywr.)r#rrs r2rkz#test_nested_loop..Ws#/01 ]+/"rn)rrs `r2test_nested_looprTs'/HA~./5:1X/r1ctr.)rTr8s r2raise_exceptionr\s r1c tt5tdd5}|ttdtt dgddddddy#1swYxYw#1swYyxYw)NrAr)rd)rrTr!r#rr)r s r2)test_nested_loop_with_exception_with_lokyr`su  W Q / W8 *gk*624LGO4LV4TU V WWW W WWWs"A,1A A, A) %A,,A5ctdtddfdtdDjsJy)z1Input is mutable when using the threading backendr)maxsizerAr*rdc3TK|]}tjd!ywrN)r#put)rirqs r2rkz1test_mutate_input_with_threads..js +P!NGAEEN1,=+Ps%(N)rr!rofull)rs@r2test_mutate_input_with_threadsrgs5 aA+HA{++PuQx+PP 668O8r1)rrArmctd}|Dcgc]}t|dc}t|d|Dk(sJycc}w)z.Check the keyword argument processing of pmap. rr=ryc3JK|]}tt|dywrrNr#r]rs r2rkz'test_parallel_kwargs..rs%?  1?!#N)ror]r!)relstr<s r2test_parallel_kwargsrnsO )C" #!Aa1I #'>xv'>?$'?(   #sActd}|Dcgc]}t|d}}td|5}|j}||d|Dk(sJ||d|Dk(sJt"t |t |jusJdddtt jJ|d|Dk(sJtt |jJyycc}w#1swY^xYw) Nrrrrrdc3JK|]}tt|dywrrrs r2rkz3test_parallel_as_context_manager.. =AZWQZQ//=rc3JK|]}tt|dywrrrs r2rkz3test_parallel_as_context_manager..rrc3JK|]}tt|dywrrrs r2rkz3test_parallel_as_context_manager..s 9A++9r)ror]r!_backendr r:)r9rr<rpmanaged_backends r2 test_parallel_as_context_managerrws )C#&'a!q 'H' G , K**1======1====== >/;qzz3JJ JJ K  ~1::&... q9S99 99 9 ~1::&.../( K KsC(AC--C6cGddtttd5tddfdt d Dd d d y #1swYy xYw) z`Check that pmap captures the errors when it is passed an object that cannot be pickled. ceZdZdZy)1test_parallel_pickling..UnpicklableObjectctd)N123) RuntimeErrorrGs r2 __reduce__z.UnpicklableObject.__reduce__s u% %r1N)rNrOrPrr0r1r2UnpicklableObjectr s &r1rzthe task to sendmatchrAr)rdc3RK|]}tt ywr.r#id)rirrs r2rkz)test_parallel_pickling..s%+ 12KGBK)+ ,+ $'rN)objectrrr!ro)rs@r2test_parallel_picklingrsS &F&  %8 9 *6*+ 6;Bi+     s &AA byteorder)<>= max_nbytes1Mcdtjdjdj|djj }t dd|fdtd D}|D]G\}}||k(sJ||jj k(sJtjj|Iy) Nc2||jjfSr.)dtyperrBs r2inspect_byteorderz=test_parallel_byteorder_corruption..inspect_byteorders!''####r1)rArmi4rAr))rer9rc3@K|]}tywr.r#)rirr!r<s r2rkz5test_parallel_byteorder_corruption..s&G*+"!"1%Grm) rarangereshapeviewr rr!rotestingassert_array_equal)rrinitial_np_byteorderresult x_returnedbyteorder_in_workerr!r<s @@r2"test_parallel_byteorder_corruptionr0s $ ! V$))YKr*:;A77,, FXQ: FG/4QxGF,25' '"&::::"j&6&6&@&@@@@ %%a45r1chttd|ddtdDdk(sJy)NrArer9timeoutc3FK|]}ttdyw)gMbP?Nrrs r2rkz0test_parallel_timeout_success..s <*+u%<rlr)rr!ror8s r2test_parallel_timeout_successr6sD ;HAw ;</4Ry<     r1ctt5td|ddtdDdddy#1swYyxYw)NrArr3c3FK|]}ttdywrNrrs r2rkz-test_parallel_timeout_fail..s : #$NGEN2 : rlr)rrr!ror8s r2test_parallel_timeout_failr:sE   97D9: (-b :     %>A sequential return_as generatorgenerator_unorderedc tt5ttd||ddt dDdddttd||ddt dDy#1swY8xYw)NrAr)rer9r=r4c3FK|]}ttdywr9rrs r2rkz.s"R'(r"Rrlrc3FK|]}ttdywrrrs r2rkz.s"M %&NGEN4 M rl)rrlistr!ror9r=s r2)test_parallel_timeout_fail_with_generatorrEs    QHAw)S QR,1"IR    L7iLM */)M    s /A66A?c fttt5td|t ddDcgc]\}}t t ||c}}dddtt5td|dDcgc]}t t|c}dddtd|5}t|jJt|j}tt5|t ddDcgc]\}}t t ||c}}dddt|jJt|j|usJtdDcgc]}t|dc}|dtdDk(sJt|j}tt5|dDcgc]}t t|c}dddt|jJt|j|usJtdDcgc]}t|dc}|d tdDk(s3J|j|j|j|j f dddtjMJtt5td dDcgc]}t t|c}dddtt5td t ddDcgc]\}}t t ||c}}dddtt"5tdd d tdDdddycc}}w#1swYNxYwcc}w#1swYxYwcc}}w#1swYxYwcc}wcc}w#1swYxYwcc}w#1swYWxYwcc}w#1swYxYwcc}}w#1swYxYw#1swYyxYw)NrArd)rr)rrrrrc3JK|]}tt|dywrrrs r2rkz%test_error_capture..%>'(  1"">rc3JK|]}tt|dywrrrs r2rkz%test_error_capture..rHrryr)rerc3JK|]}tt|dyw)T)rUNr#rVrhs r2rkz%test_error_capture..+s# TQ &W% &q4 @ @ Trr2)r rZeroDivisionErrorr!zipr#r>rYrZr:rror] _iteratingn_completed_tasksn_dispatched_tasks _abortingrE)r9r<r=r original_workerss r2test_error_capturerSs  ~ % &  /HAw /585HITQ""1a(I   % &  /HAw /7=>!*)*1->   Q 0# Hx001= ==*8+<+<= )* Tc&&>QRda+'(+Aq1RS T x001= ==x0019II II',Ry1!Aa1I1X>,1"I>6  +8+<+<= )* IG13'"23A6GH Ix001= ==x0019II II',Ry1!Aa1I1X>,1"I>6 ##**++""  9# N8,,-555 % & O HA fM 9(8 9! <M N O ! "VFF@ST1-GH-a3TUV * + %1% T%PR) T   uJ  ?  S T T2 H I I29# # T N O O UVV  sN( N" N(3N:N5 !N:=.my_generator2s1r AH} !9::G &(rArdrWrc3FK|]}tt|ywr.rgrhs r2rkz.test_error_in_task_iterator..; CQogfoa Crl)rXc3FK|]}tt|ywr.rgrhs r2rkz.test_error_in_task_iterator..@r\rlrc3FK|]}tt|ywr.rgrhs r2rkz.test_error_in_task_iterator..Ds DQogfoa Drlr)r!rrT)r9rYrs r2test_error_in_task_iteratorra/s G , E J&> ? D C,*B C C D J&> ? D C,*B C C DJ&> ? E D,*C D D E E E D D  D D E E E EsQCB,CB8+CCC,B5 1C8C =CC CCc,|jd|zy)Nz Consumed %s)r)queueitems r2consumerreGs LL%&r1zbatch_size, expected_queue) Produced 0 Consumed 0 Produced 1 Consumed 1 Produced 2 Consumed 2 Produced 3 Consumed 3 Produced 4 Consumed 4 Produced 5 Consumed 5r) rfrhrjrlrgrirkrmrnrprorqctfd}td||fd|D|k(sJtdk(sJy)z=Test that with only one job, Parallel does act as a iterator.c3^KtdD]}jd|z|ywNr"z Produced %irorrjrcs r2producerz'test_dispatch_one_job..producerx0q A LL* +G *-rre batch_sizer9c3JK|]}tt|ywr.r#re)rir<rcs r2rkz(test_dispatch_one_job..}s%?()%#? # N)rCr!r)r9r{expected_queuerwrcs @r2test_dispatch_one_jobrKsXV FE ?HA*g>?-5Z? N "" " u:  r1cNtj}|jfd}tddd|fd|Dt}|ddk(sJ|d d j d }|d kDsJ|j d }||kDsJt dk(sJy )zRCheck that using pre_dispatch Parallel does indeed dispatch items lazily. c3^KtdD]}jd|z|ywrtrurvs r2rwz/test_dispatch_multiprocessing..producerrxryrArrm)rer{ pre_dispatchr9c3JK|]}ttdyw)rNr})rirrcs r2rkz0test_dispatch_multiprocessing..s'F,-%'Fr~rrfNrz Consumed anyrxrlr)r ManagerrCr!indexr)r9managerrwqueue_contentsfirst_consumption_indexproduced_3_indexrcs @r2test_dispatch_multiprocessingrs jjlG LLNE FHA!!WEF19F%[N !  ,, ,-Ra066~F "R '' '%++L9 5 55 5 u:  r1ctddd5}|dtdD|jjdk(sJ dddy#1swYyxYw)NrAautor*rzc3FK|]}tt|ywr.rrhs r2rkz/test_batching_auto_threading.. .Q+'"+a. .rlrr!rorcompute_batch_size)rs r2test_batching_auto_threadingrsT v{ C4q .%+ ..zz,,.!333444 8AActdd|5}|dtdD|jjdkDsJ dddy#1swYyxYw)NrArrzc3FK|]}tt|ywr.rrhs r2rkz2test_batching_auto_subprocesses..rrlrrr)r9rs r2test_batching_auto_subprocessesrsV vw ?31 .%+ .. zz,,.222 333rctt5tddddtdDdddy#1swYyxYw)zCMake sure that exception raised during dispatch are indeed capturedrAr)rerrc3FK|]}tt|ywr.rKrhs r2rkz*test_exception_dispatch..s#7 -. %G$ %a (7 rlr2N)rrTr!ror0r1r2test_exception_dispatchrsC   6A67 27)7     r;cHtddtdDy)NrAryc3FK|]}tt|ywr.rKrijs r2rkz(nested_function_inner..sG0w/03Grlr2rnrjs r2nested_function_innerrsHAGU2YGGr1cHtddtdDy)NrAryc3FK|]}tt|ywr.)r#rrs r2rkz(nested_function_outer..sLQ5w45a8Lrlr2rnrs r2nested_function_outerrsHAL%)LLr1z'https://github.com/joblib/loky/pull/255reasonchtt5}td|dtdDdddt j |j |j}dj|}d|vsJd|vsJd |vsJt |j tusJy#1swYuxYw) zEnsure errors for nested joblib cases gets propagated We rely on the Python 3 built-in __cause__ system that already report this kind of information to the user. rArdc3FK|]}tt|ywr.)r#rrhs r2rkz1test_nested_exception_dispatch..s#, 23 *G) *1 -, rlr2NrrrV) rrTr!ror r`rtbr)r9excinfo report_linesreports r2test_nested_exception_dispatchrs   w+7+, 7.s 21OGFOA  2rlrc3>K|]\}}|t|k(ywr.)rC)rirjrs r2rkz.test_njobs_converted_to_int..s9$!QqF1I~9r)r!_effective_n_jobsror enumerate)r9rerrs r2test_njobs_converted_to_intrsS 0A   A %% % 2b 2 2C 9)C.9 99 9r1c tdtdtvsJtdtk(sJ td=y#td=wxYw)N test_backend)r'rr r0r1r2test_register_parallel_backendrsE%!.2EF)))'+>>>> ^ $H^ $s ,7 Ac tj}ttk(sJ t dt ddtt k(sJ |t_ttk(sJy#|t_wxYw)Nr*T) make_default)r r/rbr3r'r r)default_backend_origs r2test_overwrite_default_backendr%sw#33  !%A%C CC C8!+x /DSWX#%)9999$8  !%A%C CC C$8 s (A55 BzOnly without multiprocessingcttd5tddtdDdddt d5tdtdDdddy#1swY?xYw#1swYyxYw)Nz)joblib backend '.*' is not available on.*rr)r8c3FK|]}tt|ywr.rgrhs r2rkz2test_backend_no_multiprocessing..4s F!3 Frlrmc3FK|]}tt|ywr.rgrhs r2rkz2test_backend_no_multiprocessing..8s8!?76?1%8rl)r UserWarningr!ror&r0r1r2test_backend_no_multiprocessingr1sy {"M NG  FU1X FFG  (9 8uQx8899 GG99s#A3 !A?3A<?Bc||d5tj\}}|dk(sJtddk(sJt}|jdk(sJ|dk(r0t |t usJt |jt usJ|dk(r0t |tusJt |jtusxJ|dk(r0t |tusJt |jtusCJ|jdr0t |tusJt |jtusJdddy#1swYyxYw)Nrmryr(r)r*test_) r rar$r!rer`rrrr startswithr)rw backend_nameactive_backend active_n_jobsrs r2check_backend_context_managerr;sA a (;(0(C(C(E% !!!"a''' Jxx1}} , ,'+AA AA #'== == V #';6 66 #{2 22 [ ('+;; ;; #'77 77  $ $W -'+>> >> #':: ::#;;;s D*D>>Ermztest_backend_%dc.|tvr|jt|ttt k(sJt |tt k(sJt ddfdtDtt k(sJy)NrAr*rdc3NK|]}|stt|ywr.)r#r)rirIrws r2rkz/test_backend_context_manager..cs+,  /-.w:,r)r setitemrrbr3rr! all_backends_for_context_manager)rr9rws `r2test_backend_context_managerrTshHg/BC  !%A%C CC C!'73 !%A%C CC C,HA{+,1, !%A%C CC Cr1ceZdZdZddZy)ParameterizedParallelBackendz9Pretends to run conncurrently while running sequentially.Nc,| td||_y)Nzparam should not be None)rTparam)rGrs r2rLz%ParameterizedParallelBackend.__init__ps =78 8 r1r.rMr0r1r2rrms Cr1rc8|jtdttt k(sJ|ddd5t j \}}t|tusJ|jdk(sJ|dk(sJt}|jdk(sJ|j|usJ|dtdD}dddtdDcgc] }t|c}k(sJtt k(sJy#1swYHxYwcc}w)N param_backend*rm)rrec3FK|]}tt|ywr.r#rrhs r2rkz=test_parameterized_backend_context_manager..7MGDM!$7rlr)rr rrbr3r rar`rr!rerror)rrwrrrrrjs r2*test_parameterized_backend_context_managerrvs/3OP  !%A%C CC C 1 58(0(C(C(E% N#'CCCC##r)))!!! Jxx1}}zz^+++7eAh778 a11tAw1 11 1 !%A%C CC C882sBD D Dcttk(sJ|tdd5tj\}}t |tusJ|j dk(sJ|dk(sJt}|jdk(sJ|j|usJ|dtdD}dddtdDcgc] }t|c}k(sJttk(sJy#1swYHxYwcc}w)N+)rrryc3FK|]}tt|ywr.rrhs r2rkzFtest_directly_parameterized_backend_context_manager..rrl) rbr3rr rar`rr!rerror)rwrrrrrjs r23test_directly_parameterized_backend_context_managerrs  !%A%C CC C -B7 B8(0(C(C(E% N#'CCCC##r)))!!! Jxx1}}zz^+++7eAh778 a11tAw1 11 1 !%A%C CC C882sBC9D9Dc@tdtjS)Nr)rosgetpidr0r1r2sleep_and_return_pidrs #J 99;r1cttk(sJtjdk(sJtddt dDS)NrrAryc3DK|]}ttywr.)r#rrs r2rkz"get_nested_pids..sP!;g&:;=P )rbrr!rror0r1r2get_nested_pidsrsM  !%5 55 5 : ' ' )Q .. . 81 PuQxP PPr1c"eZdZdZfdZxZS) MyBackendz:Backend to test backward compatibility with older backendsc.tt| dSr_)superrget_nested_backend)rG __class__s r2rzMyBackend.get_nested_backendsY8:1==r1)rNrOrPrQr __classcell__)rs@r2rrsD>>r1rback_compat_backend)r*r)r(rc||5tddtdD}|D]}tt|dk(rJ dddy#1swYyxYw)NrAryc3DK|]}ttywr.)r#rrs r2rkz6test_nested_backend_context_manager..s'Vq(@(@(B'Vrrr)r!rorr)rwr9 pid_groups pid_groups r2#test_nested_backend_context_managerrsc  ,'XQ''VERTI'VV # ,Is9~&!+ ++ ,,,,s?A AA)rArxNcdtdfdtdD|5tdfdtdDdddy#1swYyxYw)Nctt|k(sJt|}tj |k(sJyr.)rbr r$r!r)expected_backend_typeexpected_n_jobs r2check_nested_backendz?test_nested_backend_in_sequential..check_nested_backends@$%2G)HHHH*.9z++-???r1rryc3^K|]$}ttjd&ywr)r#r r/)rirrs r2rkz4test_nested_backend_in_sequential..s,GH%$%h&>&>Bryrc3BK|]}tywr.r%)rirr9rres r2rkz4test_nested_backend_in_sequential..s& ?@ )G( )'6 : srn)r9rerwrs`` @r2!test_nested_backend_in_sequentialr sm@HALQRTI  (  DI"I     s 'A%%A.c||5}|tur|d}|tur|d}j|k(sJ dddy#1swYyxYw)Nr9r)r&r% nesting_level)rw inner_backendexpected_levelctxr9s r2check_nesting_levelrsX  73 o %)nG & &!fG$$666 777s ,?A outer_backendr ct|dtd|fdtdDd5tfdtdDdddy#1swYyxYw)NrrArdc3LK|]}ttdywrr#rrirrwr s r2rkz-test_backend_nesting_level..s(.DE$#$WmQ?.!$rryc3LK|]}ttdywrrrs r2rkz-test_backend_nesting_level..s( HI (G' (- C r)rr!ro)rwrr s` `r2test_backend_nesting_levelrsr 2-HA}-.INr. q )   MRSUY     s $A--A6with_retrieve_callbackTcFddlGfddt}td|d|d5}tdfdt d D|t ur|d j d k(sJ|tur|dj d k(sJdddy#1swYyxYw) Nrc>eZdZdZWZWj dZy))test_retrieval_context..MyBackendrc3>K|xjdz c_dywrzrr s r2retrieval_contextz;test_retrieval_context..MyBackend.retrieval_context s FFaKF rN)rNrOrPrjsupports_retrieve_callbackcontextmanagerr) contextlibrsr2rrs& %;"  " "  # r1r retrievalcFtddt|DS)NrAryc3FK|]}tt|ywr.rrhs r2rkz>test_retrieval_context..nested_call..s!CQ+'"+a.!Crlrn)ns r2 nested_callz+test_retrieval_context..nested_calls!xq!!C%(!CCCr1rAryc3@K|]}t|ywr.r%)rirjr&s r2rkz)test_retrieval_context..sEq/7;/2Er&rr9r)r!rr'r!ror&rjr%)rwrrrr!r&s ` @@r2test_retrieval_contextr(s$k95D  !EE!HEE o %y>##q( (( & &q688q= = !!!s ABB r{)rrxgQ?cdtt5t|dddy#1swYyxYw)Nr{)rrTr!r*s r2test_invalid_batch_sizer+s'  (J'(((s &/z)n_tasks, n_jobs, pre_dispatch, batch_size) )rArArr)rArArerrrArer)irArerr,)rrrer)rrer)r 2 * n_jobsr)rrr)r0rr/rS)r-rr/rc|||d}t|Dcgc] }t|}}tdi|dt|D}||k(sJycc}w)N)rerr{c3FK|]}tt|ywr.rgrhs r2rkz/test_dispatch_race_condition..;s L!3 Lrlr0r)rrerr{paramsrjrrs r2test_dispatch_race_conditionr4%s\( J WF#(>2aq 2H2 h  LU7^ LLG h  3sActj}tdd}|jj d}|j}||k(sJy)NrAr(rdrw)r get_start_methodr!_backend_kwargsget)mp_start_methodrrw start_methods r2test_default_mp_contextr;?sO))+O#45A##I.G++-L ? ** *r1c&|dk(rtjd}tjj d}|j ddtj jtd|fdtdDy)Nr(r,rirArdc3rK|].}ttjj0ywr.)r#rdotT)rirjrHs r2rkzAtest_no_blas_crash_or_freeze_with_subprocesses..^s&'RA133(?'Rs47) r r+rrandom RandomStaterandnr>r?r!ro)r9rngrHs @r2.test_no_blas_crash_or_freeze_with_subprocessesrDHst##..) ))   #C $AFF1accN(HAw''Rq'RRr1afrom joblib import Parallel, delayed def square(x): return x ** 2 backend = "{}" if backend == "spawn": from multiprocessing import get_context backend = get_context(backend) print(Parallel(n_jobs=2, backend=backend)( delayed(square)(i) for i in range(5))) c|dk(r,tjdk7rtjdtj |}t tjd|gddy)Nr(forkzVRequire fork start method to use interactively defined functions with multiprocessing.z-cr\[0, 1, 4, 9, 16\]r4 stdout_regex) r r6pytestskip,UNPICKLABLE_CALLABLE_SCRIPT_TEMPLATE_NO_MAINformatrsys executable)r9codes r22test_parallel_with_interactively_defined_functionsrQqs[ ##(;(;(=(G  .  8 > >w GD t$b?Tr1aimport sys # Make sure that joblib is importable in the subprocess launching this # script. This is needed in case we run the tests from the joblib root # folder without having installed joblib sys.path.insert(0, {joblib_root_folder!r}) from joblib import Parallel, delayed def run(f, x): return f(x) {define_func} if __name__ == "__main__": backend = "{backend}" if backend == "spawn": from multiprocessing import get_context backend = get_context(backend) callable_position = "{callable_position}" if callable_position == "delayed": print(Parallel(n_jobs=2, backend=backend)( delayed(square)(i) for i in range(5))) elif callable_position == "args": print(Parallel(n_jobs=2, backend=backend)( delayed(run)(square, i) for i in range(5))) else: print(Parallel(n_jobs=2, backend=backend)( delayed(run)(f=square, x=i) for i in range(5))) z!def square(x): return x ** 2 zcdef gen_square(): def square(x): return x ** 2 return square square = gen_square() zsquare = lambda x: x ** 2 define_funccallable_position)r#rkwargsc |dvr1|tk7stjdk(rtjdt j |||tjjtjjtj}|jd}|j|ttj|j gddy) N)r(r,win32zNot picklable with pickle)rRr9rSjoblib_root_folderzunpicklable_func_script.pyrrGrH) SQUARE_MAINrNplatformrJrK)UNPICKLABLE_CALLABLE_SCRIPT_TEMPLATE_MAINrMrpathdirnamejoblib__file__rwriterrOstrpath)r9rRrStmpdirrP code_files r20test_parallel_with_unpicklable_functions_in_argsrcs..{"cllg&= /0 4 ; ;+77??277??6??+KL < D  89I OOD **+*r1aimport sys import faulthandler # Make sure that joblib is importable in the subprocess launching this # script. This is needed in case we run the tests from the joblib root # folder without having installed joblib sys.path.insert(0, {joblib_root_folder!r}) from joblib import Parallel, delayed from functools import partial class MyClass: '''Class defined in the __main__ namespace''' def __init__(self, value): self.value = value def square(x, ignored=None, ignored2=None): '''Function defined in the __main__ namespace''' return x.value ** 2 square2 = partial(square, ignored2='something') # Here, we do not need the `if __name__ == "__main__":` safeguard when # using the default `loky` backend (even on Windows). # To make debugging easier faulthandler.dump_traceback_later(30, exit=True) # The following baroque function call is meant to check that joblib # introspection rightfully uses cloudpickle instead of the (faster) pickle # module of the standard library when necessary. In particular cloudpickle is # necessary for functions and instances of classes interactively defined in the # __main__ module. print(Parallel(backend="loky", n_jobs=2)( delayed(square2)(MyClass(i), ignored=[dict(a=MyClass(1))]) for i in range(5) )) )rWc|jd}|jtttj |j gddy)Nz(joblib_interactively_defined_function.pyrG)rIr4)rr_5INTERACTIVE_DEFINED_FUNCTION_AND_CLASS_SCRIPT_CONTENTrrNrOr`rascripts r27test_parallel_with_interactively_defined_functions_lokyrhs> [[C DF LLFG (*r1aimport sys # Make sure that joblib is importable in the subprocess launching this # script. This is needed in case we run the tests from the joblib root # folder without having installed joblib sys.path.insert(0, {joblib_root_folder!r}) from joblib import Parallel, delayed, hash import multiprocessing as mp mp.util.log_to_stderr(5) class MyList(list): '''MyList is interactively defined by MyList.append is a built-in''' def __hash__(self): # XXX: workaround limitation in cloudpickle return hash(self).__hash__() l = MyList() print(Parallel(backend="loky", n_jobs=2)( delayed(l.append)(i) for i in range(3) )) c|jd}|jtttj |j gdddy)Nz)joblib_interactive_bound_method_script.pyz\[None, None, None\] LokyProcess)rI stderr_regexr4)rr_9INTERACTIVELY_DEFINED_SUBCLASS_WITH_METHOD_SCRIPT_CONTENTrrNrOr`rfs r2:test_parallel_with_interactively_defined_bound_method_lokyrns? [[D EF LLJK (,# r1cHtg}td|gk(sJy)NrAry)iterr!)exhausted_iterators r2%test_parallel_with_exhausted_iteratorrr)s'b 81 0 1R 77 7r1ct|tjstdt ||j S)Nz#Expected np.memmap instance, got %r)rrmemmap TypeErrorr`copy)rHs r2 check_memmaprw.s- a #=tAwGG 668Or1cd}tdd|d|dD}t||t|D]%\}}tjj ||'tdd|d|dD}t||t|D]%\}}tjj ||'y) Nc3Kt|D],}tjdtj|z.yw)Nrr )roronesfloat32)r%rjs r2generate_arrayszBtest_auto_memmap_on_arrays_from_generator..generate_arrays<s3q 4A''"BJJ/!3 3 4s<>rArrerr9c3FK|]}tt|ywr.r#rwrirHs r2rkz.B$@%& a @rlrrc3FK|]}tt|ywr.rrs r2rkz.Krrl)r!rMrrr*r+)r9r}rr-rs r2)test_auto_memmap_on_arrays_from_generatorr4s4 @haAw?@*9#*>@G W)FG8 %%h78 @haAw?@*9#*>@G W)FG8 %%h78r1c|Sr.r0)args r2identityrRs Jr1c|jdj}tj}t j |dt j |dg}t||t|dtdfdd D\}td tjsJd j|kDsJtjj||y) Nz test.mmapuint8rzr) mmap_moderAryc3HK|]}ttywr.)r#r)rirrts r2rkz.test_memmap_with_big_offset..^s"J#478#4V#<"Jrr`r)rr`mmapALLOCATIONGRANULARITYrzerosr{r r r!rrtoffsetr*r+)rafnamesizeobjr-rts @r2test_memmap_with_big_offsetrVs KK $ , ,E  % %D 88D ("''$g*F GCe %3 'F"""Jc"JJIV fQi ++ + !9  d "" "JJ!!#v.r1c6tjd5}tdddtdDdddt dk(sJ|d}t |j tsJt|j dk(sJy#1swYUxYw) NTrr)rer4c3FK|]}tt|ywr.rgrhs r2rkzFtest_warning_about_timeout_not_supported_by_backend..fs%LQogfoa&8%Lrl2rzThe backend class 'SequentialBackend' does not support timeout. You have set 'timeout=1' in Parallel but the 'timeout' parameter will not be used.) rrr!rorrrrr)rrs r23test_warning_about_timeout_not_supported_by_backendrds   -M%1%%L%)%LLM x=A   A aii -- - qyy>    MMs $BBc|||<|Sr.r0) input_listrrs r2set_list_valuerqsJu Lr1)rrArcdgdzt|ddfdtdD}t|}tdt |DsJy)Nrrr>r*rer=r9c3LK|]}tt||ywr.)r#r)rirjrs r2rkzPtest_parallel_return_order_with_return_as_generator_parameter..|s)Q67 Aq1Qrc3,K|] \}}||k(ywr.r0rivrs r2rkzPtest_parallel_return_order_with_return_as_generator_parameter..s:$!QqAv:)r!rorCrrM)rer-rs @r2=test_parallel_return_order_with_return_as_generator_parameterrvs^ qJ PXV{K PQ;@8QF &\F :#j&"9: :: :r1c2|r tdt|S)Nr2)rr)edelays r2_sqrt_with_delayrs  b 7Nr1ct|d|dtdDtfdtdD}dgttddz}t d t ||DsJy) Nr?rc3TK|] }tt|dz|dk("yw)rArN)r#rrhs r2rkzK_test_parallel_unordered_generator_returns_fastest_first..s/W67! !!Q$a1WrZrc34K|]}tywr.)next)rirr-s r2rkzK_test_parallel_unordered_generator_returns_fastest_first..s=qd6l=s rrAc3,K|] \}}||k(ywr.r0rs r2rkzK_test_parallel_unordered_generator_returns_fastest_first..sS$!QqAvSr)r!rosortedrCrrM)r9requickly_returnedexpected_quickly_returnedr-s @r28_test_parallel_unordered_generator_returns_fastest_firstrs~WXV/Dg VW;@9WF=E!H==!"d5B<&8 8 S#&?AQ"RS SS Sr1ct||yr.)rr9res r27test_parallel_unordered_generator_returns_fastest_firstrs=WfMr1)rrArrxcdgdgdzz}tt5tj}t||d|Ddddtjz }|dksJy#1swY(xYw)NrHrrrdc3ZK|]#}ttj|%ywr.r#rXrrhs r2rkz%test_abort_backend..s!0XA1D1DQ1G0Xs)+rV)rrurXr!)rer9delayst_startdts r2test_abort_backendrswUbTCZ F  Y))+000XQW0XXY w B 7N7 YYs /A00A9cXtjtdt}d|d<|S)NgArzFr)rr{intbool)rr-s r2get_large_objectrs$ WWS\ .FF1I Mr1ct|||5}|dtdD}t|t|~dddy#1swYyxYw)N)rer9r=c3FK|]}tt|ywr.)r#rrhs r2rkz0_test_deadlock_with_generator..sJ13'"23A6Jrlr)r!ror)r9r=rer r-s r2_test_deadlock_with_generatorrsL I F(Jb JJ V  V  s 0AAct|||yr.)r)r9r=res r2test_deadlock_with_generatorrs "'9f=r1c8ttd5t|||}|dtdD}t j}|dtdD}dddt jz dksJd ~y#1swY,xYw) N)This Parallel instance is already runningrrDc3FK|]}ttdywrrrs r2rkz/test_multiple_generator_call..s:1^WU^A&:rlrc3FK|]}tt|ywr.rrhs r2rkz/test_multiple_generator_call..s;1  A;rlrrA[The error should be raised immediately when submitting a new task but it took more than 2s.)rr r!rorX)r9r=rer grgen2s r2test_multiple_generator_callrs  $O P<FGyI :b : :))+;c ;; < 99; 1 $ $ $ <.s;A^WU^B';rlrrrc3FK|]}tt|ywr.rrhs r2rkz7test_multiple_generator_call_managed..s=Q+'"+a.=rlrrAr)r!rorXrr )r9r=rer rrg2s r2$test_multiple_generator_call_managedrs &'Y ?  8 ;r; ;))+ L(S T >=%*==B >yy{W$q(  ( (    > >   s#>B& B&)B&B# B&&B0 return_as_1 return_as_2c tt|||dtdD}t|||dtddD}|dk(r t|}|dk(r t|}tdt |tdDsJtdt |tddDsJy) NrDc3LK|]}tt|dzywrANrrhs r2rkz9test_multiple_generator_call_separated..s&A !  adA"$rc3LK|]}tt|dzywrrrhs r2rkz9test_multiple_generator_call_separated..s&B !  adBrrVr?c3,K|] \}}||k(ywr.r0rirrjs r2rkz9test_multiple_generator_call_separated..s8FCsax8rc3,K|] \}}||k(ywr.r0rs r2rkz9test_multiple_generator_call_separated..s=FCsax=r)r!rorrrM)r9rrrerrs r2&test_multiple_generator_call_separatedrs AK@A%*2YA A B&'[ AB%*2r]B B++ 1I++ BZ 8c!U2Y&78 88 8 =c"eBm&<= == =r1zbackend, error))r)T)r*F)r<Fc >|dk(rttjdtd||}|dt dD}t j |d}|rttd n t}|5tj}td||d t dd D}|d k(r t|}td t|t dd DsJ dddtjz dksJd} |jr1| dkr,| dz } tjd|jr| dkr,tj|z dksJ|j dk7r|j"sJyy#1swYxYw)Nr)zRequires multiprocessingrArDc3FK|]}ttdywr9rrhs r2rkz.&s7#7rlrctdS)NzGenerator collected)rr0r1r2z;test_multiple_generator_call_separated_gc..'s u-B'Cr1z The executor underlying Parallelrc3LK|]}tt|dzywrrrhs r2rkz.3s&@ $%MGDM!Q$ @ rrVr?c3,K|] \}}||k(ywr.r0rs r2rkz.:s@Q3!8@rrrrmrr)r rJrKr!roweakreffinalizerr rrXrrrMaliverrrQ) r9rrerrorr rg_wrrrretrys r2)test_multiple_generator_call_separated_gcrs 6  ./7kBH7U2Y77A   AC DD  |#EF ] A ))+ ?HQ; ?@ ).r2@   / /q A@#ar2*?@@@@ A 99; 1 $$ $ E **   3 ** 99; 1 $$ $!x!!#q(!!!!)1 A As =A2FFcz|j}tdd||5}|dtjjdgdzDt t j |dkDsJ dddtdD]$}t j |sntd&td tdd| }|d tjjdgdzDtdD]$}t j |sytd&td #1swYxYw) NrAr)rerr9 temp_folderc3FK|]}tt|ywr.rrs r2rkz(test_memmapping_leaks..Vs Gq ',  " Grlrrrrz/temporary directory of Parallel was not removedr~c3FK|]}tt|ywr.rrs r2rkz(test_memmapping_leaks..fsC1glACrl) r`r!rr@rrlistdirrorAssertionError)r9rarrs r2test_memmapping_leaksrKs! ^^F q'v N+RS GRYY-=-=b-A,BQ,F GG2::f%&*** +3ZPzz&!  c P NOO a9AC)9)9")=(>(BCC 3ZPzz&!  c P NOO1++s AD11D:)Nr)r*ctd|dtdD}|tdDcgc]}|dz c}k(sJycc}w)NrArdc3@K|]}td|yw)c |dzSr@r0rBs r2rz2test_lambda_expression...vs !Q$r1Nr%rhs r2rkz)test_lambda_expression..us"2'("2srrn)r9rrjs r2test_lambda_expressionrpsN 2ha12,1"I2G U2Y/q!t/ // //s Acd}d}d|z td||}|fdt|D|jj|jjk(sJ|jj |jj k(sJ|fdt|D|jj|jjk(sJ|jj |jj k(sJy) zCTest that a parallel backend correctly resets its batch statistics.rAig@r)rrer9c3\K|]#}ttj%ywr.rrirj task_times r2rkz6test_backend_batch_statistics_reset..">gdjj)$>),c3\K|]#}ttj%ywr.rrs r2rkz6test_backend_batch_statistics_reset..rrN)r!ror_effective_batch_size_DEFAULT_EFFECTIVE_BATCH_SIZE_smoothed_batch_duration _DEFAULT_SMOOTHED_BATCH_DURATION)r9ren_inputsrrs @r2#test_backend_batch_statistics_resetr{sFHhIFGeHo>> :: + +qzz/W/W WW W ++ :: 6 6 7 7>eHo>> :: + +qzz/W/W WW W ++ :: 6 6 7 7r1cdD]}tt|jtusJt|d}t|jtusJt|d}t|jt usJt|d}t|jturJtdd d }t|jt usJ|d d5td }t|jt usJ|j dk(sJ ddd|d d5td d}t|jt usJ|j d k(sJ ddd|d d5td }t|jtusJ|j dk(sJ ddd|d d5td d}t|jtusJ|j d k(sJ dddy#1swYxYw#1swYxYw#1swYqxYw#1swYyxYw)N)rrArxrythreadsreprefer processes sharedmemrerequirerAr))rer9rrrmrr)r`r!rr3rrre)rwrers r2$test_backend_hinting_and_constraintsr s 4HF+4459U9WWWW F9 5AJJ#3333 F; 7AJJ;... FK 8AJJ#3333 4 6).MyCustomThreadingBackendTcyr.r0r s r2 apply_asynczgtest_backend_hinting_and_constraints_with_custom_backends..MyCustomThreadingBackend.apply_async r1c|Sr.r0rs r2r$zltest_backend_hinting_and_constraints_with_custom_backends..MyCustomThreadingBackend.effective_n_jobsMr1NrNrOrPsupports_sharedmem use_threadsrr$r0r1r2MyCustomThreadingBackendrs!   r1rrArrrrc eZdZdZdZdZdZy)\test_backend_hinting_and_constraints_with_custom_backends..MyCustomProcessingBackendFcyr.r0r s r2rzhtest_backend_hinting_and_constraints_with_custom_backends..MyCustomProcessingBackend.apply_asyncrr1c|Sr.r0rs r2r$zmtest_backend_hinting_and_constraints_with_custom_backends..MyCustomProcessingBackend.effective_n_jobsrr1Nrr0r1r2MyCustomProcessingBackendrs"   r1rrr)rerrzUsing ThreadingBackend as joblib backend instead of MyCustomProcessingBackend as the latter does not provide shared memory semantics.r9r) rr!r`r readouterrrstriprrT)capsysrwrrrouterrrs r29test_backend_hinting_and_constraints_with_custom_backendsr#s#6 )+ ,< Ak 2AJJ#;;;; A{ 3AJJ#;;;; <$7 *, - Ak 2AJJ#<<<<$$&Sbyybyy A{B ?AJJ#3333$$&S 8  yy{h&&&byy%(  K24kJKKK<<"(KKs%A E'B$E3 E?'E03E<?Fctt5tddddtt5tddddtt5tdddddtKtt5tdddddtt5td ddddyy#1swYxYw#1swYxYw#1swYsxYw#1swYTxYw#1swYyxYw) Ninvalidr r rr)rrr)rr()rrTr!r r0r1r2,test_invalid_backend_hinting_and_constraintsr&s  # "#  $#$  :  [9:  ~J  : V[ 9 : J  E . D E E ##$$:: : : E Es: C C CC$(C0C  CC!$C-0C9c td5}t|jj|jjfg}dk(r |cdddS|fdt dD}||dzcdddS#1swYyxYw)zCPerform nested parallel calls and introspect the backend on the wayrAryrNc3RK|]}ttdddz i yw)limitrNr0)r#_recursive_backend_info)rirjrTr)s r2rkz*_recursive_backend_info..s2  -G+ , G519 G G rr)r!r`rrNr ro)r)rTr this_levelrs`` r2r*r*s  'qAJJ'00!**2J2JKL A:'' 1X  GAJ&'''s=B"BB c||d5t}ddd|jdz}|dfdddg}|k(sJy#1swY,xYw)NrAryBackendr)rr)rrA)r*title)rwr9backend_types_and_levelstop_level_backend_typeexpected_types_and_levelss r2test_nested_parallelism_limitr2si  #=#:#< =%]]_y8 #  ! $'@ @@ @==s AA cBtdtdDS)z6A horrible function that does recursive parallel callsc3DK|]}ttywr.)r#_recursive_parallelrhs r2rkz&_recursive_parallel..0sG2g124GrrArn) nesting_limits r2r5r5.s 8:GeAhG GGr1c>||d5tt5}tddddddj}|dk(r-ddlm}t ||tfrtjdt |tsJy#1swYbxYw#1swYfxYw)NrAryr)r)TerminatedWorkerErrorz1Loky worker crash when serializing RecursionError) r BaseExceptionr5r&joblib.externals.loky.process_executorr8rrrJxfailRecursionError)rwr9rexcr8s r2test_thread_bomb_mitigationr>3s  #" M " "g  ! "" --C& Q c1=A B LLL M c> ** *# " """s!B BBB BBcri}dD]$}tjj|||<&|tdfS)N)OMP_NUM_THREADSOPENBLAS_NUM_THREADSMKL_NUM_THREADSVECLIB_MAXIMUM_THREADSNUMEXPR_NUM_THREADSNUMBA_NUM_THREADS ENABLE_IPCr)renvironr8r)env_varsvars r2_run_parallel_sumrJOsBH , s+  , \#& &&r1zNeed OpenMP helper compiledc*td|dtdD}ttdzd}|D]T\}}||k(sJ|j D]5\}}|j dr|t |k(r&J|dk(sJ|dk(r5JVy)NrArdc3DK|]}ttywr.)r#rJrs r2rkz-test_parallel_thread_limit..as!2)*"!"$2rr_THREADSrF1)r!rorr"itemsendswithr)r9rexpected_num_threadsworker_env_varsomp_num_threadsrrs r2test_parallel_thread_limitrT^s2ha12.3Ah2Gy{a/3,3$("6666*002 $KD%}}Z($8 9999|+++|#|  $$r1zThis test requires daskc`ttd5|ddddy#1swYyxYw)NzPlease install daskrdask)rrT)rws r2)test_dask_backend_when_dask_not_installedrWos+  "7 8s $-cGddt}d}||5tjt|5t ddt dDddddddy#1swYxYw#1swYyxYw)Nc eZdZdZddZdZy)3test_zero_worker_backend..ZeroWorkerBackendcyr_r0)rGrrTs r2rz=test_zero_worker_backend..ZeroWorkerBackend.configure{r1Nctd)NzNo worker availabler)rGfunccallbacks r2rz?test_zero_worker_backend..ZeroWorkerBackend.apply_async~s45 5r1cyr_r0rs r2r$zDtest_zero_worker_backend..ZeroWorkerBackend.effective_n_jobsr\r1r.)rNrOrPrrr$r0r1r2ZeroWorkerBackendrZzs  6 r1raz&ZeroWorkerBackend has no active workerrrAryc3FK|]}tt|ywr.rrhs r2rkz+test_zero_worker_backend..s@!{wr{1~@rl)rrJrr r!ro)rwra expected_msgs r2test_zero_worker_backendrdvs,.check_globalss!!r1rAryc3>K|]}tywr.r%rirjrhs r2rkz.!1%&  1z changed valuec3>K|]}tywr.r%rjs r2rkz.rkrl)rgr!ror)workers_global_variablerhs @r2)test_globals_update_at_each_parallel_callros *" ?. .. .0ha01*/(1 & ',<+= == =) ?o -- -0ha01*/(1 & 'O+< << rJ importorskipthreadpool_info)rrHrqs r2_check_numpy_threadpool_limitsrusI S!A BFF1aL''8M  ( ( **r1ch|D]}|d|dk(s|dcStdj|)Nfilepath num_threadsz,An unexpected module was loaded in child: {})rTrM) child_module parent_info parent_modules r2_parent_max_num_threads_forr|sK$0  $ Z(@ @ / /0 7>>|L r1cf|D],}|D]%}t||}t|||h}|d|vr%J.y)Nrx)r|min) workers_inforzrxchild_threadpool_inforyparent_max_num_threadsrs r2check_child_num_threadsrsZ".;1 ;L%@k& "K)?@+NH .(: ::  ;;r1)rArrrxc t}t|dk(rtjdt d|dt dD}t |}|dk(r |dd }ntt|zd}t|||y) Nr&Need a version of numpy linked to BLASrr)rc3DK|]}ttywr.r#rurhs r2rkz;test_threadpool_limitation_in_child_loky..s#G67/./1GrrArrx rurrJrKr!ror$rr"r)rerzworkers_threadpool_infosexpected_child_num_threadss r2(test_threadpool_limitation_in_child_lokyrs12K ;1 CDFxvFG;@8G f %F {%0^M%B"%()>%B" +/Ir1inner_max_num_threads)rrArNcft}t|dk(rtjd|d|5t |dt dD}dddt |}|d k(r |dd }n|tt|zd }n|}t||y#1swYLxYw) Nrrrr)rryc3DK|]}ttywr.rrhs r2rkz>test_threadpool_limitation_in_child_context..s!; :; 3G2 3 5; rrArrxr)rwrerrzrrs r2+test_threadpool_limitation_in_child_contextrs12K ;1 CD /D E #:86#:; ?DQx; $   f %F {%0^M%B"  &%()>%B"%:" +/I  s #B''B0var_name)rArBr@ct|dk(rtjdtdj dt j j} dt j <t|fdtd D}|ddgk(sJ|d d 5t|fd td D}ddd|d d gk(sJ |t j =y|t j <y#1swY;xYw#|t j =w|t j <wxYw)NrzSkip test when n_jobs == 1Treusec@tjj|Sr.)rrGr8)rs r2_get_envz>test_threadpool_limitation_in_child_override.._get_envszz~~h''r14ryc3@K|]}tywr.r%rirjrrs r2rkz?test_threadpool_limitation_in_child_override.."s)X!*;'(*;H*E)Xr&rAr)rc3@K|]}tywr.r%rs r2rkz?test_threadpool_limitation_in_child_override..&s#.01!!(+.r&rN) r$rJrKrshutdownrrGr8r!ro)rwreroriginal_var_valuerrs ` @r2,test_threadpool_limitation_in_child_overrider s'1$ 01%..0(16" 8)(&))XuUVx)XX3*$$$ V1 5 -hf-.5:1X.G 3*$$$  % 8$#5BJJx     % 8$#5BJJx s%#A D.&DDDD)E)rArrxcd}||td}tdD]}||td}||k(rJy)Ncdtdt|fdtdDy)Nrryc3HK|]}ttywr.)r#sum)rirjr<s r2rkzAtest_loky_reuse_workers..parallel_call..:sCA  QCr)ror!)rer<s @r2 parallel_callz.test_loky_reuse_workers..parallel_call8s% "ICrCCr1Trr)rro)rerfirst_executorrrs r2test_loky_reuse_workersr2sQ D &*6N2Y*f(t4>)))*r1c2d|tj<y)N initialized)rr)statuss r2_set_initializedrHs'F299;r1cltj}|j|d}|dvs Jd||syd||<tjdz}t |j Dcgc] \}}|dk(s |c}}}tj|kri||krdtj dt |j Dcgc] \}}|dk(s |c}}}tj|kr||krdtj|k\r tdScc}}wcc}}w)N)rstartedz2worker should have been in initialized state, got rr2rz-Waited more than 30s to start all the workers)rrr8rXrrOrr)rre wait_workerspidstatedeadliner n_starteds r2 _check_statusrLs  ))+C JJsD !E . . LaQ)^LM  ))+ Y%7 yy{hJKK JIMs, D* :D*  D0 D0 ctj}|j||tf5t fdt dDdddy#1swYyxYw)Nr9re initializerinitargsc3JK|]}ttywr.r#rrirjrers r2rkz+test_initializer_context..ss Na)7=)&&9Nr~r)r rdictrr!ro)rer9rwrrs` @r2test_initializer_contextrcsd jjlG \\^F $  O  N5:NNOOOs $A%%A.ctj}|jt|tffdt dDy)Nrc3JK|]}ttywr.rrs r2rkz,test_initializer_parallel..s AgmVV,Ar~r)r rrr!rro)rer9rrs` @r2test_initializer_parallelrvsJjjlG \\^FH$  BeCjA Br1c@d}tj}|jt}t |D]I}t dt ffdt D}|jt|}Kt|k(sJdy)Nrmr)rc3NK|]}ttdywT)rNrrs r2rkz*test_initializer_reused..,  #GM "66 E E rz=The workers should be reused when the initializer is the same) r rrrror!runionrre n_repetitionsrpidsrjrrs` @r2test_initializer_reusedrs MjjlG \\^F 5D = ! ( ((Y   6]  zz#g,' ( t9 G r1cFd}tj}t}t|D]Y}|j t dt ffdtD}|jt|}[t||zk(sJdy)Nrmr)rc3NK|]}ttdywrrrs r2rkz.test_initializer_not_reused..rrzFThe workers should not be reused when the initializer arguments change) r rrrorr!rrrrs` @r2test_initializer_not_reusedrs MjjlG 5D = ! ( ((Y   6]  zz#g,' ( t9 . .P .r1)F)rr)rmr.)rQrrrrNr*rXrrr!rmathrr(rpicklerr tracebackr rJr]r r r joblib._multiprocessing_helpersr joblib.test.commonrrrrjoblib.testingrrrrrjoblib.externals.lokyrrcrposix ImportError _openmp_test_helper.parallel_sumr distributedjoblib._parallel_backendsrrrrrjoblib.parallelr r!r"r#r$r%r&r'rvRETURN_GENERATOR_BACKENDSpoprkeysALL_VALID_BACKENDSPROCESS_BACKENDSPARALLEL_BACKENDShasattrrr+r3r:r>rC ExceptionrErVrZr]rbrqrsrvmarkrrrrrrrrrrrrrrrr0r6r:rrErSrarerrrrrrrr;rrrrrrrrrrextendrorrrrrr_parallel_backendsrrr rrr(r+r4r;rDrLrQrZrX SQUARE_LOCAL SQUARE_LAMBDArcrMr[r\r^rerhrmrnrrrwrrrrrrrrrrrrrrrrrrrrr r#r&r*r2r5no_coverr>rJrTrWrdrorur|rrrrrrrrrrr) backend_strrjs00r2rsu  "( & ''. UT>;>   *HMMO/6Vf]X]]_55(K;,x ,.KK:)62$ }4 2}nbnnW56.I    2" Y*:;<% b""- . :0 . =. Y*+ X~& Y %,&',,  Y*+ X1v,( Y*+','T0 +  2 Y@A B Q _h' x()( WW  Xy!" Y)*/+/:      [/* \At9%5&+ 5$ Y)* +  Y)* +  Y56,GH [;(=>?@I$ Y()C *C L Y!E"E,' Y!     $   %$'P Q'"R   Y)*+<4 Y()3*3 HM Y)*CD-E+-, +   Y*+H,H$ Y)* X}%:&+:% D$=>9?9;*$5Q#7 ''a(Q1):Q)>(QR Y89 Y)?;<D=:D,#4 Y*:;<D=D& Y*:;<D=D( Q>))55>/; YWX Y*:;<,=Y, X}% Y)* Y*:;< =+& 07 _/0 _/0 Y*:;<  =11   Y*:;< %e}5!6=!: \=)(*(  /  ! ++  Y()S* S, 0,  Y() * -)@    Y("*B7)LM ][, FG "?@AHN,(9P Fbggoobggoofoo.NOFPQ6V  =, Fbggoobggoofoo.NOFP-:28    Y()8* 86  /  /  9- ;. ;"Aq6* Y56,GHNI + N Y*+ X~&',  Y12 [;(=>? X~&>'@3 > Y12 [;(=>? X~& '@3 & Y12 [;(=>? X~& '@3 & Y12 ][*?@A ][*?@A X~&>'BB3>&  ][*?@A ][*?@A)"BB)"X  Y()P* PD  rz{#7R00 Y()*, Y*:;<+=+\ Y*:;<3K=3KlE* ' Y-. Y*:;< A=/ AH  Y*:;< Y"*+6;:OQ+R=+2 ' Y2>$vG %BC $DH $ Y*:;< 4(ABC=  Y*:;<A=A&=H + ;  X~&' 0  $o6 X2w Y*:;<= 7 6 X2w ZWX Y*:;< 6=Y  6F Xz"*#*((. X1v Y() Y*:;< O=* O X1v Y() B* BAq6*+0Aq6*+mC E L K6L~)RsBk; l l?l%? l*;ll lll"!l"