o ~_L;@sdZddlmZddlmZddlmZddlmZddlmZddlZddl m Z dd l m Z dd l m Z Gd d d eZ Gd ddeZGdddeZGdddeZdS)z=Define some lazy data structures and functions acting on them)print_function)map)next)range)objectN)log)robust)utilc@seZdZdZeddZeddZeddZedd Zed d Z ed d Z edddfddZ eddZ eddZ eddZeddZeddZedddZdS) Iterz:Hold static methods for the manipulation of lazy iteratorsccs|D] }||r |VqdS)z5Like filter in a lazy functional programming languageN) predicateiteratorir r 0/usr/lib/python3/dist-packages/duplicity/lazy.pyfilter)s z Iter.filterccs|D]}||VqdS)z2Like map in a lazy functional programming languageNr functionr rr r rr0s zIter.mapcCs|D]}||qdS)z(Run function on each element in iteratorNr rr r rforeach6s z Iter.foreachcg |D] }|D]}|VqqdS)zLazily concatenate iteratorsNr )itersiterrr r rcat< zIter.catccr)z6Lazily concatenate iterators, iterated by big iteratorNr ) iter_of_itersrrr r rcat2Crz Iter.cat2cCs|D]}dSdS)zTrue if iterator has length 0Nr rrr r remptyJsz Iter.emptyNcCs||kSNr )xyr r rRsz Iter.c Cs|D]/}zt|}Wnty|rtd|fYdSw|||s1|r.td||fdSqzt|}Wn tyBYdSw|rLtd|fdS)zxTrue if iterator 1 has same elements as iterator 2 Use equality operator, or == if it is unspecified. zEnd when i1 = %sNz%s not equal to %srzEnd when i2 = %s)r StopIterationprint)iter1iter2verboseoperatori1i2r r requalQs*     z Iter.equalcCsd}|D]}|r |Sq|S)z:True if any element in iterator is true. Short circuitingNr rr r rOrk zIter.OrcCsd}|D]}|s |Sq|S)z.cSr7rr r r r rr!r8Ncs|dkr8z dtWnty(kr&ds&dd<twtD] }|d7<q-|}|d8<t}|dvrg||dksZJ|d|d=|S)z-Return the next element requested by fork_numr9rNrr6)insertrr"rr/)fork_numr return_valblen)buffercalled_closing_func closing_func final_func forkpositionr num_of_forksstarting_forkpositionr rget_nexts*      z Iter.multiplex..get_nextc3s. z|}Wn tyYdSw|Vqr)r")r;ret)rEr r make_iterators  z%Iter.multiplex..make_iterator)IterMultiplex2yieldayieldbtuplerlistr)rrCrAr@im2rGr ) r>r?r@rArBrErrCrDr multiplexs    zIter.multiplex)NN)__name__ __module__ __qualname____doc__ staticmethodrrrrrrr*r+r-r/r0r5rNr r r rr &s8            r c@s(eZdZdZddZddZddZdS) rHzMultiplex an iterator into 2 parts This is a special optimized case of the Iter.multiplex function, used when there is no closing_func or final_func, and we only want to split it into 2. By profiling, this is a time sensitive class. cCsd|_g|_||_dS)Nr) a_leading_byr>r)selfrr r r__init__s zIterMultiplex2.__init__ccsj|j|j}} |jdkr%zt|}Wn tyYdSw||n|d}|jd7_|Vq )zReturn first iteratorrrNr>rrTrr"appendpoprUbufrelemr r rrI     zIterMultiplex2.yieldaccsj|j|j}} |jdkr%zt|}Wn tyYdSw||n|d}|jd8_|Vq )zReturn second iteratorrrNrWrZr r rrJr]zIterMultiplex2.yieldbN)rOrPrQrRrVrIrJr r r rrHs  rHc@s@eZdZdZddZddZddZdd Zd d Zd d Z dS)IterTreeReduceraHTree style reducer object for iterator - stolen from rdiff-backup The indicies of a RORPIter form a tree type structure. This class can be used on each element of an iter in sequence and the result will be as if the corresponding tree was reduced. This tries to bridge the gap between the tree nature of directories, and the iterator nature of the connection between hosts and the temporal order in which the files are processed. This will usually be used by subclassing ITRBranch below and then call the initializer below with the new class. cCs*||_||_d|_|||_|jg|_dS)zITR initializerN) branch_class branch_argsindex root_branchbranches)rUr_r`r r rrVs  zIterTreeReducer.__init__cCsV|j} |d}|j}||dt|kr(||d=|s dS|d|ndSq)aRun Finish() on all branches index has passed When we pass out of a branch, delete it and process it with the parent. The innermost branches will be the last in the list. Return None if we are out of the entire tree, and 1 otherwise. rr9N)rc base_indexr/ call_end_procbranch_process)rUrarcto_be_finishedrdr r rfinish_branches#s zIterTreeReducer.finish_branchescCs|j|j}|j||S)z;Return branch of type self.branch_class, add to branch list)r_r`rcrXrUbranchr r r add_branch:s  zIterTreeReducer.add_branchcCs(t|j|j||jsd|_||_dS)z"Run start_process on latest branchrN)rcheck_common_erroron_error start_processcaught_exceptionstart_successfulrd)rUrarjargsr r rprocess_w_branch@s  z IterTreeReducer.process_w_branchcCs0 |j}||jsdS|jd|q)z,Call at end of sequence to tie everything uprr9N)rcrYrerf)rUrgr r rFinishHs zIterTreeReducer.FinishcGs|d}|jdur|||j|||_dS||jkr/ttdt|jt|fdS||dur8dS|j d}|j r[|j |rOt |j|j|n|}||||n||||_dS)adProcess args, where args[0] is current position in iterator Returns true if args successfully processed, false if index is not in the current tree and thus the final result is available. Also note below we set self.index after doing the necessary start processing, in case there is a crash in the middle. rNrz#Warning: oldindex %s >= newindex %sr9)rarrrbrWarn_r uindexrhrcrpcan_fast_processrrlrm fast_processrklog_prev_error)rUrqra last_branchrjr r r__call__Qs.      zIterTreeReducer.__call__N) rOrPrQrRrVrhrkrrrsr{r r r rr^ s  r^c@sdeZdZdZdZZdZdZZddZ ddZ ddZ d d Z d d Z d dZddZddZdS) ITRBranchaHelper class for IterTreeReducer above There are five stub functions below: start_process, end_process, branch_process, fast_process, and can_fast_process. A class that subclasses this one will probably fill in these functions to do more. NcCs,|js|js d|_t|j|jd|_dS)z1Runs the end_process on self, checking for errorsrN)finishedrprorrlrm end_processrUr r rres  zITRBranch.call_end_proccGdS)z!Do some initial processing (stub)Nr rUrqr r rrnzITRBranch.start_processcCr)z4Do any final processing before leaving branch (stub)Nr rr r rr~rzITRBranch.end_processcCs|jsJdS)z2Process a branch right after it is finished (stub)N)r}rir r rrfs zITRBranch.branch_processcGr)z9True if object can be processed without new branch (stub)Nr rr r rrwrzITRBranch.can_fast_processcGr)z,Process args without new child branch (stub)Nr rr r rrxrzITRBranch.fast_processcGszd|_|r|drt|dtrtjj|d}n |jr$tjj|j}nd}tt d|t |ftj j t |dS)z1This is run on any exception in start/end-processrr.zError '%s' processing %sN)ro isinstancerKospathjoinrarrtrur fsdecode WarningCodecannot_processescape)rUexcrqfilenamer r rrmszITRBranch.on_errorcCs@|sd}ntjj|}ttdt|tjj t |dS)z%Call function if no pending exceptionrz%Skipping %s because of previous errorN) rrrrrtrur rrprocess_skippedr)rUra index_strr r rrys zITRBranch.log_prev_error)rOrPrQrRrdrar}rorprernr~rfrwrxrmryr r r rr|xs  r|)rR __future__rbuiltinsrrrrr duplicityrrr r rHr^r|r r r rs        80k