o ]Lb@sHdZddlmZddlZddlZddlmZddlmZddl m Z dd l m Z m Z mZmZmZdd lmZdd lmZmZmZdd lmZdZGd ddeZd1ddZd1ddZd2ddZd1ddZddZddZ ej!dddGdddeZ"d d!Z#d"d#Z$d$Z%d%d&Z&d'd(Z'd)d*Z(d1d+d,Z)Gd-d.d.eZ*d/d0Z+dS)3z4Helper class to compute deltas stored inside revlogs)absolute_importNnullrev)_)getattr)COMP_MODE_DEFAULTCOMP_MODE_INLINECOMP_MODE_PLAINREVIDX_ISCENSOREDREVIDX_RAWTEXT_CHANGING_FLAGS)attr)errormdiffutil)flagutilc@sBeZdZdZdddZddZd d Zd d Zd dZddZ dS) _testrevlogz)minimalist fake revlog to use in doctests?rcCs&||_||_||_t||_d|_dS)z.data is an list of revision payload boundariesN)_data_srdensitythreshold _srmingapsizeset _snapshotindex)selfdatadensitymingapsnapshotrr>/usr/lib/python3/dist-packages/mercurial/revlogutils/deltas.py__init__-s   z_testrevlog.__init__cCs&|tkrdS|dkr dS|j|dSNrrrrrrevrrr!start5s z_testrevlog.startcCs|tkrdS|j|SNrr$r%rrr!end< z_testrevlog.endcCs||||SNr)r'r%rrr!lengthAsz_testrevlog.lengthcCs t|jSr+)lenr)rrrr!__len__D z_testrevlog.__len__cCs|tkrdS||jvS)NT)rrr%rrr! issnapshotGr*z_testrevlog.issnapshotN)rrr) __name__ __module__ __qualname____doc__r"r'r)r-r/r1rrrr!r*s  rc#sj|dur t|j}tjdd}|durfdd}||jjD]}t||D]}|Vq,q$dS)aslice revs to reduce the amount of unrelated data to be read from disk. ``revs`` is sliced into groups that should be read in one time. Assume that revs are sorted. The initial chunk is sliced until the overall density (payload/chunks-span ratio) is above `revlog._srdensitythreshold`. No gap smaller than `revlog._srmingapsize` is skipped. If `targetsize` is set, no chunk larger than `targetsize` will be yield. For consistency with other slicing choice, this limit won't go lower than `revlog._srmingapsize`. If individual revisions chunk are larger than this limit, they will still be raised individually. >>> data = [ ... 5, #00 (5) ... 10, #01 (5) ... 12, #02 (2) ... 12, #03 (empty) ... 27, #04 (15) ... 31, #05 (4) ... 31, #06 (empty) ... 42, #07 (11) ... 47, #08 (5) ... 47, #09 (empty) ... 48, #10 (1) ... 51, #11 (3) ... 74, #12 (23) ... 85, #13 (11) ... 86, #14 (1) ... 91, #15 (5) ... ] >>> revlog = _testrevlog(data, snapshot=range(16)) >>> list(slicechunk(revlog, list(range(16)))) [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]] >>> list(slicechunk(revlog, [0, 15])) [[0], [15]] >>> list(slicechunk(revlog, [0, 11, 15])) [[0], [11], [15]] >>> list(slicechunk(revlog, [0, 11, 13, 15])) [[0], [11, 13, 15]] >>> list(slicechunk(revlog, [1, 2, 3, 5, 8, 10, 11, 14])) [[1, 2], [5, 8, 10, 11], [14]] Slicing with a maximum chunk size >>> list(slicechunk(revlog, [0, 11, 13, 15], targetsize=15)) [[0], [11], [13], [15]] >>> list(slicechunk(revlog, [0, 11, 13, 15], targetsize=20)) [[0], [11], [13, 15]] Slicing involving nullrev >>> list(slicechunk(revlog, [-1, 0, 11, 13, 15], targetsize=20)) [[-1, 0], [11], [13, 15]] >>> list(slicechunk(revlog, [-1, 13, 15], targetsize=5)) [[-1], [13], [15]] Nslicechunktodensitycst|||Sr+)_slicechunktodensity)xyzrevlogrr!zslicechunk..)maxrrrr_slicechunktosize)r<revs targetsizedensityslicingchunksubchunkrr;r! slicechunkMs<   rFccs|dus d|ks J||d}||d}||}|dus%||kr*|VdSd}d}t|}t||D]6\} } || |} || } | |krS| rS| d}nt||||} | r_| V| }|| }| d}| snnq8t|}|||kr|}||dkrnL|||d}||} | |kr||dkrn|||d8}|||d}||} | |kst||||} | r| V|}|||}|||ksyt|||} | r| VdSdS)aJ slice revs to match the target size This is intended to be used on chunk that density slicing selected by that are still too large compared to the read garantee of revlog. This might happens when "minimal gap size" interrupted the slicing or when chain are built in a way that create large blocks next to each other. >>> data = [ ... 3, #0 (3) ... 5, #1 (2) ... 6, #2 (1) ... 8, #3 (2) ... 8, #4 (empty) ... 11, #5 (3) ... 12, #6 (1) ... 13, #7 (1) ... 14, #8 (1) ... ] == All snapshots cases == >>> revlog = _testrevlog(data, snapshot=range(9)) Cases where chunk is already small enough >>> list(_slicechunktosize(revlog, [0], 3)) [[0]] >>> list(_slicechunktosize(revlog, [6, 7], 3)) [[6, 7]] >>> list(_slicechunktosize(revlog, [0], None)) [[0]] >>> list(_slicechunktosize(revlog, [6, 7], None)) [[6, 7]] cases where we need actual slicing >>> list(_slicechunktosize(revlog, [0, 1], 3)) [[0], [1]] >>> list(_slicechunktosize(revlog, [1, 3], 3)) [[1], [3]] >>> list(_slicechunktosize(revlog, [1, 2, 3], 3)) [[1, 2], [3]] >>> list(_slicechunktosize(revlog, [3, 5], 3)) [[3], [5]] >>> list(_slicechunktosize(revlog, [3, 4, 5], 3)) [[3], [5]] >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3)) [[5], [6, 7, 8]] >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3)) [[0], [1, 2], [3], [5], [6, 7, 8]] Case with too large individual chunk (must return valid chunk) >>> list(_slicechunktosize(revlog, [0, 1], 2)) [[0], [1]] >>> list(_slicechunktosize(revlog, [1, 3], 1)) [[1], [3]] >>> list(_slicechunktosize(revlog, [3, 4, 5], 2)) [[3], [5]] == No Snapshot cases == >>> revlog = _testrevlog(data) Cases where chunk is already small enough >>> list(_slicechunktosize(revlog, [0], 3)) [[0]] >>> list(_slicechunktosize(revlog, [6, 7], 3)) [[6, 7]] >>> list(_slicechunktosize(revlog, [0], None)) [[0]] >>> list(_slicechunktosize(revlog, [6, 7], None)) [[6, 7]] cases where we need actual slicing >>> list(_slicechunktosize(revlog, [0, 1], 3)) [[0], [1]] >>> list(_slicechunktosize(revlog, [1, 3], 3)) [[1], [3]] >>> list(_slicechunktosize(revlog, [1, 2, 3], 3)) [[1], [2, 3]] >>> list(_slicechunktosize(revlog, [3, 5], 3)) [[3], [5]] >>> list(_slicechunktosize(revlog, [3, 4, 5], 3)) [[3], [4, 5]] >>> list(_slicechunktosize(revlog, [5, 6, 7, 8], 3)) [[5], [6, 7, 8]] >>> list(_slicechunktosize(revlog, [0, 1, 2, 3, 4, 5, 6, 7, 8], 3)) [[0], [1, 2], [3], [5], [6, 7, 8]] Case with too large individual chunk (must return valid chunk) >>> list(_slicechunktosize(revlog, [0, 1], 2)) [[0], [1]] >>> list(_slicechunktosize(revlog, [1, 3], 1)) [[1], [3]] >>> list(_slicechunktosize(revlog, [3, 4, 5], 2)) [[3], [5]] == mixed case == >>> revlog = _testrevlog(data, snapshot=[0, 1, 2]) >>> list(_slicechunktosize(revlog, list(range(9)), 5)) [[0, 1], [2], [3, 4, 5], [6, 7, 8]] Nrrr)r'r) enumeratenextr1 _trimchunkr.)r<rArB startdataenddatafullspan startrevidx endrevidxiterrevsidxrspanr rDnbitem localenddatarrr!r@sdc          r@rc#s|j}|jt|dkr|VdSt||}||kr |VdS|}tfdd|D}|r6|t|}nd}||krA|VdSg} d} t|D](\} } || } | }|dkrZqI| durm| | }||krm| || f| |} qI| g}| r||kr| \}}||||8}|dkr|t|}nd}| r||ks~|d}|D]}t ||||}|r|V|}qt |||}|r|VdSdS)aslice revs to reduce the amount of unrelated data to be read from disk. ``revs`` is sliced into groups that should be read in one time. Assume that revs are sorted. The initial chunk is sliced until the overall density (payload/chunks-span ratio) is above `targetdensity`. No gap smaller than `mingapsize` is skipped. >>> revlog = _testrevlog([ ... 5, #00 (5) ... 10, #01 (5) ... 12, #02 (2) ... 12, #03 (empty) ... 27, #04 (15) ... 31, #05 (4) ... 31, #06 (empty) ... 42, #07 (11) ... 47, #08 (5) ... 47, #09 (empty) ... 48, #10 (1) ... 51, #11 (3) ... 74, #12 (23) ... 85, #13 (11) ... 86, #14 (1) ... 91, #15 (5) ... ]) >>> list(_slicechunktodensity(revlog, list(range(16)))) [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]] >>> list(_slicechunktodensity(revlog, [0, 15])) [[0], [15]] >>> list(_slicechunktodensity(revlog, [0, 11, 15])) [[0], [11], [15]] >>> list(_slicechunktodensity(revlog, [0, 11, 13, 15])) [[0], [11, 13, 15]] >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14])) [[1, 2], [5, 8, 10, 11], [14]] >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14], ... mingapsize=20)) [[1, 2, 3, 5, 8, 10, 11], [14]] >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14], ... targetdensity=0.95)) [[1, 2], [5], [8, 10, 11], [14]] >>> list(_slicechunktodensity(revlog, [1, 2, 3, 5, 8, 10, 11, 14], ... targetdensity=0.95, mingapsize=12)) [[1, 2], [5, 8, 10, 11], [14]] rNc3s|]}|VqdSr+r.0rRr-rr! qsz'_slicechunktodensity..g?r) r'r-r. segmentspansumfloatrHappendsortpoprJ)r<rA targetdensity mingapsizer'deltachainspanreaddata chainpayloadrgapsprevendir&revstartrevlengapsizeselectedgapidxprevidxrQrDrrXr!r72sf1          r7cCs|j}|dur t|}||dt|kr=|dkr=||kr=|||ddkr=|d8}|dkr=||kr=|||ddks'|||S)areturns revs[startidx:endidx] without empty trailing revs Doctest Setup >>> revlog = _testrevlog([ ... 5, #0 ... 10, #1 ... 12, #2 ... 12, #3 (empty) ... 17, #4 ... 21, #5 ... 21, #6 (empty) ... ]) Contiguous cases: >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0) [0, 1, 2, 3, 4, 5] >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 5) [0, 1, 2, 3, 4] >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 0, 4) [0, 1, 2] >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 2, 4) [2] >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3) [3, 4, 5] >>> _trimchunk(revlog, [0, 1, 2, 3, 4, 5, 6], 3, 5) [3, 4] Discontiguous cases: >>> _trimchunk(revlog, [1, 3, 5, 6], 0) [1, 3, 5] >>> _trimchunk(revlog, [1, 3, 5, 6], 0, 2) [1] >>> _trimchunk(revlog, [1, 3, 5, 6], 1, 3) [3, 5] >>> _trimchunk(revlog, [1, 3, 5, 6], 1) [3, 5] Nrr)r-r.)r<rAstartidxendidxr-rrr!rJs&$$ rJcCs(|sdS||d}|||dS)aGet the byte span of a segment of revisions revs is a sorted array of revision numbers >>> revlog = _testrevlog([ ... 5, #0 ... 10, #1 ... 12, #2 ... 12, #3 (empty) ... 17, #4 ... ]) >>> segmentspan(revlog, [0, 1, 2, 3, 4]) 17 >>> segmentspan(revlog, [0, 4]) 17 >>> segmentspan(revlog, [3, 4]) 5 >>> segmentspan(revlog, [1, 2, 3,]) 7 >>> segmentspan(revlog, [1, 3]) 7 rrGr,)r<rAr)rrr!rZsrZc Cstd}|d|t||t||kr||d} n |j||d} t| |} z"t || |} | r?|j | |||d|t @rLt td|W| St jy^|t @s[Y| Sw)zlllN_df)p1p2snode %s is not censored)structcalcsizerreplacediffheaderrawsizer.revisionpatchrprocessflagsraw checkhashr r StorageErrorrCensoredNodeError) fhr<baserevdeltarrrsflags expectednodehlenfulltextbasetext validatehashrrr!_textfromdeltas.   rT)slotsfrozenc@sLeZdZeZeZeZeZeZ eZ eZ eZ dS) _deltainfoN) r2r3r4ribdistancedeltalenrbase chainbasechainlencompresseddeltalen snapshotdepthrrrr!r#s rc CsN|jddksJ|jdt|j|jdd|jdf|j|j|j|j|jS)zturn into a "u" (no-compression) into no-compression without header This is useful for revlog format that has better compression method. rur) rrrrrrrrr)rrrr!drop_u_compression/s rcCs|durdS|j}|d}|j}|s|j}t||}|js$||jkr$dS||jkr+dS|t|jkr4dS|jr?|j|j kr?dS|j durN||j ?|jkrNdS|j r\| |j |jkr\dSdS)zReturns True if the given delta is good. Good means that it is within the disk span, disk size, and chain length bounds that we know to be performant.NFT) textlen_maxdeltachainspanrr? _sparserevlogrLIMIT_DELTA2TEXTr _maxchainlenrrr-r)r< deltainforevinfor defaultmaxmaxdistrrr!isgooddeltainfoAs0    riccst|r|js dVdS|j}|j}|j}d}|t} th} t||||} | |} | dur1ng} | D]}|j r^|tks^|| vs^||s^| |||}|j r^|tks^|| vs^||rF|tkrcq5|| vrhq5| || ||kruq5|r| ||t krq5| |t@rq5||\}}|jr||jkrq5| |krq5|r|jdur|j}||tf}||vr||r||}| |}t||d}||}||?}||krq5||}||krq5| |q5| rt| V}q'dVdS)zProvides group of revision to be tested as delta base This top level function focus on emitting groups with unique and worthwhile content. See _raw_candidate_groups for details about the group order. NTr)r._storedeltachainsr- deltaparentrrr_refinedgroupssend _generaldeltaaddrwLIMIT_BASE2TEXTrr _chaininforupperboundcompr1rr?r]tuple)r<rrrrs cachedelta deltalengthrsparsegood deltas_limittested candidates temptativegroupr&r chainsizemaxcomp basenotsnaprrevsizerawsizedistancelowestrealisticdeltalen snapshotlimit revlengthrrr!_candidategroupsst          FrcCsZt|jdr|j||dS|j}|j}||D]}||r*||||qdS)z#find snapshot from start_rev to tips findsnapshotsN)r safehasattrr findsnapshotsrr1rAr])r<cache start_revrr1r&rrr!_findsnapshotssrc csd}|r|jr|jr|dfV}|durdVdStt}t|||||D] }|V}|dur2nq'|js;dVdS|dur|||fvr||rd}||krd|}||}|t kr\n|fV}||ksP|snt |||dd}||kr|}t t dd||D} | V}||kstdVdS)Nrrcss|]}|VqdSr+r)rWcrrr!rY#sz!_refinedgroups..) r_lazydeltabase collections defaultdictlist _rawgroupsrr1rrrrsorted) r<rrrsrr snapshotsrpreviousrchildrenrrr!rsD    rc#sXj}|oj}t}|d}fdd|rEdd||fD} jsz_rawgroups..cSsg|]}|tkr|qSrrrWprrr! ;z_rawgroups..rrNcsg|]}|qSrrr) deltachainrr!rRsT)reversecsg|]}|kr|qSrrrV)floorrr!rr)rrr._deltabothparentsr^rrrrrHr1rminrupdaterAr?keysritemsrr)r<rrrsrrgdeltarcurrprevparents parents_snapscandidate_chainschainrQs snapfloorall_revsotherrR max_depthsnapssiblingsr)rrr<r!r*sz                 rc@s>eZdZddZddZddZddZd d Zdd d Zd S) deltacomputercCs ||_dSr+r;)rr<rrr!r"r0zdeltacomputer.__init__c Cs`|j}|ddur |dS|j}|j}|d}|d}t|||||j|j|j|j}|d<|S)zBuilds a fulltext version of a revision revinfo: revisioninfo instance that contains all needed info fh: file handle to either the .i or the .d revlog file, depending on whether it is inlined or not rNr)btextr<rrrrrsrnode) rrr~rr<rrrrrrr! buildtexts$   zdeltacomputer.buildtextc Cs\|j}|||}||rt||t|}||}|S|j||d}t||}|S)Nrp) r<r iscensoredrrvrwr.rawdatatextdiff) rrrr~r<theaderrptextrrr!_builddeltadiffs   zdeltacomputer._builddeltadiffc Cs|j}||}|jr|}n|}d}|jr|tkrd}n)|jrE||rE||j||j}} ||| fvrE||rEt | |d}d} |j r|j \} } | } | tkrw| |krw|j | dkrw|j | } | tkrw| |krw|j | dksa|jjr| |kr|j d} | dur||||} |jdur|rt | |j}|j|?}||krdS| ||krdS|| \}}t |t |}|t |d}||||}||\}}|d7}||7}t||||f|||||Sr#)r<rrrrr1r&rrrsr.rrr-r _lazydeltarrrcompressr)r'rr)rrrr~r<r deltabaserrrrsr cachebase cachediff currentbaserrrrroffsetdistrrrrr!_builddeltainfosf     zdeltacomputer._builddeltainfoc Cs^|||}|j|}t|dt|d}}}|} } d} d} t|||| | | || S)Nrr)rr<rr.r) rr~rrrawtextrrrrrrrrrrr!_fullsnapshotinfo s   zdeltacomputer._fullsnapshotinfoNcCsX|dur t|j}|js||||S|durt}|jt@r&||||S|j}|j}|j }|j}d} | || |} } t |j|j| | |} t | } | durg}| dur\| | | D]#}||vreq^||krjq^||||}|durt|j||r| |q^|rt|ddd} | dur| | j} nt | } | dusQ| dur||||} | S)ayFind an acceptable delta against a candidate revision revinfo: information about the revision (instance of _revisioninfo) fh: file handle to either the .i or the .d revlog file, depending on whether it is inlined or not Returns the first acceptable candidate revision, as ordered by _candidategroups If no suitable deltabase is found, we return delta info for a full snapshot. `excluded_bases` is an optional set of revision that cannot be used as a delta base. Use this to recompute delta suitable in censor or strip context. NcSs|jSr+)r)r8rrr!r=Zsz-deltacomputer.finddeltainfo..)key)r.r<rrrrr rrrrsr&rrIr]rrrrr)rrr~excluded_bases target_revrrrrsr<rp1rp2rgroups candidaterevsnominateddeltas candidaterevcandidatedeltarrr! finddeltainfosR    zdeltacomputer.finddeltainfo)NN) r2r3r4r"rrrrrrrrr!rs;rcCsz|j\}}t}|s|st}||fS|s-|dd}|dkr#t}||fS||kr)t}||fS|dkr9d}t}t|}||fS)zreturn (COMPRESSION_MODE, deltainfo) used by revlog v2+ format to dispatch between PLAIN and DEFAULT compression. rrrr)rr r r r)default_compression_headerrhdcompression_moderrrr!delta_compressiones$   r r+)rr),r5 __future__rrrtrri18nrpycompatr constantsr r r r r thirdpartyrrrrrrobjectrrFr@r7rJrZrrrrrrrrrrrr rrrr!s@       # J  ~6  L] 1y C