o ]LbE@s@ddlmZddlZddlZddlmZdZdZdZdZ dZ dZ d Z d Z d Zd Zd ZdZdZdZdZd ZdZeded>eBZede d>eBZede d>eBZedZedZedZedZedZ dZ!dZ"ddZ#ddZ$ddZ%dLd!d"Z&d#d$Z'd%d&Z(d'd(Z)d)d*Z*d+d,Z+d-d.Z,d/d0Z-d1d2Z.d3d4Z/e0e$e1e'ej2e'e3e(e4e(e5e,e6e+e7e.e8de/i Z9d5d6Z:Gd7d8d8e;Zdkrd9d:Z?nd;d:Z?edZ@ed<ZAed=ZBed>ZCdZDdZEdZFdZGdZHdZIdMd?d@ZJdNdBdCZKGdDdEdEe0ZLGdFdGdGeMZNGdHdIdIeMZOdJdKZPdS)O)absolute_importN)pycompatiz>Bz>BBz>BHz>BLz>BQcCs|dkr t|d>|BS|dkrt|d>dB|S|dkr)t|d>dB|S|dkr7t|d>dB|St|d>dB|S) z6Obtain a value encoding the major type and its length.rrlrr)ENCODED_LENGTH_1packENCODED_LENGTH_2ENCODED_LENGTH_3ENCODED_LENGTH_4ENCODED_LENGTH_5) majortypelengthr:/usr/lib/python3/dist-packages/mercurial/utils/cborutil.py encodelengthEsr!ccsttt|V|VdSN)r!MAJOR_TYPE_BYTESTRINGlenvrrr streamencodebytestringSs r'ccs2tV|D] }ttt|V|VqtVdS)zConvert an iterator of chunks to an indefinite bytestring. Given an input that is iterable and each element in the iterator is representable as bytes, emit an indefinite length bytestring. N)BEGIN_INDEFINITE_BYTESTRINGr!r#r$BREAK)itchunkrrr streamencodebytestringfromiterXs  r,rccs^tVd}t|} ||||}|t|7}ttt|V|V||kr)nq tVdS)zGiven a large source buffer, emit as an indefinite length bytestring. This is a generator of chunks constituting the encoded CBOR data. rTN)r(r$r!r#r))source chunksizeilr+rrr streamencodeindefinitebytestringgs  r1ccsJ|dks |dkr td|dkrtt|VdSttt|dVdS)Nllsbig integers not supportedrr) ValueErrorr!MAJOR_TYPE_UINTMAJOR_TYPE_NEGINTabsr%rrr streamencodeint~s r6ccs4ttt|V|D] }t|D]}|Vqq dS)z)Encode a known size iterable to an array.N)r!MAJOR_TYPE_ARRAYr$ streamencode)r0r/r+rrr streamencodearrays r9ccs0tV|D] }t|D]}|Vq qtVdS)z:Encode an iterator of items to an indefinite length array.N)BEGIN_INDEFINITE_ARRAYr8r))r*r/r+rrr streamencodearrayfromiters  r;cCst|j|fSr")type__name__r%rrr _mixedtypesortkeysr>ccs.tttVtt|tdD]}|VqdS)Nkey)r!MAJOR_TYPE_SEMANTICSEMANTIC_TAG_FINITE_SETr9sortedr>)sr+rrr streamencodesets  rEccs^ttt|Vtt|dddD]\}}t|D]}|Vqt|D]}|Vq&qdS)z_Encode dictionary to a generator. Does not supporting indefinite length dictionaries. cSs t|dSNr)r>)xrrr s z!streamencodemap..r?N)r!MAJOR_TYPE_MAPr$rCr iteritemsr8)dr@valuer+rrr streamencodemaps  rMccsHtV|D]\}}t|D]}|Vqt|D]}|VqqtVdS)zFGiven an iterable of (key, value), encode to an indefinite length map.N)BEGIN_INDEFINITE_MAPr8r))r*r@rLr+rrr streamencodemapfromiters    rOccs|rdVdSdVdS)Nr)brrr streamencodeboolsrSccs dVdS)Nrr%rrr streamencodenones rUcCsJt|j}|stD] }t||sq t|}|s!tdt|||S)a1Encode a value in a streaming manner. Given an input object, encode it to CBOR recursively. Returns a generator of CBOR encoded bytes. There is no guarantee that each emitted chunk fully decodes to a value or sub-value. Encoding is deterministic - unordered collections are sorted. sdo not know how to encode %s)STREAM_ENCODERSget __class__ isinstancer2r<)r&fntyrrr r8s  r8c@seZdZdZdS)CBORDecodeErrorz"Represents an error decoding CBOR.N)r= __module__ __qualname____doc__rrrr r\sr\cCs||Sr"rrRr/rrr _elementtointegersracCs t||Sr")ordr`rrr ras s>Hs>Ls>Qc Cst||}|d7}|d?}|t@}|tkr.t|||\}}}|r(d||dtfSdd|tfS|tkrNt|||\}}}|rHd| d|dtfSdd|tfS|tkrt|||dd\}}}|sedd|tfS|dur|||t|kr||||||}d|||dtfSt||||} dd| tfSdddtfS|t krt d|t krt|||\}}}|rd||dt fSdd|tfS|t krt|||\}}}|rd||dtfSdd|tfS|tkr8t|||\}} }|sdd|tfS| tkr2||t|kr dddtfSt|||\}}} } |sdd| tfS| t kr(t d d||| dtfSt d | |tkro|tkrHdddtfS|tkrSdddtfS|tkr^dddtfS|tkridddtfSt d |J) aDecode a new CBOR value from a buffer at offset. This function attempts to decode up to one complete CBOR value from ``b`` starting at offset ``offset``. The beginning of a collection (such as an array, map, set, or indefinite length bytestring) counts as a single value. For these special cases, a state flag will indicate that a special value was seen. When called, the function either returns a decoded value or gives a hint as to how many more bytes are needed to do so. By calling the function repeatedly given a stream of bytes, the caller can build up the original values. Returns a tuple with the following elements: * Bool indicating whether a complete value was decoded. * A decoded value if first value is True otherwise None * Integer number of bytes. If positive, the number of bytes read. If negative, the number of bytes we need to read to decode this value or the next chunk in this value. * One of the ``SPECIAL_*`` constants indicating special treatment for this value. ``SPECIAL_NONE`` means this is a fully decoded simple value (such as an integer or bool). rrTFN)allowindefinitesstring major type not supporteds,expected array after finite set semantic tagssemantic tag %d not allowedsspecial type %d not allowed)ra SUBTYPE_MASKr3 decodeuint SPECIAL_NONEr4r#r$#SPECIAL_START_INDEFINITE_BYTESTRINGMAJOR_TYPE_STRINGr\r7SPECIAL_START_ARRAYrISPECIAL_START_MAPrArB decodeitemSPECIAL_START_SETMAJOR_TYPE_SPECIAL SUBTYPE_FALSE SUBTYPE_TRUE SUBTYPE_NULLSUBTYPE_INDEFINITESPECIAL_INDEFINITE_BREAK) rRoffsetinitialrsubtypecompleterL readcountsizewantedtagvalue readcount2specialrrr rls                           rlFcCs|dkr d|dfS|dkr|rdStd|dkrtd||dkr&t}n|d kr-t}n|d kr4t}n |d kr;t}ntd t|||jkrTd|||d|jfSd dt|||jfS)aDecode an unsigned integer. ``subtype`` is the lower 5 bits from the initial byte CBOR item "header." ``b`` is a buffer containing bytes. ``offset`` points to the index of the first byte after the byte that ``subtype`` was derived from. ``allowindefinite`` allows the special indefinite length value indicator. Returns a 3-tuple of (successful, value, count). The first element is a bool indicating if decoding completed. The 2nd is the decoded integer value or None if not fully decoded or the subtype is 31 and ``allowindefinite`` is True. The 3rd value is the count of bytes. If positive, it is the number of additional bytes decoded. If negative, it is the number of additional bytes needed to decode this value. rTrr )TNrs'indefinite length uint not allowed heres'unsupported subtype on integer type: %drrrs#bounds condition checking violationFN)r\STRUCT_BIG_UBYTESTRUCT_BIG_USHORTSTRUCT_BIG_ULONGSTRUCT_BIG_ULONGLONGr$ry unpack_from)rvrRrtrcrDrrr rfs, rfc@seZdZdZdddZdS)bytestringchunkaRepresents a chunk/segment in an indefinite length bytestring. This behaves like a ``bytes`` but in addition has the ``isfirst`` and ``islast`` attributes indicating whether this chunk is the first or last in an indefinite length bytestring. FcCst||}||_||_|Sr")bytes__new__isfirstislast)clsr&firstlastselfrrr rs zbytestringchunk.__new__N)FF)r=r]r^r_rrrrr rsrc@sReZdZdZdZdZdZdZdZdZ dZ d d Z e d d Z dd dZddZdS) sansiodecoderaA CBOR decoder that doesn't perform its own I/O. To use, construct an instance and feed it segments containing CBOR-encoded bytes via ``decode()``. The return value from ``decode()`` indicates whether a fully-decoded value is available, how many bytes were consumed, and offers a hint as to how many bytes should be fed in next time to decode the next value. The decoder assumes it will decode N discrete CBOR values, not just a single value. i.e. if the bytestream contains uints packed one after the other, the decoder will decode them all, rather than just the initial one. When ``decode()`` indicates a value is available, call ``getavailable()`` to return all fully decoded values. ``decode()`` can partially decode input. It is up to the caller to keep track of what data was consumed and to pass unconsumed data in on the next invocation. The decoder decodes atomically at the *item* level. See ``decodeitem()``. If an *item* cannot be fully decoded, the decoder won't record it as partially consumed. Instead, the caller will be instructed to pass in the initial bytes of this item on the next invocation. This does result in some redundant parsing. But the overhead should be minimal. This decoder only supports a subset of CBOR as required by Mercurial. It lacks support for: * Indefinite length arrays * Indefinite length maps * Use of indefinite length bytestrings as keys or values within arrays, maps, or sets. * Nested arrays, maps, or sets within sets * Any semantic tag that isn't a mathematical finite set * Floating point numbers * Undefined special value CBOR types are decoded to Python types as follows: uint -> int negint -> int bytestring -> bytes map -> dict array -> list True -> bool False -> bool null -> None indefinite length bytestring chunk -> [bytestringchunk] The only non-obvious mapping here is an indefinite length bytestring to the ``bytestringchunk`` type. This is to facilitate streaming indefinite length bytestrings out of the decoder and to differentiate a regular bytestring from an indefinite length bytestring. rrrrrrr cCs$d|_|j|_g|_d|_g|_dSrF)decodedbytecount _STATE_NONE_state_collectionstack_currentmapkey_decodedvaluesrrrr __init__4s  zsansiodecoder.__init__cCs |j|jkS)z2Whether the decoder has partially decoded a value.)rrrrrr inprogressGs zsansiodecoder.inprogressc Csd|s t|jddfS|}|t|kr(t||\}}}}|dkr'|j|7_|s:|dks/Jt|j||| fS||7}|j|jkr|tkrP|j|n|t krc|j |gd|j |_ny|t krv|j |id|j |_nf|tkr|j |td|j|_nR|tkr|j|_nHtd||j|j krD|tkr|j d}|d||dd8<n!|t kr|j d} g} | d| | dd8<|j || dn|t kr |j d} i} | d| | dd8<|j || d|j |_n|tkr5|j d} t} | d| | dd8<|j || d|j|_n|tkr>tdtd ||j|j krt|tkrY||_|j|_n|tkrbtd |t t tfvrntd td ||j|jkr(|tkr|j d} || d|j<| dd8<|j |_n|t kr|j d} g} | | d|j<| dd8<|j || d|j |_nb|t kr|j d} i} | | d|j<| dd8<|j || d|j |_n9|tkr|j d} t} | | d|j<| dd8<|j || d|j|_n|tkrtd td|d|_n|j|jkrd|tkrI|j d} | d|| dd8<n|tkrRtd|t t tfvr^tdtd||j|jkr|tkr|jt|dd|j|_n]|tkr|jtdddd|j rJ|j|_nBtd||j|jkr|tkr|jt|n'|tkr|jtddd|j rJ|j|_n td|td|j |j|j |j |jfvrn8|j d} | drn-|j |j rt|j t|j t|jit|j dd|_n |j| d|j|_q|t|kst|j||dfS)aAttempt to decode bytes from an input buffer. ``b`` is a collection of bytes and ``offset`` is the byte offset within that buffer from which to begin reading data. ``b`` must support ``len()`` and accessing bytes slices via ``__slice__``. Typically ``bytes`` instances are used. Returns a tuple with the following fields: * Bool indicating whether values are available for retrieval. * Integer indicating the number of bytes that were fully consumed, starting from ``offset``. * Integer indicating the number of bytes that are desired for the next call in order to decode an item. r) remainingvsunhandled special state: %drdrrrs9indefinite length bytestrings not allowed as array valuess5unhandled special item when expecting array value: %ds5indefinite length bytestrings not allowed as map keyss%collections not supported as map keyss1unhandled special item when expecting map key: %ds7indefinite length bytestrings not allowed as map valuess3unhandled special item when expecting map value: %dNs7indefinite length bytestrings not allowed as set valuess%collections not allowed as set valuess3unhandled special item when expecting set value: %dT)r)rrs<unexpected special value when expecting bytestring chunk: %d)rsunhandled decoder state: %d)boolrr$rlrrrrgappendrjr_STATE_WANT_ARRAY_VALUErk_STATE_WANT_MAP_KEYrmset_STATE_WANT_SET_VALUErh"_STATE_WANT_BYTESTRING_CHUNK_FIRSTr\r_STATE_WANT_MAP_VALUEaddr'_STATE_WANT_BYTESTRING_CHUNK_SUBSEQUENTrspoplistdictr<) rrRrt initialoffsetrwrLrxr}clastcnewvaluerrr decodeLs                                                jzsansiodecoder.decodecCst|j}g|_|S)zReturns an iterator over fully decoded values. Once values are retrieved, they won't be available on the next call. )rr)rr0rrr getavailables zsansiodecoder.getavailableNr)r=r]r^r_rrrrrrrrpropertyrrrrrrr rs"8   rc@s(eZdZdZddZddZddZdS) bufferingdecoderahA CBOR decoder that buffers undecoded input. This is a glorified wrapper around ``sansiodecoder`` that adds a buffering layer. All input that isn't consumed by ``sansiodecoder`` will be buffered and concatenated with any new input that arrives later. TODO consider adding limits as to the maximum amount of data that can be buffered. cCst|_g|_d|_dSrF)r_decoder_chunks_wantedrrrr rs zbufferingdecoder.__init__cCst|tr t|}|jrBt||jkr(|j||jt|8_dd|jfSt|}|j|d|j}g|_t||}nd}|j |\}}}||_|t|kr`|j||d||||fS)a0Attempt to decode bytes to CBOR values. Returns a tuple with the following fields: * Bool indicating whether new values are available for retrieval. * Integer number of bytes decoded from the new input. * Integer number of bytes wanted to decode the next value. FrrN) rY bytearrayrrr$rrjoinrr)rrRnewlenoldlen availablerxrzrrr rs$       zbufferingdecoder.decodecCs |jSr")rrrrrr r#s zbufferingdecoder.getavailableN)r=r]r^r_rrrrrrr rs   3rcCsH|sgSt}||\}}}|t|krtd|jr td|S)avDecode all CBOR items present in an iterable of bytes. In addition to regular decode errors, raises CBORDecodeError if the entirety of the passed buffer does not fully decode to complete CBOR values. This includes failure to decode any value, incomplete collection types, incomplete indefinite length items, and extra data at the end of the buffer. sinput data not fully consumedsinput data not complete)rrr$r\rr)rRdecoder havevaluesrx wantbytesrrr decodeall's  r)rr)rF)Q __future__rstructsysrr3r4r#rir7rIrArnrerorprqSUBTYPE_HALF_FLOATSUBTYPE_SINGLE_FLOATSUBTYPE_DOUBLE_FLOATrrrBrr(r:rNStructrrrrrr) BREAK_INTr!r'r,r1r6r9r;r>rErMrOrSrUrintlongrtuplerrrr<rVr8 Exceptionr\ version_infomajorrarrrrrgrhrjrkrmrsrlrfrobjectrrrrrrr s                          3p G