o .&ai@sddlmZddlmZddlZddlZddlZddlZddlm Z ddlm Z ddl m Z m Z eeZGdddeZGd d d eZGd d d eZGd ddeZGdddeZGdddeZdS))tee)sixN)PaginationError)zip)set_value_from_jmespath merge_dictsc@s8eZdZdZddZddZddZdd Zd d Zd S) TokenEncoderabEncodes dictionaries into opaque strings. This for the most part json dumps + base64 encoding, but also supports having bytes in the dictionary in addition to the types that json can handle by default. This is intended for use in encoding pagination tokens, which in some cases can be complex structures and / or contain bytes. c Cs^zt|}Wnttfy#||g\}}||d<t|}Ynwt|ddS)a.Encodes a dictionary to an opaque string. :type token: dict :param token: A dictionary containing pagination information, particularly the service pagination token(s) but also other boto metadata. :rtype: str :returns: An opaque string boto_encoded_keysutf-8) jsondumps TypeErrorUnicodeDecodeError_encodebase64 b64encodeencodedecode)selftoken json_string encoded_token encoded_keysr3/usr/lib/python3/dist-packages/botocore/paginate.pyr)s  zTokenEncoder.encodecCsLt|tr |||St|tr|||St|tjr"|||S|gfS)z@Encode bytes in given data, keeping track of the path traversed.) isinstancedict _encode_dictlist _encode_listr binary_type _encode_bytesrdatapathrrrrFs      zTokenEncoder._encodec CsPg}g}t|D]\}}||g}|||\}} |||| q||fS)z@Encode any bytes in a list, noting the index of what is encoded.) enumeraterappendextend) rr#r$new_dataencodedivaluenew_path new_value new_encodedrrrrQs   zTokenEncoder._encode_listc CsNi}g}|D]\}}||g}|||\}} |||<|| q||fS)z@Encode any bytes in a dict, noting the index of what is encoded.)itemsrr') rr#r$r(r)keyr+r,r-r.rrrr\s  zTokenEncoder._encode_dictcCst|d|gfS)zBase64 encode a byte string.r )rrrr"rrrr!gszTokenEncoder._encode_bytesN) __name__ __module__ __qualname____doc__rrrrr!rrrrrs   rc@s0eZdZdZddZddZddZdd Zd S) TokenDecoderzDecodes token strings back into dictionaries. This performs the inverse operation to the TokenEncoder, accepting opaque strings and decoding them into a useable form. cCsDt|dd}t|}|dd}|dur|S|||S)adDecodes an opaque string to a dictionary. :type token: str :param token: A token string given by the botocore pagination interface. :rtype: dict :returns: A dictionary containing pagination information, particularly the service pagination token(s) but also other boto metadata. r r N)r b64decoderrr loadspop_decode)rrr decoded_tokenrrrrrss    zTokenDecoder.decodecCs8|D]}|||}t|d}||||q|S)z&Find each encoded value and decode it.r ) _path_getrr6r _path_set)rrrr0r)decodedrrrr9s  zTokenDecoder._decodecCs|}|D]}||}q|S)zReturn the nested data at the given path. For instance: data = {'foo': ['bar', 'baz']} path = ['foo', 0] ==> 'bar' r)rr#r$dsteprrrr;s  zTokenDecoder._path_getcCs$|||dd}|||d<dS)zSet the value of a key in the given data. Example: data = {'foo': ['bar', 'baz']} path = ['foo', 1] value = 'bin' ==> data = {'foo': ['bar', 'bin']} N)r;)rr#r$r+ containerrrrr<s zTokenDecoder._path_setN)r1r2r3r4rr9r;r<rrrrr5ls  r5c@seZdZddZddZdS)PaginatorModelcCs|d|_dS)N pagination)_paginator_config)rpaginator_configrrr__init__szPaginatorModel.__init__cCs,z|j|}W|Stytd|w)Nz*Paginator for operation does not exist: %s)rDKeyError ValueError)roperation_namesingle_paginator_configrrr get_paginators  zPaginatorModel.get_paginatorN)r1r2r3rFrKrrrrrBs rBc@seZdZddZeddZeddZejddZedd Zd d Z d d Z ddZ ddZ ddZ ddZddZddZddZddZddZd d!Zd"d#Zd$d%Zd&d'Zd(S)) PageIteratorc Csb||_||_||_||_||_||_||_| |_| |_| |_ d|_ ||_ i|_ t |_t|_dSN)_method _input_token _output_token _more_results _result_keys _max_items _limit_key_starting_token _page_size _op_kwargs _resume_token_non_aggregate_key_exprs_non_aggregate_partr_token_encoderr5_token_decoder) rmethod input_token output_token more_results result_keysnon_aggregate_keys limit_key max_itemsstarting_token page_size op_kwargsrrrrFs zPageIterator.__init__cC|jSrMrRrrrrrazPageIterator.result_keyscCrh)z&Token to specify to resume pagination.)rXrjrrr resume_tokenszPageIterator.resume_tokencCslt|ts td|d|vrt|jdg}nt|j}t|}||kr0|j||_dStd|)NBad starting token: %sboto_truncate_amount) rrrHsortedrOkeysr[rrX)rr+ token_keys dict_keysrrrrls     cCrhrM)rZrjrrrnon_aggregate_partrkzPageIterator.non_aggregate_partccs~|j}d}tdd|jD}|jdur|d}d}d}|jd}d}|| ||}||} |rL|jdurD| | ||}d}| | nd}| | } | durYg} t | } d} |j durk|| |j } | dkr}|| || |||VdS|V|| 7}|| }tdd|DrdS|j dur||j kr||_dS|dur||krd|} t| d||||}q,) Ncss|]}|dfVqdSrMr).0r0rrr z(PageIterator.__iter__..rTFcss|]}|duVqdSrMr)rttrrrru#rvz*The same next token was received twice: %smessage)rWrrOrU_parse_starting_tokenra_inject_starting_params _make_request_extract_parsed_response_handle_first_request _record_non_aggregate_key_valuessearchlenrS_truncate_response_get_next_tokenallvaluesrlr_inject_token_into_kwargs)rcurrent_kwargsprevious_next_token next_token total_items first_requestprimary_result_keystarting_truncationresponseparsedcurrent_responsenum_current_responsetruncate_amountryrrr__iter__sn                zPageIterator.__iter__ccsFt|}|D]}||}t|tr|D]}|Vqq|VqdS)aApplies a JMESPath expression to a paginator Each page of results is searched using the provided JMESPath expression. If the result is not a list, it is yielded directly. If the result is a list, each element in the result is yielded individually (essentially implementing a flatmap in which the JMESPath search is the mapping function). :type expression: str :param expression: JMESPath expression to apply to each page. :return: Returns an iterator that yields the individual elements of applying a JMESPath expression to each page of results. N)jmespathcompilerrr)r expressioncompiledpageresultselementrrrr3s   zPageIterator.searchcCs|jdi|S)Nr)rN)rrrrrr|MszPageIterator._make_requestcCs|SrMr)rrrrrr}Psz%PageIterator._extract_parsed_responsecCs2i}|jD]}||}t||j|q||_dSrM)rYrrrrZ)rrrbrresultrrrrSs   z-PageIterator._record_non_aggregate_key_valuescCs@|jdur|d}||||jdur|j||j<dSdS)Nr)rUrzrrVrT)rrgrrrrr{\s    z$PageIterator._inject_starting_paramscCs>|D]\}}|dur|dkr|||<q||vr||=qdS)NNone)r/)rrgrnamerrrrris z&PageIterator._inject_token_into_kwargsc Cs|d}||}t|ttjfr||d}nd}t||j||jD]0}||kr-q&||}t|tr:g}nt|tjrCd}n t|t t frMd}nd}t||j|q&|S)Nr) rzrrrr string_typesrrraintfloat) rrrrall_datar#rsample empty_valuerrrr~ps.      z"PageIterator._handle_first_requestc CsR||}|dur g}t||}|d|}t||j||||d<||_dS)Nrn)rrrrrl) rrrrrroriginalamount_to_keep truncatedrrrrs    zPageIterator._truncate_responsecCsZ|jdur |j|s iSi}t|j|jD]\}}||}|r&|||<qd||<q|SrM)rQrrrPrO)rr next_tokensr_ input_keyrrrrrs      zPageIterator._get_next_tokencCs&t|t|j}ddt||jDS)NcSsg|] \}}t||qSr)ResultKeyIterator)rtr* result_keyrrr sz1PageIterator.result_key_iters..)rrrar)r teed_resultsrrrresult_key_iterss zPageIterator.result_key_iterscCsi}|D]Q}|}t|trt|dkr|d}|jD]:}||}|dur&q||}|dur7t||j|qt|trB||qt|t t t j frTt||j||qqt ||j|jdurf|j|d<|S)Nr NextToken)rtuplerrarrrrr'rrrrrrsrl)rcomplete_resultrrresult_expression result_valueexisting_valuerrrbuild_full_results:        zPageIterator.build_full_resultc Csz|jdurdS|j}z|j|}d}d|vr$|d}|d=W||fSW||fSttfy<|\}}Y||fSw)Nrrn)rUr\rgetrHr _parse_starting_token_deprecated)rrindexrrrrzs   z"PageIterator._parse_starting_tokencCstd|j|jdurdS|jd}g}d}t|t|jdkr:zt|}Wn ty9|jg}Ynw|D]}|dkrH| dq<| |q<| ||fS)z| This handles parsing of old style starting tokens, and attempts to coerce them into the new style. zCAttempting to fall back to old starting token parser. For token: %sN___rrr) logdebugrUsplitrrOrr8rHr&"_convert_deprecated_starting_token)rpartsrrpartrrrrs&      z-PageIterator._parse_starting_token_deprecatedcCsft|}t|j}||krtd|j||kr+tdt||D]}|dq#tt |j|S)zb This attempts to convert a deprecated starting token into the new style. rmzaOld format starting token does not contain all input tokens. Setting the rest, in order, as None.N) rrOrHrUrrranger&rr)rdeprecated_tokenlen_deprecated_tokenlen_input_tokenr*rrrrs   z/PageIterator._convert_deprecated_starting_tokenN)r1r2r3rFpropertyrarlsetterrsrrr|r}rr{rr~rrrrrzrrrrrrrLs2    E  !- rLc@sdeZdZeZddZeddZddZddZ d d Z d d Z d dZ ddZ ddZddZdS) PaginatorcCsj||_||_||_||j|_||j|_||j|_| |j|_ | |j|_ | |j|_dSrM)_modelrN_pagination_cfg_get_output_tokensrP_get_input_tokensrO_get_more_results_tokenrQ_get_non_aggregate_keys_non_aggregate_keys_get_result_keysrR_get_limit_keyrT)rr]pagination_configmodelrrrrF.szPaginator.__init__cCrhrMrirjrrrra:rkzPaginator.result_keyscCs*g}|dgD] }|t|q|S)Nrb)rr&rr)rconfigrpr0rrrr>sz!Paginator._get_non_aggregate_keyscCs:g}|d}t|ts|g}|D] }|t|q|S)Nr_)rrr&rr)rroutputr_rrrrDs zPaginator._get_output_tokenscCs|jd}t|ts |g}|S)Nr^)rrr)rrr^rrrrMs  zPaginator._get_input_tokenscCs |d}|durt|SdS)Nr`)rrr)rrr`rrrrSs  z!Paginator._get_more_results_tokencCs8|d}|durt|ts|g}dd|D}|SdS)NrcSsg|]}t|qSr)rr)rtrkrrrr]sz.Paginator._get_result_keys..)rrr)rrrrrrrXs  zPaginator._get_result_keyscCs |dS)Nrc)r)rrrrrr`s zPaginator._get_limit_keyc KsB||}||j|j|j|j|j|j|j|d|d|d| S)zCreate paginator object for an operation. This returns an iterable object. Iterating over this object will yield a single page of a response at a time. MaxItems StartingTokenPageSize) _extract_paging_paramsPAGE_ITERATOR_CLSrNrOrPrQrRrrT)rkwargs page_paramsrrrpaginatecs zPaginator.paginatecCs|di}|dd}|durt|}|dd}|durG|jdur(tdd|jjj}||j}|jdkrCt |t j sBt |}nt|}||dd|dS) NPaginationConfigrrzTPageSize parameter is not supported for the pagination interface for this operation.rxstringr)rrr) r8rrrTrr input_shapemembers type_namerrrstr)rrrrdrf input_memberslimit_key_shaperrrrvs*         z Paginator._extract_paging_paramsN)r1r2r3rLrrFrrarrrrrrrrrrrrr+s   rc@s eZdZdZddZddZdS)raIterates over the results of paginated responses. Each iterator is associated with a single result key. Iterating over this object will give you each element in the result key list. :param pages_iterator: An iterator that will give you pages of results (a ``PageIterator`` class). :param result_key: The JMESPath expression representing the result key. cCs||_||_dSrM)_pages_iteratorr)rpages_iteratorrrrrrFs zResultKeyIterator.__init__ccs:|jD]}|j|}|durg}|D]}|VqqdSrM)rrr)rrrrrrrrs  zResultKeyIterator.__iter__N)r1r2r3r4rFrrrrrrs r) itertoolsrbotocore.compatrrr rloggingbotocore.exceptionsrrbotocore.utilsrr getLoggerr1robjectrr5rBrLrrrrrrs$    ND pd