o .&ah@s8ddlmZddlZddlZddlZddlZddlmZddlmZddl m Z ddl m Z ddl m Z ddlmZmZdd lmZdd lmZdd lmZdd lmZdd lmZeeZedgdZdefddZeddgZedgdZedZeddgZ edZ!eddgZ"ede"dZ#edddgZ$ed d!gZ%Gd"d#d#e&Z'Gd$d%d%eZ(Gd&d'd'e(Z)Gd(d)d)e)Z*Gd*d+d+e(Z+Gd,d-d-e+Z,Gd.d/d/e(Z-Gd0d1d1e(Z.Gd2d3d3e&Z/Gd4d5d5e/Z0Gd6d7d7e/Z1Gd8d9d9e1Z2Gd:d;d;e1Z3Gdd?d?e&Z6dS)@)divisionN) namedtuple) defaultdict)CancelledError) FatalError)BaseSubscriber)queueensure_text_type relative_path)human_readable_size) uni_print) WarningResult)OnDoneFilteredSubscriber BaseResult) transfer_typesrcdestcCs,t|j}|r ||7}t|t|||fiSN)list_fieldstyper)name extra_fieldsbase_clsfieldsrB/usr/lib/python3/dist-packages/awscli/customizations/s3/results.py_create_new_result_cls's r QueuedResulttotal_transfer_sizeProgressResult)bytes_transferredr timestamp SuccessResult FailureResult exception DryRunResult ErrorResult CtrlCResult)r CommandResultnum_tasks_failednum_tasks_warnedFinalTotalSubmissionsResulttotal_submissionsc@s eZdZdS)ShutdownThreadRequestN)__name__ __module__ __qualname__rrrrr/Ksr/c@sReZdZdZdddZddZddZdd Zd d Zd d Z ddZ ddZ dS)BaseResultSubscriberNcCs*||_i|_||_|dur|j|_dSdS)zSubscriber to send result notifications during transfer process :param result_queue: The queue to place results to be processed later on. N) _result_queue_result_kwargs_cache_transfer_type TRANSFER_TYPE)self result_queuerrrr__init__Rs  zBaseResultSubscriber.__init__cKs6|||j|jj}tdi|}|j|dSNr)_add_to_result_kwargs_cacher5meta transfer_idrr4put)r8futurekwargs result_kwargs queued_resultrrr on_queued^s zBaseResultSubscriber.on_queuedcKs6|j|jj}td|td|}|j|dS)N)r"r#r)r5r=r>r!timer4r?)r8r@r"rArBprogress_resultrrr on_progressdsz BaseResultSubscriber.on_progresscCs$||}|jtdi|dSr;)%_on_done_pop_from_result_kwargs_cacher4r?r$r8r@rBrrr _on_successks z BaseResultSubscriber._on_successcCsZ||}t|trt}t|trt}|j||ddS|jtdd|i|dS)Nr&r&r) rH isinstancerr)rr(r4r?r%)r8r@erBerror_result_clsrrr _on_failureos   z BaseResultSubscriber._on_failurecCs4||\}}|j|||jjd}||j|jj<dS)N)rrrr ) _get_src_destr6r=sizer5r>)r8r@rrrBrrrr<ysz0BaseResultSubscriber._add_to_result_kwargs_cachecCs|j|jj}|d|S)Nr )r5popr=r>rIrrrrHs z:BaseResultSubscriber._on_done_pop_from_result_kwargs_cachecCtd)Nz_get_src_dest()NotImplementedError)r8r@rrrrPz"BaseResultSubscriber._get_src_destr) r0r1r2r7r:rDrGrJrOr<rHrPrrrrr3Os    r3c@ eZdZdZddZddZdS)UploadResultSubscriberuploadcCs0|jj}||j}d|jd|j}||fSNs3:///)r= call_args_get_srcfileobjbucketkeyr8r@r]rrrrrrPs z$UploadResultSubscriber._get_src_destcCt|Srr r8r_rrrr^rVzUploadResultSubscriber._get_srcN)r0r1r2r7rPr^rrrrrX rXc@eZdZddZdS)UploadStreamResultSubscribercCdSN-rrdrrrr^z%UploadStreamResultSubscriber._get_srcN)r0r1r2r^rrrrrg rgc@rW)DownloadResultSubscriberdownloadcCs0|jj}d|jd|j}||j}||fSrZ)r=r]r`ra _get_destr_rbrrrrPs z&DownloadResultSubscriber._get_src_destcCrcrr rdrrrrorVz"DownloadResultSubscriber._get_destN)r0r1r2r7rProrrrrrmrermc@rf)DownloadStreamResultSubscribercCrhrirrdrrrrorkz(DownloadStreamResultSubscriber._get_destN)r0r1r2rorrrrrprlrpc@eZdZdZddZdS)CopyResultSubscribercopycCsB|jj}|j}d|dd|d}d|jd|j}||fS)Nr[Bucketr\Key)r=r] copy_sourcer`ra)r8r@r]rvrrrrrrPs z"CopyResultSubscriber._get_src_destNr0r1r2r7rPrrrrrr rrc@rq)DeleteResultSubscriberdeletecCs$|jj}d|jd|j}|dfSrZ)r=r]r`ra)r8r@r]rrrrrPsz$DeleteResultSubscriber._get_src_destNrwrrrrryrxryc@rq)BaseResultHandlerz6Base handler class to be called in the ResultProcessorcCrS)Nz __call__()rTr8resultrrr__call__rVzBaseResultHandler.__call__N)r0r1r2__doc__r~rrrrr{ r{c@seZdZdZddZddZddZdd Zd d Zd d Z ddZ ddZ ddZ ddZ ddZddZddZddZdS)ResultRecorderz?Records and track transfer statistics based on results receivedcCsd|_d|_d|_d|_d|_d|_d|_d|_d|_d|_ d|_ t t |_ i|_t|jt|jt|jt|jt|jt|jt|jt|ji|_dSNr)r"bytes_failed_to_transferfiles_transferred files_failed files_warnederrorsexpected_bytes_transferredexpected_files_transferred final_expected_files_transferred start_timebytes_transfer_speedrint_ongoing_progress_ongoing_total_sizesr_record_queued_resultr!_record_progress_resultr$_record_success_resultr%_record_failure_resultr_record_warning_resultr(_record_error_resultr)r-_record_final_expected_files_result_handler_mapr8rrrr:s,  zResultRecorder.__init__cCs |j|jkSr)rrrrrrexpected_totals_are_finalsz(ResultRecorder.expected_totals_are_finalcC|jt||j|ddS)z0Record the result of an individual Result objectr}N)rgetr _record_noopr|rrrr~ zResultRecorder.__call__cCsTt|ts tdt|g}|j|j|jfD] }|dur$|t|qd |S)NzdAny result using _get_ongoing_dict_key must subclass from BaseResult. Provided result is of type: %s:) rLr ValueErrorrrrrappendr join)r8r} key_partsresult_propertyrrr_get_ongoing_dict_keys  z$ResultRecorder._get_ongoing_dict_keycCs.||}|j|d}|j|d}||fSr)rrrRr)r8r} ongoing_keytotal_progresstotal_file_sizerrr_pop_result_from_ongoing_dictss z-ResultRecorder._pop_result_from_ongoing_dictscKdSrrr8rArrrrzResultRecorder._record_noopcKsN|jdur t|_|j}||j||<|r|j|7_|jd7_dSN)rrEr rrrr)r8r}rAr rrrr s  z$ResultRecorder._record_queued_resultcKs^|j}|||j|||7<|j|7_|j|jkr-|j|j|j|_dSdSr)r"(_update_ongoing_transfer_size_if_unknownrrr#rr)r8r}rAr"rrrrs    z&ResultRecorder._record_progress_resultcCsn||}|j|dur5|j}|jdur+||j|<|j|}||}|j|7_dS|j|j7_dSdSr)rrr rrr")r8r}rr ongoing_progressunaccounted_bytesrrrr&s    z7ResultRecorder._update_ongoing_transfer_size_if_unknowncKs|||jd7_dSr)rrr8r}rArrrr@s z%ResultRecorder._record_success_resultcKsL||\}}|dur||}|j|7_|jd7_|jd7_dSr)rrrr)r8r}rArr progress_leftrrrrDsz%ResultRecorder._record_failure_resultcK|jd7_dSr)rrrrrrQz%ResultRecorder._record_warning_resultcKrr)rrrrrrTrz#ResultRecorder._record_error_resultcKs |j|_dSr)r.rrrrrrWs z+ResultRecorder._record_final_expected_filesN)r0r1r2rr:rr~rrrrrrrrrrrrrrrrs     rc@seZdZdZdZdZdeZdeZdZdeZ dZ d Z d Z d Z d Zd Zd8ddZddZddZddZddZddZddZddZdd Zd!d"Zd#d$Zd%d&Zd'd(Zd)d*Zd+d,Zd9d.d/Zd0d1Z d2d3Z!d4d5Z"d6d7Z#dS): ResultPrinterz#{remaining_files} file(s) remainingz~{expected_total}z (calculating...)zOCompleted {bytes_completed}/{expected_bytes_completed} ({transfer_speed}) with z)Completed {files_completed} file(s) with z${transfer_type}: {transfer_location}z (dryrun) z7{transfer_type} failed: {transfer_location} {exception}z {message}zfatal error: {exception}zcancelled: ctrl-c receivedz{src} to {dest}z{src}NcCsv||_||_|jdurtj|_||_|jdurtj|_d|_t|jt |j t |j t |jt|jt|jt|jt|ji|_dS)aPrints status of ongoing transfer :type result_recorder: ResultRecorder :param result_recorder: The associated result recorder :type out_file: file-like obj :param out_file: Location to write progress and success statements. By default, the location is sys.stdout. :type error_file: file-like obj :param error_file: Location to write warnings and errors. By default, the location is sys.stderr. Nr)_result_recorder _out_filesysstdout _error_filestderr_progress_lengthr!_print_progressr$_print_successr%_print_failurer_print_warningr( _print_errorr) _print_ctrl_cr'_print_dry_runr--_clear_progress_if_no_more_expected_transfersr)r8result_recorderout_file error_filerrrr:zs$   zResultPrinter.__init__cCr)z|rtjd|dd|jt|d|dS|dS)Nz-Exception caught during command execution: %sTr rK)rrr9r?r(r)r8exc_type exc_valueargsrrr__exit__s zCommandResultRecorder.__exit__N) r0r1r2r:rrrrrrrrrrrls  r)7 __future__rloggingrrrE collectionsrrs3transfer.exceptionsrrs3transfer.subscribersr awscli.compatrr awscli.customizations.s3.utilsr r awscli.customizations.utilsr rr getLoggerr0rrrrr!r$r%r'r(r)r*r-objectr/r3rXrgrmrprrryr{rrrrrrrrrrrsf                =    S 0