qgh=dZddlZddlZddlZddlZddlZddlZddlmZddl m Z ddl m Z m Z ddlmZddlmZ ddlZdZGd d e Zy#e$rd ZYwxYw) a Sync Media to S3 ================ Django command that scans all files in your settings.MEDIA_ROOT and settings.STATIC_ROOT folders and uploads them to S3 with the same directory structure. This command can optionally do the following but it is off by default: * gzip compress any CSS and Javascript files it finds and adds the appropriate 'Content-Encoding' header. * set a far future 'Expires' header for optimal caching. * upload only media or static files. * use any other provider compatible with Amazon S3. * set other than 'public-read' ACL. Note: This script requires the Python boto library and valid Amazon Web Services API keys. Required settings.py variables: AWS_ACCESS_KEY_ID = '' AWS_SECRET_ACCESS_KEY = '' AWS_BUCKET_NAME = '' When you call this command with the `--renamegzip` param, it will add the '.gz' extension to the file name. But Safari just doesn't recognize '.gz' files and your site won't work on it! To fix this problem, you can set any other extension (like .jgz) in the `SYNC_S3_RENAME_GZIP_EXT` variable. Command options are: -p PREFIX, --prefix=PREFIX The prefix to prepend to the path on S3. --gzip Enables gzipping CSS and Javascript files. --expires Enables setting a far future expires header. --force Skip the file mtime check to force upload of all files. --filter-list Override default directory and file exclusion filters. (enter as comma separated line) --renamegzip Enables renaming of gzipped files by appending '.gz'. to the original file name. This way your original assets will not be replaced by the gzipped ones. You can change the extension setting the `SYNC_S3_RENAME_GZIP_EXT` var in your settings.py file. --invalidate Invalidates the objects in CloudFront after uploading stuff to s3. --media-only Only MEDIA_ROOT files will be uploaded to S3. --static-only Only STATIC_ROOT files will be uploaded to S3. --s3host Override default s3 host. --acl Override default ACL settings ('public-read' if settings.AWS_DEFAULT_ACL is not defined). TODO: * Use fnmatch (or regex) to allow more complex FILTER_LIST rules. N)List)settings) BaseCommand CommandError)StringIO) signalcommandTFceZdZdZdZdZdZdZdZgdZ dZ gZ dZ dZ dZdZdZfdZed Zd Zd Zd Zd ZdZdZdZxZS)Command)z .DS_Storez.svnz.hgz.gitz Thumbs.db)ztext/csszapplication/javascriptzapplication/x-javascriptztext/javascriptrzSSyncs the complete MEDIA_ROOT structure and files to S3 into the given bucket name. bucket_nameTc t|||jdddttddd|jdd d d |jd dttddd|jddttddd|jddddd|jddddd|jddd dd!|jd"dd#dd$|jd%d&d'dd()|jd*d+ddd,-|jd.d/ddd0-|jd1d2ddd3-y)4Nz-pz--prefixprefixSYNC_S3_PREFIXr z(The prefix to prepend to the path on S3.)destdefaulthelpz-dz--dirdirz#Custom static root directory to use)rrz--s3hosts3host AWS_S3_HOSTz;The s3 host (enables connecting to other providers/regions)z--aclaclAWS_DEFAULT_ACLz public-readz.Enables to override default acl (public-read).z--gzip store_truegzipFz*Enables gzipping CSS and Javascript files.)actionrrrz --renamegzip renamegzipzJEnables renaming of gzipped assets to have '.gz' appended to the filename.z --expiresexpiresz,Enables setting a far future expires header.z--forceforcez7Skip the file mtime check to force upload of all files.z --filter-list filter_liststorezVOverride default directory and file exclusion filters. (enter as comma seperated line))rrrrz --invalidate invalidatez0Invalidates the associated objects in CloudFront)rrrrz --media-only media_onlyz,Only MEDIA_ROOT files will be uploaded to S3z --static-only static_onlyz-Only STATIC_ROOT files will be uploaded to S3)super add_arguments add_argumentgetattrr)selfparser __class__s k/var/www/skyplay_api_hub/venv/lib/python3.12/site-packages/django_extensions/management/commands/sync_s3.pyr$zCommand.add_argumentsls f% *H&6;;   '6   HmR8N   H&7GA   fe=   lE]   i?   guJ   -Bi   uC   r?   -@  cts tdttdrttds tdtj|_tj |_ttds tdtj s tdtj |_ttds td tjs td ttd d |_ ttd d |_ |d|_ |d|_ |d|_ |d|_|d|_|d|_|d|_|d|_|d|_|d|_ttd|j*|_|d}|r|j-d|_|d|_|d|_|j.r|j0r td|j.rtjg|_ng|j0rtj2g|_nD|j$r|j$g|_n%tjtj2g|_|j5|j"r|j7t9d t9d|j:zt9d|j<zy) Nz>Please install the 'boto' Python library. ($ pip install boto)AWS_ACCESS_KEY_IDAWS_SECRET_ACCESS_KEYzeMissing AWS keys from settings file. Please supply both AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.AWS_BUCKET_NAMEz]Missing bucket name from settings file. Please add the AWS_BUCKET_NAME to your settings file.z AWS_BUCKET_NAME cannot be empty. MEDIA_ROOTz(MEDIA_ROOT must be set in your settings.AWS_CLOUDFRONT_DISTRIBUTIONr SYNC_S3_RENAME_GZIP_EXTz.gz verbosityrrrrrr rrr FILTER_LISTr,r!r"zMCan't use --media-only and --static-only together. Better not use anything...z%d files uploaded.z%d files skipped.)HAS_BOTOrhasattrrr-r.r/r0r&r1r2r3rdo_gzip rename_gzip do_expiresdo_forcer DIRECTORIESr default_aclr4splitr!r" STATIC_ROOTsync_s3invalidate_objects_cfprint upload_count skip_count)r'argsoptionsrs r*handlezCommand.handles_` `x!45WXOf=g GH H%-%?%?D ")1)G)GD &x!23~ ++"#EFF'77x.IJ J&&"#MNN+28=Z\^+_( H7 ? $!-h' v "<0!),( !,/"5>h' "5>"8]D$>@ @r+c|js tdd}|j}|j}t dt ||Dcgc] }||||z }}|D]}|j |j| ycc}w)z8Split the invalidation request in groups of 1000 objectszrAn object invalidation was requested but the variable AWS_CLOUDFRONT_DISTRIBUTION is not present in your settings.irN)r1rrKuploaded_filesrangelencreate_invalidation_request)r'chunkconnobjsichunkspathss r*rAzCommand.invalidate_objects_cfs//OP P ||~""-21c$i-GH$qU#HH 9E  , ,00% 9 9IsBc|j\}}|jD]B}tj|D](\}}}|j |||j |f|||*Dy)z7Walk the media/static directories and syncs files to S3N)open_s3r<oswalk upload_s3r/)r'bucketkey directoryrootdirsfiless r*r@zCommand.sync_s3sllln )) bI%'WWY%7 b!dET-A-A9MtUZ\`a b br+ct}tjdd|}|j||j |j S)zGzip a given string.wb)mode compresslevelfileobj)rrGzipFilewriteclosegetvalue)r'szbufzfiles r*compress_stringzCommand.compress_strings;z 4q$G A }}r+c@i}|jr|j|d<|S)z"Return connection kwargs as a dicthost)r)r'kwargss r*get_s3connection_kwargszCommand.get_s3connection_kwargs#s  ;;![[F6N r+c~tj|j|jfi|j } |j |j }|tjjj|fS#tjj$r|j|j }YewxYw)z.Open connection to S3 returning bucket and key) rI connect_s3r-r.rs get_bucketr/ exceptionS3ResponseError create_buckets3r]Key)r'rRr\s r*rXzCommand.open_s3*s  " "  & &.**,. >__T%9%9:Ftww{{v...~~-- >''(<(<=F >sB8B<;B<c |\}}}}tjj||jvr1tjj ||j vr|dd=y|j tjjs|tjjz}|D]v} i} | |jvrtjj|| } tjj| rU| t|d} |jr|jd| } |js|j| } | rtjtj | j"ddd}tjj%tj&| j(}||kr4|xj*dz c_|j,dkDrt/d| zG|j,dkDrt/d| zt1j2| d}|r|| d<nd | d<t5| d }tj6|j9j:}|j=}|j>rt|d kDro||j@vra|jC|}|jDr| d |jF} d | d<|j,dkDrt/d|d z t|d z fz|jHrdtJjLjOtjPtjjStjTdzjWz| d<ddz| d<|j,dkDr"t/d| dzt/d| dz | |_,|j[|| d|j\|xj^dz c_/|j`jc| |jmyy#tdjfjh$r}t/d|zYd}~Dd}~wtj$r}t/|d}~wwxYw)N/z%a, %d %b %Y %H:%M:%S %Zrrdz6File %s hasn't been modified since last being uploadedzUploading %s...z Content-Typezapplication/octet-streamrbi.rzContent-Encodingz gzipped: %dk to %dkz%s GMTi)daysExpiresz max-age %digz Cache-Controlz expires: %sz cache-control: %sT)replacepolicyz Failed: %s)7rYpathbasenamer4dirnamer<endswithsepjoinisdirrOrr;get_keydatetimetimestrptime last_modifiedutcfromtimestampstatst_mtimerDr3rB mimetypes guess_typeopenfstatfilenost_sizereadr8GZIP_CONTENT_TYPESror9r2r:emailUtils formatdatemktimenow timedelta timetuplenameset_contents_from_stringr=rCrMappendrIrw S3CreateError Exceptionrj)r'argrnamesr`r\r]r root_dirfileheadersfilenamefile_keys3_key s3_datetimelocal_datetime content_typefile_obj file_sizefiledataes r*r[zCommand.upload_s36s-0*[( 77  G $(8(8 8RWW__W=UY]YiYi=iQ   -"''++-HM DGt'''ww||GT2Hww}}X&H /H{{&*kk8<==1"*"3"3T]],,.H6JJKA6O#PK%-%6%6%G%G)22&4N% 31,>>A-!"Z]e"ef ~~!'(23$//9!>A%/GI,>>?/'/2JJK 5#,,Xw484D4D-F!!Q&!##**84 NN [M H>>// (lQ&'' a s$=&P''Q4Q Q4# Q//Q4)__name__ __module__ __qualname__r-r.r/r1r2r<r4rrMrCrDrrEcan_import_settingsr$rrGrKrAr@rorsrXr[ __classcell__)r)s@r*r r RsO"$ KCKNLJ `D D@ DG5G5R@ 9.b /Zr+r )__doc__rrrrrYrtypingr django.confrdjango.core.management.baserrior"django_extensions.management.utilsrrIr6 ImportErrorr r+r*rsa8r   A<H~k~ HsAAA