curl - Proxying PUT HTTP requests to AWS S3 fails -


i working on creating proxy relay http requests amazon s3. proxy uses nginx right ngx_aws_auth module adds necessary headers http requests client not need to.

nginx configuration:

http {     include       mime.types;     default_type  application/octet-stream;     sendfile        on;     client_body_in_file_only clean;     client_body_buffer_size 32k;      client_max_body_size 300m;      send_timeout 300s;      keepalive_timeout  65;      gzip  on;   server {     listen     8000;      location / {       proxy_pass https://test-bucket.s3.amazonaws.com;        aws_access_key xxxxx;       aws_secret_key yyyyyy;       s3_bucket test-bucket;        proxy_set_header authorization $s3_auth_token;       proxy_set_header x-amz-date $aws_date;     }   } }

almost except when trying upload larger file.

get:

$ curl -vx http://192.168.99.131:8000/a.txt ; echo * connect() 192.168.99.131 port 8000 (#0) *   trying 192.168.99.131... * connected 192.168.99.131 (192.168.99.131) port 8000 (#0) > /a.txt http/1.1 > user-agent: curl/7.29.0 > host: 192.168.99.131:8000 > accept: */* > < http/1.1 200 ok < server: nginx/1.6.3 < date: sun, 30 aug 2015 18:18:50 gmt < content-type: application/x-www-form-urlencoded < content-length: 1 < connection: keep-alive < x-amz-id-2: 1f/eqtkju= < x-amz-request-id: a970d9bcf5d5890b < last-modified: sat, 29 aug 2015 00:58:33 gmt < etag: "0cc175b9c0f1b6a831c399e269772661" < accept-ranges: bytes < * connection #0 host 192.168.99.131 left intact

put small file:

$ echo 'a'>a;curl -vx put -d @a http://192.168.99.131:8000/a.txt * connect() 192.168.99.131 port 8000 (#0) *   trying 192.168.99.131... * connected 192.168.99.131 (192.168.99.131) port 8000 (#0) > put /a.txt http/1.1 > user-agent: curl/7.29.0 > host: 192.168.99.131:8000 > accept: */* > content-length: 1 > content-type: application/x-www-form-urlencoded > * upload sent off: 1 out of 1 bytes < http/1.1 200 ok < server: nginx/1.6.3 < date: sun, 30 aug 2015 18:20:03 gmt < content-length: 0 < connection: keep-alive < x-amz-id-2: gf/hvpsr+r2r/j7qquf0= < x-amz-request-id: f499d02f2d28b6eb < etag: "0cc175b9c0f1b6a831c399e269772661" < * connection #0 host 192.168.99.131 left intact

put larger file without adjusting header:

curl -vvvx put -d @1408133479717.jpg http://192.168.99.131:8000/a.jpg * connect() 192.168.99.131 port 8000 (#0) *   trying 192.168.99.131... * connected 192.168.99.131 (192.168.99.131) port 8000 (#0) > put /a.jpg http/1.1 > user-agent: curl/7.29.0 > host: 192.168.99.131:8000 > accept: */* > content-length: 320356 > content-type: application/x-www-form-urlencoded > expect: 100-continue > < http/1.1 100 continue < http/1.1 200 ok < server: nginx/1.6.3 < date: sun, 30 aug 2015 18:33:45 gmt < content-length: 0 < connection: keep-alive < x-amz-id-2: im+nmggfzccp+/mfldwdr/zwjfs= < x-amz-request-id: 3b86e54874009d73 < etag: "8d04fbe9d623bf03bb828594ca272063" < * connection #0 host 192.168.99.131 left intact

this uploads partial file size of 320356 bytes, original file 825210 bytes.

put larger file , adjusting header:

[vagrant@gripper nginx]$ curl -vvvx put -h "content-length: 825210" -d @1408133479717.jpg http://192.168.99.131:8000/a.jpg * connect() 192.168.99.131 port 8000 (#0) *   trying 192.168.99.131... * connected 192.168.99.131 (192.168.99.131) port 8000 (#0) > put /a.jpg http/1.1 > user-agent: curl/7.29.0 > host: 192.168.99.131:8000 > accept: */* > content-length: 825210 > content-type: application/x-www-form-urlencoded > expect: 100-continue > < http/1.1 100 continue * empty reply server * connection #0 host 192.168.99.131 left intact curl: (52) empty reply server

my question is:

how can upload file correct size using curl?


-

Comments

Post a Comment

Popular posts from this blog

c# - Binding a comma separated list to a List<int> in asp.net web api -

the default binder in web api expecst http://url.com/webapi/report/?pageids=3243&pageids=2365 to bind public ihttpactionresult report(list<int> pageids){ // exciting webapi code} i wish bind http://url.com/webapi/report/?pageids=3243,2365 as running out of space in url get . i have created class public class commaseparatedmodelbinder : system.web.http.modelbinding.imodelbinder { public bool bindmodel(httpactioncontext actioncontext, modelbindingcontext bindingcontext) { //binding in here } } and registered in webapiconfig.cs var provider = new simplemodelbinderprovider( typeof(list<int>), new commaseparatedmodelbinder()); config.services.insert(typeof(modelbinderprovider), 0, provider); i have altered method signature use model binder so public ihttpactionresult report( [modelbinder] list<int> pageids){ // exciting webapi code} however break point in binder not being
Read more

Delphi 7 and decode UTF-8 base64 -

in delphi 7, have widestring encoded base64(that received web service widestring result) : pd94bwwgdmvyc2lvbj0ims4wij8+dqo8c3ryaw5nptiq2lpyqjwvc3ryaw5npg== when decoded it, result not utf-8: <?xml version="1.0"?> <string>طھط³طھ</string> but when decoded base64decode.org, result true : <?xml version="1.0"?> <string>تست</string> i have use encddecd unit decodestring function. the problem have using decodestring . function, in delphi 7, treats decoded binary data being ansi encoded. , problem text utf-8 encoded. to continue encddecd unit have couple of options. can switch decodestream . instance, code produce utf-8 encoded text file data: {$apptype console} uses classes, encddecd; const data = 'pd94bwwgdmvyc2lvbj0ims4wij8+dqo8c3ryaw5nptiq2lpyqjwvc3ryaw5npg=='; var input: tstringstream; output: tfilestream; begin input := tstringstream.create(data); try output := tfilestream.
Read more

html - Is there any way to exclude a single element from the style? (Bootstrap) -

so, have img logo, overlaid on div styled background-image: url("...") , , both elements inside 2 div classes .to-color , .to-overlay respectively, black transparent overlay. is there way exclude logo style given these 2 classes? here's jsfiddle example: link html , css example: - html - <div class="to-color"> <div class="to-overlay"> <div class="respo"> <div class="row"> <div class="col-xs-4 col-xs-offset-4 text-center"> <img src="http://ansonalex.com/wp-content/uploads/2011/06/google-logo-768x260.png" alt="..." class="img-responsive"> </div> </div> </div> </div> </div> - css - .to-color { background-color: #000; } .to-overlay { opacity : 0.5; } .respo { background-image: url("https://source.unsplash.com/category/objects/1600x900"); background-position: center center;
Read more