Luminous Landscape Forum

Site & Board Matters => Luminous Landscape Video => Topic started by: Christopher Sanderson on March 14, 2012, 11:59:56 am

Title: Test Amazon S3 downloads?
Post by: Christopher Sanderson on March 14, 2012, 11:59:56 am
This is a call for some volunteers to help us decide whether to migrate our download videos from the existing server to Amazon's S3.

If you own Camera to Print & Screen 2011, could you please try downloading a file and post the speed of the download? C2PS is the only download currently on S3.

We have had mixed results so far (past 12 hours) with N.American downloads quite speedy but European/UK downloads slower than normal.

We'd love to get some Australia/N-Z & Asia feedback
Title: Re: Test Amazon S3 downloads?
Post by: Christopher Sanderson on March 14, 2012, 12:09:16 pm
Click the Poll thingy above but also post your speed, location and comments if you have the time - Many thanks!
Title: Re: Test Amazon S3 downloads?
Post by: michaelphilip on March 14, 2012, 12:18:46 pm
Hi Chris,

I just tried downloading 2 of the Camera to Print and Screen (2011) files and was getting an average speed of about 3MB/sec. (ranged between 2.8 and 3.2 MB/sec.).
This compared to speeds last week for the LR4 tutorial of around 40-50 KB/sec.!

BTW, this is in N. America (Massachusetts).

Mike P.
Title: Re: Test Amazon S3 downloads?
Post by: tatuvaaj on March 14, 2012, 12:28:09 pm
Chris,

Downloading from Finland (EU) @ 73 KB/s. It is a bit faster than before but not exactly fast ;)

I would still prefer S3 if it would allow reliable multiple simultaneous downloads.
Title: Re: Test Amazon S3 downloads?
Post by: Robert-Peter Westphal on March 14, 2012, 12:41:45 pm
Hi Chris,

I downloaded C2PS02.zip ( from Germany) with a speed changing between 250kB/s and 800kB/s, which is in my opinion pretty good.

My internet-connection is capable of 1MB/s, so the peaks I could reach are very near the maximum speed available for me.

Best wishes

Robert
Title: Re: Test Amazon S3 downloads?
Post by: Alan Goldhammer on March 14, 2012, 12:53:00 pm
1.6 MB/sec which is much faster than last week LR4 downloads which were about 40 kB/sec.

Alan - Bethesda, MD; USA
Title: Re: Test Amazon S3 downloads?
Post by: Christopher Sanderson on March 14, 2012, 01:00:41 pm
That's very helpful. Many thanks.

Please keep the comments coming.

Hey Oz! When you wake up....
Title: Re: Test Amazon S3 downloads?
Post by: StuartOnline on March 14, 2012, 02:45:49 pm
That's very helpful. Many thanks.

Please keep the comments coming.

Hey Oz! When you wake up....

Hi Chris,

I am not at home where I have a very high speed connection, but I am at a restrurant that I am involved  with but the speed is not that fast.
Was down loading one of the file that was 412MB it was going to take about 43 minutes to down load. But again this is not a real fast connection.
Will give it a try when I return home.

Best,

Stu
Title: Re: Test Amazon S3 downloads?
Post by: aduke on March 14, 2012, 03:03:23 pm
Downloaded C2ps-02 at 11:45MST 14:45EDT.
  Initial Maximum - 670KB/sec
  Later Minimum- 64KB/sec
  Later Maximum- 685KB/sec

  The above readings were taken in the order given; the first one was after the first quarter of the file. The last quarter of the file was relatively steady, averaging about 350KB/sec.

Alan
Title: Re: Test Amazon S3 downloads?
Post by: AlanShaw on March 14, 2012, 04:32:01 pm
Australia here Chris.

Downloading for me at around 300 kb/s . About 10 times the speed of last week.

Cheers,
Alan.
Title: Re: Test Amazon S3 downloads?
Post by: michaelphilip on March 14, 2012, 04:55:06 pm
Chris,
One additional data point which might be of interest. (Or not?)
Out of curiosity I just tried downloading one of the LR4 tutorial files for comparison, and it was getting up to 1.3MB/sec., or about 30 times faster than last week during crunch time.
So the S3 still appears to be ~2.5x faster even when your old server isn't stressed.

(In N. America, Massachusetts, at about 4:45 EDT.)

MP
Title: Re: Test Amazon S3 downloads?
Post by: Christopher Sanderson on March 14, 2012, 05:36:29 pm
Chris,
One additional data point which might be of interest. (Or not?)
Out of curiosity I just tried downloading one of the LR4 tutorial files for comparison, and it was getting up to 1.3MB/sec., or about 30 times faster than last week during crunch time.
So the S3 still appears to be ~2.5x faster even when your old server isn't stressed.

(In N. America, Massachusetts, at about 4:45 EDT.)

MP

Yes, we know our server works well when there are less than 100 customers plugged in to the Store - it's the maxed out situation that remains to be seen...

Many thanks to all for their input. Please keep 'em coming.
Title: Re: Test Amazon S3 downloads?
Post by: paulturton on March 14, 2012, 05:58:01 pm
450 KB/sec on Rogers Roger Hub which is quite fast for this time of day - suppertime. Certainly better than the 18 to 28 KB/sec download speed I had for most of the for the LR4 videos.
Title: Re: Test Amazon S3 downloads?
Post by: Wolfgang Haeussler on March 14, 2012, 07:11:59 pm
Chris,

some data for Germany: c2ps-23.zip and c2ps-28.zip at an average download rate of 2 MB/s (12:00 AM Central European Time)
Title: Re: Test Amazon S3 downloads?
Post by: Eric Myrvaagnes on March 14, 2012, 07:17:39 pm
I just started 6 simultaneous downloads of files bigger than 200MB each (#1, 2, 5, 6, 7, and 9), and while all six were downloading, the download speeds ranged from 300KB/s to 960KB/s. Downloading the LR4 videos a week after they were released (once the feeding frenzy had subsided) I was getting speeds around 100KB/s, so I think this is much faster.

This is from Boston, MA. (7:00 pm on Wednesday, at a time when the main LuLa site seems slow to respond).

Eric
Title: Re: Test Amazon S3 downloads?
Post by: Alan Smallbone on March 14, 2012, 08:55:14 pm
I downloaded a C2PS and then a LR4 at two different locations here are the numbers:

office
C2PS  ranged from 600kb to 980kb/s
LR4    ranged from 600lb to 1.2mb/s

home
C2PS  ranged from 1.9mb to 3.2mb/s avg 2.6-2.7mb/s
LR4    ranged from 1.6mb to 1.9mb/s

Hope this helps, late last week I was downloading around 200kb to 400kb/s at home


Alan
Title: Re: Test Amazon S3 downloads?
Post by: Christopher Sanderson on March 14, 2012, 09:58:31 pm
Ok! I think we will try putting the existing LR4 files on S3.

I'll post here when they are up
Title: Re: Test Amazon S3 downloads?
Post by: OnyimBob on March 14, 2012, 10:22:39 pm
Bob here in Australia - 171 MB's at 52 kb/s approx 56 mins - on satellite broadband.
Title: Re: Test Amazon S3 downloads?
Post by: kevk on March 15, 2012, 05:03:30 am
Kevin in Brisbane, Australia - getting 812 kB/sec from S3 (eg. c2ps-02.zip (282 MB) arrived in 6 minutes).

Also tried LR4_11 (264 MB) from your old-not-stressed server and was getting 265 kB/sec.
When I was downloading LR4 tutoral during stress time I was getting 15-20 kB/sec.

(My connection is ADSL2+ and a speed test on it says I get 6.5 Mbit/sec from a server in Australia)

Time - 7:00 pm Australian eastern standard time (UTC+10), which equates to 5:00 am in New York and 9:00 am in London.
Sorry about the late reply - I dared to stray from my computer today!  :)

Good job Chris!!!!
Title: Re: Test Amazon S3 downloads?
Post by: Christopher Sanderson on March 15, 2012, 07:54:21 am
So the results are pretty good and we will proceed with the switch-over but....

The 'hacks' to our shopping cart have 'broken' a couple of other things in there on the Admin side. So we have to back out of Amazon for a few hours and try and fix those issues before continuing the migration. Stay tuned

Many thanks to everyone for your feedback. BTW there still seem to be some routing issues with Amazon S3 in Ireland. I'd love to hear from anyone in IE who can test the C2PS download...
Title: Re: Test Amazon S3 downloads?
Post by: lcowgill on March 15, 2012, 09:12:13 am
Chris,

237 meg file in less than a minute.  Location GA.
Title: Re: Test Amazon S3 downloads?
Post by: Christopher Sanderson on March 16, 2012, 11:22:51 am
OK so the available LR4 files are now up on Amazon. I'd appreciate again if a few people would try a download and post their location, speed and comments

Chris
Title: Re: Test Amazon S3 downloads?
Post by: StuartOnline on March 16, 2012, 11:33:02 am
OK so the available LR4 files are now up on Amazon. I'd appreciate again if a few people would try a download and post their location, speed and comments

Chris

Hi Chris,

Just down loaded from home connection LR4_13-1 (219MB) at 1.9MB/per sec.  Total time was just at 2 minutes.
Now I am on AT&T Uverse with download 18Mbps and upload 1.4Mbps.
Let add that I am on a MacBook Pro 2.66 (June 2009) with 8GB DDR3 Ram with a 500GB 7200rpm Hard Drive.

Best,

Stu
Title: Re: Test Amazon S3 downloads?
Post by: Christopher Sanderson on March 16, 2012, 12:42:22 pm
Just down loaded from home connection LR4_13-1 (219MB) at 1.9MB/per sec.  Total time was just at 2 minutes.

Wow - you US folk are going to like Amazon!
Title: Re: Test Amazon S3 downloads?
Post by: greyscale on March 16, 2012, 01:45:09 pm
Just d/l LR4_13.zip avg speed 5.1 mb per sec on a Verizon Fios home system Win 7 pro. NJ, USA.

John
Title: Re: Test Amazon S3 downloads?
Post by: martinreed22 on March 16, 2012, 03:00:16 pm
In UK, LR4_06.zip @ 600-900kB/sec

 ;D

martin
Title: Re: Test Amazon S3 downloads?
Post by: Ronny Nilsen on March 16, 2012, 03:54:00 pm
I´m getting the max speed of my line at home here in Norway (6Mbit/sec -> 800KByte/sec)

Ronny
Title: Re: Test Amazon S3 downloads?
Post by: Christopher Sanderson on March 16, 2012, 04:50:54 pm
Many thanks again for all of the input. It has given us the confidence to migrate everything over to Amazon's S3 - but over time. A lot of our content will still come through the regular Store server for the next several weeks.
 
Right now only LR4 & C2PS are coming off AmazonS3

I will be publishing the next videos of LR4 in less than five days and will do a mass email to all customers. That will be the acid test to see if a few thousand hits on the same download file will still give acceptable rates. But don't go to the Store for new LR4 stuff yet, I will post here (http://www.luminous-landscape.com/forum/index.php?topic=63341.0) when it's available (and prior to the mass email)
Title: Re: Test Amazon S3 downloads?
Post by: Nigel Johnson on March 16, 2012, 05:07:09 pm
In the UK I downloaded LR4_11.zip at 807 kbytes/s which is slightly less than half my maximum download speed.

In comparison c2ps-30.zip from Camera to Print and Screen (a similar sized file) downloaded at 831 kbytes/sec within half an hour of the LR4 download.

Therefore S3 seems to be slightly slower than your normal server in the UK when there is not a 'feeding frenzy' for new downloads - the download speed for the LR file was much slower when I download it in the first 48 hours after issue, unfortunately I didn't note the speed.

Nigel
Title: Re: Test Amazon S3 downloads?
Post by: Nigel Johnson on March 16, 2012, 05:14:25 pm
Chris

I obviously don't know how Amazon S3 will stand up to large numbers of simultaneous connections but the test I just carried out, while you were posting, showed a worse performance for S3 than your normal server for a UK download. The test was carried out between about 8.30 pm and 9 pm local time on a Friday evening. If it is of any use to you I could retest at a different time of day.

Regards
Nigel

PS a slightly slower download for your legacy files is not a personal concern, as I have either already bought and downloaded all the LL videos or for the earlier LLVJs have the DVDs.
Title: Re: Test Amazon S3 downloads?
Post by: David Hufford on March 16, 2012, 07:55:53 pm
Just tried one and averaged 2.44mb per second in Tokyo.
Title: Re: Test Amazon S3 downloads?
Post by: kevk on March 16, 2012, 10:02:50 pm
Brisbane, Australia - Getting mostly 812 kB/sec for LR4_11 (270 MB) so it took 5-6 mins.
I am very happy with that.
(My connection is ADSL2+ , 6.5 Mbit/sec download)
Looking forward to the next instalment!

Kevin
Title: Re: Test Amazon S3 downloads?
Post by: rbudge on March 17, 2012, 03:04:32 am
From Brisbane Australia

Camera to print  download between 1 - 1.2MB/sec
Where the **** pictures download straight after varied between 116 - 125KB/sec
at 6pm AEDT
Title: Re: Test Amazon S3 downloads?
Post by: StuartOnline on March 17, 2012, 05:47:40 pm
Chris,

Just downloaded:
LR4 14 (230MB) 630.4KB/Sec about 8 minutes.
LR4 15 (245MB) 1.4MB/Sec was under 5 minutes.

Download was via home connection with AT&T Uverse download 18Mbps and upload 1.4Mbps (US).
I am using a MacBook Pro 2.66 (June 2009) with 8GB DDR3 Ram with a 500GB 7200rpm Hard Drive.

Best,

Stu
Title: Re: Test Amazon S3 downloads?
Post by: aduke on March 17, 2012, 06:39:17 pm
Chris,

Just finished downloading LR4 14. Took between 40 and 45 minutes. The lowest transfer speed seen was about 50KB/sec, highest was over 200KB/sec.

On the basis of my download of C2ps from Amazon, I'm not terribly surprised by this performance. It is significantly better than the experience downloading the LR4 basic files which I did after the announcement that the rush was over.

Alan
Title: Re: Test Amazon S3 downloads?
Post by: WallyM on March 17, 2012, 07:13:40 pm
I was able to get both videos at more or less my maximum connection speed - about 150 - 160 KBps.  I am on 1.5mbps DSL connection with OS X SnowLeopard and Firefox 10 in SW Washington state.

Oddly enough, when I started it settled down to a roaring 15 KBps with time estimated at over 3 hours.  I killed the download, rebooted the DSL modem and tried again, and promptly went up to about 150ish KBps.

Wally
Title: Re: Test Amazon S3 downloads?
Post by: AlanShaw on March 17, 2012, 07:37:48 pm
In Melbourne, Australia.

Downloads ran at around 300kb/s. About 10 minutes for each file.  That's my Sunday morning sorted.

Cheers,
Alan.
Title: Re: Test Amazon S3 downloads?
Post by: alain on March 17, 2012, 08:02:45 pm
Hi

I got 1,7MB/s from Europe or about 4 times as fast.

Alain
Title: Re: Test Amazon S3 downloads?
Post by: kencameron on March 17, 2012, 08:44:39 pm
Going slow as an elderly canine in Canberra, Australia, at the moment - 17 Kb/s. I am assuming that I don't have to do anything special to get the Amazon server, just click on the files on my  order information page - or am I mistaken about that? Is there some different path to using the Amazon server?
Title: Re: Test Amazon S3 downloads?
Post by: Triolo on March 17, 2012, 08:48:18 pm
Trying to download the two new videos from New Jersey.
Maximum speed of the download is impressive at 2.35 MB/sec., but the downloads are aborted before completion giving a error "the server retured an error response of 403=forbidden".
The previous server was slower, but with the same problem.
Title: Re: Test Amazon S3 downloads?
Post by: Box Brownie on March 17, 2012, 09:37:53 pm
I was tempted to 'tick' "Much faster" but the speed did fluctuate a bit so selected the step under the max :)  Now bearing in mind the mad rush of the first few days has disappeared I would say using the Amazon cloud does seem to be an overall more pleasing experience, so many thanks for taking that step  :D
Title: Re: Test Amazon S3 downloads?
Post by: Christopher Sanderson on March 17, 2012, 10:39:45 pm
Going slow as an elderly canine in Canberra, Australia, at the moment - 17 Kb/s. I am assuming that I don't have to do anything special to get the Amazon server, just click on the files on my  order information page - or am I mistaken about that? Is there some different path to using the Amazon server?
No special route - it is automatic. You may want to try a different browser. Sounds like some local problem perhaps?
Title: Re: Test Amazon S3 downloads?
Post by: Andrew Makiejewski on March 17, 2012, 10:42:50 pm
I am just about finished testing the download of c2ps-27.zip which is 384MB. It took 10 minutes to D/L. It started out quite fast at about 1.4MB/s then slowed down to about 450KB/s about a third of the way in.

I am in Toronto, ON , Canada.

FYI, I D/L the 2 new LR4 zips and they came down really fast. Did not pay attention to the info , but under 2 minutes each.

Andrew
Title: Re: Test Amazon S3 downloads?
Post by: kencameron on March 18, 2012, 04:05:58 am
No special route - it is automatic. You may want to try a different browser. Sounds like some local problem perhaps?

Yes, I think so. Going nicely now.
Title: Re: Test Amazon S3 downloads?
Post by: Robert-Peter Westphal on March 18, 2012, 05:08:30 am
Hi,

I've downloaded both packages at around 600-800kB/s, which is pretty good.

Thanks Chris - for me you have fixed the problem very fast and in a very good way !!!

Robert
Title: Re: Test Amazon S3 downloads?
Post by: Christopher Sanderson on March 18, 2012, 09:46:36 am
Thanks Chris - for me you have fixed the problem very fast and in a very good way !!!
...with a lot of help from our IT brains: Mark Guertin (Gerk)
Title: Re: Test Amazon S3 downloads?
Post by: JCVG on March 18, 2012, 01:17:28 pm
I'm located in Bangkok, Thailand and actually for me it's slower than the first server and I couldn't wait the 7hrs required to download one file (few kb/s). I'll try again in few days.
Title: Re: Test Amazon S3 downloads?
Post by: Photo Op on March 18, 2012, 01:59:53 pm
Chris- downloaded the most recent files separately at 2.4mb/sec at 7:00am EDT from Massachusetts. Time for each <2 min.

Thanks!

DaveB
Title: Re: Test Amazon S3 downloads?
Post by: WallyM on March 18, 2012, 06:24:22 pm
I'm located in Bangkok, Thailand and actually for me it's slower than the first server and I couldn't wait the 7hrs required to download one file (few kb/s). I'll try again in few days.

Try cancelling the download if you get a connection with very poor transfer rates; then restart the download in a minute or so and see if it improves.  My first try resulted in about 15KBps and a three hour time estimate.  I broke the connection, rebooted my DSL modem and restarted the connection and got full speed transfer on both files.  I don't know if rebooting the modem helped, but at least one other person has reported an initial download speed of 10 or 15 KBps that improved later.  Maybe an occasional connection to Amazon has a problem that goes away when the link is broken and then reconnected.

Wally
Title: Re: Test Amazon S3 downloads?
Post by: Alan Goldhammer on March 19, 2012, 11:08:28 am
Chris,

Just downloaded Zips 14 & 15 at 1.6 Mb/sec which is the max I get from my Verizon FIOS connection.  Looks like Amazon is doing well to get the files out.

Alan
Title: Re: Test Amazon S3 downloads?
Post by: JCVG on March 19, 2012, 11:20:10 am
Try cancelling the download if you get a connection with very poor transfer rates; then restart the download in a minute or so and see if it improves.  My first try resulted in about 15KBps and a three hour time estimate.  I broke the connection, rebooted my DSL modem and restarted the connection and got full speed transfer on both files.  I don't know if rebooting the modem helped, but at least one other person has reported an initial download speed of 10 or 15 KBps that improved later.  Maybe an occasional connection to Amazon has a problem that goes away when the link is broken and then reconnected.

Wally


Thanks for the tip Wally. I am now in France and tried to download the files from Safari (Windows 7) and Chrome but I get 2 or 3 kb, 15 hrs to download 1 file. When I was in Bangkok I've tried from my Mac. Different computers and different network but still the same problem. But not a big deal for me, I guess after a few days it will be better when everybody will be done with the download. I can wait but I just want to share my experience. (I'll be in Tunisia in few days, I'll try from there  ;D ).

Cheers

JC
Title: Re: Test Amazon S3 downloads?
Post by: s4e on March 24, 2012, 03:23:47 pm
The download no works like it should. Thank you for improving this one. I was close to skip next video because of all the problems downloading but now I ready for your next video ;D

Sverre
Title: Re: Test Amazon S3 downloads?
Post by: Christopher Sanderson on March 24, 2012, 04:23:31 pm
The download no[w] works like it should. Thank you for improving this one. I was close to skip next video because of all the problems downloading but now I ready for your next video ;D

Sverre

added what I presume is a missing 'w'
Title: Re: Test Amazon S3 downloads?
Post by: mac_paolo on May 07, 2012, 02:37:34 am
I've just put the last four archives on download.
All together they get downloaded within the 75-100KB/s range. Which not only is unacceptable, but also far less than the previous performance with local hosted files.
Let's wait those 3 hours...  :-\

My location: Italy

Edit: I should have said 30-100KB/s  >:(
Title: Re: Test Amazon S3 downloads?
Post by: Box Brownie on May 07, 2012, 07:02:48 am
I've just put the last four archives on download.
All together they get downloaded within the 75-100KB/s range. Which not only is unacceptable, but also far less than the previous performance with local hosted files.
Let's wait those 3 hours...  :-\

My location: Italy

Edit: I should have said 30-100KB/s  >:(

For what it is worth have you tried them one at a time???  I have found with any multiple downloads that when all are selected it is not queue'ing that occurs but they all start to download and the 'rate' of the first one selected can drop quite markedly while the next ones go at a crawl :(

So do try them one at a time and see how that goes :) I have been doing this when pressed for time (means you have to return to the PC regularly rather than just walking away!) and I think the last lot were collected in approx and hour???
Title: Re: Test Amazon S3 downloads?
Post by: francois on May 07, 2012, 07:45:42 am
I've just put the last four archives on download.
All together they get downloaded within the 75-100KB/s range. Which not only is unacceptable, but also far less than the previous performance with local hosted files.
Let's wait those 3 hours...  :-\

My location: Italy

Edit: I should have said 30-100KB/s  >:(

I did the same here but got much higher speeds, it was a matter of a few minutes. I downloaded the first three all at once and the last one (alone) went down at +3MB/sec. I redid the test right now (13h30 European time) and DL speed was at 2.9MB/sec and going up.
Title: Re: Test Amazon S3 downloads?
Post by: mac_paolo on May 07, 2012, 08:04:02 am
So do try them one at a time and see how that goes :)
Next time I will.
Title: Re: Test Amazon S3 downloads?
Post by: mac_paolo on May 07, 2012, 08:06:09 am
I did the same here but got much higher speeds, it was a matter of a few minutes. I downloaded the first three all at once and the last one (alone) went down at +3MB/sec. I redid the test right now (13h30 European time) and DL speed was at 2.9MB/sec and going up.
It depends on a zillion of factors.
The point is, taken into account my DSL parameters, with previous local hosted files I go better speeds (more or less 200-300KB/s), while I now get way slower ones.
It's strange, as Amazon S3 is an excellent redundant service... but these are the numbers I have.
Title: Re: Test Amazon S3 downloads?
Post by: StuartOnline on May 07, 2012, 09:04:29 am
Chris,

I down loaded the latest videos (20-23) with no problems the other day.
Each video took about 2 minutes at a rate of 1.5Mbps.
Currently I am on AT&T Uverse here in the US with a downstream speed of up to 18 Mbps.
Computer being used is an Apple Macbook Pro 2.66 (June 2009) with 8GB DDR3 Ram.

Best,

Stu