Author |
Message |
|
I made a change in my cc_config.xml file which seems to improve file uploads.
I set the max to 1 at a time
Previously when I had it at the default of 2 per project it took about 12-13 minutes for the largest file to complete, while the shorter ones completed during that time.
With setting it to 1 max, all the files went through in about 9 minutes total.
Host Project ID Date Message
PS301 PS3GRID 23 7/29/2007 8:23:13 PM [file_xfer] Started upload of file Ny23493-WF5-3-nodelete_0_0
PS301 PS3GRID 67 7/29/2007 8:30:48 PM [file_xfer] Finished upload of file Ny23493-WF5-3-nodelete_0_0
PS301 PS3GRID 68 7/29/2007 8:30:48 PM [file_xfer] Throughput 54063 bytes/sec
PS301 PS3GRID 69 7/29/2007 8:30:48 PM [file_xfer] Started upload of file Ny23493-WF5-3-nodelete_0_1
PS301 PS3GRID 70 7/29/2007 8:31:06 PM [file_xfer] Finished upload of file Ny23493-WF5-3-nodelete_0_1
PS301 PS3GRID 71 7/29/2007 8:31:06 PM [file_xfer] Throughput 51779 bytes/sec
PS301 PS3GRID 72 7/29/2007 8:31:06 PM [file_xfer] Started upload of file Ny23493-WF5-3-nodelete_0_2
PS301 PS3GRID 73 7/29/2007 8:31:24 PM [file_xfer] Finished upload of file Ny23493-WF5-3-nodelete_0_2
PS301 PS3GRID 74 7/29/2007 8:31:24 PM [file_xfer] Throughput 51067 bytes/sec
PS301 PS3GRID 75 7/29/2007 8:31:24 PM [file_xfer] Started upload of file Ny23493-WF5-3-nodelete_0_3
PS301 PS3GRID 76 7/29/2007 8:31:27 PM [file_xfer] Finished upload of file Ny23493-WF5-3-nodelete_0_3
PS301 PS3GRID 77 7/29/2007 8:31:27 PM [file_xfer] Throughput 1385 bytes/sec
PS301 PS3GRID 78 7/29/2007 8:31:27 PM [file_xfer] Started upload of file Ny23493-WF5-3-nodelete_0_4
PS301 PS3GRID 79 7/29/2007 8:31:50 PM [file_xfer] Finished upload of file Ny23493-WF5-3-nodelete_0_4
PS301 PS3GRID 80 7/29/2007 8:31:50 PM [file_xfer] Throughput 52684 bytes/sec
Lines I added to cc_config.xml
<max_file_xfers>1</max_file_xfers>
<max_file_xfers_per_project>1</max_file_xfers_per_project>
these go in the between the <options> tags
____________
Alpha Tester ~~ BOINCin since 10-Apr-2004 (2.28) ~~~ Join team USA |
|
|
|
For my last WU, result files for upload are very huge:
>20 Mo
This is this one http://www.ps3grid.net/result.php?resultid=6171
Is this normal?
And why i can\'t have more than 1 WU crunching, and no one in my waiting task?
I check other users, and they have one WU crunching, and another waiting for calculations.
|
|
|
GDFVolunteer moderator Project administrator Project developer Project tester Volunteer developer Volunteer tester Project scientist Send message
Joined: 14 Mar 07 Posts: 1957 Credit: 629,356 RAC: 0 Level
Scientific publications
|
For my last WU, result files for upload are very huge:
>20 Mo
This is this one http://www.ps3grid.net/result.php?resultid=6171
Is this normal?
And why i can\'t have more than 1 WU crunching, and no one in my waiting task?
I check other users, and they have one WU crunching, and another waiting for calculations.
Hi Jim,
file size is ok. It has been like this for a while now. The app needs to return the MD trajectories which I have tried to keep at the minimum size. Also, files upload are concurrent with calculations, so no time is lost.
Because WUs are chained in a workflow the server maintains a limit of 2 per host such that there is a sufficient buffer but they return as quickly as possible.
You have only one because there are few wus left in this batch of runs. Every wu finished creates another one and you have been unlucky so far.
I am preparing a new large set of runs. It should be out after testing in a few days.
g |
|
|
|
For my last WU, result files for upload are very huge:
>20 Mo
This is this one http://www.ps3grid.net/result.php?resultid=6171
Is this normal?
And why i can\'t have more than 1 WU crunching, and no one in my waiting task?
I check other users, and they have one WU crunching, and another waiting for calculations.
Hi Jim,
file size is ok. It has been like this for a while now. The app needs to return the MD trajectories which I have tried to keep at the minimum size. Also, files upload are concurrent with calculations, so no time is lost.
Because WUs are chained in a workflow the server maintains a limit of 2 per host such that there is a sufficient buffer but they return as quickly as possible.
You have only one because there are few wus left in this batch of runs. Every wu finished creates another one and you have been unlucky so far.
I am preparing a new large set of runs. It should be out after testing in a few days.
g
Ok, i just notice the size yesterday, when i saw that i don\'t have another WU to crunch, and saw the WU finished and then uploading.
And yes i was not lucky yesterday, no WU before the end of the morning.
And may the force be with you until September!
|
|
|
|
I had a download problem today.
It started about 7 hours ago, I just noticed it.
My PS3 had no work except for one it was trying to download for the past 7 hours.
It had 3 files named input.pdb, input.psf and input.txt. Log is showing temporarily failed, http error every time it tried these downloads.
I exited the core client and restarted but this did not help.
I finally just aborted the task and the downloads.
Then did update and got another task, which downloaded without problem, this one downloaded 10 files, but none of them were named input.anything
Heres the wu for the bad one, maybe there is something wrong with it ?
http://www.ps3grid.net/PS3GRID/workunit.php?wuid=5515
Looking back in the log the problem actually started before that another 7 hours, it actually shows at 1:53AM local time that 2 files, named input.coor and input.vel downloaded OK, the next file input.pdb, then input.psf and then input.txt did not download and show the http error.
Log in reverse order:
Host Project ID Date Message
PS301 PS3GRID 2799 9/3/2007 1:53:42 AM Backing off 1 min 0 sec on download of file input.txt
PS301 PS3GRID 2798 9/3/2007 1:53:42 AM [file_xfer] Temporarily failed download of input.txt: http error
PS301 PS3GRID 2797 9/3/2007 1:53:39 AM [file_xfer] Started download of file input.txt
PS301 PS3GRID 2796 9/3/2007 1:53:39 AM Backing off 1 min 0 sec on download of file input.psf
PS301 PS3GRID 2795 9/3/2007 1:53:39 AM [file_xfer] Temporarily failed download of input.psf: http error
PS301 PS3GRID 2794 9/3/2007 1:53:35 AM [file_xfer] Started download of file input.psf
PS301 PS3GRID 2793 9/3/2007 1:53:35 AM Backing off 1 min 0 sec on download of file input.pdb
PS301 PS3GRID 2792 9/3/2007 1:53:35 AM [file_xfer] Temporarily failed download of input.pdb: http error
PS301 PS3GRID 2791 9/3/2007 1:53:33 AM [file_xfer] Started download of file input.pdb
PS301 PS3GRID 2790 9/3/2007 1:53:33 AM [file_xfer] Throughput 148013 bytes/sec
PS301 PS3GRID 2789 9/3/2007 1:53:33 AM [file_xfer] Finished download of file input.vel
PS301 PS3GRID 2788 9/3/2007 1:53:27 AM [file_xfer] Started download of file input.vel
PS301 PS3GRID 2787 9/3/2007 1:53:27 AM [file_xfer] Throughput 122640 bytes/sec
PS301 PS3GRID 2786 9/3/2007 1:53:27 AM [file_xfer] Finished download of file input.coor
PS301 PS3GRID 2785 9/3/2007 1:53:19 AM [file_xfer] Started download of file input.coor
|
|
|
GDFVolunteer moderator Project administrator Project developer Project tester Volunteer developer Volunteer tester Project scientist Send message
Joined: 14 Mar 07 Posts: 1957 Credit: 629,356 RAC: 0 Level
Scientific publications
|
Yes, this is a test wu which did not work.
We are working hard for the next big step of PS3GRID.
|
|
|
|
I also had a WU with this download problem two days ago. I aborted the transfer files, and it successfully downloaded another WU. Just FYI.
____________
Reno, NV
Team: SETI.USA
|
|
|
|
Where do I find this cc_config file? It does not appear in my BOINC directory....
I made a change in my cc_config.xml file which seems to improve file uploads.
I set the max to 1 at a time
Previously when I had it at the default of 2 per project it took about 12-13 minutes for the largest file to complete, while the shorter ones completed during that time.
With setting it to 1 max, all the files went through in about 9 minutes total.
Host Project ID Date Message
PS301 PS3GRID 23 7/29/2007 8:23:13 PM [file_xfer] Started upload of file Ny23493-WF5-3-nodelete_0_0
PS301 PS3GRID 67 7/29/2007 8:30:48 PM [file_xfer] Finished upload of file Ny23493-WF5-3-nodelete_0_0
PS301 PS3GRID 68 7/29/2007 8:30:48 PM [file_xfer] Throughput 54063 bytes/sec
PS301 PS3GRID 69 7/29/2007 8:30:48 PM [file_xfer] Started upload of file Ny23493-WF5-3-nodelete_0_1
PS301 PS3GRID 70 7/29/2007 8:31:06 PM [file_xfer] Finished upload of file Ny23493-WF5-3-nodelete_0_1
PS301 PS3GRID 71 7/29/2007 8:31:06 PM [file_xfer] Throughput 51779 bytes/sec
PS301 PS3GRID 72 7/29/2007 8:31:06 PM [file_xfer] Started upload of file Ny23493-WF5-3-nodelete_0_2
PS301 PS3GRID 73 7/29/2007 8:31:24 PM [file_xfer] Finished upload of file Ny23493-WF5-3-nodelete_0_2
PS301 PS3GRID 74 7/29/2007 8:31:24 PM [file_xfer] Throughput 51067 bytes/sec
PS301 PS3GRID 75 7/29/2007 8:31:24 PM [file_xfer] Started upload of file Ny23493-WF5-3-nodelete_0_3
PS301 PS3GRID 76 7/29/2007 8:31:27 PM [file_xfer] Finished upload of file Ny23493-WF5-3-nodelete_0_3
PS301 PS3GRID 77 7/29/2007 8:31:27 PM [file_xfer] Throughput 1385 bytes/sec
PS301 PS3GRID 78 7/29/2007 8:31:27 PM [file_xfer] Started upload of file Ny23493-WF5-3-nodelete_0_4
PS301 PS3GRID 79 7/29/2007 8:31:50 PM [file_xfer] Finished upload of file Ny23493-WF5-3-nodelete_0_4
PS301 PS3GRID 80 7/29/2007 8:31:50 PM [file_xfer] Throughput 52684 bytes/sec
Lines I added to cc_config.xml
<max_file_xfers>1</max_file_xfers>
<max_file_xfers_per_project>1</max_file_xfers_per_project>
these go in the between the <options> tags
____________
Consciousness: That annoying time between naps......
Experience is a wonderful thing: it enables you to recognize a mistake every time you repeat it. |
|
|
|
Never mind.
____________
Consciousness: That annoying time between naps......
Experience is a wonderful thing: it enables you to recognize a mistake every time you repeat it.
|
|
|