Panther transfer on finish file size limit question

[laat0003]

Howdy,

Is there a file size limit to what the Panther run manager will transfer. I recently stopped a prior-Monte Carlo run because transfers were being initiated (in RMR file and a file created in master folder) but the files were all 0KB (no actual data transfer). Models definitely ran to completion and were recorded as complete. I tested it with a 1GB file and it worked fine but the files I was attempting to transfer during the failed run were around 2.5GB, hence the question. I'm doing a NOPTMAX =0 test right now but the model runs slower than molasses so I figured I'd ask the question in the meantime.

Thanks in advance,

[jtwhite79]

Did the rmr report any data being transferred? There is not a theoretical file size limit but there are practical limits!

[laat0003]

Yeah the RMR had an entry in it for initiating the transfer but the subsequent entries that record actual data transfer were absent. So files were created with correct naming according to the sequence of model completion, but they were all empty. The NOPTMAX = 0 test failed the transfer as well. So, I guess that I'm past the practical limit, sigh! I know what you are thinking but it's not my model. I'm just the schmuck who has to deal with it.

I managed to script up a workaround where the agents keep the HDS files and the I run a collector script when it is all finished so no biggie.

[p-ortega]

I had the same problem before (0kb files, no entries in RMR) and it was because I ran out of space on the disk where the master is.

[jtwhite79]

Good call. Probably ought to be checking the status of the file streams to see that they are actually writing successfully...

[laat0003]

I did consider the space issue before committing to the run and made sure I had sufficient storage on the machine that was running the master to hold all the transfers and still have 0.5TB buffer. My workaround, having the agents hold the files wasn't so lucky, with one of the machines running out of space dropping 20 models in the process. I extracted those reals from the prior csv and am running them now.