I updated the Project Validation XLS (filename: Project_Validation.Rev002 - sjs.xlsx) to include a column for validation of Azure. Please remember that we need Cray Japan/KyotoU to validate the custom development projects on Azure BEFORE we deploy on Cray US XC, KyotoU ja01, and Kyoto Temporary system. Once we receive confirmation of a project being validated on Azure, we will deploy on the other systems. Again, we are NOT deploying the projects onto Cray US XC, KyotoU ja01, and Kyoto Temporary systems until we receive your validation on Azure.
Thank you for confirming issues #1, #2, #4, #6, #7, and #10.
BTW, I noticed there were two #15, so I updated the table. I have provided some updates within XLS (filename: PBS_issue.Rev004 - sjs.xlsx)
WRT #3, #5, #9, #11, #12, #13, #15, #16, and #17, which are related to the qsub wrapper, I will need for Clinton to review and provide feedback. I have put Clinton in the TO: line of this email.
WRT #8, I am not understanding your comment
Nov-30: [CRAY: Do not set the max memory size of calogin01 in qsub wrapper.]
The qsub wrapper should NOT be setting unlimited.
Per the requirements, if the user does NOT specify memory at submission time, then the default memory defined in qsub wrapper is 1355mb. Is this correct?
If this is correct, then you can see the example below illustrating that 1355mb was requested by default because there was no memory requested at submission time.
[scott@ja01 ~]$ qsub -q normal -I
qsub: waiting for job 3484.ja01 to start
qsub: job 3484.ja01 ready
[scott@calogin01 ~]$ qstat -f 3484 | grep mem
resources_used.mem = 5776kb
resources_used.vmem = 434248kb
exec_vnode = (calogin01-17:ncpus=272:mem=1387520kb)
Resource_List.mem = 1355mb
Resource_List.select = 1:ncpus=272:mpiprocs=1:mem=1355mb:nppus=68:vntype=cr
comment = Job run at Tue Nov 29 at 02:40 on (calogin01-17:ncpus=272:mem=138
QSUB_PPN=1 -l select=1:ncpus=272:mpiprocs=1:mem=1355mb:nppus=68:vntype
[scott@calogin01 ~]$ ulimit -a | mem
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) 1387520
virtual memory (kbytes, -v) unlimited
The max memory size will be set based on the user's memory request. Please see cell I8 for the example of the max memory size being updated based on the user's request.
I see that you did the following:
$ ssh calogin01 'ulimit -a|grep "max memory size"'
max memory size (kbytes, -m) unlimited
What were you trying to accomplish by showing this ssh output?
The max memory size was changed to "unlimited" because your spreadsheet (cell E8) requested "Please set the max memory size to unlimited.". Please confirm what you want.
WRT #14, yesterday, Graham updated the Azure system to rejects jobs requesting more than Rstd. This is implemented in a queuejob hook. Please confirm on Azure that this is correct.
WRT the XLS files that Horigome-san provided are outdated. We had introduced new resources based on the email exchanges wit Nakashima-san. I am attaching a PDF (filename: Node Allocator PBS custom resources 20161130.pdf) that identifies the resources that are specific to node allocator. I have also provided a new XLS that describes the queue and server attributes (filename: Queue and Server Attributes 20161130.xlsx)