<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Aptos;
panose-1:2 11 0 4 2 2 2 2 2 4;}
@font-face
{font-family:"Times New Roman \(Body CS\)";
panose-1:2 11 6 4 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Aptos",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:14.0pt;font-family:"Arial",sans-serif">To the original question about the size of the pagepool being 323908133683 bytes, I think that should only be applied to the IO nodes (the ones handling all the NSD IOs) not
every node in the cluster. The large pagepool size is needed on the IO nodes for GNR to function properly.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:14.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
<div id="mail-editor-reference-message-container">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="color:black">From:
</span></b><span style="color:black">gpfsug-discuss <gpfsug-discuss-bounces@gpfsug.org> on behalf of Jonathan Buzzard <jonathan.buzzard@strath.ac.uk><br>
<b>Date: </b>Friday, March 8, 2024 at 11:27</span><span style="font-family:"Arial",sans-serif;color:black"> </span><span style="color:black">AM<br>
<b>To: </b>gpfsug-discuss@gpfsug.org <gpfsug-discuss@gpfsug.org><br>
<b>Subject: </b>[EXTERNAL] Re: [gpfsug-discuss] pagepool<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">On 08/03/2024 14:50, Jarsulic, Michael [BSD] wrote:<br>
<br>
> Ibán,<br>
> <br>
> What are you using for your scheduler?<br>
> <br>
> On my compute nodes, I am setting the pagepool to 16 GB and setting <br>
> aside specialized memory for GPFS that will not be allocated to jobs.<br>
> <br>
<br>
What you would normally do is create a node class<br>
<br>
mmcrnodeclass compute -N node001,node002,node003,node004,.....<br>
<br>
then set the pagepool appropriately<br>
<br>
mmchconfig pagepool=16G -i -N compute<br>
<br>
We then use slurm to limit the maximum amount of RAM a job can have on a <br>
node to be physical RAM minus the pagepool size minus a bit more for <br>
good measure to allow for the OS.<br>
<br>
If the OOM is kicking in then you need to reduce the RAM limit in slurm <br>
some more till it stops.<br>
<br>
Note we also twiddle with some other limits for compute nodes<br>
<br>
mmchconfig maxFilesToCache=8000 -N compute<br>
mmchconfig maxStatCache=16000 -N compute<br>
<br>
We have a slew of node classes where these settings are tweaked to <br>
account for their RAM and their role so dssg, compute, <br>
gpu,protocol,teaching, and login. All nodes belong to one or more node <br>
classes. Which reminds me I need a gui node class now.<br>
<br>
<br>
JAB.<br>
<br>
-- <br>
Jonathan A. Buzzard Tel: +44141-5483420<br>
HPC System Administrator, ARCHIE-WeSt.<br>
University of Strathclyde, John Anderson Building, Glasgow. G4 0NG<br>
<br>
<br>
_______________________________________________<br>
gpfsug-discuss mailing list<br>
gpfsug-discuss at gpfsug.org<br>
</span><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org"><span style="font-size:11.0pt">http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org</span></a><span style="font-size:11.0pt">
<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</body>
</html>