<div dir="ltr"><div>first good that the problem at least is solved, it would be great if you could open a PMR so this gets properly fixed, the daemon shouldn't segfault, but rather print a message that the device is too big. </div><div><br></div>on the caching , it only gets used when you run out of pagepool or when you run out of full file objects . so what benchmark, test did you run to push data into LROC ?<div><br></div><div>sven<br><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr">On Thu, Dec 29, 2016 at 5:41 PM Matt Weil <<a href="mailto:mweil@wustl.edu">mweil@wustl.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000" class="gmail_msg">
<p class="gmail_msg">after restart. still doesn't seem to be in use.<br class="gmail_msg">
</p>
<p class="gmail_msg">
<blockquote type="cite" class="gmail_msg">[root@ces1 ~]# mmdiag --lroc</blockquote></p></div><div bgcolor="#FFFFFF" text="#000000" class="gmail_msg"><p class="gmail_msg"><blockquote type="cite" class="gmail_msg"><br class="gmail_msg">
<br class="gmail_msg">
=== mmdiag: lroc ===<br class="gmail_msg">
LROC Device(s): '0A6403AA5865389E#/dev/nvme0n1;' status Running<br class="gmail_msg">
Cache inodes 1 dirs 1 data 1 Config: maxFile 1073741824
stubFile 1073741824<br class="gmail_msg">
Max capacity: 1526184 MB, currently in use: 0 MB<br class="gmail_msg"></blockquote></p></div><div bgcolor="#FFFFFF" text="#000000" class="gmail_msg"><p class="gmail_msg"><blockquote type="cite" class="gmail_msg">
Statistics from: Thu Dec 29 10:35:32 2016</blockquote></p></div><div bgcolor="#FFFFFF" text="#000000" class="gmail_msg"><p class="gmail_msg"><blockquote type="cite" class="gmail_msg"><br class="gmail_msg">
<br class="gmail_msg">
Total objects stored 0 (0 MB) recalled 0 (0 MB)<br class="gmail_msg">
objects failed to store 0 failed to recall 0 failed to
inval 0<br class="gmail_msg">
objects queried 0 (0 MB) not found 0 = 0.00 %<br class="gmail_msg">
objects invalidated 0 (0 MB)<br class="gmail_msg">
</blockquote></p></div><div bgcolor="#FFFFFF" text="#000000" class="gmail_msg">
<br class="gmail_msg">
<div class="m_-5832568848232937267moz-cite-prefix gmail_msg">On 12/29/16 10:28 AM, Matt Weil wrote:<br class="gmail_msg">
</div>
<blockquote type="cite" class="gmail_msg">
<p class="gmail_msg">wow that was it.</p>
<p class="gmail_msg"> </p>
<blockquote type="cite" class="gmail_msg"> mmdiag --lroc<br class="gmail_msg">
<br class="gmail_msg">
=== mmdiag: lroc ===<br class="gmail_msg">
LROC Device(s): '0A6403AA5865389E#/dev/nvme0n1;' status Running<br class="gmail_msg">
Cache inodes 1 dirs 1 data 1 Config: maxFile 1073741824
stubFile 1073741824<br class="gmail_msg">
Max capacity: 1526184 MB, currently in use: 0 MB<br class="gmail_msg">
Statistics from: Thu Dec 29 10:08:58 2016</blockquote>
It is not caching however. I will restart gpfs to see if that
makes it start working.<br class="gmail_msg">
<br class="gmail_msg">
<div class="m_-5832568848232937267moz-cite-prefix gmail_msg">On 12/29/16 10:18 AM, Matt Weil
wrote:<br class="gmail_msg">
</div>
<blockquote type="cite" class="gmail_msg">
<p class="gmail_msg"><br class="gmail_msg">
</p>
<br class="gmail_msg">
<div class="m_-5832568848232937267moz-cite-prefix gmail_msg">On 12/29/16 10:09 AM, Sven Oehme
wrote:<br class="gmail_msg">
</div>
<blockquote type="cite" class="gmail_msg">
<div dir="ltr" class="gmail_msg">i agree that is a very long name , given this
is a nvme device it should show up as /dev/nvmeXYZ
<div class="gmail_msg">i suggest to report exactly that in nsddevices and
retry.</div>
<div class="gmail_msg">i vaguely remember we have some fixed length device
name limitation , but i don't remember what the length is,
so this would be my first guess too that the long name is
causing trouble. <br class="gmail_msg">
</div>
</div>
</blockquote>
I will try that. I was attempting to not need to write a custom
udev rule for those. Also to keep the names persistent. Rhel 7
has a default rule that makes a sym link in /dev/disk/by-id.<br class="gmail_msg">
0 lrwxrwxrwx 1 root root 13 Dec 29 10:08
nvme-Dell_Express_Flash_NVMe_SM1715_1.6TB_SFF_______S29GNYAH200016
-> ../../nvme0n1<br class="gmail_msg">
0 lrwxrwxrwx 1 root root 13 Dec 27 11:20
nvme-Dell_Express_Flash_NVMe_SM1715_1.6TB_SFF_______S29GNYAH300161
-> ../../nvme1n1<br class="gmail_msg">
<blockquote type="cite" class="gmail_msg">
<div dir="ltr" class="gmail_msg">
<div class="gmail_msg"><br class="gmail_msg">
</div>
</div>
<br class="gmail_msg">
<div class="gmail_quote gmail_msg">
<div dir="ltr" class="gmail_msg">On Thu, Dec 29, 2016 at 5:02 PM Aaron Knister
<<a href="mailto:aaron.s.knister@nasa.gov" class="gmail_msg" target="_blank">aaron.s.knister@nasa.gov</a>>
wrote:<br class="gmail_msg">
</div>
<blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Interesting.
Thanks Matt. I admit I'm somewhat grasping at straws here.<br class="gmail_msg">
<br class="gmail_msg">
That's a *really* long device path (and nested too), I
wonder if that's<br class="gmail_msg">
causing issues.<br class="gmail_msg">
<br class="gmail_msg">
What does a "tspreparedisk -S" show on that node?<br class="gmail_msg">
<br class="gmail_msg">
Also, what does your nsddevices script look like? I'm
wondering if you<br class="gmail_msg">
could have it give back "/dev/dm-XXX" paths instead of
"/dev/disk/by-id"<br class="gmail_msg">
paths if that would help things here.<br class="gmail_msg">
<br class="gmail_msg">
-Aaron<br class="gmail_msg">
<br class="gmail_msg">
On 12/29/16 10:57 AM, Matt Weil wrote:<br class="gmail_msg">
><br class="gmail_msg">
><br class="gmail_msg">
>> ro_cache_S29GNYAH200016 0A6403AA586531E1<br class="gmail_msg">
>>
/dev/disk/by-id/nvme-Dell_Express_Flash_NVMe_SM1715_1.6TB_SFF_______S29GNYAH200016<br class="gmail_msg">
>> dmm <a href="http://ces1.gsc.wustl.edu" rel="noreferrer" class="gmail_msg" target="_blank">ces1.gsc.wustl.edu</a>
server node<br class="gmail_msg">
><br class="gmail_msg">
><br class="gmail_msg">
> On 12/28/16 5:19 PM, Aaron Knister wrote:<br class="gmail_msg">
>> mmlssnsd -X | grep 0A6403AA58641546<br class="gmail_msg">
><br class="gmail_msg">
> _______________________________________________<br class="gmail_msg">
> gpfsug-discuss mailing list<br class="gmail_msg">
> gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" class="gmail_msg" target="_blank">spectrumscale.org</a><br class="gmail_msg">
> <a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" class="gmail_msg" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br class="gmail_msg">
><br class="gmail_msg">
<br class="gmail_msg">
--<br class="gmail_msg">
Aaron Knister<br class="gmail_msg">
NASA Center for Climate Simulation (Code 606.2)<br class="gmail_msg">
Goddard Space Flight Center<br class="gmail_msg">
<a href="tel:%28301%29%20286-2776" value="+13012862776" class="gmail_msg" target="_blank">(301)
286-2776</a><br class="gmail_msg">
_______________________________________________<br class="gmail_msg">
gpfsug-discuss mailing list<br class="gmail_msg">
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" class="gmail_msg" target="_blank">spectrumscale.org</a><br class="gmail_msg">
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" class="gmail_msg" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br class="gmail_msg">
</blockquote>
</div>
<br class="gmail_msg">
<fieldset class="m_-5832568848232937267mimeAttachmentHeader gmail_msg"></fieldset>
<br class="gmail_msg">
<pre class="gmail_msg">_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at <a href="http://spectrumscale.org" class="gmail_msg" target="_blank">spectrumscale.org</a>
<a class="m_-5832568848232937267moz-txt-link-freetext gmail_msg" href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a>
</pre>
</blockquote>
<br class="gmail_msg">
<br class="gmail_msg">
<fieldset class="m_-5832568848232937267mimeAttachmentHeader gmail_msg"></fieldset>
<br class="gmail_msg">
<pre class="gmail_msg">_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at <a href="http://spectrumscale.org" class="gmail_msg" target="_blank">spectrumscale.org</a>
<a class="m_-5832568848232937267moz-txt-link-freetext gmail_msg" href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a>
</pre>
</blockquote>
<br class="gmail_msg">
</blockquote>
<br class="gmail_msg">
</div>
_______________________________________________<br class="gmail_msg">
gpfsug-discuss mailing list<br class="gmail_msg">
gpfsug-discuss at <a href="http://spectrumscale.org" rel="noreferrer" class="gmail_msg" target="_blank">spectrumscale.org</a><br class="gmail_msg">
<a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss" rel="noreferrer" class="gmail_msg" target="_blank">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br class="gmail_msg">
</blockquote></div>