[gpfsug-discuss] LROC 100% utilized in terms of IOs
Matt Weil
mweil at wustl.edu
Fri Jan 27 15:49:12 GMT 2017
turning off data seems to have helped this issue
Thanks all
On 1/26/17 9:37 AM, Matt Weil wrote:
100% utilized are bursts above 200,000 IO's. Any way to tell ganesha.nfsd to cache more?
On 1/25/17 3:51 PM, Matt Weil wrote:
[ces1,ces2,ces3]
maxStatCache 80000
worker1Threads 2000
maxFilesToCache 500000
pagepool 100G
maxStatCache 80000
lrocData no
378G system memory.
On 1/25/17 3:29 PM, Sven Oehme wrote:
have you tried to just leave lrocInodes and lrocDirectories on and turn data off ?
yes data I just turned off
also did you increase maxstatcache so LROC actually has some compact objects to use ?
if you send value for maxfilestocache,maxfilestocache,workerthreads and available memory of the node i can provide a start point.
On Wed, Jan 25, 2017 at 10:20 PM Matt Weil <mweil at wustl.edu<mailto:mweil at wustl.edu>> wrote:
On 1/25/17 3:00 PM, Sven Oehme wrote:
Matt,
the assumption was that the remote devices are slower than LROC. there is some attempts in the code to not schedule more than a maximum numbers of outstanding i/os to the LROC device, but this doesn't help in all cases and is depending on what kernel level parameters for the device are set. the best way is to reduce the max size of data to be cached into lroc.
I just turned LROC file caching completely off. most if not all of the IO is metadata. Which is what I wanted to keep fast. It is amazing once you drop the latency the IO's go up way more than they ever where before. I guess we will need another nvme.
sven
On Wed, Jan 25, 2017 at 9:50 PM Matt Weil <mweil at wustl.edu<mailto:mweil at wustl.edu>> wrote:
Hello all,
We are having an issue where the LROC on a CES node gets overrun 100%
utilized. Processes then start to backup waiting for the LROC to
return data. Any way to have the GPFS client go direct if LROC gets to
busy?
Thanks
Matt
________________________________
The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://spectrumscale.org>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://spectrumscale.org>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
________________________________
The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://spectrumscale.org>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
________________________________
The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
________________________________
The materials in this message are private and may contain Protected Healthcare Information or other information of a sensitive nature. If you are not the intended recipient, be advised that any unauthorized use, disclosure, copying or the taking of any action in reliance on the contents of this information is strictly prohibited. If you have received this email in error, please immediately notify the sender via telephone or return mail.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20170127/bcd9a8eb/attachment-0002.htm>
More information about the gpfsug-discuss
mailing list