[gpfsug-discuss] GPFS nodes crashing during policy scan

[Simon Thompson]

Just a gentle reminder that this is a community based list and that we expect people to be respectful of each other on the list.

Simon
________________________________________
From: gpfsug-discuss-bounces at spectrumscale.org [gpfsug-discuss-bounces at spectrumscale.org] on behalf of matthew.robinson02 at gmail.com [matthew.robinson02 at gmail.com]
Sent: 06 December 2018 17:41
To: gpfsug main discussion list
Subject: Re: [gpfsug-discuss] GPFS nodes crashing during policy scan

<Snip>


On Thu, Dec 6, 2018 at 11:46 AM Ratliff, John <jdratlif at iu.edu<mailto:jdratlif at iu.edu>> wrote:
We’re trying to run a policy scan to get a list of all the files in one of our filesets. There are approximately 600 million inodes in this space. We’re running GPFS 3.5. Every time we run the policy scan, the node that is running it ends up crashing. It makes it through a quarter of the inodes before crashing (i.e. kernel panic and system reboot). Nothing in the GPFS logs shows anything. It just notes that the node rebooted.

In the crash logs of all the systems we’ve tried this on, we see the same line.

<1>BUG: unable to handle kernel NULL pointer dereference at 00000000000000d8
<1>IP: [<ffffffffa03fb7b2>] _ZN6Direct5dreadEP15KernelOperationRK7FileUIDxiiiPvPFiS5_PKcixyS5_EPx+0xf2/0x590 [mmfs26]

Our policy scan rule is pretty simple:

RULE 'list-homedirs'
    LIST 'list-homedirs'

mmapplypolicy /gs/home -A 607 -g /gpfs/tmp -f /gpfs/policy/output -N gpfs1,gpfs2,gpfs3,gpfs4 -P /tmp/homedirs.policy -I defer -L 1

Has anyone experienced something like this or have any suggestions on what to do to avoid it?

Thanks.

John Ratliff | Pervasive Technology Institute | UITS | Research Storage – Indiana University | http://pti.iu.edu

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org<http://spectrumscale.org>
http://gpfsug.org/mailman/listinfo/gpfsug-discuss


--
Matthew Robinson
Comptia A+, Net+
919.909.0494
matthew.robinson02 at gmail.com<mailto:matthew.robinson02 at gmail.com>

The greatest discovery of my generation is that man can alter his life simply by altering his attitude of mind. - William James, Harvard Psychologist.