[gpfsug-discuss] Backend corruption
Stef Coene
stef.coene at docum.org
Tue Aug 4 07:41:26 BST 2020
Hi,
I tried to use a policy to find out what files are located on the broken
disks.
But this is not finding any files or directories (I cleaned some of the
output):
[I] GPFS Current Data Pool Utilization in KB and %
Pool_Name KB_Occupied KB_Total Percent_Occupied
V500003 173121536 69877104640 0.247751444%
[I] 29609813 of 198522880 inodes used: 14.915063%.
[I] Loaded policy rules from test.rule.
rule 'ListRule'
list 'ListName'
from pool 'V500003'
[I] Directories scan: 28649029 files, 960844 directories, 0 other
objects, 0 'skipped' files and/or errors.
[I] Inodes scan: 28649029 files, 960844 directories, 0 other objects, 0
'skipped' files and/or errors.
[I] Summary of Rule Applicability and File Choices:
Rule# Hit_Cnt KB_Hit Chosen KB_Chosen
KB_Ill Rule
0 0 0 0 0
0 RULE 'ListRule' LIST 'ListName' FROM POOL 'V500003'
[I] Filesystem objects with no applicable rules: 29609873.
[I] A total of 0 files have been migrated, deleted or processed by an
EXTERNAL EXEC/script;
0 'skipped' files and/or errors.
So the policy is not finding any files but there is still some data on
the V50003 pool?
Stef
On 2020-08-03 17:21, Uwe Falke wrote:
> Hi, Stef,
>
> if just that V5000 has provided the storage for one of your pools
> entirely, and if your metadata are still incorrupted, a inode scan with a
> suited policy should yield the list of files on that pool.
> If I am not mistaken, the list policy could look like
>
> RULE 'list_v5000' LIST 'v5000_filelist' FROM POOL <your_v5000_pool>
>
> paut it into a (policy) file, run that by mmapplypolicy against the file
> system in question, it should produce a file listing in
> /tmp/v5000_filelist. If it doesn#T work exactly like that (I might have
> made one or mor mistakes), check out the information lifycacle section in
> the scal admin guide.
>
> If the prereqs for the above are not met, you need to run more expensive
> investigations (using tsdbfs for all block addresses on v5000-provided
> NSDs).
>
> Mit freundlichen Grüßen / Kind regards
>
> Dr. Uwe Falke
> IT Specialist
> Global Technology Services / Project Services Delivery / High Performance
> Computing
> +49 175 575 2877 Mobile
> Rathausstr. 7, 09111 Chemnitz, Germany
> uwefalke at de.ibm.com
>
> IBM Services
>
> IBM Data Privacy Statement
>
> IBM Deutschland Business & Technology Services GmbH
> Geschäftsführung: Dr. Thomas Wolter, Sven Schooss
> Sitz der Gesellschaft: Ehningen
> Registergericht: Amtsgericht Stuttgart, HRB 17122
>
>
>
> From: Stef Coene <stef.coene at docum.org>
> To: gpfsug-discuss at spectrumscale.org
> Date: 03/08/2020 16:07
> Subject: [EXTERNAL] [gpfsug-discuss] Backend corruption
> Sent by: gpfsug-discuss-bounces at spectrumscale.org
>
>
>
> Hi,
>
> We have a GPFS file system which uses, among other storage, a V5000 as
> backend.
> There was an error in the fire detection alarm in the datacenter and a
> fire alarm was triggered.
> The result was that the V5000 had a lot of broken disks. Most of the
> disks recovered fine after a reseat, but some data is corrupted on the
> V5000.
>
> This means that for 22MB of data, the V5000 returns a read error to the
> GPFS.
>
> We migrated most of the data to an disks but there is still 165 GB left
> on the V5000 pool.
>
> When we try to remove the disks with mmdeldisk, it fails after a while
> and places some of the disks as down.
> It generated a file with inodes, this an example of a 2 lines:
> 9168519 0:0 0 1 1
> exposed illreplicated illplaced REGULAR_FILE Error: 218 Input/output
> error
> 9251611 0:0 0 1 1
> exposed illreplicated REGULAR_FILE Error: 218 Input/output error
>
>
> How can I get a list of files that uses data of the V5000 pool?
> The data is written by CommVault. When I have a list of files, I can
> determine the impact on the application.
>
>
> Stef
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=fTuVGtgq6A14KiNeaGfNZzOOgtHW5Lm4crZU6lJxtB8&m=HhbxQEWLNTXFDCFT5LDpMD4YvYTUEdl6Nt6IgjdVlNo&s=fxsoDddp4OUnP7gORNUOnAmrnHPIU57OQMnraXEEO0k&e=
>
>
>
>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>
More information about the gpfsug-discuss
mailing list