[gpfsug-discuss] mmcrfs issue

Daniel Kidger daniel.kidger at uk.ibm.com
Sat Mar 11 10:37:47 GMT 2017


On the subject of using zvols for software Raid/ replication, can ask as a
quick poll, how many people are doing this?

And any feedback on stability, tuning and performance?

Daniel
IBM Technical Presales

> On 10 Mar 2017, at 22:44, Aaron Knister <aaron.s.knister at nasa.gov> wrote:
>
> Those look like zvol's. Out of curiosity have you set sync=always on the
> filesystem root or zvols themselves? It's my understanding that without
> that you risk data loss since GPFS won't ever cause a sync to be issued
> to the zvol for zfs to flush acknowledged but uncommitted writes.
>
> -Aaron
>
>> On 3/10/17 4:36 PM, Sanchez, Paul wrote:
>> See:
>>
https://www.ibm.com/support/knowledgecenter/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.adm.doc/bl1adm_nsddevices.htm

>>
>>
>>
>> GPFS has a limited set of device search specs it uses to find connected
>> NSDs.  When using exotic devices, you need to whitelist the devices
>> yourself using the user exit script at /var/mmfs/etc/nsddevices.
>>
>>
>>
>> *From:*gpfsug-discuss-bounces at spectrumscale.org
>> [mailto:gpfsug-discuss-bounces at spectrumscale.org] *On Behalf Of
>> *Buterbaugh, Kevin L
>> *Sent:* Friday, March 10, 2017 3:44 PM
>> *To:* gpfsug main discussion list
>> *Subject:* [gpfsug-discuss] mmcrfs issue
>>
>>
>>
>> Hi All,
>>
>>
>>
>> We are testing out some flash storage.  I created a couple of NSDs
>> successfully (?):
>>
>>
>>
>> root at nec:~/gpfs# mmlsnsd -F
>>
>>
>>
>> File system   Disk name    NSD servers
>>
>>
---------------------------------------------------------------------------
>>
>> (free disk)   nsd1         nec
>>
>> (free disk)   nsd2         nec
>>
>>
>>
>> root at nec:~/gpfs#
>>
>>
>>
>> So I tried to create a filesystem:
>>
>>
>>
>> root at nec:~/gpfs# mmcrfs gpfs0 -F ~/gpfs/flash.stanza -A yes -B 1M -j
>> scatter -k all -m 1 -M 3 -Q no -r 1 -R 3 -T /gpfs0
>>
>> GPFS: 6027-441 Unable to open disk 'nsd2' on node nec.
>>
>> GPFS: 6027-441 Unable to open disk 'nsd1' on node nec.
>>
>> No such device
>>
>> No such device
>>
>> GPFS: 6027-538 Error accessing disks.
>>
>> mmcrfs: 6027-1200 tscrfs failed.  Cannot create gpfs0
>>
>> mmcrfs: 6027-1639 Command failed. Examine previous error messages to
>> determine cause.
>>
>> root at nec:~/gpfs#
>>
>>
>>
>> Does this output from readdescraw look normal?
>>
>>
>>
>> root at nec:~/gpfs# mmfsadm test readdescraw /dev/zd16
>>
>> NSD descriptor in sector 64 of /dev/zd16
>>
>>    NSDid: 0A0023D258C1C02C  format version: 1403 Label:
>>
>>    Paxos sector: -1931478434 number of sectors: 8192 isPdisk: 0
>>
>>    Comment: NSD descriptor for <unknown> Thu Mar  9 14:50:52 2017
>>
>> No Disk descriptor in sector 96 of /dev/zd16
>>
>> No FS descriptor in sector 2048 of /dev/zd16
>>
>> root at nec:~/gpfs# mmfsadm test readdescraw /dev/zd32
>>
>> NSD descriptor in sector 64 of /dev/zd32
>>
>>    NSDid: 0A0023D258C1C02B  format version: 1403 Label:
>>
>>    Paxos sector: -1880562609 number of sectors: 8192 isPdisk: 0
>>
>>    Comment: NSD descriptor for <unknown> Thu Mar  9 14:50:51 2017
>>
>> No Disk descriptor in sector 96 of /dev/zd32
>>
>> No FS descriptor in sector 2048 of /dev/zd32
>>
>> root at nec:~/gpfs#
>>
>>
>>
>> Thanks in advance, all…
>>
>>
>>
>> Kevin
>>
>>>>
>> Kevin Buterbaugh - Senior System Administrator
>>
>> Vanderbilt University - Advanced Computing Center for Research and
Education
>>
>> Kevin.Buterbaugh at vanderbilt.edu
>> <mailto:Kevin.Buterbaugh at vanderbilt.edu> - (615)875-9633
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> gpfsug-discuss mailing list
>> gpfsug-discuss at spectrumscale.org
>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>>
>
> --
> Aaron Knister
> NASA Center for Climate Simulation (Code 606.2)
> Goddard Space Flight Center
> (301) 286-2776
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598. 
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20170311/714907a9/attachment-0002.htm>


More information about the gpfsug-discuss mailing list