[gpfsug-discuss] mmcrfs issue

[Aaron Knister]

Those look like zvol's. Out of curiosity have you set sync=always on the 
filesystem root or zvols themselves? It's my understanding that without 
that you risk data loss since GPFS won't ever cause a sync to be issued 
to the zvol for zfs to flush acknowledged but uncommitted writes.

-Aaron

On 3/10/17 4:36 PM, Sanchez, Paul wrote:
> See:
> https://www.ibm.com/support/knowledgecenter/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.adm.doc/bl1adm_nsddevices.htm
>
>
>
> GPFS has a limited set of device search specs it uses to find connected
> NSDs.  When using exotic devices, you need to whitelist the devices
> yourself using the user exit script at /var/mmfs/etc/nsddevices.
>
>
>
> *From:*gpfsug-discuss-bounces at spectrumscale.org
> [mailto:gpfsug-discuss-bounces at spectrumscale.org] *On Behalf Of
> *Buterbaugh, Kevin L
> *Sent:* Friday, March 10, 2017 3:44 PM
> *To:* gpfsug main discussion list
> *Subject:* [gpfsug-discuss] mmcrfs issue
>
>
>
> Hi All,
>
>
>
> We are testing out some flash storage.  I created a couple of NSDs
> successfully (?):
>
>
>
> root at nec:~/gpfs# mmlsnsd -F
>
>
>
>  File system   Disk name    NSD servers
>
> ---------------------------------------------------------------------------
>
>  (free disk)   nsd1         nec
>
>  (free disk)   nsd2         nec
>
>
>
> root at nec:~/gpfs#
>
>
>
> So I tried to create a filesystem:
>
>
>
> root at nec:~/gpfs# mmcrfs gpfs0 -F ~/gpfs/flash.stanza -A yes -B 1M -j
> scatter -k all -m 1 -M 3 -Q no -r 1 -R 3 -T /gpfs0
>
> GPFS: 6027-441 Unable to open disk 'nsd2' on node nec.
>
> GPFS: 6027-441 Unable to open disk 'nsd1' on node nec.
>
> No such device
>
> No such device
>
> GPFS: 6027-538 Error accessing disks.
>
> mmcrfs: 6027-1200 tscrfs failed.  Cannot create gpfs0
>
> mmcrfs: 6027-1639 Command failed. Examine previous error messages to
> determine cause.
>
> root at nec:~/gpfs#
>
>
>
> Does this output from readdescraw look normal?
>
>
>
> root at nec:~/gpfs# mmfsadm test readdescraw /dev/zd16
>
> NSD descriptor in sector 64 of /dev/zd16
>
>     NSDid: 0A0023D258C1C02C  format version: 1403 Label:
>
>     Paxos sector: -1931478434 number of sectors: 8192 isPdisk: 0
>
>     Comment: NSD descriptor for <unknown> Thu Mar  9 14:50:52 2017
>
> No Disk descriptor in sector 96 of /dev/zd16
>
> No FS descriptor in sector 2048 of /dev/zd16
>
> root at nec:~/gpfs# mmfsadm test readdescraw /dev/zd32
>
> NSD descriptor in sector 64 of /dev/zd32
>
>     NSDid: 0A0023D258C1C02B  format version: 1403 Label:
>
>     Paxos sector: -1880562609 number of sectors: 8192 isPdisk: 0
>
>     Comment: NSD descriptor for <unknown> Thu Mar  9 14:50:51 2017
>
> No Disk descriptor in sector 96 of /dev/zd32
>
> No FS descriptor in sector 2048 of /dev/zd32
>
> root at nec:~/gpfs#
>
>
>
> Thanks in advance, all…
>
>
>
> Kevin
>
> —
>
> Kevin Buterbaugh - Senior System Administrator
>
> Vanderbilt University - Advanced Computing Center for Research and Education
>
> Kevin.Buterbaugh at vanderbilt.edu
> <mailto:Kevin.Buterbaugh at vanderbilt.edu> - (615)875-9633
>
>
>
>
>
>
>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at spectrumscale.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss
>

-- 
Aaron Knister
NASA Center for Climate Simulation (Code 606.2)
Goddard Space Flight Center
(301) 286-2776