[gpfsug-discuss] Multipath configurations

Jonathan Buzzard jonathan at buzzard.me.uk
Thu Sep 19 16:38:05 BST 2013

On Thu, 2013-09-19 at 15:53 +0100, Orlando Richards wrote:


> Along the way we discovered an odd GPFS "feature" - if some nodes in the 
> cluster use RDAC (and thus have /dev/sdXX devices) and some use 
> multipathd (and thus use /dev/dm-XX devices), then the nodes can either 
> fail to find attached NSD devices (in the case of the RDAC host where 
> the NSD's were initially created on a multipath host) or can try to talk 
> to them down the wrong device (for instance - talking to /dev/sdXX 
> rather than /dev/dm-XX). We just set up this mixed environment to 
> compare rdac vs dm-multipath, and don't expect to put it into production 
> - but it's the kind of thing which could end up cropping up in a system 
> migrating from RDAC to dm-multipath, or vice versa. It seems that on 
> creation, the nsd is tagged somewhere as either "dmm" (dm-multipath) or 
> "generic" (rdac), and servers using one type can't see the other.

That is all covered in the GPFS documentation under "don't do that" ;-)


> Thankfully, we have no plans to mix the environment - but for future 
> reference it could be important (if ever migrating existing systems from 
> rdac to dm-multipath, for instance).

The recommendation is roughly umount GPFS everywhere, export the file
system, change everything to dm-multipath/RDAC depending on your poison,
import file system and mount again. I think you can skip the
export/import by doing some fiddling to switch the NSD type. 


Jonathan A. Buzzard                 Email: jonathan (at) buzzard.me.uk
Fife, United Kingdom.

More information about the gpfsug-discuss mailing list