[gpfsug-discuss] Spectrum Scale and Firewalls

[Simon Thompson]

Hi,

We’re having some issues bringing up firewalls on some of our NSD nodes. The problem I was actually trying to diagnose I don’t think is firewall related but still …

We have port 22 and 1191 open and also 60000-61000, we also set:
# mmlsconfig tscTcpPort
tscTcpPort 1191
# mmlsconfig tscCmdPortRange
tscCmdPortRange 60000-61000

https://www.ibm.com/support/knowledgecenter/en/STXKQY_5.0.0/com.ibm.spectrum.scale.v5r00.doc/bl1adv_firewallforinternalcommn.htm
Claims this is sufficient …

Running mmnetverify:
# mmnetverify all --target-nodes rds-er-mgr01

rds-pg-mgr01 checking local configuration.
  Operation interface: Success.

rds-pg-mgr01 checking communication with node rds-er-mgr01.
  Operation resolution: Success.
  Operation ping: Success.
  Operation shell: Success.
  Operation copy: Success.
  Operation time: Success.
  Operation daemon-port: Success.
  Operation sdrserv-port: Success.
  Operation tsccmd-port: Success.
  Operation data-small: Success.
  Operation data-medium: Success.
  Operation data-large: Success.
Could not connect to port 46326 on node rds-pg-mgr01 (10.20.0.56): timed out.
This may indicate a firewall configuration issue.
  Operation bandwidth-node: Fail.

rds-pg-mgr01 checking cluster communications.

Issues Found:
rds-er-mgr01 could not connect to rds-pg-mgr01 (TCP, port 46326).

mmnetverify: Command failed. Examine previous error messages to determine cause.


Note that the port number mentioned changes if we run mmnetverify multiple times. The two clients in this test are running 5.0.2 code. If I run in verbose mode I see:
<snip>
  Checking network communication with node rds-er-mgr01.
    Port range restricted by cluster configuration: 60000 - 61000.
    rds-er-mgr01: connecting to node rds-pg-mgr01.
    rds-er-mgr01: exchanged 256.0M bytes with rds-pg-mgr01.
      Write size: 16.0M bytes.
    Network statistics for rds-er-mgr01 during data exchange:
      packets sent: 68112
      packets received: 72452
    Network Traffic between rds-er-mgr01 and rds-pg-mgr01 port 60000 ok.
  Operation data-large: Success.
  Checking network bandwidth.
    rds-er-mgr01: connecting to node rds-pg-mgr01.
Could not connect to port 36277 on node rds-pg-mgr01 (10.20.0.56): timed out.
This may indicate a firewall configuration issue.
  Operation bandwidth-node: Fail.
<snip>

So for many of the tests it looks like its using port 60000 as expected, is this just a bug in mmnetverify or am I doing something silly?

Thanks

Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20181019/0bdcbf2e/attachment-0001.htm>