[gpfsug-discuss] gpfsug-discuss Digest, Vol 108, Issue 18

Thu Jan 28 19:54:38 GMT 2021

sounds quite complicated. 
if all public holidays can be ignored it is simple: 
the algorithm has only to run on week days (the effective age of files 
would not change on weekend days.). 
To find the latest date to remove files:

Now, enumerate the weekdays, starting with Mon=1

If your max age is T
find the integer multiple of 5 and the remainder such that T=T_i*5 +R

Determine the current DoW in terms of your enumeration. 

if DoW - R > 0, your max age date is Dx=D-(R+7*T_i)
else your max age date is Dx=D-(R+2+7*T_i

dates can be easily compiled in epoch, like D_e=$(date +%s), Dx_e = D_e - 
86400*(R+7*T_i) or Dx_e = D_e - 86400*(R+2+7*T_i)
you then need to convert the found epoch time back into a christian date 
which could be done by date --date='@<epoch_time'

example: 
T=16
today is Thu, Jan 28. 
T_i=3
R=1
DoW=4

D_e=1611863080  ## y.m.m.v
Dx_e=1611863080-86400*(2+7*3)=1609875880

$ date --date='@1609875880'
Tue  5 Jan 20:44:40 CET 2021

Just made that up, but it should do the trick

Mit freundlichen Grüßen / Kind regards

Dr. Uwe Falke
IT Specialist
Hybrid Cloud Infrastructure / Technology Consulting & Implementation 
Services
+49 175 575 2877 Mobile
Rochlitzer Str. 19, 09111 Chemnitz, Germany
uwefalke at de.ibm.com

IBM Services

IBM Data Privacy Statement

IBM Deutschland Business & Technology Services GmbH
Geschäftsführung: Sven Schooss, Stefan Hierl
Sitz der Gesellschaft: Ehningen
Registergericht: Amtsgericht Stuttgart, HRB 17122

From:   "Madhav Ponamgi1" <mzp at us.ibm.com>
To:     gpfsug-discuss at spectrumscale.org
Date:   28/01/2021 19:43
Subject:        [EXTERNAL] Re: [gpfsug-discuss] gpfsug-discuss Digest, Vol 
108, Issue 18
Sent by:        gpfsug-discuss-bounces at spectrumscale.org

To calculate this directly (if you don't want to depend on a utility) 
consider the following steps. There
are many more such algorithms in the wonderful book Calenderical 
Calculations.

1.      Take the last two digits of the year. 
2.      Divide by 4, discarding any fraction. 
3.      Add the day of the month. 
4.      Add the month's key value: JFM AMJ JAS OND 144 025 036 146 
5.      Subtract 1 for January or February of a leap year. 
6.      For a Gregorian date, add 0 for 1900's, 6 for 2000's, 4 for 
1700's, 2 for 1800's; for other years, add or subtract multiples of 400. 
7.      For a Julian date, add 1 for 1700's, and 1 for every additional 
century you go back. 
8.      Add the last two digits of the year. 
9.      Divide by 7 and take the remainder. 
---
Madhav
mzp at us.ibm.com

gpfsug-discuss-request---01/28/2021 01:32:13 PM---Send gpfsug-discuss 
mailing list submissions to gpfsug-discuss at spectrumscale.org

From: gpfsug-discuss-request at spectrumscale.org
To: gpfsug-discuss at spectrumscale.org
Date: 01/28/2021 01:32 PM
Subject: [EXTERNAL] gpfsug-discuss Digest, Vol 108, Issue 18
Sent by: gpfsug-discuss-bounces at spectrumscale.org

Send gpfsug-discuss mailing list submissions to
gpfsug-discuss at spectrumscale.org

To subscribe or unsubscribe via the World Wide Web, visit
http://gpfsug.org/mailman/listinfo/gpfsug-discuss 
or, via email, send a message with subject or body 'help' to
gpfsug-discuss-request at spectrumscale.org

You can reach the person managing the list at
gpfsug-discuss-owner at spectrumscale.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of gpfsug-discuss digest..."

Today's Topics:

  1. Re: Policy Rules Syntax to find files older than X days
     excluding weekends in the calculation.... (Owen Morgan)
  2. Number of vCPUs exceeded (Mervini, Joseph A)

----------------------------------------------------------------------

Message: 1
Date: Thu, 28 Jan 2021 14:27:35 +0000
From: Owen Morgan <owen.morgan at motionpicturesolutions.com>
To: "mark.bergman at uphs.upenn.edu" <mark.bergman at uphs.upenn.edu>,
"gpfsug-discuss at spectrumscale.org" <gpfsug-discuss at spectrumscale.org>
Subject: Re: [gpfsug-discuss] Policy Rules Syntax to find files older
than X days excluding weekends in the calculation....
Message-ID:
<VI1PR09MB399809E3E571E7471276FD0B88BA9 at VI1PR09MB3998.eurprd09.prod.outlook.com>

Content-Type: text/plain; charset="utf-8"

Mark,

Thank you for taking the time to comment, I genuinely appreciate it!

I will digest and look at the mmfind examples (to be honest, didn't know 
it was a thing.....).

Everything I know about Spectrum Scale (and Spectrum Archive) has been 
self taught so...... I'm pretty sure I'm missing Soooooooooo much useful 
info!

I wish there was like a dummies guide (I've read the redbooks and admin 
guides as best I can but I know my knowledge is patchy at best)!

Once digested I may, or may not, have further questions but I genuinely 
thank you for your assistance.

Owen.
[Sent from Front]

Owen Morgan
Data Wrangler
Motion Picture Solutions Ltd
T: 
E: owen.morgan at motionpicturesolutions.com | W: motionpicturesolutions.com
A: Mission Hall, 9-11 North End Road, London, W14 8ST
Motion Picture Solutions Ltd is a company registered in England and Wales 
under number 5388229, VAT number 201330482
On Wed, Jan 27 at 11:53 pm, <mark.bergman at uphs.upenn.edu<
mailto:mark.bergman at uphs.upenn.edu>> mark.bergman at uphs.upenn.edu<
mailto:mark.bergman at uphs.upenn.edu> wrote:

In the message dated: Wed, 27 Jan 2021 22:17:09 +0000,
The pithy ruminations from Owen Morgan on
[[External] [gpfsug-discuss] Policy Rules Syntax to find files older than 
X days excluding weekends in the calculation....] were:
=> Hi Everyone,
=>
=> First question from me I appreciate this is policy engine thing as
=> opposed to more fundamental Spectrum Scale so hope its ok!

It's great.

=>
=> I'm trying to find a 'neat' way within a couple of policy rules to
=> measure different time intervals (in days) but solely interested in 
WEEK
=> DAYS only (ie delete files older than X week days only).

Policy SQL syntax gives me a headache. For this kind of task, I find
that mmfind is your friend -- it's in the "examples" source dir within
/usr/lpp/mmfs. Trivial to compile & install. Easier to debug, and it
will generate the SQL.

=>
=> An example is one of the rules a team would like implemented is delete
=> all files older than 10 business days (ie week days only. We are

What about "delete all files older than 12 calendar days" -- by
definition, those files are older than 10 business days as well.

=> ignoring public holidays as if they don't exist). Followed by a 
separate
=> rule for a different folder of deleting all files older than 4 business
=> days.

Or, older than 6 calendar days.

Or, run this nightly:

#! /bin/bash
dateOffset=0
if [ `date '+%u'` -le 4 ] ; then
# Mon=1, Tue=2, Wed=3, Thu=4
#
# For a file to be more than 4 business days old on-or-before the
# 4th day of the week, it must span the weekend, so offset the number
# of required days in the file age
dateOffset=2
fi

mmfind -mtime $((4 + $dateOffset)) /path/to/Nuke/After/4/Days -xarg rm -f

=>
=> Thanks in advance,
=>
=> Owen. [Sent from Front]
=>
=> Owen Morgan Data Wrangler Motion Picture Solutions Ltd T: E:
=> owen.morgan at motionpicturesolutions.com<
mailto:owen.morgan at motionpicturesolutions.com> | W: 
motionpicturesolutions.com<http://motionpicturesolutions.com >
=> A: Mission Hall, 9-11 North End Road, London, W14 8ST Motion Picture
=> Solutions Ltd is a company registered in England and Wales under number
=> 5388229, VAT number 201330482
=>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20210128/201a280e/attachment-0001.html 
>

------------------------------

Message: 2
Date: Thu, 28 Jan 2021 18:26:37 +0000
From: "Mervini, Joseph A" <jamervi at sandia.gov>
To: "gpfsug-discuss at spectrumscale.org"
<gpfsug-discuss at spectrumscale.org>
Subject: [gpfsug-discuss] Number of vCPUs exceeded
Message-ID: <59193954-B649-4DF5-AD21-652922E49FD9 at contoso.com>
Content-Type: text/plain; charset="utf-8"

Hi,

I haven?t seen this before but one of my remote cluster users reported the 
system in question is experiencing high loads and is with Scale unmounting 
the file system.

This is the output she is seeing:

Wed Jan 27 22:18:34.168 2021: [I] GPFS vCPU limits: Low warning limit 3 
vCPUs, High warning limit 256 vCPUs, Hard limit 1536 vCPUs.
Wed Jan 27 22:18:34.169 2021: [I] GPFS vCPU limits include all vCPUs that 
Linux sees as online or possibly online via hot add, ht/smt changes, etc.
Wed Jan 27 22:18:34.170 2021: [X] GPFS detected 1792 vCPUs. This exceeds 
the warning limit of 256 vCPUs and the hard limit of 1536 vCPUs. GPFS will 
shutdown

Any help will be appreciated.

Thanks,
Joe

====

Joe Mervini
Sandia National Laboratories
High Performance Computing
505.844.6770
jamervi at sandia.gov<mailto:jamervi at sandia.gov>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20210128/930fadb1/attachment.html 
>

------------------------------

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss 

End of gpfsug-discuss Digest, Vol 108, Issue 18
***********************************************

_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss 