[gpfsug-discuss] default value of SIZE in EXTERNAL POOL and EXTERNAL LIST rule

Alec anacreo at gmail.com
Thu Mar 31 00:39:24 BST 2022 

I'd like to offer a dissenting opinion...  I believe this is a way to limit
the number of actual bytes that any execution must work on....  As in only
send 500gb worth of files to each invocation.  Whereas the -B is the actual
number of files to be passed to each invocation.  I would think that size
has no default limit as I've had a single invocation stuck with 30TB of
data in its 100 file grouping, lol.

I however use a different mechanism to divy up my files because I need to
worry about my backup streams getting choked up, and I want to take a cross
section of each directory on different nights...

I call mmapplypolicy with -M shares=<someincrementingnumber>, so let's say
I increase my number by 1 every day, starting at 0, over the course of a
week...

In my policy I have this line:
  WHERE MOD(INODE,6)<=shares

So then I work on 1/7th of the files in my file system and cumulatively
pick up any INODES until I've hit all of the files... I could pick just the
matching segment by saying = instead of <=.

This is happening at the file selection phase, so the application of -B is
after this selection, as my files are quite huge, I tend to use -B 1.

Alec

On Wed, Mar 30, 2022 at 3:44 PM IBM Spectrum Scale <scale at us.ibm.com> wrote:

> Hi shao feng,
>
> It is controlled by -B option.
>
> *-B* *MaxFiles*
>          Specifies how many files are passed for each invocation
>          of the EXEC script. The default value is 100.
>
>          If the number of files exceeds the value specified for
>          MaxFiles, *mmapplypolicy* invokes the external
>          program multiple times.
>
>          For more information about file list records, refer to
>          the IBM Spectrum Scale: Administration Guide.
>
> Regards, The Spectrum Scale (GPFS) team
>
>
> ------------------------------------------------------------------------------------------------------------------
> If your query concerns a potential software error in Spectrum Scale (GPFS)
> and you have an IBM software maintenance contract please contact
>  1-800-237-5511 in the United States or your local IBM Service Center in
> other countries.
>
> [image: Inactive hide details for IBM Spectrum Scale---03/30/2022 09:53:51
> AM---Hi Diane, Can you please look at the below policy relat]IBM Spectrum
> Scale---03/30/2022 09:53:51 AM---Hi Diane, Can you please look at the below
> policy related query.
>
> From: IBM Spectrum Scale/Poughkeepsie/IBM at IBMUS
> To: "gpfsug main discussion list" <gpfsug-discuss at gpfsug.org>, Diane
> Chung/Poughkeepsie/IBM at IBMMail
> Cc: "gpfsug-discuss" <gpfsug-discuss-bounces at gpfsug.org>
> Date: 03/30/2022 09:53 AM
> Subject: [EXTERNAL] Re: [gpfsug-discuss] default value of SIZE in
> EXTERNAL POOL and EXTERNAL LIST rule
> Sent by: "gpfsug-discuss" <gpfsug-discuss-bounces at gpfsug.org>
> ------------------------------
>
>
>
> Hi Diane,
>
> Can you please look at the below policy related query.
>
> Regards, The Spectrum Scale (GPFS) team
>
>
> ------------------------------------------------------------------------------------------------------------------
> If you feel that your question can benefit other users of  Spectrum Scale
> (GPFS), then please post it to the public IBM developerWroks Forum at
> *https://www.ibm.com/developerworks/community/forums/html/forum?id=11111111-0000-0000-0000-000000000479*
> <https://www.ibm.com/developerworks/community/forums/html/forum?id=11111111-0000-0000-0000-000000000479>.
>
>
> If your query concerns a potential software error in Spectrum Scale (GPFS)
> and you have an IBM software maintenance contract please contact
>  1-800-237-5511 in the United States or your local IBM Service Center in
> other countries.
>
> The forum is informally monitored as time permits and should not be used
> for priority messages to the Spectrum Scale (GPFS) team.
>
> [image: Inactive hide details for "shao feng" ---28-03-2022 03.21.39
> PM---Hello, What is the default value of SIZE in EXTERNAL POOL in]"shao
> feng" ---28-03-2022 03.21.39 PM---Hello, What is the default value of SIZE
> in EXTERNAL POOL in filesystem policy?
>
> From: "shao feng" <shaof777 at gmail.com>
> To: "gpfsug main discussion list" <gpfsug-discuss at gpfsug.org>
> Date: 28-03-2022 03.21 PM
> Subject: [EXTERNAL] [gpfsug-discuss] default value of SIZE in EXTERNAL
> POOL and EXTERNAL LIST rule
> Sent by: "gpfsug-discuss" <gpfsug-discuss-bounces at gpfsug.org>
> ------------------------------
>
>
>
> Hello,
>
> What is the default value of SIZE in EXTERNAL POOL in filesystem policy?
>
> The document did not mention that, but I found it is not infinity if not
> specified.
>
> Thank you!
>
> *SIZE **sum-number*
> Is an optional clause of the *EXTERNAL POOL* and *EXTERNAL LIST* rules.
> *sum-number* limits the total number of bytes in all of the files named
> in each list of files passed to your EXEC 'script'. If a single file is
> larger than *sum-number*, it is passed to your EXEC 'script' as the only
> entry in a “singleton” file list.
>
> Specify *sum-number* as a numeric constant or a floating-point value.
>
> *Note:* The value of *sum-number* is in kilobytes.
> *https://www.ibm.com/docs/en/spectrum-scale/5.1.2?topic=rules-policy-terms*
> <https://www.ibm.com/docs/en/spectrum-scale/5.1.2?topic=rules-policy-terms>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> *http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org*
> <http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org>
>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
>
>
>
>
> _______________________________________________
> gpfsug-discuss mailing list
> gpfsug-discuss at gpfsug.org
> http://gpfsug.org/mailman/listinfo/gpfsug-discuss_gpfsug.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20220330/34296ca4/attachment-0002.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://gpfsug.org/pipermail/gpfsug-discuss_gpfsug.org/attachments/20220330/34296ca4/attachment-0002.gif>

More information about the gpfsug-discuss mailing list