[gpfsug-discuss] ILM: migrating between internal pools while premigrated to external
Uwe Falke
UWEFALKE at de.ibm.com
Tue Apr 24 16:20:14 BST 2018
(Sorry, vi commands and my mailing client do not co-operate well ...Pls
forgive the pre-mature posting just sent. )
Hi, the system is not in action yet.
I am just to plan the migrations right now.
1) + 2)
premigration would be (not listing excludes here, actual cos and FS name
replaced by nn, xxx), executed once per hour:
RULE EXTERNAL POOL 'hsm_aggr_cosnn' EXEC '/opt/hpss/bin/ghi_migrate' OPTS
'-a -c nn'
/* Migrate all files that are smaller than 1 GB to hpss as aggregates. */
RULE 'toHsm_aggr_cosnn' MIGRATE FROM POOL 'pool1'
WEIGHT(CURRENT_TIMESTAMP - ACCESS_TIME)
TO POOL 'hsm_aggr_cosnn' SHOW ('-s' FILE_SIZE)
WHERE path_name like '/xxx/%' AND FILE_SIZE <= 1073741824
AND (CURRENT_TIMESTAMP - MODIFICATION_TIME > INTERVAL '120' MINUTES)
Internal Migration was originally intended like
RULE "pool0_to_pool1" MIGRATE FROM POOL 'pool0' TO POOL 'pool1' WHERE
(CURRENT_TIMESTAMP - MODIFICATION_TIME > INTERVAL '720' MINUTES)
to be ran once per day (plus a thrshold-policy to prevent filling up of
the relative small pool0 should something go wrong)
3) + 4) as that is not yet set up - no answer. I would like to prevent
such things to happen in the first place, therefore asking.
5) I do not understand your question. I am planning the migration set up,
and a suppose there might be the risk for trouble when doing internal
migrations while haveing data pre-migrated to external.
Indeed I am not a developer of the ILM stuff in Scale, so I cannot fully
foresee what'll happen. Therefore asking.
Mit freundlichen Grüßen / Kind regards
Dr. Uwe Falke
IT Specialist
High Performance Computing Services / Integrated Technology Services /
Data Center Services
-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland
Rathausstr. 7
09111 Chemnitz
Phone: +49 371 6978 2165
Mobile: +49 175 575 2877
E-Mail: uwefalke at de.ibm.com
-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland Business & Technology Services GmbH / Geschäftsführung:
Thomas Wolter, Sven Schooß
Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart,
HRB 17122
From: "Marc A Kaplan" <makaplan at us.ibm.com>
To: gpfsug main discussion list <gpfsug-discuss at spectrumscale.org>
Date: 24/04/2018 14:38
Subject: Re: [gpfsug-discuss] ILM: migrating between internal pools
while premigrated to external
Sent by: gpfsug-discuss-bounces at spectrumscale.org
To address this question or problem, please be more specific about:
1) commands and policy rules used to perform or control the migrations and
pre-migrations.
2) how the commands are scheduled, how often, and/or by events and
mmXXcallbacks.
3) how many files are "okay" and how many are ill-placed.
4) how many blocks or KB of storage are ill-placed. (One could determine
this by looking at pool free/used blocks stats, then running
restripe to correct all ill-placements and then looking at pool free/used
stats again.)
5) As commands and migrations are executed, what is happening that you do
not understand or that you believe is incorrect?
Marc K of GPFS (aka "Mister Mmapplypolicy")
From: "Uwe Falke" <UWEFALKE at de.ibm.com>
To: "gpfsug main discussion list"
<gpfsug-discuss at spectrumscale.org>
Date: 04/24/2018 06:43 AM
Subject: [gpfsug-discuss] ILM: migrating between internal pools
while premigrated to external
Sent by: gpfsug-discuss-bounces at spectrumscale.org
Hello, World,
I'd asked this on dW last week but got no reactions:
I am unsure on the viability of this approach:
In a given file system, there are two internal storage pools and one
external (GHI/HPSS).
Certain files are created in one internal pool and are to be migrated to
the other internal pool about one day after file closure.
In addition, all files are to be (pre-)migrated to the external pool
within one hour after closure.
Hence: typically, blocks of a file being already pre-migrated to external
(but not purged on the initial internal pool) are to be moved from one
internal pool to the other.
A slight indication that there might be some issue with such an approach
I've found in a dW post some years back:
https://www.ibm.com/developerworks/community/forums/html/topic?id=77777777-0000-0000-0000-000014797943&ps=25
However, time has passed and this might work now or might not. Is there
some knowledge about that in the community?
If it would not be a reasonable way, we would do the internal migration
before the external one, but that imposes a timely dependance we'd not
have otherwise.
Mit freundlichen Grüßen / Kind regards
Dr. Uwe Falke
IT Specialist
High Performance Computing Services / Integrated Technology Services /
Data Center Services
-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland
Rathausstr. 7
09111 Chemnitz
Phone: +49 371 6978 2165
Mobile: +49 175 575 2877
E-Mail: uwefalke at de.ibm.com
-------------------------------------------------------------------------------------------------------------------------------------------
IBM Deutschland Business & Technology Services GmbH / Geschäftsführung:
Thomas Wolter, Sven Schooß
Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart,
HRB 17122
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwIFAw&c=jf_iaSHvJObTbx-siA1ZOg&r=cvpnBBH0j41aQy0RPiG2xRL_M8mTc1izuQD3_PmtjZ8&m=j-JJhNxx8XhgizXxQDuJolplgYxVRVTS6zalCPshD-0&s=oHJJ8ZT4qE3GTSPKyNRVdybeeCQHTeyoDiLg5CvC5JM&e=
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
More information about the gpfsug-discuss
mailing list