<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto">Thank you! Would you mind letting me know in what era you made your evaluation?<div><br></div><div>I’m not suggesting you should change anything at all, but when I make recommendations for my own customers I like to be able to associate the level of GPFS with the anecdotes. I view the software as more of a stream of features and capabilities than as a set product.</div><div><br></div><div>Different clients have different requirements, so every implementation could be different. When I add someone else’s judgement to my own, I just like getting as close to their actual evaluation scenario as possible.</div><div><br></div><div>Your original post was very thoughtful, and I appreciate your time.</div><div><br></div><div><div dir="ltr"> <span style="text-align: -webkit-auto; orphans: 2; widows: 2; background-color: rgba(255, 255, 255, 0);">-- </span><div class="" style="orphans: 2; widows: 2; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><div style="orphans: auto; widows: auto;"><span style="background-color: rgba(255, 255, 255, 0);">Stephen</span></div><div style="orphans: auto; widows: auto;"><span style="background-color: rgba(255, 255, 255, 0);"><br></span></div></div></div><div dir="ltr"><br><blockquote type="cite">On Mar 11, 2021, at 7:58 AM, Tagliavini, Enrico <enrico.tagliavini@fmi.ch> wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div>Hello Stephen,</div>
<div><br>
</div>
<div>actually not a dumb question at all. We evaluated AFM quite a bit before turning it down.</div>
<div><br>
</div>
<div>The horror stories about it and massive data loss are too scary. Plus we had actual reports of very bad performance. Personally I think AFM is very complicated, overcomplicated for what we need. We need the data safe, we don't need active / active DR or
anything like that. While AFM can technically do what we need the complexity of its design makes it too easy to make a mistake and cause a service disruption or, even worst, data loss. We are a very small institute with a small IT team, so investing time in
making it right was also not really worth it due to the high TCO.</div>
<div><br>
</div>
<div>Kind regards.</div>
<div><br>
</div>
<div><span>
<pre>-- <br></pre>
<pre>Enrico Tagliavini
Systems / Software Engineer
enrico.tagliavini@fmi.ch
Friedrich Miescher Institute for Biomedical Research
Infomatics
Maulbeerstrasse 66
4058 Basel
Switzerland
</pre>
</span></div>
<div><br>
</div>
<div>On Thu, 2021-03-11 at 08:17 -0500, Stephen Ulmer wrote:</div>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div>I’m going to ask what may be a dumb question:</div>
<div><br>
Given that you have GPFS on both ends, what made you decide to NOT use AFM?</div>
<div><br>
<div dir="ltr"> <span style="text-align: -webkit-auto; orphans: 2; widows: 2; background-color: rgba(255, 255, 255, 0);">-- </span>
<div class="" style="orphans: 2; widows: 2; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div style="orphans: auto; widows: auto;"><span style="background-color: rgba(255, 255, 255, 0);">Stephen</span></div>
<div style="orphans: auto; widows: auto;"><span style="background-color: rgba(255, 255, 255, 0);"><br>
</span></div>
</div>
</div>
<div dir="ltr"><br>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div>On Mar 11, 2021, at 3:56 AM, Tagliavini, Enrico <enrico.tagliavini@fmi.ch> wrote:<br>
<br>
</div>
</blockquote>
</div>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div dir="ltr"><span>Hello William,</span><br>
<span></span><br>
<span>I've got your email forwarded my another user and I decided to subscribe to give you my two cents.</span><br>
<span></span><br>
<span>I would like to warn you about the risk of dong what you have in mind. Using the GPFS policy engine to get a list of file to rsync is</span><br>
<span>easily going to get you with missing data in the backup. The problem is that there are cases that are not covered by it. For example</span><br>
<span>if you mv a folder with a lot of nested subfolders and files none of the subfolders would show up in your list of files to be updated.</span><br>
<span></span><br>
<span>DM API would be the way to go, as you could replicate the mv on the backup side, but you must not miss any event, which scares me</span><br>
<span>enough not to go that route.</span><br>
<span></span><br>
<span>What I ended up doing instead: we run GPFS on both side, main and backup storage. So I use the policy engine on both sides and just</span><br>
<span>build up the differences. We have about 250 million files and this is surprisingly fast. On top of that add all the files for which</span><br>
<span>the ctime changes in the last couple of days (to update metadata info).</span><br>
<span></span><br>
<span>Good luck.</span><br>
<span>Kind regards.</span><br>
<span></span><br>
<span>-- </span><br>
<span></span><br>
<span>Enrico Tagliavini</span><br>
<span>Systems / Software Engineer</span><br>
<span></span><br>
<span>enrico.tagliavini@fmi.ch</span><br>
<span></span><br>
<span>Friedrich Miescher Institute for Biomedical Research</span><br>
<span>Infomatics</span><br>
<span></span><br>
<span>Maulbeerstrasse 66</span><br>
<span>4058 Basel</span><br>
<span>Switzerland</span><br>
<span></span><br>
<span></span><br>
<span></span><br>
<span></span><br>
<span>-------- Forwarded Message --------</span><br>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span></span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>-----Original Message-----</span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>From: gpfsug-discuss-bounces@spectrumscale.org <gpfsug-discuss-bounces@spectrumscale.org> On Behalf Of Ryan Novosielski</span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>Sent: Wednesday, March 10, 2021 3:22 AM</span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org></span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>Subject: Re: [gpfsug-discuss] Backing up GPFS with Rsync</span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span></span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>Yup, you want to use the policy engine:</span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span></span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>https://www.ibm.com/support/knowledgecenter/STXKQY_4.2.0/com.ibm.spectrum.scale.v4r2.adv.doc/bl1adv_policyrules.htm</span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span></span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>Something in here ought to help. We do something like this (but I’m reluctant to provide examples as I’m actually suspicious that we</span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>don’t have it quite right and are passing far too much stuff to rsync).</span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span></span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>--</span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>#BlackLivesMatter</span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>____</span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>\\UTGERS, |---------------------------*O*---------------------------</span><br>
</div>
</blockquote>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>_// the State | Ryan Novosielski - novosirj@rutgers.edu</span><br>
</div>
</blockquote>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>\\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus</span><br>
</div>
</blockquote>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span> \\ of NJ | Office of Advanced Research Computing - MSB C630, Newark</span><br>
</div>
</blockquote>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span> `'</span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span></span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>On Mar 9, 2021, at 9:19 PM, William Burke <bill.burke.860@gmail.com> wrote:</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span></span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span> I would like to know what files were modified/created/deleted (only for the current day) on the GPFS's file system so that I</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>could rsync ONLY those files to a predetermined external location. I am running GPFS 4.2.3.9</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span></span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>Is there a way to access the GPFS's metadata directly so that I do not have to traverse the filesystem looking for these files? If</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>i use the rsync tool it will scan the file system which is 400+ million files. Obviously this will be problematic to complete a</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>scan in a day, if it would ever complete single-threaded. There are tools or scripts that run multithreaded rsync but it's still a</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>brute force attempt. and it would be nice to know where the delta of files that have changed.</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span></span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>I began looking at Spectrum Scale Data Management (DM) API but I am not sure if this is the best approach to looking at the GPFS</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>metadata - inodes, modify times, creation times, etc.</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span></span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span></span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span></span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>--</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span></span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>Best Regards,</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span></span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>William Burke (he/him)</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>Lead HPC Engineer</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>Advance Research Computing</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>860.255.8832 m | LinkedIn</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>_______________________________________________</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>gpfsug-discuss mailing list</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>gpfsug-discuss at spectrumscale.org</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</span><br>
</div>
</blockquote>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span></span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>_______________________________________________</span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>gpfsug-discuss mailing list</span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>gpfsug-discuss at spectrumscale.org</span><br>
</div>
</blockquote>
<blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex">
<div><span>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</span><br>
</div>
</blockquote>
<span>_______________________________________________</span><br>
<span>gpfsug-discuss mailing list</span><br>
<span>gpfsug-discuss at spectrumscale.org</span><br>
<span>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</span><br>
</div>
</blockquote>
</div>
<div>_______________________________________________<br>
</div>
<div>gpfsug-discuss mailing list<br>
</div>
<div>gpfsug-discuss at spectrumscale.org<br>
</div>
<div><a href="http://gpfsug.org/mailman/listinfo/gpfsug-discuss">http://gpfsug.org/mailman/listinfo/gpfsug-discuss</a><br>
</div>
</blockquote>
<span>_______________________________________________</span><br><span>gpfsug-discuss mailing list</span><br><span>gpfsug-discuss at spectrumscale.org</span><br><span>http://gpfsug.org/mailman/listinfo/gpfsug-discuss</span><br></div></blockquote></div></body></html>