[gpfsug-discuss] Intro, and Spectrum Archive self-service recall interface question

[Todd Ruston]

Greetings all,

First post here, so by way of introduction we are a fairly new Spectrum Scale and Archive customer (installed last year and live in production Q1 this year). We have a four node (plus EMS) ESS system with ~520TB of mixed spinning disk and SSD. Client access to the system is via CES (NFS and SMB, running on two protocol nodes), integrated with Active Directory, for a mixed population of Windows, Mac, and Linux clients. A separate pair of nodes run Spectrum Archive, with a TS4500 LTO-8 library behind them. 

We use the system for general institute data, with the largest data types being HD video, multibeam sonar, and hydrophone data. Video is the currently active data type in production; we will be migrating the rest over time. So far things are running pretty well.

Our archive approach is to premigrate data, particularly the large, unchanging data like the above mentioned data types, almost immediately upon landing in the system. Then we migrate those that have not been accessed in a period of time (or manually if space demands require it). We do wish to allow users to recall archived data on demand as needed.

Because we have a large contingent of Mac clients (accessing the system via SMB), one issue we want to get ahead of is inadvertent recalls triggered by Mac preview generation, Quick Look, Cover Flow/Gallery view, and the like. Going in we knew this was going to be something we'd need to address, and we anticipated being able to configure Finder to disable preview generation and train users to avoid Quick Look unless they intended to trigger a recall. In our testing however, even with those features disabled/avoided, we have seen Mac clients trigger inadvertent recalls just from CLI 'ls -lshrt' interactions with the system.

While brainstorming ways to prevent these inadvertent recalls while still allowing users to initiate recalls on their own when needed, one thought that came to us is we might be able to turn off recalls via SMB (setgpfs:recalls = no via mmsmb), and create a simple self-service web portal that would allow users to browse the Scale file system with a web browser, select files for recall, and initiate the recall from there. The web interface could run on one of the Archive nodes, and the back end of it would simply send a list of selected file paths to ltfsee recall.

Before possibly reinventing the wheel, I thought I'd check to see if something like this may already exist, either from IBM, the Scale user community, or a third-party/open source tool that could be leveraged for the purpose. I searched the list archive and didn't find anything, but please let me know if I missed something. And please let me know if you know of something that would fit this need, or other ideas as well.

Cheers,

-- 
Todd E. Ruston
Information Systems Manager
Monterey Bay Aquarium Research Institute (MBARI)
7700 Sandholdt Road, Moss Landing, CA, 95039
Phone 831-775-1997      Fax 831-775-1652      http://www.mbari.org