[gpfsug-discuss] SC20 Sessions - Dates and times are settled, please join us!

[Christopher Black]

Could you talk about upcoming work to address excessive prefetch when reading small fractions of many large files?
Some bioinformatics workloads have a client node reading relatively small regions of multiple 50GB+ files. We've seen this trigger excessive prefetch bandwidth (especially on 16MB block filesystem). Investigation shows that much of the prefetched data is never read, but cache gets full, evicts blocks, then more prefetch happens. We can avoid this by turning prefetch off, but that reduces speed of other workloads that read full files sequentially.  Turning prefetch on and off based on job won't work well for our users.

We've heard this would be addressed in gpfs 5.1 at the earliest and have provided an example workload to devs. They've done some great analysis and determined the problem is worse on large (16M) block filesystems (which are now the recommended and default on new ess filesystems with sub-block allocation enabled).

Best,
Chris

On 10/29/20, 5:49 PM, "gpfsug-discuss-bounces at spectrumscale.org on behalf of Kristy Kallback-Rose" <gpfsug-discuss-bounces at spectrumscale.org on behalf of kkr at lbl.gov> wrote:

    Hi all,

    The Spectrum Scale User Group will be hosting two 90 minute sessions at SC20 this year and we hope you can join us. The first one is:

     "Storage for AI" and will be held Monday, Nov. 16th, from 11:00-12:30 EST

    and the second one is

    "What's new in Spectrum Scale 5.1?" and will be held Wednesday, Nov. 18th from 11:00-12:30 EST.

    Please see the calendar at https://urldefense.com/v3/__https://www.spectrumscaleug.org/eventslist/2020-11/__;!!C6sPl7C9qQ!G0wT65UH3HoMnjBM6_ZAVfZwWwJz5SoLE5gpB_LM0N8SNSU3TXItF31dfxG_8Pow$  and register by clicking on a session on the calendar and then the "Please register here to join the session" link.

    Best,
    Kristy

    Kristy Kallback-Rose
    Senior HPC Storage Systems Analyst
    National Energy Research Scientific Computing Center
    Lawrence Berkeley National Laboratory

    _______________________________________________
    gpfsug-discuss mailing list
    gpfsug-discuss at spectrumscale.org
    https://urldefense.com/v3/__http://gpfsug.org/mailman/listinfo/gpfsug-discuss__;!!C6sPl7C9qQ!G0wT65UH3HoMnjBM6_ZAVfZwWwJz5SoLE5gpB_LM0N8SNSU3TXItF31df0lybvoA$

________________________________

This message is for the recipient’s use only, and may contain confidential, privileged or protected information. Any unauthorized use or dissemination of this communication is prohibited. If you received this message in error, please immediately notify the sender and destroy all copies of this message. The recipient should check this email and any attachments for the presence of viruses, as we accept no liability for any damage caused by any virus transmitted by this email.