<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang=en-CH link=blue vlink=purple style='word-wrap:break-word'><div class=WordSection1><p class=MsoNormal><span lang=EN-US>Hello,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>I need to update/replace our metadata disks but I want to keep old and new in parallel for a while before I remove the old storage: as active/active pair with double copies. This will allow an immediate fall-back if we ever need.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Maybe you want to comment on this procedure – I probably found it some time ago on this mailing list, sorry if I don’t remember the author.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Some concerns I have<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>I’m a bit worried what happens when we remove the vdisks with the second copy in the last step – will be get millions of error messages? Is the sequence of commands right? <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>I need to change the failure groups of some vdisks while in use – I wonder if this poses some risk? As I understand this will change the order of block allocation among the nsds (not the allocation map I guess)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>This will double metadata write-io, the systems should be able to handle it. <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>We will get better metadata read-io during the transition than what we’ll finally get.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>== Start – ‘old’ ESS/GNR only<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US> -m 1 (single metadata copy), <o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US> -M 2<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US> -K whenpossible<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US> Metadata vdisks in failure groups 1 and 2<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>== Preparation<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Use’ mmvdisk filesystems’ and move all metadata vdisk-sets to a single failuregroup (this has to be done in operation, under load)<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Set -m 2, while we still have one failure group only. -K whenpossible will keep us running<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Add the new vdisk-set with a second failure group on ‘new’ ESS/GNR systems<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Now all new inodes have two copies, one on old and one on new<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>== Create copies on ‘old’ and ‘new’<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US> mmrestripefs -R with –qos maintenance and -N helper-nodes to minimize the load. This may create some locks on the filesystem/cluster and interfere with backup and snapshots??<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Maybe better: use a policy ‘replicate’ rule to replicate all files, I can run this in small chunks and run mmrestripefs just once to crosscheck.<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>== Observe for some days, handle remaining filesystems in the same way<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>== Finally<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Suspend ‘old’ disks<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Run mmrestripefs -m<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Remove ‘old’ vdisk sets with mmvidisk – this will run another mmdeldisk<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Change to -m 1<o:p></o:p></span></p><p class=MsoNormal style='margin-left:36.0pt'><span lang=EN-US>Run fix replication setting mmrestripefs -R (if needed?)<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Thank you for reading and for any comments or suggestions.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Heiner<o:p></o:p></span></p></div></body></html>