moving *finally*

March 10, 2008 by
Filed under: Work 

Migrating to a new host has been in the works for a long time. It’s been stalled for a couple months while we worked with the developers of the unionfs filesystem (mostly by providing them with crash reports) to stabilize it to the point where we felt comfortable putting it to use on a production service. That time has finally come. The alternative to unionfs (keeping 9 terrabytes of disk space online at once, completely dedicated to the FTP staging process) was really not cost effective, and it was really worth waiting for.

This Thursday, March 13th, we’ll be switching the DNS for the domain name to point at the new box. The SSH host keys have been copied over, so most people will never notice. The existing box, effective immediately, is accessible using the name If for any reason you’re nervous about switching your upload process to the new box this coming Thursday, you can use the stage-old name to keep using the old one, for now. Stage-old will go away on Tuesday, March 25th, so you’ll have until then to resolve any issues you have (or get IT to resolve them if you think it’s our problem). Feel free to file a bug against Server Ops if you need help.

Be sure to read my previous post for details of how the new system will work (it is a little different, though most well-behaved upload scripts shouldn’t be affected). The most noticeable change will be that there will now be a slight delay between when you upload something and when it shows up on the ftp server, since we’re now virus-scanning what you upload before it gets made available on the ftp server.  On the current system that we’re moving away from, the virus scan runs after the files are placed on the ftp server, and then the files are yanked afterwards if they get flagged. This of course, could let something get out there briefly, which we obviously don’t want.


4 Responses to “ moving *finally*”

  1. Gen Kanai on March 11th, 2008 12:13 am

    On behalf of those of us who use stage.mozilla we thank you :)

    Also, can I suggest you file this in “Work” and not “Personal”?

  2. justdave on March 11th, 2008 12:24 am

    Thanks for the heads-up. I could swear I checked “Work” when I started working on it, but I did have a crash while writing it, and I must have gotten bitten by the session restore or something.

  3. Grey Hodge on March 12th, 2008 5:16 pm

    My god, 9TB? What all is that comprised of?

  4. justdave on March 12th, 2008 6:01 pm

    The FTP server has about 2.6 TB of stuff on it last time I looked. The 9 TB reference is because without the fancy unionfs scheme, we’d need 3 copies of it online at the same time to be able to virus scan things without pushing them to the visible ftp server.

    It’s mostly a selection of nightly builds since the begining of time, and so forth. :)

Feel free to leave a comment...
and oh, if you want a pic to show with your comment, go get a gravatar!