scheduled maintenance November 25 at 5pm PST

November 23, 2005 by · Comment
Filed under: Work 

On Friday, November 25th, we will be cutting over to new faster Internet uplinks and new firewalls in the Meer collocation facility. This is to increase security and capacity to support our core infrastructure.

The upgrade affects many services, including Bugzilla, Bonsai, Tinderbox, LXR, CVS, mail, mail, Mozilla Developer Center, the wikis, IRC, the Report a Broken Website tool, and the mailing lists. We will do everything possible to minimize downtime and impact to users, but we anticipate some effect when we do the final cutover and failover testing. During the outage window, we will also be rebooting several of the servers in order to install some pending upgrades.

We will start this work at 5pm PST (1am UTC Saturday) and will be done by 7pm PST (3am UTC). Expect sporadic access to any Internet facing infrastructure during this time.

Once complete we will be performing a full suite of tests to ensure stability and functionality. We will send out an update when we have completed the work. If you see any issues relating to the upgrade after 7pm, please email or, or comment on the #bmo channel on

Bugzilla performance boost

November 10, 2005 by · Comment
Filed under: Work 

It was discovered after we upgraded to version 2.20 that some of the architectural changes in how Bugzilla talks to the database servers had quite by accident sideswiped the ability to run searches against a replicated database server, causing those searches to always run against the master. The net effect on Bugzilla is that on large installations (such as ours), Bugzilla would periodically lock up for a bit because of having to wait for database locks.

I applied the patch from bug 313695 to correct this problem to about 3 hours ago, and so far things are looking pretty good. I haven’t seen any freezeups yet since then, and things are staying pretty snappy. On the other hand, we’re also right at the bottom end of our off-peak time of day for traffic levels.

I also bumped up the send timeout in apache to match what we had before we moved to the new server a few months back (bug 299577). This allows some longer-running searches (and sending of bugmail on bugs with lots of CCs) to actually complete.

If you notice any new problems with Bugzilla as of this evening, come find me on #bmo on

For any non-Mozilla Bugzilla folks reading this, yes, this fix will be included in 2.20.1. :)

Happy Birthday…

November 9, 2005 by · Comment
Filed under: Work 

…to both me and to Firefox. :)

One year ago yesterday, I joined the Mozilla Foundation as a paid sysadmin, just in time to wrestle with the server overload as we released Firefox 1.0 the following day. It was pretty cool to see Firefox 1.0 get released on my birthday last year. Now both Firefox and I are another year older.

Here’s to another great year!

  • Find me on Twitter

    Me: @justdavemiller
    The Bugzilla Project: @bugzilla
  • Recent Tweets by @justdavemiller

    justdavemiller @EnglishMossop @nsIAnswers Receiptionist would never get anything done if she didn't let it go to voicemail half the time.
    justdavemiller @EnglishMossop @nsIAnswers makes sense. Main number rings off the hook, it gets answered ocassionally.
    justdavemiller @Support Can we do 2FA with multiple accounts yet without needing a separate phone number for each one?
    justdavemiller @PFCBSA your Twitter thing is short-coding your links - it'd be nice if you told me something with the link to make me want to click it.
    justdavemiller @EnglishMossop @nsIAnswers HR should have a direct number they can give you that bypasses the phone tree.
  • Archives