Tag: firefox

  • Writing and Running Automated Firefox Browser Chrome Tests

    As a followup to making my first patch for Firefox, it came time to write some automated tests for it. Turns out Mozilla has an entire suite of automated testing tools for their platforms. All thoroughly documented, they range from the ultra low-level compiled code tests to “record and play” macro testing for human UI interactions.

    Selecting the Test Suite

    The hardest part was figuring out what test suite would best compliment my patch, but given that my patch affected the Browser Chrome I went with that section of the Mochitest suite. Browser Chrome tests are really just javascript files running at elevated chrome-level privileges. This means that in them you get to access the tab browser API, more window objects, and other goodies. They’re really simple to write, just follow one convention: put everything in function test(). This is the main line of your test mini-application. You can add more functions, but they’ll be ignored by the tester.

    At first I’d put my js file in the source tree directory only ($srcRoot/browser/base/content/test) but turns out that builds compiled with testing enabled pull from a different directory ($objDir/_tests/testing/mochitest/browser/browser/base/content/test). Oh yeah that’s right: Firefox has to be built with the –enable-tests option. This pushes the test code from the source into the appropriate sub-directory of your output object directory (notice how both end in “/browser/base/content/test”?). Since I had written my test after compiling, I had to copy it over myself!

    With the right location, it was time to write the test. Simple enough, I just looked at existing tests. The browser_allTabsPanel.js one looked very similar to what I was trying to do, so I used it to get a feel for how to write a test.

    Writing the Test

    When writing a test you must work by assertions, specifying a message to output when the expected value matches the actual. A few assertion functions are:

    ok(val, "val exists!");                        // Tests existence
    is(val1, val2, "val1 equals val2");            // Tests equality
    isnot(val1, val2, "val1 does not equal val2"); // Tests if not equal

    For a successful test run, you want all of the above comparisons (if used) to be true. As for writing the code itself: I wanted to test the addTab function I’d modified in tabbrowser.xml. To do this, I used the gBrowser object, which seems to be the global object holding the current browser (and giving access to functions in tabbrowser.xml). Here was my test function:

    function test() {
     var stubTab = gBrowser.addTab();
    
     var tabs = gBrowser.tabs;
     var owner;
    
     is(tabs.length, 2, "2 tabs are open");
     is(gBrowser.selectedTab._tPos, 0, "First tab is selected");
    
     var newTab = gBrowser.addTab();
    
     is(gBrowser.selectedTab, tabs[0], "First tab is still selected");
     is(gBrowser.selectedTab._tPos+1, newTab._tPos, "Was inserted at #2");
     is(newTab._tPos+1, stubTab._tPos, "Old #2 shifted to #3");
    
     gBrowser.moveTabTo(newTab,2);
     is(newTab._tPos-1,stubTab._tPos, "Successfully moved new tab, older one shifted down");
    
     gBrowser.removeTab(stubTab);
     is(newTab, tabs[1],"Successfully deleted stub tab, new tab moved down");
    
     var newTab2 = gBrowser.addTab();
     is(newTab, tabs[2],"Successfully moved newTab down on newTab2 creation");
     is(newTab2, tabs[1],"Successfully added newTab2 adjacent to selected tab");
    
     gBrowser.selectTabAtIndex(1);
     var newTab3 = gBrowser.addTab();
     is(newTab3, tabs[2],"Successfully added newTab3 in middle of list");
    
     while (tabs.length > 1)
     gBrowser.removeCurrentTab();
    }

    Running the Test

    With the test written, it was time to run it. For that, I had to go back to command line and from the source root run

    make -C $(OBJDIR) mochitest-browser-chrome

    This ran the entire suite! I didn’t want to wait around for my one test to run, so instead I specified the TEST_PATH variable before running it:

    TEST_PATH=browser/base/content/test/browser_tab_addBeside.js make -C $(OBJDIR) mochitest-browser-chrome

    Much better, it only runs my test. And 11/11 passed! Perfect.

  • Modding Firefox

    As the semester is drawing to a close and we the open source crew are feeling comfortable with the JavaScript language, Dave took us to the next level. From the content scripts we had been working on, we got our first exposure to chrome scripts: JavaScript that doesn’t just run in the browser: it runs the browser. With elevated privileges (like hard drive access) and a larger object model (like controlling the tabs) things were getting interesting. And how else would we work with this than by rewriting a very small part of Firefox to give it some added spark?

    So that’s exactly what we had to do. Something simple, just changing the behaviour of new tabs from by default being appending to the end of the tab collection to appearing just to the right of the currently active tab. It could be done in just a few lines of code, he said. He was right but it took sifting through millions.

    Enter MXR: Mozilla Cross-Reference. It’s a powerful web tool that allows you to search through and jump all over the source code through cross-referenced links. Every function call and definition, regular expression and plain text searching, everything was made easier with this tool. The hard part was finding a good starting ground. Kenneth in class suggested a search for “New Tab”. This didn’t get us exactly what we wanted, but it was the best we could hope for considering our ignorance and we ended up following a quick chain.

    All of this was done by the class, but conveniently we ran out of time before we could get right to the bottom of how to do this mini-assignment. Though ending on the cusp of discovery, some self research into the rest of tabbrowser.xml quickly found me a solution. 2 lines of code later, I had something that seemed to be working. After generating a diff file using Mercurial…

    hg diff -p -U 8 > changes.diff

    I had a portable way to model and ship my changes. I had been working off an older code base (current from when I had last built Firefox) so line numbers are slightly skewed with respect to the current code. It seemed to work on my computer, but I had to test it on other’s. So I got experimenting some more and found out I’d stopped just shy when building Firefox last time: I had never packaged it!

    make -C objdir/browser/installer                // ZIPs/Tarballs to objdir/dist
    make -C objdir/browser/installer installer      // Builds MSI to objdir/dist/install/sea

    After a few quick commands from the source root (and replacing “objdir” with my object dir), I was ready. Gave the ZIP file to a friend on USB and she tried running it. Error. Hmm. Turns out there were a few missing DLLs. My compiler (Visual Studio 10) had a few dependent dlls in C:\Windows\system32 that hadn’t been archived. Remedying that though, and everything was a-ok.

    What did I take away from this? I felt this was a great way to not only learn about chrome scripts, but drilling down from the UI “New Tab” label to the code that made it happen gave me an appreciation not just for the size of the Mozilla code base, but the architecture as well. I learned more about various tools at public disposal to make the 500 foot dive into the Mozilla project seem more like 5. And most of all, I learned the excitement of shaping Firefox.

  • Building Mozilla Firefox on Windows 7

    I’d always thought it would be scary building a large application like Mozilla Firefox completely from source, but a few weeks ago I did some initial looking into it. With a recent nudge from a professor, I went ahead and successfully retrieved the source and compiled Firefox. Here’s how I did it.

    Step 0: I read the documentation.

    Not just a little either, but everything you can to build Firefox on Windows. Mozilla Developer Central has an extensive series of wiki entries dedicated solely to multi-platform building. The build system is complex and issues may come up. If you’re like me you want to get your hands dirty and you’ll ignore the “Impatient” route they suggest.

    Step 1: Review Build Preqrequisites.

    An extensive system has extensive prerequisites for various versions. Feeling ambitious, I decided to try building the most-recent code for the upcoming Firefox 4. Going off a checklist, I worked my way down:

    Hardware Requirements

    Mozilla requires a “development-class system” to compile. With the following hardware, I was above their recommended specs:

    • AMD Phenom X3 8450
    • 3 GB RAM
    • 100+ GB HDD space
    • Windows 7 Professional 32-bit

    Software Requirements

    These are the tools required by the Mozilla build system. I already had a Visual C++ Compiler and Windows SDK, but still to be downloaded was the MozillaBuild System (found here). Given my target product of Firefox 4, my compiler version needed to supported. Firefox 4 is compatible with Visual Studio 2005, 2008 and 2010. My configuration for compiling is below:

    • Visual Studio Ultimate 2010
    • Windows SDK 6.0A, 7.0A, and 7.1
    • MozillaBuild 1.5.1

    MozillaBuild was installed by default to “C:\mozilla-build” (you can change this, just don’t include spaces in the path name!) and with it came a bevy of tools. Windows versions of VIM and python are there, along with the software source control program used by Mozilla (Mercurial, found in “C:\mozilla-build\hg”). With hardware and software dependencies taken care of, it was time to get the source code!

    Step 2: Grabbing the Source Code

    Mozilla uses Mercurial to manage source control and versioning. Mercurial is an application which, if you’re following along to the T should have been installed along with MozillaBuild in Step 1. If not, you can find installation instructions here. Once downloading it though, it came time to configure Mercurial. Mercurial uses an ini file to store settings, which must be created from scratch after installation. Here’s how to do it:

    1. Open Notepad (or another text editor)
    2. Save the empty file to Mercurial’s install directory (“C:\mozilla-build\hg”)

    This file will drive the source retrieval process. Mozilla has a few recommended settings listed on their wiki, copied into my own Mercurial.ini and shown below:

    [ui]
    username = Your Real Name <user@example.com>
    merge = your-merge-program (or internal:merge)


    git = 1
    showfunc = 1
    unified = 8

    [defaults]
    commit = -v

    Name is easy enough to fill in, but the merge tool entry may be one of many things. A few of the wiki editors on MDC prefer tools such as kdiff, but having already had TortoiseSVN and TortoiseMerge installed, I decided to use TortoiseMerge by setting the value to the absolute path of the program (“C:\Program Files\TortoiseSVN\bin\TortoiseMerge.exe”). Merge tools come in many flavours though, with a comprehensive listing here.

    Actually Grabbing the Code

    With the configuration all set, I just had to choose a location to store the source code and then to retrieve it. The location should have no spaces (I chose D:\FFsrc\src20). Then simply call “hg clone http://hg.mozilla.org/mozilla-central/ D:\FFsrc\src20″ from the command line with the parameter after clone being the location to clone from and the last argument being the location to close to. Mozilla Central is the trunk, which at this point will become Firefox 4.

    Mercurial at Work
    Mercurial at Work

    By default, the MozillaBuild installation does not modify the PATH variable on your computer and Windows won’t know what program hg is unless you are in the Mercurial root directory (C:\mozilla-build\hg). You can fix this like so:

    1. Right click on My Computer
    2. Choose Properties
    3. Choose “Advanced” (may show as “Advanced Settings”)
    4. Click “Environment Variables”
    5. Scroll down the list of System variables until you find PATH
    6. Double-click the entry
    7. Move to the end of the string and enter “;C:\mozilla-build\hg” without quotes
    8. Click OK
    9. Click OK
    10. Click OK

    At this point the command to pull code from Mozilla’s source repository should work when run from anywhere. Restarting the system may be required for any changes made to the PATH variable to take effect. Pulling the files will take a while: the complete source as of today’s date weighs in at a hefty 670 MB.

    Step 3: Configuring the Build

    Firefox uses a file (.mozconfig) to specify build options, which are then passed to the make file. This is an ordinary text file with options specified one-per-line, located at the root of the source tree (in my case, D:\FFsrc\src20). There are an extensive number of options which may enable the creation of one of any of the Mozilla products all outlined in their documentation. At the bottom of that page is a listing of several recommended configurations for builds such as Release Firefox, Firefox Debug, Thunderbird Debug and SeaMonkey Optimized (not statically linked). I went with the Firefox release configuration, shown below:

    . $topsrcdir/browser/config/mozconfig
    mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/ff-opt
    ac_add_options --disable-tests

    After creating a .mozconfig file (much the same way you create the Mercurial.ini file) enter the above in your .mozconfig and save it to the root for your source (D:\FFsrc\src20). While this doesn’t seem like many commands, a great many which are not specified have default values, and often defaults result in the best overall performance. A dissection of the options shows a variable, topsrcdir,with relative paths specified for object generation (MOZ_OBJDIR). This will be the location all compilation output (including our final Firefox executables) will reside within.

    With everything in place, it’s time to cross our fingers and build Firefox.

    Step 4: Building Firefox

    Here is where all the diligent work above pays off. The build takes a long time (50 minutes using the hardware, software and build configurations listed above) and may easily take more depending on your build options.

    Compilation is performed through a unix-style shell included in the MozillaBuild install from Step 1. This is run in the form of one of many batch scripts (located at the root install directory for MozillaBuild), each with customizations for whether the compiler used will be Visual Studio 2003, 2005, 2008 or 2010. Using Visual Studio 2010, the appropriate file for me is start-msvc10.bat. While I am on a 32-bit OS and their are 64-bit versions of the scripts, developers on the wiki caution against using the experimental 64-bit versions.

    Whichever shell is right for you, the final steps to building are:

    1. Run the appropriate build file
    2. Make the source root your current directory (cd “D:\FFSrc\src20”)
    3. Type “make -f client.mk build” without quotes and hit enter

    Building Firefox
    Building Firefox

    Every stage of the build is written to standard output (the console) by default, so you can watch the build progress every step of the way. By the end of the build process you should have your very own build of Firefox. You can find it in the default Windows location of MOZ_OBJDIR\dist\bin\firefox.exe, as specified in your .mozconfig file in Step 3.

    Enjoy your custom-built Firefox!

  • My First Bug

    After identifying and analyzing various performance issues I noted related to hardware acceleration in Firefox 4 nightlies, it was now time to make my findings public. Bugzilla would be the end result, but there were quite a few steps in between. First, there was consultation. I’d never filed a bug before and as punctual as I try to be, I had the great luck of a one-time schedule conflict occurring when we were covering bug filing in class. I already knew a bit from earlier classes and my own playing around with Bugzilla, but like most of my colleagues I consulted with Dave on IRC before submitting. That’s when things got interesting.

    IRC is yet another new thing to me in DPS909. I’ve used chat clients like MSN, but within 5 minutes of conversing with Dave he had pulled in a Senior Software Developer from Mozilla to #seneca where we were talking. After a short bit, when we figured that the performance issue was related to Direct2D, he pulled in a “Platform Graphics Guy” at Mozilla. Bam, bam bam. Within 10 minutes of consulting with Dave on IRC I’d spoken with and been helped by two extremely knowledgeable guys located in various places around North America. The connectivity enabled by IRC still boggles me.

    With all of their help, I now had some guidance as to how to go about filing this performance bug. I knew it needed to be filed in the first place. I knew it was in some way related to Direct2D. And I knew to CC Dave on it and add the “[chromeexperiments]” tag to it. With all this new information to process, I consider myself lucky to have remembered 75% of it (thanks for adding the [chromeexperiments] tag, Dave!). After filing the bug, the Open Source wheels churned to life.

    Just as Linus’ Law applies to bug identification and resolution, it seems to also apply to bug spotting in a different light. Between the 4 viewers on Bugzilla and another two on IRC, those six sets of eyes were able to spot a duplicate bug, this one from four months prior. A dialog is now going and some of the best programmers in the world are now looking at resolving this with collated effort. Release early, release often indeed.

  • Getting ready to file my first bug using xperf

    If you’ll remember from my last blog entry, I’ve been stress-testing Minefield using Chromeexperiments in an attempt to identify bugs and bottlenecks. Without being able to identify any crashes or unusual non-performance related behaviour, I then had the task of investigating profiling tools to find the issue. And then things got messy.

    I thought things would be as simple the tool, running the test and the tool would tell you exactly what was wrong. That assumed two things: 1) That I knew how to use the tool and 2) That the tool knew how to identify problems. Turns out I started with incorrect assumptions on both. I had never used a profiler before, and the Windows SDK-supplied xperf had a lot of options. Mozilla was great at giving me the setup, but there was more to learn. Firstly, that profilers are not sentient and that their results still require analysis.

    The idea behind a profiler is that it generally runs in the background, taking samples of CPU and memory states (or other conditions still, like registry accesses or file io) periodically while all processes are running. Good ones like xperf can be configured to hook into kernel events to create a stack trace to show state at every function call.

    This tool was great for discounting some of my earlier theories by using the Windows-default perfmon tool. The  ability of xperf to monitor both CPU and memory during stack-walking disproved some of my earlier causal theories and showed me that in most cases, CPU and memory usage was eaten up within DirectDraw drivers.

    To figure out where the information I wanted was required a fair bit of googling. Even something so simple as viewing a stack walk took a bit of detective work though for those looking for the Coles notes, here’s how to capture both Heap and CPU state for each stack frame (courtesy of Mozilla and MSDN links above):

    xperf -on latency -stackwalk profile
    call xperf -start heapsession -heap -PidNewProcess “%platform% %siteToGo% %args%” -stackwalk HeapAlloc+HeapRealloc -BufferSize 512 -MinBuffers 128 -MaxBuffers 512

    %platform% would be the absolute path for the executable to run (in this case, Minefield)

    %siteToGo% is the web site you wish to test (saves on collecting unneccessary data from loading home page or google)

    %args% would be any other arguments you wish to pass to the application. In this case, I opened Minefield with “-P Testing -no-remote” (no quotes) to use my test profile. The test profile was a very stripped-down profile to eliminate any background work in Firefox which may have thrown in extra data. The complete script can be found here.

    Various options can be specified in the “On” parameter when calling xperf as Richard Russell shows, but I went with the latency option suggested above by Mozilla. The result were large (1.7 GB for 5 minutes) files, so perhaps for CPU monitoring I would have done well to stick with the “PROC_THREAD” option. Most of the size was heap data, the result of the second call to xperf but every bit helps. Once finished, I  stopped the profiler instances and merged the results:

    xperf -stop heapsession -d heap.etl
    xperf -d main.etl
    xperf -merge main.etl heap.etl result.etl

    Once the report was compiled, I loaded it and saw some complex graphs. It took some getting used to where everything was, but after a long while I learned the following best practices:

    To load symbols (so as to view symbolic function names), enter them under “Trace -> Configure Symbol Paths” and then select “Trace -> Load Symbols” to actually load the symbols. From here, the app has probably become non-responsive for a short time while it associates everything and you’re left still looking at the same graphs. Here’s where the magic happens:

    Right click on a graph (“CPU Sampling by Thread” and “Heap Total Allocation Size” work well) and select “Summary Table” or “Simple Summary Table”. From here there was a tree-view allowing a drill down from process to thread to dll to individual function calls. Richard Russell’s blog  entry was so helpful, I’ll link it here as well, when I say how much it helped describe using the GUI.

    In the end, xperf helped identify some choke-points on some of the experiments I was testing, specifically isolating the need to test both with and without hardware acceleration. Speaking of which, hardware acceleration in Minefield and the upcoming Firefox 4 can be adjusted by use of the following options (enter about:config into your Firefox address bar to adjust):

    gfx.direct2d.disabled
    gfx.direct2d.force-enabled
    gfx.font_rendering.directwrite.enabled
    layers.accelerate-all
    layers.accelerate-none

    The first 3 are for DirectDraw, which handles 2D graphics and text, while the last 2 are for Direct3D, which handles 3D drawing. This is a lot to work with to test hardware acceleration, so Joe the graphics guru from Mozilla has an easier way to test hardware acceleration (Windows only at present).

    At around  848 words this seems to be by far my largest blog post, which given that I’m blogging about what I’ve learned through bug filing must be a good thing. Ah yes, bug filing: the ends to these means. That’s the next entry in this blog post queue.

  • Performance testing Minefield and Chrome using Chrome Experiments

    Now that I have my Firefox testing environment, it’s time for my first task: Performance Testing! For this, I needed to know what to test (in this case, nightly builds of Minefield and Chromium), how to test (the Chrome Experiments) and what to determine (speed, smoothness, and responsiveness). There are quite a few experiments, but as a class of 12 we split the 120 up fairly easily. I was tasked with #71-80.

    To begin testing, I had to try and think about what variables could be eliminated. I decided to launch all experiments directly from the command line in an attempt to standardize application startup. For this I wrote a batch file script with a few parameters to help with this. I also used Windows’ built in PerfMon tool to assist with performance measurement so that I could be free to stress-test the application. With this combination, I felt I could gather information quickly and efficiently, and Perfmon also provided lots of great graphs for later analysis. It almost seemed like overkill at the time, but it helped give extremely measurable results.

    Given the target implementation platform (Chrome) for these experiments, I wasn’t surprised to find that a few of them weren’t performing quite as well on Minefield as on Google’s browser… yet. What did surprise me was the creativity and range of the experiments.

    Now that I had my data, it was time to analyze further and see if I could spot any large discrepancies, and to try and figure out why they occurred. My testing measures focused on the Working Set (Memory) and CPU utilization for the browser process. The jsCanvasBike, Animated Harmonograph and Liquid Particles seemed to require the most tuning, so my attention turned now to profiling rather than monitoring. As perfmon shows, not all of these problems were visible simply by monitoring the CPU.

    Luckily, the Windows SDK comes with their own profiling tool (xperf) which features extremely detailed kernel debugging on stack traces. Mozilla even has its own help page with instructions tailored specifically for use with Firefox. The results can look a little daunting, but it appears to be a very powerful tool and I was glad to have so many resources available. I feel as if I’ve entered a whole new realm of testing, but there is still a great deal more to experiment with.

  • Setting up a Firefox testing environment

    In light of jumping into open source development, I first decided to explore Firefox and to try and learn a bit about it, its architecture and how to accommodate a testing environment for it.

    First thing was first: downloading a prerelease version of Firefox. I had two choices: download the current (but less stable) nightly build or the more stable but less current beta. I chose to test a nightly build, but could have chosen both and simply installed to different directories. Installation was simple and straight-forward and just like release versions of Firefox. One thing of note was that pre-release versions were installed in a separate location from release versions, under the name Minefield. Genius, this way I could have both installed concurrently!

    From here, I came into an issue:my release version of Firefox had been heavily customized by me, but what if I wanted to test the default configuration? After some research I changed the properties of the shortcut for Minefield to add the command-line argument -P (apparently identical to the -ProfileManager argument), which opened up a profile manager for me to create a secondary profile. Firefox allows for different profiles (which group configuration, extensions and more) to be used to run Firefox, but there is always a default. Upon making my new profile (named Testing), Firefox opens normally. I then closed Firefox and re-modified the command line in the shortcut to include the new profile name in quotes after the -P like so:

    This was all fine and well, except that Firefox is designed so that by default only one instance can be run at once. By that I mean if run my release version, minimize and attempt to open Minefield, another instance of the already-running release version pops up. Through lecture with David Humphrey, I learned that the way to resolve this was simply to add the -no-remote command line option with the -P option (after the profile name) to allow multiple instances to be run under different profiles. With this little tweak, I now have completely independent testing environments setup for concurrently testing various Firefox builds.