Snippets Service¶

The Snippets Service hosts small chunks of HTML, CSS, and JavaScript that are displayed on about:home. in Firefox, among other places.

Contents¶

Overview¶

This document describes the snippet service and how it relates to about:home, the home page for Firefox.

What is a snippet?¶

about:home is the default home page in Firefox. When you visit about:home, you normally see, among other things, a Firefox logo, a search bar, and a bit of text and an icon below the search bar. This text and icon are what is called a “snippet”.

Snippets can be more than just text and an icon, though. Sometimes a snippet can replace the Firefox logo with a larger image. A snippet could also pop up a video for you to watch. Snippets are made of HTML, CSS, and JavaScript, and can modify the homepage in interesting and fun ways.

A snippet that replaced the Firefox logo with a larger, more detailed image.

The Snippets Service is a web service that serves the code for snippets. Firefox downloads code from the Snippet Service and injects the code into about:home. This allows administrators using the service to control which snippets are being shown by Firefox without having to update Firefox itself.

How are snippets retrieved by Firefox?¶

$digraph snippet_download_flow { load_abouthome[label="User loads\nabout:home"]; check_cache_timeout[label="Has it been\n4 hours since\nsnippets were fetched?" shape=diamond]; load_cached_snippets[label="Retrieve snippets from\nIndexedDB" shape=rectangle]; fetch_snippets[label="Fetch snippets from\nsnippets.mozilla.com" shape=rectangle]; store_snippets[label="Store new snippets in\nIndexedDB" shape=rectangle]; insert_snippets[label="Insert snippet HTML\ninto about:home"]; load_abouthome -> check_cache_timeout; check_cache_timeout -> load_cached_snippets[label="No"]; check_cache_timeout -> fetch_snippets[label="Yes"]; fetch_snippets -> store_snippets; store_snippets -> load_cached_snippets; load_cached_snippets -> insert_snippets; }$

Firefox maintains a cache of snippet code downloaded from the Snippet Service for at least 4 hours since the last download. The cache (and a few other useful pieces of information) are stored in IndexedDB, and can be accessed by code on about:home using a global JavaScript object named gSnippetsMap.

When a user visits about:home, Firefox checks to see when it last downloaded new snippet code. If it has been at least 4 hours, Firefox requests new snippet code from the Snippet Service and stores it in the cache along with the current time. After this, or if it hasn’t been 4 hours, Firefox loads the snippet code from the cache and injects it directly into about:home.

Note

All Firefox does is download snippet code from the service and inject it into the page. The rest of the logic behind displaying snippets is determined by the snippet code itself, as explained in the next section.

Note

Firefox for Android caches snippets for 24 hours.

How are snippets displayed?¶

The snippet code downloaded from the Snippet Service consists of three parts:

A small block of CSS that is common to all snippets. Among other things, this hides all the snippets initially so that only the one chosen to be displayed is visible to the user.
The code for each individual snippet, surrounded by some minimal HTML.
A block of JavaScript that handles displaying the snippets and other tasks.

Note

It’s important to understand that all the code for every snippet a user might see is injected into the page. This means that any JavaScript or CSS that is included in a snippet might conflict with JavaScript or CSS from another snippet.

For this reason it is important to ensure that snippet code is well-isolated and avoids overly-broad CSS selectors or the global namespace in JavaScript.

Once the code is injected, the included JavaScript:

Identifies all elements in the snippet code with the snippet class as potential snippets to display.
Filters out snippets that don’t match the user’s location. See GeoLocation for information on how we retrieve and store geolocation data.
Filters out snippets that are only supposed to be shown to users without a Firefox account.
Filters out snippets that are only supposed to be shown to users with a certain search provider.
Chooses a random snippet from the set based on their “weight” (a higher weight makes a snippet show more often relative to snippets with lower weights).
Displays the snippet.
Triggers a show_snippet event on the .snippet element.
Modifies all <a> tags in the snippet to add the snippet ID as a URL parameter.
Logs an impression for the displayed snippet by sending a request to the snippets metrics server. These requests are sampled and only go out 10% of the time. See also Data Collection chapter for more information on the data send to the metrics server.

If no snippets are available, the code falls back to showing default snippets included within Firefox itself.

Developing a Snippet Template¶

The following document explains how to develop and test a snippet template for use in the Snippets Service. It is assumed that you’ve already filed a bug in the Snippets Campaign component in Bugzilla to inform the snippets team that you want to create a snippet template.

Setting up your development environment¶

In order to develop a snippet template you need to setup have a snippets server. You can either setup the full Snippets Service or the Simple Snippets Server for Template Development. The latter is strongly recommended and it will be all you’ll need for template development. It needs no configuration and the code served to the browsers is the same the original Snippets Service serves. The major difference is that all the Client Matching rules are ignored. All snippets are served all the time and on every request which makes this ideal for template development.

Template or snippet?¶

A snippet is an individual piece of code that is displayed on about:home to users. Snippets are, in turn, generated from templates, which contain code shared between multiple snippets. Templates are useful for snippets where you want an admin to be able to substitute different text for localization or other customization purposes. Templates are written in the Jinja2 template language.

How are snippets loaded?¶

When a user requests snippets from the snippets service, the service finds all snippets that match the user’s client, generates the code for each snippet, and concatenates all of the code together, along with some default CSS and JavaScript.

When the user views about:home, this chunk of code is injected into the page. The default CSS hides all tags with the class snippet. Once injected, the default JavaScript selects a snippet to show and un-hides the hidden tag. Finally, a show_snippet event is triggered on the snippet tag to signal to the snippet that it is being displayed.

Snippet requirements¶

All snippets must have an outermost tag with the class snippet, and no tags outside of that except style or script tags:
```
<div class="snippet">
  <img class="icon" src="some-data-uri" />
  <p>Foo bar baz biz</p>
</div>
```
Snippet code must be valid XML. This includes:
- Closing all tags, and using self-closing tags like <br />.
- Avoiding HTML entities; use their numeric entities instead, e.g. » instead of ».
- Using CDATA comments within all script tags:
```
<script>
  // <![CDATA[
  alert('LOOK BEHIND YOU');
  // ]]>
</script>
```
Avoid loading remote resources if possible. For images and other media that you must include, use data URIs to include them directly in your snippet code.
Due to performance concerns, avoid going over 500 kilobytes in filesize for a snippet. Snippets over 500 kilobytes large must be cleared with the development team first.

Helpers¶

Accessing snippet id¶

To get the snippet id within a snippet template use snippet_id Jinja2 variable like this:

<div class="snippet">
  This is snippet id {{ snippet_id }}.
</div>

The syntax in a snippet is slightly different and uses square brackets [[snippet_id]]. Here is an example that uses the Raw Template:

<div class="snippet">
  This is snippet id [[snippet_id]].
</div>
Warning

Beware that in this case spacing matters and [[ snippet_id ]] will not work.

Custom Metric Pings¶

Snippet events can be captured and sent to our metrics server. By default snippet impressions get captured and sent to our metrics server tagged as impression. Clicks on <a> elements with defined href get captured too and get sent back as click.

Snippet developers can customize the metric name of clicks by setting the metric data attribute on the link. For example clicking on the link of the following snippet:

<div class="snippet">
  <p class="message">
    Click this <a href="http://example.com" data-metric="custom-click">link!</a>
  </p>
</div>

will send back a custom-click ping instead of a click ping.

Warning

Avoid setting up event listeners on links for click events and manually sending metric pings, or pings may get sent both by your click handler and the global click handler resulting in inaccurate numbers.

In addition to impressions and clicks snippet developers can send custom pings to capture interactions using the sendMetric function like this:

<!-- Use Raw Template to try this out -->
<div class="snippet" id="ping-snippet-[[snippet_id]]">
  <p class="message">Foo!</p>
</div>
<script type="text/javascript">
  //<![CDATA[
  (function() {
    var snippet = document.getElementById('ping-snippet-[[snippet_id]]');
    snippet.addEventListener('show_snippet', function() {
      (function () {
        var callback = function() {
          alert('Success!');
        };
        var metric_name = 'success-ping-[[snippet_id]]';
        sendMetric(metric_name, callback);
      })();
    }, false);
  })();
//]]>
</script>

Note

Callback function is optional.

Note

Only 10% of the pings reach the server. We sample at the browser level. See sendMetric function for implementation details.

Using MozUITour¶

Snippets and snippet templates can use MozUiTour to interact with the browser. Developer can directly use the following MozUITour functions:

Mozilla.UITour.showHighlight
Mozilla.UITour.hideHighlight
Mozilla.UITour.showMenu
Mozilla.UITour.hideMenu
Mozilla.UITour.getConfiguration
Mozilla.UITour.setConfiguration

For example to determine whether Firefox is the default browser can you use the following function in a snippet:

function isDefault (yesDefault, noDefault) {
    Mozilla.UITour.getConfiguration('appinfo', function(config) {
        if (config && config.defaultBrowser === true) {
            firefoxIsDefault();
        } else if (config && config.defaultBrowser === false) {
            firefoxIsNotDefault();
        } else {
            firefoxIsDefault();
        }
    });
}

You can even use the low level MozUITour functions:

_sendEvent
_generateCallbackID
_waitForCallback

to trigger more events. For example to trigger Firefox Accounts:

var fire_event = function() {
    var event = new CustomEvent(
        'mozUITour',
        { bubbles: true, detail: { action:'showFirefoxAccounts', data: {}}}
    );
    document.dispatchEvent(event);
};

Snippet Block List¶

Snippets can be prevented from showing using a block list. By default the block list is empty and the intention is to allow users to block specific snippets from showing by taking an action. Snippet service automatically assigns the block functionality to all elements of snippet with class block-snippet-button. For example a disruptive snippet can include a special Do not display again link that adds the snippet into the block list:

<!-- Use Raw Template to try this out -->
<div class="snippet" id="block-snippet-[[snippet_id]]">
  Foo! <a href="#" class="block-snippet-button">Do not show again</a>
</div>

If you need more control you can directly access the low-level function addToBlockList:

<!-- Use Raw Template to try this out -->
<div class="snippet" id="block-snippet-[[snippet_id]]">
  Foo! <a href="#" id="block-snippet-link">Do not show again</a>
</div>
<script type="text/javascript">
  //<![CDATA[
  (function() {
    var snippet = document.getElementById('block-snippet-[[snippet_id]]');
    snippet.addEventListener('show_snippet', function() {
      (function () {
        var link = document.getElementById('block-snippet-link');
        link.onclick = function() {
          // Add currently showing snippet to block list
          addToBlockList();
          window.location.reload();
        }
      })();
    }, false);
  })();
//]]>
</script>

Note

In this case we don’t utilize the special block-snippet-button class.

More low level functions are popFromBlockList and getBlockList.

Function addToBlockList will by default add to block list the campaign of the current snippet thus preventing all snippets -even the ones created in the future- with the same campaign name to get blocked. This is particularly useful when we do A/B testing. A user who blocked a variation of a snippet will not see any of the variations either, as long as they share the same snippet campaign.

If there’s no campaign set for showing snippet addToBlockList will block the ID from the snippet.

The function also accepts an argument to explicitly set the blocking value:

.. code-html::
   addToBlockList('foo-bar');

Will add foo-bar to block list instead of the showing snippet’s campaign or ID.

In bug 1172579 close button assets are provided to build a image button in your snippet. Refer to the simple snippet code on how to do this.

Testing¶

Once your snippet is done and ready for testing, you can use the snippet-switcher add-on to set the host for your about:home snippets to point to https://snippets.allizom.org or http://localhost:8000, depending on which server you are using for development.

Alternatively to using the add-on you can change the browser.aboutHomeSnippets.updateUrl perf from about:config to point to your server. For example

http://localhost:8000/%STARTPAGE_VERSION%/%NAME%/%VERSION%/%APPBUILDID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/

If you are using the staging server, the developer who set up your account and snippet should give you instructions on a Name value to use in the add-on’s settings in order to view your snippet specifically.

With the add-on installed or the perf change made, your about:home should load the latest snippet code from your local snippets instance (after a short delay). If the code doesn’t seem to update, try force-refreshing with Cmd-Shift-R or Ctrl-Shift-R and deleting local snippet storage by typing in a web console:

gSnippetsMap.clear()

What versions of Firefox should I test?¶

Depending on the complexity of your snippet, you should choose the oldest reasonable version of Firefox you want to support for your snippet, and test roughly every other version from that up until the latest released Firefox, and probably Nightly as well.

So, for example, if you wanted to support Firefox 26 and up, and the latest version was Firefox 30, you’d test Firefox 26, 28, 30, and Nightly.

What should I test for?¶

Basic functionality of your snippet. Make sure it works as you expect it to do.
Ensure that your snippet does not interfere with other snippets. The staging server has a normal text+icon snippet that is sent to all clients, which will help you ensure that the normal snippet can be shown without being altered by your snippet.
Ensure that your snippet can run alongside multiple instances of itself.
Ensure that the normal about:home functionality, such as the search box, links at the bottom, and session restore function properly.

Code review¶

There is a snippets Github repo that keeps track of the code for snippets we’ve run. Once your snippet is finished, you should submit a pull request to the snippets repo adding your snippet or template code for a code review. A snippets developer should respond with a review or direct your PR to the right person for review. If your snippet is already on the staging server, include the URL for editing it to make it easier for the reviewer to test it.

Contributing¶

Docker Setup¶

These instructions help you set up a copy of the site using Docker and fig.

Install Docker.
Install fig.

Grab the source code using git:

$ git clone --recursive git://github.com/mozilla/snippets-service.git
$ cd snippets-service

Note

Make sure you use --recursive when checking the repo out! If you didn’t, you can load all the submodules with git submodule update --init --recursive.

Build the Docker containers using fig:
```
$ fig build
```
Run the database migrations to set up the database:
```
$ fig run web ./bin/run-migrations.sh
```

Once you’ve finished these steps, you should be able to start the site by running:

$ fig up

If you’re running Docker directly (via Linux), the site should be available at http://localhost:8000. If you’re running boot2docker, the site should be available on port 8000 at the IP output by running:

$ boot2docker ip

Local Instance Setup¶

These instructions will set up an instance of the website running on your computer directly, and assume you have git and pip installed. If you don’t have pip installed, you can install it with easy_install pip.

Start by getting the source:

$ git clone --recursive git://github.com/mozilla/snippets-service.git
$ cd snippets-service

Note

Make sure you use --recursive when checking the repo out! If you didn’t, you can load all the submodules with git submodule update --init --recursive.

Create a virtual environment for the libraries. Skip the first step if you already have virtualenv installed:

$ pip install virtualenv
$ virtualenv venv
$ source venv/bin/activate
$ pip install -r requirements/dev.txt

Note

The adventurous may prefer to use virtualenvwrapper instead of manually creating a virtualenv.

Set up a local MySQL database. The MySQL Installation Documentation explains this fairly well.
Configure your local settings by copying snippets/settings/local.py-dist to snippets/settings/local.py and customizing the settings in it:
```
$ cp snippets/settings/local.py-dist snippets/settings/local.py
```
The file is commented to explain what each setting does and how to customize them.

Initialize your database structure:

$ python manage.py syncdb
$ python manage.py migrate

Once you’ve followed the steps above, you can launch the development server like so:

$ python manage.py runserver

Data Collection¶

The Snippets Service and the code that it embeds onto about:home collect data about user interaction with snippets in order to help us determine the effectiveness of certain types of snippets and measure whether a specific snippet is successful. This document outlines the types of data we collect and how it is handled.

Retrieving Snippets¶

The Overview document describes how Firefox retrieves snippets. The actual URL that Firefox uses for fetching snippets can be found under the about:config preference browser.aboutHomeSnippets.updateUrl and defaults to:

https://snippets.cdn.mozilla.net/%STARTPAGE_VERSION%/%NAME%/%VERSION%/%APPBUILDID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/

The names surrounded by % symbols are special values that Firefox replaces with information about the user’s browser.

STARTPAGE_VERSION

A hard-coded number within Firefox specifying which version of about:home is retrieving snippets. We sometimes increase this when about:home changes in a way that may break certain snippets.

Example: 1

NAME

The name of the product being used.

Example: Firefox

VERSION

The Firefox version number currently being used.

Example: 29.0.1

APPBUILDID

A string uniquely identifying the build of Firefox in use, usually in the form of the date the build occurred with a number appended.

Example: 2007083015

BUILD_TARGET

A string describing the platform and configuration used when building Firefox.

Example: Darwin_x86-gcc3

LOCALE

The locale that the current Firefox was built for. We use this for showing snippets in different languages only to users who can read that language.

Example: en-US

CHANNEL

The release channel for the current Firefox. This is typically one of release, beta, aurora, or nightly.

Example: aurora

OS_VERSION

A string describing the operating system that this Firefox was built for.

Example: Darwin%208.10.1

DISTRIBUTION

A string used to describe custom distributions of Firefox, such as when providing custom builds with partners. This is set to default for most instances of Firefox.

Example: default

DISTRIBUTION_VERSION

Version of the customized distribution. This is also default for most instances of Firefox.

Example: default

Metrics¶

Snippet code, which is executed on about:home, sends HTTP requests to a server located at https://snippets-stats.moz.works and/or https://snippets-stats.mozilla.org whenever an event occurs that we would like to measure. These requests are sampled at a rate of 1%, meaning that only 1% of the time an event occurs will a request be made.

Requests sent to snippets-stats.mozilla.org contain the following data (sent as URL parameters in the query string) in addition to the normal data available from an HTTP request:

Snippet Name: Unique name referring to the snippet that was being viewed when the request was sent.
Locale: The locale of the current Firefox instance (the same locale value described in the snippet URL from the previous section).
Country: The country code corresponding to the country the user is currently located in. This is determined via the user’s IP address and is cached locally within Firefox for 30 days. This value may be empty in cases where we can’t retrieve the user’s country.
Metric: A string describing the type of event being measured, such as a snippet impression or a link click.
Campaign: A string describing the snippet campaign this snippet is related to. We use this to help group metrics across multiple snippets related to a single campaign. This value may be empty.

Types of Metrics Gathered¶

The following is a list of the types of events that we collect data for as described in the previous section:

Impressions¶

An impression is whenever a user is shown a specific snippet.

Snippet Clicks¶

Whenever a link in a snippet is clicked, we trigger an event that notes which particular link was clicked. This includes links that may trigger an action besides opening up a new page, such as links that trigger browser menus.

Video Plays, Pauses, Replays¶

Some snippets allow users to view videos. Some of these snippets trigger events when the video is played or paused, when the end of the video is reached, or when the user replays the video after it finishes.

Default Browser¶

Some snippets trigger an event that tracks whether Firefox is the default browser on the user’s system. These snippets also trigger an event when the user makes Firefox their default browser by either clicking a link in the snippet or by setting the default outside of the browser.

Browser UI Events¶

Some snippets trigger events when the user clicks specific buttons in the Firefox user interface (as opposed to the in-page snippet). Examples of the elements that can be tracked this way include:

The “Email”, “Copy Link”, and “Start Conversation” buttons within the Firefox Hello dialog.

Google Analytics¶

The snippets statistics server may proxy data to Google Analytics, with stripped IP information and with a randomly generated UID unique to every request. Google Analytics is never loaded within about:home. Some Mozilla websites use Google Analytics to collect data about user behavior so that we can improve our sites.

GeoLocation¶

Some snippets target specific countries. For example a snippet about a greek national holiday would target only browsers requesting snippets from Greece.

To preserve user’s privacy the geolocation happens on the browser level and not on the service level. Snippet bundles contain a list of targeted countries among the actual snippet data, the snippet weight and other info.

The Browser pings Mozilla Location Service (MLS) to convert it’s IP to a country code. Upon successful response the result is cached for 30 days. Thus if a user travels from Greece to Italy for a week snippets targeting Greece will be shown while the user is in Italy.

The GeoTargeting code is part of the JS included in the snippet bundle and it’s managed from the snippets service itself. It is not part of Firefox.

Snippets Service¶

Contents¶

Overview¶

What is a snippet?¶

How are snippets retrieved by Firefox?¶

How are snippets displayed?¶

Developing a Snippet Template¶

Setting up your development environment¶

Template or snippet?¶

How are snippets loaded?¶

Snippet requirements¶

Helpers¶

Accessing snippet id¶

Custom Metric Pings¶

Using MozUITour¶

Snippet Block List¶

Testing¶

What versions of Firefox should I test?¶

What should I test for?¶

Code review¶

Contributing¶

Docker Setup¶

Local Instance Setup¶

Data Collection¶

Retrieving Snippets¶

Metrics¶

Types of Metrics Gathered¶

Impressions¶

Snippet Clicks¶

Video Plays, Pauses, Replays¶

Social Sharing¶

Default Browser¶

Browser UI Events¶

Google Analytics¶

GeoLocation¶