Snippets Service¶
The Snippets Service hosts small chunks of HTML, CSS, and JavaScript that are displayed on about:home. in Firefox, among other places.
Contents¶
Overview¶
This document describes the snippet service and how it relates to about:home, the home page for Firefox.
What is a snippet?¶
about:home is the default home page in Firefox. When you visit about:home, you normally see, among other things, a Firefox logo, a search bar, and a bit of text and an icon below the search bar. This text and icon are what is called a “snippet”.
Snippets can be more than just text and an icon, though. Sometimes a snippet can replace the Firefox logo with a larger image. A snippet could also pop up a video for you to watch. Snippets are made of HTML, CSS, and JavaScript, and can modify the homepage in interesting and fun ways.
The Snippets Service is a web service that serves the code for snippets. Firefox downloads code from the Snippet Service and injects the code into about:home. This allows administrators using the service to control which snippets are being shown by Firefox without having to update Firefox itself.
How are snippets retrieved by Firefox?¶
Firefox maintains a cache of snippet code downloaded from the Snippet Service
for at least 4 hours since the last download. The cache (and a few other
useful pieces of information) are stored in IndexedDB, and can be accessed by
code on about:home using a global JavaScript object named gSnippetsMap
.
When a user visits about:home, Firefox checks to see when it last downloaded new snippet code. If it has been at least 4 hours, Firefox requests new snippet code from the Snippet Service and stores it in the cache along with the current time. After this, or if it hasn’t been 4 hours, Firefox loads the snippet code from the cache and injects it directly into about:home.
Note
All Firefox does is download snippet code from the service and inject it into the page. The rest of the logic behind displaying snippets is determined by the snippet code itself, as explained in the next section.
Note
Firefox for Android caches snippets for 24 hours.
See also
- aboutHome.js
- The JavaScript within Firefox that, among other things, handles downloading and injecting snippet code.
How are snippets displayed?¶
The snippet code downloaded from the Snippet Service consists of three parts:
- A small block of CSS that is common to all snippets. Among other things, this hides all the snippets initially so that only the one chosen to be displayed is visible to the user.
- The code for each individual snippet, surrounded by some minimal HTML.
- A block of JavaScript that handles displaying the snippets and other tasks.
Note
It’s important to understand that all the code for every snippet a user might see is injected into the page. This means that any JavaScript or CSS that is included in a snippet might conflict with JavaScript or CSS from another snippet.
For this reason it is important to ensure that snippet code is well-isolated and avoids overly-broad CSS selectors or the global namespace in JavaScript.
Once the code is injected, the included JavaScript:
- Identifies all elements in the snippet code with the
snippet
class as potential snippets to display. - Filters out snippets that don’t match the user’s location. See GeoLocation for information on how we retrieve and store geolocation data.
- Filters out snippets that are only supposed to be shown to users without a Firefox account.
- Filters out snippets that are only supposed to be shown to users with a certain search provider.
- Chooses a random snippet from the set based on their “weight” (a higher weight makes a snippet show more often relative to snippets with lower weights).
- Displays the snippet.
- Triggers a
show_snippet
event on the.snippet
element. - Modifies all
<a>
tags in the snippet to add the snippet ID as a URL parameter. - Logs an impression for the displayed snippet by sending a request to the snippets metrics server. These requests are sampled and only go out 10% of the time. See also Data Collection chapter for more information on the data send to the metrics server.
If no snippets are available, the code falls back to showing default snippets included within Firefox itself.
Developing a Snippet Template¶
The following document explains how to develop and test a snippet template for use in the Snippets Service. It is assumed that you’ve already filed a bug in the Snippets Campaign component in Bugzilla to inform the snippets team that you want to create a snippet template.
Setting up your development environment¶
In order to develop a snippet template you need to setup have a snippets server. You can either setup the full Snippets Service or the Simple Snippets Server for Template Development. The latter is strongly recommended and it will be all you’ll need for template development. It needs no configuration and the code served to the browsers is the same the original Snippets Service serves. The major difference is that all the Client Matching rules are ignored. All snippets are served all the time and on every request which makes this ideal for template development.
Template or snippet?¶
A snippet is an individual piece of code that is displayed on about:home to users. Snippets are, in turn, generated from templates, which contain code shared between multiple snippets. Templates are useful for snippets where you want an admin to be able to substitute different text for localization or other customization purposes. Templates are written in the Jinja2 template language.
How are snippets loaded?¶
When a user requests snippets from the snippets service, the service finds all snippets that match the user’s client, generates the code for each snippet, and concatenates all of the code together, along with some default CSS and JavaScript.
When the user views about:home, this chunk of code is injected into the page.
The default CSS hides all tags with the class snippet
. Once injected, the
default JavaScript selects a snippet to show and un-hides the hidden tag.
Finally, a show_snippet
event is triggered on the snippet
tag to signal
to the snippet that it is being displayed.
Snippet requirements¶
All snippets must have an outermost tag with the class
snippet
, and no tags outside of that exceptstyle
orscript
tags:<div class="snippet"> <img class="icon" src="some-data-uri" /> <p>Foo bar baz biz</p> </div>
Snippet code must be valid XML. This includes:
Closing all tags, and using self-closing tags like
<br />
.Avoiding HTML entities; use their numeric entities instead, e.g.
»
instead of»
.Using
CDATA
comments within allscript
tags:<script> // <![CDATA[ alert('LOOK BEHIND YOU'); // ]]> </script>
Avoid loading remote resources if possible. For images and other media that you must include, use data URIs to include them directly in your snippet code.
Due to performance concerns, avoid going over 500 kilobytes in filesize for a snippet. Snippets over 500 kilobytes large must be cleared with the development team first.
Helpers¶
Accessing snippet id¶
To get the snippet id within a snippet template use snippet_id Jinja2 variable like this:
<div class="snippet"> This is snippet id {{ snippet_id }}. </div>
The syntax in a snippet is slightly different and uses square brackets [[snippet_id]]. Here is an example that uses the Raw Template:
<div class="snippet"> This is snippet id [[snippet_id]]. </div>Warning
Beware that in this case spacing matters and [[ snippet_id ]] will not work.
Custom Metric Pings¶
Snippet events can be captured and sent to our metrics server. By default snippet impressions get captured and sent to our metrics server tagged as impression. Clicks on <a> elements with defined href get captured too and get sent back as click.
Snippet developers can customize the metric name of clicks by setting the metric data attribute on the link. For example clicking on the link of the following snippet:
<div class="snippet"> <p class="message"> Click this <a href="http://example.com" data-metric="custom-click">link!</a> </p> </div>
will send back a custom-click ping instead of a click ping.
Warning
Avoid setting up event listeners on links for click events and manually sending metric pings, or pings may get sent both by your click handler and the global click handler resulting in inaccurate numbers.
In addition to impressions and clicks snippet developers can send custom pings to capture interactions using the sendMetric function like this:
<!-- Use Raw Template to try this out --> <div class="snippet" id="ping-snippet-[[snippet_id]]"> <p class="message">Foo!</p> </div> <script type="text/javascript"> //<![CDATA[ (function() { var snippet = document.getElementById('ping-snippet-[[snippet_id]]'); snippet.addEventListener('show_snippet', function() { (function () { var callback = function() { alert('Success!'); }; var metric_name = 'success-ping-[[snippet_id]]'; sendMetric(metric_name, callback); })(); }, false); })(); //]]> </script>Note
Callback function is optional.
Note
Only 10% of the pings reach the server. We sample at the browser level. See sendMetric function for implementation details.
Using MozUITour¶
Snippets and snippet templates can use MozUiTour to interact with the browser. Developer can directly use the following MozUITour functions:
- Mozilla.UITour.showHighlight
- Mozilla.UITour.hideHighlight
- Mozilla.UITour.showMenu
- Mozilla.UITour.hideMenu
- Mozilla.UITour.getConfiguration
- Mozilla.UITour.setConfiguration
For example to determine whether Firefox is the default browser can you use the following function in a snippet:
function isDefault (yesDefault, noDefault) { Mozilla.UITour.getConfiguration('appinfo', function(config) { if (config && config.defaultBrowser === true) { firefoxIsDefault(); } else if (config && config.defaultBrowser === false) { firefoxIsNotDefault(); } else { firefoxIsDefault(); } }); }
You can even use the low level MozUITour functions:
- _sendEvent
- _generateCallbackID
- _waitForCallback
to trigger more events. For example to trigger Firefox Accounts:
var fire_event = function() { var event = new CustomEvent( 'mozUITour', { bubbles: true, detail: { action:'showFirefoxAccounts', data: {}}} ); document.dispatchEvent(event); };
Snippet Block List¶
Snippets can be prevented from showing using a block list. By default the block list is empty and the intention is to allow users to block specific snippets from showing by taking an action. Snippet service automatically assigns the block functionality to all elements of snippet with class block-snippet-button. For example a disruptive snippet can include a special Do not display again link that adds the snippet into the block list:
<!-- Use Raw Template to try this out --> <div class="snippet" id="block-snippet-[[snippet_id]]"> Foo! <a href="#" class="block-snippet-button">Do not show again</a> </div>
If you need more control you can directly access the low-level function addToBlockList:
<!-- Use Raw Template to try this out --> <div class="snippet" id="block-snippet-[[snippet_id]]"> Foo! <a href="#" id="block-snippet-link">Do not show again</a> </div> <script type="text/javascript"> //<![CDATA[ (function() { var snippet = document.getElementById('block-snippet-[[snippet_id]]'); snippet.addEventListener('show_snippet', function() { (function () { var link = document.getElementById('block-snippet-link'); link.onclick = function() { // Add currently showing snippet to block list addToBlockList(); window.location.reload(); } })(); }, false); })(); //]]> </script>Note
In this case we don’t utilize the special block-snippet-button class.
More low level functions are popFromBlockList and getBlockList.
Function addToBlockList will by default add to block list the campaign of the current snippet thus preventing all snippets -even the ones created in the future- with the same campaign name to get blocked. This is particularly useful when we do A/B testing. A user who blocked a variation of a snippet will not see any of the variations either, as long as they share the same snippet campaign.
If there’s no campaign set for showing snippet addToBlockList will block the ID from the snippet.
The function also accepts an argument to explicitly set the blocking value:
.. code-html::
addToBlockList('foo-bar');
Will add foo-bar to block list instead of the showing snippet’s campaign or ID.
In bug 1172579 close button assets are provided to build a image button in your snippet. Refer to the simple snippet code on how to do this.
Testing¶
Once your snippet is done and ready for testing, you can use the
snippet-switcher add-on to set
the host for your about:home snippets to point to
https://snippets.allizom.org
or http://localhost:8000
, depending on
which server you are using for development.
Alternatively to using the add-on you can change the browser.aboutHomeSnippets.updateUrl perf from about:config to point to your server. For example
http://localhost:8000/%STARTPAGE_VERSION%/%NAME%/%VERSION%/%APPBUILDID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/
If you are using the staging server, the developer who set up your account and snippet should give you instructions on a Name value to use in the add-on’s settings in order to view your snippet specifically.
With the add-on installed or the perf change made, your about:home should load the latest snippet code from your local snippets instance (after a short delay). If the code doesn’t seem to update, try force-refreshing with Cmd-Shift-R or Ctrl-Shift-R and deleting local snippet storage by typing in a web console:
gSnippetsMap.clear()
What versions of Firefox should I test?¶
Depending on the complexity of your snippet, you should choose the oldest reasonable version of Firefox you want to support for your snippet, and test roughly every other version from that up until the latest released Firefox, and probably Nightly as well.
So, for example, if you wanted to support Firefox 26 and up, and the latest version was Firefox 30, you’d test Firefox 26, 28, 30, and Nightly.
What should I test for?¶
- Basic functionality of your snippet. Make sure it works as you expect it to do.
- Ensure that your snippet does not interfere with other snippets. The staging server has a normal text+icon snippet that is sent to all clients, which will help you ensure that the normal snippet can be shown without being altered by your snippet.
- Ensure that your snippet can run alongside multiple instances of itself.
- Ensure that the normal about:home functionality, such as the search box, links at the bottom, and session restore function properly.
Code review¶
There is a snippets Github repo that keeps track of the code for snippets we’ve run. Once your snippet is finished, you should submit a pull request to the snippets repo adding your snippet or template code for a code review. A snippets developer should respond with a review or direct your PR to the right person for review. If your snippet is already on the staging server, include the URL for editing it to make it easier for the reviewer to test it.
Contributing¶
Docker Setup¶
These instructions help you set up a copy of the site using Docker and fig.
Grab the source code using git:
$ git clone --recursive git://github.com/mozilla/snippets-service.git $ cd snippets-service
Note
Make sure you use --recursive
when checking the repo out! If you
didn’t, you can load all the submodules with git submodule update --init
--recursive
.
Build the Docker containers using fig:
$ fig build
Run the database migrations to set up the database:
$ fig run web ./bin/run-migrations.sh
Once you’ve finished these steps, you should be able to start the site by running:
$ fig up
If you’re running Docker directly (via Linux), the site should be available at http://localhost:8000. If you’re running boot2docker, the site should be available on port 8000 at the IP output by running:
$ boot2docker ip
Local Instance Setup¶
These instructions will set up an instance of the website running on your
computer directly, and assume you have git
and pip
installed. If you
don’t have pip
installed, you can install it with easy_install pip
.
Start by getting the source:
$ git clone --recursive git://github.com/mozilla/snippets-service.git $ cd snippets-service
Note
Make sure you use --recursive
when checking the repo out! If you
didn’t, you can load all the submodules with git submodule update --init
--recursive
.
Create a virtual environment for the libraries. Skip the first step if you already have
virtualenv
installed:$ pip install virtualenv $ virtualenv venv $ source venv/bin/activate $ pip install -r requirements/dev.txt
Note
The adventurous may prefer to use virtualenvwrapper instead of manually creating a virtualenv.
Set up a local MySQL database. The MySQL Installation Documentation explains this fairly well.
Configure your local settings by copying
snippets/settings/local.py-dist
tosnippets/settings/local.py
and customizing the settings in it:$ cp snippets/settings/local.py-dist snippets/settings/local.py
The file is commented to explain what each setting does and how to customize them.
Initialize your database structure:
$ python manage.py syncdb $ python manage.py migrate
Once you’ve followed the steps above, you can launch the development server like so:
$ python manage.py runserver
Data Collection¶
The Snippets Service and the code that it embeds onto about:home collect data about user interaction with snippets in order to help us determine the effectiveness of certain types of snippets and measure whether a specific snippet is successful. This document outlines the types of data we collect and how it is handled.
Retrieving Snippets¶
The Overview document describes how Firefox retrieves snippets. The
actual URL that Firefox uses for fetching snippets can be found under the
about:config preference browser.aboutHomeSnippets.updateUrl
and defaults
to:
https://snippets.cdn.mozilla.net/%STARTPAGE_VERSION%/%NAME%/%VERSION%/%APPBUILDID%/%BUILD_TARGET%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/
The names surrounded by %
symbols are special values that Firefox replaces
with information about the user’s browser.
STARTPAGE_VERSION
A hard-coded number within Firefox specifying which version of about:home is retrieving snippets. We sometimes increase this when about:home changes in a way that may break certain snippets.
Example:
1
NAME
The name of the product being used.
Example:
Firefox
VERSION
The Firefox version number currently being used.
Example:
29.0.1
APPBUILDID
A string uniquely identifying the build of Firefox in use, usually in the form of the date the build occurred with a number appended.
Example:
2007083015
BUILD_TARGET
A string describing the platform and configuration used when building Firefox.
Example:
Darwin_x86-gcc3
LOCALE
The locale that the current Firefox was built for. We use this for showing snippets in different languages only to users who can read that language.
Example:
en-US
CHANNEL
The release channel for the current Firefox. This is typically one of
release
,beta
,aurora
, ornightly
.Example:
aurora
OS_VERSION
A string describing the operating system that this Firefox was built for.
Example:
Darwin%208.10.1
DISTRIBUTION
A string used to describe custom distributions of Firefox, such as when providing custom builds with partners. This is set to
default
for most instances of Firefox.Example:
default
DISTRIBUTION_VERSION
Version of the customized distribution. This is also
default
for most instances of Firefox.Example:
default
Metrics¶
Snippet code, which is executed on about:home, sends HTTP requests to a server located at https://snippets-stats.moz.works and/or https://snippets-stats.mozilla.org whenever an event occurs that we would like to measure. These requests are sampled at a rate of 1%, meaning that only 1% of the time an event occurs will a request be made.
Requests sent to snippets-stats.mozilla.org contain the following data (sent as URL parameters in the query string) in addition to the normal data available from an HTTP request:
- Snippet Name
- Unique name referring to the snippet that was being viewed when the request was sent.
- Locale
- The locale of the current Firefox instance (the same locale value described in the snippet URL from the previous section).
- Country
- The country code corresponding to the country the user is currently located in. This is determined via the user’s IP address and is cached locally within Firefox for 30 days. This value may be empty in cases where we can’t retrieve the user’s country.
- Metric
- A string describing the type of event being measured, such as a snippet impression or a link click.
- Campaign
- A string describing the snippet campaign this snippet is related to. We use this to help group metrics across multiple snippets related to a single campaign. This value may be empty.
Types of Metrics Gathered¶
The following is a list of the types of events that we collect data for as described in the previous section:
Impressions¶
An impression is whenever a user is shown a specific snippet.
Snippet Clicks¶
Whenever a link in a snippet is clicked, we trigger an event that notes which particular link was clicked. This includes links that may trigger an action besides opening up a new page, such as links that trigger browser menus.
Video Plays, Pauses, Replays¶
Some snippets allow users to view videos. Some of these snippets trigger events when the video is played or paused, when the end of the video is reached, or when the user replays the video after it finishes.
Default Browser¶
Some snippets trigger an event that tracks whether Firefox is the default browser on the user’s system. These snippets also trigger an event when the user makes Firefox their default browser by either clicking a link in the snippet or by setting the default outside of the browser.
Browser UI Events¶
Some snippets trigger events when the user clicks specific buttons in the Firefox user interface (as opposed to the in-page snippet). Examples of the elements that can be tracked this way include:
- The “Email”, “Copy Link”, and “Start Conversation” buttons within the Firefox Hello dialog.
Google Analytics¶
The snippets statistics server may proxy data to Google Analytics, with stripped IP information and with a randomly generated UID unique to every request. Google Analytics is never loaded within about:home. Some Mozilla websites use Google Analytics to collect data about user behavior so that we can improve our sites.
GeoLocation¶
Some snippets target specific countries. For example a snippet about a greek national holiday would target only browsers requesting snippets from Greece.
To preserve user’s privacy the geolocation happens on the browser level and not on the service level. Snippet bundles contain a list of targeted countries among the actual snippet data, the snippet weight and other info.
The Browser pings Mozilla Location Service (MLS) to convert it’s IP to a country code. Upon successful response the result is cached for 30 days. Thus if a user travels from Greece to Italy for a week snippets targeting Greece will be shown while the user is in Italy.
The GeoTargeting code is part of the JS included in the snippet bundle and it’s managed from the snippets service itself. It is not part of Firefox.
Social Sharing¶
Some snippets contain popup windows to share content on social networks, such as Facebook or Twitter. Most of these snippets trigger an event when the user launches the popup window.