Introduction

I like to prototype things. To me—and surely not me alone—design is how something looks, feels, and works. These are very hard to gauge just by looking at a screen. Impossible, some would argue. Designs (and by extension, designers) solve problems. They address real needs. During the creative process, I need to see them in action, to hold them in my hand, on the street and on the train. In real life. So, as often as not, I need to build something.

Recently we’ve been exploring a few interesting ideas for Firefox Mobile. One advantage we have, as a mobile browser, is the idea of context—we can know where you are, what time it is, what your network connection is like—and more so than on other platforms utilize that context to provide a better experience. Not many people shop at the mall or wait in line at the bank with their laptop open, but in many of Firefox’s primary markets, people will have their phone with them. Some of this context could help us surface better content or shortcuts in different situations… we think.

Scope

The first step was to decide on scope. I sketched a few of these ideas out and decided on which I would test as a proof of concept: location aware shortcut links, a grouped history view, and some attempt at time-of-day recommendations. I wanted to test these ideas with real data (which, in my opinion, is the only legitimate way to test features of this nature), so I needed to find a way to make my history and other browser data available to my prototype. This data is available in our native apps, so whatever form my prototype took it would need to have access to this data in some way. In many apps or products, the content is the primary focus of the experience, so making sure you shift from static/dummy content to real/dynamic content as quickly as possible is important. Hitting edge cases like ultra-long titles or poor-quality images are real problems your design should address, and these will surface far sooner if you’re able to see your design with real content.

Technology

Next I decided (quickly) on some technology. The only criteria here was to use things that would get me to a testable product as quickly as possible. That meant using languages I know, frameworks to take shortcuts, and to ask for help when I was beyond my expertise. Don’t waste time writing highly abstracted, super-modular code or designing an entire library of icons for your prototypes… take shortcuts, use open-source artwork or frameworks, and just write code that works.

I am most comfortable with web technologies—I do work at Mozilla, after all—so I figured I’d make something with HTML and CSS, and likely some Javascript. However, our mobile clients (Firefox for Android and Firefox for iOS) are written in native code. I carry an iPhone most days, so I looked at our iOS app, which is written in Swift. I figured I could swap out one of the views with a web view to display my own page, but I still needed some way to get my browsing data (history, bookmarks, etc.) down into that view. Turns out, the first step in my plan was a bit of a roadblock.

Thankfully, I work with a team of incredible engineers, and my oft-co-conspirator Steph¹ said he could put something together later that week. It took him an afternoon, I think. Onward. Even if I thought I hack this together, I wasn’t sure, and didn’t want to waste time.

In the meantime, I got my web stack all set up: using an off-the-shelf boilerplate for webpack and React (which I had used before), I got the skeleton of my idea together. Maybe overkill at this point, but having this in place would let me quickly swap new components in and out to test other ideas down the road, so I figured the investment was worth it. Because the location idea was not dependent on the users existing browser data, I could get started on that while Steph built the WebPanel for me.

Working for now in Firefox on the desktop, I used the Geolocation API to get the current coordinates of the user. Appending that to a Foursquare API url and performing a GET request, I now had a list of nearby locations. Using Lodash.js I filtered them to only include records with attached URLs, then sorted by proximity.

var query = "FoursquareAPI+MyClientID";

navigator.geolocation.getCurrentPosition(function (position) {
	var ll = position.coords.latitude + "," + position.coords.longitude;
	$.get(query + ll, function (data) {
		data = _.filter(data.response.venues, function (venue) {
			return venue.url != null;
		});
		comp.setState({
			foursquareData: _.sortBy(data, function (venue) {
				return venue.location.distance;
			}),
		});
	});
});

Step 1 of my prototype, done. Well, it worked in the desktop browser at least. I knew our mobile web view supported the same Geo API, so I was confident this would work there as well (and, it did).

At this point, Steph had built some stuff I could work with. By building a special branch of Firefox iOS, I now had a field in the settings app which let me define a URL which would load in one of my home panels instead of the default native views. One of the benefits of this approach is that I could update the web-app remotely and not have to rebuild/redeploy the native app with each change. And by using a tool like ngrok I could actually have that panel powered by a dev server running on my machine’s localhost.

Steph’s WebPanel.swift provided me with a simple API to query the native profile for data, seen here:

window.addEventListener("load", function () {
	webkit.messageHandlers.mozAPI.postMessage({
		method: "getSitesByLastVisit",
		params: {
			limit: 10000,
		},
		callback: "receivedHistory",
	});
});

Here, I’m firing off a message to our mozAPI once the page has loaded, and passing it some parameters: the method I’d like to run and the limit on the number of records returned. Lastly, the name of a callback for the iOS app to pass the result of the query to.

window.receivedHistory = function (err, data) {
	store.dispatch(updateHistory(data));
};

This is the callback in my app, which just updates the flux store with the data passed from the native code.

At this point, I had a flux-powered app that could display native browser data through react views. This was enough to get going with, and let me start to build some of the UI.

Steph had stubbed out the API for me and was passing down a JSONified collection of history visits, including the URL and title for each visit. To build the UI I had in mind, however, I needed the timestamps and icons, too. Thankfully, I contributed a few hundred lines of Swift to Firefox 1.0, and could hack these in:

    extension Site: DictionaryView {
        func toDictionary() -> [String: AnyObject] {
            let iconURL = icon?.url != nil ? icon?.url : ""
            return [
                "title": title,
                "url": url,
                "date": NSNumber(unsignedLongLong: (latestVisit?.date)!),
                "iconURL": iconURL!,
            ]
        }
    }

Which gave me the following JSON passed to the web view:

[
	{
		title: "We're building a better internet — Mozilla",
		url: "http://mozilla.org",
		date: 1454514630131,
		iconUrl: "/media/img/favicon.52506929be4c.ico",
	},
];

Firstly, try not to judge my Swift skills. The purpose here was to get it working as quickly as possible, not to ship this code. Hacks are allowed, and encouraged, when prototyping. I added a date and iconURL field to the history record object and before long, I was off to the races.

Smart Dashboard

With timestamps and icons in hand, I could build the rest of the UI. A simple history view that batched visits by domain (so 14 Gmail links would collapse to 3 and “11 more…”), and a quick attempt at a time-based recommendation engine.

This algorithm may be ugly, but naively does one thing: depending on what time of day and day of the week it was, return to me some guesses of which sites I may be interested in (based on past browsing behaviour). It worked simply by following these steps:

1. Filter my entire history to only include visits from the same day type (weekday vs. weekend)
2. Exclude domains that are useless, like t.co or bit.ly
3. Further filter the set of visits to only include visits +/- some buffer around the current time: the initial prototype used a buffer of +/- one hour
4. Group the visits by their TLD + one level of path (i.e. google.com/document), which gave me better groups to work with
5. Sort these groups by length, to provide an array with the most popular domain at the beginning (and limit this output to the top n domains, 10 in my case)

The output is similar to the following:

[
	{
		"domain": "http://flickr.com/photos",
		"date": "Wed Feb 03 2016 10:54:02 GMT-0500",
		"count": 3
	},
	{
		"domain": "www.dpreview.com/forums",
		"date": "Wed Feb 03 2016 10:54:02 GMT-0500",
		"count": 2
	}
]

Awesome. Now I have a Foursquare-powered component at the top which lists nearby URLs. Below that, a component that shows me the 5 websites I visit most often around this time of day. And next, a component that shows my history in a slightly improved format, with domain-grouping and truncation of long lists of related visits. All with my actual data, ready for me to use this week and evaluate these ideas.

Favicons

One problem surfaces, though. Any visits that are from another device (through Firefox Sync) have no icon attached to them (right now, we don’t sync favicons across devices), which leaves us with large series of visits with no icon. One of the hypotheses we want to confirm is that the favicon (among other visual cues) help the user parse and understand the lists of URLs we present them with.

Again, I turn to my trusty toolbox, where I know the tools and how to use them. In this case that was Node and Express, and after a few hours I had an app running on Heroku with a simple API. I could POST an array of URLs to my endpoint /icons and my Node app would spin up a series of parallel tasks (using async.parallel). Each task would load the url via node’s request module, and would hand the HTML in the response over to cheerio, a server-side analog for jQuery. Using cheerio I could grab all the <meta> tags, check for a number of known values (‘icon’, ‘apple-touch-icon’, etc) and grab the URL associated with it. While I was there, I figured I might as well capture a few other tags, such as Facebooks OpenGraph og:image tag.

Once each of the parallel requests either completed or timed out, I combined all the extracted data into one JSON object and sent it back down the wire. A sample request may look like this:

    POST to '/icons'

    { urls: ["facebook.com"] }

And the response would look like this (keyed to the URL so the app which requested it could associate icons/images to the right URL… the above array could contain any number of URLs, and the blow response would just have more top-level keys):

{
	"facebook.com": {
		"icons": [
			{
				"type": "shortcut-icon",
				"url": "https://static.xx.fbcdn.net/rsrc.php/yV/r/hzMapiNYYpW.ico"
			}
		],
		"images": [
			{
				"type": "og-image",
				"url": "http://images.apple.com/ca/home/images/og.jpg?201601060653"
			}
		]
	}
}

Again, maybe not the best API design, but it works and only took a few hours. I added a simple in-memory cache so that subsequent requests for icons or images for URLs we’ve already fetched are returned instantly and with no delay. The entire Express app was 164 lines of Javascript, including all requires, comments, and error handling. It’s also generic enough that I can now use it for other prototypes where metadata such as favicons or lead images are needed for any number or URLs.

The finished prototype.

Conclusion

So why do all this work? Easy: because we have to. Things that just look pretty have become a commodity, and beyond being nice to look at they don’t serve much purpose. As designers, product managers, engineers—anyone who makes things—we are responsible for delivering real value to our users. Features and apps they will actually use, and when they use them, they will work well. They will work as expected, and even go out of their way to provide a moment of delight from time to time. It should be clear that the people who designed this “thing” actually used it. That it went through a number of iterations to get right. That it was no accident or coincidence that what you are holding in your hands ended up they way it is. That the designers didn’t just guess at how to solve the problem, but actually tried a few things to really understand it at a fundamental level.

Designers are problem solvers, not magicians. It is relatively cheap to pivot an idea or tweak an interface in the design phase, versus learning something in development (or worse, post-launch) and having to eat the cost of redesigning and rebuilding the feature. Simple ideas often become high-value features once their utility is seen with real use. Sometimes you get 95% of the way, and see how a minor revision can really push an idea across the finish line. And, realistically, sometimes great ideas on paper utterly flop in the field. Better to crash and burn in the hands of your trusted peers than out in the market, though.

Test your ideas, test them with real content or data, and test them with real people.