On designing the social user experience

RITE differs from a “traditional” usability test by emphasizing extremely rapid changes and verification of the effectiveness of these changes. […] Changes to the user interface are made as soon as a problem is identified and a solution is clear. […]

The results are summarized in Figure 1, which is a record of all the failures and errors over time (as represented by the participants) on the Age of Empires II tutorial. In addition, the graph shows the points at which the tutorial was revised. Every vertical line on the graph indicates a point at which a different version of the tutorial was used. Changes were implemented between participants 1, 2, 5, 8, 9, and 10 (6 iterations).

Using the RITE Method to Improve Products: a Definition and a Case Study (Playtest Research)

You want to squash all your bugs and deal with the major design issues before you try to get your big launch spike. Otherwise, you might get a spike plagued with bugs and 2 star reviews – not good.

I got a couple great comments about how to do this, by stealthily releasing and rebranding. Matt Brezina’s genius comment below:

One thing we did was launch 2-3 versions of our product under a different apple account, without our personal names on the app, before we launched Postagram. When we did a PR launch the product was basically just a branded version of our work from the past 4 months; we knew it would function well and we knew users would love it. Since then we’ve never spent more than 4 weeks developing a release. And we particularly use Android for quick experiments – the apple 1 week app approval delay can really slow down the iteration cycles

— How mobile startups can iterate better, faster, stronger

While Facebook for iOS is much faster than it was before, the speed comes with one compromise: the company can no longer roll out daily updates

— Facebook for iOS goes native, waves goodbye to HTML 5

We came up with a different plan to let you use the newest features without requiring you to update the entire app: a “fallback” renderer. When the news feed team designs a new type of story, Facebook for iOS downloads something it doesn’t quite understand. When we detect this, we use a “fallback” renderer to show the pertinent information in the new story in a format the app already understands. In the meantime, we create a new custom renderer and have it ready for our next app update. For areas within the app where we anticipate making changes more often, we will continue to utilize HTML5 code, as we can push updates server side without requiring people to download a new version of the app.

— Under the hood: Rebuilding Facebook for iOS

If we run an experiment large enough, we want to tell everybody, “Hey, we’re doing an experiment.”

One of the great things about working on products at Google is that we can try stuff out and launch it as an experiment and get tons of data. As a designer, I feel very lucky because I have access to enough data and enough users that I can actually get statistically significant information on whether or not something should be this tall, or this tall, or this tall.

Some designers, it drives them mad. But I think, if you knew that this mock-up is actually, objectively a better experience for people than this mock-up, you’d totally take that.

On the toolbar, this is actually a very complicated design problem. One of the dials that you’re turning is making it a functional, useful, discoverable tool. But other dial you’re turning is that you want to make sure you have enough space on a variety of devices and screens to make sure that the application [the user] is in is given as much space as possible.

Sometimes these trade-offs come into conflict. We have to do a lot of designs, a lot of experiments, a lot of iterations, and so in this particular case, we’re in the middle of that process. We’re trying some stuff, we’re looking at it, we’re seeing how it works, taking that learning, walking it back and iterating on that set of designs.

We’re pretty open about what we do, so we iterate in public a lot and try different things out. This is what happens on search in particular. We’re running tons of experiments all the time to tweak and fix and change.

— Don’t Break Search: Q&A with Google Lead Designer Jon Wiley

We are actually straight up injecting the site inside these apps […] betting on the browser is just kind of something that we do at Facebook. It is what we are good at and it is our ultimate opportunity to move fast and we are betting hard on the Web because that is what got us here today. So, we basically want to put the browser inside the Facebook for iPhone app.

Iterate Daily

So, what does this get us? It means that you can ship daily. You do not have to wait for an app store approval, which sucks, and you don’t have to wait for every user to download this version on the binary which has to be shipped over into their phone. That sucks. You can ship every day. You can break it every day, but you can ship every day. […]

Being able to write it once today and ship it tomorrow? That is something that Facebook is really good at and that we love doing and that is at the center of being able to move fast. Move fast has an implicit third clause - move fast, break things, and fix things fast. That is very difficult to do if you have already shipped your binary to Apple or Android and they have to download another version of it.

— How Facebook Mobile Was Designed to Write Once, Run Everywhere

Engagement & Retention

Biggest Challenges:

• Although testing and iterating is how social devs have found success on Facebook, rapid iteration is not so easy or fluid on mobile platforms like iOS. It can take from three days to two weeks to receive approval for each new version from the App Store, making it difficult to fine-tune to optimize on the fly.

• Many users don’t update their apps regularly, resulting in outdated, or “dirty,” data from the combination of “old” and “new” app version users. This makes it challenging to iterate and improve your app based on user data without being misled, unless your data tools easily filter users by app version.

Biggest Opportunities:

• Design the app to use data-driven back-end systems to control user experience in-app when a connection is available on the device. This way, developers can dynamically tweak play balance, economy balance, etc., and then perform split-testing to further optimize customer economics.

• Use an analytics solution that allows filtering data by app version, location and device type (for Android). This is very important in addressing the dirty data issue, and in determining which platforms and devices in which to commit time, effort and resources.

— Applying Lessons Learned on Facebook to Mobile App Development