How we transformed our Agile process to include research and design
Discovery work planning meeting
Like most tech startups, Optimizely uses an Agile process to build software. Agile has been great at helping us plan what we’re building, coordinate across teams, and ship iteratively. Agile’s focus, however, is on shipping working software, which meant design and research were awkwardly shoehorned into the process.
Scrum, according to HBO’s Silicon Valley
Practically speaking, this meant we got into a habit of building a feature while it was being designed. This put pressure on designers to produce mockups of a feature in order to “unblock” engineers, which didn’t give them room to understand the problem and iterate on solutions. And it rarely provided space for our researchers to gather data to understand the problem, or do usability testing.
This caused problems on the engineering side, too: since engineers were building a feature while it was being designed, the requirements were shifting as we explored solutions. This slowed down the development process since they would have questions about what to build, and features would have to be rewritten as the designs evolved. This also made it hard to estimate how long it would take to build, which led to missed deadlines, cutting corners (such as not writing unit tests and or doing formal QA), and low morale.
In short, our process didn’t have high velocity or produce high quality solutions.
To improve this situation, we split our product development process into two phases: Discovery and Delivery. In the Discovery phase, the goal is to understand and solve our customer’s problems. The output is finished designs that solve for all use cases. After designs are done, work moves into the Delivery phase. The goal of this phase is to ship the finished solution to customers, and uses our standard Agile process to manage and plan the work.
The key to this system is that design and research are an explicit part of the process, and there is acceptance criteria that must be met before engineers write any code.
Diagram of Optimizely’s product development process
To help with organizational buy-in and planning, Discovery work is tracked on a physical kanban board in our office. The board is split into two parts: Problem Understanding (Research), and Solution Exploration (Design). In the Problem Understanding phase we gather data to help us understand and frame the problem. Data includes both qualitative and quantitative data, such as customer interviews, surveys, support tickets, sales call feedback, product usage, and NPS feedback. That data either becomes general company knowledge, such as our user personas, or feeds directly into the Solution Exploration phase.
In Solution Exploration, designers use the data gathered during Problem Understanding to frame the problem to be solved. Designers explicitly write down what the problem is, use cases, target users, and anything that’s out of scope. After getting buy-in from the PM and team, they explore solutions by creating sketches, wireframes, and prototypes. Engineers are looped in to provide feedback on technical feasibility. Researchers do usability testing in this phase as well. Finally, the output is finished designs that are fully thought through. This means there are no open questions about what a feature does that would slow down an engineer during development.
Optimizely’s Discovery kanban board
Is this waterfall?
This process is more structured than our previous Agile process, but not as rigid as a typical waterfall process. We don’t “throw work over the wall” to each other, stop requirements from changing, or rely on documentation to communicate across teams.
Additionally, designers still sit with scrum teams and attend standups. This keeps whole team involved throughout the entire process. Although engineers aren’t building anything during the Discovery phase, they are aware of what problem we’re solving, why we’re solving it, and proposed solutions. And designers are included in the Delivery phase to make sure the finished feature matches what they designed.
Since rolling out this system across our scrum teams, our design and development process has been much smoother. Researchers have a stronger voice in product development, designers have space to iterate, and engineers are shipping faster. By giving designers and researchers an explicit place in our product development process, we’ve improved our planning, increased coordination and alignment across teams, and upped our velocity and quality.
These posts all informed my thinking for why I implemented Discovery Kanban at Optimizely:
In the spirit of sharing my work, I thought it would be fun to look back at some early web sites I designed and built.
First site: CED Ventura
I built this site while I was in high school in 2000 or 2001 for my dad, who manages the Ventura branch of CED. It was the first full site I designed and built. I built it as the final project of a web design class I took in high school, and it went live shortly after that (you can still see it on the Wayback Machine). Prior to this I had been experimenting with HTML and made some sample sites, but had never made a full site.
Fun facts about this project:
No CSS. CSS barely existed back then and didn’t become practical to use until the mid-2000s. I had never even heard of CSS when I made this site. All styling comes from HTML tags like <font>, images, and some style attributes like valign.
Frames. The whole site is built in a <frameset> — one for the top logo area, another for the navigation, and a third for the content. Back then that was the best way to reuse content across pages, since static site generators didn’t exist and I didn’t know anything about server-side programming at the time.
Tables for positioning. Since I knew nothing about CSS back then, and it wasn’t a viable option anyway, I used tables to position everything.
Spacer gifs! In addition to using tables for positioning, I also sprinkled spacer gifs throughout the site to position elements.
Hand-coded. Yep, the whole site was coded by hand, before that became a retro, throwback badge of honor. I seem to recall using Dreamweaver to build the site, but was somehow already experienced enough to know the code it generated was shit, so I primarily lived in the code view.
Animated gifs. You can’t tell from this static screenshot, but the homepage had not 1, but 2 animated gifs on it: the logo in the top left that says “CED Consolidated Electrical Distributors,” which fades in one letter at a time over the excruciatingly long period of about 10 seconds (longer than your typical Snapchat); and the map, which features each dot lighting up to emphasize how many locations CED has nationwide. I thought I was the shit for making each of these animated gifs. Once again, the Wayback Machine will let you experience these gifs in all their glory.
1,300px wide top banner. At the time, a 1,300 pixel wide image for the banner seemed absurdly wide and I didn’t think there’d ever be a monitor large enough to display it. Boy was I wrong. When viewed on any computer today the image will start to repeat.
The Local Oafs
This is the second full website I designed and built. In true American tradition, I made this site for my closest friends’ band, The Local Oafs (named after a Simpsons reference, if you were wondering). I’m not sure it was ever on the public internet, but if so it was probably hosted on Geocities.
For whatever reason, I added a splash page to the site. I guess that was all the rage back then. I also designed the sweet logo with one of the band members. We also printed up stickers of the logo, and stuck it on a shirt.
Pretty sweet grunge look going on here, which I made in Photoshop. This site also features a table layout, spacer gifs, and no CSS.
I made bio pages for each band member, adding a different rad Photoshop effect to each bio pic. I really hope we were being ironic, but I fear we probably thought this was actually cool. I was also the band’s photographer, and did a photoshoot to populate the site with pictures.
Anacapa Fine Yarns
This is the first site I actually got paid for, which was hugely exciting at the time (2004 or 2005, I believe). I think I got a couple hundred bucks, which felt like highway robbery. I built the site for my parents’ friend who opened this yarn shop in Ventura.
Oof those colors. Not sure what I was thinking there. And the nav items are green when hovered 😬. And such centering!
For whatever reason I have two versions of the site on my computer. I think I rebuilt the site for her about a year after the initial version. Unfortunately the design updates I made didn’t include a better color palette. The biggest update, however, was that I built the site with CSS. This must have been the first real site that I used CSS on. It features great selectors like div#main.
The properties are surprisingly well organized, though. I grouped them by position, width/height, typography, and finally visual styles (background, border, etc.). The selectors are all quite flat, and I indented them to indicate the nesting in the HTML. Which seems pretty forward thinking considering pre-processors didn’t exist yet and I was pretty new to CSS.
Hamlet on the Moon
In college a close friend of mine did an adaption of Hamlet set on the moon, aptly named “Hamlet on the Moon.” It was actually a full play that was produced and performed at UCSB. It featured Polonius as a robot named “Poloniusbot,” Ophelia as a cyborg named “Opheliatron,” and Rosencrantz and Guildenstern as a two-headed monster. I designed and built the website, posters, and program for them.
Can you tell this play takes place on the moon? You may need to look again to pick up on the space theme. I especially love the NASA-y font for the nav, and that I made it a literal block of text by adjusting the type size to maintain a fixed width. And there’s virtually no vertical space between them. I had to do the text with images since web fonts were still years away.
And don’t miss that great pseudo-drop cap in the first paragraph.
This is one of two posters I created for them, which I believe they actually printed and put up around campus. The oddest thing about the posters is that neither feature dates, times, location, or the website’s URL. I have no idea why we chose to only have the email address on the poster.
The program, on the other hand, does include the date, time, and location, even though you’d have to be at the play already to be holding one of these. But overall it actually came out pretty decently. I was taking typography and print layout classes at the time and put those skills to use. It’s at least a more understated and mature design than the website.
Around 2008 or 2009 my close friend (who also did Hamlet on the Moon) started building Pixie, a web-based pixel art editor. I helped out with design and frontend engineering. I also built a couple features, like fill. I believe this was my first exposure to HAML and SASS. Looking back on it, this was the first time I did product design (although I wouldn’t have called it that back then).
The design is pretty clean. Probably a little too clean. Those left icons are about to float off the page. And are they even clickable? I kind of like the way the line draws your eye from the top left to bottom right, even though the line itself strangles the page a bit. The left button style and icons are weirdly inconsistent with the top button style and icons. Perhaps I was trying to draw a distinction between image editing tools, like drawing and erasing, and document controls, like “Save”? And what is that bottom icon supposed to do? The “logo” is also odd – where did that orange come from? I still hadn’t really figured out “color palettes.”
Cal Cooking Club
In 2010 I was in grad school and a member of the Cal Cooking Club, so I made their website. It features web fonts and a clean, modern design! I believe I used this for a class project, too. I think I made a static version of the site first, and then converted it to a Wordpress theme so the officers could update the site themselves.
Not the pinnacle of design, but it at least doesn’t make me cringe. The oddest thing about the design is that I tried to keep the site on a grid, which is why the nav items have weird spacing between them. I think I had just read Ordering Disorder by Khoi Vinh and went a bit too far trying to put his principles into practice.
Privacy Patterns is a project spearheaded by one of my grad school professors and a PhD student, with the goal of documenting best practices for building software that supports privacy. I got involved to help them design and build the site.
We built the site using Hyde, a static site generator. All the content was stored in Markdown for ease of authoring and portability. I believe it’s the first statically-generated site I built (with the help of a fellow grad student). Surprisingly I didn’t use SASS, even though I had at least experimented with it at this point. If I had to guess, it was probably because Hyde is built in Python, and SASS in Ruby.
Overall the site isn’t too bad. Clean design, no weird colors, and good type hierarchy.
It’s humbling to look back at where I started and reflect on how far I’ve come since. Best practices, technologies, and design trends all shaped my work, and have changed a lot over the years. It will be fascinating to look back at the work I’m doing now and marvel at the misguided decisions that seemed so right at the time.
In The Elements of Style, the seminal writing and grammar book by Strunk and White, the authors have a style rule that states, “Do not take shortcuts at the cost of clarity.” This rule advises writers to spell out acronyms in full unless they’re readily understood. For example, not everyone knows that madd is Mothers Against Drunk Driving.
Acronyms come at the cost of clarity. “Many shortcuts are self-defeating,” the authors say, “they waste the reader’s time instead of conserving it.”
Icons are the acronyms of design. Designers often rely on them to communicate what an action or object does, instead of simply stating what the action or object is. Unless you’re using universally-recognized icons (which are rare), you’re more likely to harm the usability of an interface.
Do you know what the icons on the left mean?
So as Strunk and White advise, don’t take shortcuts at the cost of clarity. “The longest way round is usually the shortest way home.”
Action is the bedrock of drama. Action drives the play forward and makes for a compelling story that propels the audience to the end. And an engaging play is just a series of connected actions, according to David Ball in Backwards & Forwards.
Like a play, the user’s journey through your product is also a series of connected actions. Every click, tap, and swipe is an action users take. But unlike the audience of a play, which is just along for the ride, your users are in the driver’s seat trying to reach a specific goal. If you, as the designer, don’t make the series of actions to reach that goal clear, your users will get lost and your product will fail.
To help authors write engaging plays, David Ball recommends starting at the end of the play and identifying each preceding action, all the way back to the beginning. By looking backwards, you can see the specific steps that led to a particular outcome. “The present demands and reveals a specific past. One particular, identifiable event lies immediately before any other,” he says.
Looking forward, on the other hand, presents unlimited possibilities. The outcome of an action can trigger any number of other actions. You can only know which specific action comes next by looking backwards from the end.
This technique applies just as well to designing user experiences as it does to writing plays. Start by identifying the user’s goal, then walk backwards through each action they must take to get there.
We identified the user’s goal as creating their first A/B test. We arrived there by understanding the problem that A/B testing solves for our customers, which is to improve their app and ultimately make their business more successful.
If we had started at the beginning and worked our way forward, it would have been easy to stop once they installed our mobile SDK. But installing an SDK isn’t the customer’s goal. There’s no inherit value in that – it’s just a stepping stone to getting value out of our product.
Then we walked backwards through each step a user must take to reach that goal:
Goal: create your first A/B test.
To create an A/B test, you must install the SDK.
To install the SDK, you need to download it and add an account-specific identifier to your app.
To download and set up the SDK, you need an account and a project ID.
To create an account and a project, you must sign up by entering your info (name, email, billing info, etc.) in a form on our website.
Just by writing out each step like this, we eliminated excess steps and didn’t get distracted by edge cases or side flows. We had a focused list of tasks to design for. And at the conclusion of each task, we knew the next task to lead the user to.
Using this series of steps as a skeleton, we were able to design an onboarding flow that seamlessly led users to their goal. The experience has been praised by customers, and none of them have needed any help from our support team to create their first test.
So next time you’re designing a complex flow, start with the user’s goal and work your way backwards through each action they must take to get there. This technique will put you in an empathetic mindset that will result in user experiences that are clear and focused.
“Of such adjacent links is life — and drama — made up,” says David Ball. And so is product design.
Like many Americans, I started learning guitar back in high school. I began where everyone did – strumming basic chords and melodies and building up my finger strength. I got a little better every day, and could eventually play simple songs.
I kept practicing and getting better and pushed myself to learn advanced techniques. I wanted to know how to play all the notes — every scale, every chord, alternate picking, sweep picking, tapping, and so on. I didn’t want my technical proficiency to limit what I could play.
Over time I built up a repertoire of techniques to use. Even though I could play a lot of technically advanced parts, I didn’t necessarily know how to play the guitar well.
When he was developing as a writer, David Foster Wallace (author of Infinite Jest) had a similar focus on the advanced stuff. In his multi-day interview of the author, Although of Course You End Up Becoming Yourself, interviewer David Lipsky asks Wallace what his younger self would think of his new work, and if he thought things like character were pointless. Wallace responded with:
Not pointless but that they were easy. And that the hard stuff was more, you know, front of the head. It’s never as stark as pointless or not pointless. It’s, you know, what’s interesting, what’s advanced, what’s next? It’s gotta be — right? Not what’s true, but what’s fresh and novel and whatever. It’s very difficult to get out of that.
In his early work, David pushed himself to produce advanced work that would be considered “fresh” and “novel.” Not because that helped him communicate a larger truth to his readers, but because he wanted to push himself as a writer.
I’ve observed this focus on technical mastery in every creative field. Young filmmakers care more about getting the perfect lighting and shooting on film (rather than digital) than they do about story (check out the HBO series Project Greenlight for great examples of this). Digital product designers create slick UIs that look great on Dribbble, but aren’t usable or feasible or valuable. Programmers build technically impressive solutions to problems that don’t exist.
It’s easy to focus on this “hard” stuff because it has a clear path forward. Just practice what you aren’t good at and you’ll improve. And by learning the “hard” stuff, you’ll distinguish yourself from amateurs and beginners. When I was in high school, this is what I thought it meant to “master” the guitar.
As a result, I overlooked the “easy” stuff. The “easy” stuff is knowing when to use advanced techniques, and when to do something simple. It’s using your skills in service of achieving a higher goal – writing a song, communicating a truth to the reader, telling a good story, building something useful for people, etc.
I’ve since learned that to truly master your craft, you need to know the “hard” technical skills, and how to use those skills. So don’t just focus on learning all the notes. Learn when to play the right note, too.