AWS CONFIG "Timeline"

System design

In 2014, there was not a way to access full history of changes to AWS customers resources and relationships over time.

I was part of an ambitious project to release a new console experience for one of the most important cloud computing companies.

To comply with my non-disclosure agreement, I have omitted and obfuscated confidential information in this case study. All information in this case study is my own and does not necessarily reflect the views of AWS.

AWS Timeline

DESIGN BY INNOVATION

AWS Config (Amazon Web Services Config) is an Amazon cloud auditing tool that provides an inventory of existing resources, allowing an administrator to accurately track AWS assets to analyze compliance levels and security. It also enables an administrator to troubleshoot why a resource may have stopped working properly.

With Config, you can review changes in configurations and relationships between AWS resources, dive into detailed resource configuration histories, and determine your overall compliance against the configurations specified in your internal guidelines. This enables you to simplify compliance auditing, security analysis, change management, and operational troubleshooting.

THE CHALLENGE

DESIGN THE MAGIC IN 3 MONTHS

My goal for the project was to design a system that accesses full history of changes to the customer resources and relationships over time. Also, how users can discover AWS resources that exist within an account at any point in time.

  1. Discover AWS resources that exist within an account at any point in time.
  2. Get a consistent view of resource configuration across different AWS resource types.
  3. Access full history of changes to your resources and relationships over time.
  4. Get details about configurations of every resource.
  5. Visualize dependencies between different AWS resources through the published relationship model.

MY ROLE

I led the design of the console experience between August 2014 and August 2016.

In addition, I worked alongside a Researcher and 2 Product Managers.

The console launched globally on November 2014 during the AWS Re:Invent conference. It was featured as a Keynote.

AWS Logo

Customer Insights & Ideation

I partnered with one project manager to uncover insights and translate concepts into features that address customer behaviors and motivations.

Experience Strategy & Vision

I created frameworks and prototypes to share the vision, design principles and content strategy. This helped to evangelize ideas, gain alignment and drive decision making.

Planning & Scope Definition

I defined the product with my project manager partner. I evangelized customer goals and balanced business goals. I prioritized and negotiated features for launch and beyond.

Oversight & Coordination

I designed to translate product features for each platform context.

Design Execution & Validation

I designed down on Desktop. I executed wireframes, prototypes and design specs.

Leadership

I designed up and presented works to gain buy‐in from executives, senior stakeholders and many other Amazon teams throughout the project lifecycle.

"HIGH NOON" situation

The biggest challenge for this project consisted of convincing leadership that the use of a timeline is the correct approach to solve our customer problem.

The main purpose of the console is to allow users to navigate through time in order to be able to search, browse and analyze data. At that point in time, AWS lacked a control to allow the customer to navigate in time. The only control in the UI library close to what I was looking for consisted of a date picker control.

Skepticism was one of the biggest challenges that I faced during the completion of this project.

The following design process example shows how I was able to succeed with my proposal.

KICKOFF

UNDERSTANDING THE USER

“As a System Administrator I want to navigate in time to inventory resources that exist in my account, understand current and previous configurations for their resources and evaluate how a configuration change to one resource affects related AWS resources.”

—Auric Goldgrant.

I synthesized my personas from observations, research and stakeholder interviews. To create a thorough persona I normally include social and demographic characteristics, needs, desires, goals, habits, expertise, cultural background, and motivations.

.

WHAT ELSE HAS AWS TRIED?

It was challenging to design a tool for navigating in time to see changes without breaking some UXDG patterns. The first attempt was to use only UXDG components.

Feedback from users suggested investigating another way to allow customers to navigate through time in a more intuitive and aesthetical way.

The responses obtained from users after showing the previous mock stated that using a Date Picker control, to see changes of resources over time, was not the correct solution. Date Pickers consist of input controls where users enter data to obtain a result. The user feels like in an Easter Egg hunting trying to see changes. Contrary, we were looking for a control that helps users to visualize a series of changes over time without entering any input.

.

IS THERE ALREADY ANOTHER SOLUTION? IF SO, DOES IT WORK?

Google Analytics mobile app has a very interesting UI and IxD from Google Analytics. Data and time are linked through a Time Line.

WHY A GOOD CANDIDATE?

  1. Simple component like in AWS.
  2. The design works on small port views.
  3. Responsiveness.
.
Deeper Insights

WORKING BACKWARDS FROM PERFECT

Before I could jump into designing, it was important to define success and understand the requirements.

Due to the complexity of the system I was designing for, creating a flowchart helped me represent the logic sequence of the user flow from the starting point to accomplishing a task.

.
The purpose of a flow chart is to provide people with a common language or reference point when dealing with a project or process.
Detailed Design

Communicating Design

Amazon upholds infamously high standards for the work it produces both externally for customers and internally for team members to consume.

This has created a culture which seeks to earn trust through accountability, diving deep into the details and inviting others to scrutinize work. Heavy documentation is the artifact of such a culture.

The size of this project and structured waterfall approach meant that I needed to have everything figured out before teams would commit to moving forward with the work. Many teams involved in the project needed to see it in a tangible document. This risk-averse mindset meant I created a lot of reference documentation that was widely distributed and a high overhead to maintain.

GENERATE IDEAS CONSTANTLY

.

For each feature phase, I went through cycles of requirements, consensus, approvals, detailed specs and handoffs.

My process involved sketching and white‐boarding concepts and flows with my PM partner and then translating these directly into hi‐fidelity design comps. Since I was working with many existing design patterns, it was relatively easy to move straight into hi‐fidelity designs.

REFRAMING THE PROBLEM

Sketches are about exploring ideas

Prototypes are about testing ideas

“...how might we help customers navigate through time?”

This begged the question, how might we help customers navigate through time? Our proposal was a Timeline.

TASK WIREFRAMES

This documentation required the most rework during the project and was the highest overhead to maintain.

I also experimented and came up with complementary documentation to communicate animation and timing keyframes for our micro‐interactions.

My next step involved slicing the comps and piecing them together with SKETCH or Adobe Illustrator into a prototype. In the early stages I focussed only on representing the highest risk areas of the design. Later phases allowed me to focus on micro‐interactions, which I created in Axure RP.

I use Tasks Wireframes constantly during the early phase of prototyping because it helps to present the flow and interactions of the design without spending time developing an interactive prototype. Each task is explained so stakeholders can review and comment on the design. After the wireframe is approved then I start working on a high-fi prototype where developers can visualize and interact with the design.

PROTOTYPING

Prototyping was the most effective way to gain meaningful feedback from the team, consensus from stakeholders and approval from senior leadership. I was able to easily distribute these as videos and recycle them for Usability Testing.

For the TimeLine prototype I decided to use Axure RP to show the design and interaction of the idea. One of the main reasons why I decided to use Axure to develop the prototype was the limit of time that I had to transform the idea into a playable artifact.

One problem that I had to face as a result of using Axure for developing the prototype consisted of the animation limitations of the tool. It was complicated to match the effect that we were looking for when navigating through pages using the direction controls. However, with a little research and help from other peers, I was able to represent with accuracy the animation using Axure RP.

As a result, Axure RP is a very powerful tool for prototyping. Sometimes some investigation is required to find solutions and examples for your design requirements but in the end, Axure RP is a very reliable tool.

STORY OF A TIMELINE

The different iterations based on user feedback

Just request and we do the rest

I was surprised by the issues we found testing the first concept. The main problem consisted of locating the "changes point" indicators on the timeline.

We wanted to make sure that users understand that changes happen at the beginning of a period of time. As a result, if the "Change Point" is in the middle of the "timeblock", it creates confusion to users regarding when the change occurred.

Solution. We justified the "Change Point" marker to the left. Even though that it seems like a complex concept, users did understand the meaning having the indicator of the changes in the left position on the "TimeBlock".

.
First iteration

Better Understanding Where the User Is

Another issue consisted of comparing the current time and the "Custom" time selected by the user. The time located below the TimeLine (Left corner) shows what time is at the moment. On the other hand, the timestamp under "Custom" represents the time that users have selected to see changes.

This was a clear example of engineering overthinking. Why show the current time when users can just look at the computer’s time to see what time it is now? The current time was removed from the UI after.

.
What time is it?
“...the information seems to be too clustered.”

We've got your back

The next design shows another variation of the TimeLine. For this case, we display all the possible information inside a "block time", including the number of changes.

The problem with grouping all the information raises the difficulty of users to scan for a determined type of information. “The information seems to be too clustered.”

.
"Block Time".
“...is this a flowchart?.”

Starting Point

Another concept that displays a starting and ending point with the changes in the middle.

Adding a starting point to the time line creates confusion to users. The position of the starting point may vary. As a result, the interaction generates a continuous “jumping around” effect that disorients the user.

The solution is to position the starting point always at beginning of the time line (Left side).

.
Flowchart look and feel.

Goodbye flowchart

New approach in the design. This look eliminates the "flowchart" feeling from the previous designs.

The arrows on the "TimeBlocks" produces a "flowchart" effect rather than a time line feeling. In addition, the direction of the arrows gives the wrong information about the direction of the timeline. In our case, we want to represent that time starts from left to right.

Also, circles with a number represent the number of changes. The change helps to separate the two most important pieces of information, Time and Changes.

Discoverability is an issue in this design. Many users did not understand what the number inside the circle represents. They need to mouse over to see the "Changes" label.

.
Almost there...

...and the winner is...

Final version presented at Re:invent 2014.

Several visual design improvements help to obtain positive feedback from users. ready to test!

.
Re:invent 2014.

See timeline different interactions

— HTML page
THE DISCOVERY

USER RESOURCES CHANGED OVER TIME

“Traveling on time revealed an opportunity to perfect the navigation experience.”
Evaluate

TESTING...ONE, TWO, THREE...

Study conducted in October 2014

We tested the new Starling timeline UI, looking for usability problems. We also talked to participants about what they would use a tool like Starling for, looking for what kind of functionality is needed to solve the users’ real problems.

We had 7 participants, where 4 were already recruited as beta users of Starling and the remaining 3 were from our usability panel. 5 out of 7 users managed large AWS installations, while the remaining two had less than 10 instances.

WHAT WENT WELL?

  1. All participants liked having a visual timeline to navigate the data.
  2. All participants navigated the timeline well.
  3. Starling was thought particularly useful for reverse lookups, such as seeing which instances were associated with which security group or autoscaling group.

MAIN ISSUES

  1. Two users were confused about how to read time line changes.
  2. Three users wanted to do see all changes between two points in time for a resource, without having to click on each change point.

PRODUCE

Design Production

Steps you take once you have a finalized design solution that you want to see implemented.

Can include: Design specs, wireframes, style guides all the way to implementation, engineering, bug testing, user documentation, etc.

.
Style guide used to show developers the design aspects of the TimeLine.

Launching Is Only The Beginning

The UCD Process is circular.

New features and improvements start the process again.

Reflections

WHAT I LEARNED

I am often asked if I am proud of launching AWS Config. I’m partially lying when I say I am. Let me explain why.

One of Amazon’s leadership principles is having a bias‐for‐action. Amazonians are proud to insist that product decisions are reversible and spending time doing is better than overanalyzing.

Throughout this project, I observed how bias‐for‐action mutated into a bias‐for‐delivery. Our team disproportionately focused on measuring outputs, rather than learning and measuring outcomes. This inevitably led to a lot of waste, short‐sightedness, and distraction for the team.

We let the question “how quickly can we build it?” define it, more than we let our customers define it. We let the phrase “let’s just get something out there” define quality, more than we let our customer define quality.

If we had asked “are we building the right thing?” as much as we asked “are we going to meet our date?“, we would have launched a more reliable, intuitive and polished product, sooner.

Viability should have been defined by our customers way before the technology and date already did.

I value simplicity, focus, and utility. I aspire to make people happy by designing experiences that just get out of the way. Craftsmanship and carefully thought out details are important to me.

At the time of launch, I had difficulty accepting the reality of this product, because I knew where all the dead bodies were hidden. I knew which critical features were missing. I knew how much waste was incurred building non‐critical features and inevitably how the performance and reliability of what mattered most were compromised.

My dissatisfaction is not a case of perfectionism, but rather an insistence on quality. Quality that should never be compromised, even in the first version of a product. Quality is the responsibility of an entire group and I have learned that beautiful experiences are only possible if the whole team truly shares the same values and aspirations.

Fast forward to the present, and I feel that my satisfaction and insistence for quality does not seem to matter at all. The success of this product had nothing to do with how I feel, but everything to do with if and how the product is being used.

So, if you ask me if I am I proud of what I launched I would still say no, but I would then tell you that I am very proud of what I accomplished for Amazon. I am proud that the team is in a better position to learn, and that launching this product needed to happen in order to expose how badly things were broken—both in the product and in the way we were working. I believe that great design takes time and wisdom, which is only possible if the entire team is in a position with an accompanying mindset to learn.

Today, millions of customers are enjoying AWS Config and we are exceeding our business objectives. And while I can’t prove how much the extra mile would have benefited our business, at least the product is not considered done.

Jeff Bezos’ famous saying at Amazon is that “it’s still day‐one”. For AWS Config, this could not be truer.