I worked on a team of four, collaborating with the City’s Dept. of Innovation and Technology (DoIT) on the budget site for fiscal year 2019 - 2020 for better data and information visualizations, site structure, and financial literacy.
Helping the People Understand Important, Complex Financial Statements
We provided the City of Boston's Department of Innovation & Technology with some research and wireframes for the budget site, budget.boston.gov. Our goal was to make it so the average citizen could go to the site, know what they were looking at, and find the information they cared about easily. Timeframe: 1 month
Pencil and paper
Google apps for documentation and sharing
User interface and interaction designer
Visual designer (and adherence to established brand guidelines)
Which arrangement of blocks works better for you? A or B?
After our last project, we decided to take on another one and do some work on the City's budget site, budget.boston.gov. The next fiscal year is already finalized, but they wanted to generate some initial ideas while they plan for the following year and the website's future. We were tasked with providing some researched lo-fi wireframes, and we wanted to give them different block wireframe options with layout and structure to continue their testing.
We discovered that it is really hard to test layout without any context or details. Participants in usability studies struggled with conceptualizing something in lower fidelity and being able to give feedback. We had to incorporate some content and give some context.
People love Budget and financial jargon
And by "love" I mean "get confused by or feel lost with." It's hard to juggle financial terms with conversational copy. The City of Boston does a good job with being mindful of accessible language, but this isn't about using synonyms and easing syntax. It's a whole different challenge when the words are specific to industry and that's just what it is.
This made card sorting and usability testing tough, because many participants weren't familiar with the language. When participants feel daunted by not understanding the matters at hand, it can have a negative effect on their experience and affect our results--showing that the same thing probably happens with users, too.
The budget site is content- and narrative-heavy, overwhelming users with walls of text that exhausts them and maxes out cognitive load before they can find the information they need.
By cutting down text and replacing it with interactive data visualizations, we believe people will be more engaged and be able to understand the budget more easily. This will help average citizens go to budget.boston.gov and feel satisfied in what they discovered.
Short on time, tough on strategy
We only had one month, and each of us was working part time on this. To start, we strategized to align ourselves and have goals, because we didn't want to waste any of the time we did have.
We wanted to know about and measure two things:
- Expectations. What is the mental model of the average citizen going to the site? What do they expect to find out? What do they want to find out? Where are those expectations not being met?
- Engagement. What is interesting on the current site? What are people looking at and reading about? What catches and holds their attention?
We planned out the methods we would use to achieve our study goals. For expectations, we decided to use:
- User interviews
- Card sorting prioritization
- 5-second test
For engagement, we decided to use these methods:
- Simulated contextual inquiry
- Post-inquiry interview
- Time passed metric
- Desirability study/product reaction cards
Quick, what do you think you will see here?
We sent out a link to a 5-second test through our channels that had the budget site on it. Participants were able to look and scroll for 5 seconds before the site disappeared and they were prompted to tell us what they expected to see. Most participants answered "fiscal report" or "budget information," indicating that they expect to see information that is dry and technical. Many other participants also answered they expected charts and graphs (i.e., data viz) or tools and interaction.
Now, What do you Want to see Here?
We ran closed card sorts, giving the participants three buckets regarding the information they wanted to see on the home page of the budget site, ranked by necessity:
- Need to know
- Would be nice to know
- Do not care
We put topics and components that are currently on the site on each of the cards. The most important takeaway was that almost no one understood what the topics and components were, meaning people are having a hard time accessing the language. Nonetheless, the results were what we had assumed.
Contextual Inquiry, Kind Of
We recruited Boston citizens for guerrilla-style, prompted contextual inquiry. Because they were residents of the city, they qualified for our study; however, almost half our participants mentioned they would never look at the budget site, which makes it not full contextual inquiry. We asked them to just be themselves and think about the issues they care about and money concerns about the City, and to go explore the site. My teammate and I switched off facilitating the session and taking notes in a notes grid. We found the following:
- Everyone was overwhelmed by the amount of text and lack of visuals
- Anytime they encountered visuals or interactive content, they became engaged
- They wanted more clarity in all areas
- They didn't feel like they understood where money was coming or going by the end of their sessions
Gauging Feelings & Reactions
To quantify qualitative data, we used two metrics: time passed and product reaction cards. For time passed, we simply timed how long the users were on the site, and then asked them after how long they felt like they were exploring.
Result: Every participant except one felt that they were on the site longer than they actually were, meaning engagement was low.
We also provided users with a list of 60 words and asked them to circle 5. We didn't allow them to write in any words or pick more or less than 5. This ensured that we wouldn't have biased or unclean data.
Result: Though the visual design was clean and aesthetically attractive, usability and engagement was not considered as positively.
Ideation & Information Architecture
At this point in the project, we had about a week and a half left. Our friend Steve came to add another mind to the ideation sessions. We sketched our ideas during a design studio exercise. Because we each came up with different ideas, we then wrote each component of our sketches on the whiteboard and voted on what we think would be the most effective, given the goals and features we prioritized.
We wanted to accomplish the following goals:
Visualize the most important info
Improve in-page navigation
Tie data more tightly to explanations
Consolidate or separate?
We had two options: consolidate all our ideas into one and test it a lot, or come up with different ideas and give the City a bunch of ideas, components, and layouts to move forward with and test. This is where we revisited the original prompt. The City wanted ideas, not a final solution, so we covered different areas.
rethink the nav
Part of the problem with the site was the information architecture and the navigation. There were parts of the site that weren't connected to anything, pages where you could only access them through one link down one page somewhere, column and hierarchical issues, taxonomy issues, and issues with readability and being able to follow along the page.
We wanted to focus on this executive summary as a home page idea, because currently, it does not operate as home. In the structure, it operates as if the other pages are on the same level.
We suggested to the City that they should rethink the structure for the sake of navigation, making the executive summary true to the name--a brief of the details and an anchor.
Handoff: Layouts & Components
Layouts with Building Blocks
At the end of our sessions, we mocked up two block-level, lo-fi wireframes to give them structures we came up with during ideation sessions. We wanted to focus on:
- interactive media
- content that is digestible and understandable
- putting the "how it works" video at the top of the page
- engaging visuals
- navigational summaries about the other parts of the budget site
- a feedback mechanism that allows citizens to reach out and tell the City what they need to know
Components as Building Blocks
We also passed on some places to start when it came to filling in those blocks in the layouts. Along with the navigation, we thought a little bit about other components to add and suggest. One was the feedback mechanism. We created a quick example of what that would look like. Our thought in adding this went back to the specific and project goals: increase civic engagement through innovative solutions and identify opportunities to improve communications about the budget to citizens. Having a feedback mechanism will passively tell the City what its constituents care about, and what the City can do to increase the chances of engagement.
Lastly, we created a higher fidelity wireframe that drew from many current parts of the site, but incorporated the most important insights we gathered during discovery:
- cutting copy and
- showing engaging media and visualizations of data.
We called it quick wins because it would be the easiest to implement, considering it kept the same site structure and navigational system. Take a look at the PDFs of the wireframes and annotations by clicking the button below.
What I Learned
does it matter if content is important and interesting?
Not if it isn't presented in the right way--approachable and clear. There are so many important and interesting bits of information in the current budget site, but it was presented almost 100% through text. Our usability study participants never actually accessed the important or interesting information, because they couldn't reach it in the way it was shown to them. With information as important as your city's money, your tax dollar, and your government, it has to be presented in the right way. By taking the steps to approach this through design and inquiry, the digital team is doing so much more than other cities when it comes to finding the best way to present the information.
If a design falls in the woods, and no one is there to read it, does usability matter?
Another issue that the City's digital team is facing is getting the word out there about their content, including the budget site. The budget site is a different circumstance, though, because one of the main user groups identified by the digital team's prior research was government employees. That means that many of the readers are familiar with the site already and the language. If most of the readers (government employees) aren't having the same issues (navigation, engagement, expectations) as potential readers (the average citizen), should we put so much effort into working on it?
I say again, yes.
Civic tech is different, because everything should always be done for the people. So it never mattered about the numbers and visitors to the page--this project just made me think about usability and frequency of use. In a situation where most people aren't using the product, it isn't always the idea of the product that is deterring users. Sometimes, making it more usable and engaging is a driving force in changing things so that more people do start using it.
This project gave me a rule of thumb: you never know, and really, who cares if lots of people aren't going to use it, make it usable no matter what.
I have worked with Billy and Amanda on two projects now. Thank you to them for what I have learned from those experiences. Thank you also to Steve for coming in and helping.