RESEARCH PROJECT ON art museum website
This research project was part a group project between myself and three other students from my UX Design course at IADT. We were asked to complete a formal usability evaluation of a website of our choosing. We chose to imagine that the project was commissioned by the Irish Museum of Modern Art (IMMA) to evaluate the usability and user experience of their website, www.imma.ie.
Conducting Usability Testing Sessions
Two usability goals were chosen for this project “memorability” and “safety”. Memorability was chosen as initial research indicated museum websites are not visited very frequently, so the user needed to easily remember how to perform tasks. While “safety” was chosen to ensure that users were not making unnecessary errors and getting frustrated.
The user experience goal of “enjoyable” was chosen to reflect the experience of visiting IMMA in person, all stakeholders felt that the website needed to reflect this in its design, as well-designed interfaces can elicit pleasure in a user. To improve these, we first needed to understand what users thought of the website.
We combined qualitative and quantitative research methods to help us better understand and empathise with the users. At the very beginning of the project each member of the UX team conducted heuristic analysis was conducted of the IMMA website and also on competitor websites. This proved valuable in gathering information to help the UX team to prepare questions for the surveys and usability testing sessions.
An online survey was created to give quantifiable data about the general public’s interactions with museums and IMMA. From this survey it would be determined, how people interacted with the site, what devices they used, how often they visit the site, what tasks they performed and their experience with previous gallery/museum sites.
The survey would also act as a baseline to compare the researchers users results to. One of the drawbacks for this project was that it was based on convenience sampling by the researchers with the results not being as generalized as a larger study.
The survey was created in Google Forms and shared online through social media, gathering 202 responses. Some of the questions were:
- What art museum websites have you visited?
- How often would you visit the art museum site?
- What would you expect to do on a museum website (Click all that apply) – find opening hours, purchase tickets for exhibitions/events, browse the collection online, purchase from the museum shop, how to get there, read the blog, join the mailing list
- Have you ever visited IMMA (Irish Museum of Modern Art) website?
- If yes, How would you rate your experience of using the website?
The response format was a mixture of checkboxes, ranges, and rating scales (Likert and semantic). A call to action was included at the end of the survey to offer participants the opportunity to take part in a focus group. The quantitative data gathered was analysed using the survey tool and affinity diagrams.
Would expect to find opening hours on the website
Would expect to be able to purchase tickets online
would expect details online on how to get there
use a smartphone when browsing the internet
The usability testing goals were to find out how long it took users to perform certain tasks. The team created three scenarios for the tests with three tasks related to each scenario. A post task usability questionnaire was then completed at the end of the session.
12 test participants were to complete the testing. All research was carried out with the researcher and the participant interacting with a laptop computer using an online tool to record their website interactions and commentary.
The sessions were conducted in rooms where the session would not be interrupted and there were no outside distractions. This helped to minimise variables in the research settings, where possible, and to keep them to a constant.
The users were asked to look over the home page without interacting with it for a couple of minutes to get familiar with the website and questioned about their first impressions of the site. Three tasks in three scenarios were performed by the user until they were completed, or the user felt they could not complete it. While the users were performing their tasks, they ‘were asked to think out loud’ so that the researchers could understand what they were thinking during the tasks.
Once the tasks had been completed, the users were asked about the usability/user goals such as memorability, safety and enjoyable and their answers recorded. They were then given a post task questionnaire to complete to add quantitative data to the qualitative findings of the recording sessions.
Scenarios & Tasks
Scenario A – Tourist
You are a tourist who is travelling to Dublin and wants to visit IMMA. You want to know if the museum is near your hotel, what the opening times are, what is on at the museum, if there is a shop and if you can get something to eat while you’re there.
- What public transport you can get to the museum.
- What are the opening times for the museum café?
- Find out if you can buy online in the IMMA shop.
Scenario B – OAP
You are an old age pensioner who will be coming up to Dublin in May to visit family. As you will be in the city for a month you’re interested to see if there are any classes available. You’re also interested in potentially becoming an Imma Patron.
- Find out what events are scheduled for the month of May specifically.
- Find out what IMMA programmes are available to adults.
- Become and Imma Silver Patron.
Scenario C – Student
You are an art student who is interested in visiting IMMA. You have been learning about a particular artist in college and his work may be part of the collection so you want to see if that information is available on the website. You are also interested in doing a residency and would like to know when the open call is.
- Find out if it is it free for students.
- Search the collection for the artist Benny Andrews.
- When is the open call for the IMMA Residency Program?
A Usability Task Performance sheet was used while reviewing the user interaction videos. The user goal of “memorability” was analysed by comparing the amount of time that users took to complete tasks. A rating system for any usability issues, ranked from a minor design suggestion, to something which could prevent a task from being completed. Tasks were marked as successful or unsuccessful to see how easily users performed them.
For “safety”, any slips or mistakes the users made were noted and how these impacted on the tasks they were performing.
The user experience of “enjoyable” was analysed through whether or not tasks were successful or unsuccessful, as this can cause considerable frustration for the user. The users pre/post task commentary and interviews were also used to determine the users feelings about the website.
The post task questionnaire was also analysed. We made simple calculations by creating averages and percentages from timing the tasks and the closed questions.
Usability Task Performance Sheet Example
We were able to determine patterns and trends through affinity diagramming, using the quantitative and qualitative data gathered from heuristic evaluation, usability testing sessions, general questionnaire and post task usability questionnaire.
These were first themed under the heading of the user goals to start, then broken down into more themes as they emerged.
The most common patterns found were on navigation, responsiveness, volume of text, and overall design.
Content & Design
- The findings showed users are being overloaded with text, comments like “too much text” and “it’s very text heavy” show that this was an issue with more than one user.
- Some users were unclear on the language used on the website, making it easier for users to make an error e.g. “user confused about what a patron is and believes member, supporter and benefactor are all the same.” Users took the longest of all the tasks to find the IMMA programmes available to adults, it also had users looking for the most prompts with users confused by the different terms.
- logo was not a link to the homepage, there were comments from several users on this.
- During the heuristic evaluation the UX team highlighted the multiple navigation areas with duplicate menu items as a potential hazard for users. Further analysis of the data showed comments like “too many links” and “there are 4 separate areas for navigation”. Further findings from the post task questionnaire showed six of the twelve participants strongly agreed the website was “unnecessarily complex” and five strongly disagreed with the statement “the organisation of information on the website was clear”.
- One user expected to see a map when performing the task on public transport, they scanned over the information which was relevant to them in search of a map. Similarly users expected the calendar of events dates to show the dates specific for the month that they chose, not a range of dates that encompasses the year. One user clicked three times on the different events for May and stated “is there no exhibitions for May? […] I’m very confused by this”. This task took the third longest but had users looking for the most prompts.
were dissatisfied with how easy it was to use the website
were satisfied with the overall functions the website had
strongly agreed that the website was unnecessarily complex
strongly disagreed that they were satisfied with the website
- Five out of 12 users disagreed that it was easy to see what public transport to get to the museum
- Five out of 12 users agreed that they found the opening times for the cafe quickly
- Six out of 12 users strongly agreed/agreed that they found the opening times for the cafe quickly
- Five out of 12 users disagreed they could easily find an event scheduled for May
- Six out of 12 users strongly disagreed/disagreed they found information on the adult programmes easily
- Five out of 12 users agreed that they were able to become a silver patron easily
- Five out of 12 users disagreed that it was easy to find the cost for students
- Four out of 12 users disagreed that searching the collection was easy
- Five out of 12 users agreed that they were able to find information on the residency programme easily
- The navigation needs to be clear
- Using mental models is very important e.g. making the logo link to the home page, using maps for location.
- Use simple language as English could be a second language for your users.
- Keeping your site uncluttered enables users to perform tasks quicker.
- Talking with users is invaluable as it can lead to conversational tangents and bring up issues you may never have thought of.
- Use less text to convey your messages.
- Have clear CTAs on your site.
The sections above have been taken from the full research report that I completed with the aid of three other UX designers. This project was submitted as part of the Fundamentals of User Experience Design certificate course I completed at IADT.
A selection of related projects that I have worked on are shown below.