UX Research in eLearning Platform — Plan & Analysis

Case Study

A little about me…

Hello, I am Kat! I am a business psychologist with a background in business management, trying to break into the magical world of UX. Just recently, I acquired an opportunity to help a little Austrian startup company. Here is my research plan.

Project Description

The company building an eLearning platform for architects and other design enthusiasts. Their content is about controversial futuristic design that is often missing in an universities’ curriculum.

Advantages

• a global potential
• support in the architecture community
• existing Facebook page with access to over 50k fans
• a lot of existing content as a base to start with
• the beta-version of the website that was ready to be tested

Problem

• no financial capital
• the CEO is an architect with no experience in business management or product development
• no product team
• lack of structure & direction
• weak user knowledge
• usability issues with current beta-version

My Role

Initially, I was involved as a UX Researcher. However, as there was no Product Manager, UX Designer, or Business Coordinator, I had to take a leading role and do everything I could to point this project the right direction.

I worked directly with:

• the CEO and leading content creator,
• developers,
• marketing manager,
• users in the Facebook group.

This way I could align all the business and prdocut development activities.

I prioritised all the needs of the business as follows:

• I analysed the business’s needs using socio-technical systems thinking model and utilised principles of Lewin’s change model (read about it here)
• In parallel, I started with foundational research(read about it here).
• Lastly, I conducted directional research (read about it here).

Outputs and Deliverables

• personas and job stories
• pyramid of user’s needs
• competitor analysis
• job stories
• story map
• unique value
• short-, medium-, and long-term product strategy
• internal wiki
• wireframes
• sitemaps

Interview Data Analysis Workshop

Example of Persona

HIerarchy of Users’ Needs

Ideation Workshop

Table of competitors

Product Strategy

Wireframes

Sitemap

Timeline

Plan: 2 weeks

• Environmental analysis and industry overview: 1 week (I worked full time so I did not have much time during the day).
• Research plan, interview questions, interviewees’ specifications, email invitations: 1 week.

Empathise: 1.5 week

Define: 3 days

Ideate & Validate: 2 weeks

Measure & Learn: 1 week

Prototype: 1.5 weeks

Research Statement & Objectives

Before I set research objectives:

• I set a meeting with the CEO to find out business priorities and what she would like to get from this research.
• I reviewed existing materials to draw some assumptions about potential users and our competitors.

Subsequently, I set these research questions:

• Who are our real competitors?
• What activities to prioritise and how?
• What are usability issues in the beta-version?
• Who are our primary users, what are their frustrations, attitudes, feelings, goals, and needs?

These questions were divided in two research phases:

• Foundational research, that answered who were our users, competitors and how to prioritise.
• Directional research, that covered usability issues with the existing beta-version.

Recruitment criteria & Process

Recruitment & Participant’s Specifications

I aimed for 5+ participants which is the industry rule of thumb that reveals about 80% of all there is to know.

Foundational Research

The aim was to have geographically dispersed people from universities with varied students’ support. These were recruited via the Facebook group.

• 2 people from the best universities for architecture (e.g. Austria, UK)
• 3 people from other parts of the world from lower-ranking universities
• level of spoken English — advanced (to limit any assumptions I would have to make if they would not be able to express themselves well)

For the survey, I used the Facebook group and asked one of the tutors to share the survey (as his name is well-respected in the community). I also directly sent the survey to those participants from the prior interviews, who agreed to participate in further research.

Directional Research

Participants for usability testing were recruited via the Facebook group. I created a short questionnaire in Google Forms that helped us identify which ones of volunteers fit our personas.

After I created personas, I recruited 2 people that fitted each persona (6 in total). I asked a few people who I interviewed before as I was able to build a rapport during the interview.

• Open card sorting — I sent out the link to participants that completed the online validation survey and agreed to participate in further research. I asked the interview participants to complete it too.
• Closed card sorting — I sent a direct email to participants from the earlier survey they wished to be contacted in further research and fit our personas. I also asked the prior interview participants to complete the card sorting exercise.

In this article, I summarised used methods and analysis just briefly. If you want to see how I ran the sessions in more detail, here are the links for:

• foundational research,
• directional research.

Empathise

Research Methods & Tools

I opted for free tools, or tools that enabled me to create free accounts, having in mind the financial constraint.

Foundational Research

• in-depth interviews, MS Teams
• survey, Google Forms

Directional Research

• supervised open card sorting, Miro and MS Teams
• unsupervised closed card sorting, Maze.design
• moderated usability testing, Miro and MS Teams
• mockups Design, Miro, MS Teams

Card sorting and Mockup Design in MIRO

Define

Analysis

Interviews

• organised collaborative workshop to get a more accurate interpretation of interviews, and empathising with the users on a cross-functional level
• we clustered information into categories
• then divided into attitudes, goals, needs, and pains
• I collected feedback from experienced people from the industry and added additional information

Survey

I used the formula:

Opportunity = Importance + (Importance — Satisfaction)

Moderated Usability Testing

• average time of tasks’ completion
• tracked task success, calculated percentages

Open Card Sorting

• frequencies were quantified
• a matrix in an MS Excel, listing categories in the columns and cards in the rows
• raw counts into my spreadsheet and calculated percentages
• sorted cards according to their values and compared the percentages

Closed Card Sorting

• report from Maze.design
• compared percentages of cards in more than one category

Closed Card Sorting

Ideate

• organised ideation workshop
• result: ‘content crafting’ product strategy visualising short-, medium-, and long-term strategy

Prototype

• sitemaps for home and profile pages, using Miro
• high-fidelity wireframes for the home, profile, and course pages, using Figma

Impact

• company’s vision & mission
• business objectives for 5 years
• visually expressed business strategy presented to potential investors (“content” crafting model)
• wireframes for new landing, profile, and course pages
• product roadmap & product backlog
• foundation for marketing strategy
• team alignment & better cross-team collaboration

Next Steps and Recommendations

Build & Measure & Learn

As this is a young lean startup, an ongoing learning process is crucial. Firstly, I suggested implementing UX metrics for the redesigned website. Following HEART framework, I already used the task success and happiness elements in previously conducted directional research.

• Task Success. I measured if participants could achieve a set of given tasks and I measured how long did it take.
• Happiness. I asked about satisfaction and ease of use on a scale from 1 to 5.

This will enable me to compare this information with the new design and quantify changes.

With redesigned website and deployment of marketing strategies, I suggested installing Hotjar to see how users interact with the page. This will help to identify users’ behaviours and what needs to be investigated in further research. This can be done by adopting of retention and adoption metrics.

• Retention & adoption metrics. In next few months, it is crucial that the website tracks how many new users joined, how many of them returned, and how many dropped out to be able to pinpoint potential usability issues as soon as possible.

The company has a lined up collaboration with a university that encourages its design/architecture students to use the product. I would plan to recruit 15 volunteers for a simple diary study. This is another way that would help to discover potential usability issues with redesigned website.

Assumptions & Validations

Assumptions

1. The vast majority of students will be interested in our product.
2. Students lack opportunities for practical experience.
3. Students lack guidance & community support.
4. Students from the best universities will be keen to use our product.
5. All the students have the same needs.

What we validated

1. Students lack opportunities for practical experience.
2. Students lack guidance & community support.

What we disproved

1. The vast majority of students will be interested in our product.
2. Students from the best universities will be keen to use our product.
3. All the students have the same needs.