APPROACH & LEARNINGS (CHATBOT) (2)

Contents

APPROACH TAKEN
RESEARCH PLANNING
SWIM-LANE WORKSHOPS
Workshop examples
STRATEGY PROPOSAL
UI IMPROVEMENTS
Example
OUTCOME
WHAT DID I LEARN

APPROACH TAKEN

Conversocial structured design projects using the double diamond framework

To be able to try quick changes and understand their impact I chose to condense the initial discovery phase and run further discovery phases throughout:

Discover Research planning Evaluating what we already knew and what further insight was needed for our goals

Discover Swim-lane Workshops Identifying the processes for making bots & learning what the people involved thought & experienced

Define Strategy Proposal 3 streams for tactical and strategy deliveries

Develop UI Improvements Usability improvements that would be quick to deliver & measure

RESEARCH PLANNING

Evaluating what we already knew and what further insight was needed for our goals

We had two sets of related interviews which summarised how people felt using the bot platform and we saw 5 patterns:

Using the platform wasn't felt to be causing them delays
Getting copy & bot flows from clients was time-consuming and challenging
UI usability was reported to be difficult in places
Different types of chatbots have different complexity that wasn't inherently understood or planned for
Re-use between similar chatbots wasn't happening despite indications it should be

💡

However, these interviews didn't uncover the actual process people used for selling, making & maintaining bots. Usability frustrations were aired but without specifics being recorded.

Additional research to investigate these gaps in our evidence were required to achieve our goals:

Workshops to map process & highlight challenges
Interviews with sales & support colleagues
Ad-hoc interviews during an off-site with the Bot product team (product, design & developers)
A retrospective into the bot that was deployed in a broken state

SWIM-LANE WORKSHOPS

Identifying the processes for making bots & learning what the people involved thought & experienced

The workshops mapped process steps and their Actors. Pain points & opportunities were marked by the participants

Workshops

I coordinated a set of workshops to identify the processes for selling, creating & maintaining bots:

Two in-person workshops (EU) with the most experienced sellers and builders.

These took advantage of an 'off-site' being run for the acquired bot team

Four distributed workshops (UK & USA) bringing insight from the wider company

Including people selling bots
Less-experienced bot builders
The support team who supported bots after the 'build phase'
I facilitated these using Miro (online whiteboard) & video conferencing

Workshop examples

In-Person whiteboard swimlanes

The first board that begun our understanding of how bots were sold. Post-it comments come from the people making & selling bots

Red post-its indicate pain points our colleagues identified they where experiencing. Yellow indicate their thoughts for improvement opportunities

Blue indicates additional information for that step

Distributed swimplanes

Using Miro so each person could add to the flows

Capturing part of the sales & contract processes

Purple markers indicate missing detail that needed information from other participants

Example where requirements gathering steps would vary depending on the client or people involved in that bot

STRATEGY PROPOSAL

3 streams for tactical and strategy deliveries

Findings from the workshops & our compressed timelines lead me to propose 3 streams for our next steps

Discover: Complete the process mapping and conduct further analysis
Develop: Tactical changes to try within 6 months

Create a trained 'Bot squad' team that would specialise in bot building
Enable 'self-service' learning for bot builders with fresh platform guides
Check-list for making & deploying bots to introduce some quick standardisation to testing
Prioritise UI improvements to key areas of the platform identified as causing confusion and increases to defects

Deliver: With insight from the other two streams. A strategic stream to implement wider changes that would reduce the inconsistencies in bot building and provide Conversoical with the capability to build bots more reliably.

3 streams proposal with delivery and high level time lines

UI IMPROVEMENTS

Usability improvements that would be quick to deliver & measure

Conversocial was running a strategic project to add the Apple Business Chat (ABC) channel to its products, and I used this project to make UI improvements to ABC List Picker.

It was an opportunity to

Fix the identified usability issues
Showcase guerilla research to validate the UI changes before development
Introduce a more flexible & robust interaction model for complex channel components:
Feed the strategic streams with feedback from the changes

Example

ABC List Picker

The full-screen modal was chosen to reduce cognitive load about bot elements that are unrelated. The modal would be reusable for other projects as it was testing well for communicating a component's features

The end-user representative UI helped bot builders comprehend the nuances of a list picker

Untitled Document

sskh8h.axshare.com

Example Axure prototype & specification shared with stakeholders and developers

OUTCOME

⛔

As we began the tactical work, Conversoical went through management changes pivoting the company objectives. This stopped the project and triggered a replan

Through a combination of handovers and sprint work the tactical fixes were delivered:

The 'Bot squad' team was created and trained
Product documentation was completed
Small UI improvements had been made

The strategic streams were deprioritised in favour of a new product

WHAT DID I LEARN

Running workshops in a distributed manner is surprisingly insightful

Miro enabled all participants to draw onto the digital 'whiteboard'. We were able to continue the collaborative approach from the in-person workshops
We were able to spread the workshops to fit around people's schedules and timezones. By running smaller sessions I witnessed more engaged participants
Conversocial is a distributed company. This helped the transition to run workshops remotely as colleagues were already experienced in discussing detailed topics over a webcam or screen share

Incomplete Research Analysis can still be useful insight

The workshops provided a lot of detail and whilst we never completed the analysis, what was done did help other projects
I was able to find small opportunities to increase bot robustness or the platform usability based on the findings
The tactical fixes turned out to be accurate recommendations for delivering improvements. Bot creation and robustness improved. Bots started going live with fewer defects

APPROACH & LEARNINGS (CHATBOT) (2)

Contents

APPROACH TAKEN
RESEARCH PLANNING
SWIM-LANE WORKSHOPS
Workshop examples
STRATEGY PROPOSAL
UI IMPROVEMENTS
Example
OUTCOME
WHAT DID I LEARN

APPROACH TAKEN

Conversocial structured design projects using the double diamond framework

To be able to try quick changes and understand their impact I chose to condense the initial discovery phase and run further discovery phases throughout:

Discover Research planning Evaluating what we already knew and what further insight was needed for our goals

Discover Swim-lane Workshops Identifying the processes for making bots & learning what the people involved thought & experienced

Define Strategy Proposal 3 streams for tactical and strategy deliveries

Develop UI Improvements Usability improvements that would be quick to deliver & measure

RESEARCH PLANNING

Evaluating what we already knew and what further insight was needed for our goals

We had two sets of related interviews which summarised how people felt using the bot platform and we saw 5 patterns:

Using the platform wasn't felt to be causing them delays
Getting copy & bot flows from clients was time-consuming and challenging
UI usability was reported to be difficult in places
Different types of chatbots have different complexity that wasn't inherently understood or planned for
Re-use between similar chatbots wasn't happening despite indications it should be

💡

However, these interviews didn't uncover the actual process people used for selling, making & maintaining bots. Usability frustrations were aired but without specifics being recorded.

Additional research to investigate these gaps in our evidence were required to achieve our goals:

Workshops to map process & highlight challenges
Interviews with sales & support colleagues
Ad-hoc interviews during an off-site with the Bot product team (product, design & developers)
A retrospective into the bot that was deployed in a broken state

SWIM-LANE WORKSHOPS

Identifying the processes for making bots & learning what the people involved thought & experienced

The workshops mapped process steps and their Actors. Pain points & opportunities were marked by the participants

Workshops

I coordinated a set of workshops to identify the processes for selling, creating & maintaining bots:

Two in-person workshops (EU) with the most experienced sellers and builders.

These took advantage of an 'off-site' being run for the acquired bot team

Four distributed workshops (UK & USA) bringing insight from the wider company

Including people selling bots
Less-experienced bot builders
The support team who supported bots after the 'build phase'
I facilitated these using Miro (online whiteboard) & video conferencing

Workshop examples

In-Person whiteboard swimlanes

The first board that begun our understanding of how bots were sold. Post-it comments come from the people making & selling bots

Red post-its indicate pain points our colleagues identified they where experiencing. Yellow indicate their thoughts for improvement opportunities

Blue indicates additional information for that step

Distributed swimplanes

Using Miro so each person could add to the flows

Capturing part of the sales & contract processes

Purple markers indicate missing detail that needed information from other participants

Example where requirements gathering steps would vary depending on the client or people involved in that bot

STRATEGY PROPOSAL

3 streams for tactical and strategy deliveries

Findings from the workshops & our compressed timelines lead me to propose 3 streams for our next steps

Discover: Complete the process mapping and conduct further analysis
Develop: Tactical changes to try within 6 months

Create a trained 'Bot squad' team that would specialise in bot building
Enable 'self-service' learning for bot builders with fresh platform guides
Check-list for making & deploying bots to introduce some quick standardisation to testing
Prioritise UI improvements to key areas of the platform identified as causing confusion and increases to defects

Deliver: With insight from the other two streams. A strategic stream to implement wider changes that would reduce the inconsistencies in bot building and provide Conversoical with the capability to build bots more reliably.

3 streams proposal with delivery and high level time lines

UI IMPROVEMENTS

Usability improvements that would be quick to deliver & measure

Conversocial was running a strategic project to add the Apple Business Chat (ABC) channel to its products, and I used this project to make UI improvements to ABC List Picker.

It was an opportunity to

Fix the identified usability issues
Showcase guerilla research to validate the UI changes before development
Introduce a more flexible & robust interaction model for complex channel components:
Feed the strategic streams with feedback from the changes

Example

ABC List Picker

The end-user representative UI helped bot builders comprehend the nuances of a list picker

Untitled Document

sskh8h.axshare.com

Example Axure prototype & specification shared with stakeholders and developers

OUTCOME

⛔

As we began the tactical work, Conversoical went through management changes pivoting the company objectives. This stopped the project and triggered a replan

Through a combination of handovers and sprint work the tactical fixes were delivered:

The 'Bot squad' team was created and trained
Product documentation was completed
Small UI improvements had been made

The strategic streams were deprioritised in favour of a new product

WHAT DID I LEARN

Running workshops in a distributed manner is surprisingly insightful

Miro enabled all participants to draw onto the digital 'whiteboard'. We were able to continue the collaborative approach from the in-person workshops
We were able to spread the workshops to fit around people's schedules and timezones. By running smaller sessions I witnessed more engaged participants
Conversocial is a distributed company. This helped the transition to run workshops remotely as colleagues were already experienced in discussing detailed topics over a webcam or screen share

Incomplete Research Analysis can still be useful insight

The workshops provided a lot of detail and whilst we never completed the analysis, what was done did help other projects
I was able to find small opportunities to increase bot robustness or the platform usability based on the findings
The tactical fixes turned out to be accurate recommendations for delivering improvements. Bot creation and robustness improved. Bots started going live with fewer defects