Richer Insights with AI Follow-ups

Enterprise · RESPONSIVE WEB · AI

Summary

As the Lead Product Designer in the Engagement product I led the discovery and redesign of Culture Amp’s core survey design experience.

​

This project modernised a legacy left mainly untouched for ~6 years and laid the foundation for AI-powered survey tools.

​

Timeline: ~6 months (discovery → validation → delivery/release)

How do we collect deeper insights?

Surveys were effective at collecting structured data, but often lacked depth. Admins could ask rating, multiple choice, or free text questions but they previously only had one opportunity to collect further commentary from participants in an “Add Comment” feature which led to:

  • Generally limited context behind responses, which required post-survey follow up
  • Missed insights, especially for nuanced sentiment
  • A tendency to make surveys longer/with more questions to compensate

Insights without increasing survey fatigue

While we wanted to collect these deeper responses, it was also about doing it in a way that didn’t slow people down or make surveys feel heavier as this could potentially decrease participation which is a key metric in survey success.

​

One of the first decisions was to scope AI follow-ups at the question level, rather than across an entire survey.

​

This allowed admins to:

  • Apply follow-ups only to high-value questions
  • Avoid overusing the feature
  • Reduce the risk of participant fatigue

Working within constraints

In parallel we redesigned the core survey design experience. We landed on a multi-panel layout with:

  • Clear separation between editing, settings, and preview
  • Less of a “everything in one” UI
  • More modular, easier to add/scale with new features

​

To validate the new experience I ran unmoderated Maze testing with 20 admins across different segments and regions with the goal to:

  • Test existing workflows and learnability
  • Assess against usability metrics like SUS (>80) and SEQ (>5.5)

​

Feedback came back positive, users liked being able to edit and preview at the same time. The structure felt familiar even though there was some learning curve there was expectation it would eventually be faster overall.

Prototyping the AI experience

To properly test the concept, I built a working prototype using Claude that could simulate real LLM-generated follow-ups.

I worked closely with People Scientists to define:

    • How prompts should be structured
    • When follow-ups should stop
    • How to avoid sensitive or inappropriate responses
    • How to ensure questions aligned with best practice

This allowed us to test not just usability, but whether AI follow-ups genuinely improved the quality of responses.

Validating with users

We tested the experience through a mix of Maze testing and 1:1 research with admins across different segments.

​

One key insight was around drop-off as ~50% of participants responded to the first follow-up, but engagement dropped off to ~30% for each subsequent follow up. This helped inform our follow-up setting maximums which also helped with keeping the surveys from feeling too heavy with follow ups.

An illustrative sketch of a flower

Impact and Future

The feature helped close the gap with competitors and strengthened Culture Amp’s positioning in AI-powered survey capabilities.

​

Customers reported significantly more comments than in standard surveys, providing richer qualitative data for leaders and HR teams.

​

Looking ahead, I would explore:

  • More adaptive logic for when follow-ups should trigger
  • Better continuity across multiple questions in a survey
  • Stronger measurement of long-term impact on insight quality

Want to learn more?

CV/Resume

Linkedin

Email

Richer Insights with AI Follow-ups

Enterprise · RESPONSIVE WEB · AI

Summary

As Lead Product Designer I led the discovery and design for AI-generated follow-up questions in Culture Amp’s Survey product that helped organisations capture richer, more actionable insights.

​

Our competitors were beginning to position themselves around AI-powered survey capabilities, and there was growing pressure from the business and customers to leverage our expertise in employee engagement to bolster capabilities in our existing survey designer.

How do we collect deeper insights?

Surveys were effective at collecting structured data, but often lacked depth. Admins could ask rating, multiple choice, or free text questions but they previously only had one opportunity to collect further commentary from participants in an “Add Comment” feature which led to:

  • Generally limited context behind responses, which required post-survey follow up
  • Missed insights, especially for nuanced sentiment
  • A tendency to make surveys longer/with more questions to compensate

Insights without increasing survey fatigue

While we wanted to collect these deeper responses, it was also about doing it in a way that didn’t slow people down or make surveys feel heavier as this could potentially decrease participation which is a key metric in survey success.

​

One of the first decisions was to scope AI follow-ups at the question level, rather than across an entire survey.

​

This allowed admins to:

  • Apply follow-ups only to high-value questions
  • Avoid overusing the feature
  • Reduce the risk of participant fatigue

Working within constraints

In parallel we redesigned the core survey design experience. We landed on a multi-panel layout with:

  • Clear separation between editing, settings, and preview
  • Less of a “everything in one” UI
  • More modular, easier to add/scale with new features

​

To validate the new experience I ran unmoderated Maze testing with 20 admins across different segments and regions with the goal to:

  • Test existing workflows and learnability
  • Assess against usability metrics like SUS (>80) and SEQ (>5.5)

​

Feedback came back positive, users liked being able to edit and preview at the same time. The structure felt familiar even though there was some learning curve there was expectation it would eventually be faster overall.

Prototyping the AI experience

To properly test the concept, I built a working prototype using Claude that could simulate real LLM-generated follow-ups.

I worked closely with People Scientists to define:

    • How prompts should be structured
    • When follow-ups should stop
    • How to avoid sensitive or inappropriate responses
    • How to ensure questions aligned with best practice

This allowed us to test not just usability, but whether AI follow-ups genuinely improved the quality of responses.

Validating with users

We tested the experience through a mix of Maze testing and 1:1 research with admins across different segments.

​

One key insight was around drop-off as ~50% of participants responded to the first follow-up, but engagement dropped off to ~30% for each subsequent follow up. This helped inform our follow-up setting maximums which also helped with keeping the surveys from feeling too heavy with follow ups.

An illustrative sketch of a flower

Impact and Future

The feature helped close the gap with competitors and strengthened Culture Amp’s positioning in AI-powered survey capabilities.

​

Customers reported significantly more comments than in standard surveys, providing richer qualitative data for leaders and HR teams.

​

Looking ahead, I would explore:

  • More adaptive logic for when follow-ups should trigger
  • Better continuity across multiple questions in a survey
  • Stronger measurement of long-term impact on insight quality

Want to learn more?

CV/Resume

Linkedin

Email

Richer Insights with AI Follow-ups

Enterprise · RESPONSIVE WEB · AI

Summary

As Lead Product Designer I led the discovery and design for AI-generated follow-up questions in Culture Amp’s Survey product that helped organisations capture richer, more actionable insights.

​

Our competitors were beginning to position themselves around AI-powered survey capabilities, and there was growing pressure from the business and customers to leverage our expertise in employee engagement to bolster capabilities in our existing survey designer.

How do we collect deeper insights?

Surveys were effective at collecting structured data, but often lacked depth. Admins could ask rating, multiple choice, or free text questions but they previously only had one opportunity to collect further commentary from participants in an “Add Comment” feature which led to:

  • Generally limited context behind responses, which required post-survey follow up
  • Missed insights, especially for nuanced sentiment
  • A tendency to make surveys longer/with more questions to compensate

Insights without increasing survey fatigue

While we wanted to collect these deeper responses, it was also about doing it in a way that didn’t slow people down or make surveys feel heavier as this could potentially decrease participation which is a key metric in survey success.

​

One of the first decisions was to scope AI follow-ups at the question level, rather than across an entire survey.

​

This allowed admins to:

  • Apply follow-ups only to high-value questions
  • Avoid overusing the feature
  • Reduce the risk of participant fatigue

Working within constraints

Ideally, this experience would have been paired with a redesigned survey-taking flow (e.g. one question at a time). However, a full rebuild was not in scope.

​

So instead, we:

  • Integrated follow-ups into the existing survey experience
  • Ensured they felt lightweight and non-disruptive
  • Prioritised clarity over introducing new interaction models

Prototyping the AI experience

To properly test the concept, I built a working prototype using Claude that could simulate real LLM-generated follow-ups.

I worked closely with People Scientists to define:

    • How prompts should be structured
    • When follow-ups should stop
    • How to avoid sensitive or inappropriate responses
    • How to ensure questions aligned with best practice

This allowed us to test not just usability, but whether AI follow-ups genuinely improved the quality of responses.

Validating with users

We tested the experience through a mix of Maze testing and 1:1 research with admins across different segments.

​

One key insight was around drop-off as ~50% of participants responded to the first follow-up, but engagement dropped off to ~30% for each subsequent follow up. This helped inform our follow-up setting maximums which also helped with keeping the surveys from feeling too heavy with follow ups.

An illustrative sketch of a flower

Impact and Future

The feature helped close the gap with competitors and strengthened Culture Amp’s positioning in AI-powered survey capabilities.

​

Customers reported significantly more comments than in standard surveys, providing richer qualitative data for leaders and HR teams.

​

Looking ahead, I would explore:

  • More adaptive logic for when follow-ups should trigger
  • Better continuity across multiple questions in a survey
  • Stronger measurement of long-term impact on insight quality