Thousands of partner responses — underutilized

Agoda runs a Partner Satisfaction Survey (PSAT) every quarter, completed by roughly 12,000 properties — around 4,000 of whom leave open-text responses. It is a rich, direct signal from accommodation partners about what was and wasn't working. But we had no scalable way to act on it.

To address this, we set out to:

Three steps to standardize, automate, and visualize

We designed a three-step solution in collaboration across Product Design, UX Research, and Data Analytics.

We started by looking at previous partner pain-points to identify the most common issues, then grouped them into pre-defined categories — such as Extranet, Payment, Support Quality, and so on. Within each category, we broke them down into specific tags that mapped to features or areas partners interact with.

For each tag, we applied a three-layer definition system to make sure the AI could tag accurately and consistently:

I collaborated with Supply, Partner Experience, and Product teams to make sure the definitions made sense across the board — resulting in 53 pain-point tags across 7 categories as a shared language for talking about partner pain-points.

With the standardized tags and definitions in place, we needed a way to automatically tag thousands of partner responses. Our data analyst trained the AI model using the three-layer definitions. Responses were auto-translated where needed, then tagged with the category, the tag chosen, and the reason behind it.

I then reviewed the output to check tagging accuracy. If something was off, we went back to revise the definitions and ran the model again. After a few rounds of this, accuracy reached around 95%. From there, a data pipeline was built so that when new PSAT responses come in each quarter, the AI reads each open-text response and tags it automatically — consistently, without manual effort.

With automated tagging in place, we needed a way for everyone to access and explore this data. I designed the PSAT Dashboard on Tableau, which has three main views:

From a two-week manual exercise to an always-on system

The dashboard made it easy for any team to look at partner pain-points longitudinally — tracking which issues had improved, which had worsened, and what was driving the changes. For example, 2025 data showed a notable drop in complaints around promotions and performance (linked directly to Supply Discount scrum work and page load improvements across Property Portal scrums), while finance-related complaints rose to become the top issue at 16%.

Teams across the organization now reference this data regularly — from product discovery, prioritization, and roadmap planning to end-of-quarter reviews.

Anyone can filter by a specific tag — say, "Guest Messages" — to see what percentage of total partner complaints it accounts for, then download the verbatims and run their own GPT analysis to build hypotheses. This is now a standard step in the discovery phase for several product teams, replacing or supplementing what used to require a dedicated UX Research engagement.

• Example prompt used by teams:
  "Analyse these verbatims and tell me the top 5 categories of user pain-points, what each means, the % for each, and 2–3 verbatim examples per category."
• From the Guest Messages analysis:
  34% of complaints were about messaging restrictions (links, phone numbers, and PDFs blocked), and 26% were about delivery and notification failures.

Teams running A/B experiments can now cross-reference PSAT data to validate whether a design change affected partner sentiment — not just the primary metric. In one example, an experiment on deactivating promotions showed that the B-side (with the new deactivation flow) had fewer promo-related complaints (2% vs 4%) and a higher PSAT score (3.41 vs 3.07), directionally supporting the design decision.

The sample size is limited to survey respondents during the experiment window, so results are used directionally — but this adds a qualitative signal layer that wasn't previously available at all.

The system keeps growing

This project was different from most of my product design work. I wasn't just solving a specific partner-facing UX problem — I was building internal infrastructure that made it possible for others to solve partner problems faster and with better evidence. What started as a Supply Design initiative is now a shared system used across the organization.

With the dashboard now used across teams, the next step is expanding the taxonomy further — more granular tagging within existing categories, broader coverage across new ones, and exploring how to identify and surface unknown pain-points that fall outside the current taxonomy.

Rather than expecting stakeholders to visit the dashboard, we're exploring whether there's a better way to bring insights to them. With more AI capabilities, one direction is proactively sending teams an AI-generated summary of the pain-points most relevant to their product area — delivered directly, without needing to look it up.

AI across design projects

Two product milestones, one Analytics Center

Agoda Partner Portal's Analytics Center scope split into two revamps of existing pages that kicked off in parallel — Market Demand reporting and Margin Program performance reporting — each with a different PM, scrum, and weekly sync cadence. I was the only designer across both.

The groundwork started months before Market Demand and Margin Program became Q3 2026 product milestones — I'd already run UX audits on the current designs, helped with user research, conducted competitor analyses, and sat in on early discussions with product, market managers, business, and other designers about what to improve. My old process would've been gathering all of that manually into a Figma moodboard. This time I wanted AI in from the start: I set up a single Cursor project so everything could live in one place, then used MCP to pull in what I already had — competitor analysis, user research, market manager insights — and distilled it into a single markdown context file, so the rest of the process — starting with design exploration — stemmed from one consistent source of truth.

Market demand

Before

Partner need

Spot uncaptured demand & pricing opportunity

A few pain points

• Lacks forward-looking demand data
• No visibility into demand being missed
• No clear next action

Margin program performance

Before

Partner need

Understand program performance & bookings across Agoda's flagship partner programs — AGP, Boost Rank, Sponsored Listing, ADR+

A few pain points

• Raw counts only — impressions, clicks, bookings, revenue
• No before/after comparison
• ROAS calculated incorrectly

With context in place, I connected Agoda's design system and Recharts inside Cursor and built a shared skeleton first, then split into sandboxes within the same prototype — one per product milestone. Each sandbox became a playground for exploration: I'd prompt AI for ideas, then guide and refine with my own take until one held up. Once a sandbox looked right, I asked AI to piece the components together into the working prototype.

I published the prototype to GitLab Pages so PM, business, and BI could try it out themselves. Syncs then focused on which directions were worth building for partners, and what BI could support. After each sync, an AI skill I built converted the transcript into a same-day action-item list and rolled it into one Confluence doc organized by partner jobs-to-be-done — decided, pending design, pending BI — so I could jump straight into the next iteration. It took a few rounds to reach agreement.

Both prototypes are close to final — the last steps are locking the remaining UI details and preparing the engineering handoff.

Market demand

Program performance

On working with AI this way

This project is a small slice of what AI can do for design — a sandbox, a decision log, a working prototype. AI's role in my process is only going to grow from here, and I'm excited to keep figuring out what that looks like as the tools improve.