Facebook | Campaign Recommendations
Designed with Product Marketing, Content Strategy, UX Research and Engineering Functional PartnersKey takeaways
Project Breakdown
Project Type:
Framework Development + Experience Design
Contributions + Role: Research, Workstream Prioritization, Solution Discovery, Concept and Design Development
Redesigned recommendation cards in Account Overview featuring a card that was previously snoozed, and 2 unread recommendations
Summary of impact
Improved advertiser adoption of recommendations
Enabled partner teams are developing their own recommendations due to establishing a framework
Improved internal understanding of advertiser behavior in context to campaign recommendations
Established foundation for improving recommendation ranking system, and level of relevance for advertisers
Context
Ad Manager is the starting point for running ad campaigns on Facebook properties. It is where ads are managed, created, and where advertisers go to understand how their campaigns are performing against goals.
Mid-Flight Recommendations are a way for advertisers to increase their ROI, and be empowered to make better performance decisions as their campaigns are running. It serves to notify advertisers when our system detects these opportunities. In the current state, use cases for Mid-flight recommendations include;
When campaigns are competing with each other
When there isn’t enough budget for the ad to be delivered enough for the Machine Learning (ML) to learn which best cohort of people it should deliver ads
When ad creative is exhausted and no longer attracting the attention of consumers
When a audience-related input is narrowing how the campaign can perform (e.g. the geography is limited to customers in San Francisco, but in actuality the business can serve everyone in the state of California, etc)
Digram highlighting where the area of focus is relative to similar guidance types
Problems
For advertisers, the relevance or of recommendations vary
Adoption rates are indicative of a lack of trust, largely due to the fact the recommendations aren’t personalized enough to frame how the recommendation could contribute to advertiser goals
Advertisers aren’t seeing recommendations at points they could readily adopt
Advertisers learn to ignore recommendations when they aren’t applicable, or aren’t ready for them. When advertisers see recommendations they are on the platform to perform an unrelated task.
Internally, scaling the recommendation inventory is hard
To include new cases is inefficient due to the lack of a cohesive design and content framework. Internal approvals on details per Recommendation use case was a bottleneck.
Goals
Increase adoption rates of recommendations
Build advertiser trust and improve comprehension of mid-flight recommendations cases
Develop framework to reduce overhead for teams and unblock our internal ability to introduce new recommendations faster
Capture feedback in-product from advertisers to improve recommendation relevance
Simplify workflows of current recommendations where possible
Process + Execution
First, start with an audit
Collect all current recommendation inventory, and prospective recommendation use cases to build a foundational understanding of their properties, and structure. We need to know;
If there are any similarities in structure that could act as a pattern for a UI framework
Where advertisers go to look to validate whether or not the recommendation is right for them to adopt
What actions are being asked of advertisers to adopt a given recommendation
If there are any opportunities to simplify or streamline against all current cases, and establish flexible guidance for future recommendations
Audit of current and prospective recommendation use cases
Research
Understand what keeps advertisers from adopting recommendations
Understand how advertisers decide to make changes to currently running campaigns
Understand what advertisers need to see in order to trust a given recommendation
Understand what makes one recommendation more relevant to them versus another
Concept study version of recommendation cards designed to trigger feedback from advertisers to understand what upfront knowledge they needed to adopt, content, clarity, expectations on system feedback, and required actions to adopt.
Research synthesis. Mapping of necessary content, context, and components necessary for advertisers to consider adoption of a recommendation case
Visual framework developed with Interfaces Platform team to get more specific in component usage, or future component needs across different recommendation workflow types
Updated awareness recommendation card anatomy as informed by research and framework
Fragmentation use case before framework applied
Audience Saturation use case with framework applied
ranking + Feedback
Context
In a workstream developing in tandem with Mid-flight recommendations I was trying to solve for advertiser relevance across all recommendation types, first starting with Mid-flight that looked further into the future. In Mid-flight recommendation research we heard many times that advertisers often were not ready to adopt a recommendation when the first saw it, and often forgot about it hours or days later when they would return to Facebook Ads Manager. Also, we found that in some cases though the system accurately detected an opportunity for campaign improvement, some advertisers strategies or real-world conditions made certain use cases irrelevant.
Design concept study version of recommendation card feedback
Dimensions to solve for
Understand what signals we can passively detect as a “feedback signal” coming from advertisers (e.g. adopting a recommendation without other inputs, etc”
Understand how we need to prioritize recommendations in the case an advertiser is eligible for more than 1 at a time
Understand what expression types best map to advertiser mental models and needs
Get clear signal on advertiser expectations on what feedback options do in the moment, and what the impacts are in the future (e.g. how does the system internalize their positive/negative feedback inputs?)
Develop framework that considers when a recommendation should reset or return after an initial dismissal
Worksheet to help cross-functional team to align on expression types, interface behaviors, logging strategy, and expression influence on ranking system
Instance of a recommendation that has retuned after an initial snooze/dismissal where the conditions had changed
Simplified framework to highlight expression types to UI Behaviors
Prototype Illustrating Interactions and feedback options
Impact
Framework + Workflow Improvements
Improved advertiser adoption of recommendations
Enabled partner teams are developing their own recommendations due to establishing a framework
Improved internal understanding of advertiser behavior in context to campaign recommendations
Established foundation for improving recommendation ranking system, and level of relevance for advertisers
Feedback Expressions
Launched in 2021
Improved what can be understood from a logging perspective (data science)
Laid groundwork for engineering-centric work stream focused on improving recommendation ranking across delivery system, and partner teams