Pinterest Head of Growth Marketing Lisa Sullivan-Cross | Amplify 2019

Please welcome Pinterest's Head of Growth Marketing, Lisa Sullivan-Cross Hi there

I'm Lisa Sullivan-Cross, as you heard I'm the head of growth marketing at Pinterest, where over 300 million people come on a monthly basis to find the inspiration to create a life they love So prior to Pinterest I founded the growth discipline at Pandora where over about a four and a half year period my team and I built the infrastructure to track, measure and run paid performance campaigns and paid growth campaigns in addition to contributing to growing the business and the user base All right So growth marketing, big buzz word, ton of definitions out there, just wanted to share with you how I define growth marketing for my teams and what works for us to find success in this area So I define it as a cross-functional, data-driven focus on growing users or customers and of course revenue, which is what really matters in the long run

It's a technical analysis of every aspect of a user's journey, and quickly implementing and testing insights to achieve sustainable growth So it's really important to keep that in mind on a daily basis within the growth org And there are six principles that I've learned over time really make a growth org succeed So one is data-driven decisions And there seemed to always be other groups or teams who want to launch things quickly within the product or within the paid marketing program or owned marketing programs, email, mobile, push to achieve quick short-term revenue goals

If revenue number needs to be hit for the month and it's the day before month end, but I found it's really important to push back on those, and to make sure that things are tested, and you can test things so quickly in regards to digital products and in performance marketing before anything's rolled out Creative solutions is another one I know a pretty simple creative solution would be an example from Pandora I guess when we needed to increase our mobile push opt-in rate because push was a really strong driver of growth for us And so as most of you probably know when you serve a push notification prompt from Apple's iOS system if that user says no and declines it you can never serve them another one in their lifetime, cross-device And so what we did is we served our own pop-up before we served the iOS pop up that talked about the benefits of opting in to push, why should somebody receive a push notification from Pandora, and then if they said, "yes," we'd serve them the iOS prompt

If they said, "no," we wouldn't, and then maybe six months, a year from now we might have a more compelling benefit, and we can contact that user again So that was a creative solution to being able to increase our opt-ins Product focus is another Product and the quality of the product is the most important thing That is the rising tide that will raise all boats

And so that always has to be first and foremost, having a good product, whether it be digital or physical product Iterative approach, failing fast, testing and learning and then putting the customer or user first, and this is a tough one And this we have to remind teams constantly to put the user first or put the customer first because we could do things at Pinterest, we could send a ton more mobile push notifications or more emails and likely would not see any immediate harm done there, like any decrease in opt-out rates or anything like that or decrease in response rates, but what it will do is annoy the customer and long-term that's gonna cause a negative association with the brand And so we're really careful to balance that and really with any product or marketing decision So those those are some really important elements

So I'm just gonna show a quick video It's a B2B ad that we're running It'll just give you a sense for what B2B is at Pinterest, and then I'll talk through that a little more So the case study I'm going to share with you today is around building a B2B performance growth marketing program from the ground up at Pinterest, and this – I joined the company in January – and this is when we started building it Everything I'm going to share with you today however can also be applied to the consumer side, and has been for me at companies such as Pandora, Art

com and Pinterest so just keep that in mind, even though it's focused on B2B So it can be hard to make B2B ads interesting, so we really strive to do that We strive to infuse our overall brand into our B2B ads, and it takes a lot of extra testing to be able to do that, because sometimes you can over-rotate and make things pretty or sexy, and they won't perform well So it's just – I challenge my teams to try to work on making performance ads more, bringing as many brand elements as you can into those ads without hurting performance So this is an insight that I gained early on in building growth teams that really helps guide what we do on the team

And so it's that a truly cross-functional growth org with shared goals translates into optimal business growth And I'll talk a little more about that later but having those shared goals and having all of the cross-functional teams have skin in the game essentially it's what really makes us work So what does b2b marketing at Pinterest? So what we're doing is we are targeting small to medium sized businesses and trying to get them to come and use Pinterest's self-serve promoted pin tool That's what we're doing with our paid performance marketing program that I'm going to talk about today The cross-functional team – this is not just for the B2B, but this is also our cross-functional team for the consumer side at Pinterest, so product, engineering, marketing, analytics and data science, and those are equally shared as far as goals and projects and and workload

Then our KPIs for the B2B piece, our cost per new spender – or new advertiser, months to break even, so that's essentially an ROI breakeven number, and it varies by stage the company is in, whether you want to be at a 12-month break-even or or longer – if you're in a high-end and investing in growth mode And then the third sort of master metric is our three-year LTV over a CAC and that number is – builds a little bit more in than just a row as number because you're looking at retention over a three-year period and then it pulls the acquisition cost into that Alright, so where do we start? Before we spent anything we built that cross-functional team out as I mentioned but this was it at every every company that I've been doing growth marketing at the past ten years It started with an LTV analysis so this is both on the consumer side and the business side So on the business side we looked at what is the LTV of all of our advertisers and then we did one click down

What is the LTV of advertisers through our self-serve tool? And six months after our paid performance marketing was up and running, we now look at what is the the LTV of paid market specific cohorts But those first two I mentioned – getting a read on LTV in the beginning – really helped us set our acquisition cost and what we could pay for a new user, because we whittled that LTV down to a one-year customer value And then that as I mentioned set our our CPA and our LTV over CAC and ROI breakeven goals So how have we scaled that while still hitting our KPIs? Because that's that's the tough part and I'll talk about the results at the end, but these are five of the key things we did in that first six months to be able to scale that and hit those tough KPIs So we had an experiment framework that we built on the consumer side, and we tailored that to the B2B side so that we could use that, and one of the most important things we do with that is determine incrementality

So on the consumer side we look at cost per incremental Pinner or incremental user, and on the on the B2B side we're looking at incremental advertisers, incremental business accounts And then we also use publisher systems So for facebook for example the conversion lift test, but in order to validate what they're telling us, we run with retargeting an audiences and incrementality test through our framework and then we can use that multiplier for acquisition tests And not surprisingly the numbers do differ from a Facebook or Google and what they're reporting So I think it's really important to know for your business how do they differ and then really applying that multiplier to your numbers

The next one is identifying leading indicators to your master KPIs So for us its first spender or a billing conversion and so identifying those really allows you to optimize in the moment, quickly, real time and then you're still working to achieve those high level KPIs because you know that they're indicators for them So building systems and automation: so automating APIs and things like that, I'll talk a little more about other things that we've automated, but the more you can automate the better because those systems learn over time and then you come in with new ideas, it frees people up on a team to think more about ideating new things to do in the area Lifecycle marketing mindset: so this this is really important Lifecycle marketing allows you to not be short-sighted and just think about getting the user, our customer, in the door

But what do you do to keep them coming back? And then if they fall off, what do you do to resurrect them? And I can provide some specific examples later on that And then product UX landing page testing It's constantly testing there And so these are the three areas I'll go into I'll go into prospecting, retargeting and resurrection

So in the prospecting side we enabled automation, like I mentioned, of APIs We optimize bidding to high-value down funnel events So we actually early on at Pinterest like in the first, look – in the third month, I think, so within the first quarter we switched from optimizing on cost per business account signup to cost per billing complete, so somebody who completed the credit card and hit that landing page, and we saw a 25x increase in our performance or decrease in our CPA just from that one change And so we're continuing to work to get further and further down funnel on optimizing using first party data

So this could involve excluding your low value audiences or creating lookalike audiences So at Pinterest we have a lookalike audience on people who've spent more than a hundred dollars in the first month, and that works really well for us We actually had similar findings at Pandora on the consumer side, and creating those lookalike audiences, rapid creative testing, unlocking mobile is important on the B2B side and is challenging Still, we've come a long way there where now I think about 40% of our conversions are from mobile, but it started at zero because this is a business product and most people are on their desktops at work placing their ad buys on Pinterest However, they are on their phones all the time whether they're doing that while they're on their computer, at work, or at lunch or whatever it may be

So we have – and plus, 80 plus percent of Facebook's inventory for example, is mobile so we had to crack this nut and get it to work, and we have through a lot of working really closely with product and eng, and building out flows that increase conversion on mobile web and on mobile app And then diversifying channels: So started out with the usual suspects, the Facebooks and Googles, tested into programmatic DSPs, into native content ads We're actually testing direct mail right now, and queuing up a podcast test I think it's really important to test channel so you're not dependent on one channel, and to do that you really need to build out a multi-touch attribution system And then the second of the three areas of focus is retargeting

So how to breathe, essentially, life back into dead leads? People who maybe did start to spend and tapered off, or even people who interacted with your ad View through or click through converted and just didn't come back So setting tracking pixels on multiple pages so you can get a sense for again that lifecycle of a user So whether they come and visit the business site or they set up an account or they set up a promoted pin or whatever it may be So being able to track all of that activity

Automating the data exchanges: We already talked about Building heuristic models to retarget those most likely to spend is interesting We looked at business accounts because on Pinterest you can sign up for a free business account and have basically a presence on Pinterest for your business account that doesn't cost money, but then we want you to spend money to boost those pins and get people to see them And so we looked at what attributes of a business account are going to lead to somebody spending and then we can actually target on those and one of the surprising things we found is that it wasn't just business accounts, but when we looked at the heuristic model unlikelihood to spend of pinners, so consumers, we were able to find pockets and subsets of pinners with similar traits that go on to be advertisers And so now we're able to really open up our retargeting not just to business accounts but to our pinners and consumers as well because we know they will likely be interested in advertising

Another example of this is is that at Pandora we did a similar thing where we built a likelihood to subscribe model and we were able to determine what actions or behaviors do pinners take that make them more likely to be a subscriber so we can make sure that we're only serving ads to those people or within products we're only upselling to those people so that we're not basically pissing off the rest of the users who are never going to subscribe and annoying them And so I think this is a really valuable thing to do And then life- cycle marketing and product changes to increase retention So just since I just mentioned Pandora, another example there would be when we determined that we wanted to increase usage and we saw that some users were only using Pandora during commute hours We could actually target them say on Friday or Saturday night with another use case, "Hey did you think about using Pandora for your dinner party?" and we found that because we understood their behavior and could retarget them at a different point in their in their sort of day or life that that worked really well

And we're finding the same similar things on Pinterest on the consumer side in regards to use cases And then on the B2B side as well And then the last area of focus for us is resurrection So these are churned users and getting them to come back – or churned advertisers in this case So the first thing we did here, because we just started doing this, because again we just launched this B2B program in January, so we just started working on resurrection in the last couple months, and so we did some opportunity sizing to make sure there was a "there" there essentially

And then we identified the highest potential small, medium-sized businesses to retarget So are they businesses who've been gone for 14 days, 30 days, 90 days? Whatever it may be And again this applies to the consumer side absolutely as well, and then finding that sweet spot And that's really where you hone in and make sure that you're remarketing to those people This is a really important one: so understanding reasons for churn

So this is through a combination of looking at behavior patterns and qualitative surveys and being able to identify why people are leaving, and then really address those in your marketing So an example on the Pinterest business side is that we identified a cohort of users because through both their activity and surveys we determined this was one reason people we're leaving is that their ads were not performing They were were expecting them to perform like their performance ads on other platforms for example And so when we just launched our new 30-day view through attribution window on Pinterest so that advertisers could look at view through conversions and attach that to to revenue rather than just quick conversions, we were able to reach out to those users we identified as having those issues, and let them know that we had this new product, and that really brought them back And not only did it bring them back but the issue of why their ad didn't perform was actually addressed

And so it performed well when they did come back This was also used at Pandora on the consumer side We built a churn probability score for each user so that we knew at any point in time what was the probability of churn of this user based on the behaviors they were taking And then we match that with marketing so we could serve them a certain message at a certain time in a certain place if we knew that their churn probability had fallen to 30% And so we got to know what messages and what channels were best to decrease that churn probability

Communicating product improvements I kind of covered, and in my previous statement, and then incrementally measurement is really important on the resurrection side because there are a big group of users who will come back on their own, and also a group of users who will come back with non-paid marketing so mobile push or email and that sort of thing And so really understanding what with – so you don't waste your money until your money is spent efficiently, which users to target with that paid spend and which to not And then making sure to move those resurrected users who return back into the retention loop and back into the lifecycle All right so results This is just within the first quarter

So with all of these tactics that we applied, strategies tactically applied, we saw a 20% decrease in CPA within the first quarter and a 15x improvement in first spend rate And the exciting thing is we were able to maintain that efficiency as we scaled spend So now in Q3 we're spending four times more on a weekly basis than we were spending in Q1, and those KPIs are holding, which is pretty exciting for us So I mean I just put together a few trends that I see continuing and heightening in 2020, things that we're starting to leverage and see now, but I think that they're really going to grow next year

So one is this cross-functional growth org I talked about So more and more companies are forming growth orgs instead of having the teams live in their respective groups whether it be engineering, product ,marketing, and we found at Pinterest that just having that collaboration is so important, and you know it actually kind of forces marketing, for example, not to be so singular and looking at paid marketing spend or email and mobile push because the growth product and engs teams not only implement projects that help with our paid performance marketing like implementing pixels or creating APIs or landing page testing, but that same team because it's a holistic growth team is also working on in-product growth, growth hacking or changes to the products that make consumers have a better experience And it's not specific to the marketing piece, but it helps marketing because it increases retention and increases conversion and that makes our LTV go up, higher LTV of our CAC means you can spend more money on on the channels that you're on So extreme personalization is another one I know we're in a pretty advanced spot as an industry on being able to target people one-to-one with a specific message that's based on on what they like or don't like but we're seeing that at Pinterest on the consumer side

We've built out a mobile push and email machine learning automated platform that most of our email goes through so we don't do actually any batch and blast emails that a, say, marketing manager is putting together However, marketing, product do collaborate on if we have a new idea for an email we'll test that and throw that into the system and things like that But all that means is that machine learning is enabling us to deliver the most relevant message to each person really based on what we know that they're gonna like and based on their behavior Automation: So at Pinterest we've built our own automation tools so we have a creative optimization tool where we can throw hundreds of creatives into this tool and it will disseminate them across the different channels and based on performance And then we have a built a bidding tool, automated bidding tool for Facebook, so I think just automating these areas within in growth marketing is going to be continued to be heightened next year

And then lastly I think that sometimes that laser focus on data-driven marketing can be, it could be tempting to not think of the customer first, so I think that more and more customers are skeptical of data targeting and really going to demand that the customer be put first, and that could in many cases mean less specific data targeting and more making sure that we're using that data to better the customer experience and have a real value, build a real value for them So wraps it up Thank you so much

Source: Youtube