Predicting Content Success with Python

You can see in the SERPs a news report of November 3 mentioning lower donations in B.C.

It makes sense that searches would spike to learn more about it.

Do you see how this technique could help discover new content opportunities?

Really powerful stuff!

Semantically Matching Queries

So, we have a number of new queries and corresponding content ideas.

But, we don’t have historical performance to gauge each opportunity.

As these queries are potentially never seen before, they are unlikely to show up in keyword tools.

How do we know which ones to prioritize?

Here is one clever technique we can try.

We are going to borrow an idea from this patent analysis post from sensei Bill Slawski.

The patents he discusses talk about canonical queries.

Canonical queries imply there are duplicate ways to write the same search.

While the patents focus on syntactical duplication, we can expand the concept and consider semantic duplication as well.

Queries that mean the same, but are expressed using different words.

Now, let’s translate this idea into Python code.

First, let’s install this handy library.

!pip install sentence-transformers

We can build a model with a couple of lines of code.

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer('distilbert-base-nli-stsb-mean-tokens')

Once we have a model, we will find semantically similar queries by following the following process:

• Turn the queries/sentences into embeddings (numbers that simplify proximity calculations)
• Calculate proximity scores between pairs of queries.
• Filter the closest matches based on a cutoff threshold we can determine empirically.

The highest score is 1.

Here is the code to do this with our queries.

First, we create the embeddings.

fifteen_percent_list = list(fifteen_percent)

#Compute embedding for both lists

embeddings1 = model.encode(fifteen_percent_list, convert_to_tensor=True)

# try on a smaller set of 10k, as it takes too long to run on full set of queries

rest_of_queries_list = list(set(rest_of_queries))[:10000]

embeddings2 = model.encode( rest_of_queries_list, convert_to_tensor=True)

Next, we want to find semantically similar queries in the historical list that closely match the new unknown queries.

#Compute cosine similarities

cosine_scores = util.pytorch_cos_sim(embeddings1, embeddings2)

Finally, here is the code to filter the similar queries.

#Output the pairs with their score

for i in range(len(fifteen_percent_list)):

  score = cosine_scores[i][i]

  if score > 0.7:

    print(f"{i}. {fifteen_percent_list[i]} <> {rest_of_queries_list[i]} \nScore: {score:.4f}")

I was only able to check 10,000 queries and there were no matches above 0.5.

I will try later with the full dataset and report the outcome on Twitter.

Please feel free to report what you get in your experiments.

Prioritizing the Most Promising Ideas

When I visually inspected the unknown queries, I found a couple of interesting patterns worth investigating further.

One about fundraising ideas and another about [no-profit] near me.

Let’s use a data-driven approach to evaluate the potential impact of each one with visualization and forecasting.

First, I’m going to create two filtered data frames where the query contains the pattern keyword.

ideas_df = df[df['query'].str.contains("idea")]

near_me_df = df[df['query'].str.contains("near me")]

We are going to set the date column as the index of the data frame.

ideas_df = ideas_df.set_index("date")

near_me_df = near_me_df.set_index("date")

We also use D, which stands for calendar day frequency.

You can find the full list time series frequency aliases here.

Let’s increase the frequency to days.

Here is what the plots look like with daily data.

You can see the model is trying to fit as many points as it can in the pattern, but it fails to fit the big outlier around May.

This year many things have been unpredictable, so no surprise here.

In practice, the predictions dramatically improve when you have a dataset of at least a couple of years.

The final step is forecasting the traffic of several candidate ideas, sum it up and rank ideas based on the best traffic potential.