Zapier makes processing and moving data between different tools extremely easy.
Unfortunately, just like any other tool, it has its quirks. For example, it will sometimes return said data in a different format than we’d like it to.

A common problem when working with Zapier data is duplicates.

In this article, you’ll see how to easily remove Zapier data duplicates with Python.

Thankfully, sometimes, all you need to solve a Zap problem are just a few lines of code.

The Problem – Zapier Duplicate Data

As an example, I’ll use a Zap I set that adds blog posts I read from my ClickUp to a Google Spreadsheet.

Whenever I set this task to Closed, it triggers a Zap which then adds the blog post to a list in a Google sheet.

That way, I can keep track of the blog posts I read, and when I read them. It also allows me to easily get back to them whenever I need to look up an idea or reference them.

The entire workflow is made up of just 4 steps:

Setting Up Zapier Automation flow

In the first step, the Zap looks for a task getting closed in a specific list.

In the second step, I use Text Formatter to extract the URL from the ClickUp task. Basically, I have a special link-type field in ClickUp where I add blog posts I intend to read:

Example of a link custom field in ClickUp

Unfortunately, out of the box, Text Formatter by Zapier outputs duplicated URLs:

But, that’s where the “Run Python in Code by Zapier” step comes in.

Using Python Code to Remove Zapier Duplicate Data

First, you need to add the right Step into your Zapier workflow – “Run Python Code”:

Setting Up the "Run Python in Code by Zapier" step

Then, click “Set up action” and enter the code. As data, pick the output from the Zapier text formatter. Remember to use ‘name’ for input data when setting up the action (or to edit the code):

Adding Python code to Zapier

Here’s the code if you want to copy-paste it:

unique_value = list(set((input_data['name']).split(',')))

output = [{'id': 123, 'value': unique_value}]

The code takes input data and splits it into a list. It then turns that list into a set (which cannot have duplicate values).

Lastly, it turns it once again into a list – this time made up of a single element. The reason we need this last step is that you can’t use a set in Zapier output.

And the output – a single link which you can then use in other steps:

And that’s it! The beauty of this is that it will work even if you have more than one pair of duplicate data.

Of course, don’t forget to check out other blog posts. And, if you need a Zapier workflow setup – schedule a quick discovery call and let’s talk about how automation can help your business save time and money.

Jacek Piotrowski
Jacek Piotrowski

Hey, I’m Jacek. I’m the founder and Chief Automator at

I’m on a mission to help you use automation to reclaim your time and achieve more in your business.

You can find out more about me – and why I started Clickleo – over on this page


Share on activity feed

Powered by WP LinkPress

Pin It on Pinterest

Share This