10 Oct 2024 for v2, see
Outreachee
Outreachee
26 Jul 2022
I have tried a few email automation & sequencing platforms over the last 10 years, starting with Toutapp (acquired since by Marketo) and up to Outreach in more recent days.
Though the products are good generally, I usually have precise ideas about what I want to achieve or the level of automation I expect.
Every time, I got frustrated - by product limitations, deliverability issues... and pricing.
Last year, as part of my "Sales-as-a-Service" engagements, I faced the need again to use an email sequencing solution.
Knowing now how to write automation scripts in Python, I decided to write my own email automation solution.
The idea was to have something that would fit my needs in terms of flexibility, including:
- same basic capabilities as off-the-shelf tools: email sequencing, etc..
- easy re-use & automations where possible: across both campaigns AND client engagements
- multi-lingual: send emails in English, French & German (my main working languages) depending on the recipient's
- time personalisation: greetings based on time of day
- snippets insertion: customise emails very flexibly with any contact-level information
- integration: working as part of my stack and other automations, eg Twittee (lead generation using the Twitter API) Twittee
Overall structure
3 components to the system:
- data management: spreadsheet-like "source of truth" of recipients and sequences, using Grist Grist | The Evolution of Spreadsheets
- email content: text files
- sending emails: python script
DATA MANAGEMENT
I turned to my online spreadsheet solution of choice (Grist: Grist | The Evolution of Spreadsheets) to manage the list of contacts and sequences.
Example
Here is an overview of a campaign in Grist:
This particular campaign was based on reaching out to webinar organisers, based on webinar data I gathered from Twitter (see Twittee)
The spreadsheet format enables to maintain quick and full visibility of the entire campaign, easy to filter as needed.
It includes the following fields:
email
domain
: enables to filter by company (TODO: add logic to space out emails from same company)li
: Linkedin search link, with patternhttps://www.linkedin.com/search/results/all/?keywords={first_name}%20{company_name}
first
title
track
: different messaging by track, eg for webinar campaign each recipient assigned to track based on current solution used. Track name inserted in emails as personlisation.search
: search link when company URL not readily available, using patternhttps://duckduckgo.com/?q=!ducky+{domain}
company
url_reg
: webinar registration linkwebinar_title
: to be inserted in emails for personalisation.tz
: timezone from the webinar (assuming same as organiser)status
:SEND
(ready to send),DISCARD
,DNE
(Do Not Email),Engaged
,Hold
,No
,Queue
,Ready
notes
tweet
: original tweet where lead/webinar was foundanswer
: date answer received (average response time can be calculated)src
: source of the email addresswebinar_date
v
: version of the email, egA
,B
, etc..s_test_date
: test fields_1_date
: send date of email #1s_2_date
: send date of email #2s_3_date
: send date of email #3, etc..lang
:EN
,FR
,DE
created
: record creation dateupdated
: record last updatedup
: duplicate flagradar
: add to Radarsend_from
: logic to send from different email accounts
EMAIL CONTENT
Each email is a text file in a folder for the campaign.
The filename of the text file follows this pattern, enabling support for tracks, language, versions and sequences:
TRACK_LANGUAGE_VERSION_SEQUENCE_SUBJECT LINE.txt
eg. GoToWebinar_E_A_1_GoToWebinar contract.txt
is the email content for the GoToWebinar
track (ie recipients currently using GoToWebinar), in English (E
), version A
(enabling A/B testing, up to Z), 1
st email in the sequence, where the subject line of the email will be GoToWebinar contract
.
It's therefore very easy to quickly create a new email version, sequence or language, just by creating a new text file with the right name.
For a single email campaign:
example of email cadence
Email #1
Filename: GoToWebinar_E_A_1_GoToWebinar contract.txt
= Track: GoToWebinar
= Language: English
= Version: A
= Sequence: 1
= Subject: GoToWebinar contract
Body (content of text file):
Hi{first},
I have seen with your webinar {webinar_title} that you are using GoToWebinar.
Who is responsible for the webinar contract at {company_name}?
And when does it expire/is set to renew?
We can help you improve participants' experience, increase engagement and results.
Webinar.net offers a platform built from the ground up for webinars.
It is like organising a seminar in an auditorium and not in a meeting room.
If you are looking to upgrade/improve your webinar program this year, let me know who would be the right person at {company_name}, and when we should connect?
Thanks,
Nic
{signature}
Email #2
Filename: GoToWebinar_E_A_2_Re GoToWebinar contract.txt
= Track: GoToWebinar
= Language: English
= Version: A
= Sequence: 2
= Subject: Re GoToWebinar contract
Body:
Hi{first},
I hate to pester, especially the wrong person at the wrong time :)
Our clients all were looking to upgrade their webinar program, be it the experience for participants/presenters/producers and/or the results (engagement, number and quality of leads, etc..).
If this might be of interest when renewal is in question for you, can you confirm who the person to talk to about webinars would be, and when to connect in 2022 (ie when your contract with GoToWebinar expires)?
Let me know,
Thanks,
Nic
{signature}
Email #3
Filename: GoToWebinar_E_A_3_Re GoToWebinar contract.txt
= Track: GoToWebinar
= Language: English
= Version: A
= Sequence: 3
= Subject: Re GoToWebinar contract
Body:
Hi{first},
Upgrading to a webinar platform designed from the ground up for that purpose, brings benefits for all involved - participants, presenters and producers.
From not losing participants because a plug-in needs to be installed, to a stress-free presenter experience, a participant experience leading to a more engaged audience, better qualified leads, and more.
Obviously if you are happy with what GoToWebinar has to offer and don't see improvement potential for better business results, just let me know.
Else, when should we connect this year?
Thanks,
Nic
{signature}
etc...
Cadences can go up to as many emails as desired and A/B testing is possible.
I have seen good success with the above campaign, as it was personalised with the solution they used, their own webinar title, and targeting the webinar organiser(s) directly.
Typical other cadences or tests might include dropping other clients in same industry, success metrics, etc...
See Outbound Lead Generation for more.
SENDING EMAILS
Sending methods
Script allows for 3 ways to send emails:
send manually
MAN
: email gets generated ready to send. Provides ability to personalise even more before hitting send.
Here is an example, with the added feature of launching both the recipient's Linkedin profile (this campaign had that info) and the company's website alongside generating the ready-to-send email. This makes hyper-personalisation of each email very efficient.
send automatically using keyboard automation
AUTO_KB
: email gets generated on desktop and sent, using keyboard shortcuts automation.
This is my preferred method.
Why?
Emails are sent 100% as if sent manually, from email client/desktop, thus circumventing any spam filtering based on automation identification.
Plus this allows for inclusion of email signature without having to fiddle with inline image insertion via SMTP which can be a bit fiddly.
send automatically via SMTP
AUTO_SMTP
: fully "behind-the-scenes"
The most "elegant" way to run it (vs the "hacky" approach above), used by all the emailing platforms.
Though generating emails that "look" as if they were sent manually from a metadata perspective (headers, etc..) is painful.
Sending is easy - circumventing complex spam filtering looking at email metadata is trickier.
test
+ TEST
which prints what would be sent, without sending the email / or send to Mailtrap:
Script
06 Sep 2022
This is my current script - this one without opening Linkedin & Company Website (TODO: add example).
Comment inline.
# my default Python boilerplate
from datetime import datetime
print(f"Starting at {datetime.now().strftime('%H:%M:%S')}")
import time
start_time = time.time()
import os
# appending paths to my modules
import sys
sys.path.append("/path/to/project/with/my/modules/in/it")
from dotenv import load_dotenv # to store environment variables
load_dotenv() # initiate load of .env
# pretty printer, always handy to have
import pprint
pp = pprint.PrettyPrinter(indent=4)
# I always have a counter in my boilerplate/footer, to add easily when coding for verification purposes
count = 0
# my collection of utilities. See my-utils
import my_utils
# my Grist interface modules. Anonymised here with "XX" - one module per Grist workspace (ie per client in my case). See Grist | The Evolution of Spreadsheets
import grist_XX
import grist_XX
import grist_XX
# script specific imports
import webbrowser
import time
import random
from datetime import date
from urllib.parse import quote
import datetime
import re
from pynput.keyboard import Key, Controller
####################
import re
import webbrowser
from urllib.parse import quote
import random
from pynput.keyboard import Key, Controller
import smtplib
from email.mime.text import MIMEText
# for automated emails sent via Gmail/SMTP
import yagmail
per_run = 3
stop_mailtrap = 0
send_type = "AUTO_KB" # TEST or MAN or AUTO_KB or AUTO_SMTP
# email_version = 'A'
wait_from = 1
wait_to = 3
####################
# Global Variables
date_time_today = datetime.datetime.now()
date_today = time.mktime(date_time_today.timetuple())
campaign = os.path.basename(__file__)[:-3]
# Client identifcation:
# I assign a 2-letter code to each of my client, as a unique identifier
# XXript name would start with that 2 letter code, enabling automations below
client = campaign[:2]
# Campaign identification
# same as above, each campaign gets a 5 character code assigned, comprising the first 2 being the client ID, with 3 digits added
# the full 5 characters represent the campaign ID
# the XXript name contains the campaign name after the ID
campaign_code = campaign[:5]
# initiate yagmail
if client == 'XX':
yag = yagmail.SMTP(XX_EMAIL, XX_PASSWORD)
if client == 'yy':
yag = yagmail.SMTP(YY_EMAIL, YY_PASSWORD)
# current_time = datetime.datetime.now()
# Bcc Hubspot based on client
if client == 'XX':
bcc = 'xxxxxxx@bcc.hubspot.com'
elif client == 'YY':
bcc = 'yyyyyyy@bcc.hubspot.com'
elif client == 'ZZ':
bcc = 'zzzzzzzz@bcc.hubspot.com'
# Sender based on campaign name
if client == 'XX':
sender = 'nicolas@xxxx.com'
elif client == 'YY':
sender = 'nicolas@yyyy.com'
elif client == 'ZZ':
sender = 'nicolas@zzzz.com'
# Language assignment based on code
french = ['FR', 'F']
german = ['DE', 'D', 'AT']
# Fetching campaign data from Grist (list of namedtuples)
data_grist_campaign = grist_XX.XX003.fetch_table('Master')
# Count left to send based on empty "send date" field
count_rest_to_send = 0
for record in data_grist_campaign:
if record.send_date == None:
count_rest_to_send += 1
# Counters
count_sent = 0
count_replied = 0
count_recipients = 0
recipients = {}
list_sent = []
# Regex to validate email format if needed
validEmail = "[^@]+@[^@]+\.[^@]+"
####################
# Fetch Messages
message_path = f'/path/to/folder/{campaign_code}/emails'
directory = os.fsencode(message_path)
dict_emails = {}
# Dict of email messages
for file in os.listdir(directory):
filename = os.fsdecode(file)
if filename.endswith(".txt"):
print(filename)
filename_regex = re.match(r"(.*)_(.*)_(.*)_(.*)_(.*).txt", filename)
email_lan = filename_regex[1] # Language
email_ver = filename_regex[2] # Version
email_seq = filename_regex[3] # Sequence
email_tra = filename_regex[4] # Track
email_sub = filename_regex[5] # Subject
email_path = f"{message_path}/{filename}"
dict_emails[email_path] = {'email_ver': email_ver, 'email_lan': email_lan, 'email_sub': str(email_sub), 'email_seq': email_seq, 'email_tra': email_tra}
####################
# Functions
def select_email(email_seq, email_lan, email_tra):
path_to_return = ''
email_version = ''
for path, attribute in dict_emails.items():
if attribute['email_seq'] == email_seq and attribute['email_lan'] == email_lan and attribute['email_tra'] == email_tra:
if path_to_return == '':
path_to_return = path
email_version = attribute['email_ver']
else:
if path > path_to_return:
path_to_return = path
email_version = attribute['email_ver']
if path_to_return == '': # Backup to generic email templates if none specific found
email_tra = 'XXX'
for path, attribute in dict_emails.items():
if attribute['email_seq'] == email_seq and attribute['email_lan'] == email_lan and attribute['email_tra'] == email_tra:
if path_to_return == '':
path_to_return = path
email_version = attribute['email_ver']
else:
if path > path_to_return:
path_to_return = path
email_version = attribute['email_ver']
# return path_to_return
dict_to_return = {'email_version': email_version, 'email_path': path_to_return}
return dict_to_return
def mailto(email, subject, body, bcc):
return webbrowser.open(f'mailto:{email}?subject={quote(subject)}&body={quote(body)}&bcc={quote(bcc)}')
def send_email(email, subject, body, send_type, id):
global bcc
global count_sent
global sender
global client
global campaign
global count_test_send # to manage Mailtrap limit
global stop_mailtrap
# global send_type
if send_type == "MAN":
mailto(email.strip(), subject, body, bcc)
print_output = f"Email generated for {email}"
mark_as_sent(id, email_seq, email_version)
time.sleep(1)
count_sent += 1
elif send_type == "TEST":
msg = MIMEText(body)
msg['Subject'] = subject
msg['From'] = sender
msg['To'] = email
# if count_test_send < stop_mailtrap: # To minimise Mailtrap credit
# with smtplib.SMTP("smtp.mailtrap:2525") as server:
# server.ehlo()
# server.starttls()
# server.login("xxxxxxxxxxxx", "xxxxxxxxxxxxx")
# # message = 'Subject: {}\n\n{}'.format(subject, body)
# # server.sendmail(sender, email, message)
# server.sendmail(sender, email, msg.as_string())
# server.quit()
# mark_as_sent(id, email_seq, email_version)
count_sent += 1
# list_sent.append(email)
print_output = f"func send_email: email sent to Mailtrap for {email}"
elif send_type == "AUTO_KB":
mailto(email.strip(), subject, body, bcc)
random.randint(wait_from, wait_to)
print(f"waiting {wait} seconds...")
for x in range(wait):
print(f"{x + 1}..")
time.sleep(1)
# Send automatically by clicking Cmd + Enter
keyb = Controller()
with keyb.pressed(Key.cmd):
keyb.press(Key.enter)
mark_as_sent(id)
count_sent += 1
print_output = f"Email sent to {email} via keyboard automation"
elif send_type == "AUTO_SMTP":
content = [body,
# '',
# yagmail.inline("minions.gif"),
# '\n\n\n\n\n\nNic',
]
# yag.send(email, subject, content)
yag.send(to=email, bcc=bcc, subject=subject, contents=content)
mark_as_sent(id)
count_sent += 1
# Print for verification
# print()
# print(subject)
# print('----')
# print(body[:160], "...")
print_output = f"Email sent to {email} via SMTP automation"
wait = random.randint(wait_from, wait_to)
print(f"waiting {wait} seconds...")
time.sleep(wait)
return print(print_output)
def mark_as_sent(id):
global date_today
global campaign
global client
# Comment out to test marking as Sent in proper column
if send_type == "TEST":
email_seq = 'test'
if client == 'XX':
grist_XX.XX003.update_records('Master', [
{ 'id': id,
f's_{email_seq}_date': date_today,
f's_{email_seq}_v': email_version,
}
])
if client == 'YY':
grist_YY.YY001.update_records('Master', [
{ 'id': id,
f's_{email_seq}_date': date_today,
f's_{email_seq}_v': email_version,
}
])
print(f"{id} marked as Sent")
####################
# Loop
list_dne = [x.email for x in grist_XX.Contacts.fetch_table('Master') if x.dne in [True, 'True']]
# blacklist = my_utils.get_blacklist_domains() + my_utils.get_blacklist_freemail_domains()
count_row = 0
count_sequence_finished = 0
count_left_to_send = 0
count_test_send = 0
list_missing_data_to_send = []
list_missing_email_template = []
for contact in data_grist_campaign:
# time.sleep(1)
count_row += 1
id = int(contact.id)
first = contact.first
email = contact.email
# if email not in list_dne and my_utils.domain_from_email(email) not in blacklist:
if email not in list_dne:
### Send only to marked as Send and without a Do Not Email flag
# if contact.send_date in [None, '', 0] and contact.priority in [None, '']:
if contact.send_date in [None, '', 0]:
count_recipients += 1
if count_recipients < per_run + 1: # Limit per run
# print(f"\n{count_recipients}/{count_rest_to_send}") # Need to find logic for count_rest_to_send
print(f"\n\n{datetime.datetime.now().strftime('%H:%M:%S')} #{count_row} / Recipient {count_recipients} / Send {send_type}")
print(email)
print(f"First: {first}")
message_path = '/path/to/folder/mailingee/XX003/emails/E_A_1_0_EMAIL SUBJECT LINE.txt'
# Subject
fileName = os.path.basename(os.path.splitext(message_path)[0])
subject = re.match(r"(.*)_(.*)_(.*)_(.*)", fileName)
subject = subject[4]
# Body
with open(message_path) as file:
body = file.read()
# Insert First Name
if first not in [None, '', '--']:
body = body.replace(
"{first}", f" {first}")
else:
body = body.replace("{first}", "")
# SEND EMAIL
if send_type == 'TEST':
send_email(email.strip(), subject, body, "TEST", id)
list_sent.append([email, first])
elif send_type == 'MAN':
send_email(email.strip(), subject, body, "MAN", id)
list_sent.append([email, first])
elif send_type == 'AUTO_KB':
send_email(email.strip(), subject, body, "AUTO_KB", id)
list_sent.append([email, first])
elif send_type == 'AUTO_SMTP':
send_email(email.strip(), subject, body, "AUTO_SMTP", id)
list_sent.append([email, first])
else:
count_recipients -= 1
print(f"NO EMAIL SENT FOR {email}")
continue
else:
count_left_to_send += 1
# my default Python footer
########################################################################################################
if __name__ == '__main__':
print('\n\n')
print('-------------------------------')
print(f"count = {count}")
print(f"count_row = {count_row}")
print(f"count_recipients = {count_recipients}")
print(f"count_left_to_send = {count_left_to_send} / For next run")
print()
print(f"count_sent = {count_sent}")
print()
print()
print('-------------------------------')
run_time = round((time.time() - start_time)/60, 1)
print(f'{os.path.basename(__file__)} finished in {run_time} minutes.')
print(datetime.datetime.now().strftime('%H:%M:%S'))
print()
19 Apr 2023
Improved the system a lot over the last few weeks. Amongst other changes:
- different intro/outro depending on day of the week
- different call-to-action write up if email sent to a team, or an individual (“who best to connect with” vs “let’s connect?“)
- different tagline depending if email is .edu /.ac(“.. platform for Universities”) or .org (“..platform for Associations”) or neither (just “…platform”).
- different phone number based on country TLD (my German number only if .de, .at, UK only if .uk, both otherwise)
TODO write-up.
about tracking
this script/solution does not cater for tracking emails - though it could, leveraging services like this one:
Though I had deliverability issues in the past with off-the-shelves solutions (eg Toutapp), where I think the tracking pixel was a culprit (or perhaps in combination with other factors).
Furthermore, I experienced also "false positives" - engaging prospects thinking they had seen/read my email, when they clearly had not.
Last but not least, I'm a data privacy geek and avoid tracking myself as much as possible. So feels weird to employ those tactics on others.
Long story short, I'm underwhelmed by "tracking emails" and do not think at this point that my script/system should support it.