Table of Contents
26 Nov 2022
I have built websites in the past using no-code tools (and a couple never published using Macromedia's Dreamweaver - an old WYSIWYG tool).
Building this site here since June, using a static site generator (!python/library-pelican) taught me more about how these are built.
As I need a website to promote my B2B Sales services (https://btobsales.eu/), I decided to give it a shot at building it from scratch, just with code - Python, HTML, CSS & JS.
Benefits:
- learning opportunity in those 4 languages
- free website (hosted on Github Pages Hosting my static website(s) with Github Pages)
Alternative considered last month was to use something like Webflow (https://webflow.com/) - a better no-code tool than the one I had used 5 years ago for my startup's website.. supposedly.
I did invest ~10 hours of my time with Webflow, going through the tutorial and building a version 0.1 with it.
What made me stop this and go with a pure code approach:
- limitations: I have ideas for things I'd want to automate with Python for the site (eg directories), which might have been possible but more cumbersome.
- abstraction: that's the crux of every no-code tool.. abstracting the code to provide a GUI instead means making opinionated decisions & implementing a logic that is not... logical (to me). I struggled to understand basic things I knew with HTML & CSS which felt like defeating the purpose.
- costs: 250/year. Not the end of the world, but a) still painful considering the indirect costs (time spent learning the platform, somewhat being locked-in, etc..) and b) cost could go much higher if I decide to automate something generating many pages/ content (eg company profiles).
- time investment: while everything takes time to learn, these days and with experience, I struggle to invest my time in a proprietary solution where the learnings are only moderately transferrable.
Phase 1: static website
Started a few weeks ago and got the site to work with 7 pages, based on a single repo committed to Github where I created the HTML, CSS & JS.
Should have started this note earlier to keep track!
A few days ago, the pain of having to duplicate changes in repeating sections (eg footer, header, etc..) across multiple pages hit home.
I decided to start building a generator with Python, so I could manage my project (the way it should) with templates.
Phase 2: static website generator
Decided to go with Jinja, for which I had some exposure already (underlying the Python library Pelican & used by my dev while we were building Officebots).
Jinja is a templating engine library for Python, meaning it helps deal with templates from which actual content (ie pages) get generated.
Current working code:
from jinja2 import Environment, FileSystemLoader
import os
### Global Variables
test = True
if test:
output_folder = '/path/to/folder/test'
else:
output_folder = '/path/to/folder/live'
print(f"\n{output_folder=}\n")
count_pages = 0
count_resources = 0
### Main
environment = Environment(loader=FileSystemLoader("/path/to/folder/templates/"))
# LOOP
print(f"Starting loop...\n")
for root, dirs, files in os.walk("/path/to/folder/templates"):
for name in files:
if name.endswith((".html")):
file_path = f"{root}/{name}"
template = environment.get_template(name)
# INDEX
if 'index' in name:
page_name = name.replace("page_", "").replace(".html", "")
print(f"\nRendering INDEX...\n{name=}\n{page_name=}\n{file_path=}")
# root
if test:
ROOT = ''
else:
ROOT = 'https://btobsales.eu/'
print(f"{ROOT=}")
# render
output = template.render(ROOT=ROOT)
output_path = f"{output_folder}/index.html"
with open(output_path, "w") as f:
f.write(output)
print(f"Generated at {output_path=}\n")
count_pages += 1
# OTHER PAGES
elif name.startswith('page_'):
page_name = name.replace("page_", "").replace(".html", "")
print(f"\nRendering PAGE {page_name}...\n{name=}\n{page_name=}\n{file_path=}")
# root
if test:
ROOT = '../'
else:
ROOT = 'https://btobsales.eu/'
print(f"{ROOT=}")
# render
output = template.render(ROOT=ROOT)
output_path = f"{output_folder}/{page_name}/index.html"
try:
with open(output_path, "w") as f:
f.write(output)
except:
os.mkdir(f"{output_folder}/{page_name}")
with open(output_path, "w") as f:
f.write(output)
print(f"Generated at {output_path=}\n")
count_pages += 1
# RESOURCES
elif name.startswith('resource_'):
page_name = name.replace("resource_", "").replace(".html", "")
print(f"\nRendering RESOURCE {page_name}...\n{name=}\n{page_name=}\n{file_path=}")
# root
if test:
ROOT = '../../'
else:
ROOT = 'https://btobsales.eu/resources/'
print(f"{ROOT=}")
# render
output = template.render(ROOT=ROOT)
output_path = f"{output_folder}/resources/{page_name}/index.html"
# create 'resources' folder if missing
if not os.path.isdir(f"{output_folder}/resources/"):
os.mkdir(f"{output_folder}/resources/")
# create 'resources/page_name' folder if missing
if not os.path.isdir(f"{output_folder}/resources/{page_name}"):
os.mkdir(f"{output_folder}/resources/{page_name}")
with open(output_path, "w") as f:
f.write(output)
print(f"Generated at {output_path=}\n")
count_resources += 1
28 Nov 2022
Added directory sections & sitemap generator:
from jinja2 import Environment, FileSystemLoader
import os
### Global Variables
test = True
REFRESH = True
# if test:
# output_folder = '/Users/nic/Python/btobee/test'
# else:
# output_folder = '/Users/nic/Python/btobee/test_live' # /Users/nic/Github/btobsales.github.io/
output_folder = '/Users/nic/Github/btobsales.github.io'
print(f"\n{output_folder=}\n")
count_pages = 0
count_resources = 0
timestamp = datetime.now().strftime('%Y%m%d%H%M%S')
### Functions
### Main
environment = Environment(loader=FileSystemLoader("/Users/nic/Python/btobee/templates/"))
### Section Recruiters
count_recs = 0
if REFRESH:
from generate_section_recruiters import generate_recruiters_list
recs = generate_recruiters_list()
list_recruiters = recs['list_recruiters']
count_recs = recs['count_recruiters_published']
print(f"\n{count_recs=}\n")
# root
if test:
ROOT = '../../'
else:
ROOT = 'https://btobsales.eu/resources/'
print(f"{ROOT=}")
template = environment.get_template("template_section_directory_recruiters.html") # use template for the section
output = template.render(
ROOT=ROOT,
records=list_recruiters,
count_recs=count_recs,
ts_publish=ts_publish,
)
output_path = f"/path/to/templates/section_directory_recruiters.html" # generate section file to use in loop below
with open(output_path, "w") as f:
f.write(output)
### Section VCs
count_vcs = 0
if REFRESH:
from generate_section_vcs import generate_vcs_list
vcs = generate_vcs_list()
list_vcs = vcs['list_vcs']
count_vcs = vcs['count_vcs_published']
print(f"\n{count_vcs=}\n")
# root
if test:
ROOT = '../../'
else:
ROOT = 'https://btobsales.eu/resources/'
print(f"{ROOT=}")
template = environment.get_template("template_section_directory_vcs.html") # use template for the section
output = template.render(
ROOT=ROOT,
records=list_vcs,
count_vcs=count_vcs,
ts_publish=ts_publish,
)
output_path = f"/path/to/templates/section_directory_vcs.html" # generate section file to use in loop below
with open(output_path, "w") as f:
f.write(output)
# LOOP
print(f"Starting loop...\n")
for root, dirs, files in os.walk("/path/to/templates"):
for name in files:
if name.endswith((".html")):
file_path = f"{root}/{name}"
template = environment.get_template(name)
# INDEX
if 'index' in name:
page_name = name.replace("page_", "").replace(".html", "")
print(f"\nRendering INDEX...\n{name=}\n{page_name=}\n{file_path=}")
# root
if test:
ROOT = ''
else:
ROOT = 'https://btobsales.eu/'
print(f"{ROOT=}")
# render
output = template.render(
ROOT=ROOT,
TS=timestamp,
)
output_path = f"{output_folder}/index.html"
with open(output_path, "w") as f:
f.write(output)
print(f"Generated at {output_path=}\n")
count_pages += 1
# OTHER PAGES
elif name.startswith('page_'):
page_name = name.replace("page_", "").replace(".html", "")
print(f"\nRendering PAGE {page_name}...\n{name=}\n{page_name=}\n{file_path=}")
# root
if test:
ROOT = '../'
else:
ROOT = 'https://btobsales.eu/'
print(f"{ROOT=}")
# render
output = template.render(
ROOT=ROOT,
TS=timestamp,
)
output_path = f"{output_folder}/{page_name}/index.html"
try:
with open(output_path, "w") as f:
f.write(output)
except:
os.mkdir(f"{output_folder}/{page_name}")
with open(output_path, "w") as f:
f.write(output)
print(f"Generated at {output_path=}\n")
count_pages += 1
# RESOURCES
elif name.startswith('resource_'):
page_name = name.replace("resource_", "").replace(".html", "")
print(f"\nRendering RESOURCE {page_name}...\n{name=}\n{page_name=}\n{file_path=}")
# root
if test:
ROOT = '../../'
else:
ROOT = 'https://btobsales.eu/resources/'
print(f"{ROOT=}")
# render
output = template.render(
ROOT=ROOT,
TS=timestamp,
count_recs=count_recs, # for resources/directory_recruiters
count_vcs=count_vcs, # for resources/directory_vcs
)
output_path = f"{output_folder}/resources/{page_name}/index.html"
# create 'resources' folder if missing
if not os.path.isdir(f"{output_folder}/resources/"):
os.mkdir(f"{output_folder}/resources/")
# create 'resources/page_name' folder if missing
if not os.path.isdir(f"{output_folder}/resources/{page_name}"):
os.mkdir(f"{output_folder}/resources/{page_name}")
with open(output_path, "w") as f:
f.write(output)
print(f"Generated at {output_path=}\n")
count_resources += 1
# Generate Sitemap
from generate_sitemap import generate_sm
generate_sm()
generate data for directory sections
Logic to generate the data for the directory sections, eg here with VCs:
Source is a Grist spreadsheet Grist | The Evolution of Spreadsheets
# Generate data for BB VCs to be used in generate_btobsales.py for https://btobsales.eu/resources/directory-vcs
def generate_vcs_list():
from collections import namedtuple
import pickle
test = False
v = False
refresh_countries = True
# Boilerplate
root = os.path.dirname(os.path.abspath(__file__))
#################### FETCH DATA
countries_data_pickle_path = "test/countries_data.pickle"
print(f"Fetching Countries data...")
if refresh_countries:
countries_data = grist_IX.indeXall.fetch_table('Countries')
with open(countries_data_pickle_path, 'wb') as pf:
pickle.dump(countries_data, pf)
if not refresh_countries:
countries_data_pickle = open(countries_data_pickle_path, "rb")
countries_data = pickle.load(countries_data_pickle)
dict_countries = {x.name:x.id for x in countries_data}
dict_countries_flags = {x.id:x.flag for x in countries_data}
dict_countries_names = {x.code:x.name for x in countries_data}
dict_countries_codes = {x.id:x.code.lower() for x in countries_data}
list_countries_codes = [x.code.lower() for x in countries_data]
print(len(dict_countries))
print(f"\nFetching recs data...")
all_recs_data = grist_BB.VCs.fetch_table('Master')
print(len(all_recs_data))
list_vcs = []
vc = namedtuple('VC', [
'name',
'founded',
'socials',
'hq',
'url',
'domain',
'sort_by',
'country',
'target',
'contact',
])
## Loop vcS
print(f"\nLooping through vcs data...")
for rec in tqdm(all_recs_data):
count_row_recs += 1
if not rec.do_not_publish:
# domain
if rec.domain in [None, '']:
rec = rec._replace(domain=my_utils.domain_from_url(rec.url))
# sort_by
rec_name = rec.name
if rec_name not in [None, '']:
rec_name = rec_name.lower()
if 'é' in rec_name:
rec_name = rec_name.replace('é', 'e')
sort_by = rec_name
rec = rec._replace(sort_by=sort_by)
else:
sort_by = f"{rec.domain.lower()}"
rec = rec._replace(sort_by=sort_by)
if v:
print(f"\n#{get_linenumber()} {rec_name=}")
# founded
if rec.founded in [None, 'None', 0, '0']:
rec = rec._replace(founded='')
# name
if rec.name in [None, 'None', 0, '0']:
extract = tldextract.extract(rec.domain)
domain_name = extract.domain
if domain_name not in [None, '']:
domain_name = domain_name.strip()
if '-' in domain_name:
domain_name = string.capwords(domain_name, sep='-')
domain_name= domain_name.replace('-', ' ')
else:
domain_name = string.capwords(domain_name, sep=None)
rec = rec._replace(name=domain_name)
### Socials
socials_html = ''
logo_width = '25'
logo_height = '25'
if rec.twitter not in [None, 'None', '']:
socials_html += f"<a class=\"\" href=\"https://twitter.com/{rec.twitter}\" target=\"_blank\" id=\"{rec.twitter}\"><img alt=\"{rec.twitter}\" src=\"https://ik.imagekit.io/vhucnsp9j1u/logos/logo_twitter_black.svg\" width=\"{logo_width}\" height=\"{logo_height}\"></a>"
else:
socials_html += f"<img src=\"https://ik.imagekit.io/vhucnsp9j1u/logos/logo_twitter_light.svg\" alt=\"no_twitter_account\" width=\"{logo_width}\" height=\"{logo_height}\" />"
if rec.linkedin not in [None, 'None', '']:
socials_html += f" <a class=\"\" href=\"{rec.linkedin}\" target=\"_blank\" id=\"{rec.linkedin}\"><img alt=\"{rec.linkedin}\" src=\"https://ik.imagekit.io/vhucnsp9j1u/logos/logo_linkedin_black.svg\" width=\"{logo_width}\" height=\"{logo_height}\"></a>"
else:
socials_html += f" <img src=\"https://ik.imagekit.io/vhucnsp9j1u/logos/logo_linkedin_light.svg\" alt=\"no_linkedin_account\" width=\"{logo_width}\" height=\"{logo_height}\" />"
### Contact
contact = rec.contact
if rec.contact in [None, ''] and rec.email not in [None, '']:
contact = rec.email
if contact not in [None, '']:
if contact.startswith('http'):
contact = f'<div class="contact"><a href="{contact}" target="_blank" id="{contact}"><img src="https://ik.imagekit.io/vhucnsp9j1u/contact-form.svg" alt="Contact form {contact}" width="30" /></a></div>'
elif '@' in contact:
contact = f'<div class="contact"><a href="mailto:{contact}" target="_blank" id="{contact}"><img src="https://ik.imagekit.io/vhucnsp9j1u/email.svg" alt="Email {contact}" width="30" /></a></div>'
else:
contact = ''
### Country
if rec.country not in [None, 'None', 0, '0']:
if isinstance(rec.country, int): # to avoid keyerror when looping dexees and flag URL already inserted
country_html = f'<div class="country_div"><img src="{dict_countries_flags[rec.country]}" alt="{dict_countries_codes[rec.country]}" width="25" /></div>'
else:
country_html=''
### Target
if rec.target in [None, 'None', 0, '0']:
target = ''
else:
target = rec.target
### Create final record
list_vcs.append(
vc( name=rec.name,
founded=rec.founded,
socials=socials_html,
hq=rec.country,
url=rec.url,
domain=rec.domain,
sort_by=sort_by,
country=country_html,
target=target,
contact=contact,
)
)
count_vcs_published += 1
print()
if v:
for y in list_vcs:
print()
print(f"{y.name=}")
print(f"{y.founded=}")
print(f"{y.socials=}")
print(f"{y.hq=}")
print(f"{y.url=}")
print(f"{y.domain=}")
print(f"{y.sort_by=}")
print(f"{y.country=}")
# # FORMAT VC DATA
list_vcs = sorted(list_vcs, key=attrgetter('sort_by'))
return {'list_vcs': list_vcs, 'count_vcs_published': count_vcs_published}
generate_sitemap.py
# Generate sitemap for btobsales.eu
from pathlib import Path
def generate_sm():
test = False
v = True
blacklist = [
'cannabis',
'darknet',
]
## Generate links
site = 'https://btobsales.eu'
src = '/path/to/folder/with/html/files'
links = []
blacklist = (
'css',
'img',
'old',
'.git',
)
for root, dirs, files in os.walk(src):
for file in files:
if file.endswith(".html") and file != '404.html':
relative_path = f"{root[38:]}"
if not relative_path.startswith(blacklist):
if relative_path == '':
links.append(f"{site}")
else:
links.append(f"{site}/{relative_path}")
for link in links:
print(link)
print(f"\n{len(links)} links generated\n")
## Generate sitemap.xml
sitemap_template='''<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
{% for link in links %}
<url>
<loc>{{ link['link'] }}</loc>
<lastmod>{{ link['lastmod'] }}</lastmod>
<priority>{{ link['priority'] }}</priority>
</url>
{% endfor %}
</urlset>'''
template_sitemap = Template(sitemap_template)
lastmod_date = datetime.now().strftime('%Y-%m-%d')
list_meta_urls = []
for link in links:
if link == site:
list_meta_urls.append({'link': link, 'lastmod': lastmod_date, 'priority': '1.0'})
else:
list_meta_urls.append({'link': link, 'lastmod': lastmod_date, 'priority': '0.8'})
# Render each row / column in the sitemap
sitemap_output = template_sitemap.render(links=list_meta_urls)
if test:
sitemap_filename = f"test/test_sitemap.xml"
else:
sitemap_filename = f"/path/to/site/root/folder/sitemap.xml"
# Write the File to Your Working Folder
with open(sitemap_filename, 'wt') as f:
f.write(sitemap_output)
print(f"\nSitemap generated at {sitemap_filename}\n")
output:
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
<url>
<loc>https://btobsales.eu</loc>
<lastmod>2022-11-28</lastmod>
<priority>1.0</priority>
</url>
<url>
<loc>https://btobsales.eu/b2b-sales-consulting</loc>
<lastmod>2022-11-28</lastmod>
<priority>0.8</priority>
</url>
<url>
<loc>https://btobsales.eu/vp-sales-as-a-service</loc>
<lastmod>2022-11-28</lastmod>
<priority>0.8</priority>
</url>
<url>
<loc>https://btobsales.eu/resources/directory-recruiters</loc>
<lastmod>2022-11-28</lastmod>
<priority>0.8</priority>
</url>
<url>
<loc>https://btobsales.eu/resources/directory-ve-providers</loc>
<lastmod>2022-11-28</lastmod>
<priority>0.8</priority>
</url>
<url>
<loc>https://btobsales.eu/resources/directory-webinar-providers</loc>
<lastmod>2022-11-28</lastmod>
<priority>0.8</priority>
</url>
<url>
<loc>https://btobsales.eu/resources/directory-vcs</loc>
<lastmod>2022-11-28</lastmod>
<priority>0.8</priority>
</url>
<url>
<loc>https://btobsales.eu/vp-emea-as-a-service</loc>
<lastmod>2022-11-28</lastmod>
<priority>0.8</priority>
</url>
<url>
<loc>https://btobsales.eu/sales-as-a-service</loc>
<lastmod>2022-11-28</lastmod>
<priority>0.8</priority>
</url>
</urlset>
301 redirect pages where URL has changed
in .htaccess, added:
redirect 301 https://btobsales.eu/recruiters-directory/index.html https://btobsales.eu/resources/directory-recruiters.html