Building a static website: BtoBSales.EU

just with Python (Jinja), HTML & CSS, from scratch

26 Nov 2022

I have built websites in the past using no-code tools (and a couple never published using Macromedia's Dreamweaver - an old WYSIWYG tool).

Building this site here since June, using a static site generator (!python/library-pelican) taught me more about how these are built.

As I need a website to promote my B2B Sales services (https://btobsales.eu/), I decided to give it a shot at building it from scratch, just with code - Python, HTML, CSS & JS.

Benefits:

Alternative considered last month was to use something like Webflow (https://webflow.com/) - a better no-code tool than the one I had used 5 years ago for my startup's website.. supposedly.

I did invest ~10 hours of my time with Webflow, going through the tutorial and building a version 0.1 with it.

What made me stop this and go with a pure code approach:

  • limitations: I have ideas for things I'd want to automate with Python for the site (eg directories), which might have been possible but more cumbersome.
  • abstraction: that's the crux of every no-code tool.. abstracting the code to provide a GUI instead means making opinionated decisions & implementing a logic that is not... logical (to me). I struggled to understand basic things I knew with HTML & CSS which felt like defeating the purpose.
  • costs: 250/year. Not the end of the world, but a) still painful considering the indirect costs (time spent learning the platform, somewhat being locked-in, etc..) and b) cost could go much higher if I decide to automate something generating many pages/ content (eg company profiles).
  • time investment: while everything takes time to learn, these days and with experience, I struggle to invest my time in a proprietary solution where the learnings are only moderately transferrable.

Phase 1: static website

Started a few weeks ago and got the site to work with 7 pages, based on a single repo committed to Github where I created the HTML, CSS & JS.

Should have started this note earlier to keep track!

A few days ago, the pain of having to duplicate changes in repeating sections (eg footer, header, etc..) across multiple pages hit home.
I decided to start building a generator with Python, so I could manage my project (the way it should) with templates.

Phase 2: static website generator

Decided to go with Jinja, for which I had some exposure already (underlying the Python library Pelican & used by my dev while we were building Officebots).

Jinja is a templating engine library for Python, meaning it helps deal with templates from which actual content (ie pages) get generated.

Python Library: jinja2

Current working code:

from jinja2 import Environment, FileSystemLoader
import os

### Global Variables

test = True

if test:
    output_folder = '/path/to/folder/test'
else:
    output_folder = '/path/to/folder/live'
print(f"\n{output_folder=}\n")

count_pages = 0
count_resources = 0

### Main

environment = Environment(loader=FileSystemLoader("/path/to/folder/templates/"))

# LOOP

print(f"Starting loop...\n")

for root, dirs, files in os.walk("/path/to/folder/templates"):
    for name in files:
        if name.endswith((".html")):

            file_path = f"{root}/{name}"

            template = environment.get_template(name)

            # INDEX
            if 'index' in name:
                page_name = name.replace("page_", "").replace(".html", "")
                print(f"\nRendering INDEX...\n{name=}\n{page_name=}\n{file_path=}")
                # root
                if test:    
                    ROOT = ''
                else:
                    ROOT = 'https://btobsales.eu/'
                print(f"{ROOT=}")
                # render
                output = template.render(ROOT=ROOT)
                output_path = f"{output_folder}/index.html"
                with open(output_path, "w") as f:
                    f.write(output)
                print(f"Generated at {output_path=}\n")
                count_pages += 1

            # OTHER PAGES
            elif name.startswith('page_'):
                page_name = name.replace("page_", "").replace(".html", "")
                print(f"\nRendering PAGE {page_name}...\n{name=}\n{page_name=}\n{file_path=}")
                # root
                if test:    
                    ROOT = '../'
                else:
                    ROOT = 'https://btobsales.eu/'
                print(f"{ROOT=}")
                # render
                output = template.render(ROOT=ROOT)
                output_path = f"{output_folder}/{page_name}/index.html"
                try:
                    with open(output_path, "w") as f:
                        f.write(output)
                except:
                    os.mkdir(f"{output_folder}/{page_name}")
                    with open(output_path, "w") as f:
                        f.write(output)

                print(f"Generated at {output_path=}\n")
                count_pages += 1

            # RESOURCES
            elif name.startswith('resource_'):
                page_name = name.replace("resource_", "").replace(".html", "")
                print(f"\nRendering RESOURCE {page_name}...\n{name=}\n{page_name=}\n{file_path=}")
                # root
                if test:    
                    ROOT = '../../'
                else:
                    ROOT = 'https://btobsales.eu/resources/'
                print(f"{ROOT=}")
                # render
                output = template.render(ROOT=ROOT)
                output_path = f"{output_folder}/resources/{page_name}/index.html"

                # create 'resources' folder if missing
                if not os.path.isdir(f"{output_folder}/resources/"):
                    os.mkdir(f"{output_folder}/resources/")
                # create 'resources/page_name' folder if missing
                if not os.path.isdir(f"{output_folder}/resources/{page_name}"):
                    os.mkdir(f"{output_folder}/resources/{page_name}")

                with open(output_path, "w") as f:
                        f.write(output)

                print(f"Generated at {output_path=}\n")
                count_resources += 1

28 Nov 2022

Added directory sections & sitemap generator:

from jinja2 import Environment, FileSystemLoader
import os

### Global Variables

test = True
REFRESH = True

# if test:
#     output_folder = '/Users/nic/Python/btobee/test'
# else:
    # output_folder = '/Users/nic/Python/btobee/test_live' # /Users/nic/Github/btobsales.github.io/
output_folder = '/Users/nic/Github/btobsales.github.io' 
print(f"\n{output_folder=}\n")

count_pages = 0
count_resources = 0

timestamp = datetime.now().strftime('%Y%m%d%H%M%S')

### Functions


### Main

environment = Environment(loader=FileSystemLoader("/Users/nic/Python/btobee/templates/"))

### Section Recruiters

count_recs = 0

if REFRESH:
    from generate_section_recruiters import generate_recruiters_list
    recs = generate_recruiters_list()
    list_recruiters = recs['list_recruiters']
    count_recs = recs['count_recruiters_published']
    print(f"\n{count_recs=}\n")

    # root
    if test:    
        ROOT = '../../'
    else:
        ROOT = 'https://btobsales.eu/resources/'
    print(f"{ROOT=}")

    template = environment.get_template("template_section_directory_recruiters.html") # use template for the section
    output = template.render(
        ROOT=ROOT,
        records=list_recruiters,
        count_recs=count_recs,
        ts_publish=ts_publish,
        )
    output_path = f"/path/to/templates/section_directory_recruiters.html" # generate section file to use in loop below
    with open(output_path, "w") as f:
        f.write(output)

### Section VCs

count_vcs = 0

if REFRESH:
    from generate_section_vcs import generate_vcs_list
    vcs = generate_vcs_list()
    list_vcs = vcs['list_vcs']
    count_vcs = vcs['count_vcs_published']
    print(f"\n{count_vcs=}\n")

    # root
    if test:    
        ROOT = '../../'
    else:
        ROOT = 'https://btobsales.eu/resources/'
    print(f"{ROOT=}")

    template = environment.get_template("template_section_directory_vcs.html") # use template for the section
    output = template.render(
        ROOT=ROOT,
        records=list_vcs,
        count_vcs=count_vcs,
        ts_publish=ts_publish,
        )
    output_path = f"/path/to/templates/section_directory_vcs.html" # generate section file to use in loop below
    with open(output_path, "w") as f:
        f.write(output)

# LOOP

print(f"Starting loop...\n")

for root, dirs, files in os.walk("/path/to/templates"):
    for name in files:
        if name.endswith((".html")):

            file_path = f"{root}/{name}"

            template = environment.get_template(name)

            # INDEX
            if 'index' in name:
                page_name = name.replace("page_", "").replace(".html", "")
                print(f"\nRendering INDEX...\n{name=}\n{page_name=}\n{file_path=}")
                # root
                if test:    
                    ROOT = ''
                else:
                    ROOT = 'https://btobsales.eu/'
                print(f"{ROOT=}")
                # render
                output = template.render(
                    ROOT=ROOT,
                    TS=timestamp,
                    )
                output_path = f"{output_folder}/index.html"
                with open(output_path, "w") as f:
                    f.write(output)
                print(f"Generated at {output_path=}\n")
                count_pages += 1

            # OTHER PAGES
            elif name.startswith('page_'):
                page_name = name.replace("page_", "").replace(".html", "")
                print(f"\nRendering PAGE {page_name}...\n{name=}\n{page_name=}\n{file_path=}")
                # root
                if test:    
                    ROOT = '../'
                else:
                    ROOT = 'https://btobsales.eu/'
                print(f"{ROOT=}")
                # render
                output = template.render(
                    ROOT=ROOT,
                    TS=timestamp,
                    )
                output_path = f"{output_folder}/{page_name}/index.html"
                try:
                    with open(output_path, "w") as f:
                        f.write(output)
                except:
                    os.mkdir(f"{output_folder}/{page_name}")
                    with open(output_path, "w") as f:
                        f.write(output)

                print(f"Generated at {output_path=}\n")
                count_pages += 1

            # RESOURCES
            elif name.startswith('resource_'):
                page_name = name.replace("resource_", "").replace(".html", "")
                print(f"\nRendering RESOURCE {page_name}...\n{name=}\n{page_name=}\n{file_path=}")
                # root
                if test:    
                    ROOT = '../../'
                else:
                    ROOT = 'https://btobsales.eu/resources/'
                print(f"{ROOT=}")
                # render
                output = template.render(
                    ROOT=ROOT,
                    TS=timestamp,
                    count_recs=count_recs, # for resources/directory_recruiters
                    count_vcs=count_vcs, # for resources/directory_vcs
                    )
                output_path = f"{output_folder}/resources/{page_name}/index.html"

                # create 'resources' folder if missing
                if not os.path.isdir(f"{output_folder}/resources/"):
                    os.mkdir(f"{output_folder}/resources/")
                # create 'resources/page_name' folder if missing
                if not os.path.isdir(f"{output_folder}/resources/{page_name}"):
                    os.mkdir(f"{output_folder}/resources/{page_name}")

                with open(output_path, "w") as f:
                        f.write(output)

                print(f"Generated at {output_path=}\n")
                count_resources += 1

# Generate Sitemap

from generate_sitemap import generate_sm

generate_sm()

generate data for directory sections

Logic to generate the data for the directory sections, eg here with VCs:

Source is a Grist spreadsheet Grist | The Evolution of Spreadsheets

# Generate data for BB VCs to be used in generate_btobsales.py for https://btobsales.eu/resources/directory-vcs

def generate_vcs_list():

    from collections import namedtuple
    import pickle

    test = False
    v = False
    refresh_countries = True

    # Boilerplate

    root = os.path.dirname(os.path.abspath(__file__))

    #################### FETCH DATA

    countries_data_pickle_path = "test/countries_data.pickle"

    print(f"Fetching Countries data...")
    if refresh_countries:
        countries_data = grist_IX.indeXall.fetch_table('Countries')
        with open(countries_data_pickle_path, 'wb') as pf:
            pickle.dump(countries_data, pf) 

    if not refresh_countries:
        countries_data_pickle = open(countries_data_pickle_path, "rb")
        countries_data = pickle.load(countries_data_pickle)

    dict_countries = {x.name:x.id for x in countries_data}
    dict_countries_flags = {x.id:x.flag for x in countries_data}
    dict_countries_names = {x.code:x.name for x in countries_data}
    dict_countries_codes = {x.id:x.code.lower() for x in countries_data}
    list_countries_codes = [x.code.lower() for x in countries_data]
    print(len(dict_countries))

    print(f"\nFetching recs data...")
    all_recs_data = grist_BB.VCs.fetch_table('Master')
    print(len(all_recs_data))

    list_vcs = []

    vc = namedtuple('VC', [
        'name', 
        'founded',
        'socials',
        'hq',
        'url',
        'domain',
        'sort_by',
        'country',
        'target',
        'contact',
        ])

    ## Loop vcS

    print(f"\nLooping through vcs data...")
    for rec in tqdm(all_recs_data):
        count_row_recs += 1

        if not rec.do_not_publish:

            # domain
            if rec.domain in [None, '']:
                rec = rec._replace(domain=my_utils.domain_from_url(rec.url))

            # sort_by
            rec_name = rec.name
            if rec_name not in [None, '']:
                rec_name = rec_name.lower()
                if 'é' in rec_name:
                    rec_name = rec_name.replace('é', 'e')
                sort_by = rec_name
                rec = rec._replace(sort_by=sort_by)
            else:
                sort_by = f"{rec.domain.lower()}"
                rec = rec._replace(sort_by=sort_by)
            if v:
                print(f"\n#{get_linenumber()} {rec_name=}")

            # founded
            if rec.founded in [None, 'None', 0, '0']:
                rec = rec._replace(founded='')

            # name
            if rec.name in [None, 'None', 0, '0']:
                extract = tldextract.extract(rec.domain)
                domain_name = extract.domain
                if domain_name not in [None, '']:
                    domain_name = domain_name.strip()
                    if '-' in domain_name:
                        domain_name = string.capwords(domain_name, sep='-')
                        domain_name= domain_name.replace('-', ' ')
                    else:
                        domain_name = string.capwords(domain_name, sep=None)
                    rec = rec._replace(name=domain_name)

            ### Socials
            socials_html = ''
            logo_width = '25'
            logo_height = '25'

            if rec.twitter not in [None, 'None', '']:
                socials_html += f"<a class=\"\" href=\"https://twitter.com/{rec.twitter}\" target=\"_blank\" id=\"{rec.twitter}\"><img alt=\"{rec.twitter}\" src=\"https://ik.imagekit.io/vhucnsp9j1u/logos/logo_twitter_black.svg\" width=\"{logo_width}\" height=\"{logo_height}\"></a>"
            else:
                socials_html += f"<img src=\"https://ik.imagekit.io/vhucnsp9j1u/logos/logo_twitter_light.svg\" alt=\"no_twitter_account\" width=\"{logo_width}\" height=\"{logo_height}\" />"

            if rec.linkedin not in [None, 'None', '']:
                socials_html += f" <a class=\"\" href=\"{rec.linkedin}\" target=\"_blank\" id=\"{rec.linkedin}\"><img alt=\"{rec.linkedin}\" src=\"https://ik.imagekit.io/vhucnsp9j1u/logos/logo_linkedin_black.svg\" width=\"{logo_width}\" height=\"{logo_height}\"></a>"
            else:
                socials_html += f" <img src=\"https://ik.imagekit.io/vhucnsp9j1u/logos/logo_linkedin_light.svg\" alt=\"no_linkedin_account\" width=\"{logo_width}\" height=\"{logo_height}\" />"

            ### Contact
            contact = rec.contact
            if rec.contact in [None, ''] and rec.email not in [None, '']:
                contact = rec.email
            if contact not in [None, '']:
                if contact.startswith('http'):
                    contact = f'<div class="contact"><a href="{contact}" target="_blank" id="{contact}"><img src="https://ik.imagekit.io/vhucnsp9j1u/contact-form.svg" alt="Contact form {contact}" width="30" /></a></div>'
                elif '@' in contact:
                    contact = f'<div class="contact"><a href="mailto:{contact}" target="_blank" id="{contact}"><img src="https://ik.imagekit.io/vhucnsp9j1u/email.svg" alt="Email {contact}" width="30" /></a></div>'
                else:
                    contact = ''

            ### Country
            if rec.country not in [None, 'None', 0, '0']:
                if isinstance(rec.country, int): # to avoid keyerror when looping dexees and flag URL already inserted
                    country_html = f'<div class="country_div"><img src="{dict_countries_flags[rec.country]}" alt="{dict_countries_codes[rec.country]}" width="25" /></div>'

            else:
                country_html=''

            ### Target

            if rec.target in [None, 'None', 0, '0']:
                target = ''
            else:
                target = rec.target

            ### Create final record
            list_vcs.append(

            vc(  name=rec.name, 
                        founded=rec.founded,
                        socials=socials_html,
                        hq=rec.country,
                        url=rec.url,
                        domain=rec.domain,
                        sort_by=sort_by,
                        country=country_html,
                        target=target,
                        contact=contact,
                        )
            )

            count_vcs_published += 1

    print()

    if v:
        for y in list_vcs:
            print()
            print(f"{y.name=}")
            print(f"{y.founded=}")
            print(f"{y.socials=}")
            print(f"{y.hq=}")
            print(f"{y.url=}")
            print(f"{y.domain=}")
            print(f"{y.sort_by=}")
            print(f"{y.country=}")

    # # FORMAT VC DATA
    list_vcs = sorted(list_vcs, key=attrgetter('sort_by'))

    return {'list_vcs': list_vcs, 'count_vcs_published': count_vcs_published}

generate_sitemap.py

# Generate sitemap for btobsales.eu

from pathlib import Path

def generate_sm():

    test = False
    v = True

    blacklist = [
        'cannabis',
        'darknet',
    ]

    ## Generate links

    site = 'https://btobsales.eu'
    src = '/path/to/folder/with/html/files'


    links = []

    blacklist = (
        'css',
        'img',
        'old',
        '.git',
    )

    for root, dirs, files in os.walk(src):

        for file in files:
            if file.endswith(".html") and file != '404.html':
                relative_path = f"{root[38:]}"
                if not relative_path.startswith(blacklist):
                    if relative_path == '':
                        links.append(f"{site}")
                    else:
                        links.append(f"{site}/{relative_path}")

    for link in links:
        print(link)
    print(f"\n{len(links)} links generated\n")


    ## Generate sitemap.xml

    sitemap_template='''<?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
        {% for link in links %}
        <url>
            <loc>{{ link['link'] }}</loc>
            <lastmod>{{ link['lastmod'] }}</lastmod>
            <priority>{{ link['priority'] }}</priority>        
        </url>
        {% endfor %}
    </urlset>'''

    template_sitemap = Template(sitemap_template)

    lastmod_date = datetime.now().strftime('%Y-%m-%d')

    list_meta_urls = []

    for link in links:
        if link == site:
            list_meta_urls.append({'link': link, 'lastmod': lastmod_date, 'priority': '1.0'})
        else:
            list_meta_urls.append({'link': link, 'lastmod': lastmod_date, 'priority': '0.8'})


    # Render each row / column in the sitemap
    sitemap_output = template_sitemap.render(links=list_meta_urls) 

    if test:
        sitemap_filename = f"test/test_sitemap.xml" 
    else:
        sitemap_filename = f"/path/to/site/root/folder/sitemap.xml" 

    # Write the File to Your Working Folder
    with open(sitemap_filename, 'wt') as f:   
        f.write(sitemap_output)

    print(f"\nSitemap generated at {sitemap_filename}\n")

output:

<?xml version="1.0" encoding="UTF-8"?>
    <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">

        <url>
            <loc>https://btobsales.eu</loc>
            <lastmod>2022-11-28</lastmod>
            <priority>1.0</priority>        
        </url>

        <url>
            <loc>https://btobsales.eu/b2b-sales-consulting</loc>
            <lastmod>2022-11-28</lastmod>
            <priority>0.8</priority>        
        </url>

        <url>
            <loc>https://btobsales.eu/vp-sales-as-a-service</loc>
            <lastmod>2022-11-28</lastmod>
            <priority>0.8</priority>        
        </url>

        <url>
            <loc>https://btobsales.eu/resources/directory-recruiters</loc>
            <lastmod>2022-11-28</lastmod>
            <priority>0.8</priority>        
        </url>

        <url>
            <loc>https://btobsales.eu/resources/directory-ve-providers</loc>
            <lastmod>2022-11-28</lastmod>
            <priority>0.8</priority>        
        </url>

        <url>
            <loc>https://btobsales.eu/resources/directory-webinar-providers</loc>
            <lastmod>2022-11-28</lastmod>
            <priority>0.8</priority>        
        </url>

        <url>
            <loc>https://btobsales.eu/resources/directory-vcs</loc>
            <lastmod>2022-11-28</lastmod>
            <priority>0.8</priority>        
        </url>

        <url>
            <loc>https://btobsales.eu/vp-emea-as-a-service</loc>
            <lastmod>2022-11-28</lastmod>
            <priority>0.8</priority>        
        </url>

        <url>
            <loc>https://btobsales.eu/sales-as-a-service</loc>
            <lastmod>2022-11-28</lastmod>
            <priority>0.8</priority>        
        </url>

    </urlset>

301 redirect pages where URL has changed

in .htaccess, added:

redirect 301 https://btobsales.eu/recruiters-directory/index.html https://btobsales.eu/resources/directory-recruiters.html

links

social