19 Aug 2022
Goal: 24/7 monitoring of network & internet connection to identify source and log dropouts for reporting to ISP.
My setup:
Cable modem > Unifi Router USG > Unifi Switch US-8 > Ethernet connection to macOS Monterey
Good Resources
Python Script to Monitor Network Connection
linuxhint
Python Script to Monitor Network Connection and saving into Log File - GeeksforGeeks
geeksforgeeks
Pinging servers in Python - Stack Overflow
stackoverflow
Measuring internet consistency with speedtest.net, plotly and docker
medium.com
Resources / References
Monitor download/upload speeds
>Monitor your internet with python
pythonprogramming
Tweeting about your low internet speed and tagging your ISP
pythonprogramming
I care less about speed than just availability.
Not a good fit.
Simplemonitor
updated doc:
simplemonitor/index.rst at develop · jamesoff/simplemonitor · GitHub
github
Probably too complex for my needs.
Not a good fit.
Smokeping
For reference only. Too complex for my needs.
Not a good fit.
Speednet CLI
sivel/speedtest-cli: Command line interface for testing internet bandwidth using speedtest.net
github.com
with Python API: https://github.com/sivel/speedtest-cli/wiki
04 Sep 2022
Starting code, open-sourcing repo here:
GitHub - ndeville/pingee: Internet Connection Monitoring Script
github
Logic:
- identify network path automatically for devices to ping or hardcode internal IPs?
- ping switch (1s)
- ping router (1s)
- ping remote server #1 from random list
if success:
repeat loop
if #1 fails, ping remote server # 2
if #2 fails, ping remote server # 3
If #3 fails, log & notify as internet dropout
Assuming 3s loop (by default or with added delay), ie one ping per second.
List of IPs to ping randomly:
[
'208.67.222.222', # OpenDNS
'208.67.220.220', # OpenDNS
'1.1.1.1', # Cloudflare
'1.0.0.1', # Cloudflare
'8.8.8.8', # Google DNS
'8.8.4.4', # Google DNS
]
My questions:
- is that the right way to build the script?
- is the ping cadence too much, ie:
- will I run the risk to get blacklisted by remote server(s)?
- will it impact internal network?
Tests
06 Sep 2022
import subprocess
# Internal ping
print(f"INTERNAL PING:\n")
router = '192.168.1.1'
router_command = ['ping', '-c', '1', router]
router_ping = subprocess.call(router_command)
print(f"\n{router_ping=}\n")
# External ping
print(f"EXTERNAL PING:\n")
hostnames = [
'208.67.222.222', # OpenDNS
'208.67.220.220', # OpenDNS
'1.1.1.1', # Cloudflare
'1.0.0.1', # Cloudflare
'8.8.8.8', # Google DNS
'8.8.4.4', # Google DNS
]
for hostname in hostnames:
hostname_command = ['ping', '-c', '1', hostname]
external_ping = subprocess.call(hostname_command)
print(f"\n{external_ping=}\n")
outputs:
INTERNAL PING:
PING 192.168.1.1 (192.168.1.1): 56 data bytes
64 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=2.220 ms
--- 192.168.1.1 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 2.220/2.220/2.220/0.000 ms
router_ping=0
EXTERNAL PING:
PING 208.67.222.222 (208.67.222.222): 56 data bytes
64 bytes from 208.67.222.222: icmp_seq=0 ttl=55 time=21.802 ms
--- 208.67.222.222 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 21.802/21.802/21.802/0.000 ms
external_ping=0
PING 208.67.220.220 (208.67.220.220): 56 data bytes
64 bytes from 208.67.220.220: icmp_seq=0 ttl=55 time=22.655 ms
--- 208.67.220.220 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 22.655/22.655/22.655/0.000 ms
external_ping=0
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=56 time=20.793 ms
--- 1.1.1.1 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 20.793/20.793/20.793/0.000 ms
external_ping=0
PING 1.0.0.1 (1.0.0.1): 56 data bytes
64 bytes from 1.0.0.1: icmp_seq=0 ttl=56 time=22.733 ms
--- 1.0.0.1 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 22.733/22.733/22.733/0.000 ms
external_ping=0
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=115 time=14.300 ms
--- 8.8.8.8 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 14.300/14.300/14.300/0.000 ms
external_ping=0
PING 8.8.4.4 (8.8.4.4): 56 data bytes
64 bytes from 8.8.4.4: icmp_seq=0 ttl=116 time=12.229 ms
--- 8.8.4.4 ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 12.229/12.229/12.229/0.000 ms
external_ping=0
Need to find a way to avoid terminal printing when all OK. Lookup other method.
11 Nov 2022
Tests using the speedtest-cli library Python library: speedtest-cli
Working script:
from datetime import datetime
import os
import time
import speedtest
import csv
date = f"{datetime.now().strftime('%y%m%d')}"
tests_to_run = 3
threads = 1 # 1 simulates a typical file transfer, else None
log_file = 'path/to/log.csv'
count = 0
def test_speed(count, threads=None, servers=[]):
global date
start_test_time = time.time()
s = speedtest.Speedtest()
# s.get_servers(servers)
try:
s.get_best_server()
download = s.download(threads=threads)
download_mbps = f"{round(download / 1000000, 2)} Mbit/s"
run_time = round((time.time() - start_test_time), 1)
timestamp = datetime.now().strftime('%H:%M:%S')
print(f"#{count} [{timestamp}] {download_mbps} ({run_time}s test time)")
with open(log_file, 'a', newline='', encoding='utf-8') as i:
writer = csv.writer(i, delimiter=',', quotechar='|', quoting=csv.QUOTE_MINIMAL, lineterminator='\n')
if threads == None:
threads = 'multi'
writer.writerow([date, timestamp, int(download), download_mbps, threads, run_time, "OK"])
return download
except Exception as e:
timestamp = datetime.now().strftime('%H:%M:%S')
run_time = round((time.time() - start_test_time), 1)
error_message = f"ERROR: {e}"
with open(f"log.csv", 'a', newline='', encoding='utf-8') as i:
writer = csv.writer(i, delimiter=',', quotechar='|', quoting=csv.QUOTE_MINIMAL, lineterminator='\n')
if threads == None:
threads = 'multi'
writer.writerow([date, timestamp, "ERROR", "ERROR", threads, run_time, error_message])
print(error_message)
return "ERROR"
print(f"\nStarting speed test with {tests_to_run} runs (threads={threads})...\n")
for x in range(0, tests_to_run):
count += 1
test_speed(count, threads=threads)
print()
Though got a 403 Forbidden after a while?
Branched out speed_test.py and ping_test.py.
For ping_test.py, will use Python library ping3 Python library: ping3
11 Nov 2022
Just realised that Speedtest has its own CLI tools!
31 Dec 2022
Using a basic ping loop now:
### Basic ping loop
from datetime import datetime
hostnames = [
'208.67.222.222', # OpenDNS
'208.67.220.220', # OpenDNS
'1.1.1.1', # Cloudflare
'1.0.0.1', # Cloudflare
'8.8.8.8', # Google DNS
'8.8.4.4', # Google DNS
]
from ping3 import ping, verbose_ping
import time
for i in range(0,10000):
for host in hostnames:
p = ping(host)
# print(f"\n{p=}")
if p != None:
p_format = round(p*1000, 2)
print(f"{datetime.now().strftime('%H:%M:%S')} OK \t {host}: {p_format}ms")
# print("\r" + f"{host}: {p_format}ms", end='')
else:
print(f"{datetime.now().strftime('%H:%M:%S')} NOK \t {host}: {p}")
with open('log/log.txt', 'a') as file:
file.write(f"\n{datetime.now().strftime('%H:%M:%S')} NOK {host}: {p}")
time.sleep(1)
⬇︎
13:10:40 OK 208.67.220.220: 19.83ms
13:10:42 OK 1.1.1.1: 93.14ms
13:10:43 OK 1.0.0.1: 20.63ms
13:10:44 OK 8.8.8.8: 13.91ms
13:10:45 OK 8.8.4.4: 18.77ms
13:10:46 OK 208.67.222.222: 18.85ms
13:10:47 OK 208.67.220.220: 20.05ms
13:10:48 OK 1.1.1.1: 95.95ms
13:10:49 OK 1.0.0.1: 22.12ms
13:10:50 OK 8.8.8.8: 14.56ms
13:10:51 OK 8.8.4.4: 13.82ms
13:10:52 OK 208.67.222.222: 20.49ms
13:10:53 OK 208.67.220.220: 20.87ms
13:10:54 OK 1.1.1.1: 33.76ms
13:10:55 OK 1.0.0.1: 21.06ms
13:10:56 OK 8.8.8.8: 17.02ms
13:10:57 OK 8.8.4.4: 16.27ms
13:10:58 OK 208.67.222.222: 20.5ms
13:10:59 OK 208.67.220.220: 21.94ms
13:11:00 OK 1.1.1.1: 33.78ms
13:11:01 OK 1.0.0.1: 26.58ms
13:11:02 OK 8.8.8.8: 12.99ms
13:11:03 OK 8.8.4.4: 15.64ms
13:11:04 OK 208.67.222.222: 23.74ms
13:11:05 OK 208.67.220.220: 20.6ms
13:11:10 NOK 1.1.1.1: None
13:11:11 OK 1.0.0.1: 22.53ms
13:11:12 OK 8.8.8.8: 13.98ms
13:11:13 OK 8.8.4.4: 16.91ms
13:11:14 OK 208.67.222.222: 19.28ms
13:11:15 OK 208.67.220.220: 18.69ms
13:11:16 OK 1.1.1.1: 29.85ms
13:11:17 OK 1.0.0.1: 20.9ms
13:11:18 OK 8.8.8.8: 14.24ms
13:11:19 OK 8.8.4.4: 17.14ms
13:11:20 OK 208.67.222.222: 17.94ms
13:11:21 OK 208.67.220.220: 21.56ms
13:11:23 OK 1.1.1.1: 34.88ms
13:11:24 OK 1.0.0.1: 18.69ms
13:11:25 OK 8.8.8.8: 13.14ms
13:11:26 OK 8.8.4.4: 16.33ms
13:11:27 OK 208.67.222.222: 24.02ms
13:11:28 OK 208.67.220.220: 21.89ms
13:11:29 OK 1.1.1.1: 32.43ms
13:11:30 OK 1.0.0.1: 23.03ms
13:11:31 OK 8.8.8.8: 15.13ms
13:11:32 OK 8.8.4.4: 16.65ms
13:11:33 OK 208.67.222.222: 20.19ms
13:11:34 OK 208.67.220.220: 20.07ms
13:11:35 OK 1.1.1.1: 51.86ms
13:11:36 OK 1.0.0.1: 20.87ms
13:11:37 OK 8.8.8.8: 15.78ms
13:11:38 OK 8.8.4.4: 18.24ms
13:11:39 OK 208.67.222.222: 18.08ms
13:11:40 OK 208.67.220.220: 20.19ms
Every failed ping gets logged with timestamp in a log.txt file.
04 Jan 2023
Updated code:
from datetime import datetime
from ping3 import ping, verbose_ping
import time
print("----------")
hostnames = [
'208.67.222.222', # OpenDNS
'208.67.220.220', # OpenDNS
'1.1.1.1', # Cloudflare
'1.0.0.1', # Cloudflare
'8.8.8.8', # Google DNS
'8.8.4.4', # Google DNS
]
run_time = 24 # in hours, as integer
log_file = 'log/log.txt'
def log_message(s):
global log_file
with open(log_file, 'a') as file:
file.write(s)
loops = int((run_time * 60 * 60) / len(hostnames))
print(f"\nRunning {loops} loops...\n")
log_message(f"\n\n---------- STARTING {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
for i in range(0,loops): # 14400 seconds in 24h
for host in hostnames:
p = ping(host)
if p != None:
p_format = round(p*1000, 2)
print(f"{i}/{loops} {datetime.now().strftime('%H:%M:%S')}\tOK\t{host}\t{p_format}ms")
else:
print(f"{i}/{loops} {datetime.now().strftime('%H:%M:%S')}\tNOK\t{host}\t{p}")
log_message(f"\n{datetime.now().strftime('%H:%M:%S')}\tNOK\t{host}: {p}")
time.sleep(1)
print()
log_message(f"\n\n---------- STOPPED {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
print()
Traceroute
06 Sep 2022 ideally, I would want the script to figure out the IP addresses of the devices to ping on the internal network. Traceroute seems to be the way to go.
What is Traceroute: What Does it Do & How Does It Work?
fortinet
Terminal command:
traceroute hostname
Traceroute read:
| hop number | device name | device IP | hop time (back and forth) |
|---|---|---|---|
| 1 | router | (192.168.0.1) | 0.841 ms 0.662 ms 0.639 ms |
| 2 | xxxxxxxx.xxxxx.xxxxxx.com | (xx.xx.xx.xx) | 14.854 ms 13.750 ms 13.516 ms |
Asterisk in front of device name: indicates that the router it reached was configured to deprioritize or automatically reject ICMP packets, which is done because ICMP is not categorized as essential traffic by many routers.
“If there is an issue, you can use Simple Network Management Protocol (SNMP) to diagnose the problem.”
Working code
Divided in two parts:
ping_test.pyspeed_test.py
ping_test.py
Working (basic) script for monitoring internet connection.
Can be run for a specified amount of time, looping hosts to ping every x seconds.
Logs ping errors to a txt file.
I use it mainly as needed for quick checks, launching the script via a keyboard shortcut.
Variables:
run_time = 24 # in hours, as integer
ping_wait = 1 # in seconds, as integer
log_file = '/path/to/log/file.txt'
hostnames = {
'208.67.222.222': 'OpenDNS', # OpenDNS
'208.67.220.220': 'OpenDNS', # OpenDNS
'1.1.1.1': 'Cloudflare', # Cloudflare
'1.0.0.1': 'Cloudflare', # Cloudflare
'8.8.8.8': 'Google', # Google
'8.8.4.4': 'Google', # Google
}
prints as:
Running 14400 loops...
0/14400 17:46:42 ✅ 208.67.222.222/OpenDNS 22.01ms
0/14400 17:46:43 ✅ 208.67.220.220/OpenDNS 23.97ms
0/14400 17:46:44 ✅ 1.1.1.1/Cloudflare 20.11ms
0/14400 17:46:45 ✅ 1.0.0.1/Cloudflare 23.05ms
0/14400 17:46:46 ✅ 8.8.8.8/Google 13.31ms
0/14400 17:46:47 ✅ 8.8.4.4/Google 15.93ms
1/14400 17:46:48 ✅ 208.67.222.222/OpenDNS 21.93ms
1/14400 17:46:49 ✅ 208.67.220.220/OpenDNS 19.6ms
1/14400 17:46:50 ✅ 1.1.1.1/Cloudflare 25.94ms
1/14400 17:46:51 ✅ 1.0.0.1/Cloudflare 19.57ms
1/14400 17:46:52 ✅ 8.8.8.8/Google 16.17ms
1/14400 17:46:53 ✅ 8.8.4.4/Google 17.55ms
Full code:
### Working code for pinging a list of hosts every x second
from ping3 import ping
import time
from datetime import datetime
print("----------\n")
run_time = 24 # in hours, as integer
ping_wait = 1 # in seconds, as integer
log_file = '/Users/nic/Python/pingee/log/log.txt'
hostnames = {
'208.67.222.222': 'OpenDNS', # OpenDNS
'208.67.220.220': 'OpenDNS', # OpenDNS
'1.1.1.1': 'Cloudflare', # Cloudflare
'1.0.0.1': 'Cloudflare', # Cloudflare
'8.8.8.8': 'Google', # Google
'8.8.4.4': 'Google', # Google
}
def log_message(s):
global log_file
with open(log_file, 'a') as file:
file.write(s)
loops = int((run_time * 60 * 60) / len(hostnames))
print(f"\nRunning {loops} loops...\n")
log_message(f"\n\n---------- STARTING {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
for i in range(0,loops): # 14400 seconds in 24h
for host,hostname in hostnames.items():
p = ping(host)
# print(f"\n{p=}")
if p != None:
p_format = round(p*1000, 2)
if hostname == 'OpenDNS':
print(f"{i}/{loops}\t{datetime.now().strftime('%H:%M:%S')}\t✅\t{host}/{hostname}\t{p_format}ms")
if hostname == 'Cloudflare':
print(f"{i}/{loops}\t{datetime.now().strftime('%H:%M:%S')}\t✅\t{host}/{hostname}\t{p_format}ms")
if hostname == 'Google':
print(f"{i}/{loops}\t{datetime.now().strftime('%H:%M:%S')}\t✅\t{host}/{hostname}\t\t{p_format}ms")
else:
p = '-'
if hostname == 'OpenDNS':
print(f"{i}/{loops} {datetime.now().strftime('%H:%M:%S')}\t❌\t{host}/{hostname}\t{p}")
if hostname == 'Cloudflare':
print(f"{i}/{loops} {datetime.now().strftime('%H:%M:%S')}\t❌\t{host}/{hostname}\t{p}")
if hostname == 'Google':
print(f"{i}/{loops} {datetime.now().strftime('%H:%M:%S')}\t❌\t{host}/{hostname}\t\t{p}")
log_message(f"\n{datetime.now().strftime('%H:%M:%S')}\t❌\t{host}/{hostname}: {p}")
time.sleep(ping_wait)
print()
log_message(f"\n\n---------- STOPPED {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n\n")
speed_test.py
Using the speedtest-cli Python library, looping download tests.
Variables:
tests_to_run = 3
threads = 1 # 1 simulates a typical file transfer, else None
log_file = '/path/to/log.csv'
prints as:
Starting speed test with 3 runs (threads=1)...
#1 [10:19:44] 60.85 Mbit/s (11.3s test time)
#2 [10:19:55] 72.8 Mbit/s (11.3s test time)
#3 [10:20:06] 25.89 Mbit/s (11.4s test time)
and logs to CSV file.
Reference ping times
Here are the guidelines for required ping time when playing online:
Winning: 0-59 ms
In the game: 60-129 ms
Struggling: 130-199 ms
Game over: 200+ ms
Commercial alternatives
PingPlotter
Graphical Network Monitoring and Troubleshooting | PingPlotter
Legendary Ping and Traceroute Solutions
➤ SaaS
➤ $20 one-off for 28 days use. Or $29/month subscription.
Path Analyzer Pro
Path Analyzer Pro - Graphical Traceroute, WhoIs, Charts, Maps, Performance Testing, ip location, tracert, trace route
Path Analyzer Pro - the ultimate traceroute
➤ macOS app.
➤ starts at $29 (one-time fee) for personal use, or $89 for business use.
Pingdom
WebsitePerformance and Availability Monitoring | Pingdom - Pingdom
Simplified End User Experience Monitoring