Monitoring WebSites with Python

#Python #PythonAutomation

Mohamed El Eraki's photo
Mohamed El Eraki
·

4 min read

Monitoring WebSites with Python

Table of contents

Overview

In this example, will type down a script that fetches a WebSite page status code by using the Requests library, Requests allows you to send HTTP/1.1 requests extremely easily, Without manually adding query strings to your URLs; Just imagine The Library simplicity, you can set conditions based on the result status code (i.e. 200, 404, etc.)

Even With some help from other libraries (e.g. paramiko,) you can SSH on the server and fix the issue (e.g. restart the Web App,) At the end you'll find out that you craft a whole automated process action based on the website status code result.

Script steps explanation:

  • Run the website by using Nginx container docker run nginx

  • Using try and except for error handling.

  • Get the website status code result.

  • If the result does not match 200 (which means "OK") Take action by sending an email.

  • If the result does not match any of the HTTP response statuses (i.e. 200, 300, 400, etc) and instead gives an error, will use except to set an action that will be SSH and restart the Nginx container.

Gmail account setup

Script Code

import os
import requests
import smtplib
import paramiko

#1
EMAIL_ADDRESS = os.environ.get('EMAIL_ADDRESS_ENV') 
EMAIL_PASSWORD = os.environ.get('EMAIL_PASSWORD_ENV')

#2
def send_notification(email_msg):
    with smtplib.SMTP('smtp.gmail.com', 587) as smtp:
        smtp.starttls()  # set encryption
        smtp.ehlo()  # Identifies our application with the mail server using the encryption that has been set.
        smtp.login(EMAIL_ADDRESS, EMAIL_PASSWORD)  # login using the ENV vars
        message = f"Subject: SiteDown\n{email_msg}"  # set variable that have the the email_msg parameter value
        smtp.sendmail(EMAIL_ADDRESS, EMAIL_ADDRESS, message)  # start sending by pass >> sender=EMAIL_ADDRESS, receiver=EMAIL_ADDRESS, and the message

#3
try:
    response = requests.get('http://Example.com')  # get the website code status result
    if response.status_code == 200:  # if Okay print the below statement
        print('Application is running successfully!')
    else:  # if not Send an email
        print('Application down, Fix it!')
        msg = f"Subject: Site Down\nApplication return {response.status_code}. Fix the issue!"  # create a var that have response status code to pass it to send_notification function
        send_notification(msg)

        # Restart the application, SSH connection
        ssh = paramiko.SSHClient()  # call the ssh function
        ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())  # the connection for the 1st time will ask for set messing host key policy this step to configure this

        # ssh connection with private key
        ssh.connect(hostname='139.162.130.236', username='root', key_filename='/home/mohamed/.ssh/id_rsa')

        # execute command
        stdin, stdout, stderr = ssh.exec_command('docker start CONTAINER_ID_HERE')  # to get the input value, the output, and the error msg
        print(stdin)
        print(stdout.readlines())
        ssh.close()
        print('Application restarted')

except Exception as ex:  # catch the err in ex
    print(f'connection error happened {ex}')  # print out the error
    msg = "Subject: SiteDown\napplication not accessible at all Fix the issue!"   # create a var that have response status code to pass it to send_notification function
    send_notification(msg)
     # Restart the application, SSH connection
    ssh = paramiko.SSHClient()  # call the ssh function
    ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())  # the connection for the 1st time will ask for set messing host key policy this step to configure this

        # ssh connection with private key
    ssh.connect(hostname='139.162.130.236', username='root', key_filename='/home/mohamed/.ssh/id_rsa')

        # execute command
    stdin, stdout, stderr = ssh.exec_command('docker start CONTAINER_ID_HERE')  # to get the input value, the output, and the error msg
        print(stdin)
        print(stdout.readlines())
        ssh.close()
        print('Application restarted')

Code Explanation

#1

  • In the beginning, will need to set up our Gmail to use in Python for sending emails from

  • Go to your Gmail account and set up an "app passwords"

  • Create an environment variable on your device called EMAIL_PASSWORD_ENV that has the generated password, by Adding the following values in ~/.bash_profile for Redhat ~/.profile for ubuntu
    export EMAIL_PASSWORD_ENV=mohamed_test
    Save and restart the server.

  • Create an environment variable on your device called EMAIL_ADDRESS_ENV that has your email username value.

  • get the environment variable by using the following command: EMAIL_ADDRESS = os.environ.get('EMAIL_ADDRESS_ENV')

#2

  • Create a function for sending emails and will call it in other places

  • View the comments for code explanation.

  • if you're faced with any issue related to environment variables, try to set variables with the username and password in your script just for testing

#3

  • In this step will use try and except for error handling, Try_and_except Docs.

  • View comments for code explanation.

Summary

  • Fetch the WebPage status code result.

  • If the webpage results work as expected "200" prints a message, if not will send an email and restart the container.

  • Except will catch the error result if any in the ex variable, if there's no result like 200, 404 instead gives you an error message, send an email and restart the container.

Python