<<< Publish WeeWX to S3

Add S3 Upload to WeeWX

The final step in adding an S3 upload to WeeWX is to make a number of modifications to WeeWX, these are:

The only other thing that I have done is to modify the Standard skin.conf to reduce the amount of data that is produced, for example I no longer create skins for mobile phones. This is becasue the S3 charges are related to volume of storage and the number of PUTS that you perform so I have refined the report to optimise this.

My website can be found at my domain name, hosted on S3, https://weather.cougar.eu.com

or at the S3 address of https://weather.cougar.eu.com.s3-website-us-east-1.amazonaws.com

Granting root Access to S3

In the previous step we setup S3 access for our main Rasberry Pi users, however the WeeWX program is run by a seperate privileged account, in the case of the Rasberry Pi this is root and so for our process to work this account needs access to S3

This is as simple as copying the configuration files, we previously created, into the directory structure of our privileged account

pi@weewx: $ sudo su -
pi@weewx: $ cp -R /home/pi/.aws ~/.aws
pi@weewx: $ aws s3 ls s3://mikes-weewx --profile weewx
2020-12-31 11:49:01         88 index.html

This lets you query the file we copied up previously

Adding a new skin

Not as diffictult as it sounds

  1. Go to the destination directory as defined by WEEWX_ROOT in weewx.conf
  2. Create a new directory and give it a unique name I used S3
pi@weewx: $ cd /home/weewx/skins
pi@weewx: $ ls -al
total 40
drwxr-xr-x 10 pi root 4096 Mar 23  2020 .
drwxr-xr-x  8 pi root 4096 Dec 27 10:42 ..
drwxr-xr-x  2 pi root 4096 Mar 22  2020 Ftp
drwxr-xr-x  2 pi root 4096 Mar 22  2020 Mobile
drwxr-xr-x  2 pi root 4096 Mar 22  2020 Rsync
drwxr-xr-x  5 pi root 4096 Dec 29 09:40 Seasons
drwxr-xr-x  3 pi root 4096 Mar 22  2020 Smartphone
drwxr-xr-x  6 pi root 4096 Mar 22  2020 Standard

pi@weewx: $ mkdir S3

pi@weewx: $ ls -al
total 40
drwxr-xr-x 10 pi root 4096 Mar 23  2020 .
drwxr-xr-x  8 pi root 4096 Dec 27 10:42 ..
drwxr-xr-x  2 pi root 4096 Mar 22  2020 Ftp
drwxr-xr-x  2 pi root 4096 Mar 22  2020 Mobile
drwxr-xr-x  2 pi root 4096 Mar 22  2020 Rsync
drwxr-xr-x  2 pi pi   4096 Mar 23  2020 S3
drwxr-xr-x  5 pi root 4096 Dec 29 09:40 Seasons
drwxr-xr-x  3 pi root 4096 Mar 22  2020 Smartphone
drwxr-xr-x  6 pi root 4096 Mar 22  2020 Standard
  1. Add a new file, skin.conf, to that directory

This can be downloaded from here or you can create your own from the template below

pi@weewx: $ cd /home/weewx/skins/S3

pi@weewx: $ vi skin.conf
###############################################################################
# Copyright (c) 2018 Mike Revitt <mike@cougar.eu.com>                         #
# With credit to Tom Keffer <tkeffer@gmail.com>                               #
#                                                                             #
# AWS S3 CONFIGURATION FILE                                                   #
#   This 'report' does not generate any files. Instead, we use the report     #
#   engine to invoke aws cli, which synchronizes files between two locations. #
###############################################################################

[Generators]
    generator_list = user.s3upload.S3UploadGenerator

This file tells the WeeWX program that this skin is a generator and it is to call the S3UploadGenerator class from within the s3upload python script in the user directory under bin

Copy the S3 Python script into the bin user directory under your WeeWX home directory

Next you need to download the S3 python script s3upload.py and copy it into the user directory below bin in your WeeWX Home directory area

pi@weewx: $ cp s3upload.py /home/weewx/bin/user
pi@weewx: $ ls -al /home/weewx/bin/user
total 48
drwxr-xr-x  3 pi root  4096 Mar 23  2020 .
drwxr-xr-x 12 pi root  4096 Mar 28  2020 ..
-rw-r--r--  1 pi root   829 Mar 28  2020 cputemp.py
-rw-r--r--  1 pi root   541 Mar 14  2020 extensions.py
-rw-r--r--  1 pi root   306 Mar 14  2020 __init__.py
drwxr-xr-x  2 pi root  4096 Mar 29  2020 __pycache__
-rwxr-xr-x  1 pi pi    7327 Mar 28  2020 s3backup.py
-rwxr-xr-x  1 pi pi   11041 Mar 28  2020 s3upload.py
-rwxr-xr-x  1 pi pi    3992 Mar 28  2020 stats.py

Modifying Weewx.conf

Now that we have created a new skin and copied the new Python script into the WeeWX directory structure, we need to tell the reports engine to use this skin and Python script, to do this you need to edit weewx.conf and make the following changes

And again, I know there are some fancy ways of doing this programatically, but I simply edit the file by adding a new entry for my new Python script

Edit weewx.conf, find the [StdReport] section and add the following, I put it after [[RSYNC]] in my file

pi@weewx: $ vi weewx.conf
    [[AWSCLI]]
        # Using AWSCLI to copy the results to a webserver is treated as just
        # another report, albeit one with an unusual report generator!
        skin = S3
        enable = true

        # You must configure AWS at the command line and create some logon credentials
        # The logon credentials then go into the credentials file.
        # This is the name of the profile defined in that file that you use for the copy
        AWS_Profile = weewx

        # You must configure AWS at the command line and create some logon credentials
        # The logon credentials then go into the credentials file.
        # This is the name of the region defined in that file where the files are located
        AWS_Region = us-east-1

        # This is the name of the S3 bucket to where the files will be copied
        S3_BUCKET = mikes-weewx

In this file we are adding several values which will be picked up by the python script

And that is it, now every time WeeWX runs it will call the new Python script and copy your files upto S3 from where you can access your website, to test the result, follow the instructions in Enable S3 where we setup Hello World