Detect/notify when web page etag changes
Go to file
Emil Lerch 52eb054277
All checks were successful
continuous-integration/drone/push Build is passing
add lambda-layer implementation based on AL2/Python 3.8
2021-01-06 10:23:15 -08:00
.dockerignore add dockerignore 2021-01-05 15:11:49 -08:00
.drone.yml first makefile attempt 2021-01-04 18:23:42 -08:00
.gitignore add lambda-layer implementation based on AL2/Python 3.8 2021-01-06 10:23:15 -08:00
aws-flow.txt document usage of etags in an aws flow for ungoogled chromium 2020-11-25 12:28:39 -08:00
Dockerfile document why we use find 2021-01-06 10:22:41 -08:00
etags.py add changed data to output for command line processing 2021-01-05 23:26:33 -08:00
LICENSE add license 2020-11-25 11:59:55 -08:00
Makefile add lambda-layer implementation based on AL2/Python 3.8 2021-01-06 10:23:15 -08:00
pyvenv.cfg add pyvenv.cfg 2020-11-25 12:26:02 -08:00
README.md add README 2021-01-05 23:24:23 -08:00
requirements.txt add requirements.txt 2020-11-25 12:26:39 -08:00

etags

Checks urls for etags and reports back any changed urls. Requires DynamoDb, or something like it. For a lightweight, self-hosted version, you can try DynamoDb Bolt.

Usage: etags.py <url> ...

Environment variables

  • ETAGS_BUS_NAME: If set, this will notify on an EventBridge bus
  • ETAGS_TABLE: Table name for DynamoDB
  • DDB_ENDPOINT: By default, the application will use DynamoDb's standard endpoint Set this variable if using a non-standard endpoint or DynamoDb Bolt

This uses boto3, so all AWS Environment Variables will control authentication. If using DynamoDb Bolt, the following must be set, but can be dummy values:

  • AWS_DEFAULT_REGION
  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY

Running as Docker

Latest version can be found at https://r.lerch.org/repo/etags/tags/. Versions are tagged with the short hash of the git commit, and are built as a multi-architecture image based on a scratch image.

You can run the docker image with a command like:

docker run \
  --rm                              \
  --tmpfs /tmp                      \
  --name=ddbbolt                    \
  -e AWS_DEFAULT_REGION=us-west-2   \
  -e AWS_ACCESS_KEY_ID=AKIAEXAMPLE  \
  -e AWS_SECRET_ACCESS_KEY=dummy    \
  -e DDB_ENDPOINT=set_if_applicable \
  -e ETAGS_TABLE=etags              \
  r.lerch.org/etags:0af6716