79 Commits

Author SHA1 Message Date
asif 29a5539949 feat(systemd): Update systemd unit-file
Update systemd unit-file to support v3 parameters. Use journald logging as the template example.
2026-07-25 04:33:35 -06:00
asif 0fe3535328 chore(ide): Delete notebook 2026-07-25 04:16:23 -06:00
asif 30bd4d230d docs(logwatch): Update readme
Update readme with changes for v3
2026-07-25 04:11:32 -06:00
asif 2d0188949b chore(ide): Update dictionary 2026-07-25 04:11:05 -06:00
asif c53b89667f feat(logwatch): Update logwatch files for v3
Add new timestamp template and delete the old one. Update report script file to use new 'status' tags to generate logwatch reports.
2026-07-25 03:46:41 -06:00
asif 9654bf7472 feat(log): Add 'SUCCESS' status
Tag successfully updated entries with a 'SUCCESS' tag instead of 'OK' for better log filtering and to allow for logwatch reporting.
2026-07-25 03:45:44 -06:00
asif d20be3e602 docs(readme): Update readme with v3 changes 2026-07-25 03:23:49 -06:00
asif 563ded005f docs(script): Fix script examples
Script examples referenced old log-file switches.
2026-07-25 01:52:27 -06:00
asif 977827b92d feat(logger): Remove unneeded file
This file was a test-file while testing different methods of writing to the journal. No longer needed.
2026-07-25 01:50:22 -06:00
asif 9ed3d1adc1 docs(script): Update embedded script examples
Proofreading. Add examples of using new log backends and formats.
2026-07-25 01:28:47 -06:00
asif 8540fe16b2 fix(log): Disallow multi-logging
Make logging options mutually-exclusive to avoid confusion.
2026-07-25 01:10:48 -06:00
asif 1fb44db8df chore(ide): Update notebook settings 2026-07-25 01:10:16 -06:00
asif 1144156613 feat(func): Write to journald
Function to write log entries to journald. Update writeLog steering function to use this new backend.
2026-07-25 00:45:18 -06:00
asif 45e5e17513 feat(func): Translate standardized log levels to journald 2026-07-25 00:42:06 -06:00
asif a92b341bc3 feat(log): merge feat/structured-logging
Merge structured logging feature branch into main.

Closes 2
2026-07-25 00:20:04 -06:00
asif 08a5c823d5 refactor(log): Use CLEF for JSON logs
Use Compact Log Event Format (CLEF) for JSON logs along with custom properties.
2026-07-24 04:41:23 -06:00
asif eafba0d67d perf(log): Use compact JSON for logging 2026-07-24 04:36:26 -06:00
asif e25bd9fb4d feat(func): Add JSON logging func
Update central logging steering function to reference this new JSON logging backend function.
2026-07-24 04:02:21 -06:00
asif bff59f5c1f chore(ide): Save custom dictionary 2026-07-24 04:01:35 -06:00
asif 539b80e4ec feat(script): Use HTTPS ip discovery links 2026-07-24 03:52:30 -06:00
asif 58520b3567 refactor(log): Remove redundant debug log handling
Debug-level logging is handled by the central logging steering function. Remove redundant code in backend plaint-text logging function.
2026-07-24 03:28:49 -06:00
asif 51bd6160f3 feat(func): Use centralized log func
Create single logging function that can be called and then forwards the request to the appropriate logging backend function for completion.
2026-07-24 03:25:29 -06:00
asif 8d3489ba00 fix(vars): Add missing LOG_PROGRAM_NAME var 2026-07-24 02:48:26 -06:00
asif 3b97d112ad feat(log): Add CF-ERR to operation attr
Update 'listCFErrors' function so that 'CF-ERR' is appended to the 'operation' field of the error log.
2026-07-24 02:44:22 -06:00
asif 1506d77ee9 refactor(func): Remove no-colourize functions 2026-07-24 02:17:06 -06:00
asif e4e967e227 feat(params): Cleanup journald pre-req fail msg 2026-07-24 02:16:08 -06:00
asif aebf8042e3 feat(params): Use badparam func to manage logging 2026-07-24 02:15:14 -06:00
asif 02e3e8d391 feat(log): Log parameter errors 2026-07-24 01:59:10 -06:00
asif 95682314de fix(params): Fix pre-flight console log dedup 2026-07-24 01:46:51 -06:00
asif 5bc3e7d587 feat(log): Enable debug level filtering 2026-07-24 01:18:16 -06:00
asif fca36ac119 feat(params): Add debug logging toggle 2026-07-24 01:13:40 -06:00
asif 2b0128d0c1 fix(log): Prevent preflight duplicate logging
Prevent writing preflight error messages to the console twice when using console logging mode.
2026-07-24 01:08:54 -06:00
asif c792ec8a0a feat(params): Add log type, fmt params
Add log type and format parameters. Update defaults. Update help script.
2026-07-24 01:03:28 -06:00
asif 1732de343d refactor(func): Remove unused functions
Remvoe the logging functions replaced by the new structured implementation.
2026-07-23 23:22:17 -06:00
asif 51b2b455a2 feat(log): Use plain-text logging for all messages
Convert all logging output to plain-text structured messages using new function.
2026-07-23 23:16:12 -06:00
asif f8f3bf1539 refactor(func): Change default err code to 0 2026-07-23 22:17:15 -06:00
asif f8908d7830 feat(func): Write modified syslog fmt msg
Take input parameters and generate a modified syslog-style log entry.
2026-07-23 21:59:58 -06:00
asif 8a5a72e15a feat(func): Convert to uniform log levels
Take input text with various references to log levels and return a standardized uppercase log level name.
2026-07-23 21:57:30 -06:00
asif 1da49b06db feat(func): Convert text ucase, lcase 2026-07-23 21:56:19 -06:00
asif 1336c8e63b feat(func): Retrieve machine host name 2026-07-23 21:55:56 -06:00
asif 450733a8d5 feat(func): Generate ISO-8601 timestamp 2026-07-23 21:55:25 -06:00
asif 92e69d1b85 fix(logger): Fix JournalD log levels
Log levels were reversed in the translation function.
2026-07-23 00:20:30 -06:00
asif 00ea88780d feat(logger): Echo vs return values from Translate func 2026-07-23 00:18:03 -06:00
asif d9aa8973a3 fix(logger): Perform cmd subst outside heredoc
Perform function and command value substitution outside the heredoc forwarded to logger for JournalD entries.
2026-07-23 00:17:42 -06:00
asif ec75cffbf1 fix(logger): Ensure log levels are lcase before standardization 2026-07-23 00:16:19 -06:00
asif b8fb744623 fix(logger): Fix error in json logger
Assignment to 'pid' argument was incorrect.
2026-07-22 23:50:33 -06:00
asif d0b2c781bf feat(logger): Update test cases
Update JSON and JOURNALD tests to match text tests and use new functions.
2026-07-22 23:48:16 -06:00
asif c4d28d1f8c fix(logger): Fix default subst
Default parameter substitution in all functions was incorrect.
2026-07-22 23:44:08 -06:00
asif f085838781 feat(logger): Add test with operation, error code 2026-07-22 23:42:52 -06:00
asif 4a6c339a07 fix(logger): Fix default subst
Default variable substitution in plain-text function was incorrect.
2026-07-22 23:42:33 -06:00
asif c94f12e976 fix(logger): Fix error in uppercase func
'upper' and 'lower' translate strings were not supplied separately.
2026-07-22 23:38:45 -06:00
asif 5443612381 feat(logger): Add plain-text test output 2026-07-22 23:35:08 -06:00
asif df144a5211 fix(logger): Remove errant 'fi' statements
Remove 'fi' statements left over from previous logic checks.
2026-07-22 23:33:49 -06:00
asif 04811805c3 feat(logger): Remove unused global vars 2026-07-22 23:25:28 -06:00
asif aa446c00e6 feat(logger): Update journal logging function
Use new standardized format fields.
2026-07-22 23:24:52 -06:00
asif 56a5ad59fe refactor(logger): Rename writeJournalLog function
Rename from 'writeJournalLog' to 'writeJournalDLog' for specificity.
2026-07-22 23:19:51 -06:00
asif ce1ab86d22 fix(logger): Fix JSON logger
Level and status were swapped. JSON output was missing new prefix information (timestamp, hostname, etc.)
2026-07-22 23:18:36 -06:00
asif cae57ccf6b feat(logger): Use standardized log levels for JSON 2026-07-22 23:15:44 -06:00
asif 0fcd0d9c13 refactor(logger): Use standardized log levels for plain-text 2026-07-22 23:15:11 -06:00
asif e411a35b93 feat(logger): Add log level translator functions 2026-07-22 23:13:58 -06:00
asif 0e787be456 feat(logger): Add lowercase function 2026-07-22 23:06:34 -06:00
asif 17c33d237f fix(logger): Ensure plain-text status is uppercase 2026-07-22 22:54:25 -06:00
asif 5b55ac95a3 feat(logger): Update JSON logger
Require message, level, and status. Reorganize JSON output to mirror modified syslog-style structure.
2026-07-22 22:53:38 -06:00
asif f4dc8f44b1 feat(logger): Handle plain-text insufficient information
Return without logging if message, level and/or status is not supplied.
2026-07-22 22:49:13 -06:00
asif 3792881529 feature(logger): Add plain-text logger function 2026-07-22 22:42:49 -06:00
asif c1f54a619c refactor(logger): Use ISO8601 timestamp 2026-07-22 21:52:53 -06:00
asif 4c997a6b7d feature [logger] add status logging param 2026-07-22 18:34:14 -06:00
asif 1824d9f139 feat(logger): Framework test file 2026-07-22 17:33:46 -06:00
asif 50b4cf105f chore(ide): Update IDE settings 2026-07-22 17:33:36 -06:00
asif 1e3b545a23 chore(ide): JetBrains IDE configuration files 2026-07-19 23:37:13 -06:00
asif 380509c966 chore(git): Update git control files 2026-07-19 23:36:58 -06:00
asif 71da487aa8 style(fmt): Reformat shell script 2026-07-19 23:33:56 -06:00
asif 3be6eaee0c [preflight] Add check for logger dependency
'logger' is required to facilitate logging to the journaling system.
2026-07-19 18:52:34 -06:00
asif 5c6c8ec4ea [params] Create journal logging switch
Create boolean variable to store journal logging preference.
Set to 'false' by default. Add parameter to activate this preference.
Update help to include this new parameter.
2026-07-19 18:49:34 -06:00
asif c7297a512c [version] Update version, modified date 2026-07-19 15:44:40 -06:00
Asif Bacchus fddb406717 fix error in hyperlink 2021-05-10 03:22:22 -06:00
Asif Bacchus 4ec190af47 update readme 2021-05-10 03:18:49 -06:00
Asif Bacchus 57ce2d1ac3 fixed headings so TOC links work 2021-05-10 03:09:30 -06:00
Asif Bacchus 1ffd374a4f fix: set TERM for tput if not set
- stop warning messages in syslog from systemd invocation
2021-05-10 02:55:16 -06:00
15 changed files with 1039 additions and 675 deletions
+22 -3
View File
@@ -55,6 +55,23 @@
*.bat text eol=crlf *.bat text eol=crlf
*.cmd text eol=crlf *.cmd text eol=crlf
# web frontend stack -- force LF so SRI hashes are always correct
*.html text eol=lf
*.htm text eol=lf
*.css text eol=lf
*.min.css text eol=lf
*.js text eol=lf
*.min.js text eol=lf
*.php text eol=lf
# Visual Studio projects (Rider also)
*.cs diff=csharp
*.sln merge=union
*.csproj merge=union
*.vbproj merge=union
*.fsproj merge=union
*.dbproj merge=union
# Serialisation # Serialisation
*.json text *.json text
*.toml text *.toml text
@@ -76,6 +93,8 @@
# Exclude files from exporting # Exclude files from exporting
# #
.gitattributes export-ignore .gitattributes export-ignore
.gitignore export-ignore .gitignore export-ignore
.gitkeep export-ignore .gitkeep export-ignore
.idea export-ignore
.vscode export-ignore
+34
View File
@@ -1 +1,35 @@
#
# JetBrains exclusions
#
riderModule.iml
/_ReSharper.Caches/
# User-specific stuff
.idea/**/workspace.xml
.idea/**/tasks.xml
.idea/**/usage.statistics.xml
.idea/**/dictionaries
.idea/**/shelf
# Generated files
.idea/**/contentModel.xml
.idea/**/GitCommitMessageStorage.xml
# Sensitive or high-churn files
.idea/**/dataSources/
.idea/**/dataSources.ids
.idea/**/dataSources.local.xml
.idea/**/sqlDataSources.xml
.idea/**/dynamic.xml
.idea/**/uiDesigner.xml
.idea/**/dbnavigator.xml
# modules
.idea_modules/
# Editor-based Rest Client
.idea/httpRequests
# project-specific exclusions
*.log *.log
+6
View File
@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="DeveloperToolsToolWindowSettingsV1" lastSelectedContentNodeId="base64-encoder-decoder" pluginVersion="9.0.0">
<developerToolsConfigurations />
</component>
</project>
+15
View File
@@ -0,0 +1,15 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="GitToolBoxProjectSettings">
<option name="commitMessageIssueKeyValidationOverride">
<BoolValueOverride>
<option name="enabled" value="true" />
</BoolValueOverride>
</option>
<option name="commitMessageValidationEnabledOverride">
<BoolValueOverride>
<option name="enabled" value="true" />
</BoolValueOverride>
</option>
</component>
</project>
+8
View File
@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="UserContentModel">
<attachedFolders />
<explicitIncludes />
<explicitExcludes />
</component>
</project>
+8
View File
@@ -0,0 +1,8 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="RiderProjectSettingsUpdater">
<option name="singleClickDiffPreview" value="1" />
<option name="unhandledExceptionsIgnoreList" value="1" />
<option name="vcsConfiguration" value="3" />
</component>
</project>
Generated
+6
View File
@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="VcsDirectoryMappings">
<mapping directory="" vcs="Git" />
</component>
</project>
+6
View File
@@ -0,0 +1,6 @@
<wpf:ResourceDictionary xml:space="preserve" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:s="clr-namespace:System;assembly=mscorlib" xmlns:ss="urn:shemas-jetbrains-com:settings-storage-xaml" xmlns:wpf="http://schemas.microsoft.com/winfx/2006/xaml/presentation">
<s:Boolean x:Key="/Default/UserDictionary/Words/=cfddns/@EntryIndexedValue">True</s:Boolean>
<s:Boolean x:Key="/Default/UserDictionary/Words/=ddns/@EntryIndexedValue">True</s:Boolean>
<s:Boolean x:Key="/Default/UserDictionary/Words/=logwatch/@EntryIndexedValue">True</s:Boolean>
<s:Boolean x:Key="/Default/UserDictionary/Words/=mydomain/@EntryIndexedValue">True</s:Boolean>
<s:Boolean x:Key="/Default/UserDictionary/Words/=myserver/@EntryIndexedValue">True</s:Boolean></wpf:ResourceDictionary>
+213 -104
View File
@@ -1,49 +1,69 @@
# CloudflareDDNS <!-- omit in toc --> # CloudflareDDNS
Update your *existing* Cloudflare DNS records with your current (dynamic) IP address via systemd timers and a (POSIX) shell script. Update your *existing* Cloudflare DNS records with your current (dynamic) IP address via systemd timers and a (POSIX)
shell script.
## Contents <!-- omit in toc --> ## Contents
<!-- toc --> <!-- TOC -->
- [Prerequisites](#prerequisites) * [CloudflareDDNS](#cloudflareddns)
- [cfddns&#46;sh](#cfddns%2346sh) * [Contents](#contents)
* [Installation](#installation) * [Prerequisites](#prerequisites)
* [Usage](#usage) * [cfddns script](#cfddns-script)
+ [Parameters](#parameters) * [Installation](#installation)
- [Cloudflare credentials file](#cloudflare-credentials-file) * [Usage](#usage)
* [File structure](#file-structure) * [Parameters](#parameters)
* [Bearer token](#bearer-token) * [Cloudflare credentials file](#cloudflare-credentials-file)
* [Zone ID](#zone-id) * [File structure](#file-structure)
- [cfddns.service](#cfddnsservice) * [Bearer token](#bearer-token)
* [IP4 and/or IP6](#ip4-andor-ip6) * [Zone ID](#zone-id)
+ [Examples](#examples) * [cfddns systemd service unit](#cfddns-systemd-service-unit)
- [cfddns.timer](#cfddnstimer) * [IP4 or IP6](#ip4-or-ip6)
* [Activation](#activation) * [Examples](#examples)
- [Logging](#logging) * [cfddns systemd timer unit](#cfddns-systemd-timer-unit)
* [Using Logwatch to monitor this script](#using-logwatch-to-monitor-this-script) * [Activation](#activation)
* [Using Logrotate to control log file size](#using-logrotate-to-control-log-file-size) * [Logging](#logging)
- [Final thoughts](#final-thoughts) * [JSON format](#json-format)
* [SYSLOG format](#syslog-format)
* [Systemd journal (journald)](#systemd-journal-journald)
* [Using Logwatch to monitor this script](#using-logwatch-to-monitor-this-script)
* [Using Logrotate to control log file size](#using-logrotate-to-control-log-file-size)
* [Final thoughts](#final-thoughts)
<!-- tocstop --> <!-- TOC -->
## Prerequisites ## Prerequisites
This script requires that `curl` and `jq` are installed. `curl` is used to interact with the Cloudflare API and `jq` is used to efficiently and reliably construct/deconstruct the JSON strings and arrays which is how the Cloudflare API communicates. In most cases you can install these programs using your package manager running as root or via sudo. On Debian/Ubuntu, for example, you would run: This script requires that `curl` and `jq` are installed. `curl` is used to interact with the Cloudflare API and `jq` is
used to efficiently and reliably construct/deconstruct the JSON strings and arrays returned by the Cloudflare API. If
you want to log to the system journal (journald), then `logger` must also be installed. In most cases you can install
these programs using your package manager running as root or via sudo:
```bash ```sh
apt install -y curl jq # Debian/Ubuntu
sudo apt update && sudo apt install -y curl jq bsdutils
# Arch
sudo pacman -Sy curl jq logger
# Redhat/CentOS/Fedora
sudo dnf install curl jq util-linux
# Redhat/CentOS/Fedora (older versions)
sudo yum install curl jq util-linux
``` ```
While the script does *not* require root privileges, you will need sudo/root access to install the *systemd* service and timer. > While the script does *not* require root privileges, you will need sudo/root access to install the *systemd* service
> and timer.
## cfddns&#46;sh ## cfddns script
### Installation ### Installation
I recommend putting this script in your */usr/local/bin* directory or somewhere else in your path so it's easy to run. I recommend putting this script in your */usr/local/bin* directory or somewhere else in your path so it's easy to run.
1. Copy the script file to your desired path and rename if you want. 1. Copy the script file to your desired path and rename it if you want.
```Bash ```Bash
sudo cp cfddns.sh /usr/local/bin/ # just copy it sudo cp cfddns.sh /usr/local/bin/ # just copy it
@@ -56,11 +76,15 @@ I recommend putting this script in your */usr/local/bin* directory or somewhere
sudo chmod +x /usr/local/bin/cfddns.sh sudo chmod +x /usr/local/bin/cfddns.sh
``` ```
> Note: You can rename *cfddns.sh* to anything you want, the script will auto-update itself. However, you **must** manually update the systemd service file (*cfddns.service*) `ExecStart` line as [explained below](#cfddns.service). > Note: You can rename *cfddns.sh* to anything you want, the script will automatically use its current name. However,
> you **must** manually update the systemd service file (*cfddns.service*) `ExecStart` line
> as [explained below](#cfddns-script).
### Usage ### Usage
If you run the script with no parameters, it will display the help screen. The script accepts several parameters with only one (1) being required. The parameters are summarized below. You can access the help screen and example usage screens by running: If you run the script with no parameters, it will display the help screen. The script accepts several parameters with
only one (1) being required. The parameters are summarized below. You can access the help screen and example usage
screens by running:
```Bash ```Bash
cfddns.sh --help # display help screen cfddns.sh --help # display help screen
@@ -69,35 +93,46 @@ cfddns.sh --examples # show script usage examples
#### Parameters #### Parameters
|Parameter|Description|Default|Required?| | Parameter | Description | Default | Required? |
|---|---|---|:---:| |------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------|:---------:|
|-r<br>--record<br>--records|The fully qualified hostname(s) that should be updated with new IP addresses. You can supply a comma-delimited list (no spaces) or just one. Note that the script can only update *either* A *or* AAAA records during a single run so you may need to batch your hostnames, depending on your set-up.<br>N.B. This script will only update *existing* host records, it will **not** create new ones!|none|YES| | -r<br>--record<br>--records | The fully qualified hostname(s) that should be updated with new IP addresses. You can supply a comma-delimited list (no spaces) or just one. Note that the script can only update *either* A *or* AAAA records during a single run so you may need to batch your hostnames, depending on your set-up.<br>**N.B. This script will only update *existing* host records, it will _not_ create new ones!** | none | YES |
|-c<br>--cred<br>--creds<br>--credentials<br>-f|Full path to your CloudFlare credentials file. This file contains your access token and zone id. See the [relevant section](#cloudflare-credentials-file) of this readme for more information.|scriptPath/cloudflare.credentials|NO| | -c<br>--cred<br>--creds<br>--credentials<br>-f | Full path to your CloudFlare credentials file. This file contains your access token and zone id. See the [relevant section](#cloudflare-credentials-file) of this readme for more information. | scriptPath/cloudflare.credentials | NO |
|-i<br>--ip<br>--ip-address<br>-a<br>--address|The IP address that should be used to update Host A/AAAA records. If you omit this value, the script will attempt to auto-detect your public IP4/IP6 address and use that as appropriate. Use this option to manually force a specific IP to be used or when auto-detection fails. Note that the script does *not* check your IP addresses for correctness or proper form!|IP4 auto-detect|NO| | -i<br>--ip<br>--ip-address<br>-a<br>--address | The IP address that should be used to update Host A/AAAA records. If you omit this value, the script will attempt to auto-detect your public IP4/IP6 address and use that as appropriate. Use this option to manually force a specific IP to be used or when auto-detection fails. Note that the script does *not* check your IP addresses for correctness or proper form! | IP4 auto-detect | NO |
|-4<br>--ip4<br>--ipv4|Update Host A records only (IP4). The script can only update *either* A *or* AAAA records in a single run. If you specify this and also use the IP6 mode switch, the most recent one will take effect.|Enabled, update A records|NO| | -4<br>--ip4<br>--ipv4 | Update Host A records only (IP4). The script can only update *either* A *or* AAAA records in a single run. If you specify this and also use the IP6 mode switch, the most recent one will take effect. | Enabled, update A records | NO |
|-6<br>--ip6<br>--ipv6|Update Host AAAA records only (IP6). The script can only update *either* A *or* AAAA records in a single run. If you specify this and also use the IP4 mode switch, the most recent one will take effect.|Disabled, update A records|NO| | -6<br>--ip6<br>--ipv6 | Update Host AAAA records only (IP6). The script can only update *either* A *or* AAAA records in a single run. If you specify this and also use the IP4 mode switch, the most recent one will take effect. | Disabled, update A records | NO |
||||| | | | | |
|-l<br>--log|Full path where the script should save its log. Recommend */var/log/scriptName.log*|scriptPath/scriptName.log|NO| | --log-file | Generate a log file and save it in the specified location. Only specify this option if you want to save the log file in a non-default location. If you want to save the log in the default location, omit this option. Recommend */var/log/scriptName.log* | scriptPath/scriptName.log | NO |
|--nc<br>--no-color<br>--no-colour|Do not use ANSI colour-coding when writing to the log. This is useful if you review the logs using a reader that does not support ANSI colour-coding and instead displays control symbols which makes your log difficult to read.|Disabled, do colourful logs|NO| | --log-journal | Log to the system journal (journald) instead of a log file. Log entries are automatically tagged 'CFDDNS' for easier filtering. Format tags do NOT apply when using this option. Use `json` or `pretty-json` with `journalctl` to see all logged properties. | Disabled, write to log file | NO |
|--log-console|Output the log to the console instead of a log file. You may use `--nc` with this option also.|Disabled, write to log file|NO| | --log-console | Write log entries to the console (stdout) instead of a log file. Using `--fmt-syslog` with this option is strongly recommended since JSON is difficult to read on the console. | Disabled, write to log file | NO |
|--no-log|Do not write a log file or output to the console. You will not have **any** feedback from the script if you run in this mode so you will not know if updates were successful or not. Im not really sure why youd want this option, but its available.|Disabled, write to log file|NO| | --no-log | Do *not* write a log file, write to the journal, or output to the console. You will not have **any** feedback from the script if you run in this mode so you will not know if updates were successful or not. Im not really sure why youd want this option, but its available. | Disabled, write to log file | NO |
|-h<br>--help<br>-?|Display built-in help screen explaining these same parameters.||| | | | | |
|--examples|Display some usage examples. Sometimes it's just easier to understand by seeing rather than reading.||| | --fmt-json | Write the log file or console output in a structured *Compact Log Event Format (CLEF)* JSON format. You will probably need a log viewer to meaningfully parse the logs. | Enabled | NO |
| --fmt-syslog | Write the log file or console output in a modified plain-text *syslog* style. This option is recommended when using `--log-console` or if you need easily readable log files. While all log entries are *consistent* using this format, they are not *structured*. | Disabled, just JSON format | NO |
| | | | |
| -h<br>--help<br>-? | Display built-in help screen explaining these same parameters. | | |
| --examples | Display some usage examples. Sometimes it's easier to understand by seeing rather than reading. | | |
## Cloudflare credentials file ## Cloudflare credentials file
This repo includes a sample credentials file (*cloudflare.credentials* at the root of the repo) with pretty self-explanatory variable names. The script reads this file to get the credentials it needs to connect to your Cloudflare account and update DNS entries. It should be noted that the script is designed to use a *bearer token* and **not** your username/password or your Global API token! Lets break this down in case things are a little fuzzy still... This repo includes a sample credentials file (*cloudflare.credentials* at the root of the repo) with pretty
self-explanatory variable names. The script reads this file to get the credentials it needs to connect to your
Cloudflare account and update DNS entries. It should be noted that the script is designed to use a *bearer token* and
**not** your username/password or your Global API token! Lets break this down in case things are a little fuzzy
still...
### File structure ### File structure
The file is a basic shell script variables file. Make sure you **do not** put spaces between the variable name, equal sign and the value. Also, **do not** add any executable code since it will be run! The file should contain values for the following two variables: The file is a basic shell script variables file. Make sure you **do not** put spaces between the variable name, equal
sign and the value. Also, **do not** add any executable code since it will be run! The file should contain values for
the following two variables:
| Variable | Value | | Variable | Value |
| -------- | ------------------------------------------------------------ | |----------|----------------------------------------------------------------------------------------------|
| cfKey | The bearer token granting access to *edit* the DNS records of the zone (domain) in question. | | cfKey | The bearer token granting access to *edit* the DNS records of the zone (domain) in question. |
| cfZoneId | The Cloudflare Zone ID of the zone (domain) you wish to update. | | cfZoneId | The Cloudflare Zone ID of the zone (domain) you wish to update. |
You can add comments if youd like since the script will ignore them. In the end, your file should look something like this, but obviously with your data instead of this nonsense sample information: You can add comments if youd like since the script will ignore them. In the end, your file should look something like
this, but obviously with your data instead of this nonsense sample information:
```ini ```ini
# #
@@ -109,24 +144,44 @@ cfZoneId=83d564234134513245311b23412331dd
``` ```
You can save the file as anything you like and anywhere youd like as long as you inform the script of its location using the `--credentials` parameter. By default, the script will look for a file named *cloudflare.credentials* in the same path as the script. You can save the file as anything you like and anywhere youd like as long as you inform the script of its location
using the `--credentials` parameter. By default, the script will look for a file named *cloudflare.credentials* at the
same path as the script.
Please remember that this file basically contains a password! As a result, it should be protected and access limited to
the root account:
```bash
chown root:root /path/to/cloudflare.credentials
chmod 600 /path/to/cloudflare.credentials
```
### Bearer token ### Bearer token
I chose to use an API bearer token instead of a username/password or Global API token for security reasons. Your username/password and Global API token provide unfettered access to your account so if anyone gets hold of them, they can do anything to your account. An API bearer token, by contrast, can only do what you authorize it to do and you can revoke it at any time. Therefore, I suggest making a bearer token that is based on the “Edit zone DNS” template and restricted to the specific domain/zone you wish to update. Cloudflare provides an [excellent article](https://support.cloudflare.com/hc/en-us/articles/200167836-Managing-API-Tokens-and-Keys) on how to generate this token. I chose to use an API bearer token instead of a username/password or Global API token for security reasons. Your
username/password and Global API token provide unfettered access to your account, so if anyone gets hold of them, they
can do anything to your account. An API bearer token, by contrast, only permits actions you explicitly authorize when
you create it, and you can revoke it at any time. Therefore, I suggest making a bearer token that is based on the
“Edit zone DNS” template and restricted to the specific domain/zone you wish to update. Cloudflare provides
an [excellent article](https://support.cloudflare.com/hc/en-us/articles/200167836-Managing-API-Tokens-and-Keys) on how
to generate this token.
> N.B. This is a breaking change from previous versions of this script! > N.B. This is a breaking change from previous versions of this script!
### Zone ID ### Zone ID
This is required by the Cloudflare API so it knows which zone you are editing and can check the permissions of the bearer token. This script only caters to one zone so likely only one domain per configuration file. If you need to update multiple zones, you can have multiple configuration files and call them as required on separate invocations of the script. This is required by the Cloudflare API so it knows which zone you are editing and can check the permissions of the
bearer token. This script only caters to one zone, so likely only one domain per configuration file. If you need to
update multiple zones, you can have multiple configuration files and call them as required on separate invocations of
the script.
To get your Zone ID, log into your Cloudflare account and open the domain in question. On the overview page, scroll down a bit and look to the right. You will see your Zone ID listed there. Copy that string into your configuration file. To get your Zone ID, log into your Cloudflare account and open the domain in question. On the overview page, scroll down
a bit and look to the right. You will see your Zone ID listed there. Copy that string into your configuration file.
## cfddns systemd service unit
## cfddns.service For the easiest implementation, this file should be copied to your */etc/systemd/system* directory. If you change the
name of the *cfddns.sh* file, you must update the filename in the `ExecStart` line as shown below:
This file **must** be copied to your */etc/systemd/system* directory (or equivalent directory if you're not running Debian/Ubuntu). If you change the name of the cfddns&#46;sh file, you must update the filename in the `ExecStart` line as shown below:
````Ini ````Ini
... ...
@@ -136,32 +191,34 @@ ExecStart=/full/path/to/your/renamed.file -parameter1 -parameter2 -parameter...
... ...
```` ````
Dont forget to reload systemd after copying this file so it is recognized by the system! On most systems you can do this by running the following as root or via sudo: ### IP4 or IP6
```bash The cfddns.service file includes two *ExecStart* lines, one without a specified IP-protocol parameter (default IP4) and
systemctl daemon-reload the other with the -6 (IP6) parameter. The service will run the cfddns&#46;sh script in default (IP4) mode with
``` specified parameters first and then will run the script again in IP6 mode with specified parameters.
### IP4 and/or IP6
The cfddns.service file includes two *ExecStart* lines, one without a specified IP-protocol parameter (default IP4) and the other with the -6 (IP6) parameter. The service will run the cfddns&#46;sh script in default (IP4) mode with specified parameters first and then will run the script again in IP6 mode with specified parameters.
*Note: The parameters *can be different* in each case.* *Note: The parameters *can be different* in each case.*
> **N.B.** This is a `oneshot` type systemd unit-file, so we can have multiple `ExecStart` lines.
#### Examples #### Examples
1. **Only update A records** 1. **Only update A records**
Update *mail<span>.example.com* A record with the current auto-detected public IP4 address of this machine and log results to */var/log/cfddns.log*. Update the `A` record for *mail.example.com* with the current auto-detected public IP4 address of this machine and
log in JSON format (default) to */var/log/cfddns.log*. Note our Cloudflare credentials file is located in the *root
user's home* directory.
```Ini ```Ini
[Service] [Service]
Type=oneshot Type=oneshot
ExecStart=/usr/local/bin/cfddns.sh -c /root/cloudflare.credentials -r mail.example.com -l /var/log/cfddns.log ExecStart=/usr/local/bin/cfddns.sh -c /root/cloudflare.credentials -r mail.example.com --log-file /var/log/cfddns.log
... ...
``` ```
2. **Only update AAAA records** 2. **Only update AAAA records**
Update *git<span>.example.com* and *mail<span>.example.com* AAAA records with the current auto-detected public IP6 address of this machine. Log will be stored in the same directory as the script file (/usr/local/bin). Update `AAAA` records for *git.example.com* and *mail.example.com* with the current auto-detected public IP6 address
of this machine. The log will be stored in the same directory as the script file (default) and will be
JSON-formatted (default). In this case, the Cloudflare credentials file is located in */home/johndoe*.
```Ini ```Ini
[Service] [Service]
@@ -171,21 +228,29 @@ The cfddns.service file includes two *ExecStart* lines, one without a specified
``` ```
3. **Update A records then AAAA records** 3. **Update A records then AAAA records**
Update *mail<span>.example.com* A record with auto-detected public IP4 address of this machine and write to log file stored at */var/log/DDNS_IP4.log*. Then, update both *mail<span>.example.com* and *git<span>.example.com* AAAA records with the specified IP6 address and write to log file at */var/log/DDNS_IP6.log*. Update the `A` record for *mail.example.com* with the auto-detected public IP4 address of this machine. Then, update
`AAAA` records for both *mail.example.com* and *git.example.com* with the *specified IP6 address*. In both cases, we
are logging to the system journal (journald). Also notice that the Cloudflare credentials file is located in
different
places for each script call.
```Ini ```Ini
[Service] [Service]
Type=oneshot Type=oneshot
# update IP4 addresses # update IP4 addresses
ExecStart=/usr/local/bin/cfddns.sh -c /dir1/account.cf -r mail.example.com -l /var/log/DDNS_IP4.log ExecStart=/usr/local/bin/cfddns.sh -c /dir1/account.cf -r mail.example.com --log-journal
# update IP6 addresses # update IP6 addresses
ExecStart=/usr/local/bin/cfddns.sh -6 -c /dir2/cloudflare.details -r mail.example.com,git.example.com --ip fd3f:e6db:9817:df84::a001 -l /var/log/DDNS_IP6.log ExecStart=/usr/local/bin/cfddns.sh -6 -c /dir2/cloudflare.details -r mail.example.com,git.example.com --ip fd3f:e6db:9817:df84::a001 --log-journal
... ...
``` ```
## cfddns.timer ## cfddns systemd timer unit
This is the timer file that tells your system how often to call the *cfddns.service* file which runs the *cfddns&#46;sh* script. By default, the timer is set for 5 minutes after the system boots up (to allow for other processes to initialize even on slower systems like a RasPi) and is then run every 15 minutes thereafter. Remember when setting your timer that Cloudflare limits API calls to 1200 every 5 minutes. Again, for the easiest configuration, this file should be copied to your */etc/systemd/system* directory. This timer
file unit tells your system how often to call the *cfddns.service* file which runs the *cfddns.sh* script. By default,
the timer is set for 5 minutes after the system boots up (to allow for other processes to initialize even on slower
systems like a RasPi) and is then run every 15 minutes thereafter. Remember when setting your timer that Cloudflare
limits API calls to 1200 every 5 minutes.
You can change the timer by modifying the relevant section of the *cfddns.timer* file: You can change the timer by modifying the relevant section of the *cfddns.timer* file:
@@ -195,8 +260,17 @@ OnBootSec=5min
OnUnitActiveSec=15min OnUnitActiveSec=15min
``` ```
*OnBootSec* is how long to wait after the system boots up before executing the *cfddns.service*. *OnUnitActiveSec* will then wait the specified time from that first (after boot) call or after the timer is explicitly started before calling *cfddns.service* again. I recommend setting OnUnitActiveSec to a low value (like 2 minutes) for testing then setting it to a more reasonable time (like 15 *OnBootSec* is how long to wait after the system boots up before executing the *cfddns.service*. *OnUnitActiveSec* will
minutes) after everything is working. then wait the specified time from that first (after boot) call or after the timer is explicitly started before calling
*cfddns.service* again. I recommend setting OnUnitActiveSec to a low value (like 2 minutes) for testing, then setting it
to a more reasonable time (like 15 minutes) after everything is working.
Remember to reload `systemd` after youve copied both the systemd unit and this timer unit. Run the following as root or
via sudo:
```sh
systemctl daemon-reload
```
### Activation ### Activation
@@ -220,43 +294,78 @@ systemctl status cfddns.timer
It is NOT necessary to enable/start the *cfddns.service*, only the timer needs to be active. It is NOT necessary to enable/start the *cfddns.service*, only the timer needs to be active.
Also remember that if you make changes to settings like `OnUnitActiveSec` while testing, or after testing is complete,
you *must* reload the systemd daemon! It will restart the appropriate units for you, and your new settings will take
effect immediately.
## Logging ## Logging
The script logs every major action it takes and provides details on any errors it encounters in the log file (see the [parameters section](#parameters) for details about setting log location and name). If errors are encountered, they are colour coded red and an explanation of the error code is provided. The script logs every major action it takes and provides details on any errors it encounters in the log file. Refer
to the [parameters section](#parameters) for details about setting log location and name. Starting in version 3.0, the
entire logging system outputs structured **JSON** by default.
While the log file is as terse as I felt reasonable, you may still want to configure any log-watch programs to further filter things for you so you don't have to review this log as part of your daily routine. To make that easier, the following conventions are observed in the log file and can be used to program your log-watch system: > PLEASE remember to review your current log review/parsing system if upgrading from previous versions since this is a
> **major** change to how logging works.
- Specific update process errors: **[TIMESTAMP] ERR: message** **IMPORTANT:** This script uses the same log file for its output (unless you specify a different log file each time it
- These can be counted/filtered separately if you only care about update errors and not any other errors. is run) therefore, you should avoid switiching formats within existing log files. For example, if you are using the
- Error messages: **[TIMESTAMP] ERROR: message (code: number)** default log file, and run the script once with `--fmt-syslog` then switch to the default `--fmt-json`, parsing your log
- Only one summary error message will be displayed for any/all update errors. This message contains a tally of failed updates. If you want to count individual update errors, filter for the above process error message format. will become very difficult. If you are switching log formats, it is best to start with a fresh log file.
- While process error messages only relate to updates, these general error messages are logged for a variety of error conditions so its a good idea to include them in any filters.
- Cloudflare API error messages: **[TIMESTAMP] CF-ERR: message (code: cf-error-code)** ### JSON format
- These are only logged when update process errors occur so that you can see exactly what the Cloudflare API is complaining about.
- Specific update process warnings: **[TIMESTAMP] WARN: message** This is the default logging format since it allows for easy parsing and filtering of log entries while still retaining a
- These can be counted/filtered separately from general warning messages. Presently, there are *no* general warning messages. structured format. I chose to implement a modified [Compact Log Event Format (CLEF)](https://clef-json.org) with the
- Warning messages: **[TIMESTAMP] WARNING: message** following format for every message (expanded for readability, the log file is not 'pretty-printed'):
- Summary of each type of warning. Contains a tally of the specific warning.
- Currently, warnings are only issued for hostnames that are not found (i.e. update process warnings). ```json
- Success messages: **[TIMESTAMP] SUCCESS: message** {
- Each successful update generates a success message. There is no process or tally message. "@t": "ISO-8601 timestamp",
- Already up-to-date: **[TIMESTAMP] IP address for {fqdn} is already up-to-date** "@m": "message describing the event",
- Already up-to-date host entries generate a success message but you may still want to filter for them separately using this criteria. "@l": "log level",
- A session log always starts with **[TIMESTAMP] -- Cloudflare DDNS update-script: starting --** "hostname": "hostname of the machine executing the script",
- A successful session log always ends with **[TIMESTAMP] -- Cloudflare DDNS update-script: completed successfully --** "programName": "CFDDNS",
- A session ending with errors always ends with **[TIMESTAMP] -- Cloudflare DDNS update-script: completed with error(s) --** "pid": "PID of the process executing the script",
"status": "status of the operation being logged",
"operation": "short-name of the operation being logged",
"code": "numeric 'exit code' of the operation being logged"
}
```
### SYSLOG format
The SYSLOG format (`--fmt-syslog`) is a modified version of the typical plain-text *nix syslog format. This format is
useful when outputting the log to the console or when you need a very human-friendly format for the log files since you
will be manually parsing them. While the plain-text format is not *structured*, it is *consistent*:
```text
ISO-8601 timestamp hostname CFDDNS[pid]: LOG_LEVEL STATUS MESSAGE (OPERATION:CODE)
```
### Systemd journal (journald)
When using the `--log-journal` option, all logging will be sent to `journald` using its format. All log messages are
tagged `CFDDNS` to make filtering easier. Additional information logged with each entry can be exposed using the `json`
or `json-pretty` output options for `journalctl`.
### Using Logwatch to monitor this script ### Using Logwatch to monitor this script
If you are using the Logwatch package to monitor your system, see the README in the */etc/logwatch* folder for details about the pre-configured service files already done for you :-) If you are using the Logwatch package to monitor your system, see the README in the */etc/logwatch* folder for details
about the pre-configured service files already done for you :-)
### Using Logrotate to control log file size ### Using Logrotate to control log file size
Logrotate is pre-installed on standard Debian/Ubuntu distributions and is a great way to automatically rotate your log files and control how many old logs you keep on your system so they don't accumulate and eat up your disk space. I've included a sample configuration file you can copy to your */etc/logrotate.d/* folder. This file is set up to rotate your logs once a week, keep 3 weeks worth of history (compressed) and delete all logs older than that. The configuration file is located in this git archive at Logrotate is pre-installed on most linux distributions and is a great way to automatically rotate your log
*/etc/logrotate.d/cfddns* and is fully commented to help you customize it to suit your needs. files and control how many old logs you keep on your system so they don't accumulate and eat up your disk space. I've
included a sample configuration file you can copy to your */etc/logrotate.d/* folder. This file is set up to rotate your
logs once a week, keep 3 weeks worth of history (compressed) and delete all logs older than that. The configuration file
is located in this git archive at */etc/logrotate.d/cfddns* and is fully commented to help you customize it to suit your
needs.
## Final thoughts ## Final thoughts
Hopefully this helps you with an easy and reliable way to update your Cloudflare DNS entries with a dynamic IP address. Please feel free to comment and provide feedback and suggestions to make this script better! Hopefully, this helps you with an easy and reliable way to update your Cloudflare DNS entries with a dynamic IP address.
Please feel free to comment and provide feedback and suggestions to make this script better!
Please check out my blog at [https://mytechiethoughts.com](https://mytechiethoughts.com) where I tackle problems like this all the time and find free/cheap solutions to tech problems. Please check out my blog at [https://mytechiethoughts.com](https://mytechiethoughts.com) where I tackle problems like
this all the time and find free/inexpensive solutions to tech problems.
+2 -2
View File
@@ -4,8 +4,8 @@ After=network.target
[Service] [Service]
Type=oneshot Type=oneshot
ExecStart=/usr/local/bin/cfddns.sh -c /path/to/account.details -r server.domain.tld -l /var/log/cfddns.log ExecStart=/usr/local/bin/cfddns.sh -c /path/to/account.details -r server.domain.tld --log-journal
ExecStart=/usr/local/bin/cfddns.sh -6 -c /path/to/account.details -r server.domain.tld -l /var/log/cfddns.log ExecStart=/usr/local/bin/cfddns.sh -6 -c /path/to/account.details -r server.domain.tld --log-journal
[Install] [Install]
WantedBy=multi-user.target WantedBy=multi-user.target
+542 -438
View File
File diff suppressed because it is too large Load Diff
+156 -100
View File
@@ -1,50 +1,66 @@
# Using Logwatch to monitor Cloudflare DDNS updater script <!-- omit in toc --> # Using Logwatch to monitor the Cloudflare DDNS updater script
The Cloudflare DDNS update script's log file has been set up so that utilities like Logwatch can easily parse it. In order to make that happen, a LogFile Group file, Service and Script have to be created for Logwatch to generate reports. The correct (general) directory structure has been created in this git archive already. Below are the details of each file. The Cloudflare DDNS update script's log file has been set up so that utilities like Logwatch can easily parse it. To
make that happen, a LogFile Group file, Service and Script have to be created for Logwatch to generate reports.
The correct (general) directory structure has been created in this git archive already. Below are the details of each
file.
You can implement this setup easily by copying it into your */etc/logwatch* directory and then modifying the files as necessary: You can implement this setup easily by copying it into your */etc/logwatch* directory and then modifying the files as
necessary:
```bash ```bash
cd /etc/logwatch cd /etc/logwatch
cp -R /path/to/CloudflareDDNS_repo/etc/logwatch/* ./ cp -R /path/to/CloudflareDDNS_repo/etc/logwatch/* ./
``` ```
If you need help getting logwatch installed and set-up, please [check out my blog post](#https://mytechiethoughts.com/linux/easily-monitor-your-logs-using-logwatch/). If you need help getting logwatch installed and set-up,
please [check out my blog post](#https://mytechiethoughts.com/linux/easily-monitor-your-logs-using-logwatch/).
## Contents <!-- omit in toc --> > **IMPORTANT:** Starting with version 3.0 of the `cfddns` script, you *MUST* use the `--fmt-syslog` parameter when
> running the `cfddns` script. Otherwise, your log file will be in JSON format, and the supplied templates will not
> work.
<!-- toc --> ## Contents
- [LogFile Group file (/etc/logwatch/conf/logfiles/cfddns.conf)](#logfile-group-file-etclogwatchconflogfilescfddnsconf) <!-- TOC -->
* [Log file location](#log-file-location)
* [Archive location and name format](#archive-location-and-name-format)
* [External script for timestamp processing](#external-script-for-timestamp-processing)
- [Service definition file (/etc/logwatch/conf/services/cfddns.conf)](#service-definition-file-etclogwatchconfservicescfddnsconf)
* [LogFile Group file definition](#logfile-group-file-definition)
* [Report title](#report-title)
* [Detail level](#detail-level)
- [Service script (/etc/logwatch/scripts/services/cfddns)](#service-script-etclogwatchscriptsservicescfddns)
* [Detail levels](#detail-levels)
- [Timestamp processing script (/etc/logwatch/scripts/shared/sqfullstampanywhere)](#timestamp-processing-script-etclogwatchscriptssharedsqfullstampanywhere)
* [The time format specification](#the-time-format-specification)
* [The search REGEX](#the-search-regex)
- [Testing](#testing)
- [Final thoughts](#final-thoughts)
<!-- tocstop --> * [Using Logwatch to monitor the Cloudflare DDNS updater script](#using-logwatch-to-monitor-the-cloudflare-ddns-updater-script)
* [Contents](#contents)
* [LogFile Group file](#logfile-group-file)
* [Log file location](#log-file-location)
* [Archive location and name format](#archive-location-and-name-format)
* [External script for timestamp processing](#external-script-for-timestamp-processing)
* [Service definition file](#service-definition-file)
* [LogFile Group file definition](#logfile-group-file-definition)
* [Report title](#report-title)
* [Detail level](#detail-level)
* [Service script](#service-script)
* [Detail levels](#detail-levels)
* [Timestamp processing script](#timestamp-processing-script)
* [The time format specification](#the-time-format-specification)
* [The search REGEX](#the-search-regex)
* [Testing](#testing)
* [Final thoughts](#final-thoughts)
## LogFile Group file (/etc/logwatch/conf/logfiles/cfddns.conf) <!-- TOC -->
## LogFile Group file
> This file is located within the repo at */etc/logwatch/conf/logfiles/cfddns.conf*
### Log file location ### Log file location
Update this as needed to point to the location and name of the log file generated by the updater script. Remember, by default, the log file is created in the same directory as the script itself. Update this as needed to point to the location and name of the log file generated by the updater script. Remember, by
default, the log file is created in the same directory as the script itself.
```Ini ```Ini
LogFile = /path/to/your/cfddns.log LogFile = /path/to/your/cfddns.log
... ...
``` ```
Best practices suggest you use the `--log` flag to change this location to something like */var/log/cfddns.log*, for example. In that case, the entry would look like: Best practices suggest you use the `--log-file` flag to change this location to something like */var/log/cfddns.log*,
for
example. In that case, the entry would look like:
```Ini ```Ini
LogFile = /var/log/cfddns.log LogFile = /var/log/cfddns.log
@@ -53,7 +69,10 @@ LogFile = /var/log/cfddns.log
### Archive location and name format ### Archive location and name format
If you want Logwatch to process old (archived) log files generated by something like *Logrotate*, then you have to specify the location and file name format of those files. I've included the generalized compressed format of such rotated files as the default in the script. Suppose you store your log files in the recommended location (*/var/log/*) and are using *Logrotate* with compression enabled, the archive line would look like: If you want Logwatch to process old (archived) log files generated by something like *Logrotate*, then you have to
specify the location and file name format of those files. I've included the generalized compressed format of such
rotated files as the default in the script. Suppose you store your log files in the recommended location (*/var/log/*)
and are using *Logrotate* with compression enabled, the archive line would look like:
```Ini ```Ini
... ...
@@ -61,29 +80,37 @@ Archive = /var/log/cfddns.log.?.gz
... ...
``` ```
This would tell Logwatch, when the archive option is set to true, that your *cfddns.log* files are archived as: *cfddns.log.1.gz*, *cfddns.log.2.gz*, etc. and are all located in */var/log/*. This would tell Logwatch, when the archive option is set to true, that your *cfddns.log* files are archived as:
*cfddns.log.1.gz*, *cfddns.log.2.gz*, etc. and are all located in */var/log/*.
> Note: This line is totally optional and only used if you set the archive option in Logwatch to true. You can comment/delete this line if you wish. > Note: This line is totally optional and only used if you set the archive option in Logwatch to true. You can
> comment/delete this line if you wish.
### External script for timestamp processing ### External script for timestamp processing
Since the log file uses a non-standard (according to Logwatch) method of date-stamping, a custom filter had to be created. See the [relevant](#timestamp-processing-script-etclogwatchscriptssharedsqfullstampanywhere) section of this document for more information. Since the log file uses a non-standard (according to Logwatch) method of date-stamping, a custom filter had to be
created. See the [relevant](#timestamp-processing-script) section of this
document for more information.
The script file is called with an asterisk (*\**) before the filename. The script file is called with an asterisk (*) before the filename.
```Ini ```Ini
... ...
*sqFullStampAnywhere *iso8601timestamp
... ...
``` ```
If you change the name of this file, you will have to change this line. Remember that whatever you type here as a name is converted to all-lowercase so your filename should be all lowercase also. If you change the name of this file, you will have to change this line. Remember that whatever you type here as a name
is converted to all-lowercase so your filename should be all lowercase also.
## Service definition file (/etc/logwatch/conf/services/cfddns.conf) ## Service definition file
> This file is located within the repo at */etc/logwatch/conf/services/cfddns.conf*
### LogFile Group file definition ### LogFile Group file definition
The service file needs to know what group of log files it is responsible for processing. This MUST match the name of your *LogFile Group file*: The service file needs to know what group of log files it is responsible for processing. This MUST match the name of
your *LogFile Group file*:
```Ini ```Ini
LogFile = cfddns LogFile = cfddns
@@ -94,15 +121,20 @@ If you change your LogFile Group filename, then update it here also without the
### Report title ### Report title
The Logwatch output file (html or text) is divided into sections. You can define the title to be anything that has meaning for you. I have arbitrarily chosen *"CloudFlare DDNS update"* but you can change it to anything you want by modifying the line: The Logwatch output file (HTML or text) is divided into sections. You can define the title to be anything that has
meaning for you. I have arbitrarily chosen *"CloudFlare DDNS update"* but you can change it to anything you want by
modifying the line:
```Ini ```Ini
... ...
Title = "CloudFlare DDNS update" Title = "CloudFlare DDNS update"
``` ```
### Detail level ### Detail level
If you want to set the *detail* level of this service differently from your other services (which will use the *--detail* switch value or the value in your *logwatch.conf*), then you can define that level here. By default, it appears like this in the service configuration file: If you want to set the *detail* level of this service differently from your other services (which will use the
*--detail* switch value or the value in your *logwatch.conf*), then you can define that level here. By default, it
appears like this in the service configuration file:
```Ini ```Ini
... ...
@@ -111,7 +143,8 @@ If you want to set the *detail* level of this service differently from your othe
# Detail = 0 # Detail = 0
``` ```
Simply change it to the value you want enforced. For example, here I'm setting it to output level 5 regardless of whatever settings everything else is using. Simply change it to the value you want enforced. For example, here I'm setting it to output level 5 regardless of
whatever settings everything else is using.
```Ini ```Ini
# Override the detail level for this service # Override the detail level for this service
@@ -119,64 +152,90 @@ Simply change it to the value you want enforced. For example, here I'm setting
Detail = 5 Detail = 5
``` ```
## Service script (/etc/logwatch/scripts/services/cfddns) ## Service script
Logwatch calls any script with a name that **matches the service name**. You'll notice that I just named everything *cfddns* to keep things simple. You can change this to whatever you want. If you changed the service name to *"cloudflare*.conf", for example, you would have to rename this script file to "*cloudflare*" with no extension. Note: The script is a PERL file (note the > This file is located within the repo at */etc/logwatch/scripts/services/cfddns*
shebang) but it can be written in any language.
In essence, Logwatch just spits out the log file(s) defined in the LogFile Group file as standard input (STDIN) for the script and then takes whatever is output (STDOUT) from the script to assemble into its report. Logwatch calls any script with a name that **matches the service name**. You'll notice that I just named everything
*cfddns* to keep things simple. You can change this to whatever you want. If you changed the service name to
"*cloudflare*.conf", for example, you would have to rename this script file to "*cloudflare*" with no extension.
Note: The script is a PERL file (note the shebang), but it can be written in any language.
In essence, Logwatch just spits out the log file(s) defined in the LogFile Group file as standard input (STDIN) for the
script and then takes whatever is output (STDOUT) from the script to assemble into its report.
### Detail levels ### Detail levels
The script supports four (4) detail levels as follows: The script supports four (4) detail levels as follows:
- **Level 0: Summary output only** - **Level 0: Summary output only**
- This will display a simple aggregate of status message categories over the reporting period: - This will display a simple aggregate of status message categories over the reporting period:
- Entries successfully updated - Entries successfully updated
- Entries already up-to-date - Entries already up to date
- Hosts failed to update - Hosts failed to update
- Undefined hosts (i.e. requested to update but record doesnt exist) - Undefined hosts (i.e. requested to update but record doesnt exist)
- Total warning messages - Total warning messages
- Total errors - Total errors
- **This is the recommended reporting level.** It does not take up much space and is quick to read. If you see successful updates and/or up-to-date numbers match what you expect and no errors logged, then you can assume things are working properly. If the numbers arent right or you see errors/warnings, then you can investigate the situation by consulting the actual logs or increasing the detail level in logwatch. - **This is the recommended reporting level.** It does not take up much space and is quick to read. If you see
successful updates and/or up-to-date numbers match what you expect and no errors logged, then you can assume
- For example: Lets suppose you are running an update every 15 minutes. Doing the math... things are working properly. If the numbers arent right or you see errors/warnings, then you can investigate the
$$ situation by consulting the actual logs or increasing the detail level in logwatch.
(update_{success}) + (update_{up-to-date}) = (24h \times 60min)/15min = 96
$$ - For example, lets suppose you are running an update every 15 minutes. Doing the math...
Therefore, you expect to see Entries successfully updated and Entries already up-to-date total *96*. If thats the case and no errors or warnings are logged, things are ok. Pretty easy, right? Thats why this is the recommended filter setting. $$
(update_{success}) + (update_{up-to-date}) = (24h \times 60min)/15min = 96
$$
Therefore, you expect to see Entries successfully updated and Entries already up to date total *96*. If thats
the case and no errors or warnings are logged, things are ok. Pretty easy, right? Thats why this is the
recommended filter setting.
- **Levels 1-4: Critical messages** - **Levels 1-4: Critical messages**
- This uses the data which is summarized by Level 0 but outputs the *actual messages* in the log file. For example, you will see the actual text of the errors logged instead of just a total number of errors. This level of reporting is useful when *initially* monitoring the script's operation since you can see the text of any generated errors. - This uses the data which is summarized by Level 0 but outputs the *actual messages* in the log file. For example,
- Levels 1, 2, 3 & 4 are identical so pick your favourite number. you will see the actual text of the errors logged instead of just a total number of errors. This level of
reporting is useful when *initially* monitoring the script's operation since you can see the text of any generated
errors.
- Levels 1, 2, 3 & 4 are identical, so pick your favourite number.
- **Level 5: Verbose output** - **Level 5: Verbose output**
- Like the previous level, this outputs the actual messages found in the log file. However, it also includes *CF-ERR* tags and tally count messages. This can help you pinpoint why the Cloudflare API is rejecting your requests by letting you see things like authentication errors or malformed addresses, etc. - Like the previous level, this outputs the actual messages found in the log file. However, it also includes
- Honestly, this is not much more information than L1-L4 and is often a better choice while debugging any issues since you get the Cloudflare API messages. *CF-ERR* tags and tally count messages. This can help you pinpoint why the Cloudflare API is rejecting your
- This level of output is much more verbose than the summary report. It also takes *much* more time and patience to review so it is only recommended when youre dealing with issues. requests by letting you see things like authentication errors or malformed addresses, etc.
- This is *not* recommended for day-to-day or routine reports. - Honestly, this is not much more information than L1-L4 and is often a better choice while debugging any issues
since you get the Cloudflare API messages.
- This level of output is much more verbose than the summary report. It also takes *much* more time and patience to
review, so it is only recommended when youre dealing with issues.
- This is *not* recommended for day-to-day or routine reports.
- **Levels 6+: Complete log file dump** - **Levels 6+: Complete log file dump**
- Any number greater than 5 passed as a detail level will trigger the script to dump the entire log file out to Logwatch line-by-line. This is really only useful during debugging or dealing with serious issues where you do not have access to the actual log file. While this is an exact echo of the log file, it likely will not be colour-coded which makes it harder to review. - Any number greater than 5 passed as a detail level will trigger the script to dump the entire log file out to
- **Use this detail level only when you need to see the entire log file and cannot otherwise access the log file.** Logwatch line-by-line. This is really only useful during debugging or dealing with serious issues where you do not
- Depending on how your logwatch treats this log dump, you may see gibberish control codes like *\e[0m;]*. If this is the case, run the script with the `--no-colour` or `--nc` option to remove ANSI colour formatting. have access to the actual log file.
- **Use this detail level only when you need to see the entire log file and cannot otherwise access the log file.**
## Timestamp processing script (/etc/logwatch/scripts/shared/sqfullstampanywhere) ## Timestamp processing script
This is basically a modified version of the '*applyeurodate*' script that comes with Logwatch. It had to be modified to search within [square brackets] and to accept characters coming before the stamp (i.e. ANSI colour codes). If you change the '**stamp**' variable in the updater script to update the timestamp to your liking (which to totally fine!) then you'll probably have to update this file. There are two lines you need to modify to suit your new '**stamp**' variable. > This file is located within the repo at */etc/logwatch/scripts/shared/iso8601timestamp*
> This entire section is only applicable if you are a very curious person or if you change the hard-coded stamp function in the script. If you did not make any changes and you like a little mystery in your life, you can safely skip this entire section. This is a modified version of the '*applyeurodate*' script that comes with Logwatch. If you change
the '**stamp**' variable in the `cfddns` script to modify the timestamp to your liking (which is totally fine!) then
you'll probably have to update this file. There are two lines you need to modify to suit your new '**stamp**' variable.
> This entire section is only applicable if you are an inquisitive person or if you change the hard-coded stamp
> function in the script. If you did not make any changes and you like a little mystery in your life, you can safely
> skip this entire section.
### The time format specification ### The time format specification
'*$SearchDate*' is the variable used in the PERL script to do exactly what it says, search for the date stamp. I have it set up to look for the format '*year-month-date hour:minute:second*'. Note, we don't care about brackets or anything here, we're just defining the format of the date/time stamp. '*$SearchDate*' is the variable used in the PERL script to do exactly what it says, search for the date stamp. I have it
set up to look for the format '*year-month-dateThour:minute:secondZ*', which is the ISO-8601 format.
```Perl ```Perl
... ...
$SearchDate = TimeFilter('%Y-%m-%d %H:%M:%S'); $SearchDate = TimeFilter('%Y-%m-%dT%H:%M:%SZ');
... ...
``` ```
If you changed the '**stamp**' variable so it was formatted as '*month/day/year hour:minute*' (ex: '*[09/27/2018 18:38]*') then you'd update the **$SearchDate** variable as follows (note: no mention of the square brackets!): If you changed the '**stamp**' variable so it was formatted as '*month/day/year hour:minute*'
(ex: '*09/27/2018 18:38*') then you'd update the **$SearchDate** variable as follows:
```Perl ```Perl
... ...
@@ -186,31 +245,8 @@ $SearchDate = TimeFilter('%m/%d/%Y %H:%M');
### The search REGEX ### The search REGEX
The PERL script uses a '*regular expression*' (REGEX) to search within the log file for '*$SearchDate*'. For the default date stamp, this specification looks like: The PERL script uses a '*regular expression*' (REGEX) to search within the log file for '*$SearchDate*'. For the default
date stamp, this specification looks like:
```Perl
...
if ($ThisLine =~ m/\[$SearchDate\] /o) {
...
```
The REGEX appears between '*m/*' and '*/o*'. In this case, it searches for '*$SearchDate*' inside [square brackets] appearing anywhere on the line. This is because ANSI colour-codes often appear before the date stamp in the default log file. If you have modified this so that your date stamp appears at the beginning of the line and in the example format in the section above (using slashes instead of dashes) then you'd rewrite this REGEX as follows:
```Perl
...
if ($ThisLine =~ m/^\[$SearchDate\] /o) {
...
```
or using regular brackets anywhere on the line:
```Perl
...
if ($ThisLine =~ m/\($SearchDate\) /o) {
...
```
or without any brackets but appearing at the beginning of the line:
```Perl ```Perl
... ...
@@ -218,12 +254,30 @@ if ($ThisLine =~ m/^$SearchDate /o) {
... ...
``` ```
The REGEX appears between '*m/*' and '*/o*'. In this case, it searches for '*$SearchDate*' appearing at the
beginning (^) of the line. If you want to search anywhere in the line instead of the beginning, remove the caret (^):
```Perl
...
if ($ThisLine =~ m/$SearchDate /o) {
...
```
or maybe you changed the date stamp to appear in square brackets at the start of the line:
```Perl
...
if ($ThisLine =~ m/^\[$SearchDate\] /o) {
...
```
## Testing ## Testing
Run *logwatch --help* and note the options. You can test just this service locally on your screen with the following command (assuming you kept default names for everything): Run *logwatch --help* and note the options. You can test just this service locally on your screen with the following
command (assuming you kept default names for everything):
```Bash ```Bash
# Summary output, entire duration of log file # Summary output, entire duration of the log file
logwatch --service cfddns --output stdout --format text --range all --detail 0 logwatch --service cfddns --output stdout --format text --range all --detail 0
# Minimal detail, yesterday only # Minimal detail, yesterday only
@@ -235,4 +289,6 @@ logwatch --service cfddns --output stdout --format text --range today --detail 5
## Final thoughts ## Final thoughts
That's it! I'm a horrible PERL programmer so if anyone can optimize/improve the script file used for Logwatch then please do it! Otherwise, I hope this made sense and helped you integrate the updater script with Logwatch for easy monitoring :-) That's it! I'm a horrible PERL programmer, so if anyone can optimize/improve the script file used for Logwatch, then
please do it! Otherwise, I hope this made sense and helped you integrate the updater script with Logwatch for easy
monitoring :-)
+1 -1
View File
@@ -10,4 +10,4 @@ Archive = /path/to/your/logfile.ext.?.gz
# You don't need to change this unless you have altered the 'stamp' variable in # You don't need to change this unless you have altered the 'stamp' variable in
# the script in which case you will want to update the regex in the custom pl # the script in which case you will want to update the regex in the custom pl
# script below # script below
*sqFullStampAnywhere *iso8601timestamp
+16 -22
View File
@@ -4,7 +4,7 @@
# $Id$ # $Id$
############################################################################# #############################################################################
# Log: CloudFlare updater script (cfddns) # Log: CloudFlare updater script (cfddns)
# Revision 2.1 2021/05/08 # Revision 3.0 2026/07/25
# Written by Asif Bacchus # Written by Asif Bacchus
############################################################################# #############################################################################
@@ -31,22 +31,22 @@ my $key;
if ($detailLevel == 0) { if ($detailLevel == 0) {
### process logfile and summarize message types ### process logfile and summarize message types
while (defined(my $ThisLine = <STDIN>)) { while (defined(my $ThisLine = <STDIN>)) {
if ($ThisLine =~ /ERR: Unable to update IP address/) { if ($ThisLine =~ /[FAIL] Unable to update the IP address for/) {
$summaryFailedUpdate++; $summaryFailedUpdate++;
} }
elsif ($ThisLine =~ /ERROR: /) { elsif ($ThisLine =~ /[FAIL] /) {
$summaryErr++; $summaryErr++;
} }
elsif ($ThisLine =~ /WARN: Cannot find existing record/) { elsif ($ThisLine =~ /[WARN] Cannot find an existing record matching/) {
$summaryInvalidHost++; $summaryInvalidHost++;
} }
elsif ($ThisLine =~ /WARNING: /){ elsif ($ThisLine =~ /[WARN] /){
$summaryWarning++; $summaryWarning++;
} }
elsif ($ThisLine =~ /SUCCESS: /) { elsif ($ThisLine =~ /[SUCCESS] /) {
$summaryUpdated++; $summaryUpdated++;
} }
elsif ($ThisLine =~ /already up-to-date/) { elsif ($ThisLine =~ /already up to date/) {
$summaryUpToDate++; $summaryUpToDate++;
} }
} }
@@ -56,7 +56,7 @@ if ($detailLevel == 0) {
$reportHash{"Entries successfully updated"} = $summaryUpdated; $reportHash{"Entries successfully updated"} = $summaryUpdated;
} }
if ($summaryUpToDate > 0) { if ($summaryUpToDate > 0) {
$reportHash{"Entries already up-to-date"} = $summaryUpToDate; $reportHash{"Entries already up to date"} = $summaryUpToDate;
} }
if ($summaryFailedUpdate > 0) { if ($summaryFailedUpdate > 0) {
$reportHash{"Hosts failed to update"} = $summaryFailedUpdate; $reportHash{"Hosts failed to update"} = $summaryFailedUpdate;
@@ -80,16 +80,16 @@ if ($detailLevel == 0) {
### a summary count ### a summary count
elsif ($detailLevel >= 1 && $detailLevel <= 4) { elsif ($detailLevel >= 1 && $detailLevel <= 4) {
while (defined(my $ThisLine = <STDIN>)) { while (defined(my $ThisLine = <STDIN>)) {
if ($ThisLine =~ /ERR: /) { if ($ThisLine =~ /[ERR] /) {
print $ThisLine; print $ThisLine;
} }
elsif ($ThisLine =~ /WARN: /) { elsif ($ThisLine =~ /[WARN] /) {
print $ThisLine; print $ThisLine;
} }
elsif ($ThisLine =~ /SUCCESS: /) { elsif ($ThisLine =~ /[SUCCESS] /) {
print $ThisLine; print $ThisLine;
} }
elsif ($ThisLine =~ /already up-to-date/) { elsif ($ThisLine =~ /already up to date/) {
print $ThisLine; print $ThisLine;
} }
} }
@@ -98,25 +98,19 @@ elsif ($detailLevel >= 1 && $detailLevel <= 4) {
### debugging messages ### debugging messages
elsif ($detailLevel == 5) { elsif ($detailLevel == 5) {
while (defined(my $ThisLine = <STDIN>)) { while (defined(my $ThisLine = <STDIN>)) {
if ($ThisLine =~ /ERR: /) { if ($ThisLine =~ /[ERR] /) {
print $ThisLine;
}
elsif ($ThisLine =~ /ERROR: /) {
print $ThisLine; print $ThisLine;
} }
elsif ($ThisLine =~ /CF-ERR: /) { elsif ($ThisLine =~ /CF-ERR: /) {
print $ThisLine; print $ThisLine;
} }
elsif ($ThisLine =~ /WARN: /) { elsif ($ThisLine =~ /[WARN] /) {
print $ThisLine; print $ThisLine;
} }
elsif ($ThisLine =~ /WARNING: /) { elsif ($ThisLine =~ /[SUCCESS] /) {
print $ThisLine; print $ThisLine;
} }
elsif ($ThisLine =~ /SUCCESS: /) { elsif ($ThisLine =~ /already up to date/) {
print $ThisLine;
}
elsif ($ThisLine =~ /already up-to-date/) {
print $ThisLine; print $ThisLine;
} }
} }
@@ -5,9 +5,8 @@
########################################################################## ##########################################################################
############################################################################### ###############################################################################
## Filter dates in full-date-time international format, surrounded by square ## Filter dates in ISO-8601 format located at the start of a given line
## brackets located anywhere on a given line ## Format: '%Y-%m-%dT%H:%M:%SZ'
## Format: '[%Y-%m-%d %H:%M:%S]'
############################################################################### ###############################################################################
use Logwatch ':dates'; use Logwatch ':dates';
@@ -15,7 +14,7 @@ use Logwatch ':dates';
my $Debug = $ENV{'LOGWATCH_DEBUG'} || 0; my $Debug = $ENV{'LOGWATCH_DEBUG'} || 0;
### Specify the format of the date/time stamp itself ### Specify the format of the date/time stamp itself
$SearchDate = TimeFilter('%Y-%m-%d %H:%M:%S'); $SearchDate = TimeFilter('%Y-%m-%dT%H:%M:%SZ');
if ( $Debug > 5 ) { if ( $Debug > 5 ) {
print STDERR "DEBUG: Inside FullDateTime...\n"; print STDERR "DEBUG: Inside FullDateTime...\n";
@@ -24,7 +23,7 @@ if ( $Debug > 5 ) {
while (defined($ThisLine = <STDIN>)) { while (defined($ThisLine = <STDIN>)) {
### specify the regex that defines how to find 'SearchDate' ### specify the regex that defines how to find 'SearchDate'
if ($ThisLine =~ m/\[$SearchDate\] /o) { if ($ThisLine =~ m/^$SearchDate /o) {
print $ThisLine; print $ThisLine;
} }
} }