docs(logwatch): Update readme

Update readme with changes for v3
This commit is contained in:
2026-07-25 04:11:32 -06:00
parent 2d0188949b
commit 30bd4d230d
+122 -74
View File
@@ -1,37 +1,48 @@
# Using Logwatch to monitor Cloudflare DDNS updater script <!-- omit in toc --> # Using Logwatch to monitor the Cloudflare DDNS updater script
The Cloudflare DDNS update script's log file has been set up so that utilities like Logwatch can easily parse it. In order to make that happen, a LogFile Group file, Service and Script have to be created for Logwatch to generate reports. The correct (general) directory structure has been created in this git archive already. Below are the details of each file. The Cloudflare DDNS update script's log file has been set up so that utilities like Logwatch can easily parse it. To
make that happen, a LogFile Group file, Service and Script have to be created for Logwatch to generate reports.
The correct (general) directory structure has been created in this git archive already. Below are the details of each
file.
You can implement this setup easily by copying it into your */etc/logwatch* directory and then modifying the files as necessary: You can implement this setup easily by copying it into your */etc/logwatch* directory and then modifying the files as
necessary:
```bash ```bash
cd /etc/logwatch cd /etc/logwatch
cp -R /path/to/CloudflareDDNS_repo/etc/logwatch/* ./ cp -R /path/to/CloudflareDDNS_repo/etc/logwatch/* ./
``` ```
If you need help getting logwatch installed and set-up, please [check out my blog post](#https://mytechiethoughts.com/linux/easily-monitor-your-logs-using-logwatch/). If you need help getting logwatch installed and set-up,
please [check out my blog post](#https://mytechiethoughts.com/linux/easily-monitor-your-logs-using-logwatch/).
## Contents <!-- omit in toc --> > **IMPORTANT:** Starting with version 3.0 of the `cfddns` script, you *MUST* use the `--fmt-syslog` parameter when
> running the `cfddns` script. Otherwise, your log file will be in JSON format, and the supplied templates will not
> work.
<!-- toc --> ## Contents
- [LogFile Group file](#logfile-group-file) <!-- TOC -->
* [Using Logwatch to monitor the Cloudflare DDNS updater script](#using-logwatch-to-monitor-the-cloudflare-ddns-updater-script)
* [Contents](#contents)
* [LogFile Group file](#logfile-group-file)
* [Log file location](#log-file-location) * [Log file location](#log-file-location)
* [Archive location and name format](#archive-location-and-name-format) * [Archive location and name format](#archive-location-and-name-format)
* [External script for timestamp processing](#external-script-for-timestamp-processing) * [External script for timestamp processing](#external-script-for-timestamp-processing)
- [Service definition file](#service-definition-file) * [Service definition file](#service-definition-file)
* [LogFile Group file definition](#logfile-group-file-definition) * [LogFile Group file definition](#logfile-group-file-definition)
* [Report title](#report-title) * [Report title](#report-title)
* [Detail level](#detail-level) * [Detail level](#detail-level)
- [Service script](#service-script) * [Service script](#service-script)
* [Detail levels](#detail-levels) * [Detail levels](#detail-levels)
- [Timestamp processing script](#timestamp-processing-script) * [Timestamp processing script](#timestamp-processing-script)
* [The time format specification](#the-time-format-specification) * [The time format specification](#the-time-format-specification)
* [The search REGEX](#the-search-regex) * [The search REGEX](#the-search-regex)
- [Testing](#testing) * [Testing](#testing)
- [Final thoughts](#final-thoughts) * [Final thoughts](#final-thoughts)
<!-- tocstop --> <!-- TOC -->
## LogFile Group file ## LogFile Group file
@@ -39,14 +50,17 @@ If you need help getting logwatch installed and set-up, please [check out my blo
### Log file location ### Log file location
Update this as needed to point to the location and name of the log file generated by the updater script. Remember, by default, the log file is created in the same directory as the script itself. Update this as needed to point to the location and name of the log file generated by the updater script. Remember, by
default, the log file is created in the same directory as the script itself.
```Ini ```Ini
LogFile = /path/to/your/cfddns.log LogFile = /path/to/your/cfddns.log
... ...
``` ```
Best practices suggest you use the `--log` flag to change this location to something like */var/log/cfddns.log*, for example. In that case, the entry would look like: Best practices suggest you use the `--log-file` flag to change this location to something like */var/log/cfddns.log*,
for
example. In that case, the entry would look like:
```Ini ```Ini
LogFile = /var/log/cfddns.log LogFile = /var/log/cfddns.log
@@ -55,7 +69,10 @@ LogFile = /var/log/cfddns.log
### Archive location and name format ### Archive location and name format
If you want Logwatch to process old (archived) log files generated by something like *Logrotate*, then you have to specify the location and file name format of those files. I've included the generalized compressed format of such rotated files as the default in the script. Suppose you store your log files in the recommended location (*/var/log/*) and are using *Logrotate* with compression enabled, the archive line would look like: If you want Logwatch to process old (archived) log files generated by something like *Logrotate*, then you have to
specify the location and file name format of those files. I've included the generalized compressed format of such
rotated files as the default in the script. Suppose you store your log files in the recommended location (*/var/log/*)
and are using *Logrotate* with compression enabled, the archive line would look like:
```Ini ```Ini
... ...
@@ -63,23 +80,28 @@ Archive = /var/log/cfddns.log.?.gz
... ...
``` ```
This would tell Logwatch, when the archive option is set to true, that your *cfddns.log* files are archived as: *cfddns.log.1.gz*, *cfddns.log.2.gz*, etc. and are all located in */var/log/*. This would tell Logwatch, when the archive option is set to true, that your *cfddns.log* files are archived as:
*cfddns.log.1.gz*, *cfddns.log.2.gz*, etc. and are all located in */var/log/*.
> Note: This line is totally optional and only used if you set the archive option in Logwatch to true. You can comment/delete this line if you wish. > Note: This line is totally optional and only used if you set the archive option in Logwatch to true. You can
> comment/delete this line if you wish.
### External script for timestamp processing ### External script for timestamp processing
Since the log file uses a non-standard (according to Logwatch) method of date-stamping, a custom filter had to be created. See the [relevant](#timestamp-processing-script-etclogwatchscriptssharedsqfullstampanywhere) section of this document for more information. Since the log file uses a non-standard (according to Logwatch) method of date-stamping, a custom filter had to be
created. See the [relevant](#timestamp-processing-script) section of this
document for more information.
The script file is called with an asterisk (*\**) before the filename. The script file is called with an asterisk (*) before the filename.
```Ini ```Ini
... ...
*sqFullStampAnywhere *iso8601timestamp
... ...
``` ```
If you change the name of this file, you will have to change this line. Remember that whatever you type here as a name is converted to all-lowercase so your filename should be all lowercase also. If you change the name of this file, you will have to change this line. Remember that whatever you type here as a name
is converted to all-lowercase so your filename should be all lowercase also.
## Service definition file ## Service definition file
@@ -87,7 +109,8 @@ If you change the name of this file, you will have to change this line. Remember
### LogFile Group file definition ### LogFile Group file definition
The service file needs to know what group of log files it is responsible for processing. This MUST match the name of your *LogFile Group file*: The service file needs to know what group of log files it is responsible for processing. This MUST match the name of
your *LogFile Group file*:
```Ini ```Ini
LogFile = cfddns LogFile = cfddns
@@ -98,15 +121,20 @@ If you change your LogFile Group filename, then update it here also without the
### Report title ### Report title
The Logwatch output file (html or text) is divided into sections. You can define the title to be anything that has meaning for you. I have arbitrarily chosen *"CloudFlare DDNS update"* but you can change it to anything you want by modifying the line: The Logwatch output file (HTML or text) is divided into sections. You can define the title to be anything that has
meaning for you. I have arbitrarily chosen *"CloudFlare DDNS update"* but you can change it to anything you want by
modifying the line:
```Ini ```Ini
... ...
Title = "CloudFlare DDNS update" Title = "CloudFlare DDNS update"
``` ```
### Detail level ### Detail level
If you want to set the *detail* level of this service differently from your other services (which will use the *--detail* switch value or the value in your *logwatch.conf*), then you can define that level here. By default, it appears like this in the service configuration file: If you want to set the *detail* level of this service differently from your other services (which will use the
*--detail* switch value or the value in your *logwatch.conf*), then you can define that level here. By default, it
appears like this in the service configuration file:
```Ini ```Ini
... ...
@@ -115,7 +143,8 @@ If you want to set the *detail* level of this service differently from your othe
# Detail = 0 # Detail = 0
``` ```
Simply change it to the value you want enforced. For example, here I'm setting it to output level 5 regardless of whatever settings everything else is using. Simply change it to the value you want enforced. For example, here I'm setting it to output level 5 regardless of
whatever settings everything else is using.
```Ini ```Ini
# Override the detail level for this service # Override the detail level for this service
@@ -127,10 +156,13 @@ Detail = 5
> This file is located within the repo at */etc/logwatch/scripts/services/cfddns* > This file is located within the repo at */etc/logwatch/scripts/services/cfddns*
Logwatch calls any script with a name that **matches the service name**. You'll notice that I just named everything *cfddns* to keep things simple. You can change this to whatever you want. If you changed the service name to *"cloudflare*.conf", for example, you would have to rename this script file to "*cloudflare*" with no extension. Note: The script is a PERL file (note the Logwatch calls any script with a name that **matches the service name**. You'll notice that I just named everything
shebang) but it can be written in any language. *cfddns* to keep things simple. You can change this to whatever you want. If you changed the service name to
"*cloudflare*.conf", for example, you would have to rename this script file to "*cloudflare*" with no extension.
Note: The script is a PERL file (note the shebang), but it can be written in any language.
In essence, Logwatch just spits out the log file(s) defined in the LogFile Group file as standard input (STDIN) for the script and then takes whatever is output (STDOUT) from the script to assemble into its report. In essence, Logwatch just spits out the log file(s) defined in the LogFile Group file as standard input (STDIN) for the
script and then takes whatever is output (STDOUT) from the script to assemble into its report.
### Detail levels ### Detail levels
@@ -139,52 +171,71 @@ The script supports four (4) detail levels as follows:
- **Level 0: Summary output only** - **Level 0: Summary output only**
- This will display a simple aggregate of status message categories over the reporting period: - This will display a simple aggregate of status message categories over the reporting period:
- Entries successfully updated - Entries successfully updated
- Entries already up-to-date - Entries already up to date
- Hosts failed to update - Hosts failed to update
- Undefined hosts (i.e. requested to update but record doesnt exist) - Undefined hosts (i.e. requested to update but record doesnt exist)
- Total warning messages - Total warning messages
- Total errors - Total errors
- **This is the recommended reporting level.** It does not take up much space and is quick to read. If you see successful updates and/or up-to-date numbers match what you expect and no errors logged, then you can assume things are working properly. If the numbers arent right or you see errors/warnings, then you can investigate the situation by consulting the actual logs or increasing the detail level in logwatch. - **This is the recommended reporting level.** It does not take up much space and is quick to read. If you see
successful updates and/or up-to-date numbers match what you expect and no errors logged, then you can assume
things are working properly. If the numbers arent right or you see errors/warnings, then you can investigate the
situation by consulting the actual logs or increasing the detail level in logwatch.
- For example: Lets suppose you are running an update every 15 minutes. Doing the math... - For example, lets suppose you are running an update every 15 minutes. Doing the math...
$$ $$
(update_{success}) + (update_{up-to-date}) = (24h \times 60min)/15min = 96 (update_{success}) + (update_{up-to-date}) = (24h \times 60min)/15min = 96
$$ $$
Therefore, you expect to see Entries successfully updated and Entries already up-to-date total *96*. If thats the case and no errors or warnings are logged, things are ok. Pretty easy, right? Thats why this is the recommended filter setting. Therefore, you expect to see Entries successfully updated and Entries already up to date total *96*. If thats
the case and no errors or warnings are logged, things are ok. Pretty easy, right? Thats why this is the
recommended filter setting.
- **Levels 1-4: Critical messages** - **Levels 1-4: Critical messages**
- This uses the data which is summarized by Level 0 but outputs the *actual messages* in the log file. For example, you will see the actual text of the errors logged instead of just a total number of errors. This level of reporting is useful when *initially* monitoring the script's operation since you can see the text of any generated errors. - This uses the data which is summarized by Level 0 but outputs the *actual messages* in the log file. For example,
- Levels 1, 2, 3 & 4 are identical so pick your favourite number. you will see the actual text of the errors logged instead of just a total number of errors. This level of
reporting is useful when *initially* monitoring the script's operation since you can see the text of any generated
errors.
- Levels 1, 2, 3 & 4 are identical, so pick your favourite number.
- **Level 5: Verbose output** - **Level 5: Verbose output**
- Like the previous level, this outputs the actual messages found in the log file. However, it also includes *CF-ERR* tags and tally count messages. This can help you pinpoint why the Cloudflare API is rejecting your requests by letting you see things like authentication errors or malformed addresses, etc. - Like the previous level, this outputs the actual messages found in the log file. However, it also includes
- Honestly, this is not much more information than L1-L4 and is often a better choice while debugging any issues since you get the Cloudflare API messages. *CF-ERR* tags and tally count messages. This can help you pinpoint why the Cloudflare API is rejecting your
- This level of output is much more verbose than the summary report. It also takes *much* more time and patience to review so it is only recommended when youre dealing with issues. requests by letting you see things like authentication errors or malformed addresses, etc.
- Honestly, this is not much more information than L1-L4 and is often a better choice while debugging any issues
since you get the Cloudflare API messages.
- This level of output is much more verbose than the summary report. It also takes *much* more time and patience to
review, so it is only recommended when youre dealing with issues.
- This is *not* recommended for day-to-day or routine reports. - This is *not* recommended for day-to-day or routine reports.
- **Levels 6+: Complete log file dump** - **Levels 6+: Complete log file dump**
- Any number greater than 5 passed as a detail level will trigger the script to dump the entire log file out to Logwatch line-by-line. This is really only useful during debugging or dealing with serious issues where you do not have access to the actual log file. While this is an exact echo of the log file, it likely will not be colour-coded which makes it harder to review. - Any number greater than 5 passed as a detail level will trigger the script to dump the entire log file out to
Logwatch line-by-line. This is really only useful during debugging or dealing with serious issues where you do not
have access to the actual log file.
- **Use this detail level only when you need to see the entire log file and cannot otherwise access the log file.** - **Use this detail level only when you need to see the entire log file and cannot otherwise access the log file.**
- Depending on how your logwatch treats this log dump, you may see gibberish control codes like *\e[0m;]*. If this is the case, run the script with the `--no-colour` or `--nc` option to remove ANSI colour formatting.
## Timestamp processing script ## Timestamp processing script
> This file is located within the repo at */etc/logwatch/scripts/shared/sqfullstampanywhere* > This file is located within the repo at */etc/logwatch/scripts/shared/iso8601timestamp*
This is basically a modified version of the '*applyeurodate*' script that comes with Logwatch. It had to be modified to search within [square brackets] and to accept characters coming before the stamp (i.e. ANSI colour codes). If you change the '**stamp**' variable in the updater script to update the timestamp to your liking (which to totally fine!) then you'll probably have to update this file. There are two lines you need to modify to suit your new '**stamp**' variable. This is a modified version of the '*applyeurodate*' script that comes with Logwatch. If you change
the '**stamp**' variable in the `cfddns` script to modify the timestamp to your liking (which is totally fine!) then
you'll probably have to update this file. There are two lines you need to modify to suit your new '**stamp**' variable.
> This entire section is only applicable if you are a very curious person or if you change the hard-coded stamp function in the script. If you did not make any changes and you like a little mystery in your life, you can safely skip this entire section. > This entire section is only applicable if you are an inquisitive person or if you change the hard-coded stamp
> function in the script. If you did not make any changes and you like a little mystery in your life, you can safely
> skip this entire section.
### The time format specification ### The time format specification
'*$SearchDate*' is the variable used in the PERL script to do exactly what it says, search for the date stamp. I have it set up to look for the format '*year-month-date hour:minute:second*'. Note, we don't care about brackets or anything here, we're just defining the format of the date/time stamp. '*$SearchDate*' is the variable used in the PERL script to do exactly what it says, search for the date stamp. I have it
set up to look for the format '*year-month-dateThour:minute:secondZ*', which is the ISO-8601 format.
```Perl ```Perl
... ...
$SearchDate = TimeFilter('%Y-%m-%d %H:%M:%S'); $SearchDate = TimeFilter('%Y-%m-%dT%H:%M:%SZ');
... ...
``` ```
If you changed the '**stamp**' variable so it was formatted as '*month/day/year hour:minute*' (ex: '*[09/27/2018 18:38]*') then you'd update the **$SearchDate** variable as follows (note: no mention of the square brackets!): If you changed the '**stamp**' variable so it was formatted as '*month/day/year hour:minute*'
(ex: '*09/27/2018 18:38*') then you'd update the **$SearchDate** variable as follows:
```Perl ```Perl
... ...
@@ -194,31 +245,8 @@ $SearchDate = TimeFilter('%m/%d/%Y %H:%M');
### The search REGEX ### The search REGEX
The PERL script uses a '*regular expression*' (REGEX) to search within the log file for '*$SearchDate*'. For the default date stamp, this specification looks like: The PERL script uses a '*regular expression*' (REGEX) to search within the log file for '*$SearchDate*'. For the default
date stamp, this specification looks like:
```Perl
...
if ($ThisLine =~ m/\[$SearchDate\] /o) {
...
```
The REGEX appears between '*m/*' and '*/o*'. In this case, it searches for '*$SearchDate*' inside [square brackets] appearing anywhere on the line. This is because ANSI colour-codes often appear before the date stamp in the default log file. If you have modified this so that your date stamp appears at the beginning of the line and in the example format in the section above (using slashes instead of dashes) then you'd rewrite this REGEX as follows:
```Perl
...
if ($ThisLine =~ m/^\[$SearchDate\] /o) {
...
```
or using regular brackets anywhere on the line:
```Perl
...
if ($ThisLine =~ m/\($SearchDate\) /o) {
...
```
or without any brackets but appearing at the beginning of the line:
```Perl ```Perl
... ...
@@ -226,12 +254,30 @@ if ($ThisLine =~ m/^$SearchDate /o) {
... ...
``` ```
The REGEX appears between '*m/*' and '*/o*'. In this case, it searches for '*$SearchDate*' appearing at the
beginning (^) of the line. If you want to search anywhere in the line instead of the beginning, remove the caret (^):
```Perl
...
if ($ThisLine =~ m/$SearchDate /o) {
...
```
or maybe you changed the date stamp to appear in square brackets at the start of the line:
```Perl
...
if ($ThisLine =~ m/^\[$SearchDate\] /o) {
...
```
## Testing ## Testing
Run *logwatch --help* and note the options. You can test just this service locally on your screen with the following command (assuming you kept default names for everything): Run *logwatch --help* and note the options. You can test just this service locally on your screen with the following
command (assuming you kept default names for everything):
```Bash ```Bash
# Summary output, entire duration of log file # Summary output, entire duration of the log file
logwatch --service cfddns --output stdout --format text --range all --detail 0 logwatch --service cfddns --output stdout --format text --range all --detail 0
# Minimal detail, yesterday only # Minimal detail, yesterday only
@@ -243,4 +289,6 @@ logwatch --service cfddns --output stdout --format text --range today --detail 5
## Final thoughts ## Final thoughts
That's it! I'm a horrible PERL programmer so if anyone can optimize/improve the script file used for Logwatch then please do it! Otherwise, I hope this made sense and helped you integrate the updater script with Logwatch for easy monitoring :-) That's it! I'm a horrible PERL programmer, so if anyone can optimize/improve the script file used for Logwatch, then
please do it! Otherwise, I hope this made sense and helped you integrate the updater script with Logwatch for easy
monitoring :-)