Issue
In the forum notification checking function of vm-watcher.sh, a variable is updated inside a while read loop that follows a pipe (|), but the updated value is lost after the loop ends. Although max_id is updated within the loop, it reverts to its initial value of 0 once the loop completes.
Cause
In Bash, the right-hand side of a pipeline (|) runs in a subshell. Consequently, any variable modifications made inside the while read loop are confined to that subshell and do not propagate back to the parent shell. This is a well-known Bash pitfall.
Solution
Replace the pipeline-based while read loop with a jq-based array iteration using indices—thus avoiding pipelines altogether. First, use jq to collect all elements into a JSON array; then iterate over the array using a while loop with index-based access. Since this avoids subshells, variable updates occur directly in the parent shell.
Alternative approaches include:
- Process substitution (e.g.,
while read ... done < <(command)), - Persisting variables via temporary files,
- Enabling
shopt -s lastpipe(available in Bash 4.2+), which allows the last command in a pipeline to run in the current shell instead of a subshell.
Additional Lesson
Construct JSON output using jq -cn, never by string concatenation (e.g., echo with escaped quotes). String concatenation fails catastrophically when fields contain quotes or other special characters, resulting in invalid JSON. Our inbox was flooded with numerous null entries precisely due to this issue.