Hugo Site Publishing with Respectful Caching and IndexNow Submission

The design is intentionally narrow: CDN policy control and general deployment framework construction are outside scope; HTTP revalidation semantics are assumed, where Cache-Control: no-cache permits storage but requires validation before reuse. Filesystem mtimes then provide sufficient Last-Modified based validation, while Hugo generated artifacts keep gzip variants and IndexNow notifications consistent.

Date Semantics

Publishing distinguishes three timestamps:

• publishDate: editorial publication date;
• lastmod: meaningful content update date, used only when it differs from publishDate;
• filesystem mtime: cache validation timestamp for deployed artifacts.

The first two values belong in Hugo frontmatter and remain author controlled. The third belongs to deployment and is computed mechanically; rebuilding generated output does not rewrite article chronology, while generated artifact changes still move cache validators.

Hugo Outputs

Hugo renders three deployment files from the home page: mtimes.txt for post build touch, indexnow-<key>.json as the IndexNow submission payload, and indexnow-<key>.txt as the IndexNow verification key. The indexnow- prefix keeps the generated files identifiable; it also reduces the available key length to at most 119 characters because IndexNow keys are limited to 8 to 128 alphanumeric or - characters¹.

[outputs]
  home = ["html", "rss", "indexnow", "mtimes"]

[outputFormats]
  [outputFormats.indexnow]
    mediaType = "application/json"
    baseName = "indexnow-<key>"
    isPlainText = true
    notAlternative = true
  [outputFormats.mtimes]
    mediaType = "text/plain"
    baseName = "mtimes"
    isPlainText = true
    notAlternative = true

IndexNow Output

layouts/index.indexnow.json derives the host from baseURL, computes the IndexNow key from the output filename, publishes the key file with resources.FromString, and emits the sorted HTML permalink list as urlList.

{{- $baseURL := urls.Parse .Site.BaseURL -}}
{{- $key := "" -}}
{{- with .Site.Home.OutputFormats.Get "indexnow" -}}
  {{- $key = strings.TrimSuffix ".json" (path.Base .RelPermalink) -}}
{{- end -}}
{{- with resources.FromString (printf "%s.txt" $key) (printf "%s\n" $key) -}}
  {{- $noop := .Publish -}}
{{- end -}}
{{- $urls := slice -}}
{{- range .Site.Pages -}}
  {{- with .OutputFormats.Get "html" -}}
    {{- $urls = $urls | append .Permalink -}}
  {{- end -}}
{{- end -}}
{{- dict "host" $baseURL.Host "key" $key "urlList" ($urls | uniq | sort) | jsonify (dict "indent" "  ") -}}

Modification Time Manifest

layouts/index.mtimes.txt requires enableGitInfo = true and emits one relative path and timestamp pair for each generated output. It folds the later value of page AuthorDate and CommitDate into an optional HUGO_MINIMUM_MTIME lower bound, maps each output format to its generated file path, and formats UTC timestamps.

HUGO_MINIMUM_MTIME should be an ISO 8601 timestamp, for example the active theme’s last commit time; when absent, the template falls back to the Unix epoch beginning.

{{- $minimumMTime := time.AsTime "1970-01-01T00:00:00Z" -}}
{{- with os.Getenv "HUGO_MINIMUM_MTIME" -}}
  {{- $minimumMTime = time.AsTime . -}}
{{- end -}}
{{- $siteMTime := $minimumMTime -}}
{{- range .Site.Pages -}}
  {{- with .GitInfo -}}
    {{- $gitMTime := .AuthorDate -}}
    {{- if gt .CommitDate.Unix $gitMTime.Unix -}}
      {{- $gitMTime = .CommitDate -}}
    {{- end -}}
    {{- if gt $gitMTime.Unix $siteMTime.Unix -}}
      {{- $siteMTime = $gitMTime -}}
    {{- end -}}
  {{- end -}}
{{- end -}}
{{- range .Site.Pages -}}
  {{- $mtime := $siteMTime -}}
  {{- with .GitInfo -}}
    {{- $mtime = $minimumMTime -}}
    {{- $gitMTime := .AuthorDate -}}
    {{- if gt .CommitDate.Unix $gitMTime.Unix -}}
      {{- $gitMTime = .CommitDate -}}
    {{- end -}}
    {{- if gt $gitMTime.Unix $mtime.Unix -}}
      {{- $mtime = $gitMTime -}}
    {{- end -}}
  {{- end -}}
  {{- $touchTime := $mtime.UTC.Format "200601021504.05" -}}
  {{- range .OutputFormats -}}
    {{- if ne (strings.ToLower .Name) "mtimes" -}}
      {{- $path := strings.TrimPrefix "/" .RelPermalink -}}
      {{- if or (eq $path "") (strings.HasSuffix $path "/") -}}
        {{- $path = printf "%sindex.html" $path -}}
      {{- end }}
      {{- printf "%s %s\n" $path $touchTime -}}
    {{- end -}}
  {{- end -}}
{{- end -}}
{{- $siteTouchTime := $siteMTime.UTC.Format "200601021504.05" -}}
{{- with .Site.Home.OutputFormats.Get "indexnow" -}}
  {{- $key := strings.TrimSuffix ".json" (path.Base .RelPermalink) -}}
  {{- printf "%s.txt %s\n" $key $siteTouchTime -}}
{{- end -}}

Publishing

The publishing proceeds in ordered stages. The first stage builds a clean Hugo output and passes the optional minimum mtime as the last commit time touching .gitmodules or themes/*:

env HUGO_MINIMUM_MTIME=$(git log -1 --format=%cI -- .gitmodules themes/*) \
    hugo \
    --cleanDestinationDir \
    --gc

The generated manifest is then applied as filesystem metadata:

while read -r file mtime; do
    env TZ=UTC touch -t "$mtime" "public/$file"
done < public/mtimes.txt

mtimes.txt turns Hugo’s logical output list into HTTP validators. The clean destination removes stale files; the manifest pass prevents build time from becoming deployed modification time.

find public -type f -not -iname '*.gz' \
    -exec gzip -9 -k -n -f {} \; \
    -exec touch -m -r {} {}.gz \;

Precompression follows mtime restoration. Because find predicates are evaluated left to right, the second -exec runs only after gzip succeeds for the same file; it assigns each .gz artifact the source mtime, keeping compressed and uncompressed variants aligned for web servers that serve precompressed files directly².

rsync \
    -az \
    --delete-after \
    public/ \
    <some.server.com>:/var/www/htdocs/~user

The generated IndexNow JSON is submitted only after rsync completes:

curl \
    -XPOST \
    -d "@$(ls -1 public/indexnow-*.json | head -n 1)" \
    -H 'Content-Type: application/json' \
    https://api.indexnow.org/indexnow

The full URL list suits low volume, rarely updated sites; larger or frequently updated sites require selective change sets.

After publication, public/mtimes.txt and public/indexnow-*.json are operational artifacts: delete them before synchronisation, store them on the server as publication records, or ignore them through web server configuration.

If named publish.sh, the script can be installed as a Git pre-push hook with ln -sf ../../publish.sh .git/hooks/pre-push.