[Question] Please help troubleshooting my Caddy server. Can't get it to work since changing from IPv4 to IPv6

cross-posted from: feddit.nl/post/1094546

Hi everyone,

My router went from IPv4 to IPv6 after an update from my ISP back in April, and so I decided to try and get my selfhosted Raspberry Pi server to work with it. It’s been less trivial than I hoped it would be, though. It worked and was reachable when it still used IPv4, but it’s been out of the air since April.

I’m running Arch Linux ARM on the device and use networkd to connect it to the internet. I use now-dns.com to get a dynamic DNS and have connected it to my server using their Linux script.

This is my Caddyfile:


<span style="color:#323232;">{
</span><span style="color:#323232;">	debug
</span><span style="color:#323232;">	
</span><span style="color:#323232;">}
</span><span style="color:#323232;">
</span><span style="color:#323232;"># Jellyfin:
</span><span style="color:#323232;">https://myserver.now-dns.net:26347,
</span><span style="color:#323232;">https://myserver.now-dns.net:443,
</span><span style="color:#323232;">[(my IPv6 address here)]:26347 {
</span><span style="color:#323232;">	header / {
</span><span style="color:#323232;">		# Enable cross-site filter (XSS) 
</span><span style="color:#323232;">		# and tell browser to block detected attacks    
</span><span style="color:#323232;">		X-Frame-Options "Deny"
</span><span style="color:#323232;">		Content-Security-Policy "
</span><span style="color:#323232;">	            default-src 'self' data: blob:;
</span><span style="color:#323232;">	            style-src 'self' 'unsafe-inline' bootstrapcdn.com *.bootstrapcdn.com https://ctalvio.github.io/Monochromic/default_style.css https://ctalvio.github.io/Monochromic/jfblue_style.css https://ctalvio.github.io/Monochromic/jfpurple_style.css https://ctalvio.github.io/Monochromic/bottom-progress_style.css https://ctalvio.github.io/Monochromic/customcolor-advanced_style.css https://ctalvio.github.io/Monochromic/improve-performance_style.css https://fonts.googleapis.com/css2;
</span><span style="color:#323232;">	            script-src 'self' 'unsafe-inline' bootstrapcdn.com *.bootstrapcdn.com googleapis.com *.googleapis.com https://www.gstatic.com/cv/js/sender/v1/cast_sender.js worker-src 'self' blob:;
</span><span style="color:#323232;">	            font-src 'self' bootstrapcdn.com *.bootstrapcdn.com;
</span><span style="color:#323232;">	            img-src data: 'self' imgur.com *.imgur.com;
</span><span style="color:#323232;">	            form-action 'self';
</span><span style="color:#323232;">	            connect-src 'self' pokeapi.co;
</span><span style="color:#323232;">	            frame-ancestors 'self';
</span><span style="color:#323232;">	            report-uri {$CSP_REPORT_URI}
</span><span style="color:#323232;">	        "
</span><span style="color:#323232;">	}
</span><span style="color:#323232;">	reverse_proxy 127.0.0.1:8093
</span><span style="color:#323232;">	#reverse_proxy localhost:8093
</span><span style="color:#323232;">}
</span><span style="color:#323232;">
</span><span style="color:#323232;"># Nextcloud:
</span><span style="color:#323232;">#https://192.168.1.96:65002,
</span><span style="color:#323232;">https://myserver.now-dns.net:65001 {
</span><span style="color:#323232;">	root * /usr/share/webapps/nextcloud
</span><span style="color:#323232;">	file_server
</span><span style="color:#323232;">	#        log {
</span><span style="color:#323232;">	#                output file     /var/log/caddy/myserver.now-dns.net.log
</span><span style="color:#323232;">	#                format single_field common_log
</span><span style="color:#323232;">	#        }
</span><span style="color:#323232;">
</span><span style="color:#323232;">	#php_fastcgi 127.0.0.1:9000
</span><span style="color:#323232;">	#php_fastcgi unix//run/php-fpm/php-fpm.sock # veranderd naar correcte adres uit /etc/php/php-fpm.d/www.conf
</span><span style="color:#323232;">	php_fastcgi unix//run/nextcloud/nextcloud.sock # veranderd naar nieuwe correcte adres uit /etc/php/php-fpm.d/nextcloud.conf
</span><span style="color:#323232;">
</span><span style="color:#323232;">	header {
</span><span style="color:#323232;">		# enable HSTS
</span><span style="color:#323232;">		Strict-Transport-Security max-age=31536000;
</span><span style="color:#323232;">	}
</span><span style="color:#323232;">
</span><span style="color:#323232;">	redir /.well-known/carddav /remote.php/dav 301
</span><span style="color:#323232;">	redir /.well-known/caldav /remote.php/dav 301
</span><span style="color:#323232;">
</span><span style="color:#323232;">	# .htaccess / data / config / ... shouldn't be accessible from outside
</span><span style="color:#323232;">	@forbidden {
</span><span style="color:#323232;">		path /.htaccess
</span><span style="color:#323232;">		path /data/*
</span><span style="color:#323232;">		path /config/*
</span><span style="color:#323232;">		path /db_structure
</span><span style="color:#323232;">		path /.xml
</span><span style="color:#323232;">		path /README
</span><span style="color:#323232;">		path /3rdparty/*
</span><span style="color:#323232;">		path /lib/*
</span><span style="color:#323232;">		path /templates/*
</span><span style="color:#323232;">		path /occ
</span><span style="color:#323232;">		path /console.php
</span><span style="color:#323232;">	}
</span><span style="color:#323232;">
</span><span style="color:#323232;">	respond @forbidden 404
</span><span style="color:#323232;">}
</span>

(myserver.now-dns.net is not actually my server name, I changed it to stay a bit more anonymous. Maybe this is unnecessarily cautious, let me know if I should change this to my actual address to aid your help.)

This is a journalctl log from fresh after a Caddy restart:


<span style="color:#323232;">Aug 01 14:36:12 baspi2 systemd[1]: Starting Caddy web server...
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23895]: {"level":"info","ts":1690893373.0834036,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23895]: {"level":"warn","ts":1690893373.0915132,"msg":"No files matching import glob pattern","pattern":"/etc/caddy/conf.d/*"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23895]: {"level":"info","ts":1690893373.1047359,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x4394a00"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23895]: {"level":"info","ts":1690893373.1278725,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23895]: {"level":"info","ts":1690893373.1279871,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv1","https_port":443}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23895]: {"level":"info","ts":1690893373.1280322,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv1"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23895]: {"level":"info","ts":1690893373.128112,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv2"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23895]: {"level":"info","ts":1690893373.1328619,"logger":"tls.cache.maintenance","msg":"stopped background certificate maintenance","cache":"0x4394a00"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23895]: Valid configuration
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: caddy.HomeDir=/var/lib/caddy
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: caddy.AppDataDir=/var/lib/caddy
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: caddy.AppConfigDir=/etc/caddy
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: caddy.ConfigAutosavePath=/var/lib/caddy/autosave.json
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: caddy.Version=v2.6.4
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: runtime.GOOS=linux
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: runtime.GOARCH=arm
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: runtime.Compiler=gc
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: runtime.NumCPU=4
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: runtime.GOMAXPROCS=4
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: runtime.Version=go1.20.1
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: os.Getwd=/
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: LANG=C
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: PATH=/usr/local/sbin:/usr/local/bin:/usr/bin
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: NOTIFY_SOCKET=/run/systemd/notify
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: HOME=/var/lib/caddy
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: LOGNAME=caddy
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: USER=caddy
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: INVOCATION_ID=131202f1b6e3472bab7e6fc48933c731
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: JOURNAL_STREAM=8:2593614
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: SYSTEMD_EXEC_PID=23904
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: XDG_DATA_HOME=/var/lib
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: XDG_CONFIG_HOME=/etc
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.4270308,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":""}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"warn","ts":1690893373.4276912,"msg":"No files matching import glob pattern","pattern":"/etc/caddy/conf.d/*"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.4616253,"logger":"admin","msg":"admin endpoint started","address":"localhost:2019","enforce_origin":false,"origins":["//127.0.0.1:2019","//localhost:2019","//[::1]:2019"]}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.4650905,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x4e32000"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.4871185,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.4872386,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv1","https_port":443}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.4872835,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv1"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.4874046,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv2"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.9077604,"logger":"pki.ca.local","msg":"root certificate is already trusted by system","path":"storage:pki/authorities/local/root.crt"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.9084256,"logger":"http","msg":"enabling HTTP/3 listener","addr":":443"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.909473,"logger":"tls","msg":"cleaning storage unit","description":"FileStorage:/var/lib/caddy"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"debug","ts":1690893373.9139633,"logger":"http","msg":"starting server loop","address":"[::]:443","tls":true,"http3":true}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.9140959,"logger":"http.log","msg":"server running","name":"srv1","protocols":["h1","h2","h3"]}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.9144514,"logger":"http","msg":"enabling HTTP/3 listener","addr":":65001"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"debug","ts":1690893373.91526,"logger":"http","msg":"starting server loop","address":"[::]:65001","tls":true,"http3":true}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.9154122,"logger":"http.log","msg":"server running","name":"srv2","protocols":["h1","h2","h3"]}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"debug","ts":1690893373.9156892,"logger":"http","msg":"starting server loop","address":"[::]:80","tls":false,"http3":false}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.9158008,"logger":"http.log","msg":"server running","name":"remaining_auto_https_redirects","protocols":["h1","h2","h3"]}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.9160817,"logger":"http","msg":"enabling HTTP/3 listener","addr":":26347"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"debug","ts":1690893373.9165256,"logger":"http","msg":"starting server loop","address":"[::]:26347","tls":true,"http3":true}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.9165914,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2","h3"]}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.916624,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["(my IPv6 address here)","myserver.now-dns.net"]}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"info","ts":1690893373.917206,"logger":"tls","msg":"finished cleaning storage units"}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"warn","ts":1690893373.920347,"logger":"tls","msg":"stapling OCSP","error":"no OCSP stapling for [(my IPv6 address here)]: no OCSP server specified in certificate","identifiers":["(my IPv6 address here)"]}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"debug","ts":1690893373.920421,"logger":"tls.cache","msg":"added certificate to cache","subjects":["(my IPv6 address here)"],"expiration":1690917213,"managed":true,"issuer_key":"local","hash":"8aa98ab4d6a397ee8784859f4ba69d8df96d6d978247a3436a20cc8373cf9a8a","cache_size":1,"cache_capacity":10000}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"debug","ts":1690893373.920493,"logger":"events","msg":"event","name":"cached_managed_cert","id":"2420e703-5823-4962-ad5b-05a084aafacb","origin":"tls","data":{"sans":["(my IPv6 address here)"]}}
</span><span style="color:#323232;">Aug 01 14:36:13 baspi2 caddy[23904]: {"level":"debug","ts":1690893373.923109,"logger":"tls","msg":"loading managed certificate","domain":"myserver.now-dns.net","expiration":1697974414,"issuer_key":"acme-v02.api.letsencrypt.org-directory","storage":"FileStorage:/var/lib/caddy"}
</span><span style="color:#323232;">Aug 01 14:36:14 baspi2 caddy[23904]: {"level":"debug","ts":1690893374.1269143,"logger":"tls.cache","msg":"added certificate to cache","subjects":["myserver.now-dns.net"],"expiration":1697974414,"managed":true,"issuer_key":"acme-v02.api.letsencrypt.org-directory","hash":"7db3c32211ccb2942c5d329650e92ddd63cd9a17670eba2ce29476f3c3e3a741","cache_size":2,"cache_capacity":10000}
</span><span style="color:#323232;">Aug 01 14:36:14 baspi2 caddy[23904]: {"level":"debug","ts":1690893374.1271243,"logger":"events","msg":"event","name":"cached_managed_cert","id":"fc000be0-ac06-4ca2-aa53-c14c6fb3ae27","origin":"tls","data":{"sans":["myserver.now-dns.net"]}}
</span><span style="color:#323232;">Aug 01 14:36:14 baspi2 caddy[23904]: {"level":"info","ts":1690893374.1345215,"msg":"autosaved config (load with --resume flag)","file":"/var/lib/caddy/autosave.json"}
</span><span style="color:#323232;">Aug 01 14:36:14 baspi2 systemd[1]: Started Caddy web server.
</span><span style="color:#323232;">Aug 01 14:36:14 baspi2 caddy[23904]: {"level":"info","ts":1690893374.137206,"msg":"serving initial configuration"}
</span><span style="color:#323232;">Aug 01 14:36:14 baspi2 sudo[23887]: pam_unix(sudo:session): session closed for user root
</span>
  • I “opened” the necessary ports on my router (my router calls it that, I realise it’s probably more like I unblocked these ports in its IPv6 firewall)
  • I scanned the open ports with an IPv6 port scanner (this one) and it shows ports 80 and 443 to be open, as well as ports 65001 and 26347
  • I used my public IPv6 address to scan, as well as the myserver.now-dns.net address (which is actually another server name)
  • I used a smart phone unconnected to wifi to test, but the site cannot load there, either. I tested from other networks while away from home, too, which also didn’t work.

Unfortunately, I still cannot connect to the server with a browser. myserver.now-dns.net:26347 gives an ¨Unable to connect" error on Firefox.

I have been whittling away at this issue on and off since April and haven’t really made any big breakthroughs. What would be your first steps in troubleshooting this issue?

When I scan one of the open ports with an online tool, a message like this pops up in the journalctl log:

Aug 01 14:45:49 baspi2 caddy[23904]: {“level”:“debug”,“ts”:1690893949.6947021,“logger”:“http.stdlib”,“msg”:“http: TLS handshake error from [2a01:4f8:1c1c:2d4e::1]:50079: EOF”}

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • random
  • wartaberita
  • uselessserver093
  • Food
  • aaaaaaacccccccce
  • [email protected]
  • test
  • CafeMeta
  • testmag
  • MUD
  • RhythmGameZone
  • RSS
  • dabs
  • TheResearchGuardian
  • Ask_kbincafe
  • KbinCafe
  • Testmaggi
  • Socialism
  • feritale
  • oklahoma
  • SuperSentai
  • KamenRider
  • All magazines
    •