I think you still need to enable JXL in the config, but it seems to display just fine once enabled.
Adding support for JXL in windows was much more of a hassle and doesn’t always display properly in the file preview. Hopefully windows follows Apple’s step soon and adds native support.
I guess as a Web developer it won’t matter until the JXL toggle is enabled by default though.