Unfortunately since these are single pane windows most of the thermal performance issues will be due to the glass. Sealing this gap will help some, but it will be just a drop in the bucket.
Replacing the windows with insulated glass (two pane) windows is ideal. However that will be expensive.
The easiest/cheapest option will be the temporary cling-film type kits you can buy–you won’t be able to operate the windows, but that probably won’t be an issue in the winter. Properly sized/installed storm windows will help some too, and be significantly cheaper than new windows.