Having tried NTFS, ext4 and btrfs, the difference is not noticeable (though NTFS is buggy on Linux)
Btrfs I believe has compression built in so is good for large libraries but realistically ext4 is the easiest and simplest way to do so I just use that nowadays