Prioritize files over folders (or don't include folders) in `link-to-changelog-file` #4066

A follow-up issue for the comment in #3999 as it doesn't make much sense to track this in an already closed issue.