Include link checker in CI

[dblock]

Description

Similar to opensearch-project/documentation-website#3584.

• Run link checker in CI, fail PRs on broken links.
• Improve the homegrown link-checker a bit, make HEAD requests, etc.
• Fix many broken links.
• Closes [DOC] Broken links in sql ODBC docs documentation-website#3583

Check List

• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the BSD-3-Clause License.

[krisfreedain]

@dblock - so the whole build fails if a link it broken? I don't think that's what we want to implement for the website.
Also, while this fixed a bunch of links (thank you) - it also completely deleted a partner record - also not something we want to implement on the website, I'd rather get a report to allow us to follow up on the broken links rather than deletions and failures.

[dblock]

Thanks for the review @krisfreedain!

1. Yes, the build will fail if a link is broken. I will move this into a cron, but in the past I've seen that cause us to stop paying attention to broken links. Are you sure? Any contributor that runs into a broken link will tend to fix it if they run into that obstacle, which I think accomplishes the best outcome of links being rarely broken.
2. I think that partner no longer exists, the website is MIA. Let me know what you want me to do with it: I could instead remove the link, or something else?

[krisfreedain]

1. Yeah, I'd love to have a weekly / monthly report on broken links if you build it out as a cron job. That would be helpful (as demonstrated by the updates you found) - but I don't want to force-break the build. When a contributor files a PR, one of us that are active maintainers should be reviewing them prior to publish. Broken links are less common at that time - but a report to catch them if they break later is actionable.
2. Yeah - let's keep it, but I'm OK with removing the link - myself and @pajuric can follow up with them from the partner side and see what is going on. (this partner was added via PR: Request for Smart Search Tools and Weblink Technology as partners #1381 )

thanks @dblock !

[dblock]

@krisfreedain

1. Updated job to run on push (every change to main). It will auto-cut an issue if a link is broken, e.g. [AUTOCUT] Broken links dblock/project-website#3
2. Removed partner link, kept the partner.

[krisfreedain]

this is great - thank you @dblock