LinkedIn checks failing

[cfjedimaster]

Calls to LI urls are returning 999. I tried preferGET but it didn't help. It looks like you may need to tweak the user agent: https://stackoverflow.com/questions/27231113/999-error-code-on-head-request-to-linkedin.

[0x6b]

Hi @cfjedimaster, in order for us to investigate your issue, could you give us your test case? It looks LinkedIn's public profile seems not to be public; their authentication wall blocks me from accessing public-ish URL such as https://www.linkedin.com/company/linkedin and some other random profiles. Changing user-agent didn't help.

[cfjedimaster]

Yes, that was it I think: https://www.linkedin.com/in/raymondcamden/. I tried in Edge and got a modal about signing in to view full profiles.

[0x6b]

@cfjedimaster thanks for additional information. I confirmed that tweaking headers (user-agent, accept, accept-encoding) as the SO article did not work for the link. @azu do you have any idea? I think it might be difficult for the rule to check the actual content behind the auth wall.

$ curl --url "https://www.linkedin.com/in/raymondcamden/" -v --header "user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36" --header "accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8" --header "accept-encoding:gzip"
*   Trying 144.2.3.1...
* TCP_NODELAY set
* Connected to www.linkedin.com (144.2.3.1) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Mountain View; O=LinkedIn Corporation; CN=www.linkedin.com
*  start date: May 30 00:00:00 2018 GMT
*  expire date: Sep  1 00:00:00 2020 GMT
*  subjectAltName: host "www.linkedin.com" matched cert's "www.linkedin.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fcc0f002200)
> GET /in/raymondcamden/ HTTP/2
> Host: www.linkedin.com
> user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36
> accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
> accept-encoding:gzip
>
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 999
< date: Mon, 07 Jan 2019 12:08:29 GMT
< x-li-pop: prod-ehk1
< x-li-proto: http/2
< x-li-uuid: 17VPxp6PdxUg2yHJ4SoAAA==
< set-cookie: lidc="b=SGST04:g=2:u=1:i=1546862909:t=1546949309:s=AQG1hbPjxTW9zJWTxUYCpFgyuw6jmTc0"; Expires=Tue, 08 Jan 2019 12:08:29 GMT; domain=.linkedin.com; Path=/
< set-cookie: trkCode=gf; Max-Age=5
< set-cookie: trkInfo=AQHG340iI2OvmwAAAWgoNtZISY5e-V3UgOSyHNQih8WB-jQXU7K22YhVgMq4GAncO3x1-WvrgdMKr0cEoJ9hZAo38ow51nKJtZnCrmxzPRwuFGbNVrsD1pCSHRF7u6LGaIUOb4I=; Max-Age=5
< set-cookie: rtc=AQFDkVhoWhv4gwAAAWgoNtZIxyZD_s7ElGyJUlDJ3PoPQFl5E6FJHgThdiYiQf1kA1ujP-UUSMI_mAbMG4iFLRJ_eB9lSqqpIa1stLAl_ae6TFU2PKBB2vefR0av4oejwXKTV_odAMsY-VOehNpjvACdsRTrclMQsHhSEnN3r52LMHlpI0_UZmQik6ixeXJNIR1h7uzDvnkZehGrI05D8ZUkkNN6xDssAGcIEtPNSffKeqsi0HTyI_n8bkClqhHC9FGPW7oN; Max-Age=120; path=/; domain=.linkedin.com
< content-length: 1461
< content-type: text/html
<
<html><head>
<script type="text/javascript">
window.onload = function() {
  // Parse the tracking code from cookies.
  var trk = "bf";
  var trkInfo = "bf";
  var cookies = document.cookie.split("; ");
  for (var i = 0; i < cookies.length; ++i) {
    if ((cookies[i].indexOf("trkCode=") == 0) && (cookies[i].length > 8)) {
      trk = cookies[i].substring(8);
    }
    else if ((cookies[i].indexOf("trkInfo=") == 0) && (cookies[i].length > 8)) {
      trkInfo = cookies[i].substring(8);
    }
  }

  if (window.location.protocol == "http:") {
    // If "sl" cookie is set, redirect to https.
    for (var i = 0; i < cookies.length; ++i) {
      if ((cookies[i].indexOf("sl=") == 0) && (cookies[i].length > 3)) {
        window.location.href = "https:" + window.location.href.substring(window.location.protocol.length);
        return;
      }
    }
  }

  // Get the new domain. For international domains such as
  // fr.linkedin.com, we convert it to www.linkedin.com
  var domain = "www.linkedin.com";
  if (domain != location.host) {
    var subdomainIndex = location.host.indexOf(".linkedin");
    if (subdomainIndex != -1) {
      domain = "www" + location.host.substring(subdomainIndex);
    }
  }

  window.location.href = "https://" + domain + "/authwall?trk=" + trk + "&trkInfo=" + trkInfo +
      "&originalReferer=" + document.referrer.substr(0, 200) +
      "&sessionRedirect=" + encodeURIComponent(window.location.href);
}
</script>
* Connection #0 to host www.linkedin.com left intact
</head></html>

[azu]

I agree with @0x6b
It is overtask for this rule.

Current workaround is that just set ignore option.

{
  "rule": {
    "no-dead-link": {
      "ignore": ["https://www.linkedin.com/*"]
    }
  }
}