not finding syndication links on davidjohnmead.com

[snarfed]

...specifically on http://davidjohnmead.com/blog/ . IRC discussion. thanks again for the nudge @davidmead! cc @dshanske, @gRegorLove.

bridgy accounts:

• https://brid.gy/twitter/davidmead
• https://brid.gy/instagram/davidjohnmead

example posts:

• http://davidjohnmead.com/blog/2017/07/03/17240/
• http://davidjohnmead.com/blog/2017/07/05/17500/

mf2 looks fine: https://mf2.kylewm.com/?url=http://davidjohnmead.com/blog/

{
 "items": [
  {
   "type": ["h-feed"],
   "properties": {
    "bridgy-omit-link": [""],
    "name": ["Skip to content..."]
   },
   "children": [
    {
     "type": ["h-entry"],
     "properties": {
      "content": [
       {
        "html": "Couldn\u2019t come at a better time. Count me in!<br/>\n/cc @brad_frost @bradcolbow",
        "value": "Couldn\u2019t come at a better time. Count me in!\n/cc @brad_frost @bradcolbow"
       }
      ],
      "in-reply-to": ["https://twitter.com/RustbeltRefresh/status/882664320223653888"],
      "syndication": ["https://twitter.com/davidmead/status/882992076249595904"],
      "name": ["Posted on 2017-07-06..."]
     }
    },
    "..."

logs don't have anything, e.g. this one:

refetching h-feed for source davidmead (Twitter)
attempting to refetch h-feed for davidmead (Twitter)
fetching author url http://davidjohnmead.com/blog/
requests.get http://davidjohnmead.com/blog/ {'headers': {'User-Agent': '...'}}
Updating davidmead (Twitter) /twitter/davidmead : {'last_polled': datetime.datetime(2017, 7, 6, 0, 58, 47, 11660), 'last_hfeed_refetch': datetime.datetime(2017, 7, 6, 0, 58, 50, 796820), 'poll_status': 'ok', 'recent_private_posts': 0, 'last_public_post': datetime.datetime(2017, 7, 4, 18, 59, 21)}
Added poll task 4601411735372278371 with args {'countdown': 1936.7439229166052}

[chrisaldrich]

Perhaps related to this?
pfefferle/wordpress-webmention#116
Particularly with instagram involved? I don't see a link to his feed and his feed (/blog/) isn't on his front page, so bridgy may not be finding it for that reason.

[gRegorLove]

@chrisaldrich /blog is linked as the rel=me from Twitter so that's not it. Bridgy logs show it's looking for syndication links on /blog, it just didn't find this one for some reason. Appears to have started ~3 days ago, previously working.

[davidmead]

@snarfed Did you change anything? I just got a like on Twitter show up as a comment on my blog.

See http://davidjohnmead.com/blog/2017/07/06/17509/.

Previous ones still not showing up though.

[snarfed]

@davidmead i haven't changed anything related to this, no.

i actually wonder if @gRegorLove may be right. you only show 10 posts per page, and your post velocity is high enough (with checkins etc) that posts may occasionally drop off the front page between bridgy's crawls every 6h. still seems doubtful, but possible.

[davidmead]

@snarfed Do they have to be on the front page if I go into Brid.gy and ask it to re-scan?

[davidmead]

@snarfed You know, maybe @gRegorLove is right. I've had 4 more replies to a post syndicated to Twitter come in.

That does seem to be a bit of a downside, if the posts have to be on the homepage to get a webmention passed.

[snarfed]

@snarfed Do they have to be on the front page if I go into Brid.gy and ask it to re-scan?

no, but something else is still unhappy. i'm still debugging.

That does seem to be a bit of a downside, if the posts have to be on the homepage to get a webmention passed.

yup! you can work around it by including <link rel="feed">(s) that point to the next page(s). https://brid.gy/about#link

[snarfed]

notes to myself:

first, i was wrong, post velocity isn't the problem, since based on the Last syndication link found lines on your user pages, bridgy's normal crawls hasn't found any syndication links on your blog since your mf2 change.

second, plugging the post URLs into the resend box finds the syndication links, but doesn't retry responses it's already found. that's a bug.

[snarfed]

aha, figured it out. @davidmead the posts in your feed don't have u-urls. :P see e.g. https://mf2.kylewm.com/?url=http://davidjohnmead.com/blog/ . i'll add something about that to the docs, and i'll keep this open to track the resend bug.

[gRegorLove]

Interesting note: I bet before a few days ago, David's posts only had hEntry markup. The microformats2 backcompat parsing would correctly upgrade rel=bookmark to u-url in that case. Once a root microformats2 class was introduced (h-entry), the backcompat rules are no longer applied during the parse.

This led me to catching a bug in php-mf2. It's always upgrading rel=bookmark regardless of mf2 or mf1 root classes. mf2py has it correct here.

[snarfed]

ok, fixed the retry/discover bug. tentatively closing. thanks again for reporting, @davidmead! and just a reminder, bridgy won't handle your blog posts automatically until you add u-url to them.