add superlove & CFAA (otwarchive sites) support

calibre-plugin/plugin-defaults.ini

@@ -2690,6 +2690,330 @@ slow_down_sleep_time:2
 ## datePublished_format
 #datechapter_format:%%Y-%%m-%%d
 
+[superlove.sayitditto.net]
+## superlove.sayitditto.net is the third OTW-archive site.  Note that
+## ao3categories is still used, but labeled "superlove Categories".
+
+## Some sites require login (or login for some rated stories) The
+## program can prompt you, or you can save it in config.  In
+## commandline version, this should go in your personal.ini, not
+## defaults.ini.
+#username:YourName
+#password:yourpassword
+
+## Note that if 'Hide warnings' and/or 'Hide Additional Tags' are
+## enabled in the account settings, the OTW adapter will not be able
+## to fetch them while logged-in.
+
+## In order to get bookmarktags and bookmarksummary, you need to login
+## all the time.  This defaults to off to save time and network
+## traffic.  Requires valid username and password when true.
+#always_login:false
+
+## Some sites also require the user to confirm they are adult for
+## adult content.  In commandline version, this should go in your
+## personal.ini, not defaults.ini.
+#is_adult:true
+
+## OTW offers a page to download the entire work at once.  This can
+## save a lot of network traffic if there are many chapters.
+## True by built-in default.
+#use_view_full_work:true
+
+## superlove.sayitditto.net stories allow chapters to be added out of
+## order.  So the newest chapter may not be the last one.  FanFicFare update
+## doesn't like that.  If do_update_hook is uncommented and set true,
+## the adapter will discard all existing chapters from the newest one
+## on when updating to enforce accurate chapters.
+## Starting July 2015, FFF stores chapter URLs in the chapter files.
+## Stories downloaded after that shouldn't need this setting anymore.
+#do_update_hook:false
+
+## Some adapters collect additional meta information beyond the
+## standard ones.  They need to be defined in extra_valid_entries to
+## tell the rest of the FanFicFare system about them.  They can be
+## used in include_subject_tags, titlepage_entries,
+## extra_titlepage_entries, logpage_entries, extra_logpage_entries,
+## and include_in_* config items.  You can also add additional entries
+## here to build up composite metadata entries.
+## superlove.sayitditto.net, for example, fills genre (a standard
+## entry) as the composite offreeformtags, ao3categories in
+## include_in_genre.  If there's ever more than 4 series, add
+## series04,series04Url etc.
+extra_valid_entries:fandoms, freeformtags, freefromtags,
+ ao3categories, comments, chapterslashtotal, chapterstotal, kudos,
+ hits, bookmarks, collections, byline, bookmarked, bookmarktags,
+ bookmarksummary, bookmarkprivate, bookmarkrec, subscribed,
+ markedforlater, restricted, series00, series01, series02, series03,
+ series00Url, series01Url, series02Url, series03Url, series00HTML,
+ series01HTML, series02HTML, series03HTML
+fandoms_label:Fandoms
+freeformtags_label:Freeform Tags
+freefromtags_label:Freeform Tags
+ao3categories_label:superlove Categories
+comments_label:Comments
+chapterslashtotal_label:Chapters/Total Chapters
+chapterstotal_label:Total Chapters
+kudos_label:Kudos
+hits_label:Hits
+collections_label:Collections
+## Count of bookmarks on story by all users
+bookmarks_label:Bookmarks
+## Tags & Summary from *your* bookmark on the story.  Only collected
+## if always_login:true
+bookmarked_label:I Bookmarked Story
+bookmarktags_label:My Bookmark Tags
+bookmarksummary_label:My Bookmark Summary
+bookmarkprivate_label:My Bookmark Private
+bookmarkrec_label:My Bookmark Rec
+markedforlater_label:I Marked for Later
+subscribed_label:I Subscribed
+restricted_label:Restricted to Registered Users
+series00HTML_label:Series
+series01HTML_label:Additional Series
+series02HTML_label:Additional Series
+series03HTML_label:Additional Series
+
+## Assume entryUrl, apply to "<a class='%slink' href='%s'>%s</a>" to
+## make entryHTML.
+make_linkhtml_entries:series00,series01,series02,series03
+
+## OTW doesn't have anything it calls 'genre'.  The adapter used to be
+## hardcoded to include the site specific metadata freeformtags &
+## ao3categories in the standard metadata field genre.  By making it
+## configurable, users can change it.
+include_in_genre: freeformtags, ao3categories
+
+## OTW uses the word 'category' differently than most sites.  The
+## adapter used to be hardcoded to include the site specific metadata
+## fandom in the standard metadata field category.  By making it
+## configurable, users can change it.
+include_in_category:fandoms
+
+## freeformtags was previously typo'ed as freefromtags.  This way,
+## freefromtags will still work for people who've used it.
+include_in_freefromtags:freeformtags
+
+## chapterslashtotal contains 1/3 or 1/1 or 3/? etc as reported by OTW.
+## chapterstotal is just the total chapters part after the /
+include_in_chapterstotal:chapterslashtotal.NOREPL
+add_to_replace_metadata:
+ chapterstotal=>^([0-9])+/(.+)$=>\2
+
+## adds to titlepage_entries instead of replacing it.
+#extra_titlepage_entries: fandoms,freeformtags,ao3categories,comments,chapterslashtotal,chapterstotal,kudos,hits,bookmarks,bookmarked,bookmarktags,bookmarksummary,series01HTML,series02HTML,series03HTML,byline
+
+## adds to include_subject_tags instead of replacing it.
+#extra_subject_tags:fandoms,freeformtags,ao3categories
+
+## OTW chapters can include several different types of notes.  We've
+## traditionally included them all in the chapter text, but this allows
+## you to customize which you include.  Copy this parameter to your
+## personal.ini and list the ones you don't want.
+#exclude_notes:authorheadnotes,chaptersummary,chapterheadnotes,chapterfootnotes,authorfootnotes,inspiredlinks
+
+## OTW authorfootnotes and inspiredlinks end up in the 'last' chapter,
+## but if updated, then there's a new 'last' chapter, leaving multiple
+## copies.  This option applies the 'skip_on_ffdl_update' class to
+## those tags which means they will be removed during epub reading for
+## update.  This will only effect chapters added after turning the
+## setting on.
+## Result: Only the last chapter will have end notes.
+## Side-effect: An 'Update Always' that doesn't add a new last
+## chapter will also remove the end notes.
+#remove_authorfootnotes_on_update:false
+
+## OTW is blocking people more aggressively.  If you download fewer
+## stories less often you can likely get by with reducing this sleep.
+slow_down_sleep_time:2
+
+## OTW allows users to archive stories they didn't write in certain
+## cases.  These are indicated by showing a byline such as:
+## 'Orig Author [archived by Archivist Author]'
+## If use_archived_author is set true (and there's only one author
+## listed), author will be set to 'Orig Author' instead of 'Archivist
+## Author'.  authorUrl will still point to the Archivist Author's page.
+#use_archived_author:false
+
+## OTW chapters have 'date' available for each chapter.
+## These can be used with custom output (see
+## https://github.com/JimmXinu/FanFicFare/wiki/CustomOutput ) or with
+## chapter_title_*_pattern settings.
+## Examples for html & epub:
+#[html]
+#tocpage_entry:
+# <a href="#section${index04}">${chapter}</a> ${date}<br />
+#[epub]
+#tocpage_entry:
+# <a href="file${index04}.xhtml">${chapter}</a> ${date}<br /><br />
+
+## The 'date' value for chapters mentioned above can be formated with
+## datechapter_format.  Otherwise it will default to
+## datePublished_format
+#datechapter_format:%%Y-%%m-%%d
+
+[www.cfaarchive.org]
+## cfaa.sayitditto.net is the third OTW-archive site.  Note that
+## ao3categories is still used, but labeled "cfaa Categories".
+
+## Some sites require login (or login for some rated stories) The
+## program can prompt you, or you can save it in config.  In
+## commandline version, this should go in your personal.ini, not
+## defaults.ini.
+#username:YourName
+#password:yourpassword
+
+## Note that if 'Hide warnings' and/or 'Hide Additional Tags' are
+## enabled in the account settings, the OTW adapter will not be able
+## to fetch them while logged-in.
+
+## In order to get bookmarktags and bookmarksummary, you need to login
+## all the time.  This defaults to off to save time and network
+## traffic.  Requires valid username and password when true.
+#always_login:false
+
+## Some sites also require the user to confirm they are adult for
+## adult content.  In commandline version, this should go in your
+## personal.ini, not defaults.ini.
+#is_adult:true
+
+## OTW offers a page to download the entire work at once.  This can
+## save a lot of network traffic if there are many chapters.
+## True by built-in default.
+#use_view_full_work:true
+
+## cfaa.sayitditto.net stories allow chapters to be added out of
+## order.  So the newest chapter may not be the last one.  FanFicFare update
+## doesn't like that.  If do_update_hook is uncommented and set true,
+## the adapter will discard all existing chapters from the newest one
+## on when updating to enforce accurate chapters.
+## Starting July 2015, FFF stores chapter URLs in the chapter files.
+## Stories downloaded after that shouldn't need this setting anymore.
+#do_update_hook:false
+
+## Some adapters collect additional meta information beyond the
+## standard ones.  They need to be defined in extra_valid_entries to
+## tell the rest of the FanFicFare system about them.  They can be
+## used in include_subject_tags, titlepage_entries,
+## extra_titlepage_entries, logpage_entries, extra_logpage_entries,
+## and include_in_* config items.  You can also add additional entries
+## here to build up composite metadata entries.
+## cfaa.sayitditto.net, for example, fills genre (a standard
+## entry) as the composite offreeformtags, ao3categories in
+## include_in_genre.  If there's ever more than 4 series, add
+## series04,series04Url etc.
+extra_valid_entries:fandoms, freeformtags, freefromtags,
+ ao3categories, comments, chapterslashtotal, chapterstotal, kudos,
+ hits, bookmarks, collections, byline, bookmarked, bookmarktags,
+ bookmarksummary, bookmarkprivate, bookmarkrec, subscribed,
+ markedforlater, restricted, series00, series01, series02, series03,
+ series00Url, series01Url, series02Url, series03Url, series00HTML,
+ series01HTML, series02HTML, series03HTML
+fandoms_label:Fandoms
+freeformtags_label:Freeform Tags
+freefromtags_label:Freeform Tags
+ao3categories_label:cfaa Categories
+comments_label:Comments
+chapterslashtotal_label:Chapters/Total Chapters
+chapterstotal_label:Total Chapters
+kudos_label:Kudos
+hits_label:Hits
+collections_label:Collections
+## Count of bookmarks on story by all users
+bookmarks_label:Bookmarks
+## Tags & Summary from *your* bookmark on the story.  Only collected
+## if always_login:true
+bookmarked_label:I Bookmarked Story
+bookmarktags_label:My Bookmark Tags
+bookmarksummary_label:My Bookmark Summary
+bookmarkprivate_label:My Bookmark Private
+bookmarkrec_label:My Bookmark Rec
+markedforlater_label:I Marked for Later
+subscribed_label:I Subscribed
+restricted_label:Restricted to Registered Users
+series00HTML_label:Series
+series01HTML_label:Additional Series
+series02HTML_label:Additional Series
+series03HTML_label:Additional Series
+
+## Assume entryUrl, apply to "<a class='%slink' href='%s'>%s</a>" to
+## make entryHTML.
+make_linkhtml_entries:series00,series01,series02,series03
+
+## OTW doesn't have anything it calls 'genre'.  The adapter used to be
+## hardcoded to include the site specific metadata freeformtags &
+## ao3categories in the standard metadata field genre.  By making it
+## configurable, users can change it.
+include_in_genre: freeformtags, ao3categories
+
+## OTW uses the word 'category' differently than most sites.  The
+## adapter used to be hardcoded to include the site specific metadata
+## fandom in the standard metadata field category.  By making it
+## configurable, users can change it.
+include_in_category:fandoms
+
+## freeformtags was previously typo'ed as freefromtags.  This way,
+## freefromtags will still work for people who've used it.
+include_in_freefromtags:freeformtags
+
+## chapterslashtotal contains 1/3 or 1/1 or 3/? etc as reported by OTW.
+## chapterstotal is just the total chapters part after the /
+include_in_chapterstotal:chapterslashtotal.NOREPL
+add_to_replace_metadata:
+ chapterstotal=>^([0-9])+/(.+)$=>\2
+
+## adds to titlepage_entries instead of replacing it.
+#extra_titlepage_entries: fandoms,freeformtags,ao3categories,comments,chapterslashtotal,chapterstotal,kudos,hits,bookmarks,bookmarked,bookmarktags,bookmarksummary,series01HTML,series02HTML,series03HTML,byline
+
+## adds to include_subject_tags instead of replacing it.
+#extra_subject_tags:fandoms,freeformtags,ao3categories
+
+## OTW chapters can include several different types of notes.  We've
+## traditionally included them all in the chapter text, but this allows
+## you to customize which you include.  Copy this parameter to your
+## personal.ini and list the ones you don't want.
+#exclude_notes:authorheadnotes,chaptersummary,chapterheadnotes,chapterfootnotes,authorfootnotes,inspiredlinks
+
+## OTW authorfootnotes and inspiredlinks end up in the 'last' chapter,
+## but if updated, then there's a new 'last' chapter, leaving multiple
+## copies.  This option applies the 'skip_on_ffdl_update' class to
+## those tags which means they will be removed during epub reading for
+## update.  This will only effect chapters added after turning the
+## setting on.
+## Result: Only the last chapter will have end notes.
+## Side-effect: An 'Update Always' that doesn't add a new last
+## chapter will also remove the end notes.
+#remove_authorfootnotes_on_update:false
+
+## OTW is blocking people more aggressively.  If you download fewer
+## stories less often you can likely get by with reducing this sleep.
+slow_down_sleep_time:2
+
+## OTW allows users to archive stories they didn't write in certain
+## cases.  These are indicated by showing a byline such as:
+## 'Orig Author [archived by Archivist Author]'
+## If use_archived_author is set true (and there's only one author
+## listed), author will be set to 'Orig Author' instead of 'Archivist
+## Author'.  authorUrl will still point to the Archivist Author's page.
+#use_archived_author:false
+
+## OTW chapters have 'date' available for each chapter.
+## These can be used with custom output (see
+## https://github.com/JimmXinu/FanFicFare/wiki/CustomOutput ) or with
+## chapter_title_*_pattern settings.
+## Examples for html & epub:
+#[html]
+#tocpage_entry:
+# <a href="#section${index04}">${chapter}</a> ${date}<br />
+#[epub]
+#tocpage_entry:
+# <a href="file${index04}.xhtml">${chapter}</a> ${date}<br /><br />
+
+## The 'date' value for chapters mentioned above can be formated with
+## datechapter_format.  Otherwise it will default to
+## datePublished_format
+#datechapter_format:%%Y-%%m-%%d
+
 [starslibrary.net]
 ## Some sites require login (or login for some rated stories) The
 ## program can prompt you, or you can save it in config.  In

fanficfare/adapters/__init__.py

@@ -142,6 +142,8 @@
 from . import adapter_fanfictionsfr
 from . import adapter_touchfluffytail
 from . import adapter_spiritfanfictioncom
+from . import adapter_superlove
+from . import adapter_cfaa
 
 ## This bit of complexity allows adapters to be added by just adding
 ## importing.  It eliminates the long if/else clauses we used to need

fanficfare/adapters/adapter_cfaa.py

@@ -0,0 +1,38 @@
+#  -*- coding: utf-8 -*-
+
+# Copyright 2020 FanFicFare team
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from __future__ import absolute_import
+import logging
+logger = logging.getLogger(__name__)
+
+from .base_otw_adapter import BaseOTWAdapter
+
+def getClass():
+    return CFAAAdapter
+
+class CFAAAdapter(BaseOTWAdapter):
+
+    def __init__(self, config, url):
+        BaseOTWAdapter.__init__(self, config, url)
+
+        # Each adapter needs to have a unique site abbreviation.
+        self.story.setMetadata('siteabbrev','cfaa')
+
+    @staticmethod # must be @staticmethod, don't remove it.
+    def getSiteDomain():
+        # The site domain.  Does have www here, if it uses it.
+        return 'www.cfaarchive.org'