Figuring out nested asyncio loops

[honzajavorek]

WIP

• scrapy scraper doesn't scrape correct number of items (why?), seems like it skips some, always different ones (scrapy gives 50, actor gives 52, website says 53)
• fix pagination so that it only enqueues the next page
• actor scraper doesn't end - it's because of pagination! scrapy can correctly dedupe, actor cannot. task: get a minimal reproducible example and file a bug on SDK In the end @vdusek managed to reproduce this with minimal code and told me I don't have to debug this further
• inspect what upgrades I did recently, scrapy version, etc.
• fix parsing error
• cache optimization: gzip cache
• cache optimization: do lookups only for safe/idempotent requests leaving this up to the fingerprinter
• cache optimization: save cache under single key
• cache optimization: remove 10 expired cache keys on each close (garbage collector)
• run_scraper
• double logging, remove my custom logging mechanics
• remove Actor.log feat: Unify Apify and Scrapy to use single event loop & remove nest-asyncio apify/apify-sdk-python#390 (comment)
• merge Bump the python group with 3 updates #111
• after merging, put broken scraper back to schedule
• create pull request to apify sdk with cache