Incremental activities

[esjee]

This is a feature based on the discussion in #25. Essentially, it allows you to fetch all activities of your program incrementally. The idea here is this acts like a stream; you start at one point and keep moving forward in time until you run out of activities. As mentioned in #25, this feature has multiple usages:

• The ability to detect new activity on a report or a new report coming in using a single endpoint.
• Being able to take actions on reports based on user activity. Automatically assign after triaging, for example.
• Monitor activities on program, acting like an audit log.

The code is still a bit rough, but I envision it being used something like this:

program = HackerOne::Client::Program.find 'my_program'
updated_at_after = ... # load previous session in some fashion
incremental_activities = program.incremental_activities(updated_at_after: updated_at_after)

incremental_activities.traverse do |activity|
  # Loop through every single activity in order of creation
end

# Save progress in some fashion; any activities processed during this run will not be processed again

Before I go on writing tests, I'd be curious to see what you think of this feature, and if you have any suggestions on how to improve it.

[Willianvdv]

traverse is the word cool kids use

[esjee]

😎

[oreoshake]

flatmap

[Willianvdv]

@oreoshake wow, you're so cool

[Willianvdv]

Is your plan to store the latest fetched thing in this file? If so, that isn't going to work if you have more than one machine that uses this lib and I would let this up to the user how to store this data.

[esjee]

Yup, that's the plan! We need some kind of persistent storage to store the updated_at_after value, which essentially acts like a cursor.

I think you have a point, let's remove this logic?

[Willianvdv]

Isn't while next_page what you want?

[Willianvdv]

Nice feature! If this works, we can remove a lot of request to our servers :D

[oreoshake]

I'll take another pass over this but it I'm having some trouble tracing through the code (maybe I haven't had enough ☕

[oreoshake]

I think avoiding memoization here can clean up some of the state introduced by the ivars.

[oreoshake]

This method name is a little hard to grok.

[oreoshake]

For this file in general, I think the tiny methods might work better as inline code with a comment rather than a separate one line method that's only called in one place. I think avoiding ivars in favor of locals will help readability. Double emphasis on think.

[esjee]

I tried cleaning it up a bit. Is this helpful?

[oreoshake]

Looks like there's a merge conflict, otherwise 👍

[esjee]

It appears there's a bit of an issue with the endpoint, in that it may get stuck repeating the last page infinitely. I'm hoping I'll be able to fix this, and I'll finish off this PR at that point.

[esjee]

Alright! I'm happy with the PR in it's current state. Curious to know what you think!