Adding SFO scrapers

[patcon]

I'll just start an issue to track this, in case anyone else is working on it or interested.

Currently, I've got people scrapers working. This isn't ready for merge yet

cc: @tdooner

[patcon]

Also, I've written to a feedback email address (presumably the clerk's office) to request that more information be added to the People page, so that we have data comparable to Chicago coming directly from Legistar.

• https://sfgov.legistar.com/People.aspx
• https://chicago.legistar.com/People.aspx

[fgregg]

I would really recommend you use the legistar base scraper, like I do for Chicago and NYC. https://github.com/opencivicdata/python-legistar-scraper

[patcon]

Odd. I thought I copied this from Chicago and edited from there. I'll investigate where I went astray

EDIT: Seems ChicagoPeopleScraper is using the old way, so I'll steal from NY. Thanks for the heads up

[fgregg]

you would be better off using nyc as your template. Chicago is an older scraper, in some parts, and is not fully using python-legistar

[patcon]

Seems that I can't get most of the benefits of the new PeopleScraper until the city adds a "District" column, so going to leave the people scraper as-is until they get back to me on that.

[fgregg]

Are you saying that python-legistar assumes that a column exist that does
not exist in sf?

On Thu, Feb 18, 2016 at 11:22 PM Patrick Connolly [email protected]
wrote:

Seems that I can't get most of the benefits of the new PeopleScraper until
the city adds a "District" column, so going to leave the people scraper
as-is until they get back to me on that.

—
Reply to this email directly or view it on GitHub
#107 (comment)
.

[patcon]

Yeah, but it's a reasonable one. I'm going to push on the clerk to get district in

EDIT: Ah wait. no. I misread you there. I was saying much of the benefit of using python-legistar is lost without district. What I have pulls in just as much as python-legistar. So I'm going to hold off on re-doing it until "district" column is added.

[patcon]

But I digress. I have something working, and I'm going to leave it until I get can more value from your better way. That's all I'm saying. :)

[fgregg]

got it.

On Thu, Feb 18, 2016 at 11:57 PM Patrick Connolly [email protected]
wrote:

But I digress. I have something working, and I'm going to leave it until I
get can more value from your better way. That's all I'm saying. :)

—
Reply to this email directly or view it on GitHub
#107 (comment)
.

[fgregg]

What's the status of this?

[patcon]

Hi! It was working last I touched it, but I'm back in Toronto, and so not using it anymore. Unless someone from SF wanted to adopt, probably good to close it out?