Free/Open Licenses Data Package

[Ebnalblad]

I have been following the issues about Core Datasets as Data Packages project (http://data.okfn.org/roadmap/core-datasets) and I suggested to make data package for Free/Open Licenses datasets/awesome-data#85
@rgrp told me that you had already worked on Free/Open Licenses data, so what do you think about my suggestion ?

[rufuspollock]

I'm involved here btw - but others are too.

Here's my immediate thoughts:

• Yes, let's convert this to a simple tabular data package
• People depend on our data (this site gets deployed at http://licenses.opendefinition.org/) and has a bunch of json. We need not to break that for folks. I suggest we think properly about what API we want to offer and plan that out.
• Separately, but relatedly, we want to work out whether we want to break out the online / API stuff and move to main opendefinition repo - see Merge https://github.com/okfn/licenses with this repo ... opendefinition#7

[rufuspollock]

@Ebnalblad for context see #45

[Ebnalblad]

I am not sure I understand everything you said, I do not have very good technical knowledge.
I think we agreed to make simple tabular data package. Let me explain my idea more I am thinking of dataset contains tables, such as, the name of license, the link for it, can be use for commercial purposes ?, the last version of license and so on. By this way it would be easy for people who wants to use these licenses to get all information they want about them from single dataset.
I am open to other suggestions if I have the ability to do it .

[mlinksva]

Skimming http://data.okfn.org/doc/data-package it seems like the obvious thing to do is to add a datapackage.json file to this repository.

I'm not sure what is to be gained by converting the existing json data to tabular data, but if there's a use case of course go for it. Preferably generating one way so the data isn't maintained in yet another place.

Thanks!

[rufuspollock]

@mlinksva understood. Perhaps more importantly what do you think of #45? That really sets out a bit more clearly what is going on.

To be clear i think we will have both csv and json but standard form with be CSV. Why CSV? Simpler, anyone can open, and it makes valid tabular data packages :-)

[rufuspollock]

@Ebnalblad ok, here's what i suggest are your next steps:

• convert the list of all licenses (all.json) to csv - https://github.com/okfn/licenses/blob/master/licenses/groups/all.json - i hope it is obvious how to do this (if not just ask). I would put the csv file at the base of the repo. Make sure we do this in the master branch (i've now made default branch for repo the master branch)
• add a datapackage.json so this is a proper data package

These are all "enhancments" that take nothing away, plus in master branch so we won't affect the website (http://licenses.opendefinition.org/) which deploys off the github pages branch.

[Ebnalblad]

I searched online about how to convert json to csv and i tried my services to convert all.json file to csv file but it seems there are some mistakes in .json file
" Make sure we do this in the master branch (i've now made default branch for repo the master branch)"
You mean here https://github.com/okfn/licenses ?

[rufuspollock]

You really want to script that if you can. Very easy in a bit of code.

Maybe @jonschlinkert can help here

[Ebnalblad]

unfortunately I do not know. if there are some instructions on the web from scratch, maybe then I could do it.

[jonschlinkert]

Maybe @jonschlinkert can help here

meaning converting from csv to json?

[rufuspollock]

other way: json to csv

[jonschlinkert]

got it, thanks. I'll take a look. is this something that should be done before the other tasks, or alongside the other tasks?

@Ebnalblad what kinds of conversion issues were you getting? was it formatting-related (e.g. data structure) or were there errors in the actual content itself? (for example, nested/escaped quotes sometimes cause issues)

[Ebnalblad]

@jonschlinkert when i tried to convert the it i was getting errors messages that there are errors in some content lines .

[rufuspollock]

@jonschlinkert @Ebnalblad this should be super straight forward. We just want to turn https://github.com/okfn/licenses/blob/master/licenses/groups/all.json into a CSV and put it in a /data/ directory. (After that we create the data package).

Are you folks good to take a look at this.

[rufuspollock]

@jonschlinkert and @Ebnalblad any updates here? Should be very straightforward to do and once done we can move on to #45

[Ebnalblad]

@rgrp sorry for late i asked my friend to help me, is it ok now ?

[jonschlinkert]

sorry for the lack of response, had some deadline driven projects. I see there is a CSV in the repo, looks like that part is taken care of, what's next? something I can help with?

[rufuspollock]

@jonschlinkert the csv'ing and data packaging is done thanks to @Ebnalblad - back now to #45

[rufuspollock]

FIXED - thanks to @Ebnalblad

[mlinksva]

Is the CSV file now the one place to make changes? If it is pull requests #52 and #53 should be rewritten to update the CSV instead of individual license json files. Not clear to me what the state of this is, so asking folks who participated in the CSV'ing.

[rufuspollock]

@mlinksva good question - the basic answer should be "yes" - the issue is that is #45 is still incomplete and we need to complete that before that is truly the situation. Will seek to push forward with #45 asap. In the mean time I guess we can merge those pull requests though we should also patch the CSV file at the same time (this duping will go away with #45)

[rufuspollock]

FIXED. Closing again as i think we are all done here.