-
Notifications
You must be signed in to change notification settings - Fork 3
feat: add telemetry documentation #35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
9e0ce60
feat: add telemetry documentation
SgtPooki 41a2dbe
docs: fix markdown link to FAQs document
SgtPooki 7695322
docs: add FAQ for turning telemetry completely off
SgtPooki 1f1acc7
docs: minor updates about user data
SgtPooki ce7d99c
docs: add UI examples and steps for adding telemetry to repo
SgtPooki 788b34f
docs: reordering and rephrasing
SgtPooki 70e88fc
Update CollectedData.template.md
SgtPooki d19132a
Update docs/telemetry/CollectionPolicy.md
SgtPooki 02d9f7f
Update docs/telemetry/CollectionPolicy.md
SgtPooki 1570b54
Update docs/telemetry/CollectionPolicy.md
SgtPooki f2f2b65
Update docs/telemetry/CollectionPolicy.md
SgtPooki a3fd171
Update docs/telemetry/CollectionPolicy.md
SgtPooki 1a97fe7
docs: additional clarifications and updates
SgtPooki 8aef1e4
docs: more updates
SgtPooki 5114692
Merge branch 'main' into feature/telemetry-documentation
SgtPooki 1cd8e6d
Update README.md
SgtPooki b682ca7
Update docs/telemetry/CollectedData.template.md
SgtPooki c40f691
Update docs/telemetry/CollectionPolicy.md
SgtPooki ca11b03
Update docs/telemetry/CollectionPolicy.md
SgtPooki d698a85
chore: update countly url
SgtPooki 669bc3c
chore(docs): update collectedData.template links
SgtPooki 7d0f222
chore: use UPPER_SNAKE_CASE for telemetry docs
SgtPooki 9b93a1f
Update docs/telemetry/FAQs.md
SgtPooki 3930838
chore(docs): fix links
SgtPooki f9ff25a
chore(docs): add ipfs-gui team under lead maintainer
SgtPooki 02087b7
chore: remove multi-toggle images
SgtPooki 94ef59d
chore: grammar
SgtPooki b119380
chore: update example screenshot of ipfs-companion
SgtPooki 8181dbb
fix: exports
SgtPooki File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| # Collected telemetry data | ||
|
|
||
| Copied from https://github.com/ipfs-shipyard/ignite-metrics/blob/main/docs/telemetry/COLLECTED_DATA.template.md | ||
|
|
||
| <!-- | ||
| This is the default for projects using the ignite-metrics library. Each project's own `./docs/telemetry/COLLECTED_DATA.md` file should contain very specific information under each of the following sections: | ||
|
|
||
| * 'What metrics data DO we collect' | ||
|
|
||
| --> | ||
| ## What metrics data do we NOT collect | ||
|
|
||
| We do not collect any of the below listed items in our metric data: | ||
|
|
||
| 1. User identifiable data (names, email addresses, aliases, handles, etc.) | ||
| 2. CIDs | ||
| 3. IP addresses | ||
|
|
||
| Telemetry is sent to Countly instance at `countly.ipfs.tech`. You can read how they protect your data at [Security, Privacy, and Access FAQ](https://support.count.ly/hc/en-us/articles/360037501372-Security-Privacy-and-Access-FAQ) | ||
|
|
||
| ## What metrics data DO we collect | ||
|
|
||
| As a general rule, we collect only application data; no user data. Some examples of application data we collect are: | ||
|
|
||
| | Metric data name | Metric feature name | Metric trigger | Analytics use | Notes | | ||
| |:----------------:|---------------------|----------------|---------------|---------------| | ||
| | Fill in | Fill in | Fill in | Fill in | Fill in | | ||
| | | | | | | | ||
| | | | | | | | ||
|
|
||
| * "Metric data name" - The app-specific metric/event name we're using for this metric data. (e.g. APP_BOOTSTRAP_START) | ||
| * "Metric feature name" - The metric feature the event/metric data correlates to. The group the metric feature belongs to is defined in our [COLLECTION_POLICY](https://github.com/ipfs-shipyard/ignite-metrics/blob/main/docs/telemetry/COLLECTION_POLICY.md#metric-features-and-their-groupings). (e.g. Minimal) | ||
| * "Metric trigger" - An explanation covering when this metric data is triggered. (e.g. On Application init) | ||
| * "Analytics use" - An explanation about how this metric data is used for analytics. (e.g. Input to load time calculations) | ||
| * "Notes" - Any additional notes. (e.g. Used as a timestamp identifier for when an application is first loaded) | ||
|
|
||
| ## Other related documents | ||
|
|
||
| * [COLLECTION_POLICY](https://github.com/ipfs-shipyard/ignite-metrics/blob/main/docs/telemetry/COLLECTION_POLICY.md) | ||
| * [PRIVACY_POLICY](https://github.com/ipfs-shipyard/ignite-metrics/blob/main/docs/telemetry/PRIVACY_POLICY.md) | ||
| * [FAQs](https://github.com/ipfs-shipyard/ignite-metrics/blob/main/docs/telemetry/FAQs.md) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,138 @@ | ||
| # Telemetry collection policy | ||
|
|
||
| This document is the source of truth for our telemetry policies and practices across all Ignite team (IPFS GUI) projects. The discussion held at https://github.com/ipfs/ipfs-gui/issues/125 is the origin and inspiration for this library and all documents under `./docs/telemetry`. | ||
|
|
||
| The initial list of projects we're adding telemetry to was originally discussed at https://github.com/ipfs/ipfs-gui/issues/105. | ||
|
|
||
| For an up to date list of projects we own that follow this policy, please see: | ||
|
|
||
| * https://github.com/ipfs/ipfs-gui#primary-projects | ||
| * https://github.com/ipfs/ipfs-gui#other-ipfs-gui--tools-owned-projects | ||
|
|
||
| ## History | ||
|
|
||
| Before 2022Q4, telemetry was only implemented in https://github.com/ipfs/ipfs-webui and https://github.com/ipfs/ipfs-desktop, using our [self-hosted countly](https://count.ly/community-edition) server. The telemetry implemented within ipfs-webui and ipfs-desktop was opt-in only. i.e. metrics data was sent only if a user specifically opted-in to collecting metrics. | ||
|
|
||
| Our other projects did not implement telemetry or collection of any metrics data. The only exception was https://github.com/ipfs/ipfs-companion where we have had a [privacy policy since 2019-02-15](https://github.com/ipfs/ipfs-companion/blob/main/PRIVACY-POLICY.md). The privacy policy in ipfs-companion indicates that no data is collected whatsoever, and that will be changing in the future. | ||
|
|
||
| ## Metrics collection expectations | ||
|
|
||
| It's important to clarify that our intention for telemetry gathering within our projects is solely to prioritize our work efforts in improving the most useful features. Telemetry is used for usage analysis and is not violating our users expectations of privacy nor anonymity. | ||
|
|
||
| ### Application data vs User data | ||
|
|
||
| Our telemetry is solely focused on the collection of application data, not user data. | ||
|
|
||
| #### What is Application data? | ||
|
|
||
| Application data only contains information about events, modules, performance, and use of our applications, not the users themselves. | ||
|
|
||
| #### What is User data? | ||
|
|
||
| User data can vary across industries and applications, but is generally data that is used to classify or categorize users, typically for targeting classes or categories of users via campaigns or other marketing methods for various purposes. Think demographics, emails, physical addresses, personal preferences, and marketing. | ||
|
|
||
| As a general rule, we do not collect user or personal data. See our [Privacy Policy](./PRIVACY_POLICY.md) for more details. | ||
|
|
||
| ### Metric features and their groupings | ||
|
|
||
| The metric features in the first column name and their descriptions come directly from [countly's support website](https://support.count.ly/hc/en-us/articles/360037441932-Web-analytics-JavaScript-#features-for-consent). | ||
|
|
||
| | Metric feature name | Consent method | Metric feature group name | Metric feature description | What do we use this metric for? | | ||
| |:-------------------:|----------------|---------------------------|--------------------------------------------------------------------------------|------------------------------------------------------------------------------------------| | ||
| | sessions | Opt-out | Minimal | tracks when, how often, and how long users use your website | For determining unique daily/weekly/monthly active users in order to prioritize projects | | ||
| | views | Opt-out | Minimal | allows for the views/pages accessed by a user to be tracked | For tracking application pages and component views only | | ||
| | events | Opt-out | Minimal | allows your events to be sent to the server | Timed and one-off application events such as app_start, app_close, function_used, etc. | | ||
| | crashes | Opt-out | Performance | allows JavaScript errors to be tracked | Not used yet - Eventually for bug fixes and debugging | | ||
| | apm | Opt-out | Performance | allows performance tracking of application by recording traces | Not used yet - Eventually for bug fixes and debugging | | ||
| | scrolls | Opt-out | UX | allows a user’s scrolls to be tracked on the heatmap | Not used yet - Possibly for performance and UX optimizations | | ||
| | clicks | Opt-out | UX | allows a user’s clicks and link clicks to be tracked on the heatmap | Not used yet - Possibly for performance and UX optimizations | | ||
| | forms | Opt-out | UX | allows a user’s form submissions to be tracked | Not used yet - Possibly for bug fixes and debugging, performance and UX optimizations | | ||
| | star-rating | Opt-out | Feedback | allows user rating and feedback tracking through rating widgets | Not used yet - Possibly for getting user feedback more directly through our applications | | ||
| | feedback | Opt-out | Feedback | allows survey, nps rating and feedback tracking through feedback widgets | Not used yet - Possibly for getting user feedback more directly through our applications | | ||
| | location | Opt-out | Location | allows a user’s location (country, city area) to be recorded | N/A - we have no current need for collecting a user's location data. | | ||
| | attribution | N/A | N/A | allows the campaign from which a user came to be tracked | N/A - we do not have any plans to implement or utilize user campaigns | | ||
| | users | N/A | N/A | allows user information, including custom properties, to be collected/provided | N/A - we will not collect user information | | ||
|
|
||
| The code implementing these groupings can be found [here](./../../src/MetricsProvider.ts#L17-L30) | ||
|
|
||
| **Note:** The above groupings are our global settings. You will want to check the project specific `./docs/telemetry/COLLECTED_DATA.md` for variations. | ||
|
|
||
| ### What do the consent toggling UIs look like? | ||
|
|
||
| Consent toggling has two distinct views depending on the type of application implementing telemetry using this library: Projects without a settings page, and Projects with settings page(s). | ||
|
|
||
| Each toggle that a user sees will be tied to a specific "Metric feature group name" as listed in the above table. For now, we don't have any plans to allow more granular control than that, but please [open an issue](https://github.com/ipfs-shipyard/ignite-metrics/issues/new?assignees=&labels=need%2Ftriage&template=open_an_issue.md&title=) if you feel like that should change. | ||
|
|
||
| #### Projects without its own settings page(s) | ||
|
|
||
| In this case, we would follow the patterns discussed in https://github.com/ipfs/public-gateway-checker/issues/340 and https://github.com/ipfs-shipyard/ignite-metrics/issues/36. See https://pl-strflt.notion.site/Metrics-Collection-and-Consent-Language-0d4059f11d4d474bb76d00539d778d8d#3e2a579c4be043888b1f64bf300048b8 for up to date info. | ||
|
|
||
| 1. There will be a floating icon at the bottom left of the page: | ||
|
|
||
|  | ||
|
|
||
| 2. When clicked, that icon will expand into a modal that displays information about our telemetry collection | ||
|
|
||
|  | ||
|
|
||
|
|
||
| 3. When "manage settings" is clicked, the modal will update to allow you to enable/disable the project's implemented collection of groups of features as defined above in the "Metric features and their groupings" section. | ||
|
|
||
| Single toggle Example: | ||
|
|
||
|  | ||
|
|
||
| #### Projects with its own settings page(s) | ||
|
|
||
| The consent should conform to the UX expectations of the library it's being implemented in. | ||
|
|
||
| ##### IPFS-companion | ||
|
|
||
| It should look like the rest of the settings in the application. | ||
|
|
||
| Example with single toggle (PR at https://github.com/ipfs/ipfs-companion/pull/1117): | ||
|
|
||
|  | ||
|
|
||
| ##### IPFS-Webui & IPFS-Desktop | ||
|
|
||
| Analytics toggling and consent notifications are already implemented in the desktop app and webui. | ||
|
|
||
| Example: | ||
|
|
||
|  | ||
|
|
||
| ### Informing users of changes to this policy | ||
|
|
||
| We will not display notifications to users regarding telemetry changes, but users can follow release-notes as well as changes to this library, it's telemetry documentation (`./docs/telemetry/*`), and projects implementing this library to stay up to date on our telemetry policy changes. | ||
|
|
||
| ## SOPs (Standard Operating Procedures) | ||
|
|
||
| ### Adding telemetry collection to a project without telemetry collection | ||
|
|
||
| 1. Copy the [COLLECTED_DATA.template](./COLLECTED_DATA.template.md) file to your project at `./docs/telemetry/COLLECTED_DATA.md` | ||
| 2. Implement the library according to your project's needs | ||
| 3. Update your `COLLECTED_DATA.md` file specifying exactly which metrics your collecting, and when they're collected. | ||
|
|
||
| ### Changing what metrics data is collected by a project | ||
|
|
||
| When you add/remove metrics data that is collected by a project, you MUST follow this checklist: | ||
|
|
||
| 1. Fist ask yourself: Are you trying to collect user-identifiable, or other user data at all? STOP. | ||
| * You must get legal approval | ||
| * You must update the following documents: | ||
| * `./PRIVACY_POLICY.md` - responsibility of legal write a new doc after meeting with you and discussing what your plans are. You would then update the `PRIVACY_POLICY.md` in this repo. | ||
| * `./COLLECTION_POLICY.md` - responsibility of you to update according to the changes you're making. | ||
| * `./FAQs.md` - responsibility of you to update clarifying exactly why we're now collecting user data. | ||
| 1. Modify the project's `./docs/telemetry/COLLECTED_DATA.md` file. | ||
| * Ensure the table under `What metrics data DO we collect` is accurate and up to date | ||
|
|
||
|
|
||
| ### Changing a metric feature's grouping or opt-in/opt-out policy | ||
|
|
||
| When you move a metric feature to/from a particular feature group, or that group changes its consent method, you must update: | ||
|
|
||
| 1. The table above at `Metric features and their groupings` | ||
| 2. The code at https://github.com/ipfs-shipyard/ignite-metrics/blob/main/src/CountlyMetrics.ts#L10-L16 | ||
| 3. Possibly the code at https://github.com/ipfs-shipyard/ignite-metrics/blob/main/src/CountlyMetrics.ts#L27-L36 | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| # FAQs - Frequently asked questions | ||
|
|
||
| ## Why are you collecting metrics? | ||
|
|
||
| We are collecting metrics for all of our projects to assist with prioritizing work and provide proof that our work is valuable. Our prior state (<2023) had the following issues: | ||
|
|
||
| 1. Opt-in only metrics for webui and desktop did not send metrics data to our countly server | ||
| 1. We have no insight into the total number of users in our applications | ||
| 2. We had no insight into the percentage of users who declined or accepted metrics | ||
| 3. If a user did not accept or decline metrics, usage was not collected | ||
| 2. Lack of singular metrics collection library | ||
| 1. Consent and telemetry patterns were non-existent | ||
| 2. Privacy policy and metrics collection would have needed to be implemented for each library separately. | ||
| 3. Analytics across projects does not allow for normalized comparisons of usage. | ||
| 3. Missing telemetry | ||
| 1. We have no insight into the usage of a majority of our projects | ||
| 1. It is understood (not confirmed with data) that two of our top three projects are ipfs-webui & ipfs-desktop, and they had ****some**** telemetry in place. | ||
| 2. We have three separate usecases for ipfs-webui, and are only tracking two of them: | ||
| 1. [webui.ipfs.io](http://webui.ipfs.io) & local - tracked | ||
| 2. webui usage within ipfs-desktop - tracked as desktop usage | ||
| 3. webui bundled with kubo - not tracked | ||
|
|
||
| You can find more information about the discussions held when making these decisions at https://pl-strflt.notion.site/Telemetry-b005d4f217f44db3986902c67d922cf4 | ||
|
|
||
| ## What data is collected? | ||
|
|
||
| In order to standardize the documentation of metrics collection, we will keep a file in each project's repository that | ||
| specifies all metrics data collected by that project. The file will be located at `./docs/telemetry/COLLECTED_DATA.md`. These documents will provide answers to the following bullet-points: | ||
|
|
||
| * This is what will be collected | ||
| * This is what won't be collected | ||
| * Why we are collecting each piece of metric data. | ||
|
|
||
| You can read our [Privacy Policy](./PRIVACY_POLICY.md) and [Collection Policy](./COLLECTION_POLICY.md) for more generic details. | ||
|
|
||
| ## As an existing user, how am I notified about changes to the data that is collected? | ||
|
|
||
| You can follow updates to the metrics we collect for each repo in that repo's release notes. You can find changes to our collection policy in our [Collection Policy](./COLLECTION_POLICY.md) document. The document at `./docs/telemetry/COLLECTED_DATA.md` in each relevant repo will be updated when any metrics collection changes are made. | ||
|
|
||
| ## How do I opt-out of metrics? | ||
|
|
||
| For apps with settings pages, such as ipfs-webui, ipfs-desktop, and ipfs-companion, you can opt out in each of the applications' setting pages. For apps without settings pages (or single page applications where settings pages don't make sense), we will be following the [patterns set by the public-gateway-checker project](https://github.com/ipfs/public-gateway-checker/issues/340#issuecomment-1371410214). | ||
|
|
||
| The basic implementation is a floating icon on the bottom left of the page that displays a modal when clicked. The displayed modal allows the enabling/disabling of metrics collection for our different groupings. | ||
|
|
||
| For public-gateway-checker changes, see the following PRs and issues: | ||
|
|
||
| * https://github.com/ipfs/public-gateway-checker/pull/309 | ||
| * https://github.com/ipfs/public-gateway-checker/issues/340 | ||
| * https://github.com/ipfs/public-gateway-checker/issues/341 | ||
| * https://github.com/ipfs/public-gateway-checker/issues/342 | ||
|
|
||
| ## Can I turn telemetry completely off? | ||
|
|
||
| Yes. If you toggle all telemetry groups off (i.e. opt-out), no telemetry will be collected. The one caveat is that we send a single request when you opt-out of metrics collection. | ||
|
|
||
|
|
||
| ## I want to help development efforts, but want to be updated of any future changes. How do I stay up to date? | ||
|
|
||
| There are a few ways you can stay up to date depending on what you're interested in. | ||
|
|
||
| ### Global telemetry changes | ||
|
|
||
| Watch for changes to https://github.com/ipfs-shipyard/ignite-metrics via GitHub's watch feature. You can adjust notification preferences using GitHub's watch feature. Updates can be found in the following locations: | ||
|
|
||
| 1. PRs | ||
| 2. Release notes | ||
| 3. `./docs/telemetry/COLLECTED_DATA.template.md` | ||
|
|
||
|
|
||
| ### Project specific telemetry changes | ||
|
|
||
| Watch for changes to https://github.com/<repo-of-interest> via GitHub's watch feature. You can adjust notification preferences using GitHub's watch feature. Updates can be found in the following locations: | ||
|
|
||
| 1. PRs | ||
| 2. Release notes | ||
| 3. `./docs/telemetry/COLLECTED_DATA.md` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| # Privacy Policy | ||
|
|
||
| To be filled in by legal team based off of meeting documented at https://pl-strflt.notion.site/2023-01-05-Privacy-Policy-Updates-9bef84008b52449496ad5b806d529dfe | ||
|
|
||
| Until then, please reference the [Privacy Policy](https://ipfs.tech/companion-privacy/) for ipfs-companion (which is the only project we own with a current [privacy policy](https://github.com/ipfs/ipfs-companion/blob/main/PRIVACY-POLICY.md)). |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.