Skip to content

Commit

Permalink
Add docs on creating service (#822)
Browse files Browse the repository at this point in the history
* Add docs on creating service

* Add more docs about ooni services setup

* Improvements to docs
  • Loading branch information
hellais authored Mar 14, 2024
1 parent e506a9f commit 809d89f
Show file tree
Hide file tree
Showing 5 changed files with 475 additions and 68 deletions.
14 changes: 0 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,3 @@
# OONI backend

Welcome to the OONI backend!

We follow the principles of a building a [12 factor app](https://12factor.net/),
which can be summarized as:

- Use declarative formats for setup automation, to minimize time and cost for
new developers joining the project;
- Have a clean contract with the underlying operating system, offering maximum
portability between execution environments;
- Are suitable for deployment on modern cloud platforms, obviating the need for
servers and systems administration;
- Minimize divergence between development and production, enabling continuous
deployment for maximum agility;
- And can scale up without significant changes to tooling, architecture, or
development practices.
53 changes: 0 additions & 53 deletions ooniapi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,56 +4,3 @@ Welcome to the new OONI API!

This is still work in progress, but the end goal is to eventually reach feature
parity with backend/api/ (aka legacy flask based API).

The general idea is to break OONI API components into smaller pieces that can
be more easily deployed and managed without worrying too much about the blast
radius caused by the deployment of a larger component.

To this end, we divide the OONI API sub-components into what we call
"services".

Each service (found in the `ooniapi/services` folder) should have a narrowly
defined scope. It's optimal to slice services up depending on how critical they
are (tier0, tier1, tier2) and what kinds of resources they need to access
(databases, object store, specific hosts, etc.).

There is also a `common` folder for all shared logic common to every component.

## List of OONI API Services

### Tier0

- `ooniprobe`, (aka `probe_services`), where probes send their measurements and
get metadata to run experiments;
- `oonirun`, CRUD for editing OONI Run links, but also for probes to get the
descriptors and run tests (this fact is what makes it tier0);
- `prioritization`, CRUD for editing the priorities of URLs and populating the
appropriate clickhouse tables so they can be used by probe;
- `fastpath`, responsible for taking measurements blobs sent to `ooniprobe`
service and storing them in s3;

### Tier1

- `data`, (aka OONI Data Kraken), where Explorer and other clients access the
observations data and experiment results;
- `findings`, (aka `incidents`) backend for findings pages on explorer;
- `measurements`, backend for aggregation and list measurement endpoints (note
also probe uses this, so it's maybe on the high end of tier1);
- ``

### Tier2

- `testlists`, for proposing changes to the test-lists and submitting a github PR;

## Developer setup

We use `hatch` for managing development environments. Once you have [installed
hatch](https://hatch.pypa.io/1.9/install/), check the `Makefile` for service
specific commands.

Generally it should work something like:

```
make test
make run
```
Loading

0 comments on commit 809d89f

Please sign in to comment.