This is the codebase that runs the Devographics surveys, such as State of JS and State of CSS.
This monorepo uses pnpm, and you should start by installing it.
You can clone all repositories in the right place and create blank .env
files with the following install script:
curl -o- https://raw.githubusercontent.com/Devographics/Monorepo/main/install.sh | bash
- Clone this monorepo locally with
git clone https://github.com/Devographics/Monorepo.git
The survey apps rely on a lot of metadata. If you need to load or modify this metadata from your local filesystem instead of through our API, you can optionally also clone these other repos:
- entities: the YAML files containing all metadata for the features, sites, people, libraries, etc. mentioned in the surveys.
- surveys: contains the YAML files that define survey configs and outlines.
- locale-en-US: (or any other locale) contains the locale strings used in the survey.
I suggest using the following file structure:
devographics
parent directorydevographics/monorepo
devographics/monorepo/surveyform
devographics/monorepo/results
devographics/monorepo/...
devographics/entities
devographics/surveys
devographics/locales
devographics/locales/locale-en-US
devographics/locales/locale-fr-FR
devographics/locales/...
Whenever you need to run a project in the monorepo
directory (such as results
, surveyform
, etc.), you will need to first run pnpm install
from within its subdirectory.
cd monorepo
pnpm install
This will install dependencies for all applications of the monorepository.
You can run each project with pnpm run dev
when inside its subdirectory.
For example, for the surveyform
app:
cd monorepo
cd surveyform
pnpm run dev
If this is your first time running a project, you will run into some issues that need to be fixed by configuring your local environment variables (see below).
You can run the results
and surveyform
project by connecting to our remote production API (https://api.devographics.com/graphql).
Alternatively, if you want to use local files for the surveys, locales, etc. you will need to run a local copy of the API project (monorepo/api
) locally as well to load them. Once the API is running, you can then point the other apps to it via the API_URL
env variable.
Note that even when the API is running locally, you will still need internet access to connect to the databases and load image assets.
Each app within monorepo
needs its own environment variables defined inside a .env
files (except for surveyform
and surveyadmin
, which use a .env.development.local
file).
Here are some ways that can help with this setup:
- You can use the
.env.example
file in each project subdirectory (such as this one for the surveyform project) as a starting point by pasting its contents into your own.env
or.env.development.local
(for Next.js projects such assurveyform
) file. - You will need credentials to connect to our MongoDB and Redis staging database. You can ask me (Sacha) on Discord and I will provide them.
- When running the app with
pnpm run dev
you will get error messages indicating any remaining missing environment variables.
You can also refer to variables.yml directly and look for variables corresponding to the current app (e.g. results
).
The following apps are all contained within the monorepo.
Node.js TypeScript app.
- Make the outlines of each survey available to the survey form app.
- Connect to the database and generate the data for the results app's charts.
- Provide the internationalisation strings for each locale.
GraphQL IDE
- Make it easier to test and query the API.
Next.js TypeScript app.
- Let respondents take the survey.
Gatsby TypeScript app.
- Display the survey results.
Admin app.
- Provide a dashboard to manage all surveys.
- Handle data processing and normalization.
- Only running locally for now.
MongoDB database.
- Store the raw data entered by respondents.
- Store the "normalized" data once it's been processed.
You can plug to a staging database or run Mongo locally via Docker.
See "docker-compose.yml" and "Justfile" for our local setup. When running locally, data are stored in a ".mongo" folder in the monorepo folder. You can delete this folder to reset the local database.
Redis database.
- Cache the results of queries made to the API app.
You can plug to a staging database or run Redis locally.
Since we use Upstash, which rely on HTTP requests instead of direct Redis connection, we also setup an HTTP proxy.
See "docker-compose.yml" and "Justfile" for our local setup.
To reset the local Redis instance, the best approach is to remove the Redis container.
Locale strings
- Store locale strings for various languages as YAML files.
- https://github.com/Devographics/locale-en-US
- https://github.com/Devographics/locale-es-ES
- https://github.com/Devographics/locale-de-DE
- etc.
Outline and config files for each survey.
- Store outline and config files for each survey as YAML files.
Static image files.
- Store static images such as logos, social media preview images, etc.
Emojis to distinguish commits within the monorepo:
π ΏοΈ :parking:
for the whole monorepo ("P" for "Pnpm")- β
:gear:
for the shared folder - π
:mag:
for graphiql - π‘
:satellite:
for the api - π
:bar_chart:
for the results - π
:house:
for the homepage - β οΈ
:writing_hand:
for the surveyform - π
:lock:
for the surveyadmin - π
:globe_with_meridians:
for the locales - π
:book:
for the surveys - π
:person_pouting:
for the entities - πΌοΈ
:frame_photo:
for the images
See "shared/helpers/variables.yml" for a more up to date list.
Variable | Description | Used By |
---|---|---|
API_URL |
URL of the API | All |
Variable | Description | Used By |
---|---|---|
MONGO_URI |
URI of the Mongo database | All |
MONGO_PRIVATE_DB |
Name of the database where private data is stored | Surveyform |
MONGO_PUBLIC_DB |
Name of the database where public data is stored | API |
Variable | Description | Used By |
---|---|---|
REDIS_URL |
URL of the Redis database | Surveyform |
REDIS_TOKEN |
Redis token (needed for Upstash) | Surveyform |
Variable | Description | Used By |
---|---|---|
GITHUB_TOKEN |
GitHub access token | Results |
GITHUB_PATH_SURVEYS |
Path to surveys dir on GitHub (e.g. org/repo/(subdir) ) |
Results |
GITHUB_PATH_LOCALES |
Path to locales dir on GitHub (e.g. org/(repo)/(subdir) ) |
Results, API |
- For
GITHUB_PATH_SURVEYS
, thesubdir
segment can be omitted. It will then be assumed that the surveys data is at the root of the repo. - For
GITHUB_PATH_LOCALES
, both therepo
and thesubdir
segments can be omitted. It will then be assumed that the locales each have their own separate repo.
Variable | Description | Used By |
---|---|---|
EMAIL_OCTOPUS_APIKEY |
EmailOctopus API key | Surveyform |
DEFAULT_MAIL_FROM |
Default "from" email (e.g. [email protected] ) |
Surveyform |
SMTP_HOST |
SMTP host (e.g. email-smtp.us-east-1.amazonaws.com ) |
Surveyform |
SMTP_PORT |
SMTP port (e.g. 465 ) |
Surveyform |
SMTP_SECURE |
Set to "1" | Surveyform |
SMTP_USER |
SMTP username | Surveyform |
SMTP_PASS |
SMTP password | Surveyform |
EMAIL_OCTOPUS_APIKEY
Variable | Description | Used By |
---|---|---|
ENCRYPTION_KEY |
Encryption key to hash emails | Surveyform |
SECRET_KEY |
Secret key used to verify external webhook requests | API |
ASSETS_URL |
URL for static assets (e.g. https://assets.devographics.com/ ) |
All |
Variable | Description | Used By |
---|---|---|
SURVEYS_DIR |
Local directory from which to load survey outlines | API |
LOCALES_DIR |
Local directory from which to load locale files | API |
ENTITIES_DIR |
Local directory from which to load entities files | API |
ENABLE_CACHE |
Set to false to always load data from the API |
All |
PORT |
Which port to run the app on | All |
LOGS_DIRECTORY |
Absolute path to logs dir (e.g. /Users/sacha/monorepo/surveyform/.logs ) |
All |
- If
SURVEYS_DIR
is defined, surveys data will be loaded locally instead of from GitHub. - If
LOCALES_DIR
is defined, locales data will be loaded locally instead of from GitHub. - If
ENTITIES_DIR
is defined, entities data will be loaded locally instead of from GitHub.
- Plausible for analytics