This is the codebase that runs the Devographics surveys, such as State of JS and State of CSS.

This monorepo uses pnpm, and you should start by installing it.

You can clone all repositories in the right place and create blank .env files with the following install script:

curl -o- https://raw.githubusercontent.com/Devographics/Monorepo/main/install.sh | bash

• Clone this monorepo locally with git clone https://github.com/Devographics/Monorepo.git

The survey apps rely on a lot of metadata. If you need to load or modify this metadata from your local filesystem instead of through our API, you can optionally also clone these other repos:

1. entities: the YAML files containing all metadata for the features, sites, people, libraries, etc. mentioned in the surveys.
2. surveys: contains the YAML files that define survey configs and outlines.
3. locale-en-US: (or any other locale) contains the locale strings used in the survey.

I suggest using the following file structure:

• devographics parent directory
  • devographics/monorepo
    • devographics/monorepo/surveyform
    • devographics/monorepo/results
    • devographics/monorepo/...
  • devographics/entities
  • devographics/surveys
  • devographics/locales
    • devographics/locales/locale-en-US
    • devographics/locales/locale-fr-FR
    • devographics/locales/...

Whenever you need to run a project in the monorepo directory (such as results, surveyform, etc.), you will need to first run pnpm install from within its subdirectory.

• cd monorepo
• pnpm install

This will install dependencies for all applications of the monorepository.

You can run each project with pnpm run dev when inside its subdirectory.

For example, for the surveyform app:

• cd monorepo
• cd surveyform
• pnpm run dev

If this is your first time running a project, you will run into some issues that need to be fixed by configuring your local environment variables (see below).

You can run the results and surveyform project by connecting to our remote production API (https://api.devographics.com/graphql).

Alternatively, if you want to use local files for the surveys, locales, etc. you will need to run a local copy of the API project (monorepo/api) locally as well to load them. Once the API is running, you can then point the other apps to it via the API_URL env variable.

Note that even when the API is running locally, you will still need internet access to connect to the databases and load image assets.

Each app within monorepo needs its own environment variables defined inside a .env files (except for surveyform and surveyadmin, which use a .env.development.local file).

Here are some ways that can help with this setup:

1. You can use the .env.example file in each project subdirectory (such as this one for the surveyform project) as a starting point by pasting its contents into your own .env or .env.development.local (for Next.js projects such as surveyform) file.
2. You will need credentials to connect to our MongoDB and Redis staging database. You can ask me (Sacha) on Discord and I will provide them.
3. When running the app with pnpm run dev you will get error messages indicating any remaining missing environment variables.

You can also refer to variables.yml directly and look for variables corresponding to the current app (e.g. results).

The following apps are all contained within the monorepo.

Node.js TypeScript app.

• Make the outlines of each survey available to the survey form app.
• Connect to the database and generate the data for the results app's charts.
• Provide the internationalisation strings for each locale.

• https://github.com/Devographics/Monorepo/tree/main/api

• https://render.com

GraphQL IDE

• Make it easier to test and query the API.

• https://github.com/Devographics/Monorepo/tree/main/graphiql

• https://netlify.com

• https://api.devographics.com

Next.js TypeScript app.

• Let respondents take the survey.

• https://github.com/Devographics/Monorepo/tree/main/surveyform

• https://vercel.com

• https://survey.devographics.com

Gatsby TypeScript app.

• Display the survey results.

• https://github.com/Devographics/Monorepo/tree/main/results

• https://netlify.com

• https://2022.stateofjs.com
• https://2023.stateofcss.com
• etc.

Admin app.

• Provide a dashboard to manage all surveys.
• Handle data processing and normalization.

• Only running locally for now.

MongoDB database.

• Store the raw data entered by respondents.
• Store the "normalized" data once it's been processed.

• https://www.mongodb.com/atlas/database

You can plug to a staging database or run Mongo locally via Docker.

See "docker-compose.yml" and "Justfile" for our local setup. When running locally, data are stored in a ".mongo" folder in the monorepo folder. You can delete this folder to reset the local database.

Redis database.

• Cache the results of queries made to the API app.

• https://upstash.com/

You can plug to a staging database or run Redis locally.

Since we use Upstash, which rely on HTTP requests instead of direct Redis connection, we also setup an HTTP proxy.

See "docker-compose.yml" and "Justfile" for our local setup.

To reset the local Redis instance, the best approach is to remove the Redis container.

Locale strings

• Store locale strings for various languages as YAML files.

• https://github.com/Devographics/locale-en-US
• https://github.com/Devographics/locale-es-ES
• https://github.com/Devographics/locale-de-DE
• etc.

Outline and config files for each survey.

• Store outline and config files for each survey as YAML files.

• https://github.com/Devographics/surveys/

• https://assets.devographics.com

Static image files.

• Store static images such as logos, social media preview images, etc.

• https://github.com/Devographics/images

• https://netlify.com/

• https://assets.devographics.com

Emojis to distinguish commits within the monorepo:

• 🅿️ :parking: for the whole monorepo ("P" for "Pnpm")
• ⚙ :gear: for the shared folder
• 🔍 :mag: for graphiql
• 📡 :satellite:for the api
• 📊 :bar_chart:for the results
• 🏠 :house:for the homepage
• ✍ ️:writing_hand: for the surveyform
• 🔒 :lock: for the surveyadmin
• 🌐 :globe_with_meridians: for the locales
• 📖 :book: for the surveys
• 🙎 :person_pouting: for the entities
• 🖼️ :frame_photo: for the images

See "shared/helpers/variables.yml" for a more up to date list.

• For GITHUB_PATH_SURVEYS, the subdir segment can be omitted. It will then be assumed that the surveys data is at the root of the repo.
• For GITHUB_PATH_LOCALES, both the repo and the subdir segments can be omitted. It will then be assumed that the locales each have their own separate repo.

EMAIL_OCTOPUS_APIKEY

• If SURVEYS_DIR is defined, surveys data will be loaded locally instead of from GitHub.
• If LOCALES_DIR is defined, locales data will be loaded locally instead of from GitHub.
• If ENTITIES_DIR is defined, entities data will be loaded locally instead of from GitHub.

• Plausible for analytics

A glossary of terms used internally throughout the code and apps.

A single answer to a survey question. Note that some questions accept multiple answers.

The Node.js app that serves the GraphQL API which in turn gives all other apps (as well as any external client) access to the data.

A special auto-generated API field that returns how many respondents use 1 item in the current section, 2 items, 3 items, etc.

In the context of processing (or “coding”) freeform data, is synonymous with Token.

A dataset containing both freeform and fixed responses.

Optional comments left by respondents in relation with a specific question.

A config.yml file used to configure a given survey or survey edition.

For freeform questions that do not have a pre-established list of options, a list of options can be generated based on the dataset dynamically. This is useful for filtering.

A specific edition (i.e. year) of a survey.

The repo containing all entities definitions.

Any item, person, concept, etc. that can potentially be referenced in the surveys. Defined in the entities repo.

When used in opposition to Token, indicates a “real” item such as a feature, library, person, tc. with a homepage, social media links, or other associated metadata.

The GraphiQL Explorer can be used to navigate the GraphQL API.

Data resulting from a freeform text area and not a fixed list of questions.

The IDE that lets you query the API.

The protocol used to define and query the API.

The app used to generate an individual survey's homepage (such as https://stateofjs.com).

A special auto-generated API field that returns all items belonging to the current section.

A collection of repos (such as locale-en-US) containing localized content for the surveys.

A special auto-generated API field that returns metadata about the current item.

Data that has gone through the normalization process, which includes:

• Removing empty responses.
• Assigning entities/tokens to freeform answers.
• Copying/renaming fields.
• etc.

The MongoDB collection (a.k.a. Database Table) that stores normalized data, as created by the Surveyadmin app.

This collection or its contents can be made public, either through the GraphQL API or by sharing CSV or JSON files containing a survey edition's entire dataset.

The list of fixed options available to choose from to answer a given question.

See also: Dynamic Options.

Regex-defined string matching patterns (in a question's patterns field) used to match freeform data to entities.

Rules of patterns:

• An entity's id is automatically used to match strings.
• Any _ in the id can also match - or spaces.
• Matching is case-insensitive.
• Patterns automatically match their plural form (animation will also match animations).
• Patterns can also use the following modifiers:
  • [p] (partial): Match partial word fragments.
  • [l] (list): Comma-separated list of items to match in any order.
  • [e] (entire): Match entire answer exactly.
  • [w] (whole): Match whole words (default).

Data resulting from a fixed list of options (e.g. multiple choice questions, dropdown questions, etc. ).

Data resulting from a practically unlimited, yet still predefined list of options (such as a list of all Best of JS projects, or all GitHub repos).

A client-side query-building UI provided in the Results app that lets end users dynamically build and run GraphQL queries to generate custom charts.

A question asked in a survey.

A response to a specific question by a specific respondent. Note that some responses can contain more than one answer.

A YAML file definining the shape of a survey's questions. Stored in the surveys repo.

The raw list of answers for any given question.

The entire response document containing all of a single respondent's data for a given survey.

The app used to compile a dataset into a static, public-facing results site.

A YAML file defining the shape of a survey's results site. Currently stored in the monorepo repo but should eventually migrate to the surveys repo.

A survey section.

A sub-directory of the monorepo containing code shared by other sub-apps (and imported as local packages via PNPM).

A specific kind of survey (State of JS, State of CSS, etc.).

The app used internally to normalize surveys datasets and run scripts.

The survey form app used to collect survey responses.

The repo containing all survey definitions and associated data.

An optional tag that can be assigned to an entity to categorize it. This can then be referenced by a question to select a subset of entities.

Rules of tags:

• Entities defined in foo.yml automatically get assigned the tag foo.
• Entities also automatically get assigned the names of all parent directories as tags.
• Other tags can be explitly defined in an entity's tags field.
• Questions can reference tags in their matchTags field.
• Questions automatically reference any tag corresponding to their own id field.

In the context of question definitions (either in the questions outline or results outline), a set of predefined parameters that a question can extend to keep code DRYer.

A special sub-type of entities that correspond to concepts (such as “hard learning curve”) and not “real” items.

Data that has not gone through the normalization process yet.

The MongoDB collection (a.k.a. Database Table) that stores unnormalized data, as provided by the Surveyform app.

This collection or its contents should never be made public.