Eternaltwin

Home | Applications

Eternaltwin server configuration

This document describes the configuration of the Eternaltwin server itself: the eternaltwin.toml file read by the eternaltwin binary. The conventions every game or third-party application should follow for its own configuration are a separate document: Application configuration.

The authoritative definition is the eternaltwin_config crate, in crates/config/. The commented example at the root of the repository, eternaltwin.local.toml, is the practical starting point.

Loading

Profile

Everything starts with the profile. It selects the built-in defaults and the names of the files that are searched for.

  1. The --profile (-p) command line argument.
  2. Otherwise, the ETERNALTWIN_PROFILE environment variable, unless reading the environment has been disabled.
  3. Otherwise, dev.

Four profiles are built in: production, dev, test and sdk. Any other value is a custom profile: it picks its own file names but takes its default values from dev.

ETERNALTWIN_PROFILE is the only environment variable the configuration loader reads. There is no environment variable for any individual setting, and no ETERNALTWIN_CONFIG. Reading it can be turned off with --no-env-config (and forced back on with --env-config); the sdk profile turns it off by itself, unless the profile itself came from the environment or --env-config was passed explicitly.

Sources

Without command line arguments, the loader searches for a configuration file. It starts in the current working directory and tries these names, in order:

./eternaltwin.{profile}.local.toml
./eternaltwin.{profile}.local.json
./eternaltwin.{profile}.toml
./eternaltwin.{profile}.json
./eternaltwin.local.toml
./eternaltwin.local.json
./eternaltwin.toml
./eternaltwin.json

The first name that exists wins; the rest are ignored. If none matches, the search moves to the parent directory and repeats, up to the filesystem root. A search that finds nothing is not an error: the built-in defaults are used. The sdk profile searches for nothing at all — it is meant to be driven entirely from arguments.

Sources can also be given explicitly, and repeated:

ArgumentMeaning
--config <URL>Exact location. A file:// URL, or a relative path starting with ./ or ../.
--config-data <DATA>Inline configuration, as a string.
--config-search <PATTERN>A search, as above, with a single pattern.
--config-format <FORMAT>Format of the next source: auto (default), json or toml.

With auto, a file is parsed as TOML first and as JSON on failure; inline data is tried as JSON first. A --config-search pattern must start with ./ or ../. A --config location that does not exist is a fatal error, unlike a search that finds nothing.

Merging

The resolved configuration is a chain, merged onto the profile's built-in defaults:

  1. Start from the built-in defaults of the profile.
  2. Apply each source, from the lowest priority to the highest.

Later sources win: with several --config arguments, the last one has the final word.

A configuration file may pull in others through the top-level extends key, which takes one source or an array of them:

extends = "./eternaltwin.base.toml"

# or
extends = [
  "./eternaltwin.base.toml",
  { url = "./eternaltwin.secrets.toml", format = "toml" },
]

Relative references start with ./ or ../ and resolve against the file that declares them; anything else is parsed as an absolute URL. Only file: URLs can be read. A file always outranks what it extends, and a location that appears twice in the chain is read once.

Checking

eternaltwin config

loads the configuration exactly as the server would, prints where every source came from and the merged result, then exits. backend.secret is blanked out of that dump. Every subcommand that needs a configuration — backend, start, db, dump, job, tidsave — accepts the same arguments.

Conventions

  • Keys are lowercase with underscores, exactly as listed below.
  • Enumerated values are capitalised strings: "Memory", "Postgres", "System", "Mock", "Network", …
  • Durations are either a humantime string ("100ms", "5s", "2min") or a number of seconds.
  • Unknown keys are ignored silently. A typo, or a key that was removed in a later version, costs no error and no warning; the value simply has no effect. Use eternaltwin config to confirm that a setting landed where you think it did.
  • JSON can express null, TOML cannot. Setting a collection (seed, seed.user, opentelemetry.exporter, mailer.headers, …) to null clears it; setting one of its entries to null removes that entry. This is only reachable from a JSON source.

[backend]

The server process: what it listens on, and which implementation backs each piece of state.

KeyTypeDefaultDescription
listenstring"[::]:50320"Socket address the HTTP server binds to.
portintegerabsentOverrides the port of listen, leaving the interface alone.
channelstring"production" in production, "dev" elsewhereNames this deployment. The server identifies itself to applications as eternaltwin.{channel}.
secretstringnone in production, "dev" elsewhereMaster secret. Every token signature and every encrypted column is derived from it. Required: the server refuses to start without one.
clock"System", "Virtual"System in production and dev, Virtual in test and sdkVirtual starts at 2020-01-01T00:00:00 and only moves forward when the API is asked to move it.
mailer"Network", "Mock"Network in production, Mock elsewhereNetwork sends over SMTP, configured in [mailer]. Mock keeps messages in memory.
oauth_client"Network", "Mock"Network in production and dev, Mock in test and sdkImplementation used when Eternaltwin acts as an OAuth client of Discord and GitLab.
app_client"Network", "Mock"Network in production and dev, Mock in test and sdkImplementation used when Eternaltwin calls into a registered application.
store"Memory", "Postgres"Postgres in production and test, Memory in dev and sdkShorthand: sets every store below at once.

Memory keeps everything in RAM: the server starts without a database, and loses everything on restart. Postgres uses the database described in [postgres].

Stores

Each store can be set individually. A per-store key always wins over store, whatever their order in the file.

KeyValuesHolds
app_storeMemory, Postgres, SqliteApplications: games and third-party sites, their channels.
auth_storeMemory, PostgresSessions, cookies, tokens.
captcha_storeMemory, PostgresWhich captcha tokens have been spent, and the failed sign-in counters.
dinoparc_storeMemory, PostgresDinoparc archive.
forum_storeMemory, PostgresForum sections, threads, posts, roles.
hammerfest_storeMemory, PostgresHammerfest archive.
job_storeMemory, PostgresBackground job state.
link_storeMemory, PostgresLinks to archived game accounts: Dinoparc, Hammerfest, Twinoid.
mailer_storeMemory, Postgres, SqliteOutbound email.
oauth_provider_storeMemory, PostgresOAuth clients and their sessions, for Eternaltwin as a provider.
twinoid_storeMemory, Postgres, SqliteTwinoid archive.
user_link_storeMemory, PostgresLinks to external identity providers: Discord, GitLab.
user_storeMemory, PostgresUsers and profiles.

The three stores that accept Sqlite read [sqlite] for the database file. store cannot be set to Sqlite: only those three support it.

[frontend]

Where the site is reachable from the outside. This describes the Node frontend to the backend; it is not the frontend's own configuration, which is passed to packages/website on its command line and does not come from this file.

KeyTypeDefaultDescription
portinteger50321Sets uri to http://localhost:{port}/. An explicit uri in the same file wins.
uriURL"http://localhost:50321/"Public root of the site. The Discord and GitLab OAuth callbacks are derived from it: {uri}/oauth/callback/discord and {uri}/oauth/callback/gitlab.
forum_posts_per_pageinteger10Currently has no effect: forum pagination is fixed in the code at the same values.
forum_threads_per_pageinteger20Same.

Note that the backend defaults to port 50320 and the frontend to 50321. They are different servers; setting both to the same port cannot work.

[postgres]

Read by every store set to Postgres.

KeyTypeDefaultDescription
hoststring"localhost"
portinteger5432
namestring"eternaltwin.production" in production, "eternaltwin.dev" elsewhereDatabase name.
userstring"eternaltwin.production.main" in production, "eternaltwin.dev.main" elsewhereRole used at runtime. Also sets admin_user.
passwordstring"dev"Also sets admin_password.
admin_userstring"eternaltwin.production.admin" in production, "eternaltwin.dev.main" elsewhereRole used for schema migrations (eternaltwin db).
admin_passwordstring"dev"
max_connectionsinteger50Pool size.

admin_user and admin_password follow user and password unless set explicitly, and an explicit value wins whatever the order in the file.

[sqlite]

Read by the stores set to Sqlite.

KeyTypeDefaultDescription
filestring"./eternaltwin.{profile}.sqlite"Path to the database file.
max_connectionsinteger50Pool size.

[mailer]

Read when backend.mailer is Network.

KeyTypeDefaultDescription
hoststring"localhost"SMTP host.
usernamestring"eternaltwin_mailer"
passwordstring"dev"
senderstring"support@eternaltwin.localhost"From address.
headersarray[]Extra headers added to every message.

Each header is a table with name and value:

[[mailer.headers]]
name = "X-Deployment"
value = "eternaltwin.org"

[scrypt]

Cost of the password hasher. Both values are a ceiling the implementation tunes itself against at startup, so the right value depends on the machine.

KeyTypeDefaultDescription
max_timeduration"500ms" in production, "100ms" elsewhereTime budget for hashing one password.
max_mem_fracfloat0.1 in production, 0.05 elsewhereShare of the machine's memory one hash may use.

The server benchmarks itself against these limits on first use and caches the parameters it picks. max_time must be under a minute and max_mem_frac must be greater than 0 and at most 0.5; outside those bounds the server aborts.

[discord]

Credentials of the Discord OAuth2 application, from the Discord Developer Portal. Required when backend.oauth_client is Network; ignored under Mock.

KeyTypeDefaultDescription
client_idstringabsent
secretstringabsentServer-side secret. It must never reach the browser.

[gitlab]

Credentials of the GitLab OAuth2 application, from the instance's "Applications" settings. The application needs the read_user scope and the callback {frontend.uri}/oauth/callback/gitlab. Required when backend.oauth_client is Network; ignored under Mock.

KeyTypeDefaultDescription
client_idstringabsent
secretstringabsentServer-side secret.
base_uriURL"https://gitlab.com/"Root of the instance. GitLab is self-hostable, so the OAuth and REST endpoints are derived from this rather than hardcoded.

[captcha]

Settings of the proof-of-work captcha guarding registration and, after enough failed attempts, sign-in.

KeyTypeDefaultDescription
enabledbooleantrue in production, false elsewhereWhether a solved captcha is required at all.
challenge_countinteger50How many puzzles make up one challenge.
challenge_sizeinteger32Salt length of each puzzle, in hex characters. It does not affect the work required; it only has to be long enough that two puzzles never share a salt.
challenge_difficultyinteger4Length of the hex prefix a solution's hash must match. This is the cost dial and it is exponential.
challenge_validity_secondsinteger600How long a challenge may be solved for. It bounds how long a browser may take, so it is generous.
token_validity_secondsinteger300How long a solved captcha may be spent for. This is the window between checking the box and submitting the form, so it is much shorter.
login_attempts_before_captchainteger5Failed sign-ins from one address before the sign-in form starts asking for a captcha. Registration always asks.

Each extra character of challenge_difficulty multiplies the expected work by sixteen. At the default of 4, a challenge takes a couple of seconds in a browser. A value of 6 is not half again as hard as 4, it is 256 times as hard, and the people it locks out first are the ones on old phones. Raise it one step at a time.

enabled is off outside production because a captcha in the development profile stands between every test run and the form that test is exercising. Turning it off is a real hole: every protected endpoint then accepts a request carrying no token at all. It is a single explicit setting rather than something inferred from the profile or the store, so that turning it off is always a decision someone made.

The one piece of state the design cannot avoid — which tokens have already been spent — lives in backend.captcha_store, which follows backend.store like every other store. Memory is enough for a single-process deployment; several processes sharing a database need Postgres, or a token spent against one of them can be spent again against another.

Only enabled is published to the browser, through /api/v1/config. The difficulty and the validity windows are the server's business.

[opentelemetry]

Logs and traces.

KeyTypeDefaultDescription
enabledbooleantrueWhen false, logging and tracing are no-ops and no exporter is built.

[opentelemetry.attributes] is a free table of resource attributes attached to everything exported. Its defaults are:

[opentelemetry.attributes]
service.name = "eternaltwin"
deployment.environment = "dev"
service.version = "0.16.4"   # the running version

Setting an attribute adds or replaces it; the defaults are kept unless overridden, so deployment.environment stays "dev" in production too unless it is set.

[opentelemetry.exporter.<name>]

Any number of exporters, each under a name of your choice. Naming one that does not exist yet creates it with the defaults below.

KeyTypeDefaultDescription
typestring"Human"Human (human-readable lines on stdout), Jsonl (JSON Lines on stdout), Grpc (OTLP over gRPC), HttpJson (OTLP over HTTP). Any other value stops the server at startup.
endpointURL"http://localhost:4317/"Collector endpoint. Used by Grpc and HttpJson only.
timeoutduration"5s"Export timeout. Used by Grpc and HttpJson only.
metadatatable of strings{}gRPC metadata, or HTTP headers, sent to the collector.
colorstring"Auto"Accepted and currently unused.
targetstring"eternaltwin://stdout"Accepted and currently unused: Human and Jsonl always write to the process stdout.

Defaults per profile: production and test configure no exporter at all; dev configures one named stdout of type Human; sdk configures one named stdout, whose declared type is not among the four accepted values, so an sdk deployment that leaves opentelemetry.enabled on has to name a type itself.

Eternaltwin also receives OTLP: /v1/logs and /v1/traces on the backend's own HTTP port. There is no separate gRPC listener, and no setting for one. grpc_proxy_port, which still appears in eternaltwin.local.toml, is not a configuration key and is ignored.

[seed]

Data inserted or updated on every start. Seeding is an upsert, so it is safe to restart, and a seeded row that was edited in the database is put back the way the file describes it.

An entry set to null is removed from the seed, and seed = null clears all three collections. As with every null here, it needs a JSON source.

[seed.user.<key>]

KeyTypeDefaultDescription
idUUIDabsentForces the user id. Without it, the user is matched by username.
display_namestringthe keyThe key has to be a valid display name when this is omitted.
usernamestringabsentIf given, it must be equal to the key. With no id, the key is the username the user is matched on.
passwordstringabsent
is_administratorbooleanfalse

Users are seeded in key order. The dev and sdk profiles seed alice (an administrator), bob, charlie, dan, eve and frank, each with a ten-character password made of its initial. Production and test seed nobody.

[seed.app.<key>]

Games and third-party applications, registered as OAuth clients.

KeyTypeDefaultDescription
idUUIDabsentForces the OAuth client id.
display_namestringthe key
uriURLhttp://{key}.localhost/Homepage of the application.
oauth_callbackURLhttp://{key}.localhost/oauth/callbackOAuth callback endpoint.
event_stream_uriURLabsentWebhook the application receives events on.
secretstring"dev"Shared secret, used as the OAuth client secret.
allowed_scopesstringabsentThe scopes the application may request, space-separated, in the format of the OAuth scope parameter.

The key becomes the OAuth client key {key}@clients, and is also split on its first underscore into an application key and a channel: eternalfest_dev is the dev channel of eternalfest. A key with no underscore gets the channel production.

allowed_scopes takes base, forum:read, forum:write and forum:moderate; asking for one implies the ones below it. Omitting the key leaves the client's current allowance alone rather than resetting it to base, so a client widened by hand does not lose that at the next restart. An application that was never given one may only request base. See The forum from an application for what each forum scope buys.

The dev and sdk profiles seed brute_dev, emush_dev, eternalfest_dev, kadokadeo_dev, kingdom_dev, myhordes_dev and neoparc_dev. Production and test seed nothing.

[seed.forum_section.<key>]

KeyTypeDefaultDescription
display_namestringthe key
localelocale idabsent"fr-FR", "en-US", "es-SP", "de-DE", "eo", … Absent means the section is not tied to a language.
parentsection keyabsentKey of the section this one sits under. Absent means a root section.
orderinteger0Rank among its siblings, ascending.

Sections form a hierarchy. Seeding runs in two passes — every section without a parent first, then every section with one — so a parent always exists before its children are attached. That is one level of nesting: a section whose parent itself has a parent is not guaranteed to find it.

order sorts siblings, not the whole forum, so the numbering restarts under each parent. Leaving gaps (0, 10, 20, …) makes it possible to insert a section later without renumbering the rest.

[seed.forum_section.general]
display_name = "Général"
order = 0

[seed.forum_section.main_en]
display_name = "Main Forum (en-US)"
locale = "en-US"
parent = "general"
order = 0

[seed.forum_section.main_fr]
display_name = "Forum Général (fr-FR)"
locale = "fr-FR"
parent = "general"
order = 10

The dev and sdk profiles seed a per-game hierarchy: a root section for each game, plus general holding one section per language. Production and test seed nothing.

Design

Should default config search look into subdirectories? (e.g. .config/eternaltwin.toml)

There was a Node issue about the proliferation of config files at the repo root, with the proposition to use .config/ as an alternative. The issue with using a subdirectory is that scoping is no longer obvious. Because of this, Eternaltwin only does regular lookups.

Should config search support globs?

Globs make ordering more ambiguous, especially if allowing to traverse directories. Since Eternaltwin uses ordering for config merges, globs are not supported.

Implicit or explicit extend

Search stops at the first result instead of searching all matches. Combining configs requires an explicit extends. This is similar to TypeScript or Eslint.

Argfile support

If a CLI arg is prefixed with @, javac treats it as an argfile and expands it in place. See Javac documentation. We don't support it, but it's a neat idea.