Provision backend internals

Provision is the Drush-8 command package that does every server-side action Ægir takes — write vhosts, create databases, run installs, take backups, deploy tarballs.

It is a Drush commandfile set, not a Drupal module at runtime: provision.info declares module identity (version=7.x-3.x, read back by provision_version(), provision.inc:1371) but nothing bootstraps it inside Drupal.

On a BOA box it lives at /var/aegir/.drush/sys/provision on the Master and /data/disk/<oN>/.drush/sys/provision per Octopus satellite — cloned from omega8cc/provision.git at the branch selected by ${_BRANCH_PRN} (boa-private/lib/functions/master.sh.inc:793, satellite.sh.inc:3242-3244). The historical bare .drush/provision path is actively removed by the same code paths; never reference it.

Everything here runs as the unprivileged instance user (aegir or oN) under the omega8cc Drush 8 fork (Drush fork internals), and must stay parse- and runtime-safe on PHP 5.6 — the CLI floor on legacy servers. No ??, no scalar types, no arrow functions, no 7.x-only stdlib.

Entry point: the task round-trip

The Hostmaster frontend reaches provision through exactly one door. drush_hosting_task() (hosting/task.hosting.inc:189) re-exports the node's state as a backend context via provision-save (task.hosting.inc:208), defaults the verb to 'provision-' . $task->task_type (task.hosting.inc:223), and dispatches it with provision_backend_invoke() (task.hosting.inc:226).

That helper is provision's own (provision.inc:1274) — a drush_invoke_process() wrapper with integrate + dispatch-using-alias, also used backend-side whenever one provision command needs another (e.g. deploy unlocking the old platform).

Results and log lines ride the Drush backend channel back to the task node. The frontend half of this contract is on the Hosting API leaf.

Two gates run before any provision command, from provision_drush_init() (provision.drush.inc:51):

• _provision_drush_check_user() (:67) — refuses to run any provision-* command as root (PROVISION_IS_ROOT). Root work (BOA's own scripts, autosymlink) is always delegated, never done in-process.
• _provision_drush_check_load() (:80) — exit(1) when provision_load_critical() (:489) trips: load₁ above ncpus × critical_load_multiplier (default 5, :466) or a flat critical_load_threshold of 10 when the CPU count is unknown. provision_count_cpus() (:445) prefers BOA's daily-generated /data/all/cpuinfo over /proc/cpuinfo.

provision_drush_init() also seeds the context registry: it loads the invocation alias into d() as the root object, honouring the #name option so provision-save can address a context whose alias file does not exist yet (provision.drush.inc:55-56).

Repo layout: four commandfiles, per-verb orchestrators

Drush discovers four commandfiles inside the clone, and that split is the architecture:

Per-command logic lives in orchestrator files loaded by Drush's command-include rule (drush/includes/command.inc:1715,1723): for command a-b-c the parts are reversed and dot-joined, and any c.b.a.inc sitting next to a registered commandfile is include_onced at dispatch. So:

• provision-verify pulls in platform/verify.provision.inc
• provision-login-reset pulls in platform/reset.login.provision.inc
• provision-backup-delete pulls in platform/delete.backup.provision.inc
• hostmaster-install pulls in install.hostmaster.inc at the repo root

The same verb name can exist in several subtrees and all get included — deploy.provision.inc exists in platform/, db/ and http/, each implementing its own slice as hooks (next section).

Counts at HEAD: platform/ carries 17 *.provision.inc orchestrators plus provision_drupal.drush.inc and the platform/drupal/ engine set (38 *.inc engines + README.md); db/ and http/ carry 7 orchestrators each.

The Drush command registry

provision_drush_command() (provision.drush.inc:91) is the single registry. The file-head docblock (provision.drush.inc:2-39) is the canonical command-and-hooks enumeration — there is no api.drush.inc in this fork; that filename is an upstream-doc fiction.

Details that matter when adding or touching a command:

• provision-save merges its option table from the three context classes' static option_documentation() (provision.drush.inc:100-102) and sets allow-additional-options — unknown options are accepted and persisted, which is how extension modules smuggle their per-context properties in without touching the registry.
• Bootstrap levels are load-bearing. provision-verify and provision-delete run at DRUSH_BOOTSTRAP_DRUSH because the target may not be a bootstrappable site; the site-level bootstrap is then performed manually inside the validate hook (see verify below). provision-dlock/provision-dunlock were explicitly lowered to DRUSH with an in-callback root bootstrap, because dispatching at DRUPAL_ROOT on an unlocked D10+ platform fatals (modern typed symfony/console vs Drush 8's untyped adapter).
• Most provision-* verbs have no owner callback at all. The work happens entirely in hooks contributed by the other commandfiles — that is the pluggability model.
• provision-stats from the upstream docblock is an optional external module; it is not registered here. Do not document it as a shipped verb.

Hook choreography per command

For command provision-X, Drush 8 calls, per commandfile CF, in order: drush_CF_provision_X_validate() → drush_CF_pre_provision_X() → drush_CF_provision_X() → drush_CF_post_provision_X(), with matching *_rollback() functions unwound in reverse on failure.

Every orchestrator file in this repo is just a bag of these implementations — drush_provision_drupal_pre_provision_verify() parses as commandfile provision_drupal, phase pre, command provision-verify.

Third-party provision extensions hook the same chain the same way; that surface is enumerated in provision.api.php and worked through on the Provision API leaf.

verify

• drush_provision_drupal_provision_verify_validate() (platform/verify.provision.inc:13) — for site contexts, performs the bootstrap the DRUSH-level dispatch skipped: DRUSH_BOOTSTRAP_DRUPAL_SITE, write settings.php (_provision_drupal_create_settings_file(), provision_drupal.drush.inc:277), push site files (provision_drupal_push_site(), :204), then FULL bootstrap below D12 and DRUPAL_CONFIGURATION for D12+.
• drush_provision_drupal_pre_provision_verify() (:36) — server contexts: create config/, backups/, clients/ under aegir_root, seed config/includes/global.inc via Provision_Config_Global_Settings; platform and site branches follow in the same function.
• Owner callback drush_provision_verify() (provision.drush.inc:385) re-runs provision-save for the context, then d()->command_invoke('verify') — the context/service fan-out (verify_{type}_cmd on every subscribed service; the Nginx driver's server pass is http/Provision/Service/http/nginx.php:102). This is where vhosts and server configs are actually re-rendered.
• drush_provision_drupal_post_provision_verify() (:485) caches the verified state back into the site/platform drushrc.

install

All in platform/install.provision.inc: validate :17, pre :66 (rollback :74), implementation :102, post :442.

• The implementation branches on Drupal major. D6–D10 installs run in-process in the backend thread (provision-install-backend); D11+ installs run site-install through the platform's site-local Drush (vendor/drush/drush/drush.php under the app root found by findDrupalAppRoot()), because Ægir's Drush 8 cannot drive a modern installer. The Ægir core/console patches are reverted for the installer and re-applied after — gated by the /data/conf/_no_codebase_lock_unlock.ctrl kill-switch — and a failed or degraded install re-locks before returning so a shared platform vendor is never left in the state the patched web cannot serve. The child runs as exec '<the backend's own PHP>' -c /opt/phpNN/lib/php.ini -d log_errors=1 -d error_log=<~/log/php-site-install-<uri>.log> -d auto_prepend_file=<shutdown hook> [-d memory_limit=<floor>M] vendor/drush/drush/drush.php site-install ... (the exec form of _provision_site_local_drush_cmd(), which websh and a plain dash /bin/sh run the same way, on the interpreter's own php.ini because a modern Drush refuses any non-empty open_basedir and the account's Drush 8 ini carries one; the later user:login, pm:list and pm:uninstall calls of the install take the same form): the memory floor is a quarter of installed RAM capped at 1024 MB (or the install-memory.info pin), logged as D11/INSTALL/MEM; the hook records a fatal to that log and to stderr regardless of runtime ini changes. A non-zero exit logs D11/INSTALL/RC (status + reading) and D11/INSTALL/PHPLOG (the log tail), registers PROVISION_DRUPAL_INSTALL_FAILED and returns FALSE, so the frontend marks the task failed and rollback runs.
• The post hook rebuilds caches (_provision_drupal_rebuild_d8plus(), provision_drupal.drush.inc, on D10+ by core's own drupal_rebuild() in a Drush-free subprocess, platform/drupal/rebuild_d8plus.php, the last container write after the re-lock; _provision_drupal_rebuild_caches() below), saves the packages map (provision_drupal_system_map(), :961), creates the client symlink, and includes the cron_key engine.
• BOA-specific tail: _provision_drupal_native_symlink() (provision_drupal.drush.inc:503), called at the end of the post hook, moves the new site's files/ and private/ into the per-account static store and symlinks them back — by delegating to the root autosymlink tool through a hardened wrapper, since the store is root-managed and provision is unprivileged. Scoped to /data/disk/ accounts only, warn-not-fail, with box-wide (/data/conf/disable_native_files_symlink.cnf) and per-account (static/control/no_native_files_symlink.info) kill-switches, and fail-closed token validation before anything reaches the root tool. The clone (platform/clone.provision.inc) and migrate (platform/migrate.provision.inc) paths call the same helper with force_unshare to re-home inherited stores after cross-name copies.

deploy — the engine under clone / migrate / restore

platform/deploy.provision.inc: validate :24, pre :53 (unpack the backup; rollback :149), implementation :174, post :182. Clone, migrate and restore all end up here — deploy is "take a site package, land it at this URI on this platform, run the upgrade path".

The post hook is the dangerous part on modern Drupal, and three mechanisms in it are deliberate:

• A real unlock/re-lock window for updatedb. On D10+ the post hook unlocks the platform (DEPLOY2/LOCAL/DRUSH/UNLOCK: patches reverted, vendor/drush executable), runs the site-local Drush on the backend's own interpreter and that interpreter's php.ini (exec '<php>' -c '<ini>' vendor/drush/drush/drush.php ..., the shape _provision_site_local_drush_cmd() builds), addressed by --root=<docroot> --uri=<uri> so the site bootstraps in-process from its settings, re-locks (DEPLOY10/LOCAL/DRUSH/LOCK), and only then rebuilds the container by core itself. A locked platform runs no site-local Drush at all under PHP 8, so a chmod-only window (the previous design) ran nothing; and compiling the container in the unlocked state and leaving it is what used to bake Drush\Log\DrushLog into the persisted container and fatal every site on the platform, hence the core rebuild last. Just before the window the hook refreshes the account's Drush 10+ alias store (site:alias-convert), on the same interpreter and ini: the bare drush10 launcher takes whatever ini the shell exports, and every modern Drush refuses the account's Drush 8 ini (open_basedir). Either step failing is a warning (DRUSH/TEN/ALIASES/FAIL, DEPLOY9/LOCAL/DRUSH/FAIL), never a notice, so a deploy whose schema updates did not run cannot end SUCCESSFUL.
• Old-platform unlock/re-verify. Cross-platform moves unlock the source platform before updatedb and re-verify (re-lock) it after, both via provision_backend_invoke() against the old_platform option (deploy.provision.inc:209-217 and :339-343).
• Update-tracker clear. _provision_drupal_clear_update_fetch_tasks() (deploy.provision.inc:467) drops the re-imported update_fetch_task key-value collection after every deploy-updatedb path, so update.module never wedges on orphaned tracker rows. Non-fatal on every path.

When a D10+ platform has no executable site-local Drush, the post hook skips updatedb with an explicit recovery warning instead of falling back to Ægir's Drush 8 (which cannot bootstrap D10+ and would only fail noisily).

Version engines

provision_drupal_drush_engine_drupal() (provision_drupal.drush.inc:32) declares the drupal engine types clear, cron_key, deploy, import, install, packages, verify. drush_include_engine('drupal', 'install') and friends resolve to platform/drupal/<type>[_<major>].inc — 38 engine *.inc files that run inside the target site's own bootstrap, so each must be written for its Drupal major (see platform/drupal/README.md). This is where per-major divergence belongs; the orchestrators stay version-agnostic.

Finally, provision_drupal_drush_exit() (provision_drupal.drush.inc:52) persists successful option state to the site's drushrc.php after any provision-* command, unless provision_save_config is off.

The context system: d() and provision-save

Contexts are provision's data model: named server/platform/site objects persisted as Drush alias files. There is no database backend-side — the alias files under ~/.drush/ are the store, and the frontend refreshes them through provision-save before every task.

• d() (provision.context.inc:23) — the accessor. A static registry keyed by normalised alias name (provision_normalise_context_name(), provision.inc:1416). d('@name') returns (by reference) the context, creating it on first access via provision_context_factory() (provision.context.inc:113); d() returns the root object set at init; d('all') returns the whole registry. A guard refuses to run before Drush finishes bootstrapping (:41) — calling d() at file scope in a commandfile is a classic three-day bug. On creation the factory result gets method_invoke('init'), type_invoke('init'), then the provision_context_alter Drush hook (:74), the supported way to swap in a context subclass.
• Factory — the alias record (cached via provision_sitealias_get_record(), :89) or the CLI context_type option picks the type; the class is literally "Provision_Context_{$type}" (:130).
• Provision_Context (Provision/Context.php:12) — magic __get/__set back a $properties array, which is exactly what write_alias() (:240) hands to Provision_Config_Drushrc_Alias to write the alias file. is_oid($name) (:83) marks a property as a context reference: reading it returns d($value), which is how d('@site')->platform->server->aegir_root chains lazily across alias files. setProperty($field, $default) (:215) is the init-time idiom: take the CLI/alias option if present, else the default.
• Invocation plumbing — method_invoke() (:136) calls the method if it exists (via the static provision::method_invoke(), provision.inc:1449 — silent no-op otherwise) and then mirrors the call onto all subscribed services (services_invoke(), :290). type_invoke('save') → save_site() etc.; command_invoke('verify') → verify_site_cmd() etc. These name conventions are the extension points service classes implement.
• Subclasses (Provision/Context/): server.php derives backup_path, config_path, include_path, clients_path from aegir_root (:63-66) and spawns services; platform.php sets parent_key = 'server' and owns root/makefile; site.php sets parent_key = 'platform', owns uri, computes site_path = root . '/sites/' . uri, and runs provision_auto_fix_platform_root() so a Composer-managed web root is never mistaken for the platform root. Each class's static option_documentation() feeds the provision-save option table — adding a persisted property means adding it there.
• drush_provision_save() (provision.drush.inc:371) — type_invoke('save') + write_alias(), or unlink() of the alias on --delete.

Services: subscription, drivers, dispatch

The service layer decouples "a site needs a database" from "this server runs Percona":

• Declaration. hook_provision_services() — db (db/db.drush.inc:16) and http (http/http.drush.inc:24). The hook returns array('<service>' => <default type>).
• Driver spawn (server side). Provision_Context_server::load_services() (Provision/Context/server.php:76) iterates declared services and spawn_service() (:86) instantiates Provision_Service_{$service}_{$type} from the server's {$service}_service_type property — http_service_type=nginx → Provision_Service_http_nginx, db_service_type=mysql → Provision_Service_db_mysql. No type → Provision_Service_null. The type value is set by the frontend when it saves the server context.
• Subscription (consumer side). Static subscribe_<type> methods run during context init (Provision/Context.php:201): Provision_Service_db::subscribe_site() (db/Provision/Service/db.php:10) wires db_server (default @server_master) and service_subscribe('db', …); Provision_Service_http::subscribe_platform() (http/Provision/Service/http.php:36) wires web_server the same way. Context::service('db') (Provision/Context.php:270) then resolves through the subscription, else up the parent_key chain, ending at @server_master's spawned drivers.
• Base classes. Provision_Service (Provision/Service.php:10) extends Provision_ChainedState (Provision/ChainedState.php) — the ->succeed('…')->fail('…') chaining idiom shared with Provision_FileSystem, reachable as the provision_file() singleton (provision.file.inc:12). A service carries a $configs registry of config classes per context level, restart() (:276) with a driver-supplied default_restart_cmd(), and the verify_server_cmd/verify_platform_cmd/verify_site_cmd family the verify fan-out calls.
• Driver chains. DB: Provision_Service_db → db/pdo.php → db/mysql.php — MySQL/Percona is the only driver BOA runs (db/mysql/mysql_service.inc is an empty legacy shim). HTTP: http/public.php → http/nginx.php (registers Provision_Config_Nginx_{Server,Inc_Server,Site}, sets satellite_mode = 'boa' and the nginx_has_* capability properties in init_server(); http/nginx/ssl.php on top for SSL). The apache*, cluster and pack drivers are pre-fork upstream residue — retained, not exercised by BOA; do not treat their behaviour as authority.

Class loading

provision.inc:10 requires vendor/autoload.php (Composer-pinned deps — Symfony Process for provision_process(), provision.inc:1307).

The Provision_* classes themselves resolve through a second, dedicated \Composer\Autoload\ClassLoader created by provision_autoload() (provision.inc:19): provision_autoload_register_prefix('Provision_', …) (:42) is called with the repo root at provision.inc:66, and again by db_provision_register_autoload() (db/db.drush.inc:55) and http_provision_register_autoload() (http/http.drush.inc:13) to add db/ and http/ as additional roots for the same prefix.

That multi-root PEAR-style mapping is how Provision_Service_http_nginx resolves to http/Provision/Service/http/nginx.php. The psr-0 entry in composer.json:28-31 maps the prefix one directory too deep (Provision/Provision/…) and never matches — the runtime registration is the operative mechanism.

Extensions register their own class roots the same way (the db/http commandfiles are the template).

Config render pipeline

Provision_Config (Provision/Config.php) is the template writer every generated file goes through. write() (:195): ensure directory → load_template() → process() → render_template() (:169) → atomic-ish write with $mode/$group applied. Three alter points, all plain Drush hooks:

• hook_provision_config_load_templates() / …_load_templates_alter() (Config.php:120,126) — replace the template file outright; otherwise the class walks its own inheritance chain looking for $this->template next to each class file.
• hook_provision_config_variables_alter() (:173) — mutate the variable bag just before extract() + buffered eval('?>' . $template).
• The Nginx config classes additionally append drush_command_invoke_all('provision_nginx_server_config' / 'provision_nginx_vhost_config') output into $data['extra_config'] (http/Provision/Config/Nginx/Server.php:10, Site.php:10, and the Ssl/ twins) — free-form config injection without touching templates.

The template set itself (server.tpl.php, vhost*.tpl.php, vhost_include.tpl.php) and what BOA renders into it are web-stack material, indexed from the topic chapter.

The API surface: provision.api.php

provision.api.php (577 L) is documentation-as-code: the $options recognised from drushrc (provision_backup_suffix, provision_composer_install_platforms, provision_mysqldump_suppress_gtid_restore, …) in the file head (:11-57), then every hook:

• hook_provision_services() (:82)
• hook_provision_context_alter() (:96)
• the Nginx/Apache render hooks (:220, :239, :260)
• the template/variable alters (:274-:311)
• the directory/permission/settings alters (:354-:407)
• hook_provision_deploy_options_alter() (:420)
• hook_provision_prepare_environment() (:458)
• the DB name/user/options alters (:505-:552)

If you are extending provision rather than maintaining it, start on the Provision API leaf, which pairs this surface with a working extension skeleton; the frontend counterpart hooks are on Hosting API, and the paired-module pattern is on Extending Ægir.

Codebase lock state (BOA-fork addition)

The D10/D11 "Ægir-patched platform" model adds a state machine the upstream never had, and several commands consult it:

• provision_check_codebase_status() (provision.inc:153) — sets d()->appRoot, d()->symfonyDir and the isCorePatched / isConsolePatched / isValidatorPatched sentinels by inspecting the actual tree.
• provision_drush_local_lock_state() (provision.inc:243) — derives locked | unlocked | partial | unknown purely from on-disk reality: patch sentinels plus the 0400 (locked) / 0775 (unlocked) bits on vendor/drush and the symfony/console Input/Style dirs, resolving local-vs-above vendor layouts. No marker files.
• Lock/unlock bodies at provision.inc:677 / :300; the provision-dlock/provision-dunlock commands are thin wrappers over the same canonical functions (platform/dlock.provision.inc:21-60), so the UI task and Platform Verify share one battle-tested path, and re-running in the already-target state is a clean no-op.

Supporting and legacy files

• provision.context.inc (133 L) — d(), the factory, the alias cache; included from the tail of provision.inc (:1456-1458) together with provision.service.inc (5-line compat shim) and provision.file.inc.
• parse.backend.inc — drush_provision_backend_parse(), the backend-parse callback that renders --backend JSON from stdin human-readable.
• Pre-fork residue retained in the fork but unused by BOA: the aegir*.make family (BOA builds Hostmaster from its own make chain — see Hostmaster frontend internals), upgrade.sh.txt, debian/, scripts/ci-aegir-*, Vagrantfile, provision-tests/, and the Apache config/service classes. Verify against BOA code before citing any of them; retired pieces are tracked in Discontinued features.

Related

• Ægir backend APIs — topic chapter — the task round-trip diagram this page zooms into.
• Provision API — the hook surface for extension authors, with a skeleton.
• Hosting module suite map — the frontend modules whose task nodes become these commands.
• Drush fork internals — the Drush 8 fork provision runs under, including the extension deny-filter.
• Install & staged-setup internals — how this clone lands on a box during AegirSetup.
• Commands reference and Variables reference — consolidated CLI and _VAR tables.