Per-instance group — instgrp

Every Octopus account on a BOA server owns a private Unix group named after itself: the account o1 has the group o1, and that group is the primary group of the account's backend user (o1), its limited-shell user (o1.ftp) and every per-client sub-account (o1.<client>). The account's files carry that group. Nothing outside the account is a member of it.

Before this, every account's identities shared one box-wide primary group, users, and so did their files. A file that granted read to its group — a site's drushrc.php with the database credentials, a Drush alias — granted it to every other account on the box. The web pool identity was kept out (it runs as www-data), and the limited shell fences the paths a tenant can type, but a tenant's backend identity, or a command run by an allowed tool, could open a neighbour's credential files.

The per-instance group closes that: group read now means this account's identities, nothing wider.

What changes, and what does not

users stays on every identity on purpose. On a BOA server it is not a data group but an execute capability: the system binaries are root:users 0750 (see strict binary permissions), lshell itself included, and the shared-core directories that tenants may write to are group users. An identity outside users cannot run git, composer or even log in. Nothing removes it; the build-time guards on the BOA tree refuse any tool that would.

How an account gets there

Nothing to do. The octopus upgrade converts each account it upgrades, and an account created after this ships is born converted when the box is ready for it (the tool present and every root-run maintenance tool group-aware, the same gate the upgrade's conversion enforces); otherwise it is born on the box-wide group and the next upgrade converts it. A group name already held by another identity (a member, or a user whose primary group it is) leaves the newborn on the box-wide group too, with a NOTE and no conversion attempt in that run; every later upgrade alarms until the name is freed. Each conversion is recorded in the account's log/instance-group.txt. An account already converted costs one read-only traversal on every later upgrade (early-quit on the first path outside its group), no walk and no write.

The conversion is ordered so that no identity ever loses access it had:

1. the group is created (an old group of the same name is adopted only when nothing else holds it);
2. users, www-data and the lshell groups are asserted as supplementary groups on every identity of the account;
3. only then does the primary group move to the account's group;
4. the account's roots are walked and every path still in group users is re-grouped — a group-only change, one chown -R --from per root, that never follows a symlink and never touches the www-data paths (the immutable control files in the shell homes are handled one by one);
5. the marker is written.

The hosting queue keeps running during an octopus upgrade and a tenant's own sessions keep their old group set, so a file can be written into a directory the walk has not reached yet. The walk is re-run over such residue (twice at most); whatever still remains is reported as DRIFT and finished by the next pass. The conversion is kept, never rolled back because of files: a path left in users is the old state, not a new exposure, while a rollback would leave the busiest accounts unconverted on every release. Only a failed identity move rolls the account back.

The primary group of an identity in use cannot move: usermod refuses a user that has a process in its own root whose real, effective or saved uid is that user's. A tenant logged in over SSH or SFTP as oN.ftp or a sub-account is such a process; an FTPS session is not, because the FTP server runs it in a chroot. The conversion asks the same question: it waits up to a minute for every identity it still has to move, long enough for a cron run of the account to end, and skips the account while one stays in use, before it has changed anything. It looks again right before each move; an identity that comes into use in between rolls the moved identities back and the account is skipped the same way, or, if one of them cannot be moved back, the conversion fails and names it. The upgrade report names the identity and its processes in its alert; the next upgrade that finds the identity idle converts the account, or run instgrp convert oN once that session has ended.

The conversion refuses to start while another BOA run is live, waits for the account's own provision tasks to finish, and skips a still-busy account and an account frozen for a migration (log/proxied.pid) to the next upgrade. It also refuses when any root-run maintenance tool on the box still carries the old, box-wide form — an old tool would write users back onto the tree on its next pass, and an old websh would lock the tenant out of its shell; that deferral is a note in the upgrade report and the next upgrade converts once the tools have refreshed. It refuses a first conversion for the same reason on a box that has received the fetched tools ahead of its BOA upgrade (a staged publish): the Octopus arm that converts rides the BOA release, and until that release's barracuda pass has stamped the box (/var/log/boa/instgrp-arm.ready.txt, shown by instgrp check as arm:) the tar's own libraries would still write the box-wide group. An account that is already converted takes its read-only re-pass regardless of the stamp. A skipped (busy, logged in, frozen) or failed account is named in the upgrade report as an alert; nothing retries it before the next upgrade. Every BOA tool that writes into account trees derives the group from the account it is writing for (users until the account is converted, the account's group after), so a mixed fleet — converted here, not yet there — keeps working throughout.

Reading and steering it: instgrp

instgrp is a root-only operator command (installed in /opt/local/bin, never exposed through sudo or the limited shell):

instgrp status o1        # or: instgrp status all
instgrp convert o1       # idempotent; also what the upgrade arm runs
instgrp reclaim o1       # the file half only: no identity change, no lock; skips a frozen account like convert (--force overrides)
instgrp revert o1        # writes _INSTANCE_GROUP=NO into the account's cnf; an identity in use is finished by the worker
instgrp check            # are the fetched maintenance tools group-aware?

status prints the marker, the group, each identity's group set, the FPM pool identities (which must not be in the group), the per-group file counts under the account's roots (users, the account's, www-data, root, no group, and any other named group), and one verdict line:

A path in another named group is the case to take seriously: every box allocates account gids from the same system range, so a tree copied from another box (a migration, a root-run restore of a foreign archive) can land in the gid of a different account here, whose identities can then read the copied credential files. status counts and names those groups, and convert and reclaim claim them for the account, as they do paths in no group and paths back in users.

A marker whose recorded gid is not the account group's gid on this box is reported as stale and ignored: it recorded another box's conversion. The migration tools never carry it any more; xoct transfer, xcopy transfer, a hand-run xmass sync --live and aegir2boa-stage2 transfer run the destination's own instgrp reclaim --force after each copy (the destination's own log/proxied.pid, if any, is its demotion artefact from an earlier cutover or a freeze left by a killed import, not an account served from elsewhere; when the tool keeps deferring behind a live BOA run for the whole bounded wait, the same pass runs inline instead), and the xmass legs map the source account's group onto the destination account's group as they copy, so the 15-minute autosync of a standby never lands files in a foreign gid. The root-run backup restores (backboa, duobackboa, multiback, mybackup) re-group what they restored and run reclaim for the account, because duplicity re-applies the archived ownership. That covers a destination inside an account's tree (/data/disk/oN/... or a shell home, relative paths resolved first); a restore staged anywhere else is yours to follow with instgrp reclaim oN once the files are in place.

Between upgrades, drift is caught by the nightly: the account worker probes the credential-bearing paths (the Drush aliases, backups, config, tools, the hostmaster sites, every drushrc.php under static) and runs reclaim on a hit, and the 3-minute limited-shell worker moves an identity that fell back to the box-wide primary group back onto the account's group once that identity is no longer in use (until then its log says the move waits for an idle pass, and it tries again on the next one; a deferral that has lasted a day is raised through the worker's incident channel, /var/log/boa/manage_ltd.incident.log and one mail a day per identity unless _INCIDENT_REPORT is OFF). The same worker finishes a revert that had to leave an identity on the account's group (below). Both tools also put back a passwd home field that an interrupted group move left at a staging directory, retrying for ten seconds while the identity is in use; status reports such a home as INCONSISTENT until then. An account frozen for a migration (log/proxied.pid) is outside all of it: the nightly never visits it and reclaim skips it, so a frozen destination is healed only by the migration tool's own pass.

The exit status follows the verdict (0, 0, 2, 3; convert and reclaim add 4 skipped — busy, frozen for a migration, or a live BOA run — and convert 5 not ready, and over all the worst class wins), so status all is usable from a check script. For convert, busy includes an identity it still has to move that is in use, and revert skips with 4 in the same two cases before it starts; an identity that comes into use during a revert's file walk stays on the account's group, is named, and revert exits 1 with the group kept. That identity is recorded in /var/log/boa/instgrp.revert-pending.oN; status reports the unfinished revert, the limited-shell worker moves the identity back to users on its first pass that finds it idle and drops the record, and a second revert removes the empty group, or a convert converts again (a convert in between supersedes the pending revert). A convert whose rollback meets an identity in use records it in the same file, and the worker finishes that move back too; the nightly per-account pass writes with the box-wide group and skips its drift probe while the record exists. Every action appends a line to /var/log/boa/instgrp.log.

Opting an account out

_INSTANCE_GROUP=NO in /root/.oN.octopus.cnf (see octopus.cnf) keeps that account on the box-wide group model: the upgrade arm skips it, and a fresh account with that line is born the old way. For a new account, create /root/.oN.octopus.cnf before boa in-octopus holding just two lines, _USER="oN" and _INSTANCE_GROUP=NO; the install completes the file with its defaults and keeps the seeded lines. Never copy another account's cnf for that: it carries the other account's _DOMAIN (the install puts the derived name back, with a NOTE). It does not undo a conversion already made — run instgrp revert oN for that: files first, then identities, the group removed once no path and no identity carries it, and the line written into the cnf so the next unattended upgrade does not convert the account again (--keep-enabled leaves the cnf alone). An explicit instgrp convert ignores the switch; it is an operator's order.

Removing an account

boa cleanup purge oN removes the account's group after its identities, so the gid is not left for a later account to inherit; what the purge leaves on disk (a static store relocated under /mnt, the gems and npm trees) is handed to root first, and the group stays while any path still carries it. The purge itself is described on the Multi-Octopus page. Removing a single sub-account (through the panel or manage_ltd_users) leaves the group in place — it belongs to the account, not to the sub-account.

What this does not close

Be precise about the boundary, because "per-instance group" is easy to over-read:

• Within one account, a per-client sub-account can still read the drushrc.php of a sibling site on the same account: the group is account-wide, not site-wide. That is box-wide becoming account-wide — a strict improvement — and the sub-accounts are kept at their own site directories by the limited shell's Landlock rules, but it is not per-site isolation and must not be described as one.
• Shared codebases (/data/all/…/sites/all/{modules,libraries,themes}) stay root:users and group-writable by every account, by design: they are shared between instances. Cross-tenant write into a shared codebase is not closed by the group change; the honest fix for that is a per-account overlay, not a group.
• The master (/var/aegir) keeps its own group aegir plus users, as it always had.

Relationship to the limited shell

The limited shell confines every tenant shell process with Landlock, so a command run by an allowed tool cannot open a neighbour's files even when the filesystem would let it. The writable set is the account's own roots plus the stores BOA itself linked in as root; a link a tenant places in shared temporary space or in their own tree never widens it. The per-instance group is the complementary, discretionary half: it protects the files themselves, so the backend identity, a cron job, a task running as oN, or any path that is not a tenant shell meets the same boundary. Both are in force; neither replaces the other.

Related

• Database credentials — cloaking — what the 0440 files hold and how the web and command-line paths read them.
• Shared codebase permissions — fixrepo — the group-write repair tool; hand a root-cloned tree to the account with chown -R oN:$(id -gn oN) before running it.
• The codebase lock — the nightly ownership management, which writes the account's group.