documentation/tools/creating-new-drutopia-site.md

# Setting up a new Drutopia site

Agaric manages a large number of Drupal, primarily [Drutopia](https://drutopia.org/) sites.

Please see [the Drutopia Platform README for an overview of hosting and deploying Drutopia sites](https://gitlab.com/drutopia-platform/drutopia_host#introduction )
If you won't be deploying, skip overall setup.

Following this guide requires a working [DDEV](https://ddev.readthedocs.io/en/latest/) installation.  Deploying to Drutopia will require a locally installed composer, as well as ansible. See also the [general Git setup](git-setup) if you have not yet.

## Overall setup

In line as much as possible with [recommended locations](recommended-local-project-locations), the local setup can be done like this:

```
mkdir -p ~/Projects/drutopia-platform
cd ~/Projects/drutopia-platform
git clone git@gitlab.com:drutopia-platform/build_source.git
git clone git@gitlab.com:drutopia-platform/drutopia_host.git
cd drutopia_host
git clone git@gitlab.com:drutopia-platform/build_artifacts.git
git clone git@gitlab.com:drutopia-platform/hosting_private.git
```

```{note}
These last two repositories are private to Agaric and others maintaining the Drutopia platform on May First's Elizabeth server, so if you are following along at home you'll have to create your own analogs, per [the documentation](https://gitlab.com/drutopia-platform/drutopia_host#introduction).
```

Commands for copying throughout will assume this above setup.

## Create a new site project

For Agaric clients, the online home for the project will usually be at [Agaric's Forgejo project hosting site](https://git.agaric.com), first creating an organization (such as `example-client`, the short name that Forgejo takes first must be treated as a machine name) and then the project (such as `example-org`) that would live a URL such as `https://git.agaric.com/example-client/example-org`.

Copy the part of the URL after `git.agaric.com` so the namespace will be consistent on your local machine and create the directory at `~/Projects`.  With the examples above it would be `mkdir -p ~/Projects/example-client/example-org` and `cd ~/Projects/example-client/example-org`.

Once you have created and are in this directory, wherever you want your project to live within that namespace, you can copy-paste these commands for a quick start:

```
wget https://gitlab.com/drutopia-platform/build_source/-/raw/main/composer.json
wget https://gitlab.com/drutopia-platform/build_source/-/raw/main/composer.lock
wget https://gitlab.com/drutopia-platform/build_source/-/raw/main/.gitignore
mkdir -p scripts
wget -O scripts/pull.sh https://gitlab.com/drutopia-platform/build_source/-/raw/main/scripts/pull.sh
wget -O scripts/retain-custom-htaccess.sh https://gitlab.com/drutopia-platform/build_source/-/raw/main/scripts/retain-custom-htaccess.sh
chmod +x scripts/pull.sh
chmod +x scripts/retain-custom-htaccess.sh
ddev config --docroot=web --project-type=drupal10 --webserver-type=apache-fpm --database=mariadb:10.8 --php-version=8.1 --create-docroot
ddev start
ddev auth ssh
ddev composer install
git init
git add .
git commit -m "Begin repository based on build_source main"
```

```{note}
Webserver, PHP, and MySQL versions and types are selected here to match those used on Elizabeth and should be adjusted to match your live environment, including double-checking that they are still valid for `elizabeth.mayfirst.org`.  Please update this documentation if it changes!  The last two options (enabling Xdebug by default and not using DNS for your local site) are optional at the developer's preference. Additional ddev options can be found at [ddev.readthedocs.io/en/latest/users/cli-usage](https://ddev.readthedocs.io/en/latest/users/cli-usage/).
```

## Setting up database and configuration for the live instance

In order to get a configuration that has the proper site key, it is easiest to first deploy the site to the eventual live location, and sync that database locally.

If you are creating a specialized build of Drutopia, you will have to add that to the host vars, and build that prior to deploying the site. `ahoy vars-edit` and `ahoy deploy-build <build_target>` are used for this.

```{note}
New builds should be added **only** as absolutely required. Configuration, and themes should be leveraged as much as possible prior to resorting to a new build. If additional modules are required, consider adding them to `build_source`, knowing they will be available for *all* Drutopia SaaS sites.
```

Create a new site (member entry) per instructions in Drutopia hosting. The simplest method is to use `ahoy new-site <member-slug>` and follow its output to get started. Then use `ahoy deploy-site <member-instance>` to deploy one. Because the install using `drush site-install` that would be run when `drupal_install: true` is set in the site settings in Drutopia hosting, it is currently recommended to set this to false and use the UI installer or run `php docroot/core/scripts/drupal quick-start drutopia --no-interaction` when ssh'd into the server.

### Configure drush aliases

The [drush site aliases file](https://github.com/drush-ops/drush/blob/9.5.x/examples/example.site.yml) can be used to provide easy access to the live/test instances of a site. From the root of your project directory (e.g. `agaric/sites/example/`), you may create one with:

```bash
MY_SITE="example-com"
SERVER="drutopia.org"
mkdir -p drush/sites/
cat << EOF > drush/sites/self.site.yml
live:
  host: ${SERVER}
  paths:
    drush-script: /home/${MY_SITE//-/_}_live/site/vendor/bin/drush
  root: /home/${MY_SITE//-/_}_live/site/web
  uri: 'https://${MY_SITE}-live.drutopia.org/'
  user: ${MY_SITE//-/_}_live
test:
  host: ${SERVER}
  paths:
    drush-script: /home/${MY_SITE//-/_}_test/site/vendor/bin/drush
  root: /home/${MY_SITE//-/_}_test/site/web
  uri: 'https://${MY_SITE}-test.drutopia.org/'
  user: ${MY_SITE//-/_}_test
EOF
```

This will create a self.site.yml using the expected pattern of `site_name_INSTANCE` (e.g. `example_com_live` for the example-com live instance). Supply the URL form of the site name for the `MY_SITE` variable (i.e. with dashes, rather than underscores).

### Syncing, and setting up configuration

Our standard is to place configuration exports in `$project_root/config`. Set the this in `settings.php`:

```php
$settings['config_sync_directory'] = '../config';
```

While in `web/sites/default/settings.php` also prevent site administrators being told they can install new modules when they can't and add:

```php
$settings['allow_authorize_operations'] = FALSE;
```

This `settings.php` file is only being used for local development. Drutopia host is responsible for creating all settings during deployment.

(All this should be updated to use a distribution-wide settings.php when we have that. Until the, use `git add -f web/sites/default/settings.php` to include this in your repo)

```{note}
Do not to use `settings.ddev.php`, as this will be re-generated during `ddev start`.
```

Once you also have a working Drush installation and a live instance, you can then aquire and export the initial configuration with:

```
ddev drush sql-sync @live @self
ddev drush -y cex
```

Similarly, you can get live data and files down to your local development environment any time with:

```
ddev drush -y sql-dump > data/paranoia.sql
ddev drush -y sql-drop
ddev drush -y sql-sync @live @self
ddev drush -y rsync @live:%files @self:%files
```

And get the latest code for your distribution (here presuming [build_source](https://gitlab.com/drutopia-platform/build_source/)) like this:

```
wget https://gitlab.com/drutopia-platform/build_source/-/raw/master/composer.lock
ddev composer install
```

Or update the build source yourself, if you've got it downloaded locally per [overall setup](#overall-setup) above:

```
ddev composer update
cp composer.json ~/Projects/drutopia-platform/build_source/
cp composer.lock ~/Projects/drutopia-platform/build_source/
cd ~/Projects/drutopia-platform/build_source/
git add -p composer.json
# IMPORTANT: Make no changes to `composer.json` that shouldn't be shared by every site
git commit -m "Add x module or patch for z"
git add -p composer.lock
git commit -m "Update composer lock file"
git push
cd -
```

### Finalize hosting setup

#### Domain name

Configure the domain name to point its nameservers at:

```
a.ns.mayfirst.org
b.ns.mayfirst.org
```

Then [set up a new hosting order](https://members.mayfirst.org/cp/index.php?area=member&member_id=458&action=new).  Give the username the same name as your repository ("example-com" as used throughout this documentation).

Then under DNS change the **a** record for your full domain to IP Address `162.247.75.218` and make any additional CNames you want (at least www. for your domain, for instance "www.example.com" to go to Server Name of your domain, for instance "example.com").  You can delete the value for www. that is already there.

#### HTTPS Certificates

First, ensure that all domain names are pointed to the new location. Deploy the site with the appropriate `server_hostname`, and `server_aliases` already set. Note that if `canonical_redirect: true` you should use the default `<member>-live.drutopia.org` for `server_hostname` until the certificate is provisioned.

Currently, acquiring the certificates must be peformed directly on the server (as root) by running:
`certbot certonly --webroot -w /home/<member>/site/web -d example.org -d www.example.org -d example-live.drutopia.org`

Adjust the vault configuration to include the path to the newly generated cert and key values in the SSL settings for the member (`site_ssl_cert_path` and `site_ssl_key_path`). If there are no further changes, an `ahoy deploy-site-minimal <member>` deployment is enough to activate the certificates (if host names have been added, a normal deploy is necessary for them to reach Drupal's setting for allowed hostnames).

#### Reconfigure For Future Deployments

Once the initial site has been installed, be sure to update the vault parameters, as appropriate:

```
config_import: true
drupal_install: False (or else remove this line)
```

#### Potential Traps

If configuration is not importing: log in as root and delete the file `/home/<member>/site/CUSTOM-VERSION` and run the config-forced install option again - this tells the activate script to repopulate the config folder from the custom source.

See [this issue](https://gitlab.com/drutopia-platform/drutopia_host/-/issues/18) for the latest status on this problem.