documentation/tools/creating-new-drutopia-site.md

# Setting up a new Drutopia site

Agaric manages a large number of Drupal, primarily [Drutopia](https://drutopia.org/) sites.

Please see [the Drutopia Platform README for an overview of hosting and deploying Drutopia sites](https://gitlab.com/drutopia-platform/drutopia_host#introduction )
If you won't be deploying, skip overall setup.

Following this guide requires a working [DDEV](https://ddev.readthedocs.io/en/latest/) installation.

## Overall setup

Strongly recommended to set up locally like this:

```
mkdir -p ~/Projects/drutopia-platform
cd ~/Projects/drutopia-platform
git clone git@gitlab.com:drutopia-platform/build_source.git
git clone git@gitlab.com:drutopia-platform/drutopia_host.git
cd drutopia_host
git clone git@gitlab.com:drutopia-platform/build_artifacts.git
git clone git@gitlab.com:drutopia-platform/hosting_private.git
```

```{note}
These last two repositories are private to Agaric and others maintaining the Drutopia platform, so if you're following along at home you'll have to create your own analogs, per [the documentation](https://gitlab.com/drutopia-platform/drutopia_host#introduction).
```

Commands for copying throughout will assume this above setup.

## Create a new site project

Ordinarily this will live under https://gitlab.com/agaric/sites (for Agaric clients) or https://gitlab.com/drutopia-platform/sites (for direct Drutopia platform members).

Copy the `composer.json`, `composer.lock`, and `.gitignore` files used by the appropriate build source, such as the [default build source for the Drutopia Platform](https://gitlab.com/drutopia-platform/build_source).

Replace "example" below with name of the site, usually derived from the main domain name, for instance `example-com`:

```
MY_SITE="example"
mkdir -p ~/Projects/agaric/sites/$MY_SITE
cd ~/Projects/agaric/sites/$MY_SITE
```

Once you have created and are in this directory, whether `agaric/sites` or `drutopia-platform/sites` or wherever you want your project to live within the GitLab namespace, you can copy-paste these commands for a quick start:

```
wget https://gitlab.com/drutopia-platform/build_source/-/raw/master/composer.json
wget https://gitlab.com/drutopia-platform/build_source/-/raw/master/composer.lock
wget https://gitlab.com/drutopia-platform/build_source/-/raw/master/.gitignore
ddev config --docroot=web --project-type=drupal8 --composer-version=2 --webserver-type=apache-fpm --mariadb-version=10.1 --php-version=7.4 --create-docroot
ddev start
ddev auth ssh
ddev composer install
```

```{note}
Webserver, PHP, and MySQL versions and types are selected here to match those used on Elizabeth and should be adjusted to match your live environment, including double-checking that they are still valid for `elizabeth.mayfirst.org`.  Please update this documentation if it changes!  The last two options (enabling Xdebug by default and not using DNS for your local site) are optional at the developer's preference. Additional ddev options can be found at [ddev.readthedocs.io/en/latest/users/cli-usage](https://ddev.readthedocs.io/en/latest/users/cli-usage/).
```

## Setting up database and configuration for the live instance

In order to get a configuration that has the proper site key, it is easiest to first deploy the site to the eventual live location, and sync that database locally.

If you are creating a specialized build of Drutopia, you will have to add that to the host vars, and build that prior to deploying the site. `ahoy vars-edit` and `ahoy deploy-build <build_target>` are used for this. Note that new builds should be added ONLY as absolutely required. Configuration, and themes should be leveraged as much as possible prior to resorting to a new build. If additional/different modules are required, a new build is required - do *NOT* add them to build_source except when they are known to be required for *ALL* Drutopia basic sites.

Create a new site (member entry) per instructions in Drutopia hosting. The simplest method is to use `ahoy new-site <member-slug>` and follow it's output to get started. Then use `ahoy deploy-site <member-instance>` to deploy one.

### Configure drush aliases

The [drush site aliases file](https://github.com/drush-ops/drush/blob/9.5.x/examples/example.site.yml) can be used to provide easy access to the live/test instances of a site. From the root of your project directory (e.g. `agaric/sites/example/`), you may create one with:

```
MY_SITE="example-com"
SERVER="drutopia.org"
mkdir -p drush/sites/
cat << EOF > drush/sites/self.site.yml
live:
  host: ${SERVER}
  paths:
    drush-script: /home/${MY_SITE//-/_}_live/site/vendor/bin/drush
  root: /home/${MY_SITE//-/_}_live/site/web
  uri: 'https://${MY_SITE}-live.drutopia.org/'
  user: ${MY_SITE//-/_}_live
test:
  host: ${SERVER}
  paths:
    drush-script: /home/${MY_SITE//-/_}_test/site/vendor/bin/drush
  root: /home/${MY_SITE//-/_}_test/site/web
  uri: 'https://${MY_SITE}-test.drutopia.org/'
  user: ${MY_SITE//-/_}_test
EOF
```

This will create a self.site.yml using the expected pattern of "site_name_INSTANCE" (e.g. example_com_live for the example-com live instance). Supply the URL form of the site name for the MY_SITE variable (i.e. with dashes, rather than underscores).

### Syncing, and setting up configuration

Drutopia releases will expect the configuration in `$project_root/config/sync`. Be sure to set the appropriate variable in `settings.php` for it to be stored/retrieved from there. Note not to use `settings.ddev.php`, as this will be generated during `ddev start`:

```php
$settings['config_sync_directory'] = '../config/sync';
```

While in `web/sites/default/settings.php` also prevent site administrators being told they can install new modules when they can't and add:

```php
$conf['allow_authorize_operations'] = FALSE;
```

(All this should be updated to use a distribution-wide settings.php when we have that.)

Once you also have a working Drush installation and a live instance, you can then aquire and export the initial configuration with:

```
ddev . drush sql-sync @live @self
ddev . drush -y cex
```

Similarly, you can get live data and files down to your local development environment any time with:

```
ddev . drush -y sql-dump > data/paranoia.sql
ddev . drush -y sql-drop
ddev . drush -y sql-sync @live @self
ddev . drush -y rsync @live:%files @self:%files
```

And get the latest code for your distribution (here presuming [build_source](https://gitlab.com/drutopia-platform/build_source/)) like this:

```
wget https://gitlab.com/drutopia-platform/build_source/-/raw/master/composer.lock
ddev composer install
```

Or update the build source yourself, if you've got it downloaded locally per [overall setup](#overall-setup) above:

```
ddev composer update
cp composer.json ~/Projects/drutopia-platform/build_source/
cp composer.lock ~/Projects/drutopia-platform/build_source/
cd ~/Projects/drutopia-platform/build_source/
git add -p composer.json
# IMPORTANT: Make no changes to `composer.json` that shouldn't be shared by every site
git commit -m "Add x module or patch for z"
git add -p composer.lock
git commit -m "Update composer lock file"
git push
cd -
```

### Finalize hosting setup

#### Domain name

Configure the domain name to point its nameservers at:

```
a.ns.mayfirst.org
b.ns.mayfirst.org
```

Then [set up a new hosting order](https://members.mayfirst.org/cp/index.php?area=member&member_id=458&action=new).  Give the username the same name as your repository ("example-com" as used throughout this documentation).

Then under DNS change the **a** record for your full domain to IP Address `162.247.75.218` and make any additional CNames you want (at least www. for your domain, for instance "www.example.com" to go to Server Name of your domain, for instance "example.com").  You can delete the value for www. that is already there.

#### HTTPS Certificates

First, ensure that all domain names are pointed to the new location. Deploy the site with the appropriate `server_hostname`, and `server_aliases` already set. Note that if `canonical_redirect: true` you should use the default `<member>-live.drutopia.org` for `server_hostname` until the certificate is provisioned.

Currently, acquiring the certificates must be peformed directly on the server (as root) by running:
`certbot certonly --webroot -w /home/<member>/site/web -d example.org -d www.example.org -d example-live.drutopia.org`

Adjust the vault configuration to include the path to the newly generated cert and key values in the SSL settings for the member (`site_ssl_cert_path` and `site_ssl_key_path`). If there are no further changes, an `ahoy deploy-site-minimal <member>` deployment is enough to activate the certificates (if host names have been added, a normal deploy is necessary for them to reach Drupal's setting for allowed hostnames).

#### Reconfigure For Future Deployments

Once the initial site has been installed, be sure to update the vault parameters, as appropriate:

```
config_import: true
drupal_install: False (or else remove this line)
```

#### Potential Traps

If configuration is not importing: log in as root and delete the file `/home/<member>/site/CUSTOM-VERSION` and run the config-forced install option again - this tells the activate script to repopulate the config folder from the custom source.

See [this issue](https://gitlab.com/drutopia-platform/drutopia_host/-/issues/18) for the latest status on this problem.