87 lines
No EOL
14 KiB
Text
87 lines
No EOL
14 KiB
Text
# List of migration related Drupal modules
|
||
|
||
When one starts working with migrations, it is easy to be overwhelmed by so many modules providing migration functionality. Throughout [the series](https://understanddrupal.com/migrations), we presented many of them trying to cover module one at a time. This with the intention to help the reader understand when a particular module is truly needed and why. But we only scratched the surface. Today's article presents a list of migration related Drupal modules for quick reference. Let's get started.
|
||
|
||
## Core modules
|
||
|
||
At the time of this writing, Drupal core ships with four migration modules:
|
||
|
||
- **Migrate**: provides the base API for migrating data.
|
||
- **Migrate Drupal**: offers functionality to migrate from other Drupal installations. It serves as the foundation for upgrades from Drupal 6 and 7\. It also supports reading configuration entities from Drupal 8 sites.
|
||
- **Drupal Migrate UI**: provides a [user interface for upgrading](https://www.drupal.org/docs/8/upgrade/upgrade-using-web-browser) a Drupal 6 or 7 site to Drupal 8.
|
||
- **Migrate Drupal Multilingual**: is an experimental module required by multilingual translations. When they become stable, the module will be removed from Drupal core. See [this article](https://www.drupal.org/node/2959712) for more information.
|
||
|
||
## Migration runners
|
||
|
||
Once the migration definition files have been created, there are many options to execute them:
|
||
|
||
- [Migrate Tools](https://www.drupal.org/project/migrate_tools): provides [Drush](https://www.drush.org/) commands to [run migrations from the command line](https://understanddrupal.com/articles/tips-writing-drupal-migrations-and-understanding-their-workflow). It also exposes a [user interface to run migrations](https://understanddrupal.com/articles/executing-drupal-migrations-user-interface-migrate-tools) created as [configuration entities](https://understanddrupal.com/articles/defining-drupal-migrations-configuration-entities-migrate-plus-module). It offers support for [migration groups](https://understanddrupal.com/articles/using-migration-groups-share-configuration-among-drupal-migrations) and [tags](https://understanddrupal.com/articles/what-difference-between-migration-tags-and-migration-groups-drupal). The module *depends* on [Migrate Plus](https://www.drupal.org/project/migrate_plus).
|
||
- [Migrate Run](https://www.drupal.org/project/migrate_run): provides Drush commands to run migrations from the command line. It *does not* offer support for migration groups, but tags are supported. The module *does not* depend on Migrate Plus.
|
||
- [Migrate Upgrade](https://www.drupal.org/project/migrate_upgrade): provides [Drush support for upgrading](https://www.drupal.org/docs/8/upgrade/upgrade-using-drush) a Drupal 6 or 7 site to Drupal 8.
|
||
- [Migrate Manifest](https://www.drupal.org/project/migrate_manifest): provides a Drush command for running migrations using a manifest file. See [this article](https://www.drupal.org/node/2350651#s--running-specific-migrations-using-migrate-manifest) for an example of using this module for Drupal upgrades.
|
||
- [Migrate Scheduler](https://www.drupal.org/project/migrate_scheduler): integrates with Drupal Core's [Cron API](https://api.drupal.org/api/drupal/core!core.api.php/function/hook_cron/8.8.x) to execute migrations on predefined schedules.
|
||
- [Migrate Cron](https://www.drupal.org/project/migrate_cron): exposes a user interface to execute migrations when Cron is triggered. At the time of this writing, the module does not execute dependent migrations. Follow [this issue](https://www.drupal.org/project/migrate_cron/issues/3051619) for updates.
|
||
- [Migrate source UI](https://www.drupal.org/project/migrate_source_ui): provides a form for uploading files to use as source for already defined [CSV](https://understanddrupal.com/articles/migrating-csv-files-drupal), [JSON](https://understanddrupal.com/articles/migrating-json-files-drupal), and [XML](https://understanddrupal.com/articles/migrating-xml-files-drupal) migrations. At the time of this writing, it seems that JSON and XML migrations are not being detected. Follow [this issue](https://www.drupal.org/node/3076725) for updates.
|
||
|
||
## Source plugins
|
||
|
||
The Migrate API offers many options to fetch data from:
|
||
|
||
- **Migrate** (core module): provides the `SqlBase` abstract class to help with fetching data from a database connection. See [this article](https://www.drupal.org/docs/8/api/migrate-api/migrate-source-plugins/migrating-data-from-a-sql-source) for an example. It also exposes the `embedded_data` plugin which allows the source data to be defined inside the migration definition file. It was used extensively in the [example migrations](https://github.com/dinarcon/ud_migrations) of [this series](https://understanddrupal.com/migrations). It also offers the `empty` plugin which returns a row based on provided constants. It is used in multilingual migrations for entity references.
|
||
- [Migrate Plus](https://www.drupal.org/project/migrate_plus): combining various plugins, it allows fetching data in [JSON](https://understanddrupal.com/articles/migrating-json-files-drupal), [XML](https://understanddrupal.com/articles/migrating-xml-files-drupal), and SOAP formats. It also provides various plugins for parsing HTML. See [this article](https://isovera.com/blog/handling-html-with-drupals-migrate-api/) by [Benji Fisher](https://www.drupal.org/u/benjifisher) for an example. There is also a [patch to add support for PDF parsing](https://www.drupal.org/project/migrate_plus/issues/3019758).
|
||
- [Migrate Source CSV](https://www.drupal.org/project/migrate_source_csv): allows fetching data from [CSV](https://understanddrupal.com/articles/migrating-csv-files-drupal) files.
|
||
- [Migrate Google Sheets](https://www.drupal.org/project/migrate_google_sheets): leverages Migrate Plus functionality to allow fetching data from [Google Sheets](https://understanddrupal.com/articles/migrating-google-sheets-drupal).
|
||
- [Migrate Spreadsheet](https://www.drupal.org/project/migrate_spreadsheet): allows fetching data from [Microsoft Excel and LibreOffice Calc](https://understanddrupal.com/articles/migrating-microsoft-excel-and-libreoffice-calc-files-drupal) files.
|
||
- [Migrate Source YAML](https://www.drupal.org/project/migrate_source_yaml): allows fetching data from YAML files.
|
||
- [WP Migrate](https://www.drupal.org/project/wp_migrate): allows fetching data from a WordPress database.
|
||
|
||
## Destination plugins
|
||
|
||
The Migrate API is mostly used to move data into Drupal, but it is possible to [write to other destinations](https://understanddrupal.com/articles/drupal-migrations-understanding-etl-process):
|
||
|
||
- **Migrate** (core): provides classes for creating content and configuration entities. It also offers the `null` plugin which in itself does not write to anything. It is used in multilingual migrations for entity references.
|
||
- [Migrate Plus](https://www.drupal.org/project/migrate_plus): provides the `table` plugin for migrating into tables not registered with Drupal Schema API.
|
||
- [CSV file](https://github.com/jonathanfranks/d8migrate/tree/master/web/modules/custom/migrate_destination_csv): example destination plugin implementation to write CSV files. The module was created by [J Franks](https://www.drupal.org/u/franksj) for a [DrupalCamp presentation](https://2018.tcdrupal.org/session/drupal-8-migrate-its-not-rocket-science). Check out the [repository](https://github.com/jonathanfranks/d8migrate/tree/master/web/modules/custom/migrate_destination_csv) and [video recording](https://2018.tcdrupal.org/session/drupal-8-migrate-its-not-rocket-science).
|
||
|
||
## Development related
|
||
|
||
These modules can help with [writing Drupal migrations](https://understanddrupal.com/articles/writing-your-first-drupal-migration):
|
||
|
||
- **Migrate** (core): provides the [log](https://api.drupal.org/api/drupal/core%21modules%21migrate%21src%21Plugin%21migrate%21process%21Log.php/class/Log) process plugin. See [this article](https://understanddrupal.com/articles/how-debug-drupal-migrations-part-1) for an example of its use.
|
||
- [Migrate Devel](https://www.drupal.org/project/migrate_devel): offers Drush options for printing debug information when executing migrations. It also provides the [debug](https://git.drupalcode.org/project/migrate_devel/blob/8.x-1.x/src/Plugin/migrate/process/Debug.php) process plugin. See [this article](https://understanddrupal.com/articles/how-debug-drupal-migrations-part-2) for an example of its use.
|
||
- [Migrate Process Vardump](https://www.drupal.org/project/migrate_process_vardump): provides the [vardump](https://git.drupalcode.org/project/migrate_process_vardump/blob/8.x-1.x/src/Plugin/migrate/process/Vardump.php) plugin. It works like the `debug` plugin.
|
||
|
||
## Field and module related
|
||
|
||
- [Migrate Media Handler](https://www.drupal.org/project/migrate_media_handler): provides migration process plugins to facilitate the migration into Drupal 8 media entities. The source can be Drupal 7 file or image fields. It also supports inline file embeds in rich text. It leverages the DOM parsing plugins provided by Migrate Plus.
|
||
- [Media Migration](https://www.drupal.org/project/media_migration): provides an upgrade path from Drupal 7 to Drupal 8 media entities. The source can be image fields and fields attached to media and file entities.
|
||
- [Migrate File Entities to Media Entities](https://www.drupal.org/project/migrate_file_to_media): migrates Drupal 8.0 file entities to Drupal 8.5 media entities.
|
||
- [Migrate Files](https://www.drupal.org/project/migrate_file): provides process plugins for [migrating files and images](https://understanddrupal.com/articles/migrating-files-and-images-drupal).
|
||
- [Webform Migrate](https://www.drupal.org/project/webform_migrate): provides plugin to help migrating from the Drupal 6 and 7 versions of the Webform module.
|
||
- [Migrate HTML to Paragraphs](https://www.drupal.org/project/migrate_html_to_paragraphs): turns HTML markup into paragraph entities.
|
||
- [Commerce Migrate](https://www.drupal.org/project/commerce_migrate): offers a general purpose migration framework for bringing store information into Drupal Commerce.
|
||
- [Address](https://www.drupal.org/project/address): offers a process plugin to migrate data into fields of type address. It also provides an upgrade path from Drupal 7's [Address Field module](https://www.drupal.org/project/addressfield). See [this article](https://understanddrupal.com/articles/migrating-addresses-drupal) for an example.
|
||
- [Geofield](https://www.drupal.org/project/geofield): offers a process plugin to migrate data into fields of type geofield. See [this article](https://www.drupal.org/docs/8/api/migrate-api/migrate-process-plugins/contrib-process-plugin-geofield_latlon) for an example.
|
||
- [Office Hours](https://www.drupal.org/project/office_hours): offers a process plugin to migrate data into fields of type office hours.
|
||
- [Workbench Moderation to Content Moderation](https://www.drupal.org/project/wbm2cm): migrates configuration from one module to the other.
|
||
|
||
## Modules created by Tess Flynn (socketwench)
|
||
|
||
While doing the research for this article, we found many useful modules created by [Tess Flynn (socketwench)](https://www.drupal.org/u/socketwench). She is a [fantastic presenter](http://drupal.tv/all-videos?search_api_fulltext=socketwench) who also has written about Drupal migrations, testing, and [much more](https://deninet.com/topic/drupal). Here are some of her modules:
|
||
|
||
- [Migrate Directory](https://www.drupal.org/project/migrate_directory): imports files from a directory into Drupal as managed files.
|
||
- [Migrate Process S3](https://www.drupal.org/project/migrate_process_s3): downloads objects from an S3 bucket into Drupal.
|
||
- [Migrate Process URL](https://www.drupal.org/project/migrate_process_url): provides a process plugin to make it easier to migrate into link fields.
|
||
- [Migrate Process Vardump](https://www.drupal.org/project/migrate_process_vardump): helps with [debugging migrations](https://understanddrupal.com/articles/how-configure-xdebug-phpstorm-drupalvm-debug-drupal-migrations-drush-commands-browser).
|
||
- Many process plugins that wrap PHP functions. For example: [Migrate Process Array](https://www.drupal.org/project/migrate_process_array), [Migrate Process Trim](https://www.drupal.org/project/migrate_process_trim), [Migrate Process Regex](https://www.drupal.org/project/migrate_process_regex), [Migrate Process Skip](https://www.drupal.org/project/migrate_process_skip), and [Migrate Process XML](https://www.drupal.org/project/migrate_process_xml).
|
||
|
||
## Miscellaneous
|
||
|
||
- [Feeds Migrate](https://www.drupal.org/project/feeds_migrate): it aims to provide a user interface similar to the one from Drupal 7's [Feeds module](https://www.drupal.org/project/feeds), but working on top of Drupal 8's Migrate API.
|
||
- [Migrate Override](https://www.drupal.org/project/migrate_override): allows flagging fields in a content entity so they can be manually changed by side editors without being overridden in a subsequent migration.
|
||
- [Migrate Status](https://www.drupal.org/project/migrate_status): checks if migrations are currently running.
|
||
- [Migrate QA](https://www.drupal.org/project/migrate_qa): provides tools for validating content migrations. See [this presentation](https://events.drupal.org/seattle2019/sessions/introducing-the-migrate-qa-module) for more details.
|
||
|
||
And [many, many more modules](https://www.drupal.org/project/project_module?f[3]=drupal_core%3A7234&f[4]=sm_field_project_type%3Afull&text=migrate&solrsort=score+desc)!
|
||
|
||
What did you learn in today's blog post? Did you find a new module that could be useful in current or future projects? Did we miss a module that has been very useful to you? Share your answers in the comments. Also, I would be grateful if you shared this blog post with others. |