A visitor to the new Agaric site can see all content that was on the old agaric.com #82

Closed
opened 2018-09-06 18:48:23 +00:00 by mlncn · 39 comments
mlncn commented 2018-09-06 18:48:23 +00:00 (Migrated from gitlab.com)

@gnuget has done all the migration work. To test locally:

To get all the data to migrate:

mkdir web/sites/default/import
mkdir sql
scp -r root@sojourner.mayfirst.org:/home/members/agariclabs/sites/agaric.com/web/sites/default/files/* web/sites/default/import
scp root@sojourner.mayfirst.org:/home/members/agariclabs/sites/agaric.com/2018-09-06.sql sql/
vagrant ssh
sudo mysql drupal7 < ../sql/2018-09-06.sql

To run the migration fully fresh:

drush -y sql-drop && drush -y sql-sync @test @self && drush cr && drush -y cim
drush migrate-import --group=agaric --execute-dependencies

Work Remaining

@gnuget has done all the migration work. To test locally: To get all the data to migrate: ``` mkdir web/sites/default/import mkdir sql scp -r root@sojourner.mayfirst.org:/home/members/agariclabs/sites/agaric.com/web/sites/default/files/* web/sites/default/import scp root@sojourner.mayfirst.org:/home/members/agariclabs/sites/agaric.com/2018-09-06.sql sql/ vagrant ssh sudo mysql drupal7 < ../sql/2018-09-06.sql ``` To run the migration fully fresh: ``` drush -y sql-drop && drush -y sql-sync @test @self && drush cr && drush -y cim drush migrate-import --group=agaric --execute-dependencies ``` ## Work Remaining * [x] Migrate author into `field_authors` * [x] Migrate summary into `field_summary` * [x] Fix broken inline images, example: http://agaric-com.test/blog/daily-business-operations-using-free-software * [x] Migrate `field_tags` into `field_tags`
mlncn commented 2018-09-06 18:51:22 +00:00 (Migrated from gitlab.com)

changed the description

changed the description
mlncn commented 2018-09-06 18:52:44 +00:00 (Migrated from gitlab.com)

changed title from A visitor to the new Agaric site can see all to A visitor to the new Agaric site can see all{+ content that was on the old agaric.com+}

changed title from **A visitor to the new Agaric site can see all** to **A visitor to the new Agaric site can see all{+ content that was on the old agaric.com+}**
mlncn commented 2018-09-06 19:01:42 +00:00 (Migrated from gitlab.com)

marked this issue as related to #30

marked this issue as related to #30
mlncn commented 2018-09-06 23:40:05 +00:00 (Migrated from gitlab.com)

changed the description

changed the description
cedewey commented 2018-09-26 16:04:03 +00:00 (Migrated from gitlab.com)

assigned to @cedewey

assigned to @cedewey
cedewey commented 2018-09-26 16:04:16 +00:00 (Migrated from gitlab.com)

changed milestone to %3

changed milestone to %3
cedewey commented 2018-10-03 15:23:15 +00:00 (Migrated from gitlab.com)

assigned to @wolcen and unassigned @cedewey

assigned to @wolcen and unassigned @cedewey
cedewey commented 2018-10-03 15:25:06 +00:00 (Migrated from gitlab.com)

@wolcen I don't have root access to sojourner. I looked in the MayFirst control panel and didn't see a way to get access. I also checked Keepass.

@wolcen I don't have root access to sojourner. I looked in the MayFirst control panel and didn't see a way to get access. I also checked Keepass.
wolcen commented 2018-10-03 16:05:29 +00:00 (Migrated from gitlab.com)

@cedewey Yeah - I don't know any root pwd's, only connect by key. Yours was added.

@cedewey Yeah - I don't know any root pwd's, only connect by key. Yours was added.
cedewey commented 2018-10-03 18:11:26 +00:00 (Migrated from gitlab.com)

@mlncn @gnuget ,

When I run scp -r root@sojourner.mayfirst.org:/home/members/agariclabs/sites/agaric.com/web/sites/default/files/* web/sites/default/import

I get permission denied messages on each action. Do you know what I need to do to bypass this?

For example,

web/sites/default/import/12412508015_1e5340004b_k.jpg: Permission denied
web/sites/default/import/18-04-29-23-12-27.png: Permission denied
web/sites/default/import/18-05-02-21-17-39.png: Permission denied
web/sites/default/import/18-05-02-21-20-39.png: Permission denied
web/sites/default/import/20160408_095319.jpg: Permission denied
web/sites/default/import/20160410_094242.jpg: Permission denied
web/sites/default/import/20160410_102039.jpg: Permission denied
web/sites/default/import/20160411_162313.jpg: Permission denied
web/sites/default/import/20160411_162437.jpg: Permission denied
web/sites/default/import/20160411_173544.jpg: Permission denied
web/sites/default/import/20161011_173401.jpg: Permission denied
web/sites/default/import/20161012_132859.jpg: Permission denied
web/sites/default/import/20161012_155652.jpg: Permission denied
@mlncn @gnuget , When I run `scp -r root@sojourner.mayfirst.org:/home/members/agariclabs/sites/agaric.com/web/sites/default/files/* web/sites/default/import ` I get permission denied messages on each action. Do you know what I need to do to bypass this? For example, ``` web/sites/default/import/12412508015_1e5340004b_k.jpg: Permission denied web/sites/default/import/18-04-29-23-12-27.png: Permission denied web/sites/default/import/18-05-02-21-17-39.png: Permission denied web/sites/default/import/18-05-02-21-20-39.png: Permission denied web/sites/default/import/20160408_095319.jpg: Permission denied web/sites/default/import/20160410_094242.jpg: Permission denied web/sites/default/import/20160410_102039.jpg: Permission denied web/sites/default/import/20160411_162313.jpg: Permission denied web/sites/default/import/20160411_162437.jpg: Permission denied web/sites/default/import/20160411_173544.jpg: Permission denied web/sites/default/import/20161011_173401.jpg: Permission denied web/sites/default/import/20161012_132859.jpg: Permission denied web/sites/default/import/20161012_155652.jpg: Permission denied ```
cedewey commented 2018-10-03 18:11:36 +00:00 (Migrated from gitlab.com)

assigned to @mlncn and @gnuget and unassigned @wolcen

assigned to @mlncn and @gnuget and unassigned @wolcen
gnuget commented 2018-10-03 18:40:17 +00:00 (Migrated from gitlab.com)

My guess is that you haven't permissions on your web/sites/default/import folder, drupal usually remove the permission of /web/sites/ after to be installed.

Try with:

chmod -R 755 /web/sites

And then try again to execute the scp command.

My guess is that you haven't permissions on your `web/sites/default/import` folder, drupal usually remove the permission of `/web/sites/` after to be installed. Try with: `chmod -R 755 /web/sites` And then try again to execute the scp command.
cedewey commented 2018-10-04 16:27:50 +00:00 (Migrated from gitlab.com)

mentioned in issue #90

mentioned in issue #90
cedewey commented 2018-10-17 20:06:49 +00:00 (Migrated from gitlab.com)

@gnuget I'm trying to create a second database called drupal7 to test your migration.

I tried creating another database through Adminer, by logging in as user drupal. According to the documentation, this is the root user. However, when I tried creating a database, I received this error -

Access denied for user 'drupal'@'%' to database 'drupal7'

I tried this from the command line within the vm and got the same access error. Do you know how I can create another database in my local environment?

@gnuget I'm trying to create a second database called `drupal7` to test your migration. I tried creating another database through Adminer, by logging in as user `drupal`. [According to the documentation](http://docs.drupalvm.com/en/latest/configurations/databases-mysql/#connect-using-adminer), this is the root user. However, when I tried creating a database, I received this error - `Access denied for user 'drupal'@'%' to database 'drupal7'` I tried this from the command line within the vm and got the same access error. Do you know how I can create another database in my local environment?
cedewey commented 2018-10-17 20:06:53 +00:00 (Migrated from gitlab.com)

unassigned @mlncn

unassigned @mlncn
gnuget commented 2018-10-17 20:09:17 +00:00 (Migrated from gitlab.com)

You need to create the database as root and the only way to access the root account in mysql is using the root account.

So this should work:

$ sudo su -
# mysql -u root -proot
mysql>  create database drupal7;
mysql> GRANT ALL PRIVILEGES ON drupal7.* TO 'drupal';
You need to create the database as `root` and the only way to access the `root` account in mysql is using the `root` account. So this should work: ``` $ sudo su - # mysql -u root -proot mysql> create database drupal7; mysql> GRANT ALL PRIVILEGES ON drupal7.* TO 'drupal'; ```
cedewey commented 2018-10-23 19:14:43 +00:00 (Migrated from gitlab.com)

changed the description

changed the description
cedewey commented 2018-10-23 19:15:14 +00:00 (Migrated from gitlab.com)

@gnuget ok I've tested out the migration. It's looking good. There are a few issues I found. I've added them to the issue description.

@gnuget ok I've tested out the migration. It's looking good. There are a few issues I found. I've added them to the issue description.
gnuget commented 2018-10-29 02:12:04 +00:00 (Migrated from gitlab.com)

Migrate author into field_authors

Does this means that I need to create an Author node per each user that we have in the old Drupal site?

I mapped the nodes with the correct author already (the authored by field) not sure if we want to preserve the field_authors fields in our case.

Fix broken inline images

This is already workign, but you need to download the fiels from the old sites and put them at: sites/default/import/

You will need to rollback the files migration, put the files in place and then run the migration again.

drush migrate-rollback agaric_file
cp -r agaric-old/web/sites/default/files agaric-com/sites/default/import
drush mim agaric_file
> Migrate author into `field_authors` Does this means that I need to create an Author node per each user that we have in the old Drupal site? I mapped the nodes with the correct author already (the `authored by` field) not sure if we want to preserve the field_authors fields in our case. > Fix broken inline images This is already workign, but you need to download the fiels from the old sites and put them at: `sites/default/import/` You will need to rollback the files migration, put the files in place and then run the migration again. ``` drush migrate-rollback agaric_file cp -r agaric-old/web/sites/default/files agaric-com/sites/default/import drush mim agaric_file ```
gnuget commented 2018-10-29 02:12:11 +00:00 (Migrated from gitlab.com)

marked the checklist item Migrate summary into field_summary as completed

marked the checklist item **Migrate summary into `field_summary`** as completed
mlncn commented 2018-10-29 13:21:49 +00:00 (Migrated from gitlab.com)

@gnuget The idea with Field Authors is that, by referencing a person content type rather than a user, it makes it possible for us to do things like have a guest post without creating a user account or, much more likely for us, credit two or more people for a single post.

Since that is also Drutopia standard functionality and having a solution for migrating Drupal blogs to Drutopia blogs is very much desired, and it might still save us time ourselves... could you give it a try? :-)

@gnuget The idea with Field Authors is that, by referencing a person content type rather than a user, it makes it possible for us to do things like have a guest post without creating a user account or, much more likely for us, credit two or more people for a single post. Since that is also Drutopia standard functionality and having a solution for migrating Drupal blogs to Drutopia blogs is very much desired, and it might still save us time ourselves... could you give it a try? :-)
gnuget commented 2018-10-30 15:46:27 +00:00 (Migrated from gitlab.com)

marked the checklist item Migrate author into field_authors as completed

marked the checklist item **Migrate author into `field_authors`** as completed
gnuget commented 2018-10-30 15:46:28 +00:00 (Migrated from gitlab.com)

marked the checklist item Fix broken inline images, example: http://agaric-com.test/blog/daily-business-operations-using-free-software as completed

marked the checklist item **Fix broken inline images, example: http://agaric-com.test/blog/daily-business-operations-using-free-software** as completed
gnuget commented 2018-10-30 15:46:30 +00:00 (Migrated from gitlab.com)

marked the checklist item Migrate field_tags into field_tags as completed

marked the checklist item **Migrate `field_tags` into `field_tags`** as completed
gnuget commented 2018-10-30 15:46:35 +00:00 (Migrated from gitlab.com)

marked the checklist item Migrate field_tags into field_tags as incomplete

marked the checklist item **Migrate `field_tags` into `field_tags`** as incomplete
gnuget commented 2018-10-30 18:34:15 +00:00 (Migrated from gitlab.com)

marked the checklist item Migrate field_tags into field_tags as completed

marked the checklist item **Migrate `field_tags` into `field_tags`** as completed
gnuget commented 2018-10-30 18:35:08 +00:00 (Migrated from gitlab.com)

I created this merge request https://gitlab.com/agaric/sites/agaric-com/merge_requests/28/diffs

Which address all the issues found by @cedewey

I created this merge request https://gitlab.com/agaric/sites/agaric-com/merge_requests/28/diffs Which address all the issues found by @cedewey
cedewey commented 2018-10-31 20:04:46 +00:00 (Migrated from gitlab.com)

@gnuget I ran the migration by doing the following:

drush -y sql-drop && drush -y sql-sync @test @self && drush cr && drush -y cim
drush migrate-import --group=agaric --execute-dependencies

I needed to delete the vocabularies Services, Technologies and License for it to run successfully.

After running the migration, there is only one Blog Post - New Agaric site launches

and none of the users migrated over as People nodes.

@gnuget I ran the migration by doing the following: ``` drush -y sql-drop && drush -y sql-sync @test @self && drush cr && drush -y cim drush migrate-import --group=agaric --execute-dependencies ``` I needed to delete the vocabularies Services, Technologies and License for it to run successfully. After running the migration, there is only one Blog Post - New Agaric site launches and none of the users migrated over as People nodes.
gnuget commented 2018-10-31 20:59:06 +00:00 (Migrated from gitlab.com)

@cedewey

If you execute:

drush ms --group="agaric"

What output do you get?

@cedewey If you execute: ``` drush ms --group="agaric" ``` What output do you get?
cedewey commented 2018-10-31 21:08:19 +00:00 (Migrated from gitlab.com)
 ----------------- ------------------------- -------- ------- ---------- ------------- ---------------
  Group             Migration ID              Status   Total   Imported   Unprocessed   Last Imported
 ----------------- ------------------------- -------- ------- ---------- ------------- ---------------
  agaric (agaric)   agaric_user               Idle     15      0          15
  agaric (agaric)   agaric_file               Idle     229     0          229
  agaric (agaric)   agaric_blog_paragraph     Idle     88      0          88
  agaric (agaric)   agaric_people_paragraph   Idle     15      0          15
  agaric (agaric)   agaric_people             Idle     15      0          15
  agaric (agaric)   agaric_blog               Idle     88      0          88
  agaric (agaric)   agaric_page_paragraph     Idle     11      0          11
  agaric (agaric)   agaric_page               Idle     11      0          11
  agaric (agaric)   agaric_comment            Idle     126     0          126
 ----------------- ------------------------- -------- ------- ---------- ------------- ---------------
``` ----------------- ------------------------- -------- ------- ---------- ------------- --------------- Group Migration ID Status Total Imported Unprocessed Last Imported ----------------- ------------------------- -------- ------- ---------- ------------- --------------- agaric (agaric) agaric_user Idle 15 0 15 agaric (agaric) agaric_file Idle 229 0 229 agaric (agaric) agaric_blog_paragraph Idle 88 0 88 agaric (agaric) agaric_people_paragraph Idle 15 0 15 agaric (agaric) agaric_people Idle 15 0 15 agaric (agaric) agaric_blog Idle 88 0 88 agaric (agaric) agaric_page_paragraph Idle 11 0 11 agaric (agaric) agaric_page Idle 11 0 11 agaric (agaric) agaric_comment Idle 126 0 126 ----------------- ------------------------- -------- ------- ---------- ------------- --------------- ```
cedewey commented 2018-10-31 21:19:29 +00:00 (Migrated from gitlab.com)

I've migrated the users and content successfully.

Remaining issues:

  • Some inline images on blog posts still don't show up. I did get an error when migrating the blog posts,
LogicException while trying to render item entity:node/37:en with view mode full for search index Drutopia content index: Could not determine image width for 'public://howiviwipothabrodridishusitaslewukutiteswechugeliclerelomocromimupracicefrorocripraswogakophidivitrocechophobipregidadroriwrokukimudefruwedrecekichinacrimaphotromokolewimoueparusiclibathinituclavefrechewekutuvistudeswotarotecritricle' using image style with ID: max_1300x1300. This image style can not be used for a responsive image style mapping using the 'sizes' attribute. in _responsive_image_build_source_attributes() (line 432 of /var/www/drutopia/web/core/modules/responsive_image/responsive_image.module).
I've migrated the users and content successfully. Remaining issues: * Some inline images on blog posts still don't show up. I did get an error when migrating the blog posts, ``` LogicException while trying to render item entity:node/37:en with view mode full for search index Drutopia content index: Could not determine image width for 'public://howiviwipothabrodridishusitaslewukutiteswechugeliclerelomocromimupracicefrorocripraswogakophidivitrocechophobipregidadroriwrokukimudefruwedrecekichinacrimaphotromokolewimoueparusiclibathinituclavefrechewekutuvistudeswotarotecritricle' using image style with ID: max_1300x1300. This image style can not be used for a responsive image style mapping using the 'sizes' attribute. in _responsive_image_build_source_attributes() (line 432 of /var/www/drutopia/web/core/modules/responsive_image/responsive_image.module). ```
cedewey commented 2018-10-31 21:23:45 +00:00 (Migrated from gitlab.com)

Also, when I visit a person detail page such as /people/david-valdez I get an error,

LogicException: Could not determine image width for 'public://pafrecrislunogisofrugestiluthehotujobaspacubiluslaprithajapholasegeslawukiwrutucrigifrecilivibutheuethedrikiswichuchuchustulucripacrumewitreuijisho' using image style with ID: max_1300x1300. This image style can not be used for a responsive image style mapping using the 'sizes' attribute. in _responsive_image_build_source_attributes() (line 432 of /var/www/drutopia/web/core/modules/responsive_image/responsive_image.module).
Also, when I visit a person detail page such as /people/david-valdez I get an error, ``` LogicException: Could not determine image width for 'public://pafrecrislunogisofrugestiluthehotujobaspacubiluslaprithajapholasegeslawukiwrutucrigifrecilivibutheuethedrikiswichuchuchustulucripacrumewitreuijisho' using image style with ID: max_1300x1300. This image style can not be used for a responsive image style mapping using the 'sizes' attribute. in _responsive_image_build_source_attributes() (line 432 of /var/www/drutopia/web/core/modules/responsive_image/responsive_image.module). ```
gnuget commented 2018-10-31 21:36:25 +00:00 (Migrated from gitlab.com)

Hi @cedewey

Did you copy the files from the old site to the new site? You need to put the folder at sites/default/import folder.

Also you might need go run the agaric_file migration.

Hi @cedewey Did you copy the files from the old site to the new site? You need to put the folder at `sites/default/import` folder. Also you might need go run the agaric_file migration.
gnuget commented 2018-10-31 22:28:16 +00:00 (Migrated from gitlab.com)

I tried to start fresh and I got this error when executing vagrant provision

npm ERR! path /var/www/drutopia/web/themes/custom/agarica/patternlibrary/node_modules/grunt-cli/node_modules/atob/bin/atob.js", "npm ERR! code ENOENT", "npm ERR! errno -2", "npm ERR! syscall chmod", "npm ERR! enoent ENOENT: no such file or directory, chmod '/var/www/drutopia/web/themes/custom/agarica/patternlibrary/node_modules/grunt-cli/node_modules/atob/bin/atob.js'", "npm ERR! enoent This is related to npm not being able to find a file.", "npm ERR! enoent ", "", "npm ERR! A complete log of this run can be found in:", "npm ERR!     /root/.npm/_logs/2018-10-31T22_25_52_912Z-debug.log"], "stdout": "", "stdout_lines": []}
I tried to start fresh and I got this error when executing `vagrant provision` ``` npm ERR! path /var/www/drutopia/web/themes/custom/agarica/patternlibrary/node_modules/grunt-cli/node_modules/atob/bin/atob.js", "npm ERR! code ENOENT", "npm ERR! errno -2", "npm ERR! syscall chmod", "npm ERR! enoent ENOENT: no such file or directory, chmod '/var/www/drutopia/web/themes/custom/agarica/patternlibrary/node_modules/grunt-cli/node_modules/atob/bin/atob.js'", "npm ERR! enoent This is related to npm not being able to find a file.", "npm ERR! enoent ", "", "npm ERR! A complete log of this run can be found in:", "npm ERR! /root/.npm/_logs/2018-10-31T22_25_52_912Z-debug.log"], "stdout": "", "stdout_lines": []} ```
gnuget commented 2018-10-31 22:28:46 +00:00 (Migrated from gitlab.com)

I will try to fix the problem and run the migration

I will try to fix the problem and run the migration
gnuget commented 2018-11-01 17:03:02 +00:00 (Migrated from gitlab.com)

I found out what the problem is, but not sure how to fix it (I haven't experience with NPM)

To make it work on my local I commented these lines (in the 10-drupal.yml file):

- name: Install theme build dependencies
  npm:
    path: "{{ drupal_core_path }}/themes/custom/agarica/patternlibrary"
    state: present

@wolcen do you know what would be the correct way to fix it?

I found out what the problem is, but not sure how to fix it (I haven't experience with NPM) To make it work on my local I commented these lines (in the 10-drupal.yml file): ``` - name: Install theme build dependencies npm: path: "{{ drupal_core_path }}/themes/custom/agarica/patternlibrary" state: present ``` @wolcen do you know what would be the correct way to fix it?
wolcen commented 2018-11-02 18:01:23 +00:00 (Migrated from gitlab.com)

@gnuget just to document what we were discussing here: it appeared there were filesystem and permission issues. The node_packages and package-lock.json live in the NFS file system, so a vagrant destroy/up/provision would not reset an issue in them. We were last going to attempt a wipe of those items to re-run npm install clean - i.e.: cd /var/www/drutopia/web/themes/custom/agarica/patternlibrary && rm -rf node_packages && rm package-lock.json && npm install

@gnuget just to document what we were discussing here: it appeared there were filesystem and permission issues. The node_packages and package-lock.json live in the NFS file system, so a vagrant destroy/up/provision would not reset an issue in them. We were last going to attempt a wipe of those items to re-run npm install clean - i.e.: `cd /var/www/drutopia/web/themes/custom/agarica/patternlibrary && rm -rf node_packages && rm package-lock.json && npm install`
cedewey commented 2018-11-28 19:30:33 +00:00 (Migrated from gitlab.com)

changed milestone to %7

changed milestone to %7
cedewey commented 2018-11-28 23:27:37 +00:00 (Migrated from gitlab.com)

closed

closed
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: agaric/agaric-coop#82
No description provided.