Magento 2 Migrations

Contents

1. Introduction
2. Magento 1.x Migrations
3. Magento 2.x Migrations
4. Edmonds Commerce Migrations

Introduction

Database Migrations

See also: https://en.wikipedia.org/wiki/Schema_migration

Database migrations are a way of setting database state pragmatically, it is used in various frameworks, platforms and ORMs like Laravel’s Eloquent, Symfony’s Doctrine among many.

There are many different use cases for migrations during development, they can be used to reduce the technical debt to a database during development allowing it to be a decoupled part of the project. Internally we use migrations for newer projects to bootstrap different parts of a Magento store before it is launched, like CMS pages, blocks and configuration of the store itself.

This has the benefit of having a new store set up become much easier to manage, all the content is present and the only steps remaining for launch is to update DNS to the new platform.

Magento 1’s migrations

In Magento 1.x there is a system where by all changes in database structure and content is done incrementally using version numbering and a database table to track the status of database in comparison to a module’s version.

Install, Upgrades and Versioning

There are two sides to Magento setup and migration scripts, structure (known as sql internally) and data. When Magento encounters a new or updated module it will check if there are any changes to the database structure or contents by searching for a corresponding script in the sql and data directories. The version number of the module triggers this when it is greater than the version number that Magento has stored, Magento will then automatically trigger the structure change and data updates in turn.

Application Flow

Here is a detailed breakdown of how Magento 1 performs updates of the database

Structure

Core Mage Catalog Setup Structure (Snippet)

.
├── data
│   └── catalog_setup
│       ├── data-install-1.6.0.0.php
│       ├── data-upgrade-1.6.0.0.12-1.6.0.0.13.php
│       ├── data-upgrade-1.6.0.0.13-1.6.0.0.14.php
│       ├── data-upgrade-1.6.0.0.4-1.6.0.0.5.php
│       └── data-upgrade-1.6.0.0.8-1.6.0.0.9.php
├── etc
│   ├── config.xml
└── sql
    └── catalog_setup
        ├── install-1.6.0.0.php
        ├── mysql4-data-upgrade-0.7.57-0.7.58.php
        ├── mysql4-data-upgrade-0.7.63-0.7.64.php
        ├── mysql4-data-upgrade-1.4.0.0.28-1.4.0.0.29.php
        ├── mysql4-data-upgrade-1.4.0.0.42-1.4.0.0.43.php
        ├── mysql4-install-0.7.0.php
        ├── mysql4-install-1.4.0.0.0.php
        ├── mysql4-upgrade-0.6.40-0.7.0.php
        ├── mysql4-upgrade-0.7.0-0.7.1.php
        ├── mysql4-upgrade-0.7.1-0.7.2.php
        ├── mysql4-upgrade-0.7.11-0.7.12.php
        ├── mysql4-upgrade-0.7.12-0.7.13.php
        ├── mysql4-upgrade-0.7.13-0.7.14.php
        ├── mysql4-upgrade-0.7.14-0.7.15.php
        ├── mysql4-upgrade-0.7.15-0.7.16.php
        ├── mysql4-upgrade-0.7.16-0.7.17.php
        ├── mysql4-upgrade-0.7.17-0.7.18.php
        ├── mysql4-upgrade-0.7.18-0.7.19.php
        ├── mysql4-upgrade-0.7.19-0.7.20.php
        ├── mysql4-upgrade-0.7.20-0.7.21.php
        ├── mysql4-upgrade-0.7.2-0.7.3.php
        ├── upgrade-1.6.0.0.10-1.6.0.0.11.php
        ├── upgrade-1.6.0.0.11-1.6.0.0.12.php
        ├── upgrade-1.6.0.0.1-1.6.0.0.2.php
        ├── upgrade-1.6.0.0.14-1.6.0.0.15.php
        ├── upgrade-1.6.0.0.15-1.6.0.0.18.php
        ├── upgrade-1.6.0.0-1.6.0.0.1.php
        ├── upgrade-1.6.0.0.18-1.6.0.0.19.php
        ├── upgrade-1.6.0.0.19.1.1-1.6.0.0.19.1.2.php
        ├── upgrade-1.6.0.0.2-1.6.0.0.3.php
        ├── upgrade-1.6.0.0.3-1.6.0.0.4.php
        ├── upgrade-1.6.0.0.4-1.6.0.0.5.php
        ├── upgrade-1.6.0.0.5-1.6.0.0.6.php
        ├── upgrade-1.6.0.0.6-1.6.0.0.7.php
        ├── upgrade-1.6.0.0.7-1.6.0.0.8.php
        └── upgrade-1.6.0.0.9-1.6.0.0.10.php

Configuring Structure Changes

        <!-- app/code/core/Mage/Catalog/etc/config.xml -->
<config>
    <global>
        ...
        <resources>
            <catalog_setup>
                <setup>
                    <module>Mage_Catalog</module>
                    <class>Mage_Catalog_Model_Resource_Setup</class>
                </setup>
            </catalog_setup>
        </resources>
        ...
    </global>
</config>

Examples

Below are a few examples from Magento core modules that the data and structure of the database

Catalog Data Snippet Examples

// Create Root Catalog Node
Mage::getModel('catalog/category')
    ->load(1)
    ->setId(1)
    ->setStoreId(0)
    ->setPath(1)
    ->setLevel(0)
    ->setPosition(0)
    ->setChildrenCount(0)
    ->setName('Root Catalog')
    ->setInitialSetupFlag(true)
    ->save();

//Add Category Attribute Groups
$groups = array(
    'display'   => array(
        'name'  => 'Display Settings',
        'sort'  => 20,
        'id'    => null
    ),
    'design'    => array(
        'name'  => 'Custom Design',
        'sort'  => 30,
        'id'    => null
    )
);

foreach ($groups as $k => $groupProp) {
    $installer->addAttributeGroup($entityTypeId, $attributeSetId, $groupProp['name'], $groupProp['sort']);
    $groups[$k]['id'] = $installer->getAttributeGroupId($entityTypeId, $attributeSetId, $groupProp['name']);
}

Reviews Setup Snippets

/**
 * Create table 'review/review_entity'
 */
$table = $installer->getConnection()
    ->newTable($installer->getTable('review/review_entity'))
    ->addColumn('entity_id', Varien_Db_Ddl_Table::TYPE_SMALLINT, null, array(
        'identity'  => true,
        'unsigned'  => true,
        'nullable'  => false,
        'primary'   => true,
        ), 'Review entity id')
    ->addColumn('entity_code', Varien_Db_Ddl_Table::TYPE_TEXT, 32, array(
        'nullable'  => false
        ), 'Review entity code')
    ->setComment('Review entities');
$installer->getConnection()->createTable($table);

/**
 * Create table 'review/review_status'
 */
$table = $installer->getConnection()
    ->newTable($installer->getTable('review/review_status'))
    ->addColumn('status_id', Varien_Db_Ddl_Table::TYPE_SMALLINT, null, array(
        'identity'  => true,
        'unsigned'  => true,
        'nullable'  => false,
        'primary'   => true,
        ), 'Status id')
    ->addColumn('status_code', Varien_Db_Ddl_Table::TYPE_TEXT, 32, array(
        'nullable'  => false,
        ), 'Status code')
    ->setComment('Review statuses');
$installer->getConnection()->createTable($table);

Workflow

As mentioned in the introduction, we use migrations to bootstrap new stores (or existing stores) from install all the way through to being transactional. In the case of Magento, we rarely use the SQL install scripts and tend to use data scripts (as that is what we are working with).

Quite often we find that we are repeating ourselves when generating different content, like product attributes and over time we created a small module that standardises the creation of these elements. We aptly named the new module “migrations” and from there we have added more functionality over time to speed up the creation of new sites and passing this benefit on to clients.

Magento 2 Migrations

Differences to Magento 1

In Magento 2 there is a big shift in workflow, there are no version numbers in file names and the number of setup scripts does not increase as the module matures but the size of the scripts will do. This is one of the many stumbling blocks M1 developers will encounter when moving to Magento 2.

There is no concrete workflow on how setup scripts have to be done as there was in Magento 1, this can be confusing at first but allows a developer to define how they would like migrations to be run.

Idempotent Setup

One concept that prevails from this new approach is that migrations can now be idempotent out of the box.

Wikipedia: https://en.wikipedia.org/wiki/Idempotence > Idempotence is the property of certain operations in mathematics and computer science, that can be applied multiple times without changing the result beyond the initial application.

This approach will emphasize that checks on the current database state has to be checked before changes are made, examples include checking that we do not recreate an attribute or other element that does not exist and visa versa when removing data or modify existing tables Developers will need to be more cautious and test different variations of the setup script, it would also be a good idea to shard large setup files in to smaller categorised chunks for ease of use when making further changes.

Abstraction and Dependency Injection

In Magento 2 dependency injection and ObjectManager has replaced the much loved super object Mage meaning we now have more control over what is instantiated a class constructor. This has also had an impact on how we approach setup scripts and migrations in that we can now abstract away reusable chunks of code and tailor them to our module.

An example of this would be needing to create attributes in different ways, we could create a new class and inject the setup class used (as shown in the example above) or we could extend the setup class to add our new logic

From here we could allow the use of file based system to load content like static blocks/pages and allow faster iteration of content (instead of using the Magento admin which takes donkey’s years). An example of what this could look like:

./app/code/vendor/module
  ├── etc
  ├── registration.php
  ├── Setup
  │   ├── CategorySetup.php
  │   ├── InstallData.php
  │   ├── InstallSchema.php
  │   ├── UpgradeData.php
  │   ├── UpgradeSchema.php
  │   ├── UpgradeSchema.php
  │   └── Cms
  │       ├── Blocks
  │       │   ├── Contact.phtml
  │       │   └── Delivery.phtml
  │       │    ...
  │       ├── Pages
  │       │   ├── Home.phtml
  │       │   └── About.phtml
  │       │    ...  

Structure and Examples

Magento 2 Catalog

./vendor/magento/module-catalog
  ├── etc
  │   ├── config.xml
  │   ├── di.xml
  │   ├── module.xml
  ├── registration.php
  ├── Setup
  │   ├── CategorySetup.php
  │   ├── InstallData.php
  │   ├── InstallSchema.php
  │   ├── UpgradeData.php
  │   └── UpgradeSchema.php

$categorySetup->installEntities();
        // Create Root Catalog Node
        $categorySetup->createCategory()
            ->load($rootCategoryId)
            ->setId($rootCategoryId)
            ->setStoreId(0)
            ->setPath($rootCategoryId)
            ->setLevel(0)
            ->setPosition(0)
            ->setChildrenCount(0)
            ->setName('Root Catalog')
            ->setInitialSetupFlag(true)
            ->save();

        // Create Default Catalog Node
        $category = $categorySetup->createCategory();
        $category->load($defaultCategoryId)
            ->setId($defaultCategoryId)
            ->setStoreId(0)
            ->setPath($rootCategoryId . '/' . $defaultCategoryId)
            ->setName('Default Category')
            ->setDisplayMode('PRODUCTS')
            ->setIsActive(1)
            ->setLevel(1)
            ->setInitialSetupFlag(true)
            ->setAttributeSetId($category->getDefaultAttributeSetId())
            ->save();

EC Database Migrations Module

As mentioned previously we have a database migrations module internally that we tend to reuse and have started to migrate to Magento 2. The purpose of the module is to bootstrap tasks that are repeated across Magento 2 projects and allow for changes to be idempotent at the same time.

The module is still very young and is still under development but is shaping up to be a useful tool. There are still active issues that are being tackled and we are embracing the open source approach to the development of the module to improve the development experience of every one involved.

Install Guide

From the root of your Magento 2 project, install the composer dependency.

composer require edmondscommerce/magento2-migrations:dev-master@dev

Examples

From with in your Magento 2 module’s setup/upgrade scripts you can now inject one of the many different interfaces available from the migrations module. For the time being we have split larger parts of Magento’s infrastructure in to smaller interfaces for ease of management.

Lets take a look at changing an arbitrary config change

class InstallData implements InstallDataInterface {

    public function __construct(EavSetupFactory $eavSetupFactory, \EdmondsCommerce\Migrations\Contracts\Config\GeneralContract $generalConfig)
    {
        $this->generalConfig = $generalConfig;   
    }
    
    public function install(...)
    {
        $setup->startSetup();
        
        ...
        
        $this->generalConfig->setStoreName('My Magento 2 Store');
           
        ...
            
        $setup->endSetup();
    }
}

Conclusion

With the use of database migrations we can remove the complexity of keeping environments in sync during development and reduce the value of the database when moving between environments. From this we can always explicitly declare what should and should not be present in the database on any environment whether that be production or active development.