StoreVault to NetApp – Data Migration and Overcoming the Adversity of “Unsupported”

As I’d mentioned in my previous post about a failed attempt to get Snapdrive, Windows 2003, and a modern NetApp SAN all talking to each other, my company was attempting to decommission an old Storevault S550 SAN, which was serving corporate CIFS shares to the whole company, as well as some critical Exchange 2003 mailboxes.

Today I’d like to go deeper, and talk about how to actually make this migration possible. The team talked amongst themselves about how to move this data; there was concern — “we’ll spend a week just running xcopy to get the data onto a file server!” “How do we handle file changes during that time?”

So, the Storevault SAN is basically a rebranded NetApp, running a customized version of OnTAP. Some native NetApp features have been preserved, while others have been removed or possibly crippled. Knowing this, immediately I suggested NetApp’s SnapMirror feature, if working as expected, could be used to shuttle all the data off Storevault and onto the NetApp FAS2240 which would replace it.

Problem #1:

OnCommand v3.1 can't talk to the Storevault.

OnCommand v3.1 can’t talk to the Storevault.

OnCommand System Manager 3.1, which I’m using to administer my FAS2240, complains that it can’t talk to the old version of “OnTAP” that the Storevault runs. Attempting downgrades on a test machine didn’t work.

Problem #2:

FilerView for Storevault won't let me specify a SnapMirror destination that ISN'T itself.

FilerView for Storevault won’t let me specify a SnapMirror destination that ISN’T itself.

The StoreVault will let you use SnapMirror (if you’re licensed), however it won’t let me define a new destination OUTSIDE of the StoreVault (for instance, to a different device). I can only SnapMirror volumes within the SAN.

Problem #3: 

NetApp's compatibility matrix says SnapMirror IS possible... if you're running a newer version.

NetApp’s compatibility matrix says SnapMirror IS possible… if you’re running a newer version.

NetApp’s Volume SnapMirror InterOperability Matrix says if your source device is running 7.3.x, and your destination runs 8.2, SnapMirror IS possible — but our StoreVault runs “7.2.1S10R1”. So we’re technically going to create an unsupported situation here.

Problem #4:

The StoreVault is out of support, and NetApp’s Support portal does not provide firmware revision history for theiStoreVault line of products, which have been discontinued. A firmware upgrade is out of the question.

So, we’re stuck — right?

Nonsense, don’t give up so easily! What if we tried to set up SnapMirror anyway, quietly ignoring that the compatibility matrix makes no mention of a) StoreVault-branded OnTAP releases, or b) anything prior to 7.3.x, and we remove OnCommand from the mix here? SnapMirror relationships can be stood up manually behind the scenes, with a little more grunt-work required.

Short answer — IT WORKS! SnapMirror can sync volumes between a StoreVault running 7.2.1x and OnTAP 8.2 as long as you ignore OnCommand System Manager. Let’s see how.

This method relies on you getting cozy in the NetApp CLI, so grab putty and get connected to your SAN controllers, both source and destination.

I’m going to provide for you, my complete implementation steps; technical and verbose, but very thorough.

    1. On source STOREVAULT-SAN, enable Snapmirror
      1. Connect via SSH, run the command options snapmirror.enable on
      2. In UI, verify license for Snapmirror is installed
      3. Verify snapmirror host whitelisting is not enabled; options snapmirror.accessshould be empty
    2. On source STOREVAULT-SAN, retrieve list of volumes to mirror, and their sizes
      1. Via SSH, run vol statusfor list of volume names
      2. Our volumes: exch01_cluster_corp_pub1 150GB
        exch01_cluster_corp_tlogs_new 110GB
        exch01_cluster_mstdc 1GB
        exch01_cluster_smtpq 81GB
        exch_cluster_quorum 1GB
        exch_corp_system 300GB
        exch_corp_users 250GB
    3. On destination NetAPP FAS, create desired destination volume names, and size should be LARGER than source (0.1GB larger is fine): [you’ll see I added the prefix snapmirr_too, makes it easier to see them as a group when looking at volume list]
      1. snapmirr_L_CD_DragonEXCH_Corp_Pub_S1    150.1GB
        snapmirr_L_CD_DragonEXCH_Corp_Tlogs_S1    110.1GB
        snapmirr_L_CD_DragonEXCH_MSDTC_S1    1.1GB
        snapmirr_L_CD_DragonEXCH_SMTPQ_S1    81.1GB
        snapmirr_L_CD_DragonEXCH_Quorum_S1    1.1GB
        snapmirr_L_CD_DragonEXCH_Corp_System_S1    300.1GB
        snapmirr_L_CD_DragonEXCH_Corp_Users_S1    250.1GB
    4. Browse to admin share of destination SAN via Windows Explorer, load /etc/snapmirror.conf
      1. Add following configuration lines to bottom (WATCH THE LINE BREAKS; A NEW LINE IS A NEW ENTRY):
      2. STOREVAULT-SAN:exch01_cluster_corp_pub1 NETAPP-SAN:snapmirr_L_CD_DragonEXCH_Corp_Pub_S1 – 0 21 * *
        STOREVAULT-SAN:exch01_cluster_corp_tlogs_new NETAPP-SAN:snapmirr_L_CD_DragonEXCH_Corp_Tlogs_S1 – 0 21 * *
        STOREVAULT-SAN:exch01_cluster_mstdc NETAPP-SAN:snapmirr_L_CD_DragonEXCH_MSDTC_S1 – 0 21 * *
        STOREVAULT-SAN:exch01_cluster_smtpq NETAPP-SAN:snapmirr_L_CD_DragonEXCH_SMTPQ_S1 – 0 21 * *
        STOREVAULT-SAN:exch_cluster_quorum NETAPP-SAN:snapmirr_L_CD_DragonEXCH_Quorum_S1 – 0 21 * *
        STOREVAULT-SAN:exch_corp_system NETAPP-SAN:snapmirr_L_CD_DragonEXCH_Corp_System_S1 – 0 21 * *
        STOREVAULT-SAN:exch_corp_users NETAPP-SAN:snapmirr_L_CD_DragonEXCH_Corp_Users_S1 – 0 21 * *
      3. These parameters define NO throttle, and run on a schedule ONCE per day, at 21:00hrs; they can be modified as needed
    5. On destination, set status of every volume to Restricted— or else Snapmirror will be unable to initalize. This can be done via OnCommand System Manager.
    6. Refresh Snapmirror tab of OnCommand System Manager, verify new entries are listed. You will get an error about the STOREVAULT-SAN not being a managed system, this is OK just ignore this and work in CLI.
    7. Begin initialization process.
      1. Warning about initializing — for as long as you have multiple mirrors running their first initialization, you will likely not be able to view the VOLUMES tab in System Manager, due to some volumes being busy. UI will hang on “0 volumes loaded”. You can see the back-end effects of this by running vol status, and you’ll see some # of volumes reporting busy due to initialization.Plan accordingly (run overnight).
      2. From SSH console on destination NetApp SAN, run the following commands (just reference the destination/vol name)
        1. snapmirror initialize snapmirr_L_CD_DragonEXCH_Corp_Pub_S1
          snapmirror initialize snapmirr_L_CD_DragonEXCH_Corp_Tlogs_S1
          snapmirror initialize snapmirr_L_CD_DragonEXCH_MSDTC_S1
          snapmirror initialize snapmirr_L_CD_DragonEXCH_SMTPQ_S1
          snapmirror initialize snapmirr_L_CD_DragonEXCH_Quorum_S1
          snapmirror initialize snapmirr_L_CD_DragonEXCH_Corp_System_S1
          snapmirror initialize snapmirr_L_CD_DragonEXCH_Corp_Users_S1
      3. After all volumes have been initialized, they can be updated at will until ready for cutover
        1. snapmirror update snapmirr_L_CD_DragonEXCH_Corp_Pub_S1
          snapmirror update snapmirr_L_CD_DragonEXCH_Corp_Tlogs_S1
          snapmirror update snapmirr_L_CD_DragonEXCH_MSDTC_S1
          snapmirror update snapmirr_L_CD_DragonEXCH_SMTPQ_S1
          snapmirror update snapmirr_L_CD_DragonEXCH_Quorum_S1
          snapmirror update snapmirr_L_CD_DragonEXCH_Corp_System_S1
          snapmirror update snapmirr_L_CD_DragonEXCH_Corp_Users_S1
      4. Prepare to break the mirror
        1. If you’re mirroring LUNs, stop the applications using them, then detach the LUNs,so we don’t lose any data-in-flight. Make SnapMirror’s job as easy as possible.
        2. Run snapmirror updatefor each volume (as above in step 8A). I like to do this several times, to be paranoid/safe. My mirrors auto-updated at 21:00, and I began my cutover shortly thereafter, so you can see lag time is down. But I STILL updated manually 😀

          Snapmirror status

          Snapmirror status

    1. Quiesce all mirrors
      1. snapmirror quiesce snapmirr_L_CD_DragonEXCH_Corp_Pub_S1
        snapmirror quiesce snapmirr_L_CD_DragonEXCH_Corp_Tlogs_S1
        snapmirror quiesce snapmirr_L_CD_DragonEXCH_MSDTC_S1
        snapmirror quiesce snapmirr_L_CD_DragonEXCH_SMTPQ_S1
        snapmirror quiesce snapmirr_L_CD_DragonEXCH_Quorum_S1
        snapmirror quiesce snapmirr_L_CD_DragonEXCH_Corp_System_S1
        snapmirror quiesce snapmirr_L_CD_DragonEXCH_Corp_Users_S1
    2. Break all mirrors
      1. snapmirror break snapmirr_L_CD_DragonEXCH_Corp_Pub_S1
        snapmirror break snapmirr_L_CD_DragonEXCH_Corp_Tlogs_S1
        snapmirror break snapmirr_L_CD_DragonEXCH_MSDTC_S1
        snapmirror break snapmirr_L_CD_DragonEXCH_SMTPQ_S1
        snapmirror break snapmirr_L_CD_DragonEXCH_Quorum_S1
        snapmirror break snapmirr_L_CD_DragonEXCH_Corp_System_S1
        snapmirror break snapmirr_L_CD_DragonEXCH_Corp_Users_S1
    3. Offline the source volumes on STOREVAULT-SAN
    4. Set the destinatiaon volumes on NetApp to ONLINE (if they aren’t already in ONLINE state)
    5. Rename the destination volumes where appropriate (in my case, I will be removing the snapmirr_prefixes) (volumes can be renamed while online and serving data, no downtime)
    6. Browse to admin share of destination SAN, load /etc/snapmirror.conf
      1. Delete the recently added snapmirror line items carefully; we no longer need them
    7. Verify in OnCommand that all volumes have been thin-provisioned and have storage efficiency ENABLED if you so choose
    8. Work to reconfigure any servers referencing old STOREVAULT-SAN volumes (in our case, Exchange 2003 cluster had to be modified).

So, that’s 15 easy steps… SnapMirror synced all the data over automatically, block-by-block, ran a resync on a schedule I got to define, until we were ready to cut over, and we didn’t have to worry about changes to LUNs getting pulled over after something like an xcopy would have completed.
  • No manual xcopy — I synced a 2.3TB CIFS volume in 3 days with 0 babysitting.
  • LUNs can be initially mirrored and updated while still live
  • Changes to volumes/LUNs pulled over automatically, and on a schedule
  • SnapMirror is awesome and lets you leverage its awesome powers, impress your friends, possibly win bar bets AND look good while doing it.
  • Works around unsupported OnComand System Manager, unsupported Storevault OnTAP version, EOL device, etc (Problems 1-4)
  • Manual work required to set up SnapMirror (since we can’t use OCSM)
  • Manual work to break down the SnapMirror relationship
  • Some risk associated with manually manipulating the \etc\snapmirror.conf file (low-risk)
  • Some risk associated with working with the NetApp CLI if you aren’t familiar/comfortable (medium-risk, depending on experience level)
  • Solution not supported by NetApp
Major caveat — your new SAN has to be a NetApp 🙂
I didn’t find anyone on the net doing this, so hopefully this information can be of use to some other storage admins out there, tasked with replacing an aging StoreVault. Don’t be afraid to head down that “unsupported’ path! Yes, there are dragons and cobwebs, but you might find what you’re looking for.

What would you like to say?

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s