Solved

Migrating from NetApp CIFS shares to Acropolis File Services

  • 15 May 2018
  • 4 replies
  • 2862 views

Badge
Hi,

What would be the process to migrate from using NetApp CIFS shares to Acropolis File Services? Has anyone else gone through this process successfully before?
icon

Best answer by LW-FL 8 June 2018, 21:57

Robocopy
Start early to pre-polulate the shares so the final copy is only the changes from the night before.
In Nutanix AFS Fileserver web menu ADMINS add yourself as and admin so your actions will be done as root bypassing existing file permissions.

For large directory structures you make have to split the copyjob into several copy jobs by making one for each sub-directory. I noticed Robocopy even in multi-threaded is single threaded while doing the Tree Comparison walk. So you will maybe see little to no network used for an hour or more while it walks the tree then BAM saturates the link with 128 copy threads. Fortunately NetApp does a really good job of servicing SMB requests even when the workload is really high.

I found the best consistency running Robocopy from a Win7 box rather then a newer server OS. MS decided to change some of the default options in newer versions of Robocopy that effect both permissions verification and how it compares the file dates.

robocopy \\netapp\srcshare$\hourly.0 \\afs-fs.domain.com\destShare /NP /S /E /MT:128 /MIR /FFT /copy:DTSO /R:9 /W:30 /log:c:\CopyLog\FileCOPY.log


For the above source we created a share in NetApp pointing to the hourly snapshot.
(Enable snapdir visible and create a share mapped into the ~Snapshot directory of the volume. Once the new share is up you can disable Snapdir visible. This is not supported by NetApp but it works)

You don't have to do this but it allowed us to do the precopy during the day and not have to deal with files being locked and skipped during the copy. Sometimes the /B option which should allow copy in backup mode even if locked doesn't work.

/NP is for no progress counter in log. Doesn't actually work to stop the progress logging in Mutithreaded mode but best to have it there to keep the logs a compact as possible.
/S - All Sub Directorys
/E - Even Empty Directorys
/MT:128 - 128 threads for the copy jobs the maximum robocopy will allow per cmd session

/MIR - Update with new and changed files and remove files from target that no longer exist in source
/FFT You are copying from a linux like system to a linux system. MS gets confused about the time stamps sometimes and will end up thinking every file has been modified. Use /FFT to be more lenient about time stamp differences.

/copy:DTSO Data Time Security Owner (with S and O this can be slow as it looks like Nutanix does very little caching of SIDs so there is a lot of delay in lookup of user SID to build the permissions on the destination side. Once you have good copy's and get closer to final copy you can use just /copy:DT if you are sure the permissions haven't change in the source tree. New files will still inherit the directory permissions when copied to the new destination.

/R:9 /W:30 - retry 9 times waiting 30 sec between on errors. For our case this was needed for multihour copy jobs as out source is a snapshot. when the NetApp hourly snapshot occurs the directory disappears for several seconds to up to a minute or two depending on how busy NetApp is.

When you do your final copy, get a down time and kick everyone off the share you are migrating by making a new hidden share \\netapp\Sharename_OLD$ pointing to the same volume location and deleting the old share. At that point you can change your copy script to copy from \\netapp\Sharename_OLD$ and be assured no one is locking files on you.

/log:c:\CopyLog\FileCOPY.log - You will have to review the copy logs the make sure nothing was missed. Pay special attention to Home directory shares if roaming profiles were used where there are special permissions deep in the users tree that are restricted to OWNER only or Hidden and user restricted.

Test Test Test - before every copy, open the source and dest and make sure they are correct.

Test from though firewalls. If you are allowing SMB through a firewall, test it.
Nutanix uses DFS redirects to point your client to the right cluster server. We had issues with the redirect not working two thirds of the time through a firewall and had to remigrate a bunch of data to HOMES type shares rather then GENERAL shares. (Because you cant convert an existing share between the two types) HOMES shares will be served out from any cluster member apparently whereas a GENERAL share will always be served by the node that owns it.

Finally. Don't delete your source volumes off the Netapp until you really have too.
At some point someone is going to say you missed something and you will need to look at the old source to confirm the files really were not there to begin with.

Also, Don't forget to disable the snap schedule on the old share, The data's not changing, so stop snapping the changes and rolling off the old snapshots. You might need them.
View original

4 replies

Userlevel 4
Badge +8
First, I have not moved data to AFS yet, but we intend to, so we have been doing some testing and research. At Nutanix .NEXT just last week, there was a company who moved over 100 Tb to AFS using RoboCopy It took them a few months. I have tried that approach from Windows SMB shares on large datasets of legacy user data and it takes a while to fine tune the scripts due to legacy permission issues. I don't know if the company at .NEXT was moving from SMB or CIFS or what they were on prior.
Robocopy
Start early to pre-polulate the shares so the final copy is only the changes from the night before.
In Nutanix AFS Fileserver web menu ADMINS add yourself as and admin so your actions will be done as root bypassing existing file permissions.

For large directory structures you make have to split the copyjob into several copy jobs by making one for each sub-directory. I noticed Robocopy even in multi-threaded is single threaded while doing the Tree Comparison walk. So you will maybe see little to no network used for an hour or more while it walks the tree then BAM saturates the link with 128 copy threads. Fortunately NetApp does a really good job of servicing SMB requests even when the workload is really high.

I found the best consistency running Robocopy from a Win7 box rather then a newer server OS. MS decided to change some of the default options in newer versions of Robocopy that effect both permissions verification and how it compares the file dates.

robocopy \\netapp\srcshare$\hourly.0 \\afs-fs.domain.com\destShare /NP /S /E /MT:128 /MIR /FFT /copy:DTSO /R:9 /W:30 /log:c:\CopyLog\FileCOPY.log


For the above source we created a share in NetApp pointing to the hourly snapshot.
(Enable snapdir visible and create a share mapped into the ~Snapshot directory of the volume. Once the new share is up you can disable Snapdir visible. This is not supported by NetApp but it works)

You don't have to do this but it allowed us to do the precopy during the day and not have to deal with files being locked and skipped during the copy. Sometimes the /B option which should allow copy in backup mode even if locked doesn't work.

/NP is for no progress counter in log. Doesn't actually work to stop the progress logging in Mutithreaded mode but best to have it there to keep the logs a compact as possible.
/S - All Sub Directorys
/E - Even Empty Directorys
/MT:128 - 128 threads for the copy jobs the maximum robocopy will allow per cmd session

/MIR - Update with new and changed files and remove files from target that no longer exist in source
/FFT You are copying from a linux like system to a linux system. MS gets confused about the time stamps sometimes and will end up thinking every file has been modified. Use /FFT to be more lenient about time stamp differences.

/copy:DTSO Data Time Security Owner (with S and O this can be slow as it looks like Nutanix does very little caching of SIDs so there is a lot of delay in lookup of user SID to build the permissions on the destination side. Once you have good copy's and get closer to final copy you can use just /copy:DT if you are sure the permissions haven't change in the source tree. New files will still inherit the directory permissions when copied to the new destination.

/R:9 /W:30 - retry 9 times waiting 30 sec between on errors. For our case this was needed for multihour copy jobs as out source is a snapshot. when the NetApp hourly snapshot occurs the directory disappears for several seconds to up to a minute or two depending on how busy NetApp is.

When you do your final copy, get a down time and kick everyone off the share you are migrating by making a new hidden share \\netapp\Sharename_OLD$ pointing to the same volume location and deleting the old share. At that point you can change your copy script to copy from \\netapp\Sharename_OLD$ and be assured no one is locking files on you.

/log:c:\CopyLog\FileCOPY.log - You will have to review the copy logs the make sure nothing was missed. Pay special attention to Home directory shares if roaming profiles were used where there are special permissions deep in the users tree that are restricted to OWNER only or Hidden and user restricted.

Test Test Test - before every copy, open the source and dest and make sure they are correct.

Test from though firewalls. If you are allowing SMB through a firewall, test it.
Nutanix uses DFS redirects to point your client to the right cluster server. We had issues with the redirect not working two thirds of the time through a firewall and had to remigrate a bunch of data to HOMES type shares rather then GENERAL shares. (Because you cant convert an existing share between the two types) HOMES shares will be served out from any cluster member apparently whereas a GENERAL share will always be served by the node that owns it.

Finally. Don't delete your source volumes off the Netapp until you really have too.
At some point someone is going to say you missed something and you will need to look at the old source to confirm the files really were not there to begin with.

Also, Don't forget to disable the snap schedule on the old share, The data's not changing, so stop snapping the changes and rolling off the old snapshots. You might need them.
Userlevel 7
Badge +35
Hi @MOliver

Did any of the replied above help? If helps, please consider clicking the 'like' and 'best answer' link as that will help others in the community find the answer much quicker - Thanks for your contributions.
Badge
Thank you very much for that detailed walk through that is awesome!

Reply