Acropolis File Services

Badge +4
I am looking for any insights that are available on the network constructs of the (beta?) AFS. Some high level descriptive blogs out there talk about 'one namespace' and 'data locality' for VDI use cases. I am eager to find out how a VM will be redirected to (the files of) it's local host? Is there some dirty IP/DNS trick or is the system designed to be usable across L2 bounderies as well?


Best answer by Jon 4 June 2016, 00:44

Also, here's some doodles on how HA works

View original

17 replies

Userlevel 7
Badge +30
I think its more that in theory you could have them on the same L2 name space, so you'd never have to leave the access layer / leaf layer of the network, giving you relatively good spatial locality to the data.

If the VDIs and the AFS are on two different subnets, then there is nothing anyone can do, since it would all be L3 at that point, so you'd need to go up to the router.
Badge +4
Let's put aside the data locality -case for a moment and focus on the L2/L3 part of the question:

Can different AFS nodes (serving the same namespace) reside in different subnets? Like:

One namespace:
- Subnet A: AFS node 1
- Subnet B: AFS node 2
- Subnet C+D...Z: SMB clients
If yes: where do the clients connect to? A round-robin DNS pointing to A/B AFS node?
Userlevel 7
Badge +30
You could have different AFS nodes in different subnets, but not a single unified namespace, at least currently.

Basically, the filer has an internal and extneral network, internal connects to storage, external connects to clients, and external network is one big thing, think VIP's and failover of said VIPs

We dont yet have a mutli-external network to a single namespace.

Though, if we're just talking L3, filers could be in one namespace and subnet, and the "SMB" client subnets are whatever, as long as they have L3 reachability, should be golden
Badge +4
Thanks for clearing that up!
Focussing on that single namespace with multiple nodes sharing a VIP; is all client traffic to a specific namespace targetted to one node/filterVM then?
Please don't tell me there is something like Microsofts NLB spreading client requests over multiple filer VMs like multicast?
Badge +4
There is a short mention mention of 'DFS' in the Nutanix Bible, which is very cool if indeed used to redirect to a 'hosting File Server VM'.

My guess would be that the 'DFS namespace server role' is co-hosted with the FSVM hosting files. The namespace would than be VIP#1. This DFS 'redirects' to the correct FSVMs which might have its own VIP that can be migrated over/recovered on another FSVM within the same namespace?

Would be great if more info would be available on this topic. Thanks!
Userlevel 7
Badge +30
RE Multicast
No, none of that NLB/multicast shenanigans, that's for the 1990's, heh.

You're right, its basically using a DFS-like referral mechanism. Basically you hit something like "myFileServer12345.corp.local" (whatever you set it up as), and it goes to one of the underlying file server VM's, and you'll get referraled (i.e REDIRECT'd) over to the right place.

Very simple and slick

Here's a quick doodle we threw together in 2 minutes to show this off

Userlevel 7
Badge +30
Also, here's some doodles on how HA works

Badge +4
Thank you very much for the clarification provided with that overview!

So far I have not seen a L2-specific feature that would prevent the fileserver from being served from different subnets. But maybe the missing info is that in case of a FSVM failure (NVM1 in your schema) it's IP (+ maybe mac-address?) is added to the NIC of NVM2?
edit: I was to quick; you confirmed that HA is the cause of the L2 boundery due to IP (of MAC as well?) migration to a healthy node.
Userlevel 7
Badge +30
See the last doodles I posted on NVM HA

When one fails, the other(s) take over its public IP, which would need to be on the same L2 segment.

Otherwise, it would just go into a black hole when NVM-HA kicks in.
Userlevel 7
Badge +30
messages crossed in flight.

Yep, thats why that is.

That said, you can have MANY NVM/filers per Nutanix AOS cluster, so you could have a setup like this in a single cluster

Nutanix Cluster 01
Filer Cluster 01 = Security Zone A / subnet 101
Filer Cluster 02 = Security Zone B / subnet 202

And so on.

They are different namespaces, sure, but thats not such a bad thing. If you need to scale the namespace out, you just add more NVM's to a single filer cluster (click click, easy), and as needed, grow the underlying Nutanix AOS cluster as you need space.
Badge +4
Yep; you nailed it.
Do the FSVM arbitrate the cluster properties (election / HA etc) themselfs or are they heavily dependant on the underlying NDFS for these tasks? Read: can it be seen seperate or tightly tied to current architecture.

Also: Acropolis hypervisor only for now right? Any news regarding esx support timeline (or any GA for that matter)? Can this be tested in Community Edition?

We one rare occasions encounter a non-DFS compliant SMB client btw.. All-in-one multifunctionals for example tend to break /w scan-to-folder.
Userlevel 7
Badge +30
RE separate or stand alone
Control plane is in CVM (i.e Prism), but most other stuff is independent, such as election and HA.

It leverages a very similar architecture to NOS/AOS in that regard, since we know thats reliable/scalable/etc, we just reused quite a bit of base work there.

See doodle below

Userlevel 7
Badge +30
RE hypervisors
Yep, tech preview/beta on AHV now, will come out on other hypervisors eventually (future releases, NDA, etc, this is a public forum of course)

You should be able to test this on CE just fine. Note that its still tech preview there too, so there are some silly tech preview level restrictions that will come off with GA
Badge +4
So what about Active Directory based NTFS ACLs on files/folders? All supported?
Userlevel 7
Badge +30
Should be, as the first thing we make you do is have the filer join AD
Badge +8
Hi all,

I'm going to do the demo Acropolis File Services on NX box for customer. As I read as I know, AFS can set quota limit by user/ group belong to AD. But how the NTFS permission or File/Folder permission for Share file per User/Group?

Does it still support functionality like Windows File Server?
Userlevel 4
Badge +21
for the share permissions you can use the windows mmc for file services or you can also just right click the share and apply your ntfs security settings.