Overview

FleX Driver Development Kit

Driver development overview.

About this task

Nutanix provides the FleX Driver Development Kit (DDK) to help developers build, inject, qualify, and distribute new drivers.

Use the DDK to build a Linux kernel with one or more loadable kernel modules. The DDK builds a Linux kernel tar file that you can use as an input for Metis.

In general, the driver development procedure is as follows:

Procedure

1. Prepare the compile environment.

2. Prepare the source code to compile. The source can be an updated kernel source tree or one or more out-of-tree kernel module.

3. Compile the kernel and modules to create the kernel tar file with the driver.

4. Use the new kernel tar file as an input for Metis.

5. Use Metis to qualify the new kernel and driver.

6. Submit the driver source code (source rpm), qualification results, the procedure for building the driver, and release notes though your qualification ticket on the Elevate portal.

FleX DDK Requirements

Driver development requirements.

To use the FleX DDK, make sure that you have the following elements:

• A Linux system with more than 10 GB disk space and internet access.

• A distributable driver through the GNU General Public License without user space tools.

• Access to a Linux system with a Docker environment and non-root user access. You can obtain the Docker tools from http://docker.com.

• A copy of the kernel source code. You can obtain a copy from Nutanix through the Elevate portal.

• The most recent builds of Metis and other FleX tools from the Elevate portal.

Building the Kernel and Driver

Preparing the Build Environment

Prepare and test the environment before compiling the kernel and modules.

Procedure

1. Download the latest version of Docker for Linux from http://docker.com and install it in your Linux environment.

Install Docker and create a build user.
yum install -y docker

 groupadd docker

 useradd builduser -g docker

2.  systemctl start docker

3. Copy the DDK into the /home/builduser folder and change the owner of the file to "builduser".

Switch to builduser and run the make command in Docker.
su - builduser

 tar xvfz ddk-v1.0.tar.gz

 cd ddk

4.  ./start-docker.sh make
   The command creates tar files named kernel-phoenix-xxx.tar,kernel-ahv-nutanix-xxx.tar, andkernel-cvm-nutanix-xxx.tar.Note: These files are only test files. Create the actual files for Metis qualification as described in theCompiling a Kernel and Out-of-Tree Module section.

Compiling a Kernel and Out-of-Tree Module

Compile and install the kernel and the module for AHV, Phoenix, CVM, or a combination of these.

Procedure

1. Determine which kernel you want to use.

2. In the directory where you downloaded the DDK, locate and modify thenutanix-build.sh file for which you built the driver that includes AHV.

3. Open nutanix-build.sh in a text editor and search for the string external_driver_dirs. Add the name of your driver at the end of that section and save your changes.

4. For AHV, edit nutanix-build.sh, and run themake command in Docker.
   $ start-docker.sh kernel-ahv
   The command creates a tar file namedkernel-ahv-xxx.tar. Use the TAR file for Metis qualification.Note:
   To create only one file, use the name of the file as an argument:

   • For Phoenix, use start-docker.sh make kernel-phoenix. This commands results in thekernel-phoenix-xxx.tar file.

   • For CVM, use start-docker.sh make kernel-cvm. This commands results in the kernel-cvm-xxx.tar file.

5. To create all three files, use the make command without any arguments:

   • For AHV, Phoenix, and CVM; use start-docker.sh make. This command results in creating all the three .tar files.

RPM and Kernel Bundle in Metis

You can add an RPM and kernel bundle to a Metis project using one of the following options:

• Through the Hardware Qualification tab in the Metis home page. This option is a recommended by Nutanix.

• Through the Settings option in the Metis home page.

• Adding the RPM and Kernel Bundle Using the Hardware Qualification Option

• Adding the RPM and Kernel Bundle Using the Settings Option

Adding the RPM and Kernel Bundle Using the Hardware Qualification Option

About this task

To add the RPM and kernel bundle to a Metis project:

Procedure

1. In the Metis toolbar, click the Hardware Qualification button.

2. Click to enlarge

3. 
   
   The New Qualification Project dialog box appears.

4. Configure the Project page.

5. Click to enlarge

6. 
    

7. Click Next to configure the Targets page.

8. Click to enlarge

9. 
    

10. Click Next to go to the RPM and Kernel page.

11. Click to enlarge

12. 
     

    • If you are performing a qualification that includes bare-metal Linux tests, under CentOS RPM, either upload the RPM file or enter the path to the URL where the RPM is hosted.
      The RPMs are required if you are using a custom driver and need to inject it into our standard CentOS Linux that is used for bare-metal testing. Ensure that you obtain or create an RPM containing the driver or drivers used for the RPM injection. The RPM must be appropriate for the version of CentOS that is available on the Elevate portal.

    • If you are using custom kernels, under Custom Kernels, click the + Add Kernel Bundle button, specify the type of kernels and the name of the kernel tar files.
      In cases where two custom kernels are required, you must provide two drivers.

    • Figure. Add Kernel Bundle

    • Click to enlarge

    • 
       

    • If you are not performing a bare-metal qualification or using a custom kernel, leave all fields blank.

    • Click Save.
      For more information about configuring the other tabs, see the Metis Guide.

Adding the RPM and Kernel Bundle Using the Settings Option

About this task

To add the RPM and kernel bundle to a Metis project:

Procedure

1. In the Metis home page, select the project you want to use (or create a new one) and click Settings in the Metis sidebar.

2. Figure. Settings

3. Click to enlarge

4. 
   
    

5. Do the following:

6. Figure. CentOS RPMs and Custom Kernels

7. Click to enlarge

8. 
   
    

   • If you are performing a qualification that includes bare-metal Linux tests, under CentOS RPM, either upload the RPM file or enter the path to the URL where the RPM is hosted.
     The RPMs are required if you are using a custom driver and need to inject it into our standard CentOS Linux that is used for bare-metal testing. Ensure that you obtain or create an RPM containing the driver or drivers used for the RPM injection. The RPM must be appropriate for the version of CentOS that is available on the Elevate portal.

   • If you are using custom kernels, under Custom Kernels, click the + Add Kernel Bundle button, specify the type of kernels and the name of the kernel tar files.
     In cases where two custom kernels are required, you must provide two drivers.

   • Figure. Add kernel Bundle

   • Click to enlarge

   • 
     
      

9. Click Save to save the bundle and return to the Metis home page.

10. Use this project for Metis qualification, as described in the Metis Guide.

Example

Example: Compiling a Kernel and i40e Driver

Compile and install the kernel and driver for i40e on AHV, Phoenix, CVM, or a combination of these.

Procedure

1. Go to the Elevate portal at http://elevate.nutanix.com and download the DDK kernel source code from the Downloads section.

2. Go to Sourceforge and download the i40e driver .tar file.

3. Untar the i40E driver .tar file into the directory where you downloaded the DDK.

4. In the directory where you downloaded the DDK, open thekernel-ahv-nutanix folder, and locate thenutanix-build.sh file.

Open nutanix-build.sh in a text editor and search for the string external_driver_dirs. Add the name of your driver at the end of that section and save your changes.
external_driver_dirs=(

$(cd ../hpsa-3.4.18/drivers/scsi; pwd)

$(cd ../mpt3sas-27.00.00.00; pwd)

$(cd ../smc_sum_driver-1.6.0; pwd)

$(cd ../smartpqi-1.1.2-125; pwd)

$(cd ../i40e-2.9.21/src; pwd)

5. )
   Note: The /src folder is where the i40e make file is located.

6. For AHV, edit nutanix-build.sh, and run themake command in Docker.
   $ start-docker.sh kernel-ahv
   The command creates a tar file namedkernel-ahv-xxx.tar. Use the .tar file for Metis qualification.Note:
   To create only one file, use the name of the file as an argument:

   • For Phoenix, use start-docker.sh make kernel-phoenix. This commands results in thekernel-phoenix-xxx.tar file.

   • For CVM, use start-docker.sh make kernel-cvm. This commands results in the kernel-cvm-xxx.tar file.

7. To create all three files, use the make command without any arguments:

   • For AHV, Phoenix, and CVM, use start-docker.sh make. This command results in creating all the three TAR files.

8. The tar file contains ./boot/kernel,./lib/modules/version-ddk/extra/i40e.ko, and other kernel modules; if selected.

Glossary

Glossary of FleX Terms

AHV

The native Nutanix hypervisor.

AOS

The Nutanix operating system.

BMC

Baseboard management controller, the microcontroller that manages the motherboard.

BMC IP address

The network address that provides remote management access for the BMC.

CVM

Controller Virtual Machine, a leader VM present on every Nutanix node.

CVM IP address

The network address that provides remote management access for the CVM.

Foundation VM

The virtual machine that manages the Foundation imaging tool.

Hypervisor IP address

The network address that provides access to the hypervisor (AHV, ESXi, Hyper-V) that hosts your virtual machines.

IPMI

Intelligent platform management interface, a means of directly managing system hardware without going through the operating system or CPU.

Metis VM

The virtual machine that manages the Metis hardware validation tool.

Node IP address

A collective term for the CVM, hypervisor, and IPMI IP addresses.

Non-BMC IP address

A collective term for the CVM and hypervisor IP addresses.