Tutorial Overview

The Virtex-5 FPGA is particularly useful in Ethernet applications because it contains embedded Tri-mode 10/100/1000 Mbps Ethernet MACs. If you have done Ethernet designs before, you will know that Xilinx’s “soft” Ethernet MAC IP cores are not free and designing one yourself would be quite an undertaking. In this tutorial, we will generate an embedded Tri-mode Ethernet MAC IP wrapper using the Xilinx CORE Generator version 10.1.

Requirements

Before following this tutorial, you will need to do the following:

• You will need CORE Generator version 10.1.03 installed for this tutorial.

Generate the Tri-mode Ethernet MAC IP Wrapper

Follow these instructions for generating the Ethernet MAC IP wrapper.

1. From the “Start” menu, open Xilinx CORE Generator.
2. Select “File->New Project”.
3. Click “Browse” and select an appropriate location for the Coregen project. Select the folder where you normally place your projects, for example “C:\ML505\Projects”, and create a sub-folder called “TEMACCore”. Open this folder and click “OK”.
4. You will be asked for the specifications of the FPGA you are using. All the cores you generate under this CORE Generator project file will be customized for the FPGA you specify here. Under the “Part” tab, select these options: Family “Virtex5″, Device “xc5vlx50t”, Package “ff1136″, Speed grade “-1″. Click “OK”. Note: If you are not using the ML505 board, these specifications may not apply to you. You will have to enter the details corresponding to the specific FPGA that you are using.
   Under the “Generation” tab, you can specify how you want your IP cores to be generated. Be sure that “VHDL” output is selected as shown below. 

5. When you have created your CORE Generator project, click on the “View by Function” tab to get a list of cores that you are able to generate.
6. Open “Communication & Networking->Ethernet” and double-click on “Virtex-5 Embedded Tri-mode Ethernet MAC Wrapper”.
7. A dialog box should open to allow you to select the features of the Ethernet MAC Core you want. For each page, enter the settings as shown below.
   
   
   
8. Click “Finish”. Your Ethernet MAC wrapper will be generated and CORE Generator will display a list of all the generated files. Close this window, and close CORE Generator.

Examine the Generated Files

We will now examine the generated files and explain their purpose and utility. Open “Windows Explorer” and browse to the Coregen folder that we just created. We should see a list of files as shown below:

We can see that CORE Generator has placed some files in the coregen project folder, and it has also created a folder specifically for the Ethernet MAC wrapper called “v5_emac_v1_5″. We will discuss the files that you will find most useful.

• Instantiation Template File (v5_emac_v1_5.vho)The instantiation template provides a template for declaring the Ethernet MAC component and instantiating it. If you are already familiar with the Ethernet MAC core, you could use this template by copying and pasting it to your design and making the necessary connections. If it is your first time using the core, you might prefer to use the examples provided in the “v5_emac_v1_5″ folder.

• Ethernet MAC Core Datasheet (doc\v5_emac_ds550.pdf)The datasheet provides the technical specifications for the Ethernet MAC core such as the total resources that it occupies and a short description of its parameters. It also contains an overview of the provided example.
• Ethernet MAC Core Getting Started Guide (doc\v5_emac_gsg340.pdf)The getting started guide is probably the most useful document for people new to the Ethernet MAC core. It provides an introduction to the core and gives a detailed description of the example that is generated with the core. It also contains instructions on how to simulate/implement the example.
• Example Design folderIn the “example_design” folder you will find the source code for the example described in the Getting Started Guide and also the wrapper for the Ethernet MAC core. Remember, you will not find VHDL source for the Ethernet MAC, because it is an embedded hardware device, not a “soft” core. In this folder you will also find the .ucf constraints file for use with the example. The provided example can be implemented in ISE and it includes an address swap module that loops all received Ethernet packets back to the source after swapping the MAC source/destination addresses in the packet. The example design can be tested by connecting the ML505 board to the Ethernet socket of your PC and using the open source Wireshark software to monitor your Ethernet traffic. Read the Getting Started Guide for more details.
• Simulation folderThe “simulation” folder contains testbenches for simulating the Ethernet MAC core example.

The Coregen folder for this tutorial can be downloaded in a compressed ZIP file. Please select the file corresponding to your board, right-click on it and select “Save Link As”.

Tutorial Overview

The Virtex-5 Embedded Tri-mode Ethernet MAC is useful for designs requiring Ethernet connectivity. Fortunately, Xilinx has made it easy for us to start developing with the Ethernet MACs by providing several online examples and application notes. One of the examples can be obtained when you use CORE Generator to generate the Ethernet MAC wrapper. The generated example is a simple design that mirrors incoming Ethernet packets, swapping the source and destination MAC addresses. In this tutorial, we implement the example design provided by CORE Generator by working the example code into a custom peripheral for EDK 10.1.

This tutorial contains screenshots to guide you through the entire implementation process. Click on the images to view a higher resolution.

Requirements

Before following this tutorial, you will need to do the following:

• Generate the Virtex-5 Embedded Tri-mode Ethernet MAC Wrapper using CORE Generator. For instructions on doing this, please refer to the tutorial Generating the Ethernet MAC
• Set the J22 and J23 jumpers on the ML505 to positions 2-3 as shown below. This allows us to use an SGMII (serial) interface with the PHY.
   
• Install a copy of Wireshark on a PC with a Gigabit Ethernet network card
• Obtain a CAT5 Ethernet cable: regular or crossover, either will work because the PHY on the ML505 has an automatic switching feature that detects what type of cable you are using and switches the TX and RX pins if necessary.
• Buy an ML505/ML506/ML507 or XUPV5 board if you don’t already have one. Xilinx supplies the ML50x boards, but the best deal is the XUPV5 from Digilent. Click the Digilent link on this page for more information.

Create the Basic Project

Follow these steps to create the basic project:

1. Open XPS. From the dialog box, select “Base System Builder wizard” and OK.
2. You will be asked to specify which folder to place the project. Click “Browse” and create a new folder for the project. Click “OK”.
    
3. We are given the choice to create a new project or to create one using the template of another project. Tick “I would like to create a new design” and click “Next”.
4. On the “Select Board” page, select “Xilinx” as the board vendor. Select “Virtex 5 ML505 Evaluation Platform” as the board name. Select “1″ as the board revision. Click “Next”.
    
5. On the “Select Processor” page, we normally have a choice between using the PowerPC “hard” processor, or the Microblaze “soft” processor. Since the Virtex-5 does not contain any PowerPCs, we can only select Microblaze. Click “Next”.
    
6. On the “Configure Microblaze” page, select the clock frequency to be 125MHz. For the BRAM local memory, select “64KB”. We will use the RS232 port for debugging rather than the JTAG, so select “No debug”. Click “Next”.
    
7. In selecting the Additional IO Interfaces, leave “RS232_Uart_1″ ticked and un-tick everything else.
   
   
   
    
8. On the “Add Internal Peripherals” page, click “Next”.
9. On the “Software Setup” page, select “RS232_Uart_1″ for both STDIN and STDOUT. Un-tick “Memory Test” and leave “Peripheral Test” ticked. Click “Next”.
    
10. Click “Generate”.
11. Click “Finish”.

Create the Ethernet MAC Peripheral

We now create our Ethernet MAC peripheral using the Peripheral Wizard.

1. Select from the menu “Hardware->Create or Import Peripheral”. Click “Next”.
2. Select “Create templates for a new peripheral” and click “Next”.
    
3. We must now decide where to place the files for the peripheral. They can be placed within this project, or they can be made accessible to other projects. Select “To an XPS project”. Click “Next”.
4. On the “Name and Version” page, type “eth_mac” for the peripheral name. Click “Next”.
    
5. On the “Bus Interface” page, select “Processor Local Bus” (PLB) and click “Next”.
    
6. On the “IPIF Services” page, select “Include data phase timer”. Un-tick everything else and click “Next”.
    
7. On the “Slave Interface” page, leave the defaults and click “Next”.
    
8. On the “Peripheral Simulation Support” page, we can specify if we want the wizard to create a simulation platform for our peripheral. Click “Next” without ticking the option to generate.
9. After the “Peripheral Implementation Support” page, the wizard will generate all the template files for us. Tick “Generate ISE and XST project files” and “Generate template driver files”. Click “Next”.
10. Click “Finish”. Now our templates are created and we can modify them to include the code for the timer.

Copy the Ethernet MAC source files

We need to copy the Ethernet MAC source files generated by CORE Generator into the Ethernet MAC peripheral source folder. If you have not generated the source files using CORE Generator, please refer to the tutorial Generating the Ethernet MAC.

1. Open Windows Explorer and browse to the folder “TEMACCore\v5_emac_v1_5″. This is the folder you created with CORE Generator.
    
2. In that folder, you will find a subfolder called “example_design”. Copy the “example_design” folder into the “pcores\eth_mac_v1_00_a\hdl\vhdl” folder within your XPS project. This is the folder where you should find your “user_logic.vhd” file for the Ethernet MAC peripheral.

Modify the .PAO file

The .pao file contains a list of all the source files that compose our peripheral. We use this list when we run the Peripheral Wizard in Import mode. Now that we have added the Ethernet MAC example source files to the project, we must include them in the .pao file. Note that files must be listed in the .pao file in hierarchical order. The components at the top of the hierarchy are listed at the bottom of the file.

1. Select “File->Open” and browse to the “pcores\eth_mac_v1_00_a\data” folder. Select the file “eth_mac_v2_1_0.pao” and click “Open”.
2. At the bottom of this file you will see these two lines:

lib eth_mac_v1_00_a user_logic vhdl
lib eth_mac_v1_00_a eth_mac vhdl

1. Add the following lines just above those two lines. It is important to copy the lines exactly as shown and in the same order.

lib eth_mac_v1_00_a example_design/v5_emac_v1_5.vhd
lib eth_mac_v1_00_a example_design/physical/rx_elastic_buffer.vhd
lib eth_mac_v1_00_a example_design/physical/gtp_dual_1000X.vhd
lib eth_mac_v1_00_a example_design/physical/rocketio_wrapper_gtp.vhd
lib eth_mac_v1_00_a example_design/physical/rocketio_wrapper_gtp_tile.vhd
lib eth_mac_v1_00_a example_design/v5_emac_v1_5_block.vhd
lib eth_mac_v1_00_a example_design/client/fifo/tx_client_fifo_8.vhd
lib eth_mac_v1_00_a example_design/client/fifo/rx_client_fifo_8.vhd
lib eth_mac_v1_00_a example_design/client/fifo/eth_fifo_8.vhd
lib eth_mac_v1_00_a example_design/v5_emac_v1_5_locallink.vhd
lib eth_mac_v1_00_a example_design/client/address_swap_module_8.vhd

1. Save the file.

Now we can use this .pao file with the Peripheral Wizard when we import the Ethernet MAC peripheral.

Modify the Ethernet MAC Peripheral

If you refer back to the example design source files created by CORE Generator, you will find the top module contained in the file “v5_emac_v1_5\example_design\v5_emac_v1_5_example_design.vhd”. We copied that file into our peripheral source folder, but we wont directly use it in our design (notice that we didn’t include it in our .pao file). Instead, we will take the code from this file and work it into our “user_logic.vhd” file, the top module for our peripheral.

1. Select from the menu “File->Open” and look in the project folder.
2. Open the folders: “pcores\eth_mac_v1_00_a\hdl\vhdl”.
3. Open the file “eth_mac.vhd”.
4. Find the line of code that says “– ADD USER PORTS BELOW THIS LINE” and add the following lines of code just below.

  REFCLK_N_IN              : in std_logic;
  REFCLK_P_IN              : in std_logic;
  GTP_READY                : out std_logic;
  PHY_RESET_0              : out std_logic;
  RXP_IN                   : in std_logic;
  RXN_IN                   : in std_logic;
  TXP_OUT                  : out std_logic;
  TXN_OUT                  : out std_logic;

1. Find the line of code that says “– MAP USER PORTS BELOW THIS LINE” and add the following lines of code just below.

  REFCLK_N_IN  => REFCLK_N_IN,
  REFCLK_P_IN  => REFCLK_P_IN,
  GTP_READY    => GTP_READY,
  PHY_RESET_0  => PHY_RESET_0,
  RXP_IN       => RXP_IN,
  RXN_IN       => RXN_IN,
  TXP_OUT      => TXP_OUT,
  TXN_OUT      => TXN_OUT,

1. Save and close the file.
2. Open the file “user_logic.vhd”. We will need to modify this source code to include our example code.
3. Find the line of code that says “–USER libraries added here” and add the following lines of code just below.

library UNISIM;
use UNISIM.VCOMPONENTS.ALL;

1. Find the line of code that says “– ADD USER PORTS BELOW THIS LINE” and add the following lines of code just below.

  REFCLK_N_IN              : in std_logic;
  REFCLK_P_IN              : in std_logic;
  GTP_READY                : out std_logic;
  PHY_RESET_0              : out std_logic;
  RXP_IN                   : in std_logic;
  RXN_IN                   : in std_logic;
  TXP_OUT                  : out std_logic;
  TXN_OUT                  : out std_logic;

1. Find the line of code that says “–USER signal declarations added here” and add the following lines of code just below.

  -- Component Declaration for the TEMAC wrapper with 
  -- Local Link FIFO.
  component v5_emac_v1_5_locallink is
   port(
      -- EMAC0 Clocking
      -- 125MHz clock output from transceiver
      CLK125_OUT                : out std_logic;
      -- 125MHz clock input from BUFG
      CLK125                    : in  std_logic;
      -- Tri-speed clock output from EMAC0
      CLIENT_CLK_OUT_0          : out std_logic;
      -- EMAC0 Tri-speed clock input from BUFG
      client_clk_0              : in  std_logic;

      -- Local link Receiver Interface - EMAC0
      RX_LL_CLOCK_0             : in  std_logic; 
      RX_LL_RESET_0             : in  std_logic;
      RX_LL_DATA_0              : out std_logic_vector(7 downto 0);
      RX_LL_SOF_N_0             : out std_logic;
      RX_LL_EOF_N_0             : out std_logic;
      RX_LL_SRC_RDY_N_0         : out std_logic;
      RX_LL_DST_RDY_N_0         : in  std_logic;
      RX_LL_FIFO_STATUS_0       : out std_logic_vector(3 downto 0);

      -- Local link Transmitter Interface - EMAC0
      TX_LL_CLOCK_0             : in  std_logic;
      TX_LL_RESET_0             : in  std_logic;
      TX_LL_DATA_0              : in  std_logic_vector(7 downto 0);
      TX_LL_SOF_N_0             : in  std_logic;
      TX_LL_EOF_N_0             : in  std_logic;
      TX_LL_SRC_RDY_N_0         : in  std_logic;
      TX_LL_DST_RDY_N_0         : out std_logic;

      -- Client Receiver Interface - EMAC0
      EMAC0CLIENTRXDVLD         : out std_logic;
      EMAC0CLIENTRXFRAMEDROP    : out std_logic;
      EMAC0CLIENTRXSTATS        : out std_logic_vector(6 downto 0);
      EMAC0CLIENTRXSTATSVLD     : out std_logic;
      EMAC0CLIENTRXSTATSBYTEVLD : out std_logic;

      -- Client Transmitter Interface - EMAC0
      CLIENTEMAC0TXIFGDELAY     : in  std_logic_vector(7 downto 0);
      EMAC0CLIENTTXSTATS        : out std_logic;
      EMAC0CLIENTTXSTATSVLD     : out std_logic;
      EMAC0CLIENTTXSTATSBYTEVLD : out std_logic;

      -- MAC Control Interface - EMAC0
      CLIENTEMAC0PAUSEREQ       : in  std_logic;
      CLIENTEMAC0PAUSEVAL       : in  std_logic_vector(15 downto 0);

      --EMAC-MGT link status
      EMAC0CLIENTSYNCACQSTATUS  : out std_logic;
      -- EMAC0 Interrupt
      EMAC0ANINTERRUPT          : out std_logic;

      -- Clock Signals - EMAC0

      -- SGMII Interface - EMAC0
      TXP_0                     : out std_logic;
      TXN_0                     : out std_logic;
      RXP_0                     : in  std_logic;
      RXN_0                     : in  std_logic;
      PHYAD_0                   : in  std_logic_vector(4 downto 0);
      RESETDONE_0               : out std_logic;

      -- unused transceiver
      TXN_1_UNUSED              : out std_logic;
      TXP_1_UNUSED              : out std_logic;
      RXN_1_UNUSED              : in  std_logic;
      RXP_1_UNUSED              : in  std_logic;

      -- SGMII RocketIO Reference Clock buffer inputs 
      CLK_DS                    : in  std_logic;

      -- Asynchronous Reset
      RESET                     : in  std_logic
   );
  end component;

   ---------------------------------------------------------------------
   --  Component Declaration for 8-bit address swapping module
   ---------------------------------------------------------------------
   component address_swap_module_8
   port (
      rx_ll_clock         : in  std_logic;
      rx_ll_reset         : in  std_logic;
      rx_ll_data_in       : in  std_logic_vector(7 downto 0);
      rx_ll_sof_in_n      : in  std_logic;
      rx_ll_eof_in_n      : in  std_logic;
      rx_ll_src_rdy_in_n  : in  std_logic;
      rx_ll_data_out      : out std_logic_vector(7 downto 0);
      rx_ll_sof_out_n     : out std_logic;
      rx_ll_eof_out_n     : out std_logic;
      rx_ll_src_rdy_out_n : out std_logic;
      rx_ll_dst_rdy_in_n  : in  std_logic
      );
   end component;

  ------------------------
  -- Signal Declarations
  ------------------------

  -- address swap transmitter connections - EMAC0
  signal tx_ll_data_0_i      : std_logic_vector(7 downto 0);
  signal tx_ll_sof_n_0_i     : std_logic;
  signal tx_ll_eof_n_0_i     : std_logic;
  signal tx_ll_src_rdy_n_0_i : std_logic;
  signal tx_ll_dst_rdy_n_0_i : std_logic;

  -- address swap receiver connections - EMAC0
  signal rx_ll_data_0_i      : std_logic_vector(7 downto 0);
  signal rx_ll_sof_n_0_i     : std_logic;
  signal rx_ll_eof_n_0_i     : std_logic;
  signal rx_ll_src_rdy_n_0_i : std_logic;
  signal rx_ll_dst_rdy_n_0_i : std_logic;

  -- create a synchronous reset in the transmitter clock domain
  signal ll_pre_reset_0_i    : std_logic_vector(5 downto 0);
  signal ll_reset_0_i        : std_logic;

  attribute async_reg : string;
  attribute async_reg of ll_pre_reset_0_i : signal is "true";

  signal resetdone_0_i       : std_logic;

  -- EMAC0 Clocking signals

  -- Transceiver output clock (REFCLKOUT at 125MHz)
  signal user_clk_out              : std_logic;
  -- 125MHz clock input to wrappers
  signal user_clk                  : std_logic;
  -- Input 125MHz differential clock for transceiver
  signal ref_clk                   : std_logic;

  -- 1.25/12.5/125MHz clock signals for tri-speed SGMII
  signal client_clk_0_o            : std_logic;
  signal client_clk_0              : std_logic;

1. Find the line of code that says “–USER logic implementation added here” and add the following lines of code just below.

  -- PHY Reset logic
  PHY_RESET_0 <= not Bus2IP_Reset;   -- EMAC0 Clocking   -- Generate the clock input to the GTP   -- clk_ds can be shared between multiple MAC instances.   clkingen : IBUFDS port map (     I  => REFCLK_P_IN,
    IB => REFCLK_N_IN,
    O  => ref_clk);

  -- 125MHz from transceiver is routed through a BUFG and 
  -- input to the MAC wrappers.
  -- This clock can be shared between multiple MAC instances.
  bufg_clk125 : BUFG port map (I => user_clk_out, O => user_clk);

  -- 1.25/12.5/125MHz clock from the MAC is routed through a BUFG and  
  -- input to the MAC wrappers to clock the client interface.
  bufg_client_0 : BUFG port map (I => client_clk_0_o, O => client_clk_0);

  --------------------------------------------
  -- Instantiate the EMAC Wrapper with LL FIFO 
  -- (v5_emac_v1_5_locallink.v)
  --------------------------------------------
  v5_emac_ll : v5_emac_v1_5_locallink
  port map (
    -- EMAC0 Clocking
    -- 125MHz clock output from transceiver
    CLK125_OUT                => user_clk_out,
    -- 125MHz clock input from BUFG
    CLK125                    => user_clk,
    -- Tri-speed clock output from EMAC0
    CLIENT_CLK_OUT_0          => client_clk_0_o,
    -- EMAC0 Tri-speed clock input from BUFG
    CLIENT_CLK_0              => client_clk_0,
    -- Local link Receiver Interface - EMAC0
    RX_LL_CLOCK_0             => user_clk,
    RX_LL_RESET_0             => ll_reset_0_i,
    RX_LL_DATA_0              => rx_ll_data_0_i,
    RX_LL_SOF_N_0             => rx_ll_sof_n_0_i,
    RX_LL_EOF_N_0             => rx_ll_eof_n_0_i,
    RX_LL_SRC_RDY_N_0         => rx_ll_src_rdy_n_0_i,
    RX_LL_DST_RDY_N_0         => rx_ll_dst_rdy_n_0_i,
    RX_LL_FIFO_STATUS_0       => open,

    -- Unused Receiver signals - EMAC0
    EMAC0CLIENTRXDVLD         => open,
    EMAC0CLIENTRXFRAMEDROP    => open,
    EMAC0CLIENTRXSTATS        => open,
    EMAC0CLIENTRXSTATSVLD     => open,
    EMAC0CLIENTRXSTATSBYTEVLD => open,

    -- Local link Transmitter Interface - EMAC0
    TX_LL_CLOCK_0             => user_clk,
    TX_LL_RESET_0             => ll_reset_0_i,
    TX_LL_DATA_0              => tx_ll_data_0_i,
    TX_LL_SOF_N_0             => tx_ll_sof_n_0_i,
    TX_LL_EOF_N_0             => tx_ll_eof_n_0_i,
    TX_LL_SRC_RDY_N_0         => tx_ll_src_rdy_n_0_i,
    TX_LL_DST_RDY_N_0         => tx_ll_dst_rdy_n_0_i,

    -- Unused Transmitter signals - EMAC0
    CLIENTEMAC0TXIFGDELAY     => "00000000",
    EMAC0CLIENTTXSTATS        => open,
    EMAC0CLIENTTXSTATSVLD     => open,
    EMAC0CLIENTTXSTATSBYTEVLD => open,

    -- MAC Control Interface - EMAC0
    CLIENTEMAC0PAUSEREQ       => '0',
    CLIENTEMAC0PAUSEVAL       => "0000000000000000",

    --EMAC-MGT link status
    EMAC0CLIENTSYNCACQSTATUS  => GTP_READY,
    -- EMAC0 Interrupt
    EMAC0ANINTERRUPT          => open,

    -- Clock Signals - EMAC0
    -- SGMII Interface - EMAC0
    TXP_0                     => TXP_OUT,
    TXN_0                     => TXN_OUT,
    RXP_0                     => RXP_IN,
    RXN_0                     => RXN_IN,
    PHYAD_0                   => "00010",
    RESETDONE_0               => resetdone_0_i,

    -- unused transceiver
    TXN_1_UNUSED              => open,
    TXP_1_UNUSED              => open,
    RXN_1_UNUSED              => '1',
    RXP_1_UNUSED              => '0',

    -- SGMII RocketIO Reference Clock buffer inputs 
    CLK_DS                    => ref_clk,

    -- Asynchronous Reset
    RESET                     => Bus2IP_Reset
  );

  --------------------------------------------
  --  Instatiate the address swapping module
  --------------------------------------------
  client_side_asm_emac0 : address_swap_module_8
    port map (
      rx_ll_clock         => user_clk,
      rx_ll_reset         => ll_reset_0_i,
      rx_ll_data_in       => rx_ll_data_0_i,
      rx_ll_sof_in_n      => rx_ll_sof_n_0_i,
      rx_ll_eof_in_n      => rx_ll_eof_n_0_i,
      rx_ll_src_rdy_in_n  => rx_ll_src_rdy_n_0_i,
      rx_ll_data_out      => tx_ll_data_0_i,
      rx_ll_sof_out_n     => tx_ll_sof_n_0_i,
      rx_ll_eof_out_n     => tx_ll_eof_n_0_i,
      rx_ll_src_rdy_out_n => tx_ll_src_rdy_n_0_i,
      rx_ll_dst_rdy_in_n  => tx_ll_dst_rdy_n_0_i
  );

  rx_ll_dst_rdy_n_0_i     <= tx_ll_dst_rdy_n_0_i;

  -- Create synchronous reset in the transmitter clock domain.
  gen_ll_reset_emac0 : process (user_clk, Bus2IP_Reset)
  begin
    if Bus2IP_Reset = '1' then
      ll_pre_reset_0_i <= (others => '1');
      ll_reset_0_i     <= '1';
    elsif user_clk'event and user_clk = '1' then
      if resetdone_0_i = '1' then
        ll_pre_reset_0_i(0)          <= '0';
        ll_pre_reset_0_i(5 downto 1) <= ll_pre_reset_0_i(4 downto 0);
        ll_reset_0_i                 <= ll_pre_reset_0_i(5);
      end if;
    end if;
  end process gen_ll_reset_emac0;

1. Save and close the file.

If you examine the “v5_emac_v1_5_example_design.vhd” from the CORE Generator output, you will see that most of the code we inserted into “user_logic.vhd” was directly copied from the example code. The main differences are:

• PHY Reset: We needed to add a signal (PHY_RESET_0) to drive the external Ethernet PHY reset pin. The PHY on the ML505 board has an active low reset input, so we connect it to the logical “not” of the bus reset signal.
• The clock signal names were changed for clarity. Most Xilinx documents refer to these clocks as REFCLK and USERCLK.

Import the Ethernet MAC Peripheral

Now we will use the Peripheral Wizard again, but this time using the import function.

1. Select from the menu “Hardware->Create or Import Peripheral” and click “Next”.
2. Select “Import existing peripheral” and click “Next”.
    
3. Select “To an XPS project”, ensure that the folder chosen is the project folder, and click “Next”.
4. For the name of the peripheral, type “eth_mac”. Tick “Use version” and select the same version number that we originally created. Click “Next”. It will ask if we are willing to overwrite the existing peripheral and we should answer “Yes”.
    
5. Tick “HDL source files” and click “Next”.
    
6. Select “Use existing Peripheral Analysis Order file (*.pao)” and click “Browse”. From the project folder, go to “pcores\eth_mac_v1_00_a\data” and select the “eth_mac_v2_1_0.pao” file. Click “Next”.
    
7. On the “HDL analysis information” page, click “Next”. The wizard will mention if any errors are found in the design.
8. On the “Bus Interfaces” page, tick “PLB Slave” and click “Next”.
    
9. On the “SPLB: Port” page, click “Next”.
10. On the “SPLB: Parameter” page, click “Next”.
11. On the “Identify Interrupt Signals” page, untick “Select and Configure Interrupts” and click “Next”.
     
12. On the “Parameter Attributes” page, click “Next”.
13. On the “Port Attributes” page, click “Next”.
14. Click “Finish”.

The Ethernet MAC peripheral is now ready to use and it should be accessible through the “IP Catalog->Project Local pcores” in the XPS interface. Note that although we can access it through the IP Catalog, other projects will not find it there because it is only associated with our project, as we specified in the Peripheral Wizard.

Create an Instance of the Peripheral

Now we are ready to create an instance of the peripheral into our project.

1. From the “IP Catalog” find the “eth_mac” IP core in the “Project Local pcores” group. Right click on the core and select “Add IP”.
    
2. From the “System Assembly View” using the “Bus Interface” filter, connect the “eth_mac_0″ to the PLB bus.
    
3. Click on the “Ports” filter. Click on the “+” for “eth_mac_0″ to view its ports.
4. Click on the “Net” field for the “PHY_RESET_0″ port. Type “PHY_RESET_0″ in this field and press “Enter”. Now click again the same field and open the drop down menu. Select “Make External” and press “Enter”.
5. Click on the “Net” field for the “GTP_READY” port. Type “GTP_READY” in this field and press “Enter”. Now click again the same field and open the drop down menu. Select “Make External” and press “Enter”.
6. Click on the “Net” field for the “REFCLK_P_IN” port. Type “REFCLK_P_IN” in this field and press “Enter”. Now click again the same field and open the drop down menu. Select “Make External” and press “Enter”.
7. Click on the “Net” field for the “REFCLK_N_IN” port. Type “REFCLK_N_IN” in this field and press “Enter”. Now click again the same field and open the drop down menu. Select “Make External” and press “Enter”.
    
8. Click on the “Addresses” filter. Change the “Size” for “eth_mac_0″ to 64K. Then click “Generate Addresses”.
   

Now we have created an instance of the Ethernet MAC peripheral in our design.

Modify the Constraints file

The Ethernet MAC peripheral requires timing and pin constraints, as well as a constraint to select the RocketIO GTP we will use for a link to the PHY. The clocks used must be constrained to 125MHz while the PHY reset and GTP ready signals must be assigned to specific pins. The GTP and pins that we select here were obtained from the schematic for the ML505.

1. Click the “Project” tab and double click on the UCF file to open it.
2. Add the following lines to the end of the file:

##################################
# These constraints were adapted from the example
# design produced by CORE Generator:
# v5_emac_v1_5_example_design.ucf
##################################

CONFIG PART = 5vlx50tff1136-1;

##################################
# BLOCK Level constraints
##################################

# EMAC0 Clocking
# 125MHz clock input from BUFG
NET "*user_clk" TNM_NET = "clk_gtp";
TIMEGRP  "v5_emac_v1_5_gtp_clk" = "clk_gtp";
TIMESPEC "TS_v5_emac_v1_5_gtp_clk" = PERIOD "v5_emac_v1_5_gtp_clk" 7700 ps HIGH 50 %;
# EMAC0 Tri-speed clock input from BUFG
NET "*CLIENT_CLK_0" TNM_NET    = "clk_client0";
TIMEGRP  "v5_emac_v1_5_gtp_clk_client0"    = "clk_client0";
TIMESPEC "TS_v5_emac_v1_5_gtp_clk_client0" = PERIOD "v5_emac_v1_5_gtp_clk_client0" 7700 ps HIGH 50 %;

#-----------------------------------------------------------              
# EMAC0 Fabric Rx Elastic Buffer Timing Constraints:       - 
#-----------------------------------------------------------
NET "*GTP_DUAL_1000X_inst?RXRECCLK_0_BUFR" TNM_NET = "clk_rec_clk0";
TIMEGRP  "v5_emac_v1_5_client_rec_clk0" = "clk_rec_clk0";
TIMESPEC "TS_v5_emac_v1_5_rec_clk0"     = PERIOD "v5_emac_v1_5_client_rec_clk0" 7700 ps HIGH 50 %;

# Control Gray Code delay and skew 
INST "*GTP_DUAL_1000X_inst?rx_elastic_buffer_inst_0?rd_addr_gray_?" TNM = "rx_elastic_rd_to_wr_0";
TIMESPEC "TS_rx_elastic_rd_to_wr_0" = FROM "rx_elastic_rd_to_wr_0" TO "clk_rec_clk0" 7500 ps DATAPATHONLY;
INST "*GTP_DUAL_1000X_inst?rx_elastic_buffer_inst_0?wr_addr_gray_?" TNM = "elastic_metastable_0";
TIMESPEC "ts_elastic_meta_protect_0" = FROM "elastic_metastable_0" 5 ns DATAPATHONLY;

# Reduce clock period to allow 3 ns for metastability settling time
INST "*GTP_DUAL_1000X_inst?rx_elastic_buffer_inst_0?rd_wr_addr_gray*" TNM = "rx_graycode_0";
INST "*GTP_DUAL_1000X_inst?rx_elastic_buffer_inst_0?rd_occupancy*"    TNM = "rx_binary_0";
TIMESPEC "ts_rx_buf_meta_protect_0" = FROM "rx_graycode_0" TO "rx_binary_0" 5 ns;

##################################
# LocalLink Level constraints
##################################

# EMAC0 LocalLink client FIFO constraints.

INST "*client_side_FIFO_emac0?tx_fifo_i?rd_tran_frame_tog"    TNM = "tx_fifo_rd_to_wr_0";
INST "*client_side_FIFO_emac0?tx_fifo_i?rd_retran_frame_tog"  TNM = "tx_fifo_rd_to_wr_0";
INST "*client_side_FIFO_emac0?tx_fifo_i?rd_col_window_pipe_1" TNM = "tx_fifo_rd_to_wr_0";
INST "*client_side_FIFO_emac0?tx_fifo_i?rd_addr_txfer*"       TNM = "tx_fifo_rd_to_wr_0";
INST "*client_side_FIFO_emac0?tx_fifo_i?rd_txfer_tog"         TNM = "tx_fifo_rd_to_wr_0";
INST "*client_side_FIFO_emac0?tx_fifo_i?wr_frame_in_fifo"     TNM = "tx_fifo_wr_to_rd_0";

TIMESPEC "TS_tx_fifo_rd_to_wr_0" = FROM "tx_fifo_rd_to_wr_0" TO "v5_emac_v1_5_gtp_clk_client0" 8000 ps DATAPATHONLY;
TIMESPEC "TS_tx_fifo_wr_to_rd_0" = FROM "tx_fifo_wr_to_rd_0" TO "v5_emac_v1_5_gtp_clk_client0" 8000 ps DATAPATHONLY;

# Reduce clock period to allow 3 ns for metastability settling time
INST "*client_side_FIFO_emac0?tx_fifo_i?wr_tran_frame_tog"    TNM = "tx_metastable_0";
INST "*client_side_FIFO_emac0?tx_fifo_i?wr_rd_addr*"          TNM = "tx_metastable_0";
INST "*client_side_FIFO_emac0?tx_fifo_i?wr_txfer_tog"         TNM = "tx_metastable_0";
INST "*client_side_FIFO_emac0?tx_fifo_i?frame_in_fifo"        TNM = "tx_metastable_0";
INST "*client_side_FIFO_emac0?tx_fifo_i?wr_retran_frame_tog*" TNM = "tx_metastable_0";
INST "*client_side_FIFO_emac0?tx_fifo_i?wr_col_window_pipe_0" TNM = "tx_metastable_0";

TIMESPEC "ts_tx_meta_protect_0" = FROM "tx_metastable_0" 5 ns DATAPATHONLY;

INST "*client_side_FIFO_emac0?tx_fifo_i?rd_addr_txfer*"       TNM = "tx_addr_rd_0";
INST "*client_side_FIFO_emac0?tx_fifo_i?wr_rd_addr*"          TNM = "tx_addr_wr_0";
TIMESPEC "TS_tx_fifo_addr_0" = FROM "tx_addr_rd_0" TO "tx_addr_wr_0" 10ns;

## RX Client FIFO
# Group the clock crossing signals into timing groups
INST "*client_side_FIFO_emac0?rx_fifo_i?wr_store_frame_tog"   TNM = "rx_fifo_wr_to_rd_0";
INST "*client_side_FIFO_emac0?rx_fifo_i?rd_addr_gray*"        TNM = "rx_fifo_rd_to_wr_0";

TIMESPEC "TS_rx_fifo_wr_to_rd_0" = FROM "rx_fifo_wr_to_rd_0" TO "v5_emac_v1_5_gtp_clk_client0" 8000 ps DATAPATHONLY;
TIMESPEC "TS_rx_fifo_rd_to_wr_0" = FROM "rx_fifo_rd_to_wr_0" TO "v5_emac_v1_5_gtp_clk_client0" 8000 ps DATAPATHONLY;

# Reduce clock period to allow for metastability settling time
INST "*client_side_FIFO_emac0?rx_fifo_i?wr_rd_addr_gray_sync*" TNM = "rx_metastable_0";
INST "*client_side_FIFO_emac0?rx_fifo_i?rd_store_frame_tog"    TNM = "rx_metastable_0";

TIMESPEC "ts_rx_meta_protect_0" = FROM "rx_metastable_0" 5 ns;

##################################
# EXAMPLE DESIGN Level constraints
##################################

# Place the transceiver components. Please alter to your chosen transceiver.
INST "*GTP_DUAL_1000X_inst?GTP_1000X?tile0_rocketio_wrapper_i?gtp_dual_i" LOC = "GTP_DUAL_X0Y3";
NET REFCLK_N_IN_pin  LOC = P3;
NET REFCLK_P_IN_pin  LOC = P4;

##################################
# Additions by FPGA Developer
# http://www.fpgadeveloper.com
##################################

# GTP Ready pin
NET "GTP_READY_pin" LOC = AE24; # LED 7

# PHY Reset pin
NET "PHY_RESET_0_pin" LOC = J14; # ML505 PHY Reset 

# PHY Autonegotiate ON
INST *?v5_emac EMAC0_PHYINITAUTONEG_ENABLE = TRUE;

1. Save and close the file.

Modify the Software Application

In this example, our software application will not do anything other than send a message to Hyperterminal to let you know that it is running. All processing of Ethernet packets is done in hardware through the Ethernet MAC peripheral.

1. From the “Applications” tab, open “Sources” within the “Project: TestApp_Peripheral” tree. Open the “TestApp_Peripheral.c” source file.
2. Replace all the code in this file with the following source and save the file.

#include "xparameters.h"
#include "xbasic_types.h"
#include "xstatus.h"

int main (void)
{
  // Display message
  xil_printf("%c[2J",27);
  xil_printf("Tri-mode Ethernet MAC Loop-back");
  xil_printf(" by FPGA Developer\n\r");
  xil_printf("http://www.fpgadeveloper.com\n\r");

  // Stay in an infinite loop
  while(1){
  }
}

Download and Test the Project

1. Open a Hyperterminal window with the required settings. For the correct settings, see Hyperterminal Settings.
2. Turn on the ML505 board.
3. From the XPS software, select “Device Configuration->Download Bitstream”.

The Hyperterminal output should display the message “Tri-mode Ethernet MAC Loop-back by FPGA Developer”. When you get this message, you can continue with the following steps.

1. Open Wireshark on the PC to be used for testing. You can use any PC with a Gigabit Ethernet network card installed and working.
2. From the menu select “Edit->Preferences”. In the dialog box that opens, select “User Interface->Columns” and set the columns as shown in the screenshot below. Then click “OK”.
    
3. From the menu select “Capture->Options”. In the dialog box that opens, select the Gigabit Ethernet network card to which you will connect the ML505, then click “Start”.
    
4. Connect the CAT5 Ethernet cable between the ML505 and the PC running Wireshark.
5. You should notice that the Ethernet connection LEDs light up on both the ML505 and the PC. The connection LEDs on the PC should be on the Ethernet (RJ45) connector on the back of your PC. The connection LEDs on the ML505 are located next to the PCI edge connector and they are shown in the photo below.
   
   In order from left to right, as shown in the photo, the LEDs indicate: CONNECTION, TX, RX, 10Mbps, 100Mbps, 1000Mbps. 
6. We will produce Ethernet packets from the PC by using “ping”. From Windows, select “Start->Run” and type “cmd”. Press “Enter” and you should have a command prompt. From the command line, type “ping www.google.com”. Note that even before running “ping”, you may already see Ethernet packets in Wireshark. This can happen when your PC is trying to connect to a network.
    
7. Observe the packets in Wireshark by clicking on them. In the screenshot below, we see that the PC sent packets 1, 3 and 5, while the ML505 sent back packets 2, 4 and 6 with the MAC destination and source addresses swapped. Notice also the short time delay of 225us between the sent packet and the received copy.
    
8. Also observe that the RX and TX LEDs on the ML505 will light up at the same time, indicating that each packet received is immediately transmitted back to the sender (after swapping the MAC sender/destination addresses).

You can download the project files for this tutorial and try it on your ML50x board. Please select the file corresponding to your board, right-click on it and select “Save Link As”.

You now have a working Ethernet connection running at 1Gbps. To develop the project further, remove the address swap module and try some of these ideas:

• TCP/IP stack: Connect the Ethernet MAC LocalLink interface to read/write FIFOs and run a TCP/IP stack on the Microblaze. A lightweight TCP/IP stack for the Microblaze is discussed in the application note XAPP1026.
• Ethernet-to-Aurora Bridge: Add an Aurora core to the design and create a transparent bridge.
• Gigabit PC interface: Don’t bother with TCP/IP and use the Ethernet MAC for a high-speed PC link.
• Video/audio processing: Use VLC Media Player to stream video into the Virtex-5, then process it and send it out of the DVI interface.

Remember that the interface to the Ethernet MAC is LocalLink, so you can virtually hook up anything with an 8-bit wide LocalLink interface.

Here’s the next product in Opsero’s growing lineup of FPGA I/O cards: the Quad Gigabit Ethernet FMC. This low-pin-count FMC is loaded with four Marvell Gigabit Ethernet PHYs and enables FPGA networking applications on the ZedBoard and other LPC carriers. A demo design for the ZedBoard is available on Github at the link below and further demos will become available in the next few weeks:

https://github.com/fpgadeveloper/zedboard-qgige

Look out for more details including the technical specs, schematics and more on the soon to be launched product page. (Update 2014-11-06: the product page is now open)

If you want more information about the Quad Gigabit Ethernet FMC, or if you are interested in purchasing the board, please don’t hesitate to contact me.

Announcing that the new Ethernet FMC, a Quad Gigabit Ethernet FPGA Mezzanine Card (FMC) is now available to buy. The board is designed for easy integration with the ZedBoard, with open-source example designs so you can start designing your product sooner. The low-pin-count FMC features 4 Marvell PHYs, a 125MHz oscillator and a quad RJ45 with integrated magnetics.

The Ethernet FMC is compliant to the Vita 57.1 FMC standard except for its height (due to the RJ45 connector) and its length. The FMC is electrically and mechanically compatible with the ZedBoard and all the Xilinx evaluation boards.

Features:

• 4 x Gigabit Ethernet ports
• Vita 57.1 electrically compliant low-pin-count FMC
• 125MHz clock oscillator
• Example designs for the ZedBoard (Vivado)

Applications:

• Gigabit Ethernet Switch
• Ethernet Traffic Monitor
• Latency Measurement
• High-speed Congestion Control

Checkout the product page for more information.

Tutorial Overview

In this two-part tutorial, we’re going to create a multi-port Ethernet design in Vivado 2015.4 using both the GMII-to-RGMII and AXI Ethernet Subsystem IP cores. We’ll then test the design on hardware by running an echo server on lwIP. Our target hardware will be the ZedBoard armed with an Ethernet FMC, which adds 4 additional Gigabit Ethernet ports to our platform. Ports 0 to 2 of the Ethernet FMC will connect to separate AXI Ethernet Subsystem IPs which will be configured in DMA mode. Port 3 of the Ethernet FMC will connect to GEM1 of the Zynq PS through the GMII-to-RGMII IP, while the on-board Ethernet port of the ZedBoard will connect to GEM0.

Requirements

To go through this tutorial, you’ll need the following:

• Vivado 2015.4 (see note below)
• ZedBoard
• Ethernet FMC (standard or robust model will work)
• Platform Cable USB II (or equivalent JTAG programmer)

Note: The tutorial text and screenshots are suitable for Vivado 2015.4 however the sources in the Git repository will be regularly updated to the latest version of Vivado.

Change Vivado’s default language

Before creating our project, we need to make sure that Vivado is configured to use VHDL as it’s default language. We wont be writing any HDL code, however the constraints that we use will be dependent on the project language being set to VHDL, so it’s important that we set this:

1. Open Vivado.
2. From the menu, select Tools->Options.
3. In the “General” tab select target language : VHDL.

Create a new Vivado project

Follow these steps to create a new project in Vivado:

1. From the welcome screen, click “Create New Project”.
2. Specify a folder for the project. I’ve created a folder named “zedboard_qgige”. Click “Next”.
3. For the Project Type window, choose “RTL Project” and tick “Do not specify sources at this time”. Click “Next”.
4. For the Default Part window, select the “Boards” tab and then select the “ZedBoard Zynq Evaluation and Development Kit” and click “Next”.
5. Click “Finish” to complete the new project wizard.

Setup the Zynq PS

We start off the design by adding the Zynq PS (aka. Processor System) and make the connections specified by the ZedBoard board definition file which is included with Vivado 2015.4.

1. From the Vivado Flow Navigator, click “Create Block Design”.
2. Specify a name for the block design. Let’s go with the default “design_1” and leave it local to the project. Click “OK”.
3. In the Block Design Diagram, you will see a message that says “This design is empty. Press the (Add IP) button to add IP.”. Click on the “Add IP” icon either in the message, or in the vertical toolbar.
4. The IP catalog will appear. Go to the end of the list and double click on “ZYNQ7 Processing System” – it should be the second last on the list.
5. In the Block Design Diagram, you will see a message that says “Designer Assistance available. Run Block Automation”. Click on the “Run Block Automation” link.
6. Block Automation uses the board definition file for the ZedBoard to make connections and pin assignments to external hardware such as the DDR and the on-board Ethernet port. Just make sure that  “Apply Board Preset” is ticked and click OK.
7. Now our block diagram has changed and we can see that the DDR and FIXED_IO are connected externally. We can now configure the Zynq PS for our specific needs. Double click on the Zynq PS block to open the Re-customize IP window.
8. From the Page Navigator, select “Clock Configuration” and open the “PL Fabric Clocks” tree. Notice that “FCLK_CLK0” is enabled by default and set to 100MHz, this will serve as the clock for our AXI interfaces. Now enable “FCLK_CLK1” and “FCLK_CLK2” and set them to 125MHz and 200MHz respectively. The FCLK_CLK1 (125MHz) will be needed by the AXI Ethernet Subsystem blocks and it will be used to clock the RGMII interfaces. FCLK_CLK2 (200MHz) will be required by both the GMII-to-RGMII and AXI Ethernet Subsystem IPs and it is needed to clock the IDELAY_CTRLs.
9. Now from the Page Navigator, select “PS-PL Configuration”. By default the Master AXI GP0 interface should be enabled as you can see in the image. You must also enable the High Performance Slave AXI HP0 interface as shown. The HP Slave AXI Interface provides a high-bandwidth connection to the DDR3 memory controller – this will be needed by the DMA engines which we will create after we add the AXI Ethernet Subsystem blocks to our design.
10. The last thing to do is to enable interrupts. From the Page Navigator, select “Interrupts” and tick to enable “Fabric Interrupts” then “IRQ_F2P[15:0]”. Interrupts will be generated by all the Ethernet IPs and by the DMA engine IPs.
11. Now click “OK” to close the Re-customize IP window.
12. You will notice that the PS block has gotten a bit bigger and it has more ports. Connect FCLK_CLK0 (100MHz) to the GP Master AXI clock input (M_AXI_GP0_ACLK) by dragging a trace from one pin to the other.This action will draw a wire between the pins and make the connection.
13. Also connect the FCLK_CLK0 to the HP Slave AXI Interface clock input (S_AXI_HP0_ACLK).
14. Now open the IP Catalog and add 3 x AXI 1G/2.5G Ethernet Subsystem IPs to the design (you will have to add one at a time). Once you have done this, you should have three AXI Ethernet Subsystem blocks in your design: “axi_ethernet_0”, “axi_ethernet_1” and “axi_ethernet_2”.
15. To wire the AXI Ethernet Subsystem blocks in DMA mode, we’ll use the block automation feature, however before running this, we want to configure the “shared logic” option of the cores first. The AXI Ethernet Subsystem IP is designed with the option to include “shared logic” in the core. The shared logic includes an IDELAY_CTRL to control the IODELAYs on the RGMII interface, as well as an MMCM to generate a 90 degree skewed clock for generation of the RGMII TX clock. When we use multiple AXI Ethernet Subsystem blocks in the one design, we can save on resources by having only one of those cores include the “shared logic”. The core containing the “shared logic” will naturally share the IDELAY_CTRL with the other cores, and it will have outputs for the clocks generated by the MMCM so that it can share them too. Let’s make “axi_ethernet_0” be the one that contains the shared logic, so double click on it to bring up the Re-customize IP window.
16. Go to the “Shared Logic” tab (don’t worry about any of the other options for now). Select the “Include Shared Logic in Core” option and click OK.
17. Now open the Re-customize IP window for “axi_ethernet_1”, go to the “Shared Logic” tab and select the “Include Shared Logic in IP Example Design” option and click OK. Do the same for “axi_ethernet_2”.
18. Now we can wire up the Ethernet blocks by using the block automation feature. Notice there is a message in your block diagram saying “Designer Assistance available. Run Block Automation”. Click on the “Run Block Automation” link.
19. In the “Run Block Automation” window, you will have automation options for each of the Ethernet blocks. Tick to enable all of them, then select them one by one and make sure that they are each configured for an “RGMII” physical interface and a “DMA” connection to the AXI Streaming Interfaces. By default they will all be configured for GMII so it is important to set the physical interface correctly here and for each one of them. Then click OK.
20. After the block automation has run its course, you will notice that it has added a Clocking Wizard block called “axi_ethernet_0_refclk”. This block generates a 125MHz and 200MHz clock to feed the Ethernet blocks, however we will be using the Zynq PS to generate those clocks, so we don’t need this block. Click once on the “axi_ethernet_0_refclk” block and press Delete to remove it from the block diagram.
21. We can now use the Connection Automation feature to wire up our AXI interfaces. Click “Run Connection Automation” from the block diagram.
22. Like before, tick to enable ALL of the connections. We then we need to select each of the interfaces one-by-one and choose the right settings. Luckily the defaults are good for us in Vivado 2015.4, but check the screenshots below if you are using a different version. Then click OK.
23. When the automation feature has run its course, you will notice that again you have the option to “Run Connection Automation”. Maybe this will not be the case in future versions of Vivado, but it is the case for 2015.4. So again click “Run Connection Automation”, tick to enable all the interfaces for automation and make sure the settings are correct (defaults are good for 2015.4). They should all be configured to connect to the HP Slave AXI Interface (S_AXI_HP0) and use the “Auto” clock connection.
24. Now it’s time to add the GMII-to-RGMII block for Port 3 of the Ethernet FMC. Open the IP Catalog and double click on “Gmii to Rgmii”.
25. Double click on the GMII-to-RGMII block to open the Re-customize IP window.
26. In the “Core Functionality” tab, tick “Instantiate IDELAYCTRL in design”, set the PHY Address to 8 and select the option “Skew added by PHY”. Notice that the GMII-to-RGMII core has an MDIO input and an MDIO output. Why does the MDIO bus have to pass through the core? That’s because the GMII-to-RGMII core has logic that sits on the MDIO bus to receive commands from the MAC for configuration. The PHY address we specify here allows us to give the core a unique address on the MDIO bus, and it is very important that the address be different to that of the external PHY. On the Ethernet FMC, all the PHYs are configured with address 0, so we can give the GMII-to-RGMII core an address of 8 without creating a bus conflict. As for the “Skew added by the PHY” option, this concerns the RGMII transmit clock. Some PHYs, including the 88E1510 on the Ethernet FMC, have a feature to add a delay to the incoming RGMII TX clock so that it aligns well for sampling the incoming RGMII transmit data. The GMII-to-RGMII core allows us to specify where the skew is added: in the PHY or in the FPGA fabric (MMCM). The skew should be added by one or the other, never both, or the clock will be poorly aligned for sampling the data and the RGMII interface will fail.
27. Now open the “Shared Logic” tab and select “Include Shared Logic in Core”.
28. To connect the GMII-to-RGMII core to the PS, we need to enable GEM1 in the PS. Double click on the Zynq PS block and select “MIO Configuration” in the Page Navigator. Tick to enable “ENET 1” and select “EMIO” (Extended Multiplexed Input/Output). Selecting EMIO allows us to route GEM1 through to the FPGA fabric, so that we can then connect it to our GMII-to-RGMII core and then out to the Ethernet FMC.
29. Now you should see two extra ports on the Zynq PS block: “GMII_ETHERNET_1” and “MDIO_ETHERNET_1”. Make a connection between “MDIO_GEM” of the GMII-to-RGMII block and “MDIO_ETHERNET_1” of the Zynq PS.
30. Now make a connection between “GMII” of the GMII-to-RGMII block and “GMII_ETHERNET_1” of the Zynq PS.
31. Now we need to make the “MDIO_PHY” and “RGMII” ports external so that they can connect to the Ethernet FMC. Right click on each of these interfaces and select “Make External”.
32. Now find the external ports on the right hand side of the block diagram. We’ll need to rename them so that the names fit with the constraints that we will later add to the project. Click first on the “MDIO_PHY” port and rename it “mdio_io_port_3”. Then click on the “RGMII” port and rename it “rgmii_port_3”. The port name can be changed in the “External Interface Properties” window that normally sits just below the “Design” window and to the left of the block diagram (see image below).
33. Once you have changed the names, your ports should now look like this.
34. The GMII-to-RGMII block doesn’t provide us with a reset signal for the PHY, so we have to add some logic to provide that signal. Open the IP Catalog and add a “Utility Reduced Logic” IP.
35. Double click on the “util_reduced_logic_0” block and set the “C Size” to 1 and the “C Operation” to “and”. Then click OK.
36. Now connect the input of the “util_reduced_logic_0” block to the active-low peripheral reset output of the Processor System Reset block.
37. Now right click on the output of the “util_reduced_logic_0” block and select “Make External”.
38. The external port will have been named “Res” by default. Change this name to “reset_port_3” so that it matches the constraints we will later add to the project.
39. The MDIO, RGMII and reset ports of the 3 x AXI Ethernet Subsystem blocks will have already been externalized during the automation process, however they will have been given odd names so we need to change those names to match the constraints that we will later add to the project. Go through the ports one-by-one and rename them as follows:
    1. axi_ethernet_0 should have its external ports named “mdio_io_port_0”, “rgmii_port_0” and “reset_port_0”.
    2. axi_ethernet_1 should have its external ports named “mdio_io_port_1”, “rgmii_port_1” and “reset_port_1”.
    3. axi_ethernet_2 should have its external ports named “mdio_io_port_2”, “rgmii_port_2” and “reset_port_2”.
40. Now open the IP Catalog and add a Concat (concatenate) IP to the design. The concatenate IP takes a series of single inputs and concatenates them into a vector output. We will need this IP to be able to connect all the interrupts to the IRQ_F2P[0:0] vector input of the PS.
41. Double click on the Concat block and set the number of ports to 12. Then click OK.
42. Now we must connect all the interrupts to the Concat IP. One-by-one, go through and make all the following connections. Note that the order of pin assignment is not important because it will all be transferred to the SDK in the hardware description and be correctly mapped by the BSP.
    1. Connect axi_ethernet_0_dma/mm2s_introut to xlconcat_0/In0
    2. Connect axi_ethernet_0_dma/s2mm_introut to xlconcat_0/In1
    3. Connect axi_ethernet_1_dma/mm2s_introut to xlconcat_0/In2
    4. Connect axi_ethernet_1_dma/s2mm_introut to xlconcat_0/In3
    5. Connect axi_ethernet_2_dma/mm2s_introut to xlconcat_0/In4
    6. Connect axi_ethernet_2_dma/s2mm_introut to xlconcat_0/In5
    7. Connect axi_ethernet_0/mac_irq to xlconcat_0/In6
    8. Connect axi_ethernet_0/interrupt to xlconcat_0/In7
    9. Connect axi_ethernet_1/mac_irq to xlconcat_0/In8
    10. Connect axi_ethernet_1/interrupt to xlconcat_0/In9
    11. Connect axi_ethernet_2/mac_irq to xlconcat_0/In10
    12. Connect axi_ethernet_2/interrupt to xlconcat_0/In11
43. Now connect the Concat output to the IRQ_F2P input of the Zynq PS.
44. Now let’s connect FCLK_CLK2, the 200MHz clock, to the Ethernet blocks. First connect FCLK_CLK2 to the “ref_clk” pin of “axi_ethernet_0”.
45. Then connect FCLK_CLK2 to the “clkin” pin of the GMII-to-RGMII block. Now for those who are curious, you probably noticed that the GMII-to-RGMII block only has one clock input (clkin). This input must be connected to 200MHz which will be used to clock the IDELAY_CTRL, but also it will be used to generate three other clocks: 125MHz, 25MHz and 2.5MHz which are used for link speeds 1Gbps, 100Mbps, 10Mbps respectively. The actual link speed is determined by the PHY during the autonegotiation process and it is up to the processor to read the link speed from the PHY and then pass this value onto the GMII-to-RGMII core so that it uses the appropriate clock. By default, it is set to use the 2.5MHz clock for a link speed of 10Mbps.
46. Now let’s connect the “tx_reset” and “rx_reset” ports to the peripheral reset signal. Remember to connect BOTH of them (one at a time).
47. Now we need to connect the 125MHz clock to the “gtx_clk” port of the AXI Ethernet Subsystem block “axi_ethernet_0” (the one containing the shared logic). We did enable FCLK_CLK1 for this purpose, and you can make that connection if you wish, but for this tutorial we will explore another possibility. The Ethernet FMC has an on-board 125MHz oscillator which can also be used to supply “gtx_clk”. In order to use it, we just need to add a differential buffer to our design. Open the IP Catalog and add a “Utility Buffer” to the design.
48. Connect the output of the buffer to the “gtx_clk” input of “axi_ethernet_0”.
49. Now click on the plus (+) symbol on the input of the buffer to show the differential inputs.
50. Now right-click on each of the individual inputs of the buffer and select “Make External”. You should end up with two external input ports named “IBUF_DS_P[0:0]” and “IBUF_DS_N[0:0]”.
51. Rename those external input ports to “ref_clk_p” and “ref_clk_n” respectively.
52. Now there is only one more thing to add. The Ethernet FMC has two inputs that are used to enable the on-board 125MHz oscillator and to select it’s frequency (it can alternatively be set to 250MHz). We need to add some constants to our design to enable the oscillator and to set it’s output frequency to 125MHz. Open the IP Catalog and add two Constant IPs.
53. By default, they will be set to constant outputs of 1, which is exactly what we need. So all we must do is make their outputs external and rename them to “ref_clk_oe” and “ref_clk_fsel”. The result should be as shown in the image below.
54. Save the block diagram by clicking “File->Save Block Design”.

Create the HDL wrapper

Our Vivado block diagram is complete and we now need to create a HDL wrapper for the design.

1. Open the “Sources” tab from the Block Design window.
2. Right-click on “design_1” and select “Create HDL wrapper” from the drop-down menu.
3. From the “Create HDL wrapper” window, select “Let Vivado manage wrapper and auto-update”. Click “OK”.

Add the constraints file

The last thing we need to add to our project will be the constraints. The constraints file contains:

• Pin assignments for all the external ports of our block design, which in our case are the pins that are routed to the FMC connector and through to our Ethernet FMC
• A definition for the 125MHz reference clock that comes in from the Ethernet FMC
• IODELAY grouping constraints to assign each port to one of two groups corresponding to the I/O bank that it occupies. We don’t want the tools trying to group all the ports to the same IDELAY_CTRL, but rather there should be one instantiated for each I/O bank – in our case, there is one instantiated in the “axi_ethernet_0” and another in “gmii_to_rgmii_0”.

Follow these steps to add the constraints file to your project:

1. Download the constraints file from this link: Constraints for ZedBoard and Ethernet FMC using GMII-to-RGMII and AXI Ethernet
2. Save the constraints file somewhere on your hard disk.
3. From the Project Manager, click “Add Sources”.
4. Then click “Add or create constraints”.
5. Then click “Add files” and browse to the constraints file that you downloaded earlier.
6. Tick “Copy constraints files into project” and click Finish.
7. You should now see the constraints file in the Sources window.

Sources Git repository

Sources for re-generating this project automatically can be found on Github at the links below. There is a version of the project for the ZedBoard and the MicroZed. There is also a version that uses only the AXI Ethernet Subsystem IP.

• ZedBoard using GMII-to-RGMII and AXI Ethernet Subsystem
• ZedBoard using AXI Ethernet Subsystem only
• MicroZed using GMII-to-RGMII and AXI Ethernet Subsystem

Instructions for re-generating those projects can be found in this post: Version control for Vivado projects. We will also discuss that in the following tutorial, as well as testing the projects on actual hardware.

Testing the project on hardware

In the second part of this tutorial (yet to come) we will generate the bitstream for this project, export it to the SDK and then test an echo server application on the hardware. The echo server application runs on lwIP (light-weight IP), the open source TCP/IP stack for embedded systems.

If you have any questions about this tutorial, or if you run into problems, please leave me a comment below.

When designing a network tap on an FPGA, the logical place to start is the pass-through between two Ethernet ports. In this article, I’ll discuss a convenient way to connect two Ethernet ports at the PHY-MAC interface, which will form the basis of a network tap. The pass-through will be designed in Vivado for the ZedBoard combined with an Ethernet FMC. In future articles, I’ll discuss other aspects of an FPGA network tap design, including monitor ports, packet filtering, and opportunities for hardware acceleration in the FPGA.

Pass-through at the MAC interface (GMII, RGMII or SGMII)

The criteria for an ideal pass-through are:

• it must be completely transparent to all devices communicating over the link,
• it must preserve the fidelity of the link, and ideally,
• it should add very little latency to the link.

From those criteria we could suppose that if we could simply tap the wires of the Ethernet cable, we’d have our ideal tap. Unfortunately, due to the complexity of Gigabit Ethernet signals, we can’t do that, instead we have to break the link and connect each end to it’s own Ethernet PHY. The pass-through is implemented on the other end of the PHYs, or the MAC interface which is typically one of the following standards: GMII, RGMII or SGMII. In the case of the Ethernet FMC, which uses 4x Marvell 88E1510 Ethernet PHYs, we’re dealing with the RGMII interface.

RGMII signals are double-data-rate (DDR) and so in order to bring the data into our FPGA fabric and send it back out, we need to use the IDDR and ODDR primitives. Fortunately, there is an IP that implements the RGMII interface for us and provides us with a single-data-rate interface which we can use for the pass-through and for “tapping”. The GMII-to-RGMII IP core, included with Vivado, converts an RGMII interface, to a GMII interface. To implement our pass-through, all we have to do is instantiate two GMII-to-RGMII converters, route them to two separate Ethernet PHYs and loop together the two GMII interfaces.

The block diagram above illustrates the general idea. Port 0 and port 1 of the Ethernet FMC are each connected to a GMII-to-RGMII converter, and the GMII interfaces are passed through to the opposite port.

Use FIFOs to connect the GMII interfaces

When connecting one GMII interface to another, you will notice that the transmit interface has a separate clock to the receive interface. The GMII TX data, TX enable and TX error signals are all synchronous to the TX clock, whereas the GMII RX data, RX valid and RX error signals are all synchronous to the RX clock. So you can’t directly connect the GMII transmit interface to the GMII receive interface – you have to use proper clock domain crossing. The easy way to do that is by using a FIFO with independent read and write clocks – you’ll need two of them, one for each direction of data flow.

Wire the FIFOs as elastic buffers

The natural way to connect FIFOs to the transmit and receive interfaces is to use the “rx_dv” (RX valid) output of the GMII interfaces to drive the “write enable” inputs of the FIFOs, and to use the “valid” output of the FIFOs to drive the “tx_en” (TX enable) inputs of the GMII interface. However, in our application, there is a problem with this method. If even momentarily the FIFO is being read slightly faster than it is being written to, you will have occasions where the FIFO is empty for one clock cycle and forced to de-assert the “valid” signal. This is a problem because the GMII interface “enable” and “valid” signals are only supposed to be de-asserted at the end of a packet, so this gap effectively terminates the Ethernet packet that you are feeding to the PHY. The better solution is to feed the “enable” and “valid” signals through the FIFOs, and to design the FIFOs as elastic buffers. Remember that once you decide that a FIFO will be written to and read from constantly, using two independent clocks, it must be designed as an elastic buffer or you risk losing data due to the FIFO reaching the full or empty state. In the elastic buffer solution, we still use our “tx_en” and “rx_dv” signals, but we use them to determine what data the elastic buffer can discard at the write interface (when it’s too full), as well as when the elastic buffer can momentarily halt the read interface (when it’s too empty). An elastic buffer is not perfect and it relies on a certain amount of redundancy being present in the data, but in typical Ethernet applications, there is enough time between packets that the job of designing a reliable elastic buffer is quite simple.

So when you want to wire up a FIFO as a simple elastic buffer, there are two things to setup:

1. Programmable full and empty outputs

These signals will tell us when the FIFOs are too full or too empty and they allow us to keep the FIFO occupancy within a certain range. Typically that “range” is centered at the mid-point of the FIFO, for example, if our FIFO contains 1000 words, then we could set our desired occupancy to be between 400 and 600. In this case, the programmable full output would be set to 600, and the programmable empty output would be set to 400.

2. Write enable and read enable logic

The write and read enable inputs must be connected to logic functions that will throttle the FIFO, filling it up when it gets too empty and emptying it when it gets too full. The functions are:

• write enable <= NOT prog_full OR rx_valid
• read enable <= NOT prog_empty OR tx_valid

Configuring the GMII-to-RGMII converter

For the GMII-to-RGMII converter to operate properly, we have to let it know the actual link speed that was setup by the PHY during auto-negotiation. But how do we communicate this information to the core?

You may have noticed that the GMII-to-RGMII core contains two MDIO ports, one of which is normally connected to the MAC, and the other which is normally externalized and connected to the PHY. The GMII-to-RGMII core “sits” on the MDIO bus, as though it were another PHY, and it can be configured over that MDIO bus. So we communicate the link speed information to the core over the MDIO bus and the typical sequence is as follows:

1. Trigger the auto-negotiation sequence in the PHY (optional)
2. We read the actual link speed from the PHY after auto-negotiation has completed
3. We write the actual link speed to the GMII-to-RGMII core

The last step involves writing to a specific register within the GMII-to-RGMII core with a value that corresponds to the link speed. To do this we need the address of the register to write to (0x10) and the “PHY” address of the GMII-to-RGMII core (I quote the word PHY because the core is not a PHY). The “PHY” address of the GMII-to-RGMII core is specified in Vivado, and is 8 by default. In order to communicate with two GMII-to-RGMII cores in our design, we have connected one of the MDIO “inputs” to GEM1 of the Zynq PS. We then connected the MDIO “output” to the MDIO “input” of the second GMII-to-RGMII converter (see block diagram above). This way, we can configure both GMII-to-RGMII converters using only the MDIO port of GEM1. In Vivado, we configure the GMII-to-RGMII cores to have different “PHY addresses”, specifically 7 and 8, so that we don’t create a bus conflict.

Depending on the established link speed, we need to write the following values to register 0x10 of both of the GMII-to-RGMII converters:

• For a link speed of 1Gbps, we need to write 0x140.
• For a link speed of 100Mbps, we need to write 0x2100.
• For a link speed of 10Mbps, we need to write 0x100.

For reliable operation, the link on Port 0 should be the same speed as that on Port 1, ie. don’t try to use this pass-through to connect networks of different speeds.

Sources Git repository

The sources for re-generating this project automatically can be found on Github at the link below.

Ethernet FMC Network Tap Github Source Code

If you want to better understand how the sources are organized, you can read this post: Version control for Vivado projects.

Next on the FPGA network tap

In the next post on the FPGA network tap, we’ll hook up the other two ports of the Ethernet FMC as monitor ports which will enable “listening” by a third device. Port 2 will send a copy all packets going in one direction, while port 3 will send a copy of all packets going in the other direction, so the result will be a full gigabit network tap. We’ll also hook the ports up to soft TEMAC IPs and look at filtering the packets within the FPGA fabric.

Tutorial Overview

This tutorial is the follow-up to Using AXI Ethernet Subsystem and GMII-to-RGMII in a Multi-port Ethernet design. In this part of the tutorial we will generate the bitstream, export the hardware description to the SDK and then test the echo server application on our hardware. The echo server application runs on lwIP (light-weight IP), the open source TCP/IP stack for embedded systems. Our hardware platform is the Avnet ZedBoard combined with the Ethernet FMC.

Regenerate the Vivado project

Firstly, for those of you who did not follow the first part of this tutorial, we will use the scripts in the Git repository for this project to regenerate the Vivado project. If you followed the first part of the tutorial correctly, you should not need to complete this step. Please note that the Git repository is regularly updated for the latest version of Vivado, so you must download the last “commit” for the version of Vivado that you are using.

1. Download the sources from Github here: https://github.com/fpgadeveloper/zedboard-qgige
2. Depending on your operating system:
   • If you are using a Windows machine, open Windows Explorer, browse to the “Vivado” folder within the sources you just downloaded. Double-click on the “build.bat” file to run the batch file.
   • If you are using a Linux machine, run Vivado and then select Window->Tcl Console from the welcome screen. In the Tcl console, use the “cd” command to navigate to the “Vivado” folder within the sources you just downloaded. Then type “source build.tcl” to run the build script.
3. Once the script has finished running, the Vivado project should be regenerated and located in the “Vivado” folder. Run Vivado and open the newly generated project.

If you did not follow the first part of this tutorial, you may want to open the block diagram and get familiar with the design before continuing.

Generate the bitstream

When you are ready to generate the bitstream, click “Generate Bitstream” in the Flow Navigator.

Once the bitstream is generated, the following window will appear. Select “View Reports” and click “OK”.

Export the hardware to SDK

When the bitstream has been generated, we can export it and the hardware description to the Software Development Kit (SDK). In the SDK we will be able to generate the echo server example design and run it on our hardware.

1. In Vivado, from the File menu, select “Export->Export hardware”.
2. In the window that appears, tick “Include bitstream”, select Export to “Local to Project”, and click “OK”.
3. From the File menu, select “Launch SDK”.
4. In the window that appears, you need to specify the location of the hardware description and the location of the SDK workspace. We specified earlier to generate the hardware description local to the project (including bistream), so the Exported location must be “Local to Project”. By preference, we choose to create the SDK workspace local to the project, but you can change this if you wish. Click “OK”.

At this point, the SDK loads and a hardware platform specification will be created for your design.

Create the Echo Server application

At this point, your SDK workspace should contain only the hardware description and no applications:

We add the echo server application by selecting New->Application Project from the File menu.

In the New Project wizard, we want to name the application appropriately, so type “echo_server” as the project name then click “Next”.

The next page allows you to create the new application based on a template. Select the “lwIP Echo Server” template and click “Finish”.

The SDK will generate a new application called “echo_server” and a Board Support Package (BSP) called “echo_server_bsp”, both of which you will find in the Project Explorer as shown below.

By default, the SDK is configured to build the application automatically.

Modify the application

The echo server template application will be setup to run on the first AXI Ethernet Subsystem block in our design. This corresponds to PORT0 of the Ethernet FMC. We want to add some code to the application to allow us to select a different port if we choose.

1. Open the “main.c” file from the echo_server source folder.
2. After the last “#include” statement, add the following code:

#include "xlwipconfig.h"

/* Set the following DEFINE to the port number (0,1,2 or 3)
* of the Ethernet FMC that you want to hook up
* to the lwIP echo server. Only one port can be connected
* to it in this version of the code.
*/
#define ETH_FMC_PORT 0

/*
* NOTE: When using ports 0..2 the BSP setting "use_axieth_on_zynq"
* must be set to 1. When using port 3, it must be set to 0.
* To change BSP settings: right click on the BSP and click
* "Board Support Package Settings" from the context menu.
*/
#ifdef XLWIP_CONFIG_INCLUDE_AXIETH_ON_ZYNQ
#if ETH_FMC_PORT == 0
#define EMAC_BASEADDR XPAR_AXIETHERNET_0_BASEADDR  // Eth FMC Port 0
#endif
#if ETH_FMC_PORT == 1
#define EMAC_BASEADDR XPAR_AXIETHERNET_1_BASEADDR  // Eth FMC Port 1
#endif
#if ETH_FMC_PORT == 2
#define EMAC_BASEADDR XPAR_AXIETHERNET_2_BASEADDR  // Eth FMC Port 2
#endif
#else /* XLWIP_CONFIG_INCLUDE_AXIETH_ON_ZYNQ is not defined */
#if ETH_FMC_PORT == 3
#define EMAC_BASEADDR XPAR_XEMACPS_1_BASEADDR  // Eth FMC Port 3
#endif
#endif

3. Then go down to where the define PLATFORM_EMAC_BASEADDR is used, and replace it with EMAC_BASEADDR.

When you save the “main.c” file, the SDK should automatically start rebuilding the application.

Modify the Libraries

The BSP for this project will also have to be modified slightly, at least for Vivado 2015.4 and older versions. There are a few reasons for these modifications, but we would be going off-track to discuss those reasons in detail at this point. The modifications that apply to you will be found in the “README.md” file of the sources that you downloaded earlier. If you are using the latest version of Vivado, you can simply refer to the instructions on the front page of the Git repository.

I strongly recommend that you perform these modifications to the sources in the Vivado installation files – not the sources in the BSP of your SDK workspace. The reason is that the BSP sources will be written-over with the original sources every time that you re-build the BSP – so you’re better off modifying them at the true source.

Note: These modifications are specific to using the echo server application on the Ethernet FMC. If you are not using the Ethernet FMC, you may not need to make these modifications and you’re better off leaving the library sources as they are.

Setup the hardware

To setup our hardware, we need to configure the ZedBoard for configuration by JTAG, we need to set the VADJ voltage to the appropriate value and we need to correctly attach the Ethernet FMC. Follow these instructions to ensure that your setup is correct:

1. On the ZedBoard, set the JP7, JP8, JP9, JP10 and JP11 jumpers all to the SIG-GND position. This sets it for configuration by JTAG.
2. Set the VADJ select jumper (J18) to either 1.8V or 2.5V, depending on the version of Ethernet FMC that you are using. We are using the 2.5V version.
3. Connect the Ethernet FMC to the FMC connector of the ZedBoard. Apply pressure only to the area above and below the connector – you should feel the two connectors “snap” together.
4. Now we need to use two screws to fix the Ethernet FMC to the ZedBoard – you should find two M2.5 x 4mm screws included with the ZedBoard. Turn the ZedBoard upside down and use a Phillips head screwdriver to fix the Ethernet FMC to the ZedBoard. Please do not neglect this step, it is very important and will protect your hardware from being damaged in the event that the Ethernet FMC hinges and becomes loose. The FMC connector is not designed to be the only mechanical fixation between the carrier and mezzanine card, the screws are necessary for mechanical and electrical integrity.
5. Turn the ZedBoard around so that it is sitting the right way up.
6. Connect the USB-UART (J14) to a USB port of your PC.
7. Connect a Platform Cable USB II programmer (or similar device) to the JTAG connector. Connect the programmer to a USB port of your PC. Alternatively, if you don’t have a programmer, you can connect a USB cable to the J17 connector of the ZedBoard.
8. Connect PORT0 of the Ethernet FMC to a gigabit Ethernet port of your PC.
9. Now plug the ZedBoard power adapter into a wall socket and then into the ZedBoard.
10. Switch ON the power to the board. You should see the “POWER” LED on the ZedBoard turn on.

Test the Echo Server on hardware

To be able to read the output of the echo server application, we need to use a terminal program such as Putty. Use the following settings:

• Comport – check your device manager to find out what comport the ZedBoard popped up as. In my case, it was COM16 as shown below.
• Baud rate: 115200bps
• Data: 8 bits
• Parity: None
• Stop bits: 1

With the terminal program open, we can now load our ZedBoard with the bitstream and then run the echo server application.

1. In the SDK, from the menu, select Xilinx Tools->Program FPGA.
2. In the Program FPGA window, we select the hardware platform to program. We have only one hardware platform, so click “Program”.
3. The bitstream will be loaded onto the Zynq and we are ready to load the software application. Select the “echo_server” folder in the Project Explorer, then from the menu, select Run->Run.
4. In the Run As window, select “Launch on Hardware (GDB)” and click “OK”.
5. The application will be loaded on the Zynq PS and it will be executed. The terminal window should display this output from the echo server:

The output indicates that:

• The PHY auto-negotiation sequence has completed
• The auto-negotiated link-speed is 1Gbps
• The DHCP timeout was reached, indicating that the application was not able to get an IP address from a DHCP server
• The auto-assigned IP address is 192.168.1.10

Now that the application is running successfully, we can test the echo server by sending packets from our PC to the ZedBoard and looking at what gets sent back.

Ping Test

All Ethernet devices are required to respond to ping requests, so this is a very simple and easy test to perform using your computer. Just open a command window and type “ping 192.168.1.10”.

Packet Echoing

To test that the echo server is actually doing its job and echoing received packets, you will have to install software that allows you to send and receive arbitrary packets. The software that I use is called Packet Sender and can be downloaded here. Once the software is installed, follow the instructions below to send and receive packets:

1. Run Packet Sender.
2. Create a new packet to send using these parameters and then click “Save”:
   • Name: Test packet
   • ASCII: follow the white rabbit
   • IP Address: 192.168.1.10
   • Port: 7
   • Resend: 0
3. The packet details will be saved in the Packets tab, and we can now click on the “Send” button to send that packet whenever we want. Click “Send” and see what happens.
4. If everything went well, the Traffic Log tab should display two packets: one sent by our computer and one received by our computer. They should both occur almost instantaneously, so if you only see one, you’ve probably got a problem with your setup.

If you want to experiment, you can play around with the software by sending more packets, or different kinds of packets.

Changing ports

So far we’ve been using PORT0 of the Ethernet FMC to test the echo server, but suppose we wanted to use one of the other ports 1,2 or 3. You can configure the port on which to run lwIP by setting the ETH_FMC_PORT define that we added earlier to the main.c file of the SDK application. Valid values for ETH_FMC_PORT are 0,1,2 or 3.

One other thing to be aware of is the BSP setting called “use_axieth_on_zynq”. This parameter specifies whether the BSP will be used with AXI Ethernet Subsystem or with something else: Zynq GEM, Ethernet lite, etc. Remember that in our Vivado design we connected ports 0, 1 and 2 to an AXI Ethernet Subsystem block, and we connected port 3 to the GEM1 of the Zynq PS. Therefore, when selecting the port on which you wish to run lwIP, remember to correctly set the “use_axieth_on_zynq” parameter:

• When using ports 0..2 the BSP setting “use_axieth_on_zynq” must be set to 1.
• When using port 3, the BSP setting “use_axieth_on_zynq” must be set to 0.

The application will not compile if the correct BSP settings have not been set. To change BSP settings: right click on the BSP and click Board Support Package Settings from the context menu.

What now?

The echo server application is actually a very good starting place for developing Ethernet applications on the ZedBoard or other Xilinx FPGAs. Here are some potential ways you could “tweek” the echo server application to be useful for other things:

• Allow your FPGA designs to be controlled by TCP commands sent from a PC over an Ethernet cable – or over the Internet.
• Send data over TCP from your PC to your FPGA and leverage the FPGA for hardware acceleration.
• Connect your FPGA to the Internet and design a high-performance IoT device.

Source code Git repository

Below are the links to the source code Git repositories. There is a version of the project for the ZedBoard and the MicroZed. There is also a version that uses only the AXI Ethernet Subsystem IP.

• ZedBoard using GMII-to-RGMII and AXI Ethernet Subsystem
• ZedBoard using AXI Ethernet Subsystem only
• MicroZed using GMII-to-RGMII and AXI Ethernet Subsystem

If you enjoyed this tutorial or if you run into problems using it, please leave me a comment below.

Many FPGA-based embedded designs require connections to multiple Ethernet devices such as IP cameras, and control of those devices under an operating system, typically Linux. The development of such applications can be accelerated through the use of development boards such as the ZedBoard and the Ethernet FMC. In this tutorial, we will build a custom version of PetaLinux for the ZedBoard and bring up 4 extra Ethernet ports, made available by the Ethernet FMC. The Vivado hardware design used in this tutorial will be very similar to the one we created in a previous tutorial titled: Using AXI Ethernet Subsystem and GMII-to-RGMII in a Multi-port Ethernet design. You don’t need to have followed that tutorial to do this one, as the Vivado project can be built from the sources on Github.

Requirements

To complete this tutorial you will need the following:

• Vivado 2015.4
• PetaLinux SDK 2015.4
• ZedBoard
• Ethernet FMC
• Putty (or similar terminal program)

Tool Setup for Windows users

PetaLinux SDK 2015.4 only runs in the Linux operating system, so Windows users (like me) have to have two machines to follow this tutorial. You can either have two physical machines, which is how I work, or you can have one Windows machine and one Linux virtual machine. In this tutorial, I will assume that you have two physical machines, one running Windows and the other running Linux. My personal setup uses Windows 7 and Ubuntu 14.04 LTS on two separate machines.

If you are building your Linux setup for the first time, here are the supported OSes according to the PetaLinux SDK Installation guide:

• RHEL 5 (32-bit or 64-bit)
• RHEL 6 (32-bit or 64-bit)
• SUSE Enterprise 11 (32-bit or 64-bit)

Note: I had problems installing PetaLinux SDK 2015.4 on 32-bit Ubuntu, as did others, so I use 64-bit Ubuntu and I haven’t had any problems with my setup.

Regenerate the Vivado project

The details of the Vivado design will not be covered by this tutorial as it has already been covered in a previous tutorial – except that in this tutorial, we will be using AXI Ethernet Subsystem IP for all 4 ports. Follow these instructions to regenerate the Vivado project from scripts. Please note that the Git repository is regularly updated for the latest version of Vivado, so you must download the last “commit” for the version of Vivado that you are using.

1. Download the sources from Github here: https://github.com/fpgadeveloper/zedboard-qgige-axieth
2. Depending on your operating system:
   • If you are using a Windows machine, open Windows Explorer, browse to the “Vivado” folder within the sources you just downloaded. Double-click on the build.bat file to run the batch file.
   • If you are using a Linux machine, run Vivado and then select Window->Tcl Console from the welcome screen. In the Tcl console, use the “cd” command to navigate to the “Vivado” folder within the sources you just downloaded. Then type source build.tcl to run the build script.
3. Once the script has finished running, the Vivado project should be regenerated and located in the “Vivado” folder. Run Vivado and open the newly generated project.

Generate the bitstream

The first thing we’ll need to do is to generate the bitstream from the Vivado project.

1. Open the project in Vivado.
2. From the Flow Navigator, click “Generate Bitstream”.
3. Depending on your machine, it will take several minutes to perform synthesis and implementation. In the end, you should see the following message. Just select “View Reports” and click OK.
4. Now we need to use the “Export to SDK” feature to create a hardware description file (.hdf) for the project. From the menu, select File->Export->Export Hardware.
5. In the Export Hardware window, tick “Include bitstream” and choose “Local to Project” as the export location.

Build PetaLinux for our design

Now it’s time to move to our Linux machine and use the PetaLinux SDK to build PetaLinux for our hardware design.

1. On your Linux machine, start a command terminal.
2. Type source /<your-petalinux-install-dir>/settings.sh into the terminal and press Enter. Obviously you must insert the location of your PetaLinux installation.
3. For consistency, let’s work from a directory called projects/zedboard-multiport-ethernet in your home directory. Create that directory and then “cd” to it.
4. Use a USB stick or another method to copy the entire Vivado project directory (should be zedboard_qgige_axieth) from your Windows machine onto your Linux machine. Place it into the directory we just created.
5. Create a PetaLinux project using this command: petalinux-create --type project --template zynq --name petalinux_prj
6. Change to the “petalinux_prj” directory in the command terminal.
   Stay in the PetaLinux project folder from here on. It is important that all the following commands are run from the PetaLinux project folder that we just created.
7. Import the Vivado generated hardware description into our PetaLinux project with the command: petalinux-config --get-hw-description ../zedboard_qgige_axieth/zedboard_qgige_axieth.sdk/
8. The Linux System Configuration will open, but we don’t have any changes to make here, so simply exit and save the configuration.
9. Configure the Linux kernel with the command: petalinux-config -c kernel
10. In the Kernel configuration, we need to disable the Xilinx AXI DMA driver, as it conflicts with the AXI Ethernet driver. Disable: Device Drivers->DMA Engine support->Xilinx DMA Engines->Xilinx AXI DMA Engine, then exit and save the configuration.
11. We don’t have anything to change in the Linux root file system, but if you want to make your own changes, run the command: petalinux-config -c rootfs
12. The device tree that was generated by PetaLinux SDK will not contain the MAC addresses, nor the addresses of the Ethernet PHYs, so we have to add this information manually. Open the system-top.dts file in the petalinux-prj/subsystems/linux/configs/device-tree directory.
13. Add the following code to the end of the system-top.dts file and then save it:

&axi_ethernet_0 {
	local-mac-address = [00 0a 35 00 01 22];
	phy-handle = <&phy0>;
	xlnx,has-mdio = <0x1>;
	mdio {
		#address-cells = <1>;
		#size-cells = <0>;
		phy0: phy@0 {
			compatible = "marvell,88e1510";
			device_type = "ethernet-phy";
			reg = <0>;
			marvell,reg-init = <2 21 0xffef 0x00 0x0 0 0x7fff 0x8000>;
		};
	};
};

&axi_ethernet_1 {
	local-mac-address = [00 0a 35 00 01 23];
	phy-handle = <&phy1>;
	xlnx,has-mdio = <0x1>;
	mdio {
		#address-cells = <1>;
		#size-cells = <0>;
		phy1: phy@0 {
			compatible = "marvell,88e1510";
			device_type = "ethernet-phy";
			reg = <0>;
			marvell,reg-init = <2 21 0xffef 0x00 0x0 0 0x7fff 0x8000>;
		};
	};
};

&axi_ethernet_2 {
	local-mac-address = [00 0a 35 00 01 24];
	phy-handle = <&phy2>;
	xlnx,has-mdio = <0x1>;
	mdio {
		#address-cells = <1>;
		#size-cells = <0>;
		phy2: phy@0 {
			compatible = "marvell,88e1510";
			device_type = "ethernet-phy";
			reg = <0>;
			marvell,reg-init = <2 21 0xffef 0x00 0x0 0 0x7fff 0x8000>;
		};
	};
};

&axi_ethernet_3{
	local-mac-address = [00 0a 35 00 01 25];
	phy-handle = <&phy3>;
	xlnx,has-mdio = <0x1>;
	mdio {
		#address-cells = <1>;
		#size-cells = <0>;
		phy3: phy@0 {
			compatible = "marvell,88e1510";
			device_type = "ethernet-phy";
			reg = <0>;
			marvell,reg-init = <2 21 0xffef 0x00 0x0 0 0x7fff 0x8000>;
		};
	};
};

1. Build PetaLinux using the command: petalinux-build

PetaLinux will take a few minutes to build depending on your machine.

Boot PetaLinux from SD card

Now we will generate the boot files for an SD card, copy those files to the SD card and then boot PetaLinux on the ZedBoard.

1. Generate the boot files using these commands:
   • petalinux-package --boot --fsbl ./images/linux/zynq_fsbl.elf --fpga ../zedboard_qgige_axieth/zedboard_qgige_axieth.runs/impl_1/design_1_wrapper.bit --uboot --force
   • petalinux-package --prebuilt --fpga ../zedboard_qgige_axieth/zedboard_qgige_axieth.runs/impl_1/design_1_wrapper.bit
2. The boot files can now be found in the petalinux-prj/images/linux folder. Copy the BOOT.BIN and image.ub files into the root of your SD card.
3. Plug the SD card into your ZedBoard.
4. Make sure that your ZedBoard is configured to boot from the SD card by setting jumpers JP7, JP8, JP9, JP10 and JP11 to 00110 respectively.
5. Make sure that a USB cable connects the ZedBoard USB-UART to your PC.
6. Turn ON the ZedBoard.
7. Find the COM port associated with your ZedBoard USB-UART by going into Device Manager.
8. Open a new session in Putty using these settings and the COM port you just identified:
   • Baud rate: 115200bps
   • Data: 8 bits
   • Parity: None
   • Stop bits: 1
9. Watch PetaLinux booting up in the Putty console and wait for the login. If you don’t see anything, you probably missed the boot sequence – just press ENTER and you should see the login prompt.

If you want to see the complete boot log, click here.

Configure the Ethernet ports

Our Vivado design has 5 Ethernet ports: the on-board port of the ZedBoard plus the 4 ports of the Ethernet FMC. In PetaLinux, these ports will be assigned to eth0 (on-board port), and eth1-eth4 (Ethernet FMC ports 0-3). Using ifconfig, we will configure the Ethernet FMC ports with fixed IP addresses. We will then connect one of them to a PC and use ping to test it.

1. First login to PetaLinux using the username “root” and the password “root”.
2. Configure the Ethernet ports using the following commands:
   • ifconfig eth1 192.168.1.11 netmask 255.255.255.0 up
   • ifconfig eth2 192.168.1.12 netmask 255.255.255.0 up
   • ifconfig eth3 192.168.1.13 netmask 255.255.255.0 up
   • ifconfig eth4 192.168.1.14 netmask 255.255.255.0 up
3. When you enter each of the above commands, you should get an output that looks like this:

net eth1: Promiscuous mode disabled.
net eth1: Promiscuous mode disabled.
xilinx_axienet 41000000.ethernet eth1: Link is Down

Test the Ethernet ports

To test the Ethernet ports, we’ll need a PC with it’s own gigabit Ethernet port. Here I’m using my laptop which runs on Windows 10.

1. On the test PC, configure the Ethernet port to use a fixed IP address of 192.168.1.10.
2. Use an Ethernet cable to connect port 0 of the Ethernet FMC to the test PC. You should see the following message in Putty:

xilinx_axienet 41000000.ethernet eth1: Link is Up - 1Gbps/Full - flow control off

1. First let’s try pinging from the PC to the ZedBoard. Open a command window on the test PC and type the command: ping 192.168.1.11
2. Now to ping in the reverse direction (ZedBoard to PC), you will probably need to disable the public firewall on your PC.
3. On the Linux command line in Putty, type the command: ping -I eth1 192.168.1.10

You will have to press Ctrl-C to stop pinging. Notice that we have to use the argument “-I eth1” from the ZedBoard as there are multiple ports we could possibly ping from. We can do the same ping test to verify the other ports.

Source code Git repository

The sources for re-generating this project automatically can be found on Github here: ZedBoard Multi-port Ethernet design

Boot files for the ZedBoard

If you want to try out my boot files for the ZedBoard, download them here:

• BOOT.BIN
• image.ub

If you run into problems going through these instructions, just write me a comment below.

In this tutorial video, I bring-up the 3x Gigabit Ethernet ports on the MYD-Y7Z010 Development board from MYIR. Firstly, I create a Vivado design for this board, then I export it into the SDK and generate the echo server application for each of the 3 ports (note that the echo server application only supports one port at a time). At the end of the video, I test each of these designs on hardware and ensure that the ports are given an IP address via DHCP and that I can ping the port. I did this on the MYIR dev board but I hope that the tutorial can be of help to people bringing up Ethernet ports on other platforms or their own custom boards.

Requirements

To go through this tutorial yourself, you’ll need:

• the board files that I’ve placed in this Github repository: Projects for the MYD-Y7Z010 Development board
• Vivado 2018.1 – I’ve done the tutorial with Vivado 2018.1, but you should be able to do it in future versions without too much trouble
• an RS232 to USB converter
• a network running DHCP
• a CAT-5e/6 Ethernet cable

Description

The MYIR board is based on the Zynq 7010 device, so we’ll make use of the two build-in GEMs of the Zynq PS and we’ll use AXI Ethernet Subsystem IP for the third port. The image below shows how the ports are connected through the Ethernet PHYs to the RJ45 connectors. All of the PHYs have an RGMII interface to the MACs.

As you can see in the block diagram, one of the PHYs is on the module (SoM) and this PHY is directly connected (via MIO pins) to GEM0 of the Zynq PS. The other two PHYs are on the carrier board and they connect to the FPGA (PL) of the Zynq. For one of these PHYs, we’ll route GEM1 to the PL via EMIO, and we’ll use a GMII-to-RGMII IP to convert the GMII interface to the RGMII for the PHY connection. For the last PHY we will use the AXI Ethernet Subsystem IP.

UART for debug

When working with any board, a UART for debug is handy. This MYIR board has a few UART options but unfortunately none of them is a USB-UART:

• 3.3V TTL UART to a pin header (connects to UART1 of the Zynq PS)
• RS232 UART to the DB9 connector (connects to PL)
• RS485 UART to the DB9 connector (connects to PL)

If you’re going through this yourself, I suggest you use the UART option that is most convenient for you. In my case, I’ve got a RS232 to USB converter handy, so I’m using the RS232 option. The MYIR board comes with a rainbow ribbon cable to breakout the DB9 connector, so I’ve just wired up the RS232 signals to the DB9 connector of the converter:

• White wire (RS232 GND) to pin 5 (GND) of the DB9 of the converter
• Grey wire (RS232 TX) to pin 2 (RX) of the DB9 of the converter
• Purple wire (RS232 RX) to pin 5 (TX) of the DB9 of the converter

Board files

Before creating the Vivado project for this board, you should copy the board files into your Vivado installation. Find the board files in the Github repo here:

https://github.com/fpgadeveloper/myd-y7z010-projects/tree/master/Vivado/boards/board_files/MYD-Y7Z010/1.1

Respecting that directory structure, copy the board files into your Vivado installation here:

\Xilinx\Vivado\VERSION\data\boards\board_files

The next time you run Vivado, the MYIR board will be on the list of boards when creating a new project.

lwIP Library modifications

When we get to the SDK, we only have to generate the echo server application/template for each of the 3 ports. However, as is typical when working with the echo server application, we have to modify some of the code that deals with configuration of the PHY. The code already handles some Marvell and TI PHYs, but not the Microchip PHY that this MYIR board uses (KSZ9031). I’ve detailed the code modifications below, but I’ve also included the modified code in the Github repo.

The best way to deal with library modifications is to create a local copy of the original library and bump up the version number. You modify this local copy and then add it’s location as a repository to your SDK workspace. Then when generating the echo server application, the SDK will use your modified library instead of the original library. In the Github repo, I’ve already created this local copy of the library, but only containing the modified sources. To move the rest of the sources into this local copy, I’ve written a Tcl script that you can run. It’s a very simple script and in fact, if you prefer to just copy the files over manually, you can – just make sure not to overwrite the files that are already in the repo, as they contain the required modifications.

For those who prefer to make the modifications themselves, or who want to better understand the modifications, I’ve described them below.

File to modify

Filename: xemacpsif_physpeed.c
Location: \EmbeddedSw\ThirdParty\sw_services\lwip202_v1_09\src\contrib\ports\xilinx\netif

This file contains the code for configuration of the PHY connected to the GEMs. There are two things we need to change in this file. Firstly, we need to add a function to configure the Microchip PHY; for the most part, our function is the same as the one for the Marvell PHY, except that the detection of the link speed is done through a different register. Secondly, we need to create a define for the PHY address of the GMII-to-RGMII converter, so that the code makes the necessary register changes to the core after link-up.

Add these defines for the Microchip PHY identifier and to specify the PHY address of the GMII-to-RGMII converter:

#define PHY_MICROCHIP_IDENTIFIER 0x0022
#define XPAR_GMII2RGMIICON_0N_ETH1_ADDR 8

Add this function for the configuration of the Microchip PHY (you can place it below the get_Marvell_phy_speed function):

static u32_t get_Microchip_phy_speed(XEmacPs *xemacpsp, u32_t phy_addr)
{
	u16_t temp;
	u16_t control;
	u16_t status;
	u16_t status_speed;
	u32_t timeout_counter = 0;

	xil_printf("Start PHY autonegotiation \r\n");

	XEmacPs_PhyWrite(xemacpsp,phy_addr, IEEE_PAGE_ADDRESS_REGISTER, 2);
	XEmacPs_PhyRead(xemacpsp, phy_addr, IEEE_CONTROL_REG_MAC, &control);
	control |= IEEE_RGMII_TXRX_CLOCK_DELAYED_MASK;
	XEmacPs_PhyWrite(xemacpsp, phy_addr, IEEE_CONTROL_REG_MAC, control);

	XEmacPs_PhyWrite(xemacpsp, phy_addr, IEEE_PAGE_ADDRESS_REGISTER, 0);

	XEmacPs_PhyRead(xemacpsp, phy_addr, IEEE_AUTONEGO_ADVERTISE_REG, &control);
	control |= IEEE_ASYMMETRIC_PAUSE_MASK;
	control |= IEEE_PAUSE_MASK;
	control |= ADVERTISE_100;
	control |= ADVERTISE_10;
	XEmacPs_PhyWrite(xemacpsp, phy_addr, IEEE_AUTONEGO_ADVERTISE_REG, control);

	XEmacPs_PhyRead(xemacpsp, phy_addr, IEEE_1000_ADVERTISE_REG_OFFSET,
					&control);
	control |= ADVERTISE_1000;
	XEmacPs_PhyWrite(xemacpsp, phy_addr, IEEE_1000_ADVERTISE_REG_OFFSET,
					control);

	XEmacPs_PhyWrite(xemacpsp, phy_addr, IEEE_PAGE_ADDRESS_REGISTER, 0);
	XEmacPs_PhyRead(xemacpsp, phy_addr, IEEE_COPPER_SPECIFIC_CONTROL_REG,&control);
	control |= (7 << 12);	/* max number of gigabit attempts */
	control |= (1 << 11);	/* enable downshift */
	XEmacPs_PhyWrite(xemacpsp, phy_addr, IEEE_COPPER_SPECIFIC_CONTROL_REG,control);
	XEmacPs_PhyRead(xemacpsp, phy_addr, IEEE_CONTROL_REG_OFFSET, &control);
	control |= IEEE_CTRL_AUTONEGOTIATE_ENABLE;
	control |= IEEE_STAT_AUTONEGOTIATE_RESTART;
	XEmacPs_PhyWrite(xemacpsp, phy_addr, IEEE_CONTROL_REG_OFFSET, control);

	XEmacPs_PhyRead(xemacpsp, phy_addr, IEEE_CONTROL_REG_OFFSET, &control);
	control |= IEEE_CTRL_RESET_MASK;
	XEmacPs_PhyWrite(xemacpsp, phy_addr, IEEE_CONTROL_REG_OFFSET, control);

	while (1) {
		XEmacPs_PhyRead(xemacpsp, phy_addr, IEEE_CONTROL_REG_OFFSET, &control);
		if (control & IEEE_CTRL_RESET_MASK)
			continue;
		else
			break;
	}

	XEmacPs_PhyRead(xemacpsp, phy_addr, IEEE_STATUS_REG_OFFSET, &status);

	xil_printf("Waiting for PHY to complete autonegotiation.\r\n");

	while ( !(status & IEEE_STAT_AUTONEGOTIATE_COMPLETE) ) {
		sleep(1);
		XEmacPs_PhyRead(xemacpsp, phy_addr,
						IEEE_COPPER_SPECIFIC_STATUS_REG_2,  &temp);
		timeout_counter++;

		if (timeout_counter == 30) {
			xil_printf("Auto negotiation error \r\n");
			return XST_FAILURE;
		}
		XEmacPs_PhyRead(xemacpsp, phy_addr, IEEE_STATUS_REG_OFFSET, &status);
	}
	xil_printf("autonegotiation complete \r\n");

  // Read from Microchip page 0, register 0x1F (PHY Control)
  // http://ww1.microchip.com/downloads/en/DeviceDoc/00002117F.pdf
	XEmacPs_PhyRead(xemacpsp, phy_addr,0x1F,&status_speed);
	if (status_speed & 0x040)
		return 1000;
	else if(status_speed & 0x020)
		return 100;
	else if(status_speed & 0x010)
		return 10;

	return XST_SUCCESS;
}

Modify the get_IEEE_phy_speed function as below so that it calls the get_Microchip_phy_speed function (above) to configure the Microchip PHY:

static u32_t get_IEEE_phy_speed(XEmacPs *xemacpsp, u32_t phy_addr)
{
	u16_t phy_identity;
	u32_t RetStatus;

	XEmacPs_PhyRead(xemacpsp, phy_addr, PHY_IDENTIFIER_1_REG,
					&phy_identity);
	if (phy_identity == PHY_TI_IDENTIFIER) {
		RetStatus = get_TI_phy_speed(xemacpsp, phy_addr);
	} else if (phy_identity == PHY_REALTEK_IDENTIFIER) {
		RetStatus = get_Realtek_phy_speed(xemacpsp, phy_addr);
	} else if (phy_identity == PHY_MICROCHIP_IDENTIFIER) {
		RetStatus = get_Microchip_phy_speed(xemacpsp, phy_addr);
	} else {
		RetStatus = get_Marvell_phy_speed(xemacpsp, phy_addr);
	}

	return RetStatus;
}

File to modify

Filename: xaxiemacif_physpeed.c
Location: \EmbeddedSw\ThirdParty\sw_services\lwip202_v1_09\src\contrib\ports\xilinx\netif

This file contains the code for configuration of the PHYs connected to AXI Ethernet Subsystem IP. There are two things we need to change in this file. Firstly, we need to add a function to configure the Microchip PHY; as described earlier, our function is mostly the same as the one for the Marvell PHY, except that the detection of the link speed is done through a different register. Secondly, because we are using AXI Ethernet Subsystem IP, we need to disable the RGMII TX clock delay that is internal to the PHY (for more information on this topic, read RGMII Timing Considerations).

Add these defines to specify the Microchip identifier and the register masks for TX and RX clock delay settings:

#define PHY_MICROCHIP_IDENTIFIER 0x0022
#define IEEE_RGMII_TX_CLOCK_DELAYED_MASK 0x0010
#define IEEE_RGMII_RX_CLOCK_DELAYED_MASK 0x0020

Add this function to configure the Microchip PHY, you can add it below the get_phy_speed_88E1116R function:

unsigned int get_phy_speed_Microchip(XAxiEthernet *xaxiemacp, u32 phy_addr)
{
	u16 phy_val;
	u16 control;
	u16 status;
	u16 partner_capabilities;

	xil_printf("Start PHY autonegotiation \r\n");

  /* RGMII with only RX internal delay enabled */
	XAxiEthernet_PhyWrite(xaxiemacp,phy_addr, IEEE_PAGE_ADDRESS_REGISTER, 2);
	XAxiEthernet_PhyRead(xaxiemacp, phy_addr, IEEE_CONTROL_REG_MAC, &control);
  control &= ~IEEE_RGMII_TX_CLOCK_DELAYED_MASK;
  control |= IEEE_RGMII_RX_CLOCK_DELAYED_MASK;
	XAxiEthernet_PhyWrite(xaxiemacp, phy_addr, IEEE_CONTROL_REG_MAC, control);

	XAxiEthernet_PhyWrite(xaxiemacp, phy_addr, IEEE_PAGE_ADDRESS_REGISTER, 0);

	XAxiEthernet_PhyRead(xaxiemacp, phy_addr, IEEE_AUTONEGO_ADVERTISE_REG, &control);
	control |= IEEE_ASYMMETRIC_PAUSE_MASK;
	control |= IEEE_PAUSE_MASK;
	control |= ADVERTISE_100;
	control |= ADVERTISE_10;
	XAxiEthernet_PhyWrite(xaxiemacp, phy_addr, IEEE_AUTONEGO_ADVERTISE_REG, control);

	XAxiEthernet_PhyRead(xaxiemacp, phy_addr, IEEE_1000_ADVERTISE_REG_OFFSET,
				&control);
	control |= ADVERTISE_1000;
	XAxiEthernet_PhyWrite(xaxiemacp, phy_addr, IEEE_1000_ADVERTISE_REG_OFFSET,
				control);

	XAxiEthernet_PhyWrite(xaxiemacp, phy_addr, IEEE_PAGE_ADDRESS_REGISTER, 0);
	XAxiEthernet_PhyRead(xaxiemacp, phy_addr, IEEE_COPPER_SPECIFIC_CONTROL_REG,
				&control);
	control |= (7 << 12);	/* max number of gigabit atphy_valts */
	control |= (1 << 11);	/* enable downshift */
	XAxiEthernet_PhyWrite(xaxiemacp, phy_addr, IEEE_COPPER_SPECIFIC_CONTROL_REG,
				control);

	XAxiEthernet_PhyRead(xaxiemacp, phy_addr, IEEE_CONTROL_REG_OFFSET, &control);
	control |= IEEE_CTRL_AUTONEGOTIATE_ENABLE;
	control |= IEEE_STAT_AUTONEGOTIATE_RESTART;
	XAxiEthernet_PhyWrite(xaxiemacp, phy_addr, IEEE_CONTROL_REG_OFFSET, control);

	XAxiEthernet_PhyRead(xaxiemacp, phy_addr, IEEE_CONTROL_REG_OFFSET, &control);
	control |= IEEE_CTRL_RESET_MASK;
	XAxiEthernet_PhyWrite(xaxiemacp, phy_addr, IEEE_CONTROL_REG_OFFSET, control);
	while (1) {
		XAxiEthernet_PhyRead(xaxiemacp, phy_addr, IEEE_CONTROL_REG_OFFSET, &control);
		if (control & IEEE_CTRL_RESET_MASK)
			continue;
		else
			break;
	}

	xil_printf("Waiting for PHY to complete autonegotiation.\r\n");

	XAxiEthernet_PhyRead(xaxiemacp, phy_addr, IEEE_STATUS_REG_OFFSET, &status);
	while ( !(status & IEEE_STAT_AUTONEGOTIATE_COMPLETE) ) {
		AxiEthernetUtilPhyDelay(1);
		XAxiEthernet_PhyRead(xaxiemacp, phy_addr, IEEE_COPPER_SPECIFIC_STATUS_REG_2,
							&phy_val);
		if (phy_val & IEEE_AUTONEG_ERROR_MASK) {
			xil_printf("Auto negotiation error \r\n");
		}
		XAxiEthernet_PhyRead(xaxiemacp, phy_addr, IEEE_STATUS_REG_OFFSET,
					&status);
	}

	xil_printf("autonegotiation complete \r\n");

  // Read from Microchip page 0, register 0xA (PHY Control)
  // http://ww1.microchip.com/downloads/en/DeviceDoc/00002117F.pdf
	XAxiEthernet_PhyRead(xaxiemacp, phy_addr,0x1F,&status);
	if (status & 0x040)
		return 1000;
	else if(status & 0x020)
		return 100;
	else
		return 10;
}

Modify the get_IEEE_phy_speed function as below so that it calls the get_phy_speed_Microchip function (above) to configure the Microchip PHY:

unsigned get_IEEE_phy_speed(XAxiEthernet *xaxiemacp)
{
	u16 phy_identifier;
	u16 phy_model;
	u8 phytype;

#ifdef XPAR_AXIETHERNET_0_BASEADDR
	u32 phy_addr = detect_phy(xaxiemacp);

	/* Get the PHY Identifier and Model number */
	XAxiEthernet_PhyRead(xaxiemacp, phy_addr, PHY_IDENTIFIER_1_REG, &phy_identifier);
	XAxiEthernet_PhyRead(xaxiemacp, phy_addr, PHY_IDENTIFIER_2_REG, &phy_model);

/* Depending upon what manufacturer PHY is connected, a different mask is
 * needed to determine the specific model number of the PHY. */
	if (phy_identifier == MARVEL_PHY_IDENTIFIER) {
		phy_model = phy_model & MARVEL_PHY_MODEL_NUM_MASK;

		if (phy_model == MARVEL_PHY_88E1116R_MODEL) {
			return get_phy_speed_88E1116R(xaxiemacp, phy_addr);
		} else if (phy_model == MARVEL_PHY_88E1111_MODEL) {
			return get_phy_speed_88E1111(xaxiemacp, phy_addr);
		}
	} else if (phy_identifier == PHY_MICROCHIP_IDENTIFIER) {
		return get_phy_speed_Microchip(xaxiemacp, phy_addr);
	} else if (phy_identifier == TI_PHY_IDENTIFIER) {
		phy_model = phy_model & TI_PHY_DP83867_MODEL;
		phytype = XAxiEthernet_GetPhysicalInterface(xaxiemacp);

		if (phy_model == TI_PHY_DP83867_MODEL && phytype == XAE_PHY_TYPE_SGMII) {
			return get_phy_speed_TI_DP83867_SGMII(xaxiemacp, phy_addr);
		}

		if (phy_model == TI_PHY_DP83867_MODEL) {
			return get_phy_speed_TI_DP83867(xaxiemacp, phy_addr);
		}
	}
	else {
	    LWIP_DEBUGF(NETIF_DEBUG, ("XAxiEthernet get_IEEE_phy_speed: Detected PHY with unknown identifier/model.\r\n"));
	}
#endif
#ifdef PCM_PMA_CORE_PRESENT
	return get_phy_negotiated_speed(xaxiemacp, phy_addr);
#endif
}

Booting from SD card

When testing on hardware, unfortunately the MYIR board has the legacy 100mil pitch JTAG header, and I don't have an appropriate adapter for this. So instead of programming the board via JTAG, I generate a BOOT.bin file for each of the echo server applications, then I boot the board from the SD card. To boot the board from SD card, you will have to set the SW1 boot setting to 1-OFF, 2-ON.

What next?

Now that we've validated the hardware of this board, we could make it useful by getting Linux to run on it. When I find some time in the coming weeks, I'll generate PetaLinux for this board and test it out.

By the way, board bring-up is one of the services that we offer our customers. If your company or start-up has a custom board that you would like to bring-up, please get in touch. We are particularly good with Ethernet and PCIe interfaces.

Over the last few months I’ve been really busy working on a new product and I just want to take a step back today and share some of it. Let me start with what it is and then I’ll tell you about how and why I did it.

The product

A 4-port Gigabit Ethernet mezzanine card designed for Avnet‘s Ultra96 Zynq Ultrascale+ single board computer.

The benefits. The low cost and small form factor of the overall system makes it possible (I hope) to not only prototype new products but to go to market with this solution until investment in a custom design is justified by market demand. The other major benefit is community support – the Ultra96 has a large and growing user base and is part of an ecosystem that is supported by the 96Boards community as well as the PYNQ community. If you want to develop a product with this hardware, you’re not going it alone and you can leverage a lot of work that has already been done.

The applications. So much is done with gigabit Ethernet these days, here are some of the applications that this product will add value to: industrial controls, network security, smart NIC, network monitoring, hardware accelerated routers/switches.

The features

The 4 Ethernet ports each connect to a separate Ethernet PHY, the DP83867IS from Texas Instruments. This is a low power, robust, high immunity gigabit Ethernet PHY with many powerful features including:

• Extra low latency (TX < 90ns, RX < 290ns)
• Wake-on-LAN packet detection
• IEEE 1588 Time stamp support
• Cable diagnostics

The mezzanine has a stacked low-speed expansion connector that brings up all of the unused low-speed I/Os of the Ultra96 and can be used to attach a second mezzanine card such as the Sensors Mezzanine. The extended I/Os include:

• 10x I/Os of the programmable logic (PL) (can be reconfigured to GPIO, UART, I2C, SPI, or other low-speed interfaces)
• 3x GPIOs of the processor system (PS)
• 2x I2C buses (I2C0 and I2C1)
• 1x SPI bus (SPI0)
• Power and reset pushbuttons

The story

I’ve been closely watching the Ultra96 since Avnet launched it early last year. Its powered by the Zynq Ultrascale+ MPSoC, it has the dimensions of a credit card and it is (at the time I write this) the lowest-price ZU+ board on the market. What’s more, it’s designed to the 96Boards spec so it can carry almost any of the standardized mezzanine boards (add-on cards) now on the market. When they then released PYNQ support later in the year, I had to get one.

Why make an Ethernet mezzanine? The marketing around the Ultra96 is mainly focused on AI, which is natural because most of the 96Boards SBCs are designed with vision applications in mind, and the ZU+ has enormous potential in accelerating neural networks. But the Ultra96 is also a great fit for Ethernet – the ZU+ has 4 internal Gigabit Ethernet MACs, and it also has the ability to run SGMII links on it’s I/O pins (SGMII is a serial PHY interface that operates at 1.25Gbps). Ethernet is also a great fit for us. Through our Ethernet FMC product, we’ve helped literally hundreds of companies develop their own Ethernet products. So an opportunity eventually fell into my lap. One of our clients was looking for exactly this kind of solution and they were willing to share the design costs. So I knew we had at least one customer, and a good part of our expenses were covered.

The design

There were quite a few design challenges to overcome to make this board happen. The constrained PCB real estate and location of the expansion connectors severely limited my choice of parts, most notably the quad RJ45. The Marvell PHYs that we use on the Ethernet FMC were not suitable for this, so I couldn’t reuse that trusted design. The pin assignment of the high-speed expansion connector on the Ultra96 wasn’t exactly ideal for Ethernet. The low I/O supply voltage meant that I couldn’t be sure that it was all going to work until I built and tested the actual hardware.

Limited PCB real estate. The obvious challenge was overcoming the physical constraints. The credit card sized form factor is great but it doesn’t provide much room for a big quad RJ45 connector and 4 Ethernet PHYs. But the limited space was not the only issue – there is almost no clearance between the bottom side of the mezzanine and the tops of the USB connectors on the Ultra96, so you can’t put components under there, and you definitely can’t have the legs of an RJ45 connector poking through. The other issue is that the MDI pins of the majority of RJ45 connectors would poke through at exactly the spot where the high speed expansion connector sits. I started by looking at surface mounted connectors, which are terrible for mechanical robustness, but I didn’t see any other way. It turned out that there were very few surface mount options available, and those that were were not suitable for various reasons. From the hundreds of thru-hole options, I ended up finding about 3 connectors that fit, but only if I squeezed two of the mounting posts between the two USB connectors. Also, because I needed the MDI pins to be well clear of the expansion connector, the RJ45 had to be especially deep, so it had to take up more PCB real estate than I would have liked to give up. Needless to say, my choice of Ethernet PHY was also strongly influenced by the limited PCB real estate – I needed the smallest package requiring the least external circuitry.

SGMII. The high-speed expansion connector of the Ultra96 has 14 differential pairs routed to it, connecting to 28 I/Os. To connect 4 Ethernet PHYs through 14 pairs, there is really only one possibility: SGMII. The Marvell PHYs that we use on Ethernet FMC have got an RGMII interface, so I couldn’t use them and I couldn’t leverage any of the Ethernet FMC‘s design.

SGMII over LVDS. SGMII is a serial gigabit interface, so one interface uses only 4 pins (2 for TX and 2 for RX). Normally, running gigabit interfaces to an FPGA requires gigabit transceivers, but the pins on the expansion connector don’t route to transceivers. So I had to use “SGMII over LVDS”. SGMII over LVDS is a method for implementing SGMII on I/O pins of the FPGA, where the transceiver (data recovery, encoder/decoder) is implemented by programmable logic in the FPGA. Xilinx Vivado includes a free IP core called Ethernet PCS/PMA or SGMII that implements SGMII over LVDS.

LVDS on a 1.2V bank. The high speed expansion connector of the Ultra96 routes to pins in an I/O bank that is powered by 1.2V. Here’s the problem: you can’t use the LVDS IO standard on pins in a bank that is powered at 1.2V. The solution turned out to be to use AC coupling (required by SGMII anyway) and what they call a “pseudo differential IO standard” called DIFF_SSTL12.

Pin assignment trouble. This was the biggest risk I faced in designing this board. If you’ve ever used the Ethernet PCS/PMA or SGMII IP, on the ZU+, you would know that it requires special selection of the I/O pins – you can’t just connect it to any I/O pins. But I didn’t have that luxury, I had to use the I/O pins that were already chosen by Avnet: the pins that were routed to the high-speed expansion connector. And it turned out that they were not ideally suited for this IP core. No combination of those pins would allow me to implement 4x SGMII links. Sh#t. I told my customer that we weren’t going to get 4 ports out of this. I spent days playing around with the core in Vivado and reading the product guides. I almost retired to the idea of having only 3 ports on this board. Then I got an idea. I might have found a way to sidestep the requirement of the Ethernet PCS/PMA or SGMII IP that was preventing me from using those pins. But it was a gamble, I didn’t know if it was going to work, I couldn’t find anyone in the forums who had tried it before and I couldn’t test it until I had the hardware in my hands. I had to give it a shot. I designed the board with 4 ports, knowing that we might not be able to get all of them working. Well it turned out that I was right, I got all 4 ports working and I’m glad that I didn’t settle for 3.

Your ideas

Now I want to know what you guys think. What would you do with this compact but powerful Ethernet development platform? What mezzanines would you stack onto it? I really hope that this mezzanine delivers value to a lot of projects – what’s yours?

Here’s a short introduction to our 96B Quad Ethernet Mezzanine. For more information, see the documentation at docs.ethernet96.com and if you want to buy the product, go to the order page here.