FPGA part speed grade on U45N

[JakaBac]

In case of U45N OpenNIC is built for -2LV part. With that part Vivado fails to close timing (WNS -0.993 and TNS -2025.445). I didn't try IDR but I suspect that will also struggle a lot.

I have a SN1022 and I can see that Vccint is 0.85V and not 0.72V. That would imply that the design can be built for -2L part. I haven't tried that yet, but I suspect that it will meet timing even without IDR. From what I know SN1022 and U45N are very closely related.

An owner of an actual U45N also commented on AMD forum that Vccint on his board is 0.85V
AU45N QSFP Port numbering error and part number

But then again I also know somebody who owns a SN1022 and his card has 0.72V Vccint.

Is it possible that first versions of SN1022 had regulators configured for 0.72V and that this was later changed to 0.85V which is still the case on U45N?

[JakaBac]

I tried to build OpenNIC for -2L part and it easily meets timing. If all U45N cards have VCCINT regulators configured for 0.85V then it would make sense to make -2L as the default part?

[Jinbei43]

Hi @JakaBac , I have U45N cards and the same problem as you. I tried IDR optimization and it still failed. I would like to know if after you build and flash the shell with -2L part, can you use the card function normally when running with the ONIC driver? (I mean basic Ethernet operation, TX/RX)

[JakaBac]

In my case IDR meets timing. Which version of Vivado are you using?
Building shell with -2L part without IDR meets timing and the board works. But I did limited testing. Just be sure that the regulators on your board are configured for Vccint is 0.85V. The regulators are OTP and probably configuration is fixed during manufacturing.

However keep in mind that the only officially supported is using IDR and -2LV part.

[Jinbei43]

I currently use Vivado 2024.2 and board -2LV part from default OpenNIC shell code and AMD technical document. And when I built this default shell and used IDR (even ML IDR) like you said, it still failed as shown in the previous comment.

I tried to build shell with -2L part (renamed in board_settings section inside /script directory), but it also gives a lot of warnings about timing.

I would like to know if I need to modify the code in addition to the default provided. Because I tried to fix this problem from each of the issues answers in this repo but it still doesn't work. Or if possible, can you send me your shell code and mcs file to build and flash on my U45N?

Lastly: I'm new about this Xilinx FPGA, how can I check that my U45N board regulators are configured for VCCINT at 0.85V?

[JakaBac]

I built the shell using 2022.1 or 2023.1 I don't remember which one. I know that 2023.2 has some issues. On that version IDR run closed timing.

You can check Vccint by using SysMon in Vivado over JTAG
Please see https://docs.amd.com/r/en-US/ug908-vivado-programming-debugging/System-Monitor

Keep in mind that you need to have the ADK2 debug interface or some kind of adapter to connect to the custom card edge debug port on the U45N.

The Debug port pinout is publicly documented in:
https://docs.amd.com/r/en-US/ug1538-alveo-debug/Debug-Connector

I did very little testing of AMD OpenNIC and did not look into the warnings that are emitted during synthesis. From the screenshot it is not apparent what exactly is causing the warnings.
I used the unmodified shell code from this repository to build the bitstream, when I was testing this. I don't have the mcs file anymore, I would need to rebuild the bitstream and I also upgraded Vivado to newer version since that time.

[Jinbei43]

Thank you so much for your valuable advice! It worked when I changed Vivado's version from 2024.2 to 2022.1 with -2LV part and IDR.

However, when I loaded ONIC driver and run simple connection between both ports in the same card via QSFP28 cable, there is no response whatsoever. But after I changed my U45N card to connect to another U45N card, it was able to communicate with each other. (I tried to find the cause from the ONIC driver's issues, but I couldn't find it. But it's okay because at least my card is usable now.)

But I ran into a new problem, card performance. When I ping both cards, the latency is around tens of thousands milliseconds. And throughput when I use iPerf3 is around 5 Gb/s (I know it won't reach 100 Gb/s line-rate because of in-host bottleneck, but it shouldn't be that slow.) I don't know if this is a driver or shell issue because after I checked SysMon via ADK2, VCCINT is around 0.85V. Maybe I should raise this problem in ONIC driver's issue.

Latency result when I ping both cards together. SysMon with VCCINT value (bottom left).