01signal.com

How-To on Partial Reconfiguration with Vivado

中文 | 日本語 | 한국어 | English

Introduction

This is the second post in a series of four about Partial Reconfiguration (or Dynamic Function eXchange, DFX) with Xilinx' Vivado. The intention of this post is walk through and explain the steps for enabling Partial Reconfiguration on an FPGA design. It's recommended to read the first post if you haven't done so already, as it explains the concepts behind these steps.

Xilinx has rebranded Partial Reconfiguration as "Dynamic Function eXchange" (DFX) in 2020. DFX is the expression used in Vivado's menus and informative notes. The technical term "Partial Reconfiguration" is used here nevertheless.

For simplicity, this post assumes only one reconfigurable partition in the project. Expanding this to several partitions is fairly straightforward.

The procedure described in this post starts before the project has been enabled for Partial Reconfiguration. So to break it down roughly, the steps are:

Preparation for floorplanning

More than any other task, floorplanning is the one that requires brainwork. It's a subtle balance between not wasting area of FPGA logic and making sure that both the static partition and the reconfigurable partitions won't have any considerable obstacles during place and route.

Accordingly, the majority of this post discusses this topic.

In the distant past, floorplanning was a technique that was used for timing closure. It helped the tools in placing the logic in a sensible way. As the tools for FPGA design have improved over time, it has been many years since I last saw that floorplanning helped to achieve the timing constraints. Today, the best strategy for achieving timing constraints is almost always to let the tools make the decisions.

With Partial Reconfiguration, floorplanning is a must, so the goal is not to make things worse. Doing this is usually a matter of trial and error. However, the simplest method to achieve good results is to perform an implementation of the design without floorplanning, and then start from how the logic is placed naturally. The next step is to try to organize the areas in a way that makes sense with Partial Reconfiguration, using the initial placement of the logic as a guideline.

In the plugin usage scenario, the floorplanning can be updated as the project evolves over time. This is however not the case in the Remote Update scenario, i.e. when Partial Reconfiguration is used as a means to make version updates to a design that is released: With Remote Update, all partial bitstreams must match with the initial bitstream. Accordingly, the static logic part of the design is frozen as soon as the initial bitstream is released. This means, among others, that the floorplanning must remain the same.

So even before starting with Partial Reconfiguration, the first task is to find a proper area in the FPGA for the static logic. There's no point wasting too much time on this, just get the starting point to go on with after the project is split in two pieces.

Don't get confused: The purpose of this first step is not the floorplanning itself, but to see how Vivado places the logic without restrictions, and from that decide which area should be allocated for the static logic. So these are the steps:

Setting up a project for Partial Reconfiguration

Xilinx' UG909 suggests two work procedures for Partial Reconfiguration:

I'll go with the project flow here, even though it has some limitations, some of which relate to the type of sources in the reconfigurable module (in particular regarding using block designs). Either way, it's better to start off with the project flow, since the scripts that it generates for implementation are a good basis for the non-project flow, if so needed.

These are the steps for turning on Partial Reconfiguration support on an existing project:

Trying perform an implementation of the project at this stage will most likely fail with an error saying something like "[DRC HDPR-30] Missing PBLOCK On Reconfigurable Cell: HD.RECONFIGURABLE cell 'pr_block_ins' must have PBLOCK assigned to itself or its descendant cells". Which means, in simple words, that floorplanning is necessary.

Floorplanning

At this stage, the project is set up just enough for the floorplanning task.

Before breaking down this task into small steps, it's worth to mention a few things to keep in mind:

Now breaking it down to steps:

Correcting the floorplanning

This is probably the least pleasant part about Partial Reconfiguration: To get the floorplanning just right. If you are doing this for the Remote Update use case, this phase is extra important, as this floorplanning will remain throughout the project's lifetime.

Corrections to the floorplanning are necessary for two main reasons: As a response to Critical Warnings, and at a later stage to optimize the usage of the FPGA: The goal is to reduce the waste of resources, and at the same time avoid creating obstacles for place and route.

Making modifications isn't difficult, as it's easy to drag a Pblock's boundaries. It's also easy to expand a Pblock with additional rectangles: Right-click the Pblock and select "Add Pblock Rectangle".

The Critical Warnings often say what corrections are necessary, and nevertheless be sure to have read the relevant chapter (6, 7 or 8) in Xilinx' user guide, UG909 regarding the limitations on floorplanning of your specific FPGA.

The rest of this section discusses possible issues with series-7 FPGA. Ultrascale FPGAs are much easier to work with.

One common error with series-7 FPGA is splitting of interconnect tile columns, for example:

[Constraints 18-993] The Pblock pblock_pr_block_ins has defined an area that causes the splitting of interconnect tile columns. Dynamic Function eXchange requires that the left and right paired interconnect tile columns cannot be split by a reconfigurable boundary.  This is caused by either the left or right edge of a Pblock boundary, or by the Pblock spanning over logic types not included in the Pblock ranges.  To avoid an unroutable situation, placement will be prohibited from both of these columns. To avoid placement restrictions, modify the Pblock to avoid splitting the two columns.
The column of the split contains interconnect tile INT_L_X48Y299  (SLICE_X79Y299 SLICE_X78Y299).
Please refer to the Xilinx document on Dynamic Function eXchange.
Resolution: Set the Pblock property SNAPPING_MODE to value of ON, or modify the column/X specification of the pblock to avoid this edge.

and

[Constraints 18-996] The split between the left and right columns occurs between a reconfigurable Pblock and Static logic. The static sites are not reconfigurable. The Pblock should be adjusted to remove the column from the Pblock, unless the excluded reconfigurable and static sites are not needed for the design. Note that adjusting the Pblock will prevent prohibits and improve placement of the design, but may reduce the routability if the removed sites were needed to span across the static logic. Failure to modify the Pblock may lead to an unplaceable design if these prohibited sites are required by the design. Resolution: Set the Pblock property SNAPPING_MODE to value of ON, or modify the column/X specification of the pblock to avoid this edge. and

To fix this, set the Pblock's SNAPPING_MODE property to ROUTING or ON (odds are that ROUTING won't be good enough, so select ON) as suggested in the first warning. Doing this will probably add a lot of constraints to the XDC file, of this sort:

set_property PROHIBIT true [get_sites SLICE_X79Y349]
set_property PROHIBIT true [get_sites SLICE_X78Y349]
[ ... ]
set_property PROHIBIT true [get_sites SLICE_X79Y191]
set_property PROHIBIT true [get_sites SLICE_X78Y191]
set_property PROHIBIT true [get_sites PMV_X0Y2]
set_property PROHIBIT true [get_sites SLICE_X36Y190]
set_property PROHIBIT true [get_sites SLICE_X37Y190]
[ ... ]
set_property PROHIBIT true [get_sites SLICE_X79Y176]
set_property PROHIBIT true [get_sites SLICE_X78Y176]
set_property PROHIBIT true [get_sites T14]
set_property PROHIBIT true [get_sites R15]
set_property PROHIBIT true [get_sites XADC_X0Y0]
set_property PROHIBIT true [get_sites SLICE_X36Y175]
set_property PROHIBIT true [get_sites SLICE_X37Y175]
[ ... ]

and it goes on.

The PROHIBIT setting for slice sites are those that silence the said Critical Warning. The other PROHIBIT assignments are added for sites of logic that are included in the geometric region, but are not allowed for Partial Reconfiguration on the FPGA that is used. Ultrascale FPGAs and later produce significantly less rows with PROHIBIT, if any at all.

For the purpose of just silencing the Critical Warning, it's probably fine to remove all rows with PROHIBIT, and leave one single row for the slices only. This is done by covering the range of slices that Vivado added in response to the change in SNAPPING_MODE. So turn this range into something like this:

set_property PROHIBIT true [get_sites -range {SLICE_X79Y0 SLICE_X79Y349}]

This is the kind of row in the XDC file that may resolve the problem with interconnect splitting without making the file huge.

Regardless, a row of this sort may also appear in the XDC file. Removing this row is apparently fine as well:

set_property HD.PLATFORM_WRAPPER true [get_cells pr_block_ins]

Reducing the XDC file to the minimum that silences Critical Warnings may appear somewhat superficial, but the alternative is a huge constraints file, which is a recipe for confusion later on. In my experience, the absence of warnings on this sort can be taken as an approval that the design's floorplan is fine.

Odds are that returning the SNAPPING_MODE property back to OFF will cause problems again, regardless of changes to the XDC.

Adding a reconfigurable module

Up till now, the implementation practically achieves the same as hierarchical design, with some extra restrictions. Even though a partial bitstream is generated, it's quite useless, since loading it retains the same design.

So the goal is to create another partial bitstream, which is based upon another configurable module. This requires adding a Child Implementation.

Be sure to have the previous post fresh in your mind before going on reading this, in particular the part on Parent Implementations and Child Implementations, as well as the Dynamic Function eXchange Wizard. Also recall from above on this post, that the "Partition Definitions" tab contains the currently defined reconfigurable modules and their sources.

Open the Dynamic Function eXchange Wizard from the Tools menu, and click Next on the welcome window.

On the Edit Reconfigurable Modules window, click "+". This opens a dialog box for adding a reconfigurable module. The only interesting thing about this dialog box is the Reconfigurable Module Name: This is the name that is used to identify the reconfigurable logic with, as already explained above.

The dialog box also requires associating this module with the name of a partition definition, but there is only one such anyhow (because this post assumes that only one partition is defined).

At least one Verilog / VHDL source file needs to be added to continue: More can be added from the "Partition Definitions" tab later on. Adding the name of the toplevel module of this reconfigurable module won't hurt, in particular if it's not clear from the source files themselves.

Back at the Wizard, click Next again, to the Edit Configurations windows. Click "+", and enter a configuration name. This name's only importance is that it appears in the Design Runs window. A name like config_2 or so is fine.

A new row appears in the list of configurations. Modify the configurable module in the column for the partition, so that each configuration has a different configurable module.

The last window is Edit Configuration Runs, for assigning runs to configurations. The easy way is to delete all runs that are listed in this window (if any) and click "automatically create configuration runs". That does what you would do manually anyhow: Create a parent run, call it "impl_1", and then create child runs, call them what you want, and make them the children of "impl_1".

The Wizard selects a configuration for each run, however that's easy to change. The only important thing is which configuration is associated with the parent run.

And by the way, if you delete all runs in the Wizard, all child runs go away, but impl_1 remains.

Finally: Implementation of the design

To generate the bitstreams, just click "Generate Bitstreams" in Vivado as usual. As already mentioned in the previous post, two or three bitstreams are created for each configuration in a Partial Reconfiguration project.

For example, on an Ultrascale FPGA the bitfiles can be:

Note that the same number of bitstream files is created for all implementations. In other words, the initial bitstream file is created for the Child Implementations as well, so it's perfectly possible to load the FPGA with one of the Child Implementations' initial bitstream, and continue from there.

For a simple way to load partial bitstreams over PCIe or USB 3.x, see this page.

None of the implementations should have Critical Warnings nor fail because of complaints regarding the Pblock and floorplanning in general, because such problem should have been solved already. Should a problem like this still happen, the floorplanning must be corrected as explained above.

Sometimes, when clicking "Generate Bitstream", and changes have been made only to the child implementations, Vivado might respond with "Bitstream generation has already completed and is up-to-date. Re-run anyway?". This is somewhat confusing, but clicking "Yes" will run the child implementations properly. This whole thing with Child Implementation is a bit of an add-on to Vivado, which is also why the status row during implementation says something like "write_bitstream complete. Child running".

Reviewing the results

As Partial Reconfiguration is much about placement, it's a good idea to review the implemented designs. You can open a specific implementation by right-clicking "Open Implemented Design" and then hover over the menu item saying "Open Implemented Design" (again). Then select which implementation to open from the list. If an implementation is missing in the list, it's probably already open.

Try right-clicking the top row of the reconfigurable logic in the Netlist pane of the Implemented Design view, and pick "Highlight Leaf cells". Then do the same with the static logic, with another color.

With the same right-click, there's also "Show Connectivity", which draws straight white lines between logic elements that are connected. The actual routing paths on the FPGA is of course different, so which regions of floorplanning these lines cross has no significance. Looking at connectivity can nevertheless help spotting when the overall organization of a floorplan makes the tools struggle.

It's quite normal and fine that some cells that apparently belong to the static logic are placed inside the reconfigurable area and vice versa. The thing to be wary of is if there seems to be a congestion somewhere — if the logic seems to packed too tightly in general, or in some specific region. If possible, changes in the floorplanning can help alleviate that.

Another thing to look at is the positions of partition pins. They appear as white horizontal bars in the device view, like this (click image to enlarge):

Partition pins in Vivado's device view

As mentioned in the previous post, partition pins can be everywhere inside the reconfigurable partition. However, if the partition pins are away from the partition's edges, it can indicate that the router struggled with timing during the Parent Implementation.

It's also possible to obtain a textual list of the partition pin's coordinates with this Tcl command (change pr_block_ins to the name of the reconfigurable logic cell):

foreach s [get_pins -of [get_cells pr_block_ins]] { set partpin [get_pplocs -quiet -pins [get_pins $s]] ; puts "$s => $partpin"; }

The coordinates for Partition pins correspond to the grid of CLBs (and not slices). On the displayed drawing, these pins are referred to as "Cell pins".

Some of the cell's pins may not be assigned a partition pin. This happens when there's a mismatch between the reconfigurable module's port list (and/or widths of vectors) and its instantiation by the static logic module. Such mismatch is perfectly legal (in Verilog), but the result may be undesired. Running this Tcl command can detect unreachable ports, in particular when this is unintentional.

This ends the technical part on setting up the Vivado project, however the next post discusses an important aspect of the FPGA design: How to ensure that the replacement of logic goes through reliably and smoothly.

Copyright © 2021-2024. All rights reserved. (b4b9813f)