1 Introduction↩︎

Implementation of any complex software or hardware description design involves not only code writing. Equally important is defining metadata used for configuration, dependency management, code generation, testing, and deployment purposes. The metadata is read by software tools, which, in turn, are responsible for generating final binary or bitstream files, or for carrying out extra tasks such as unit tests. Such programs are often referred to as build systems. Some build systems are split into multiple smaller programs, and some have a monolithic structure. An introduction to the build system concept and principles can be found in [1], [2].

In the software domain, there are multiple well-established build systems, such as GNU Make [3], CMake [4], Ninja [5], or Bazel [6]. Some of them were implemented in the early days of programming. For example, GNU Make was first released in 1988 as part of the GNU Project. It was implemented to provide a free software alternative to Unix Make, which was first implemented in 1976 by Stuart Feldman at Bell Labs. Modern programming languages, such as Go, Rust, or Zig, come with build systems included in the language toolchains. The Zig build system can even be used as a build system for C/C++ projects.

In the hardware domain, the situation looks different. The earliest Electronic Design Automation (EDA) tools were implemented as proprietary programs. EDA vendors did not (and still do not) cooperate to create a standard (or at least similar) user interface for tool interaction. The main common point between various EDA tools is using the Tcl language as the interpreter for interaction with the tools. However, the tools provide custom commands and parameters even for primary, common tasks, such as adding hardware description files to a project design. For example, to add a VHDL file in AMD Vivado, one can use specific read_vhdl <path_to_vhdl_file> or generic add_files <path_to_vhdl_file> commands. To add a VHDL file in Intel Quartus, one has to use set_global_assignment -name VHDL_FILE <path_to_vhdl_file> command. EDA tools have abundant differences, making implementing a build system for hardware projects harder than for software. Despite this, there were (and still are) attempts to implement build systems that are more suitable for hardware projects.

The paper proposes a new build system for a hardware description domain. The system is Tcl-based, which makes it easy to integrate with all existing EDA tools already utilizing Tcl language as a user-tool interface. A novel, minimal common abstraction approach differentiates the implemented system from the existing build systems targeting the hardware domain.

The structure of the article is as follows. Section 2 describes related work. Section 3 discusses the requirements and their reasons imposed on the implemented build system. Section 4 introduces the internal architecture of the implemented build system. Section 5 describes commands provided by the implemented build system. Section 6 evaluates the proposed solution in terms of other build systems. Section 7 summarizes the work and highlights novel aspects of the implemented build system.

2 Related work↩︎

There already exist at least more than ten build systems targeting the hardware domain. Some of them are vendor tool specific, and some try to be generic, entirely abstracting away the tool used for synthesis or implementation. By looking at the architecture (analyzing how a tool internally works), existing hardware build systems can be classified into two approaches:

1. direct Tcl approach,

2. indirect abstract approach.

2.1 Direct Tcl approach↩︎

In the Tcl approach, the user directly utilizes the Tcl interface. The idea seems obvious. Since EDA tools already utilize Tcl as a user interface, a build system should also provide users with a Tcl interface. Usually, the user does not directly call EDA tool specific commads. Tcl approch build systems implement some wrapper commands that the user should use. Representatives of the direct Tcl approach are, for example, fpga-vbs [7], vextproj [8], [9], and OSVVM-Scripts [10].

Being able to easily execute arbitrary, custom EDA tool commands is the most significant advantage of the direct Tcl approach. As a Tcl interpreter embedded into an EDA tool executes the build system code, a user can call custom commands in almost any place.

The direct Tcl approach is not free of drawbacks. Some of them are inherent, and some result from the fact that direct Tcl build systems development was stopped at some stage by their maintainers.

The first inherent drawback is that Tcl is considered an unfriendly language compared to, for example, Python. The argument is often raised by junior hardware engineers and has a very subjective nature. However, Tcl was chosen as the EDA scripting language because it was very popular when EDA tools were developed [11]. The second inherent drawback is that Tcl is not a popular language nowadays. Although there are no physical obstacles to change this trend, it is doubtful that suddenly more developers start using Tcl instead, for example, Python as an interpreted language. This is why this drawback is also treated as inherent.

The second group of direct Tcl build systems drawbacks results from stopping the development at a certain stage. There is no direct Tcl build system that can be used to examine the limits and bottlenecks of this idea. For example, most of the direct Tcl build systems support only one EDA tool. OSVVM-Scripts is an exception. However, it can be used only for simulation. It does not support any synthesis or implementation tools. More example drawbacks common to the direct Tcl build systems include:

1. No dependency graph generation capability.

2. Lack or limited capability of automatic testbench targets detection.

3. Lack of automatic parallel testbench running.

2.2 Indirect abstract approach↩︎

The indirect abstract approach is much more popular than the direct Tcl approach. In this approach, the user declares various properties and configurations instead of directly writing Tcl code or calling EDA tools Tcl commands.

The most popular hardware build system utlizing the indirect abstract approach is probably FuseSoc [12], [13]. Other popular build systems include SiliconCompiler [14], [15], HAMMER [16], [17], Hog [18], [19], hdlmake [20], orbit [21], bender [22], bazel_rules_hdl [23], flgen [24], and Blockwork [25].

The language and syntax used for declarations vary between different build systems. Some use simple data serialization formats such as YAML (for example, FuseSoc or bender) or TOML (for example, orbit). Some come with a domain-specific language (for example, flgen), or use Python (for example, HAMMER) and Makefiles (for example, Hdlmake). Finally, some use a mix of more than one (for example, Hog). Especially using Python as a declarative language is powerful, as users can use all functionalities that come with the language.

The most significant advantage of the indirect abstract approach is the low entry threshold. The syntax of languages such as YAML, TOML, or Python is simple, which leads to high legibility. Even junior engineers without extra training can understand it. Listing [lst:fusesoc_edge_detector] presents a very basic core definition in the FuseSoc build system. The first line specifies the version of the description format. The “name” key specifies the vendor, library, name, and version (VLNV) identifier specified in the IP-XACT User Guide [26]. The remaining part is relatively self-explanatory.

CAPI=2:

name: vendor:library:edge_detector:1.0

filesets:
  src:
    files:
      - src/edge_detector.vhd
    file_type: vhdlSource-2008
    logical_name: hdl_library_name

targets:
  default:
    filesets:
      - src

The indirect abstract approach is based on the concept that the interface to EDA tools is hidden under some common abstraction layer. Whether this is an advantage or a disadvantage is debatable. Providing a common abstract layer means that users do not have to learn the interface of multiple EDA tools. Moreover, in theory, changing the EDA tool for compilation process should be limited to changing solely the EDA tool declaration. However, this is somewhat a utopian view. As practice shows the common abstraction layer leads to the following common difficulties:

1. Missing feature implementation for some EDA tools. Let us suppose the abstraction layer declares feature F, and the build system claims support for tools T1 and T2. Although T1 might provide the implementation for F, T2 might be missing implementation for this particular feature. In such a case, the user must add support for this feature to the build system. This task might not be trivial. The user must get to know the internals of the build system, prepare a pull request, wait for a review, and potentially reiterate the implementation. Yet another approach is to modify automatically generated build scripts before a compilation. However, this is against the whole idea of build systems. A direct Tcl approach is free of these difficulties because the user can directly insert custom Tcl commands. There are no intermediate abstraction layers.

2. Simple tasks, such as scoping constraints to a particular module, sometimes require verbose solutions. The solution is not only more verbose but also more complex to understand. What can be solved in one or two lines of Tcl code requires more than one file. An example is discussed in subsection 6.3.

All indirect abstract hardware build systems allow “injecting” arbitrary custom Tcl commands into the automatically generated scripts. However, these commands can not be inserted into arbitrary places. Moreover, inserting a single-line Tcl command usually requires more than one line of code or even more than one file. This can be treated as a proof that the indirect abstract approach is not a golden bullet. The idea of a common abstraction layer has significant drawbacks that no one yet discovered how to fix.

Making a truly abstract hardware build system is highlighy unliekly in the nearest feature. The main problem with the common abstraction layer is the fact that EDA tools were implemented before such a layer was proposed. Declaring a common interface for already existing tools is simply much harder task than implementing tools conforming to some formal interface.

The HBS described in the paper proposes a novel approach. It tries to incorporate the best of the direct Tcl approach and indirect abstract approach. It implements a common abstraction layer. However, the layer is:

1. minimal,

2. limited to the primary common features,

3. implemented in Tcl.

3 Hardware Build System requirements↩︎

3.1 Simplicity and ease of use↩︎

The overhead of build system implementation and maintenance is often underrated [27], [28]. McIntosh in [29] reports that the build system can account for up to 31% of the code files in a project. Moreover, up to 27% of development tasks that change the source code also require build system maintenance. The study was carried out on software projects, not on hardware designs. However, due to the complexity of hardware design, it seems reasonable to assume comparable shares.

Nejati et al. in [30] report that the complexity of build system specifications is one of the top three factors complicating the review of build specifications.

The primary goal of the hardware build system proposed in the paper is simplicity. Contemporary hardware build systems utilizing the indirect abstract approach offer simplicity only at the initial stage. However, as the project complexity grows, so does the complexity of the build system utilization.

3.2 Minimal user interface↩︎

Xia et al. in [31] report that even up to 21.35% of the bugs in software build systems belong to the external interface category. This means that one-fifth of build system bugs results from incorrect usage of the user interface or different behavior of the build system with different tools or platforms.

Fixing build system external interface bugs requires applying patches in the build system itself. Such a task is relatively time-consuming and might not be simple to implement. Build system user must report the problem to the build system developers or fix the bug himself. However, this implies that the build system user is distracted from his own work. Moreover, the fix is not applied immediately. This means that the user must also find a temporary workaround. This, in turn, leads to a technical debt [32], [33].

A minimal user interface has one more advantage. It allows developers to customize build systems to the project needs. Each project is in some way unique and might require particular approach for build problems solving. Moreover, projects usually evolve, for example, due to the customer demand. Dynamic tailoring to the project requirements is a desired feature of modern build systems [34]. Similar observations were made by Neitsh et al. in [35]. Authors report that concerns from the application and implementation domains may “leak” into the build model. The build system should be adaptable enough to provide ways to respond to these concerns. Too rigid user interfaces limit such capabilities.

3.3 Low number of dependencies↩︎

Xiao et al. in [36] report that dependency management is one of the top three factors of technical debt introduction in build systems. Such a finding is not surprising. Dependencies often break in unexpected moments. In such a case, long-term quality is sacrificed for the short-term goals. Getting a project to compile successfully is more important than fixing broken build system dependencies.

One of the goals of the HBS proposed in the paper is to minimize the probability of user distraction caused by broken dependencies. This implies that the number of dependencies should be kept as low as possible.

Moreover, a low number of dependencies potentially makes the build system more appealing for use in military designs. This is because any external dependency must be verified against potential vulnerabilities.

3.4 Ease of calling arbitrary programs in arbitrary places↩︎

The ease of calling arbitrary programs in arbitrary places results from two use cases.

The first use case originates from the complexity of designing hardware systems. To reduce manual work, engineers often automatically generate parts of the design. For example, register files are automatically generated from SystemRDL [37]. In such a case, there must be a way to call an external program generating hardware description of the register components. Existing indirect abstract hardware build systems allow calling external programs. However, they often require users to define a transitional so-called generator, which increases complexity and reduces legibility. Moreover, they usually do not allow to call external programs in arbitrary places.

The second use case includes support for EDA tools without the Tcl interface. Such tools are often open source simulators, for example, nvc [38], or verilator [39]. These tools have interfaces similar to the standard software compiler. The user simply calls programs with proper arguments and proper order in a shell. Being able to call arbitrary programs in arbitrary places is required to support this kind of tools without an extra script or Makefile generation.

3.5 Independce from version control system↩︎

Some hardware build systems are tightly coupled with a particular version control system. For example, Hog is intended to be used only with git. However, build systems and version control systems are conceptually orthogonal entities. Imposing a particular version control system limits user choice. Moreover, it potentially makes incorporating a new build system for some existing projects impossible, as they might, for example, utilize subversion instead of git.

4 Hardware Build System architecture↩︎

This section describes the architecture of the implemented HBS. The system is available on GitHub [40] with the BSD 3-Clause License.

It was decided that HBS will be implemented using Tcl language because of the following reasons:

1. Implementing a hardware build system in Tcl allows the execution of build system code during the EDA tool flow. This, in turn, gives direct access to all EDA tool custom commands. Moreover, these custom commands can be evaluated in arbitrary places. This aspect satisfies requirements 3.1 and 3.4. The complexity of existing indirect abstract build systems for less common tasks results from executing build system code before EDA tool flow. The build system only prepares scripts that are later run by EDA tools.

2. If the EDA tool provides the Tcl interface, then the Tcl shell is provided by the EDA tool vendor. The shell is installed during the installation of vendor tools. This implies that, in some cases, the build system user does not even have to install additional programs. This aspect satisfies requirement 3.3.

3. Executing arbitrary programs in arbitrary places in Tcl is very simple. There is a dedicated exec command for invoking subprocesses. If executing a subprocess requires prior dynamic arguments evaluation, the exec command call must be prepended with the eval command. Even in Python, invoking a subprocess is not so straighforward. This Tcl feature satisfies requirement 3.4.

One of the most important things while designing the HBS was the separation of common build process operations that would constitute a common abstraction layer over EDA tools. At the end, it was decided that the following actions should constitute the common abstraction layer:

1. Target device setting - some EDA tools use the term “part” instead of “device,” for example, Vivado. Simulation EDA tools do not require a device setting. However, all synthesis EDA tools require information on the target device. This is why setting the device became part of the common abstraction layer.

2. File addition - this includes support for adding files of all formats supported by a given EDA tool.

3. Library setting - setting HDL file library.

4. HDL standard setting - setting HDL file standard revision. The build system has to manage this because some tools can not handle analyzing different design units using different standard revisions, for example, nvc. In such a case, the build system must decide what the common standard revision shall be used for analyzing all HDL files.

5. Dependency specification - this is the core feature of any build system.

6. Generics/parameters setting - configuring parametric designs must be an inherent feature of any hardware build system.

7. Design top module setting - all EDA tools carrying out simulation or synthesis requires information on the top module.

8. Code generation - a hardware build system must provide a simple way for automatic arbitrary code generation. This is further explained in subsection 4.12.

4.1 General structure↩︎

The HBS is implemented in two files hbs.tcl and hbs. The first is implemented in Tcl, and the second in Python. The hbs.tcl file implements all the core features related directly to the interaction with the EDA tools. The hbs.tcl provides the following functions:

1. Dumping information about detected cores in JSON format.

2. Listing cores found in .hbs files.

3. Listing targets for a given core.

4. Running target provided as a command line argument.

The hbs is a wrapper for the hbs.tcl and serves the following additional functions:

1. Showing documentation for hbs Tcl symbols.

2. Generating dependency graph.

3. Listing testbench targets.

4. Running testbench targets.

By default, the user calls the hbs program. However, if none of the additional functions are required, the user can call the hbs.tcl directly. In such a case, the whole build system is limited to a single file. Check Table 1 to see how it compares to the size of existing hardware build systems.

4.2 Cores and cores detection↩︎

When the user executes hbs (or hbs.tcl), all directories, starting from the working directory, are recursively scanned to discover all files with the .hbs extension (symbolic links are also scanned). Files with the .hbs extension are regular Tcl files that are sourced by the hbs.tcl script. However, before sourcing .hbs files, the file list is sorted so that scripts with shorter path depth are sourced as the first ones. For example, let us assume the following three .hbs files were found:

• a/b/c/foo.hbs,

• d/bar.hbs,

• e/f/zaz.hbs.

Then, they would be sourced in the following order:

1. d/bar.hbs,

2. e/f/zaz.hbs,

3. a/b/c/foo.hbs.

Such an approach allows controlling when custom symbols (Tcl variables and procedures) are ready to use. For example, if the user has a custom procedure used in multiple .hbs files, then the user can create separate utils.hbs file contaning utility procedures, and place it in the the project root directory. Within .hbs files, the user usually defines cores and targets, although the user is free to have any valid Tcl code in .hbs files.

Listing [lst:flip-flop-hbs] presents a very basic flip-flop core definition. The flip-flop core has a single target named src. The core consists of a single VHDL file.

namespace eval flip-flop {
  proc src {} {
    hbs::AddFile flip-flop.vhd
  }
  hbs::Register
}

To register a core, the user must explicitly call hbs::Register procedure at the end of the core namespace. Such a mechanism helps to distinguish regular Tcl namespaces from Tcl namespaces representing core definitions. If the user forgets to register a core, the build system gives a potential hint. An example is presented in Listing [lst:hbs-missing-register].

[user@host tmp]$ hbs run lib::core::tb
  core 'lib::core' not found, maybe the core is not registered (hsb::Register)

Each core is identified by its unique path. The core path is equivalent to the namespace path in which hbs::Register is called. Using the namespace path as the core path gives the following possibilities:

1. The user can easily stick to the VLNV identifiers if required. This is presented in Listing [lst:flip-flop-hbs-vlnv]. In this case, the flip-flop core path is vendor::library::flip-flop::1.0.

2. The user can define arbitrary deep core paths (limited by the Tcl shell). This is presented in Listing [lst:flip-flop-hbs-deep-path]. In this case, the core path consists of 7 parts.

3. The user can nest namespaces to imitate the structure of libraries and packages. This is presented in Listing [lst:flip-flops-hbs]. Three flip-flop cores are defined in the snippet. Listing [lst:list-cores-flip-flops] presents output for listing flip-flop cores.

namespace eval vendor::library::flip-flop::1.0 {
  proc src {} {
    hbs::AddFile flip-flop.vhd
  }
  hbs::Register
}

namespace eval a::b::c::d::e::f::flip-flop {
  proc src {} {
    hbs::AddFile flip-flop.vhd
  }
  hbs::Register
}

namespace eval lib {
  namespace eval pkg1 {
    namespace eval d-flip-flop {
      proc src {} {
        hbs::AddFile d-flip-flop.vhd
      }
      hbs::Register
    }
    namespace eval t-flip-flop {
      proc src {} {
        hbs::AddFile t-flip-flop.vhd
      }
      hbs::Register
    }
  }
  namespace eval pkg2 {
    namespace eval jk-flip-flop {
      proc src {} {
        hbs::AddFile jk-flip-flop.vhd
      }
      hbs::Register
    }
  }
}

[user@host tmp]$ hbs list-cores
lib::pkg1::d-flip-flop
lib::pkg1::t-flip-flop
lib::pkg2::jk-flip-flop

4.3 Targets and targets detection↩︎

HBS automatically detects targets. Targets are all Tcl procedures defined in the scope of core namespaces (namespaces with a call to hbs::Register). However, to allow users to define custom utility procedures within cores, procedures with names starting with the floor character (_) are not treated as core targets. Listing [lst:edge-detector-hbs] presents an example edge detector core definition. The core path is vhdl::simple::edge-detector. The core has three targets: src, tb-sync, tb-comb, and one utility procedure _tb. The _tb procedure was defined to share calls common for testbench targets tb-sync and tb-comb. Moreover, all target procedures are also regular Tcl procedures. Such an approach allows for calling them in arbitrary places. The _tb procedure calls the src procedure because the edge_detector.vhd file is required for running testbench targets.

All targets are represented by a unique target path. Target path consists of the core path and target name. For example, the src target of the edge detector has the following path vhdl::simple::edge-detector::src.

namespace eval vhdl::simple::edge-detector {
  proc src {} {
    hbs::SetLib "simple"
    hbs::AddFile src/edge_detector.vhd
  }
  proc _tb {top} {
    hbs::SetTool "ghdl"
    hbs::SetTop $top
    src
    hbs::SetLib ""
  }
  proc tb-sync {} {
    _tb "tb_edge_detector_sync"
    hbs::AddFile tb/tb_sync.vhd
    hbs::Run
  }
  proc tb-comb {} {
    _tb "tb_edge_detector_comb"
    hbs::AddFile tb/tb_comb.vhd
    hbs::Run
  }
  hbs::Register
}

4.4 Testbench targets↩︎

HBS is capable of automatically detecting testbench targets. Testbench targets are targets which names:

1. start with tb- or tb_ prefix,

2. end with -tb or _tb suffix,

3. equal tb.

For example, for the edge detector from Listing [lst:edge-detector-hbs], the hbs program detects testbench targets presented in Listing [lst:edge-detector-tb-targets].

[user@host tmp]$ hbs list-tb
vhdl::simple::edge-detector::tb-comb
vhdl::simple::edge-detector::tb-sync

4.5 Running targets↩︎

HBS allows running any target of registered cores. Even if the target itself has nothing to do with the hardware design. For example, running target print from Listing [lst:print-target-hbs] results with the output shown in Listing [lst:print-target-result].

namespace eval core {
  proc print {} {
    puts "Hello!"
  }
  hbs::Register
}

[user@host tmp]$ hbs run core::print
Hello!

However, in most cases, the user wants to run a target related to the flow of the set EDA tool. In such a case, instead of manually calling all of the required tool commands, the user can call the hbs::Run procedure in the core target procedure. The hbs::Run procedure has an optional argument accepting the stage after which the tool flow should stop. This is further described in subsection 4.9. After hbs::Run returns, the user can continue processing. For example, the user can run scripts analyzing code coverage or preparing additional reports.

4.6 Target parameters↩︎

As core targets are just Tcl procedures, they can have parameters. Moreover, parameters can have optional default values. Additionally, HBS allows to provide command line arguments to the run target. This is a very convenient feature in build systems. Listing [lst:target-parameter-hbs] presents a very simplified example.

namespace eval core {
  proc target {{stage "bitstream"}} {
    puts "Running until $stage"
    # hbs::Run commented out because this is just an example.
    #hbs::Run $stage
  }
  hbs::Register
}

The core does not build any hardware design. However, the example shows how the build stage can be passed from the command line to an EDA tool. Listing [lst:target-parameter-output] presents output from running the target with different stage parameter values.

[user@host tmp]$ hbs run core::target
Running until bitstream
[user@host tmp]$ hbs run core::target synthesis
Running until synthesis

Another practical example of target parameters usage is setting the simulator for testbench target from the command line or changing the top-level module. What target parameters are used for is limited only by the user’s imagination, and Tcl semantics.

4.7 Target context↩︎

An engineer implementing a given core controls the dependencies of the core. However, the core author does not control who will use the core and how. As targets are regular Tcl procedures, there is a need for a mechanism allowing the core author to evaluate the target procedure in the invariant environment. Such a mechanism in HBS is called the target context.

The target context assures that the following variables are not affected by dependee or dependency target execution:

1. HDL library,

2. HDL standard,

3. top module name,

4. arguments prefix,

5. arguments suffix,

6. the core path,

7. the target name,

8. the target path.

Listing [lst:target-context] presents an example of the target context mechanism.

namespace eval pkg {
  namespace eval foo {
    proc src-foo {} {
      hbs::SetLib "lib-foo"
      hbs::AddDep pkg::bar::src-bar
      puts "foo lib: $hbs::Lib"
      puts "foo core: $hbs::ThisCorePath"
      puts "foo target: $hbs::ThisTarget"
    }
    hbs::Register
  }
  namespace eval bar {
    proc src-bar {} {
      hbs::SetLib "lib-bar"
      puts "bar lib: $hbs::Lib"
      puts "bar core: $hbs::ThisCorePath"
      puts "bar target: $hbs::ThisTarget"
    }
    hbs::Register
  }
}

Listing [lst:target-context-output] presents the output from running the pkg::foo:src-foo target. As can be seen, setting library in a target of one core, does not affect library in the target of another core.

[user@host tmp] hbs run pkg::foo::src-foo
bar lib: lib-bar
bar core: pkg::bar
bar target: src-bar
foo lib: lib-foo
foo core: pkg::foo
foo target: src-foo

An author of a core can not easily by mistake break execution of another core target procedure. The target context provides the isolation of target procedure evaluation environments.

4.8 Target dependencies↩︎

In HBS, targets might depend on other targets instead of cores depending on cores. Such an approach allows for fine-grained control of dependencies.

To declare target dependency, the user must call the hbs::AddDep procedure within the target procedure. The first argument is the dependency path. The remaining arguments are optional and are passed to the dependency procedure as arguments.

To add multiple distinct dependencies, the user must call hbs::AddDep multiple times. The ability to pass custom arguments to dependency was evaluated as much more advantageous than the ability to add multiple dependencies with a single hbs::AddDep call.

The hbs::AddDep internally calls the dependency procedure with the provided arguments. It also tracks dependencies so that generating a dependency graph is possible. Within a single flow, each target procedure can be run at most once with a particular set of arguments. This implies that if multiple target procedures add the same dependency with the same arguments, the dependency procedure is run only once during the first hbs::AddDep call. To enforce some target procedure rerun, the user can always directly call the target. However, enforcing target procedure rerun usually is an alert that a regular Tcl procedure shall be used instead of the core target procedure.

Listing [lst:target-dependency-hbs] contains an example core definitions for presenting target dependency rules. There are four cores: core-a, core-b, core-c, generator-core. core-a depends on core-b and core-c. core-b depends on core-c. Moreover, cores core-a and core-b depends on the generator-core. However, they use different argument values for generation.

namespace eval core-a {
  proc target {} {
    hbs::AddDep core-b::target
    hbs::AddDep core-c::target
    hbs::AddDep generator-core::gen a
    hbs::AddDep generator-core::gen x
    puts "core-a::target"
  }
  hbs::Register
}
namespace eval core-b {
  proc target {} {
    hbs::AddDep core-c::target
    hbs::AddDep generator-core::gen b
    hbs::AddDep generator-core::gen x
    puts "core-b::target"
  }
  hbs::Register
}
namespace eval core-c {
  proc target {} {
    puts "core-c::target"
  }
  hbs::Register
}
namespace eval generator-core {
  proc gen {arg} {
    puts "generator-core::gen $arg"
  }
  hbs::Register
}

Listing [lst:target-dependency-output] presents output from running target core-a::target. As can be seen, the core-c::target is run only once, even though both core-a::target and core-b::target depend on it. This is because core-c::target has no arguments and can be added as a dependency only once. On the other hand, generator-core::gen is run three times. This is because generator-core::gen is added as a dependency four times, and three times with a distinct argument value.

[user@host tmp]$ hbs run core-a::target
core-c::target
generator-core::gen b
generator-core::gen x
core-b::target
generator-core::gen a
core-a::target

4.9 EDA tool flow and stages↩︎

The primary function of any hardware build system is to provide the ability to build a design. What the term “build” actually means usually depends on the EDA tool. For some, it is only synthesis; for others, it is synthesis and implementation, and yet for others it might be simulation or just code linting. Each EDA tool is characterized by a distinct flow consisting of different stages. To mimic this behavior, HBS supports the following stages (alphabetical order):

1. analysis - HDL files analysis,

2. bitstream - device bitstream generation,

3. elaboration - design elaboration,

4. implementation - design implementation,

5. project - project creation,

6. simulation - design simulation,

7. synthesis - design synthesis.

Not all stages make sense for all EDA tools. Which stages are present in the given tool flow depends on that tool implementation. Namely, on the implementation of the hbs::<tool>::run procedure. This procedure is called at the end of the hbs::Run procedure execution.

Figure 1 shows the structure of the tool flow for the Vivado project mode. There are four stages: project, synthesis, implementation and bitstream. Each stage is wrapped by custom, user-defined callbacks. The number of callbacks is unlimited. Callbacks in any pre or post stage are executed in the order users add them. The user adds a callback by calling a dedicated HBS Application Programming Interface (API) procedure. For example, to add a post synthesis callback user can call the hbs::AddPostSynthCb procedure. Callbacks can be added with custom argument values.

The post stage callbacks and pre stage callbacks for adjacent stages were added for two reasons.

1. To clearly communicate which stage the callback refers to. For example, configuring implementation settings based on the synthesis results can be done in a post-synthesis callback or pre-implementation callback. However, as the callback modifies the implementation settings, it is probably better to add it using the hbs::AddPreImplCb, than hbs::AddPostSynthCb.

2. To introduce, to some extent, a manageable order of callbacks execution. The pre and post callbacks of a given stage are executed in the order they are added. If some nested dependencies add callbacks for the same pre or post stage, then the order of callbacks execution depends on the order of hbs::AddDep calls. However, if the result of one of the callbacks depends on the result of the other one, then relying on a user to call the hbs::AddDep procedures in proper order is error prone. In such a case, the callback that must be executed as the first one can be added to the post-synthesis callbacks, and the second callback can be added to the pre-implementation callbacks. Such an approach is immune to the order of hbs::AddDep calls.

The user can utilize stage callbacks in any desired way. However, the primary purpose of stage callbacks is to adjust the design build based on the results from a particular stage. For example, the user might want to configure additional implementation settings based on the synthesis results. The user might even terminate the tool flow in a given callback and report an error if certain conditions are not met.

4.10 EDA tool commands custom arguments↩︎

The HBS has a minimal common abstraction layer. The user performs all the actions not covered by the abstraction layer by directly calling EDA tool commands. For example, generating extra timing or clock network reports. However, some of the actions are hidden under the HBS API. For example, adding or analyzing HDL files. EDA tool commands used to perform actions hidden under common API usually have some parameters that are not used by default. Nevertheless, sometimes there is a need to specify additional parameters. In such a case, there are two possible solutions.

The first option is to bypass the HBS API and directly call the underlying EDA tool command. The drawback of this approach is that the user must manually handle the current context. For example, when adding an HDL file, the user must manually specify the library or standard revision. Bypassing HBS API also bypasses the target context.

The second option is to set the underlying command arguments prefix or suffix. This can be achieved with the hbs::SetArgsPrefix and hbs::SetArgsSuffix procedures. The argument prefix is always appended after the command name, and the argument suffix is always appended after all command arguments.

Listing [lst:eth-mdio] presents Ethernet Management Data Input/Output (MDIO) core definition. The core has one testbench target utilizing the nvc simulator. Nvc report messages occupy multiple lines by default. However, this can be changed by specifying the --messages=compact argument when running the simulation. As running the simulation is the last stage of the nvc flow, the call to hbs::SetArgsPrefix must be wrapped by the call to the hbs::AddPostElabCb procedure.

namespace eval vhdl::ethernet {
  namespace eval mdio {
    proc src {} {
      hbs::SetLib "ethernet"
      hbs::AddFile mdio.vhd
    }
    proc tb {} {
      hbs::SetTool "nvc"
      hbs::AddPostElabCb hbs::SetArgsPrefix "--messages=compact"
      src

      hbs::SetLib ""
      hbs::AddFile tb/tb-mdio.vhd
      hbs::SetTop "tb_mdio"
      hbs::Run
    }
    hbs::Register
  }
}

Listing [lst:eth-mdio-tb-output] shows commands executed by the hbs to run the simulation. The --messages=compact argument was appended right after the nvc command.

[user@host vhdl-ethernet]$ hbs run vhdl::ethernet::mdio::tb
nvc --std=2019 -L. --work=ethernet -a vhdl/vhdl-ethernet/mdio.vhd
nvc --std=2019 -L. --work=work -a vhdl/vhdl-ethernet/tb/tb-mdio.vhd
nvc --std=2019 -L. -e tb_mdio
nvc --messages=compact --std=2019 -L. -r tb_mdio --wave

4.11 HBS API extra symbols↩︎

The HBS API consists not only of symbols related to the common EDA abstraction layer. For example, there are extra hbs::Exec and hbs::CoreDir procedures. The first one is a wrapper for the Tcl standard exec procedure. Before calling exec, the hbs::Exec changes the working directory to the directory where the currently evaluated core is defined. When exec returns, the hbs::Exec restores the working directory. The hbs::CoreDir procedure allows the user to get the path of the directory in which the currently evaluated core is defined.

HBS also provides users with the following extra variables:

1. hbs::ThisCorePath - the path of the core which target is currently being run,

2. hbs::ThisTargetPath - the path of the target which is currently being run,

3. hbs::ThisTargetName - the name of the target which is currently being run,

4. hbs::TopCorePath - the path of the top core beign run,

5. hbs::TopTargetPath - the path of the top target being run,

6. hbs::TopTarget - the name of the top target being run,

7. hbs::TopTargetArgs - the list with command line arguments passed to the top target.

4.12 Code generation↩︎

Code generation in the hardware design domain is omnipresent, as it significantly speeds up the implementation process. For example, hardware-software co-design system on chip projects usually have some tool automatically generating register files [41]. Even in pure FPGA designs, it is common to generate descriptions at the register transfer level from some higher-level programming abstraction [42], [43]. That is why it is important for any hardware build system to provide as simple mechanism for code generation as possible.

Some existing hardware build systems, for example FuseSoc, do not allow the direct calling of an arbitrary external program in arbitrary places during the build process. Instead, the user has to define so-called generators [44]. Only then can the user call the generator wihtin core definitions. However, such an approach has some drawbacks:

1. The generator call requires an extra layer of indirection. Generators are defined in different places than they are used, which decreases the readability of the description.

2. The generator call syntax does not resemble shell command call syntax. Generators are usually regular applications that can be executed in a shell. Calling a generator within the build system using syntax similar to shell seems natural.

In HBS, there is no formal concept of generator. Anything can be a generator, as generators are just regular Tcl procedures. This means that generators can be target procedures (tracked by dependency system) or core internal Tcl procedures (not tracked by the dependnecy system).

Listing [lst:hbs-generator-dep] presents an example of calling an external code generator tracked by the dependnecy system. In actual usage, the call to the shell echo command would be replaced with a call to the proper code generator program. Calls to the hbs::AddFile are commented out because no EDA tool was set.

namespace eval core {
  proc top {} {
    hbs::AddDep generator::gen "foo"
    puts "Adding file top.vhd"
    # hbs::AddFile top.vhd
  }
  hbs::Register
}
namespace eval generator {
  proc gen {name} {
    exec echo "Generating $name.vhd" >@ stdout
    puts "Adding file $name.vhd"
    # hbs::AddFile "$name.vhd"
  }
  hbs::Register
}

Listing [lst:hbs-generator-no-dep] presents how to achieve the same result without tracking the generator as a dependency. This task is even more straightforward, as the user can call an external generator program directly in the target procedure.

namespace eval core {
  proc top {} {
    exec echo "Generating foo.vhd" >@ stdout
    puts "Adding file foo.vhd"
    # hbs::AddFile "foo.vhd"

    puts "Adding file top.vhd"
    # hbs::AddFile top.vhd
  }
  hbs::Register
}

5 Hardware Build System Commands↩︎

The implemented hbs program provides the user with 11 commands. Listing [lst:hbs-help-msg] presents hbs help message. The help and version commands are not discussed in this section, as their purpose is rather obvious.

Usage:

  hbs <command> [arguments]

The command is one of:

  doc           Show documentation for hbs Tcl symbol or EDA tool
  dump-cores    Dump info about cores in JSON format
  graph         Output dependency graph for given target
  help          Print help message for program or given command
  list-cores    List cores
  list-targets  List targets
  list-tb       List testbench targets
  run           Run given target
  test          Run testbench targets
  version       Print hbs version
  where         Print where given cores are defined

Type 'hbs help <command>' to obtain more
information about particular command.

5.1 Viewing HBS API documentation↩︎

The doc command was added to ease viewing documentatino for HBS Tcl symbols. Listing [lst:hbs-doc] presents an example of the doc command usage.

[user@host tmp]$ hbs doc AddFile
# AddFile adds files to the tool flow. Multiple files with different
# extensions can be added in a single call. args is the list of
# patterns used for globbing files. The file paths are relative to
# the `.hbs` file path where the proc is called.
proc AddFile {args}

5.2 Dumping cores information↩︎

The dump-cores command allows dumping information about found cores into JSON format. The generated JSON data can be used for further processing. For example, the graph command utilizes data from JSON dump to generate the dependency graph.

5.3 Generating dependency graph↩︎

The graph command allows generating target dependency graphs in PDF format. Listing [lst:graph-help] shows the graph command help message.

Output dependency graph for given target.

  hbs graph path/to/file/with/cores.json [top-target-path]

If top target path is not provided, then it is assumed to be the
same as the base of the json file path, without the '.json' suffix.

The graph command requires information about cores in the JSON format. This implies that the user must execute the dump-cores or run command before generating a dependency graph. However, this is not a major issue in practice. Dumping cores, even for large designs, does not take more than a few seconds, as the dump-cores command does not run the target tool flow.

Figure 2 presents an example dependency graph generated for VSC8211 (Ethernet PHY) chip tester design.

5.4 Listing cores↩︎

The list-cores command allows listing all cores discovered by the hbs program. Listing [lst:list-cores-help] shows list-cores command help message.

List cores found in .hbs files.

  hbs list-cores [core-path-patterns...]

If core-path-patterns are not provided, all found cores are listed.
If core-path-patterns are provided, then only cores whose path
contains at least one pattern are listed.

For example, let us assume the user wants to list all cores in a project related to the APB bus. Listing [lst:list-apb] presents the output of such an action carried out in the VSC8211 tester design.

[user@host tmp] hbs list-cores apb
vhdl::amba5::apb::bfm
vhdl::amba5::apb::checker
vhdl::amba5::apb::crossbar
vhdl::amba5::apb::mock-completer
vhdl::amba5::apb::pkg
vhdl::amba5::apb::serial-bridge
vhdl::amba5::apb::shared-bus

5.5 Listing targets↩︎

The list-targets command allows listing all targets discovered by the hbs program. Listing [lst:list-targets-help] shows the list-targets command help message. The list-targets command is analogous to the list-cores command but works on targets instead of cores.

List targets for cores found in .hbs files.

  hbs list-targets [target-path-patterns...]

If target-path-patterns are not provided, all found targets
are listed. If target-path-patterns are provided, then only
targets whose path contains at least one pattern are listed.

For example, if you want to list all targets for a given core,
simply provide the core path as the argument.

For example, let us assume the user wants to list all targets of the APB serial bridge core. Listing [lst:list-serial-bridge] presents the output of such an action carried out in the VSC8211 tester design.

[user@host tmp] hbs list-targets apb::serial
vhdl::amba5::apb::serial-bridge::src
vhdl::amba5::apb::serial-bridge::tb-write
vhdl::amba5::apb::serial-bridge::tb-read

5.6 Listing testbench targets↩︎

The list-tb command allows listing all testbench targets discovered by hbs. The list-tb command is analogous to the list-targets command, but it works on testbench targets instead of all targets. For example, let us assume the user wants to list all testbench targets of the APB serial bridge core. Listing [lst:list-tb-serial-bridge] presents the output of such an action carried out in the VSC8211 tester design. As can be seen, the src target is not included.

[user@host tmp] hbs list-tb apb::serial
vhdl::amba5::apb::serial-bridge::tb-write
vhdl::amba5::apb::serial-bridge::tb-read

5.7 Running target↩︎

The run command allows running target procedures. Usually, targets are run to carry out the build process or simulation. However, the user is free to carry out any action in the target being run. Listing [lst:run-help] shows the run command help message. Running targets is described in subsections 4.5, 4.6, and 4.7.

Run provided target.

  hbs run <target-path> [target-args...]

The target path must be absolute target path containing the core
path and target name. Target arguments are forwarded to the target
procedure call.

Example:

  hbs run core::path::target-name synthesis

5.8 Running testbench targets↩︎

The test command allows running all automatically discovered testbench targets. Listing [lst:test-help] shows the test command help message.

Run testbench targets.

  hbs test [-workers N] [target-path-patterns...]

Testbench targets are detected automatically.
A testbench target is a target which name:
  - starts with "tb-" or "tb_",
  - ends with "-tb" or "_tb",
  - equals "tb".

-workers specifies the number of targets to run simultaneously
(in parallel). N must be a positive integer. -workers must
be provided before target path patterns. Otherwise, it will
be treated as one of the target path patterns. By default,
the number of workers equals the number of available CPUs.

If target path patterns are not provided, all testbench targets
are run. If target path patterns are provided, then only testbench
targets whose path contains at least one pattern are run.

5.9 Locating core definition↩︎

The where command allows easily locating .hbs files in which given cores are defined. Listing [lst:where-help] shows the where command help message.

Print where given cores are defined.

  hbs where [core-path-patterns...]

If core-path-patterns are not provided, all found cores are printed.
If core-path-patterns are provided, then only cores whose path
contains at least one pattern are printed.

For example, let us assume the user wants to locate the APB serial bridge core. Listing [lst:where-apb] presents the output of such an action carried out in the VSC8211 tester design.

[user@host tmp] hbs where apb::serial-bridge
vhdl::amba5::apb::serial-bridge /tmp/vsc8211-tester/gw/apb/apb.hbs

6 Hardware Build System Evaluation↩︎

This section tries to evaluate the implemented HBS in terms of other hardware build systems.

Comparing build systems is not straightforward. There are no objective numerical metrics that can be used for build systems comparison. One of such metrics could be the build time. However, in the case of hardware designs, the share of time consumed by EDA tools is orders of magnitude greater than the time overhead introduced by the build system code execution.

Some generic software evaluation metrics could be used, such as numerical or memory complexity. However, the memory overhead introduced by build systems is neglectable compared to the memory utilization of EDA tools. The numerical complexity might be important for the build system developers, but it is rather irrelevant for build system users.

This section tries to evaluate the HBS from some practical viewpoints, such as dependencies, overall internal complexity, ease of use, and provided functionalities.

6.1 Dependencies↩︎

The implemented HBS has only the following three dependencies:

1. Tcl shell (mandatory) - required to run the core logic of the build system.

2. Python (optional) - required to run commands not directly related to the build (automatic testbench running and dependency graph generation).

3. Graphviz (optional) - required for the graph command to generate a dependency graph in PDF format.

None of the existing hardware build systems capable of similar features has such a low number of dependencies.

6.2 Overall internal complexity↩︎

The measure used to estimate the overall internal complexity of the build system is the number of files, the number of lines of code, and the number of dependencies. The following type of files were excluded from analysis: author files, news files, contributors files, license files, readme files, tests files, and example files.

Table 1 presents metric values for the analyzed hardware build systems. The “\(\geq N\)” signature means at least \(N\). For build systems implemented in Python, the number of dependencies was analyzed using the pipreqs [45] tool. For build systems implemented in Rust, the number of dependencies was extracted from the Cargo.toml files. For flgen and bazel_hdl_rules, the number of dependencies was analyzed manually. Determining the exact number of dependencies (except for HBS) was not straightforward, as there might be additional runtime dependencies. Hence, the need for the greater-than-equal sign.

Besides the complexity metrics, the table also includes a summary of hardware build system features. The Introduced File Extensions is the number of custom file extensions used by a build system. The Introduced File Syntaxes is the number of custom syntaxes used by a build system. For example, FuseSoc scans for files with the .core extension. However, the .core files have a standard YAML syntax. This is why the number of introduced file extensions equals 1, but the number of introduced file syntaxes equals 0.

An extra context has to be introduced to the table, as different tools support different features and EDA tools.

Direct Tcl build systems (except HBS) are not included in the table. This is because they support only one EDA tool, and they do not support automatic testbench detection and dependency graph generation. OSVVM-Script is not included because it targets only simulation tools.

The FuseSoc build tool internally utilizes the external Edalize [46] library for interaction with the EDA tools. Including Edalize, the number of files for FuseSoc equals 215, and the number of lines of code equals 19788.

The bender is only a dependency management tool. It provides a way to define dependencies among cores, execute unit tests, and verify that the source files are valid input for various simulation and synthesis tools. However, it does not run the EDA tool for building designs.

The flgen is just a file list generator. It does not run EDA tools for building designs.

Only HBS, SiliconCompiler, bender, and orbit are capable of generating dependency graphs.

Total number of lines of code of any hardware build system greatly depends on the number of supported EDA tools. FuseSoc supports the greatest number of tools, almost 40. HBS currently supports only five tools (AMD Vivado, AMD xsim, GHDL, GOWIN IDE, nvc). The most complex one is Vivado, which full support requires roughly 220 lines of code. However, supporting the nvc simulator required roughly 170 lines of code. Assuming that HBS will one day support 40 EDA tools, the number of lines of code should still be below 10000.

6.3 Ease of use↩︎

The subsection presents the ease of use of HBS in a very common task. Namely, the engineer implements a module and wants to scope some constraints to this module. As an example, a two-stage flip-flop clock domain crossing synchronizer is used. The source code of the synchronizer is not relevant to the analysis. False path is set as a constraint, although the max delay is also frequently used.

Listing [lst:fusesoc-synchronizer] presents the synchronizer core definition in FuseSoc.

CAPI=2:

name: vendor:library:synchronizer

filesets:
  src:
    files:
      - synchronizer.vhd
    file_type: vhdlSource-2008
  vivado_constr:
    files:
      - vivado-constr.xdc: {file_type: xdc}
  tcl:
    files:
      - constr.tcl: {file_type: tclSource}

targets:
  default:
    filesets:
      - src
      - "tool_vivado? (vivado_constr)"
      - tcl

Listing [lst:fusesoc-vivado-constr] presents the content of the vivado-constr.xdc file and Listing [lst:fusesoc-constr-tcl] presents the content of the constr.tcl file.

set_false_path -to [get_pins s_0_reg[*]/D]

if {[string match "Vivado*" [version]]} {
  set_property SCOPED_TO_REF Synchronizer [get_files vivado-constr.xdc]
} else {
  error "Synchronizer entity misses constraint file for your EDA tool"
}

In FuseSoc, to accomplish the task, the user must provide four files:

1. synchronizer.vhd - synchronizer HDL description,

2. vivado-constr.xdc - false path Vivado constraint for synchronizer,

3. constr.tcl - Tcl script scoping constraint to synchronizer module.

4. synchronizer.core - synchronizer core definition.

The number of required files is greater than the number of logical elements. The user describes the module in HDL, defines the core for the build system, and adds extra constraints. The number of logical elements equals three, not four. Moreover, the logic of the core definition is more complex to follow, as the user has to switch between files for a very short snippet of code. This leads to a decrease in readability.

Listing [lst:hbs-synchronizer] presents the same core definition but in the HBS format. The number of required files equals three and is the same as the number of logical elements. The constr.tcl file could be eliminated because the code of the build system is executed directly by the EDA tool. Even the error message is improved. It is possible to report precisely for which EDA tool the constraint file definition is missing. Again, this is possible because the code of the build systems is executed directly by the EDA tool.

In the case of FuseSoc, build scripts are prepared before the EDA tool is run. Within the constr.tcl script, there is no access to the FuseSoc context and variables, as the build system code execution is already finished.

namespace eval vendor::library::synchronizer {
  proc src {} {
    hbs::AddFile synchronizer.vhd

    if {$hbs::Tool == "vivado-prj"} {
      hbs::AddFile vivado-constr.xdc
      set_property SCOPED_TO_REF Synchronizer [get_files vivado-constr.xdc]
    } else {
      error "Synchronizer entity misses constraint file for $hbs::Tool"
    }
  }

  hbs::Register
}

7 Conclusions↩︎

This work proposes a new Tcl-based hardware build system with the following features:

1. Low internal code complexity.

2. Minimal command line user interface.

3. Minimal Tcl user API.

4. Minimal common abstraction layer for EDA tools.

5. Minimal number of dependencies.

6. Ease of custom interaction with EDA tools.

7. Build system code executed directly by EDA tools during the build process.

8. Version control system agnosticity.

Although the proposed HBS represents a minimal approach in multiple aspects, it does not miss any significant features present in the existing build systems. It even supports functionalities absent in the existing systems.

While the implemented build system might not be the best choice for all kinds of projects, its unique set of features makes it appealing, especially for projects targeting multiple devices with multiple configurations.

The proposed Hardware Build System is novel because:

1. It is the only Tcl-based hardware build system supporting multiple EDA tools.

2. It is the only Tcl-based hardware build system capable of dependency graph generation, automatic testbench targets detection, and parallel testbench running.

3. It is the only hardware build system providing features described in bullets 1 and 2, allowing direct interaction with EDA tools during the build time. This is possible because the HBS code is directly executed by EDA tools.

References↩︎