BraneTechnologies Documentation

Understanding the Multi-Project Structure #

At its core, the Brane SDK uses a hierarchical project organization to manage multiple applications targeting different hardware platforms. This approach strikes a balance between organization, flexibility, and reuse.

The Project Hierarchy #

When you examine the Brane SDK project structure, you’ll see a carefully designed hierarchy:

demo-project/
├── build.gradle              # Root project configuration
├── settings.gradle           # Defines which subprojects to include
└── Toolkit/                  # Main toolkit container
    └── demo/                 # Demo applications
        ├── x86_64_CPU/       # CPU-targeted applications
        │   ├── FFTConvolver/
        │   └── matrixMultiply/
        ├── coolidge/         # Kalray MPPA applications
        │   ├── FFT/
        │   ├── FluidX3D/
        │   └── matrixMultiplyOcl/
        └── fpga/             # FPGA applications
            └── gplgpu/

This structure follows a logical organization:

• The top level provides common configuration and project definitions
• The middle level separates projects by target platform
• The leaf level contains individual applications with specific implementations

This approach offers several advantages:

• You can easily find applications by their target platform
• Common settings can be shared at the appropriate level
• You can build everything at once or focus on specific platforms/applications

Project Paths: Important Note #

Throughout this documentation and in Gradle commands, projects are referenced using path notation:

:Toolkit:demo:x86_64_CPU:matrixMultiply

This path directly corresponds to the directory structure:

Toolkit/demo/x86_64_CPU/matrixMultiply

Note: When referring to projects in include statements within settings.gradle, the root project name is included:

include 'demo-project:Toolkit:demo:x86_64_CPU:matrixMultiply'

However, when running Gradle tasks from the command line, the root project name is omitted:

./gradlew :Toolkit:demo:x86_64_CPU:matrixMultiply:build

The settings.gradle File: Defining Your Project Universe #

The settings.gradle file serves as the blueprint for your project structure, telling Gradle which projects to include in the build:

rootProject.name = 'demo-project'

include 'demo-project:Toolkit:demo:x86_64_CPU:FFTConvolver'
include 'demo-project:Toolkit:demo:x86_64_CPU:matrixMultiply'

include 'demo-project:Toolkit:demo:coolidge:FFT'
include 'demo-project:Toolkit:demo:coolidge:FluidX3D'
include 'demo-project:Toolkit:demo:coolidge:matrixMultiplyOcl'

include 'demo-project:Toolkit:demo:fpga:gplgpu'

This file plays a crucial role by:

• Setting the root project name (demo-project)
• Explicitly including each subproject using directory paths (with colons as separators)
• Allowing you to selectively include or exclude projects (notice the commented-out Bitcoin project)

When working with the SDK, you can easily add your own projects by adding new include statements with the appropriate path. For example, if you wanted to add a new CPU project named “ImageProcessor”, you would add:

include 'demo-project:Toolkit:demo:x86_64_CPU:ImageProcessor'

Similarly, you can temporarily remove projects from the build by commenting out their include statements, which can be useful when you want to focus on specific projects or troubleshoot build issues.

Configuration Inheritance: The Power of Gradle Build Files #

One of the most powerful aspects of the Brane SDK build system is its use of configuration inheritance through a hierarchy of build.gradle files. This approach follows the “Don’t Repeat Yourself” (DRY) principle, placing common configuration at higher levels while allowing specific overrides at lower levels.

Root build.gradle: Global Configuration #

The root build.gradle file establishes settings that apply to all projects:

// Configure build script repositories and dependencies
buildscript {
    repositories {
        // Brane plugin repository (requires AWS credentials)
        maven {
            url "s3://com.brane.repository.maven/release/"
            credentials(AwsCredentials) {
                // AWS credentials can be provided as environment variables or in gradle.properties
                accessKey = System.getenv('AWS_ACCESS_KEY_ID') ?: findProperty('aws_access_key_id')
                secretKey = System.getenv('AWS_SECRET_ACCESS_KEY') ?: findProperty('aws_secret_access_key')
            }
        }
        // Public repositories for additional dependencies
        mavenCentral()
        jcenter()
    }
    
    // Required dependencies for the build process
    dependencies {
        // Brane-Builder plugin - core of the SDK
        classpath 'com.brane.plugins:brane-builder:2.0.1'
        // Apache Velocity for template processing
        classpath 'org.apache.velocity:velocity:1.7'
        // Add any additional build-time dependencies here
    }
}

// Common configurations that apply to all projects
allprojects {
    repositories {
        mavenCentral()
    }
}

This file typically handles:

• Repository definitions (where to find dependencies)
• Global project properties (group, version)
• Common plugin applications
• Credential management
• Environment setup

Application-Specific Configuration #

Finally, each individual application has its own build.gradle with configurations specific to that application:

/*
 * Copyright (C) 2018 Brane Technologies LLC - All Rights Reserved
 * You may use this code under the terms of the Brane license.
 *
 * You should have received a copy of the Brane license with
 * this file. If not, please write to: atif@branetechnologies.com; or visit http://branetechnologies.com
 *
 * Author: Nicolas Siret
 * Contact: nicolas@branetechnologies.com
 * 2018-10
 *
 */

plugins {
    id 'com.brane.coolidge.cpp'
}

application {
    targetMachines = [
      machines.linux.x86_64,
      coolidgeMachines.clusterOS.architecture("MPPA_v2")
    ]

    binaries.configureEach {
        def compileTask = compileTask.get()
        if (toolChain instanceof GccCompatibleToolChain) {
            compileTask.compilerArgs.addAll(['-lOpenCL', '-pthread'])
        }

        def linkTask = linkTask.get()
        if (toolChain instanceof GccCompatibleToolChain) {
            linkTask.linkerArgs.addAll(['-lOpenCL', '-pthread'])
        }
    }
}

These application-level configurations:

• Define specific dependencies unique to this application
• Override default settings when necessary
• Add custom tasks for application-specific operations
• Configure application-specific compiler flags or options

This three-level configuration hierarchy gives you the best of both worlds: consistent settings across similar projects while still allowing customization where needed.

Building and Running Projects #

The Brane SDK’s build system, build on top of Gradle build system, provides a consistent set of commands across platforms, with platform-specific extensions where needed.

Common Build Commands #

Regardless of the target platform, you can use IntelliJ or these standard Gradle commands:

# From the root directory, build everything
./gradlew build

# Build just one platform's projects
./gradlew :DevToolkit:demo:coolidge:build

# Build one specific project
./gradlew :DevToolkit:demo:coolidge:matrixMultiplyOcl:build

# Clean the build directory
./gradlew clean

# Assemble without running tests
./gradlew assemble

The hierarchical project structure gives you precise control over what gets built, saving time during development. You can:

• Build everything when preparing a complete release
• Build just one platform when focusing on platform-specific work
• Build just one application when iterating on a specific feature

For CPU Applications #

# Run the application (standard Gradle run task)
./gradlew :DevToolkit:demo:x86_64_CPU:FFTConvolver:run

# Run with specific arguments
./gradlew :DevToolkit:demo:x86_64_CPU:FFTConvolver:run --args="--input=sample.wav --output=processed.wav"

For Coolidge (MPPA) Applications #

# Run on emulator (faster but less accurate)
./gradlew :DevToolkit:demo:coolidge:FFT:emulate

# Run with detailed simulation
./gradlew :DevToolkit:demo:coolidge:FFT:simulate

# Deploy to connected hardware
./gradlew :DevToolkit:demo:coolidge:FFT:linuxRun
./gradlew :DevToolkit:demo:coolidge:FFT:jtagRun

For FPGA Applications #

# Generate FPGA project files
./gradlew :DevToolkit:demo:fpga:gplgpu:generateProject

# Run behavioral simulation
./gradlew :DevToolkit:demo:fpga:gplgpu:check

# Synthesize the design
./gradlew :DevToolkit:demo:fpga:gplgpu:compileProject

These platform-specific commands are defined by the respective Brane SDK plugins applied in the project’s build.gradle files. They abstract away the complexity of working with different toolchains and provide a consistent interface.

Advanced Build System Features #

The Brane SDK build system includes several advanced features that help manage complex multi-platform projects.

Conditional Configuration #

You can apply configuration conditionally based on various factors:

// Apply configuration based on the build type
tasks.withType(CppCompile) { task ->
    if (task.name.contains('Release')) {
        // Apply optimization for release builds
        task.compilerArgs.addAll(['-O3', '-march=native'])
    } else {
        // Enable debugging for debug builds
        task.compilerArgs.addAll(['-g', '-O0'])
    }
}

// Apply configuration based on the environment
if (System.getenv('CI') != null) {
    // Special configuration for continuous integration environment
    allprojects {
        tasks.withType(Test) {
            testLogging.showStandardStreams = true
        }
    }
}

Cross-Project Dependencies #

You can establish dependencies between projects:

// DevToolkit/demo/coolidge/FluidX3D/build.gradle
dependencies {
    // Depend on another project in the build
    implementation project(':DevToolkit:demo:coolidge:commonUtils')
}

This is powerful for creating shared libraries or utilities that multiple projects can use.

Environment-Specific Properties #

You can define properties specific to different environments or configurations:

// In gradle.properties
coolidge.default.processors=5
coolidge.high.processors=80

// In build.gradle
application {
    // Use environment-specific configuration
    def processorCount = findProperty("coolidge.${System.getenv('SCALE') ?: 'default'}.processors")
    
    binaries.configureEach {
        compileTask.get().compilerArgs.add("-DPROCESSOR_COUNT=${processorCount}")
    }
}

This approach helps manage configurations across different development, testing, and production environments.

Practical Examples and Best Practices #

Let’s explore some practical examples and best practices for working with the Brane SDK build system.

Adding a New Application #

To add a new application to the SDK, e.g.

DevToolkit/demo/x86_64_CPU/MyNewApp/src/main/cpp

include 'DevToolkit:demo:x86_64_CPU:MyNewApp'

// DevToolkit/demo/x86_64_CPU/MyNewApp/build.gradle
plugins {
    id 'com.brane.cpu.cpp-application'
}

application {
    targetMachines = [machines.linux.x86_64]
}

dependencies {
    // Add your dependencies here
}

./gradlew :DevToolkit:demo:x86_64_CPU:MyNewApp:build
./gradlew :DevToolkit:demo:x86_64_CPU:MyNewApp:run

Managing Build Performance #

For larger projects, build time can become an issue. Here are some strategies to improve build performance:

# In gradle.properties org.gradle.daemon=true

# In gradle.properties org.gradle.parallel=true

# In gradle.properties org.gradle.caching=true

# Build just what you're working on ./gradlew :DevToolkit:demo:coolidge:myProject:build

# In gradle.properties org.gradle.configureondemand=true

Troubleshooting Common Issues #

When working with the Brane SDK build system, you might encounter some common issues. Here’s how to address them:

Missing Dependencies #

Could not resolve artifact 'com.example:library:1.0.0'

Could not find method implementation() for arguments [...]

repositories {
    maven {
        url "s3://com.brane.repository.maven/release/"
        credentials(AwsCredentials) {
            accessKey = System.getenv('AWS_ACCESS_KEY_ID')
            secretKey = System.getenv('AWS_SECRET_ACCESS_KEY')
        }
    }
    mavenCentral()
}

# Verify environment variables are set echo $AWS_ACCESS_KEY_ID echo $AWS_SECRET_ACCESS_KEY

./gradlew build --refresh-dependencies

./gradlew dependencies

Incorrect Project Paths #

Project with path ':DevToolkit:demo:x86_64_CPU:matrixMultiply' could not be found

# Correct format
include 'demo-project:Toolkit:demo:x86_64_CPU:matrixMultiply'

# Correct format (note the leading colon)
./gradlew :Toolkit:demo:x86_64_CPU:matrixMultiply:build

./gradlew projects

Gradle Wrapper Issues #

chmod +x gradlew

gradle wrapper --gradle-version 7.4.2

CPU Development Issues #

binaries.configureEach {
    compileTask.get().compilerArgs.add('-std=c++17')
    linkTask.get().linkerArgs.add('-pthread')
}

# Check library paths on Linux
echo $LD_LIBRARY_PATH

# On Ubuntu/Debian
sudo apt-get install libopenblas-dev libfftw3-dev

Kalray MPPA (Coolidge) Issues #

# Check if environment variables are set
echo $KVXOPENCLDIR

# Set up environment if needed
export KVXOPENCLDIR=/opt/kalray/accesscore/lib/target/kvx-cos

// Ensure OpenCL kernels are copied to the build directory
tasks.register('copyKernels', Copy) {
    from 'src/main/opencl'
    into 'build/exe/main/debug'
    include '*.cl'
}

./gradlew :Toolkit:demo:coolidge:myApp:emulate

FPGA Development Issues #

# Check if Vivado is in PATH
echo $XILINX_VIVADO

# Set up environment if needed
export XILINX_VIVADO=/opt/Xilinx/Vivado/2022.1

HardwarePlugin {
    // Ensure constraint files are correctly specified
    constraintFiles = ['constraints/timing.xdc']
}

./gradlew :Toolkit:demo:fpga:myApp:check

Advanced Debugging Techniques #

# Get a stack trace for errors
./gradlew build --stacktrace

# Get info-level logging
./gradlew build --info

# Get debug-level logging (very verbose)
./gradlew build --debug

# Generate a build scan for detailed analysis
./gradlew build --scan

# List all tasks in the project
./gradlew tasks

# Show task execution times
./gradlew build --profile

# Show task dependencies
./gradlew :Toolkit:demo:x86_64_CPU:matrixMultiply:taskName --dry-run

Working with Different Target Platforms #

The Brane SDK supports multiple target platforms, each with its own development workflow and considerations. Let’s explore the specific needs of each platform and how the build system supports them.

CPU Application Development #

CPU applications are the most straightforward, targeting standard x86_64 processors. These projects typically:

• Use C/C++ for implementation
• May leverage multi-threading, SIMD instructions, and optimized libraries
• Have simpler debugging and deployment workflows

A typical CPU application build.gradle looks like this:

plugins {
    id 'com.brane.cpu.cpp-application'
}

application {
    // Support both Linux and Windows x86_64
    targetMachines = [
        machines.linux.x86_64,
        machines.windows.x86_64
    ]
    
    // Configure C++ compiler settings
    binaries.configureEach { binary ->
        def compileTask = binary.compileTask.get()
        
        // Enable C++17
        compileTask.compilerArgs.add('-std=c++17')
        
        // Add optimization flags for release builds
        if (binary.optimized) {
            compileTask.compilerArgs.addAll(['-O3', '-march=native'])
            
            // Enable platform-specific optimizations
            if (binary.targetMachine.operatingSystemFamily.linux) {
                compileTask.compilerArgs.add('-flto')
            }
        }
    }
}

dependencies {
    // Common performance libraries
    implementation 'org.openblas:openblas:0.3.19'
    implementation 'org.fftw:fftw3:3.3.8'
}

The development workflow for CPU applications typically involves:

1. Building: ./gradlew :DevToolkit:demo:x86_64_CPU:myApp:build
2. Testing: ./gradlew :DevToolkit:demo:x86_64_CPU:myApp:test
3. Running: ./gradlew :DevToolkit:demo:x86_64_CPU:myApp:run
4. Debugging:
   • Configure your IDE debugger (IntelliJ IDEA, VS Code, etc.)
   • Set breakpoints and debug as normal

Best practices for CPU application development:

• Leverage platform-specific optimizations where appropriate
• Use conditional compilation for platform-specific code
• Consider portability across different operating systems
• Use performance profiling tools to identify bottlenecks

Kalray MPPA (Coolidge) Development #

Developing for the Kalray MPPA Coolidge processor introduces additional complexity due to its heterogeneous nature:

• Applications typically have a host component (running on CPU) and accelerator code (running on MPPA)
• The accelerator code is often written in OpenCL
• Development involves cross-compilation and specialized deployment

A typical Coolidge application build.gradle includes:

plugins {
    id 'com.brane.coolidge.cpp'
}

application {
    // Target both standard CPU and MPPA
    targetMachines = [
        machines.linux.x86_64,
        coolidgeMachines.clusterOS.architecture("MPPA_v2")
    ]
    
    // Configure binary-specific settings
    binaries.configureEach { binary ->
        // Add Coolidge-specific compiler flags
        compileTask.get().compilerArgs.add('-DKVX_COOLIDGE')
        
        // Link with required libraries
        linkTask.get().linkerArgs.addAll(['-ldl', '-llttng-ust'])
    }
}

// Ensure OpenCL kernels are copied to the build directory
tasks.register('copyKernels', Copy) {
    from 'src/main/opencl'
    into 'build/exe/main/debug'
    include '*.cl'
}

assemble.dependsOn('copyKernels')

The Coolidge development workflow typically involves:

1. Building: ./gradlew :DevToolKit:demo:coolidge:myApp:build
2. Emulation (faster but less accurate): ./gradlew :DevToolKit:demo:coolidge:myApp:emulate
3. Simulation (slower but more accurate): ./gradlew :DevToolKit:demo:coolidge:myApp:simulate
4. Hardware Deployment: # Over Linux network ./gradlew :DevToolKit:demo:coolidge:myApp:linuxRun # Via JTAG connection ./gradlew :DevToolKit:demo:coolidge:myApp:jtagRun

Key considerations for Coolidge development:

• Memory management between host and device
• Optimization of OpenCL kernel code for MPPA architecture
• Workgroup size tuning for maximum performance
• Careful management of cluster resources

FPGA Development #

FPGA development involves hardware design using HDLs (Hardware Description Languages) like Verilog or VHDL:

• Projects describe digital circuits rather than software algorithms
• Development involves synthesis, place-and-route, and bitstream generation
• Testing uses hardware simulation rather than direct execution

A typical FPGA application build.gradle looks like:

plugins {
    id 'com.brane.hdl'           // Core HDL plugin
    id 'com.brane.hdl.vivado'    // Xilinx Vivado integration
}

HardwarePlugin {
    top = "main_design"          // Top-level module name
    sim = "test_main_design"     // Test bench module name
    revision = "rev1"            // Hardware revision
    
    // Add timing constraints
    constraintFiles = ['constraints/timing.xdc']
    
    // Set synthesis strategy
    synthStrategy = "Performance_ExploreWithRemap"
}

sourceSets {
    main {
        verilog {
            srcDirs += ['src/main/ip_cores']  // Additional source directories
        }
    }
}

The FPGA development workflow typically involves:

1. Building Project Files: ./gradlew :DevToolkit:demo:fpga:myApp:generateProject
2. Simulation: ./gradlew :DevToolkit:demo:fpga:myApp:check
3. Synthesis: ./gradlew :DevToolkit:demo:fpga:myApp:compileProject
4. Implementation (Place-and-Route): ./gradlew :DevToolkit:demo:fpga:myApp:assemble
5. Bitstream Generation: ./gradlew :DevToolkit:demo:fpga:myApp:build

Key considerations for FPGA development:

• Timing constraints and meeting timing requirements
• Resource utilization optimization
• Test bench development for thorough verification
• IP core integration and management

Customizing the Build Process #

The Brane SDK build system is highly customizable. Here are some advanced customization techniques:

Custom Plugins #

You can create custom Gradle plugins to encapsulate common configurations:

// In buildSrc/src/main/groovy/com/yourcompany/CustomPlugin.groovy
package com.yourcompany

import org.gradle.api.Plugin
import org.gradle.api.Project

class CustomPlugin implements Plugin<Project> {
    void apply(Project project) {
        // Apply standard plugins
        project.apply plugin: 'com.brane.cpu.cpp-application'
        
        // Add common configuration
        project.application {
            targetMachines = [project.machines.linux.x86_64]
        }
        
        // Register common tasks
        project.tasks.register('standardBenchmark') {
            project.dependsOn('assemble')
            doLast {
                // Benchmark logic
            }
        }
    }
}

// In build.gradle
apply plugin: com.yourcompany.CustomPlugin

Build Script Delegation #

For complex projects, you can split build logic into multiple script files:

// In build.gradle
apply from: 'gradle/cpp-config.gradle'
apply from: 'gradle/testing-config.gradle'
apply from: 'gradle/benchmark-tasks.gradle'

// In gradle/cpp-config.gradle
tasks.withType(CppCompile) {
    // C++ configuration
}

// In gradle/testing-config.gradle
tasks.withType(Test) {
    // Testing configuration
}

Continuous Integration #

The Brane SDK build system integrates well with CI/CD pipelines. Here’s how to configure it for common CI systems:

Jenkins Pipeline #

// Jenkinsfile
pipeline {
    agent any
    
    stages {
        stage('Build') {
            steps {
                sh './gradlew build'
            }
        }
        
        stage('Test') {
            steps {
                sh './gradlew test'
            }
        }
        
        stage('CPU Demo Apps') {
            steps {
                sh './gradlew :DevToolkit:demo:x86_64_CPU:build'
            }
        }
        
        stage('Coolidge Simulation') {
            steps {
                sh './gradlew :DevToolkit:demo:coolidge:simulate'
            }
        }
        
        stage('FPGA Simulation') {
            steps {
                sh './gradlew :DevToolkit:demo:fpga:check'
            }
        }
    }
    
    post {
        always {
            junit 'build/**/test-results/**/*.xml'
        }
    }
}

GitHub Actions #

# .github/workflows/build.yml
name: Build and Test

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-latest
    
    steps:
    - uses: actions/checkout@v2
    
    - name: Set up JDK
      uses: actions/setup-java@v2
      with:
        java-version: '11'
        distribution: 'adopt'
    
    - name: Cache Gradle packages
      uses: actions/cache@v2
      with:
        path: |
          ~/.gradle/caches
          ~/.gradle/wrapper
        key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
        restore-keys: |
          ${{ runner.os }}-gradle-
    
    - name: Build
      run: ./gradlew build
    
    - name: Test CPU apps
      run: ./gradlew :DevToolkit:demo:x86_64_CPU:test

Artifact Publication #

For sharing libraries or tools built with the Brane SDK:

// In build.gradle
apply plugin: 'maven-publish'

publishing {
    publications {
        mavenJava(MavenPublication) {
            from components.java
            
            groupId = 'com.yourcompany'
            artifactId = 'awesome-library'
            version = '1.0.0'
        }
    }
    
    repositories {
        maven {
            name = 'CompanyRepository'
            url = "https://repo.yourcompany.com/maven-releases"
            credentials {
                username = findProperty('repoUser') ?: System.getenv('REPO_USER')
                password = findProperty('repoPassword') ?: System.getenv('REPO_PASSWORD')
            }
        }
    }
}