Using build scan tags for ad-hoc root cause analysis

hawksbill

Recently I was helping an Android team investigate a hard-to-reproduce issue that manifested as very-long-running compilation and annotation processing tasks. Most of their builds took only a few minutes to run, but sometimes they took up to 30 minutes! In these cases, invariably, a build scan showed that some combination of Java compilation, Kotlin compilation, or annotation processing with Kapt, was the culprit. The team had no real idea of how often this problem occurred for developers, and under what conditions.

Gradle Enterprise does not, yet, provide a way to find builds based on how long a particular task took. However, custom tags make it easy to categorize and find builds for any condition that can be detected within a build. By tagging all of the too long builds we could use Gradle Enterprise to understand the impact and severity, and the context in which they occur, which accelerated the debugging.

To achieve this, we simply added something like the following snippet to the build.

// root build.gradle
subprojects {
    tasks.matching { t ->
        t instanceof org.jetbrains.kotlin.gradle.tasks.KotlinCompile
        || t instanceof org.jetbrains.kotlin.gradle.internal.KaptWithKotlincTask
        || t instanceof com.android.build.gradle.tasks.factory.AndroidJavaCompile
// || ... more ...
    }.configureEach {
        long start
        doFirst {
            start = System.currentTimeMillis()
        }
        doLast {
            long duration = System.currentTimeMillis() - start
            if (duration > 15 * 60 * 1000) { // 15 min
                buildScan.tag "TooLong"
                buildScan.value "TooLong", path
            }
        }
    }
}

This code block will tag a build with “TooLong” if any of the tasks-of-interest took longer than 15 minutes to run. Furthermore, it will add a custom value to the build with the task path as the value, so we will know exactly which task(s) are the culprit for any given build. The tag or custom value can then be used as search criteria in Gradle Enterprise to find all builds that exhibited the problem.

Scans List

This is but one example of tagging a build based on what happened. The API for tagging builds combined with Gradle’s rich and extensive API for interrogating the build open up many possibilities.

Using build scan tags for ad-hoc root cause analysis

Further reading