CloudEvents Plugin for Jenkins: Interoperability between Jenkins and other CI/CD Tools

[amurleopard]

The What, Why and How of Interoperability

With workloads and teams becoming more diverse and complex, there is an increasing need to automate various tasks in the CI/CD ecosystem of an application as a way to decrease complexity that can come with CI/CD.

A more diverse team working across different aspects of the application requires a diverse suite of CI/CD tools too, to test and deliver to a wide range of users. More often than not, we need these tools to work together and exchange data to form an effective CI/CD pipeline. However, chaining multiple services together can very easily increase complexity.

How? Each of these services use a different "language" to communicate and represent the entity(an event) which occured inside that service. In order for another service to understand this "language", the service might need to develop customized clients and agents which specialize in understanding, traversing and taking-actions based on what was transmitted to it by the first service.

One can think of it as a translator who specializes in a language called ABC, and each service who wants to communicate with the service who uses ABC will have to employ this translator, or perhaps get another trained translator. And there is no guarantee that this translator will also help communicate with other services speaking a completely different language.

We can see how easily that can grow in cost and maintenance. A preferred way is to have a common language each of these services use and understand as a way to communicate amongst each other. This way, an event which is emitted using this common language will be available to any of the interested receiver without that receiver needing a special agent. This way of communication which uses a common/standard language also creates a way for agnostic communication where the sender or the receiver are sending and receiving data without creating a tight coupling between the two.

CloudEvents specification is enabling that loosely-coupled, event-driven communication between services by enforcing a common language which defines how an event should be emitted and transferred between systems.

CloudEvents and Jenkins

A specification for describing event data in a common way

• Consistency

  • Consistent across tools and services.

• Accessibility

  • Common event format means common libraries, tooling, and infrastructure for delivering event data across environments can be used to develop with CloudEvents.

• Portability

  • Easily port event-data across tools, truly leveraging event-driven architecture.

The CloudEvents plugin for Jenkins is developed as an effort to make interoperability between Jenkins and CI/CD tools much easier. The CloudEvents plugin for Jenkins is a GSoC project, and with the help from an amazing team of mentors, this project is aimed at enhancing event-driven interoperability between cloud-native CI/CD tools, making it easier for developers to include Jenkins in their CI/CD pipelines.

With this plugin, Jenkins can send and receive CloudEvents-compliant events to and from a wide variety of CI/CD tools using CloudEvents as their event format. This plugin makes chaining Jenkins with multiple tools like Tekton, Keptn, Knative and more, very easy.

GSoC Phase 1 - CloudEvents Plugin

Using CloudEvents plugin for Jenkins

This plugin allows Jenkins to be configured as a source and sink, which can emit and consume CloudEvents from a range of tools simultaneously.

Jenkins as a Source

Configuring Jenkins as a Source enables Jenkins to send CloudEvents to a CloudEvents sink. For Phase-I of this project, there is support for HTTP Sinks, however CloudEvents supports various protocol bindings. Moving forward, there will also be support for other protocol bindings supported by CloudEvents.

To use Jenkins as a Source, the following configuration is needed:

1. Click on Manage Jenkins in the Root-Actions menu on the left.

2. Inside the Manage Jenkins UI, search for Configure System under System Configuration.

3. In the Configure System UI, scroll down to the CloudEvents plugin section, and this is where all the plugin configuration will be present. Here, you will have to enter the following information:

   • Sink Type (For now, HTTP Protocol Binding for CloudEvent and HTTP Sink is supported.)

   • Sink URL (URL of the Sink where you want the cloudevents sent.)

   • Events you want sent to the CloudEvents sink URL.

Step 1: Manage Jenkins

Step 2: Configure System

Step 3: Configure CloudEvents Sink

With Jenkins as a Source configured, Jenkins will send a POST request to the configured sink right as the selected event occurs inside Jenkins. Each event has a different payload specific to the type of the event emitted.

Event Types, Payload and Metadata

CloudEvents emitted by Jenkins follow the Binary-structure supported by CloudEvents, where the CloudEvents metadata is present inside the header, and the event-data is serialized as JSON, and present under request-body. This is the HTTP Protocol Binding for CloudEvents. Each protocol binding for CloudEvents follows a definition specific to the binding protocol.

For now, the following Jenkins events are supported in the CloudEvents Plugin-Jenkins as a Source:

• Queue Events

  • Queue Entered Waiting

  • Queue Left

• Build Events

  • Job Started

  • Job Completed

  • Job Finalized

  • Job Failed

• Job Events

  • Job Created

  • Job Updated

• Node Events

  • Node Online

  • Node Offline

Following is a table of the queue-entered waiting cloudevents metadata:

All of these fields will be present inside the HTTP-request headers since the CloudEvents format used here is the Binary structure.

Here’s also an example of event payload for the queue-entered event:

{
  "ciUrl": "http://3.101.116.80/",
  "displayName": "test2",
  "entryTime": 1626611053609,
  "exitTime": null,
  "startedBy": "shruti chaturvedi",
  "jenkinsQueueId": 25,
  "status": "ENTERED_WAITING",
  "duration": 0,
  "queueCauses": [
    {
    "reasonForWaiting": "In the quiet period. Expires in 0 ms",
    "type": "entered_waiting"
    }
  ]
}

Event Metadata Headers Key	Event Metadata Headers Value
ce-specversion	1.0
ce-type	org.jenkinsci.queue.entered_waiting
ce-source	job/test
ce-id	123-456-789

Try the Plugin

The plugin will soon be releasing as the CloudEvents Plugin under https://plugins.jenkins.io/!!

Here’s the GitHub Repo of the Plugin: CloudEvents Plugin GitHub Repo

Demo

Here is a video of the CloudEvents plugin with SockEye demoed at CDF GSoC Midterm Demos. SockEye is an open-source tool which is designed as a way to visulaize cloudevents which are sent from a sink. In this demo, we will take a look at how Jenkins installed in a multi-node K8s environment work with the CloudEvents plugin as a Source, sending events over HTTP to the SockEye sink.

Next Steps

• Jenkins as a Sink to allow Jenkins to trigger various actions as cloudevents are received from other tools.

• Enabling filtering on CloudEvents metadata to only act upon a certain kind of events recieved.

• Support for other protocol bindings in CloudEvents.

Feedback

We would absolutely love to hear your suggestions and feedback. This will help us understand the various use-cases for the plugin, and iterate to support a variety of bindings and formats.

Feel free to log an issue at the CloudEvents Plugin GitHub repository. We are on CDF slack under gsoc-2021-jenkins-cloudevents-plugin. You can also start a discussion on community.jenkins.io. I also love emails! Drop me one on: [email protected]

http://feedproxy.google.com/~r/ContinuousBlog/~3/EYrvn9vOtvI/