Machine Learning Plugin project - Coding Phase 3 blog post

amurleopard

Good to see you all again !

This is my final blog post about coding phase 3 in Jenkins Machine Learning Plugin for GSoC 2020. Being at the end of GSoC 2020, we had to finish all the pending issues and testing before a stable release in the main repository. Throughout this program, there were lots of learning and hard work will make this plugin valuable to the Data Science and Jenkins community.

Summary

With combining all of the work in phase 1, 2 and 3, initial version of Machine learning plugin( 1.0 ) was successfully released in Jenkins plugin repository. An interesting feature which allows users to connect to their existing programming language kernels more than connecting to only IPython kernel was introduced in this phase. It can be selected in multiple steps with different kernel. Images and graphs produced by Jupyter notebooks will be saved in user preferred folder in the workspace that can be used for reporting/analytic purposes later. Hoping this blog summarizes the Machine Learning’s features and future contributions. Thank you for your interest and support !!!

Main features of Machine Learning plugin

Execute Jupyter notebooks directly
Run different language scripts using multiple build steps
Convert Jupyter Notebooks to Python
Configure Jupyter kernels( IPython, IRKernel, IJulia etc) properties
Support to execute Notebooks/scripts on Agent
Extract graph/map/images from the code
Each build step can be associated with a machine learning task
Support for Windows and Linux

Future improvements

Improving performance of the plugin

Try to implement JENKINS-63377
Support parameterized definitions in Notebooks JENKINS-63478
Increasing testing code coverage

Multiple language kernel support

If there are existing kernels in the system, user will be able to configure in the global configurations in order to apply in the builder/step configuration.

Some popular interactive kernels

IPython for python
IRKernel for R
IJulia for Julia
IJavascript for javascript

More kernels and installation guides are found here. https://github.com/jupyter/jupyter/wiki/Jupyter-kernels

Dump images and graphs

Text output will be displayed in the console log. At the same time images/graphs/heat maps and HTMLs will be saved in the workspace. An action is shown in the left panel to display images in realtime. Due to the Content Security Policy of jenkins, some HTMLs which contain harmful javascript may not render in jenkins UI.