Overview
The original CI system of the XXX project was set up by the project itself and used Jenkins' traditional Job approach to implement VerifyCI, MergeCI, and DailyBuild. With the increasing scale of the project, more and more branches, and the increasing frequency of the code, the existing system presents many inconveniences. To solve these problems, the project tried to introduce a series of ready-to-use Devops R&D tools and incorporate the new Pipeline features of Jenkins 2.0 to transform the Pipeline process of this project.
Pipeline as Code is the essence of the Jenkins 2.0 release and is a key tool to help Jenkins achieve a gorgeous turn from CI to CD. The so-called Pipeline, in short, is a workflow framework that runs on Jenkins. It connects tasks that were originally run on a single node or multiple nodes to achieve a complex publishing process that is difficult for a single task to complete. At the code level, we only need to concentrate on writing the Jenkinsfile file and then host it along with the code base. Jenkins can quickly pull up the CI process of the project according to Jenkinsfile, which is convenient and efficient.
However, with the trial and landing of multiple projects, it was found that the Pipeline based on Jenkins 2.0 is powerful, but there are also the following pain points for specific applications:
Various projects are writing their own special Jenkinsfile. Many times, they copy and modify each other's code. Jenkinsfile has a very high degree of redundancy.
According to the Jenkins 1.x era, customize the CI/CD process, email notifications, and report output. The conversion process is slow and painful.
There is no ability to solve technical problems and adopt some outdated methods and tools. Other project teams have already solved some technical problems or adopted more advanced methods and tools. They cannot share each other in a timely manner.
In order to solve these problems, after repeated investigation and exploration of the project, the Pipeline co-creation library iPipeline (also known as the “plll libraryâ€) of Jenkins 2.0, which was open sourced by Zhongkai News, was used to assist the CI reconstruction process.
iPipeline is a tool set that simplifies the CI Pipeline deployment. It is a function library for developers and CI configuration administrators. It encapsulates Jenkins 2.0 common functions, integrates Gerrit, product libraries, cloud CI, metrics, alarm collection, and email notifications. Also provided is a docker packaged toolset (complexity, Klocwork, metrics analysis, metrics import, etc.). Using it can help us save a lot of energy and avoid duplication of the wheel, so it is very much in line with the needs of our project.
Problem Description
The iPipeline framework has indeed improved a lot of efficiency and well-being in the practical process of the project. However, in light of the actual use of the project, it has found that there are still some points for optimization and improvement. To meet this need for our project, we have made some optimizations and practices for iPipeline as follows.
Question 1
Using the pdocker interface provided by the plll library, it is convenient to run the specified Docker container on the specified node to complete the related CI tasks. For example, this item circle complexity check has been Dockerized, so you can use the pdocker interface to perform loop complexity checking by configuring mirror names, mapping directories, and commands and scripts that need to be run.
However, during the use of the pdocker interface, it was found that the plll library always removes the image from the node and then runs again. This is actually not necessary for the image that already exists on the local node. Therefore, it needs to be modified.
Question 2
The Update interface provided by the current Plll library only supports code update detection of the Gerrit code repository. However, the packets required for running certain use case tests in this project are stored in the SVN library. Therefore, it is necessary to extend the original interface of the Plll library to support SVN. Library code update
With these two problems, we made some corresponding optimizations and practiced the Plll library.
Optimization practice
Optimization 1: Whether the pdocker interface needs to pull the image customized by the user
Modify the pdocker interface, add parameters to control whether the pull image is needed, the interface code fragment is as follows:
/**
* Tool name: docker execution
* Tool Description:
* image - full path to the image
* needPull - whether to need pull image
* before_cmd - The command line executed as root, executed before executing user_cmd
* volumes - list of path mappings, "a:b,c:d"
* params - execution parameters
* profile - environment file, executable
* user_cmd - Command line executed by the current user (requires mirror support for adduser command)
* after_cmd - Command line executed as root, executed after executing user_cmd
* local_dir - the directory where temporary files are stored (requires permission to map to the container)
* shared_dir - the directory where the code and output files are stored (requires permissions mapped to containers)
* sh_exec - shell process, the default is sh
**/
Def call(image, needPull, before_cmd, volumes='',params='', profile='', user_cmd='', after_cmd='',
Local_dir=env.LOCAL, shared_dir=env.SHARED, sh_exec='sh'){
// ...where the code is omitted...
If( needPull =="no"){
Echo "no need to pull image !!! The image exists in local!"
writeFile file:docker_entry_file, text:"""
Set -e
Docker run --rm ${params} ${volumes} ${image} ${sh_exec} -x -c ${docker_run_file}
"""
}elseif( needPull =="yes"){
writeFile file:docker_entry_file, text:"""
Set -e
Docker pull ${image}
Docker run --rm ${params} ${volumes} ${image} ${sh_exec} -x -c ${docker_run_file}
"""
}
// ...where the code is omitted...
}
It can be seen from the code that by adding the needPull parameter, the user can customize whether the pull image is needed.
Use example:
For example, this project calls the pdocker interface to use the homemade docker image to complete the code circle complexity check:
Pnode("${env.NODE_NAME}"){
plll.Check("CCN_DOCKER","CCN_DOCKER",[
Run_execute:{ pdocker (
/* image */"docker.zte.com.cn:5000/10010891/lizard:v1",
/* need pull */"no",
/* cmd */"cd /home/code/ && chmod -R 777 * && cd script/VerifyCI/CCNCheck/ && sh +x lizard.sh",
/* volumes */"-v ${env.SHARESPACE}/${env.XXXXX_DIR}:/home",
/* params */"--privileged",
)
},
Param:[
Report_file:[]
]
]);
}
Since docker.zte.com.cn: 5000/10010891/lizard:v1 already exists on our external node, the needPull parameter is set to no here to control pdocker's internal pull image.
Optimization 2: Extension framework Update interface to support SVN code update
Optimize Plll library code, add UpdateSVN interface to support SVN library code checkout and update
/**
* Function name: Update
* Function description: Update code through SVN
**/
defUpdateSVN(name, desc, args){
LogDebug("[DEBUG] Update: ${name}, ${desc}")
Args.run_execute ={
Dir("${args.scm.path}"){
Svn_checkout( args.scm.keyid, args.scm.repo, args.scm.path )
}
}
/* Call function adaptation */
FunctionAdapter("Update","update", name, desc, args){}
Return
}
/**
* Tool Name: svn_checkout
* Tool Description: svn update code
* Parameter Description:
* - keyid: Credentials ID of the SVN library
* - repo: SVN library Repo path
* - path: code download path
**/
Def svn_checkout(keyid, repo, path){
Checkout([
$class:'SubversionSCM',
additionalCredentials:[],
Excluded CommitMessages:'',
excludedRegions:'',
excludedRevprop:'',
excludedUsers:'',
filterChangelog:false,
ignoreDirPropChanges:false,
includedRegions:'',
Locations:[
[credentialsId:"${keyid}", depthOption:'infinity', ignoreExternalsOption:true,local:".", remote:"${repo}"]
],
workspaceUpdater:[$class:'UpdateUpdater']
])
Return
}
Use example:
Users only need to configure:
Credentials ID of SVN Library
SVN library Repo path
Code download path
The SVN code checkout and update can be completed. An example is as follows:
// First configure the SVN related parameters
env.SVN_KEY_ID="89a6fe98-8f0c-4fe6-829e-6d1cbda188e1"
env.CASE_DIR ="/jenkins_ci/CASE_TEST/PATH"
env.CASE_SVN_URL="svn://XXX.XX.XXX.XXX/XXXXXXX/case/XXXX"
// Call the UpdateSVN interface to complete the SVN library code update
plll.UpdateSVN('SVN_UP', 'Updating Messages on SVN', [
Scm:[keyid:"${env.SVN_KEY_ID}", repo:"${env.CASE_SVN_URL}", path:"env.CASE_DIR"],
Run_dir:"${->OUTPUT_PATH}"
])
Promotion suggestion
This article related optimization improvements can be extended to the need to use docker image to complete the relevant CI tasks and code control projects involving the SVN library
Travel Charger Adapter is convenience for these people who always travel in many countries. Desktop Power Adapter have normal DC connector for your need, and wall power adapter have mutil plug, like US/UK/AU/EU etc. We also can produce the item according to your specific requirement. The material of this product is PC+ABS. All condition of our product is 100% brand new.
Our products built with input/output overvoltage protection, input/output overcurrent protection, over temperature protection, over power protection and short circuit protection. You can send more details of this product, so that we can offer best service to you!
Travel Charger Adapter,Portable Travel Charger Adapter,Mini Travel Charger Adapter,Travel Charger Supply
Shenzhen Waweis Technology Co., Ltd. , https://www.laptopsasdapter.com