接上篇《授之以渔-运维平台发布模块三（Jenkins篇）》，今天介绍下针对Jenkins pipeline+saltstack的发布改造。
一、 Jenkins Pipeline的总体介绍
- Pipeline，简而言之，就是一套运行于Jenkins上的工作流框架，将原本独立运行于单个或者多个节点的任务连接起来，实现单个任务难以完成的复杂流程编排与可视化。
- Pipeline是Jenkins2.X的最核心的特性，帮助Jenkins实现从CI到CD与DevOps的转变
- Pipeline是一组插件，让Jenkins可以实现持续交付管道的落地和实施。
- 持续交付管道（CD Pipeline）是将软件从版本控制阶段到交付给用户或客户的完整过程的自动化表现。软件的每一次更改（提交到源代码管理系统）都要经过一个复杂的过程才能被发布。
- Pipeline提供了一组可扩展的工具，通过Pipeline Domain Specific Language（DSL）syntax可以达到Pipeline as Code（Jenkinsfile存储在项目的源代码库）的目的。
- Stage：阶段，一个Pipeline可以划分成若干个Stage，每个Stage代表一组操作，例如：“Build”，“Test”，“Deploy”。
二、 早期设计思路
在Jenkins早期版本中，还没有Saltstack插件，所以项目在Jenkins构建后，只能是通过Jenkins调用SSH或者SFTP将项目传到目标主机上。但我们公司由于有堡垒机的存在，SSH的唯一入口只能是堡垒机，传统的方法不可行。于是就有了《授之以渔-运维平台发布模块一（Jenkins篇）》https://www.jianshu.com/p/0e052e79e134。
上文中那个可以远程回调的Saltstack接口，小名叫Salt_jenkins_post君，Jenkins上的回调参数如下curl -d "job=JOB_NAME" http://xxxx/salt_jenkins_post。是的，没错，只需要回传项目名字即可，后续的一些针对这个项目的要做的一系列动作都存在一个叫做Release表中，他包含了如下几个字段：
class Release (models.Model):
class Meta:
verbose_name = '项目发布'
verbose_name_plural = verbose_name
ordering = ['-release_time']
release_name = models.CharField('项目名称',max_length=40)
release_time = models.DateTimeField('项目发布时间')
release_svn_address = models.CharField('项目SVN地址',max_length=170)
release_no = models.CharField('项目版本号',max_length=10,blank=True)
release_next_no = models.CharField('项目下次版本号',max_length=10,blank=True)
release_hosts = models.TextField('发布主机',blank=True,max_length=1000)
release_path = models.CharField('发布路径',max_length=50)
release_fail_hosts_count = models.PositiveIntegerField('成功主机数',blank=True)
release_success_hosts_count = models.PositiveIntegerField('失败数字数',blank=True)
release_release_dir = models.TextField('发布路径',blank=True,max_length=1000)
release_reboot_sync = models.CharField('同步重启',default='0',blank=True,max_length=5)
release_monitor_process = models.CharField('监测进程',blank=True,max_length=150)
release_monitor_dir = models.CharField('监测目录',blank=True,max_length=150)
release_purge = models.CharField('清除缓存',default='0',blank=True,max_length=5)
release_purge_name = models.CharField('清除缓存项目',blank=True,max_length=40)
release_purge_dir = models.CharField('清除缓存项目路径',default='',blank=True,max_length=40)
release_pipeline = models.CharField('项目流水线',default='',blank=True,max_length=40)
release_after_commands = models.TextField('项目前置命令',blank=True,max_length=1000)
release_before_commands = models.TextField('项目后置命令',blank=True,max_length=1000)
Salt_jenkins_post君的作用就是在接到Jenkins构建后（Batch tasks）的回调命令，会根据Jenkins发布目标的JOB_NAME找到发布主机release_hosts，最后通过在通过Saltstack调用目标发布主机执行pkg.upgrade_available、pkg.mod_repo、pkg.get_repo、pkg.install来安装RPM包，最后在根据配置信息决定是否调用release_before_commands执行一些后置命令，如重启，删除及release_purge域名刷新推送，如CDN。
优点：一站式服务，全含
缺点：无法拆分每一步，无法做到发布步骤的独立步骤的重放
三、 Pipeline带来的新的设计思路
利用Stage：阶段，一个Pipeline可以划分成若干个Stage，每个Stage代表一组操作，于是Release_pipeline_model君诞生了。
class Release_pipeline_model (models.Model):
class Meta:
verbose_name = '项目发布流水线模块'
verbose_name_plural = verbose_name
release_pipeline_model_name = models.CharField('流水线模块', max_length=40)
release_pipeline_model_id = models.CharField('流水线模块简称', max_length=40)
release_pipeline_responsive_name = models.CharField('流水线Responsive名称', max_length=40)
release_pipeline_model_edit_menu = models.CharField('流水线重放可视',default='0',max_length=40)
release_pipeline_model_template = models.TextField('流水线模块模板', blank=True, max_length=1000)
release_pipeline_model_change_at = models.DateTimeField('最后更改时间', auto_now=True)
def __unicode__(self):
return str(self.release_pipeline_model_name)
他的究极完成体如下：存放着名称，标识，可视化按钮及具体的Stage内容（里面的部分内容采用变量，根据不同项目生成不同的Stage信息，Pipeline语法详见https://jenkins.io/doc/book/pipeline/）
image.png
Salt_jenkins_post君也被拆分成了Salt_jenkins_after_commands君、Salt_jenkins_before_commands君、Salt_jenkins_getrepo君、Salt_jenkins_install君、Salt_jenkins_purge君、Salt_jenkins_reboote君、Salt_jenkins_upgradeavailable君。每人负责一部分，且在Release表的release_pipeline记录着他们的排序。
image.png
最后的系统会根据
release_pipeline的排序生成Jenkins调用的Pipeline script，效果如下：
pipeline{
agent any
stages {
stage('迁出代码') {
steps{
checkout([$class: 'SubversionSCM', additionalCredentials: [], excludedCommitMessages: '', excludedRegions: '', excludedRevprop: '', excludedUsers: '', filterChangelog: false, ignoreDirPropChanges: false, includedRegions: '', locations: [[cancelProcessOnExternalsFail: true, credentialsId: 'f999860f-8121-4367-913b-21cf61969200', depthOption: 'infinity', ignoreExternalsOption: true, local: '.', remote: "http://172.17.130.96/svndata/dz.m_youth/trunk"]], quietOperation: true, workspaceUpdater: [$class: 'UpdateUpdater']])
}
}
stage('创建目录') {
steps{
sh 'mkdir -p /home/release/$JOB_NAME'
}
}
stage('打FPM包') {
steps{
sh 'fpm -s dir -x .svn -t rpm -n $JOB_NAME -v $BUILD_NUMBER --prefix /home/dz -C /var/lib/jenkins/workspace/$JOB_NAME -p /home/release/$JOB_NAME ./'
}
}
stage('更新YUM源') {
steps{
sh 'createrepo --update /home/release/$JOB_NAME/'
}
}
stage('客户端验证YUM源是否存在') {
steps{
script {
try{
out = sh(script: 'curl -d "job_id=$JOB_NAME" http://veronica.youth.cn/cmdb/salt_jenkins_getrepo/', returnStdout: true)
if (out == '{"status":1}'){
echo 'salt_jenkins_getrepo ok'
}else{
sh 'exit 1'
}
}catch(Exception e){
error("salt_jenkins_getrepo not ok")
}
}
}
}
stage('客户端检测YUM源是否更新') {
steps{
script {
try{
out = sh(script: 'curl -d "job_id=$JOB_NAME" http://veronica.youth.cn/cmdb/salt_jenkins_upgradeavailable/', returnStdout: true)
if (out == '{"status":1}'){
echo 'salt_jenkins_upgradeavailable ok'
}else{
sh 'exit 1'
}
}catch(Exception e){
error("salt_jenkins_upgradeavailable not ok")
}
}
}
}
stage('客户端RPM包安装') {
steps{
script {
try{
out = sh(script: 'curl -d "job_id=$JOB_NAME" http://veronica.youth.cn/cmdb/salt_jenkins_install/', returnStdout: true)
if (out == '{"status":1}'){
echo 'salt_jenkins_install ok'
}else{
sh 'exit 1'
}
}catch(Exception e){
error("salt_jenkins_install not ok")
}
}
}
}
stage('客户端后置命令') {
steps{
sh 'curl -d "job_id=$JOB_NAME" http://veronica.youth.cn/cmdb/salt_jenkins_before_commands/'
}
}
}
}
前台的展示是这样的：用的jquery.nestable.js
image.png
部分JS也双手奉上：
jQuery(function($){
$('.dd').nestable({
maxDepth: 1,
});
var aa=$('#nestable_list_1').nestable('serialize')
$('#release_pipeline').val(JSON.stringify(aa))
$('.dd').nestable({'group':1}).on('change', function() {
var r=$('#nestable_list_1').nestable('serialize')
console.log(JSON.stringify(r))
$('#release_pipeline').val(JSON.stringify(r))
}
);
});
function reboot_sync_hidden(obj){
if($("#release_reboot_sync_type").val()=="0"){
document.getElementById("monitor_process").style.display ="none";
document.getElementById("monitor_dir").style.display ="none";
}
else {
document.getElementById("monitor_process").style.display ="block";
document.getElementById("monitor_dir").style.display ="block";
}
}
//获取URL参数，?id=xx
function getQueryString(name) {
var reg = new RegExp('(^|&)' + name + '=([^&]*)(&|$)', 'i');
var r = window.location.search.substr(1).match(reg);
if (r != null) {
return unescape(r[2]);
}
return null;
}
function release_pipeline_edit(){
csrftokens = $.cookie('csrftoken')
$('#responsive_pipeline_ediit form').submit(function() {
$.ajax({
type: "POST",
url: "../release_pipeline_edit/",
data: $('#responsive_pipeline_ediit form').serialize(),
headers: {"X-CSRFtoken": csrftokens},
async: true,
cache: false,
dataType: "json",
beforeSend: function () {
Metronic.blockUI({animate: true});
},
complete: function () {
Metronic.unblockUI();
},
success: function (obj) {
if (obj['status'] == 999) {
alert(obj['err'])
$('#responsive_pipeline_ediit').modal('hide');
$('#responsive_pipeline_ediit form')[0].reset();
} else {
alert("流水线修改完成")
window.location.href = "../release_list";
}
}
});
return false;
})
}
$(document).ready(function(){
release_pipeline_edit();
});
function purge_hidden(obj){
if($("#release_purge_type").val()=="0"){
document.getElementById("purge_name").style.display ="none";
document.getElementById("purge_dir").style.display ="none";
}
else {
document.getElementById("purge_name").style.display ="block";
document.getElementById("purge_dir").style.display ="block";
}
}
$("#reboot").bind("click", release_pipeline_reboot_edit);
function release_pipeline_reboot_edit(){
var id = getQueryString("id");
console.log(id)
$.ajax({
type: "GET",
url: "../release_pipeline_reboot_edit_ajax/?id=" + id,
async: true,
cache: false,
dataType: "json",
beforeSend: function () {
Metronic.blockUI({animate: true});
},
complete: function () {
Metronic.unblockUI();
},
success: function (obj) {
if (obj['status']==999){
alert(obj['err'])
$('#responsive_reboot').modal('hide');
$('#responsive_reboot form')[0].reset();
}else{
var monitor_process = obj['monitor_process']
var monitor_dir = obj['monitor_dir']
var reboot_sync = obj['reboot_sync']
if (reboot_sync == "1") {
$("#release_reboot_sync_type").val(["1"]).trigger('change');
document.getElementById("monitor_process").style.display = "block";
document.getElementById("monitor_dir").style.display = "block";
$("#release_monitor_process").val(monitor_process)
$("#release_monitor_dir").val(monitor_dir)
} else if (reboot_sync == "2") {
$("#release_reboot_sync_type").val(["2"]).trigger('change');
document.getElementById("monitor_process").style.display = "block";
document.getElementById("monitor_dir").style.display = "block";
$("#release_monitor_process").val(monitor_process)
$("#release_monitor_dir").val(monitor_dir)
}
$("#job_id").val(id)
}}
})
}
$('#responsive_reboot form').submit(function(){
var id = $('#job_id').val();
var release_reboot_sync_type = $("#release_reboot_sync_type").val();
var release_monitor_process = $("#release_monitor_process").val();
var release_monitor_dir = $("#release_monitor_dir").val();
var jsonData = {
"id":id,
"release_reboot_sync":release_reboot_sync_type,
"release_monitor_process":release_monitor_process,
"release_monitor_dir":release_monitor_dir,
}
csrftokens = $.cookie('csrftoken')
$.ajax({
type: "POST",
data: jsonData,
url: "../release_pipeline_reboot_edit_ajax/",
headers:{ "X-CSRFtoken":csrftokens},
async:true,
cache: false,
dataType: "json",
beforeSend:function(){
Metronic.blockUI({animate: true});
},
complete: function() {
Metronic.unblockUI();
},
success: function(obj) {
if (obj['status']==999){
alert(obj['err'])
$('#responsive_reboot').modal('hide');
$('#responsive_reboot form')[0].reset();
}else if (obj['status'] == "1"){
$('#responsive_reboot').modal('hide');
$('#responsive_reboot form')[0].reset();
alert ("修改完成");
} else {
alert ("修改失败");
}
},
});
return false;
});
$('#responsive_reboot').on('hide.bs.modal', function () {
location.reload();
});
$("#purge").bind("click", release_purge_edit);
function release_purge_edit(){
var id = getQueryString("id");
console.log(id)
$.ajax({
type: "GET",
url: "../release_pipeline_purge_edit_ajax/?id=" + id,
async: true,
cache: false,
dataType: "json",
beforeSend: function () {
Metronic.blockUI({animate: true});
},
complete: function () {
Metronic.unblockUI();
},
success: function (obj) {
if (obj['status']==999){
alert(obj['err'])
$('#responsive_purge').modal('hide');
$('#responsive_purge form')[0].reset();
}else{
var purge_name = obj['purge_name']
var purge_dir = obj['purge_dir']
var purge = obj['purge']
if (purge == "1") {
$("#release_purge_type").val(["1"]).trigger('change');
document.getElementById("purge_name").style.display = "block";
document.getElementById("purge_dir").style.display = "block";
$("#release_purge_name").val(purge_name)
$("#release_purge_dir").val(purge_dir)
}
$("#job_id").val(id)
}}
})
}
$('#responsive_purge form').submit(function(){
var id = $('#job_id').val();
var release_purge_type = $("#release_purge_type").val();
var release_purge_name = $("#release_purge_name").val();
var release_purge_dir = $("#release_purge_dir").val();
var jsonData = {
"release_purge":release_purge_type,
"release_purge_name":release_purge_name,
"release_purge_dir":release_purge_dir,
"id":id
}
csrftokens = $.cookie('csrftoken')
$.ajax({
type: "POST",
data: jsonData,
url: "../release_pipeline_purge_edit_ajax/",
headers:{ "X-CSRFtoken":csrftokens},
async:true,
cache: false,
dataType: "json",
beforeSend:function(){
Metronic.blockUI({animate: true});
},
complete: function() {
Metronic.unblockUI();
},
success: function(obj) {
if (obj['status']==999){
alert(obj['err'])
$('#responsive_purge').modal('hide');
$('#responsive_purge form')[0].reset();
}else if (obj['status'] == "1"){
$('#responsive_purge').modal('hide');
$('#responsive_purge form')[0].reset();
alert ("修改完成");
} else {
alert ("修改失败");
}
},
});
return false;
});
$('#responsive_purge').on('hide.bs.modal', function () {
location.reload();
});
