This plugin allows Jenkins agents to be dynamically provisioned on a Kubernetes cluster. Plugin Information View Kubernetes on the plugin site for more information. Older versions of this plugin may not be safe to use. Please review the following warnings before using an older version:
See GitHub for all information.
Due to some maintenance issues, this service has been switched in read-only mode, you can find more information about the why
and how to migrate your plugin documentation in this blogpost
Overview
Content Tools
45 Comments
Unknown User (wstrange)
One the things that I am confused about (and I suspect others might be as wel)
What are the requirements for the Jenkins Master to manage Kubernetes. I assume it needs the kubectl command installed?
If Jenkins is itself running on K8S, is there a Jenkins image that has the requirements?
Unknown User (csanchez)
no need for kubectl, the
jenkins/jenkinsimage will doUnknown User (centerfire)
Having followed the setup instructions precisely, Kubernetes is spinning up slave containers that die with "connect timed out" errors when they try to JNLP connect back to my Jenkins master. Is there some additional network configuration necessary to get the slave containers to talk to the master?
Unknown User (samof76)
Brett,
I presume you would have JNLP port disabled. You have go to "Manage Jenkins > Configure Global Security", and enable either specify a "Fixed" port or a "Random" port(in case it is "Disabled").
Unknown User (snallami)
Can different Jenkins masters use the same Kubernetes cluster?
Unknown User (willmore)
Yes. Kubernetes is inherently multi-tenant.
To facilitate ease of management, consider running your various Jenkins masters (and their associated spawned agent pods) in dedicated namespaces within the shared cluster.
Unknown User (jenkinsuser20171128)
What's the best practice for how one would build docker images from within these jenkins slaves?
Unknown User (willmore)
Depends on what you are looking for regarding "best", but "Docker outside of Docker" pattern will work:
https://github.com/jenkinsci/kubernetes-plugin/blob/master/examples/docker.groovy
Unknown User (robin_smith)
I'm not sure I follow the nesting pod templates section of the user guide. What docker image is actually used when you nest the pod templates? In the examples, each pod template specifies a docker image to use (one pod uses a maven-enabled image, the other uses a docker-enabled image) - in what container are the ```sh``` steps executed, and how are the dual requirements (ability to execute docker and maven commands) met in a single container?
Unknown User (krish123)
Hai jenkinsci/jnlp-slave
Unknown User (loxley)
Then you will need to build the JNLP agent into your docker image.
Unknown User (krish123)
i builded my image taking base image as
in my dockerfile From jenkinsci/jnlp-slave
Unknown User (jandroav)
Hi. The config option "Max number of instances" is not working as I expect. I have two pod templates, one is configured with 2 max instances and the other one with 6 max instances. The "Container Cap" is set to 8 but for example the podtemplate configured with 2 max instances is creating 3 or 4 instances. Am i doing something wrong?
Unknown User (hathaways)
I'm seeing same issue. I set this number to 10 but actually 20+ containers are launched. This number has worked before, is this a regression?
Unknown User (loxley)
I found one thing that is really bad, that when a pod is created it sets these labels on all slaves that are created:
Labels: jenkins=slave
jenkins/autoscale=true
What I would like is that it sets it to a unique value as it does when deploying something normally on kubernetes:
Labels: pod-template-hash=380424406
run=net-tools
Not having a unique label will cause problems if you need to expose services, f.ex if you have function tests and hardware running outside the cluster. If multiple pods have SAME labels and you run the kubectl expose command it will use the labels as selectors resulting in multiple endpoints which is not what you want.
It would be nice to have something like this instead:
Labels: jenkins-slave-hash=<unique/random hash>
jenkins=slave
Unknown User (luckyhorang)
Hello.
I am k8s newbie. Is there an example use case? Or can you guide how to use this ?
I am setting kubenetes cluster (3 servers for example) using kubeadm.
But I have no idea for next steps...
In my case, I have one dedicated jenkins master server, I want to use kubernetes for agents. But I don't know how to do.
Thank you.
Unknown User (rkamath3)
Perhaps, you can look here. https://github.com/JeanMertz/kubernetes-plugin/tree/5f983909b1a0893b5817648eda5f68405d7aa0a0
Unknown User (rkamath3)
Hello,
When I execute the pipeline which is provided at GitHub Repo, it always fails to run.
It throws the following exception, how to resolve this? Any help would be highly appreciated.
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed: WorkflowScript: 3: Expected to find someKey "someValue" @ line 3, column 16. kubernetes { ^ WorkflowScript: 3: Missing required parameter for agent type "kubernetes": containerTemplate @ line 3, column 5. kubernetes { ^ WorkflowScript: 6: Invalid config option "yaml" for agent type "kubernetes". Valid config options are [label, containerTemplate, activeDeadlineSeconds, cloud, inheritFrom, instanceCap, nodeSelector, serviceAccount, workingDir] @ line 6, column 7. yaml """ ^ 3 errors at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310) at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085) at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603) at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581) at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558) at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298) at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268) at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688) at groovy.lang.GroovyShell.parse(GroovyShell.java:700) at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:133) at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:559) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:520) at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:323) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Finished: FAILUREUnknown User (rkamath3)
sorry, the link was wrong. I am trying to execute the Declarative pipeline with container template defined as a yaml.
Unknown User (csanchez)
You need the latest 1.6.0 released yesterday.
Questions are better addressed in jenkins-user mailing list
Unknown User (shen3lu4)
When I used kubernetes plugin 1.1.2, everything was working fine. Just upgrade the plugin to 1.6.0 and now it is having issues when creating kubernetes credentials using "OpenShift OAuth token" kind in Credentials. The only field visible to me is Scope. Nothing after that.
I am expecting to see 3 more fields: Token, ID, Description. Thanks!
Unknown User (seckler)
Hi,
I stumbled across this error, too.
Fixed it by noticing that using the "OpenShift OAuth token" credentials is deprecated.
You are supposed to use the "StringCredentials", i.e. plain text instead.
Unknown User (stevejr)
After upgrading from v1.3.2 to 1.6 I am getting a permission error (AccessDenied) when trying to access an EmptyDir volume I have created for the pod template created for my job. Is the usage of a ServiceAccount in the pod now enforced in v1.6 as at present I am not using that.
I have jenkins deployed in the cluster using the Jenkins Helm chart and the master pod does use a Service Account.
Unknown User (jchris)
How do I configure the Kube/Jenkins plugin to use the specific Service Account that's been setup for access?
> kubectl describe serviceaccounts/cd-jenkins
Name: cd-jenkins
Namespace: default
Labels: app=cd-jenkins
chart=jenkins-0.16.1
heritage=Tiller
release=cd
Annotations: <none>
Image pull secrets: <none>
Mountable secrets: cd-jenkins-token-wc2sj
Tokens: cd-jenkins-token-wc2sj
Events: <none>
# from inside the jenkins slave pod....
root@default-dzspv:~# kubectl auth can-i get pods
no - Unknown user "system:serviceaccount:default:default"
Unknown User (hooligan495)
hello,
I'm trying to understand the approach/mindset I need to take with Jenkins and the Kubernetes plugin when it comes to getting my maven settings.xml file (I've created a global one with the Config File plugin) that has my Artifactory username and password available to the agent that Kubernetes spins up.
I'm currently using a pipeline (jenkins file) job.
But also need to support non pipeline jobs.
My job is currently failing to resolve my maven dependencies due to authentication issues.
Thanks
Jay
Unknown User (rupali_a_chorghe)
Does usage of this plugin requires use of privileged mode? Also can we specify user to be used while starting up slaves?
Unknown User (oxygen)
Hi,
I want to suggest you to add a function to retain the slave after a job, thus they can be used in next job without creation again.
we can add a checkbox to indicate if user want to keep the slave after a job, also in case of a slave is offline, the slave can try to reconnect to the master periodically until it is connected.
Thanks.
Unknown User (csanchez)
that already exists, slave retention timeout
Unknown User (oxygen)
Thanks,
Unknown User (darenjabos)
How can I use a specific container for Execute Shell? I specify a label jenkins-node which is the label that I use for Kubernetes container in the plugin, but it'snot using that container, it's using busybox.
Details:
Kubernetes 1.9.6, kubernetes plugin: 1.10.1, Jenkins 2.121.1
I set up a container Template in K8s plugin. Name: jenkins-node, Docker image: darenjacobs/custom-slave:3.20.1
When I create a pipeline job, everything works fine. It says Running on "Running on jenkins-node-xxxxx" and I have all the applications I have installed in that image, specifically packer, ansible, maven, etc. I am able to get to the container I want by specifying the label that correlates to the image. Again this works just fine
When I run a Freestyle job, it will create a container "jenkins-node-8hql3",
I have the job configured to Restrict it to Label Expression: "jenkins node", which again has the Docker image I want, in the Kubernetes plugin, the job runs, and I see that it claims to grab the correct Docker image:
<grabbing the correct info>
</grabbing the correct info>
The build command is as follows:
echo "Hello World!"
sleep 10
packer --version
The job fails:
It's not running the correct image.
If I go to the console and connect to the container I can run packer just fine.
> kubectl exec -it jenkins-node-8hql3 /bin/bash
Defaulting container name to jenkins-node.
Use 'kubectl describe pod/jenkins-node-8hql3 -n default' to see all of the containers in this pod.
jenkins@jenkins-node-8hql3:~$ packer --version
1.2.4
If I'm not being clear please let me know. I just want the Execute commands to run packer, ansible, maven, etc, which are installed on the image, but not accessible from the Freestyle job.
Unknown User (raylim)
Need some help.
Anyone knows how can I over-write the "tools→home path" when the container spins up with a pipeline PodTemplate?
Unknown User (rajak)
Need help regarding the config of Kuberenetes credential in cloud section of Jenkins.
I did provide the valid URL for the Kubernetes URL, but when trying to provide credentials, its not taking or even showing them inthe drop down list.
Credentials->Add->Jenkins-Kind→Kubernetes configuration(Kubeconfig)
I provided the following details
then I did select the "Add" Button.
But the credentials with ID Test_Kube_cred are not shown in the drop down for "Credential" section.
i tried all available options of "KubeConfig"
but none of the credentials are getting displayed in the drop down to select. I am not able to configure this plugin.
Is this a bug? or i am missing some thing there. can any one help me please.
Regards,
Raja.Kavuri
Unknown User (chengjingtao)
Need For Help.
what does it mean in jenkins log ?
"Terminating Kubernetes instance for agent jenkins-slave-h34rl-8p0cz"
my jenkinsfile is
pipeline{ agent{ label "docker" } stages{ stage("Clone"){ steps{ git "https://github.com/chengjingtao/alauda-ci.git" } } stage("Build"){ steps{ script{ stash "code" def label = "ci-${UUID.randomUUID().toString()}" podTemplate( label: label, containers:[ containerTemplate(name: 'ci-container', image: "gobuild:1.8-alpine", ttyEnabled: true, envVars: [envVar(key: "LANG", value: "C.UTF-8")]) ] ){ node(label) { container('ci-container') { unstash "code" sh "make build" stash "dest" } } } dir("__dest__"){ unstash "dest" } } } } stage("Else"){ steps{ echo "some thing else" } } } }it works well. but I found some exceptions in jenkins log
did I do something wrong ?
Thanks for your help.
Unknown User (vxavier)
I have same issue above, in Jenkins log and Agent log /\.
But my pipeline not work, returning:
Happens after update my Kubernetes cluster IKS to 1.11
Somebody help?
Unknown User (ottochiu)
Hi, I could be approaching this completely wrong, but let me start by describing my organization's infrastructure. We have an official in-house cloud that provides access to many powerful nodes. However, the cluster is plagued by frequent downtime and network issues. My group has been hurt so much by this that we decided to setup our own cluster. Since we operate on a much smaller budget for compute resources, we do not have the scale of the company's cluster.
We would like our Jenkins masters to live in our own cluster and would like to spawn pods in the company's cluster, when it is operational. During downtime, we would flip the settings in Jenkins and have our cluster provide the service. Degraded service is better than no service. I am not a k8s expert but from what I understand there are network routing challenges that need to be solved so that the pods in a different cluster can communicate back to the Jenkins master. Are there any tips in what needs to be setup to accomplish this? Any help is much appreciated!
Unknown User (yrsurya)
Hi everyone , I would like to know if there any community channel where we can discuss about this plugin like gitter , slack .. ? Please let me know
Unknown User (leonbadcam)
Unknown User (vxavier) I can only support your issue. Im just testing the new IKS 1.11 as we need to switch soon to it.
With IKS 1.11 I run into the same Issue.
There is JENKINS-53297 - Getting issue details... STATUS For this, please follow
containerTemplate(name: 'java', image: 'openjdk:8-jdk', ttyEnabled: true, command: 'cat') ]) { node('mypod') { stage('Checkout') { echo "test" } container('java') { stage('Static Code Analysis') { sh "java -version" } } } }Regards, Leon
Unknown User (continuous_geek)
Unknown User (csanchez) Is it possible to define a pod template and dynamically spin up a container in a stage or build step and have it added to the pod?
Unknown User (sli720)
How is it possible to use variables from the Jenkinsfile inside the YAML? I want to mount a volume depending on the workspace directory e.g.
- name: build
mountPath: ${WORKSPACE}/build
Unknown User (ottochiu)
I am using parameters with ${params.paramName123}. I imagine it will be something like ${env.WORKSPACE}.
Unknown User (ricktbaker)
Any examples for how to have multiple agents?
Unknown User (rkamath3)
Do you mean having multiple executors in a single pod? If yes, then it is not supported by the plugin as far as I know.
If you meant creating multiple Jenkins Nodes with the plugin. Then you can create multiple pods using podTemplate.
Unknown User (ricktbaker)
Have run into a problem. Everything has been working fine. Have Jenkins running in kubernetes and builds running, with an executor that comes online for the build and then terminates once the build is done. However, I just did an upgrade of everything, jenkins and all plugins and now the executor pod doesn't come online at all. I can't find any logs or errors so I'm not even sure where to start to look as to what is going on.
I've made no changes to my values file where the Agent is defined and enabled.
Any ideas?
Unknown User (timofeee)
It's a bit unclear to me but the documentation states that "Agents are launched using JNLP, so it is expected that the image connects automatically to the Jenkins master."
Does that I mean I'm unable to use an ssh slave agent with this plugin? We were previously looking into using ssh slave agents because our master is hosted internally behind a firewall but we want to spawn the slaves in k8.
Any recommendations for specific use case?
Unknown User (bernardomk)
Hi, is it possible to use Github OAuth? We currently use the agent provided by the Swarm Plugin and it works (username and personal access token). Couldn't find in the docs if it's supported or how to supply username and password.
Thanks.