$ oc login -u <login> -p <password> https://openshift.example.com:6443
With Red Hat OpenShift Pipelines, you can create a customized CI/CD solution to build, test, and deploy your application.
To create a full-fledged, self-serving CI/CD pipeline for an application, you must perform the following tasks:
Create custom tasks, or install existing reusable tasks.
Create and define the delivery pipeline for your application.
Provide a storage volume or filesystem that is attached to a workspace for the pipeline execution using one of the following approaches:
Specify a volume claim template that creates a persistent volume claim
Specify a persistent volume claim
Create a PipelineRun
object to instantiate and invoke the pipeline.
Add triggers to capture events in the source repository.
This section uses the pipelines-tutorial
example to demonstrate the preceding tasks. The example uses a simple application which consists of:
A front-end interface, vote-ui
, with the source code in the ui-repo
Git repository.
A back-end interface, vote-api
, with the source code in the api-repo
Git repository.
The apply-manifests
and update-deployment
tasks in the pipelines-tutorial
Git repository.
You have access to an OKD cluster.
You have installed OpenShift Pipelines using the Red Hat OpenShift Pipelines Operator listed in the OpenShift OperatorHub. Once installed, it is applicable to the entire cluster.
You have installed OpenShift Pipelines CLI.
You have forked the front-end ui-repo
and back-end api-repo
Git repositories using your GitHub ID, and have Administrator access to these repositories.
Optional: You have cloned the pipelines-tutorial
Git repository.
Log in to your OKD cluster:
$ oc login -u <login> -p <password> https://openshift.example.com:6443
Create a project for the sample application. For this example workflow, create the pipelines-tutorial
project:
$ oc new-project pipelines-tutorial
If you create a project with a different name, be sure to update the resource URLs used in the example with your project name. |
View the pipeline
ServiceAccount:
Red Hat OpenShift Pipelines Operator adds and configures a ServiceAccount named pipeline
that has sufficient permissions to build and push an image. This ServiceAccount is used by PipelineRun.
$ oc get serviceaccount pipeline
Install the apply-manifests
and update-deployment
task resources from the pipelines-tutorial
repository, which contains a list of reusable tasks for pipelines:
$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.3/01_pipeline/01_apply_manifest_task.yaml
$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.3/01_pipeline/02_update_deployment_task.yaml
Use the tkn task list
command to list the tasks you created:
$ tkn task list
The output verifies that the apply-manifests
and update-deployment
task resources were created:
NAME DESCRIPTION AGE
apply-manifests 1 minute ago
update-deployment 48 seconds ago
Use the tkn clustertasks list
command to list the Operator-installed additional cluster tasks, for example, buildah
and s2i-python-3
:
To use the |
$ tkn clustertasks list
The output lists the Operator-installed ClusterTask
resources:
NAME DESCRIPTION AGE
buildah 1 day ago
git-clone 1 day ago
s2i-php 1 day ago
tkn 1 day ago
A Pipeline represents a CI/CD flow and is defined by the Tasks to be executed. It is designed to be generic and reusable in multiple applications and environments.
A Pipeline specifies how the Tasks interact with each other and their order of execution using the from
and runAfter
parameters. It uses the workspaces
field to specify one or more volumes that each Task in the Pipeline requires during execution.
In this section, you will create a Pipeline that takes the source code of the application from GitHub and then builds and deploys it on OKD.
The Pipeline performs the following tasks for the back-end application vote-api
and front-end application vote-ui
:
Clones the source code of the application from the Git repository by referring to the git-url
and git-revision
parameters.
Builds the container image using the buildah
ClusterTask.
Pushes the image to the internal image registry by referring to the image
parameter.
Deploys the new image on OKD by using the apply-manifests
and update-deployment
Tasks.
Copy the contents of the following sample Pipeline YAML file and save it:
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: build-and-deploy
spec:
workspaces:
- name: shared-workspace
params:
- name: deployment-name
type: string
description: name of the deployment to be patched
- name: git-url
type: string
description: url of the git repo for the code of deployment
- name: git-revision
type: string
description: revision to be used from repo of the code for deployment
default: "pipelines-1.3"
- name: IMAGE
type: string
description: image to be build from the code
tasks:
- name: fetch-repository
taskRef:
name: git-clone
kind: ClusterTask
workspaces:
- name: output
workspace: shared-workspace
params:
- name: url
value: $(params.git-url)
- name: subdirectory
value: ""
- name: deleteExisting
value: "true"
- name: revision
value: $(params.git-revision)
- name: build-image
taskRef:
name: buildah
kind: ClusterTask
params:
- name: TLSVERIFY
value: "false"
- name: IMAGE
value: $(params.IMAGE)
workspaces:
- name: source
workspace: shared-workspace
runAfter:
- fetch-repository
- name: apply-manifests
taskRef:
name: apply-manifests
workspaces:
- name: source
workspace: shared-workspace
runAfter:
- build-image
- name: update-deployment
taskRef:
name: update-deployment
workspaces:
- name: source
workspace: shared-workspace
params:
- name: deployment
value: $(params.deployment-name)
- name: IMAGE
value: $(params.IMAGE)
runAfter:
- apply-manifests
The Pipeline definition abstracts away the specifics of the Git source repository and image registries. These details are added as params
when a Pipeline is triggered and executed.
Create the Pipeline:
$ oc create -f <pipeline-yaml-file-name.yaml>
Alternatively, you can also execute the YAML file directly from the Git repository:
$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.3/01_pipeline/04_pipeline.yaml
Use the tkn pipeline list
command to verify that the Pipeline is added to the application:
$ tkn pipeline list
The output verifies that the build-and-deploy
Pipeline was created:
NAME AGE LAST RUN STARTED DURATION STATUS build-and-deploy 1 minute ago --- --- --- ---
To run OpenShift Pipelines in a disconnected cluster or a cluster provisioned in a restricted environment, you must ensure that either the Samples Operator is configured for a restricted network, or a cluster administrator has created a cluster with a mirrored registry.
The following procedure uses the pipelines-tutorial
example to create a pipeline for an application in a restricted environment using a cluster with a mirrored registry. To ensure that the pipelines-tutorial
example works in a restricted environment, you must mirror the respective builder images from the mirror registry for the front-end interface, vote-ui
; back-end interface, vote-api
; and the cli
.
Mirror the builder image from the mirror registry for the front-end interface, vote-ui
.
Verify that the required images tag is not imported:
$ oc describe imagestream python -n openshift
Name: python
Namespace: openshift
[...]
3.8-ubi8 (latest)
tagged from registry.redhat.io/ubi8/python-38:latest
prefer registry pullthrough when referencing this tag
Build and run Python 3.8 applications on UBI 8. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/s2i-python-container/blob/master/3.8/README.md.
Tags: builder, python
Supports: python:3.8, python
Example Repo: https://github.com/sclorg/django-ex.git
[...]
Mirror the supported image tag to the private registry:
$ oc image mirror registry.redhat.io/ubi8/python-38:latest <mirror-registry>:<port>/ubi8/python-38
Import the image:
$ oc tag <mirror-registry>:<port>/ubi8/python-38 python:latest --scheduled -n openshift
You must periodically re-import the image. The --scheduled
flag enables automatic re-import of the image.
Verify that the images with the given tag have been imported:
$ oc describe imagestream python -n openshift
Name: python
Namespace: openshift
[...]
latest
updates automatically from registry <mirror-registry>:<port>/ubi8/python-38
* <mirror-registry>:<port>/ubi8/python-38@sha256:3ee3c2e70251e75bfeac25c0c33356add9cc4abcbc9c51d858f39e4dc29c5f58
[...]
Mirror the builder image from the mirror registry for the back-end interface, vote-api
.
Verify that the required images tag is not imported:
$ oc describe imagestream golang -n openshift
Name: golang
Namespace: openshift
[...]
1.14.7-ubi8 (latest)
tagged from registry.redhat.io/ubi8/go-toolset:1.14.7
prefer registry pullthrough when referencing this tag
Build and run Go applications on UBI 8. For more information about using this builder image, including OpenShift considerations, see https://github.com/sclorg/golang-container/blob/master/README.md.
Tags: builder, golang, go
Supports: golang
Example Repo: https://github.com/sclorg/golang-ex.git
[...]
Mirror the supported image tag to the private registry:
$ oc image mirror registry.redhat.io/ubi8/go-toolset:1.14.7 <mirror-registry>:<port>/ubi8/go-toolset
Import the image:
$ oc tag <mirror-registry>:<port>/ubi8/go-toolset golang:latest --scheduled -n openshift
You must periodically re-import the image. The --scheduled
flag enables automatic re-import of the image.
Verify that the images with the given tag have been imported:
$ oc describe imagestream golang -n openshift
Name: golang
Namespace: openshift
[...]
latest
updates automatically from registry <mirror-registry>:<port>/ubi8/go-toolset
* <mirror-registry>:<port>/ubi8/go-toolset@sha256:59a74d581df3a2bd63ab55f7ac106677694bf612a1fe9e7e3e1487f55c421b37
[...]
Mirror the builder image from the mirror registry for the cli
.
Verify that the required images tag is not imported:
$ oc describe imagestream cli -n openshift
Name: cli
Namespace: openshift
[...]
latest
updates automatically from registry quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:65c68e8c22487375c4c6ce6f18ed5485915f2bf612e41fef6d41cbfcdb143551
* quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:65c68e8c22487375c4c6ce6f18ed5485915f2bf612e41fef6d41cbfcdb143551
[...]
Mirror the supported image tag to the private registry:
$ oc image mirror quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:65c68e8c22487375c4c6ce6f18ed5485915f2bf612e41fef6d41cbfcdb143551 <mirror-registry>:<port>/openshift-release-dev/ocp-v4.0-art-dev:latest
Import the image:
$ oc tag <mirror-registry>:<port>/openshift-release-dev/ocp-v4.0-art-dev cli:latest --scheduled -n openshift
You must periodically re-import the image. The --scheduled
flag enables automatic re-import of the image.
Verify that the images with the given tag have been imported:
$ oc describe imagestream cli -n openshift
Name: cli
Namespace: openshift
[...]
latest
updates automatically from registry <mirror-registry>:<port>/openshift-release-dev/ocp-v4.0-art-dev
* <mirror-registry>:<port>/openshift-release-dev/ocp-v4.0-art-dev@sha256:65c68e8c22487375c4c6ce6f18ed5485915f2bf612e41fef6d41cbfcdb143551
[...]
A PipelineRun
resource starts a pipeline and ties it to the Git and image resources that should be used for the specific invocation. It automatically creates and starts the TaskRun
resources for each task in the pipeline.
Start the pipeline for the back-end application:
$ tkn pipeline start build-and-deploy \
-w name=shared-workspace,volumeClaimTemplateFile=https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.3/01_pipeline/03_persistent_volume_claim.yaml \
-p deployment-name=vote-api \
-p git-url=https://github.com/openshift-pipelines/vote-api.git \
-p IMAGE=image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/vote-api
The previous command uses a volume claim template, which creates a persistent volume claim for the pipeline execution.
To track the progress of the pipeline run, enter the following command::
$ tkn pipelinerun logs <pipelinerun_id> -f
The <pipelinerun_id> in the above command is the ID for the PipelineRun
that was returned in the output of the previous command.
Start the Pipeline for the front-end application:
$ tkn pipeline start build-and-deploy \
-w name=shared-workspace,volumeClaimTemplateFile=https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.3/01_pipeline/03_persistent_volume_claim.yaml \
-p deployment-name=vote-ui \
-p git-url=https://github.com/openshift-pipelines/vote-ui.git \
-p IMAGE=image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/vote-ui
To track the progress of the pipeline run, enter the following command:
$ tkn pipelinerun logs <pipelinerun_id> -f
The <pipelinerun_id> in the above command is the ID for the PipelineRun
that was returned in the output of the previous command.
After a few minutes, use tkn pipelinerun list
command to verify that the Pipeline ran successfully by listing all the PipelineRuns:
$ tkn pipelinerun list
The output lists the PipelineRuns:
NAME STARTED DURATION STATUS
build-and-deploy-run-xy7rw 1 hour ago 2 minutes Succeeded
build-and-deploy-run-z2rz8 1 hour ago 19 minutes Succeeded
Get the application route:
$ oc get route vote-ui --template='http://{{.spec.host}}'
Note the output of the previous command. You can access the application using this route.
To rerun the last pipeline run, using the pipeline resources and service account of the previous pipeline, run:
$ tkn pipeline start build-and-deploy --last
Triggers enable pipelines to respond to external GitHub events, such as push events and pull requests. After you assemble and start a Pipeline for the application, add the TriggerBinding
, TriggerTemplate
, Trigger
, and EventListener
resources to capture the GitHub events.
Copy the content of the following sample TriggerBinding
YAML file and save it:
apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerBinding
metadata:
name: vote-app
spec:
params:
- name: git-repo-url
value: $(body.repository.url)
- name: git-repo-name
value: $(body.repository.name)
- name: git-revision
value: $(body.head_commit.id)
Create the TriggerBinding
resource:
$ oc create -f <triggerbinding-yaml-file-name.yaml>
Alternatively, you can create the TriggerBinding
resource directly from the pipelines-tutorial
Git repository:
$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.3/03_triggers/01_binding.yaml
Copy the content of the following sample TriggerTemplate
YAML file and save it:
apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerTemplate
metadata:
name: vote-app
spec:
params:
- name: git-repo-url
description: The git repository url
- name: git-revision
description: The git revision
default: pipelines-1.3
- name: git-repo-name
description: The name of the deployment to be created / patched
resourcetemplates:
- apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: build-deploy-$(tt.params.git-repo-name)-$(uid)
spec:
serviceAccountName: pipeline
pipelineRef:
name: build-and-deploy
params:
- name: deployment-name
value: $(tt.params.git-repo-name)
- name: git-url
value: $(tt.params.git-repo-url)
- name: git-revision
value: $(tt.params.git-revision)
- name: IMAGE
value: image-registry.openshift-image-registry.svc:5000/pipelines-tutorial/$(tt.params.git-repo-name)
workspaces:
- name: shared-workspace
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 500Mi
The template specifies a volume claim template to create a persistent volume claim for defining the storage volume for the workspace. Therefore, you do not need to create a persistent volume claim to provide data storage.
Create the TriggerTemplate
resource:
$ oc create -f <triggertemplate-yaml-file-name.yaml>
Alternatively, you can create the TriggerTemplate
resource directly from the pipelines-tutorial
Git repository:
$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.3/03_triggers/02_template.yaml
Copy the contents of the following sample Trigger
YAML file and save it:
apiVersion: triggers.tekton.dev/v1alpha1
kind: Trigger
metadata:
name: vote-trigger
spec:
serviceAccountName: pipeline
bindings:
- ref: vote-app
template:
ref: vote-app
Create the Trigger
resource:
$ oc create -f <trigger-yaml-file-name.yaml>
Alternatively, you can create the Trigger
resource directly from the pipelines-tutorial
Git repository:
$ oc create -f https://github.com/openshift/pipelines-tutorial/blob/pipelines-1.3/03_triggers/03_trigger.yaml
Copy the contents of the following sample EventListener
YAML file and save it:
apiVersion: triggers.tekton.dev/v1alpha1
kind: EventListener
metadata:
name: vote-app
spec:
serviceAccountName: pipeline
triggers:
- triggerRef: vote-trigger
Create the EventListener
resource:
$ oc create -f <eventlistener-yaml-file-name.yaml>
Alternatively, you can create the EvenListener
resource directly from the pipelines-tutorial
Git repository:
$ oc create -f https://raw.githubusercontent.com/openshift/pipelines-tutorial/pipelines-1.3/03_triggers/04_event_listener.yaml
Expose the EventListener
service as an OKD route to make it publicly accessible:
$ oc expose svc el-vote-app
Webhooks are HTTP POST messages that are received by the EventListeners whenever a configured event occurs in your repository. The event payload is then mapped to TriggerBindings, and processed by TriggerTemplates. The TriggerTemplates eventually start one or more PipelineRuns, leading to the creation and deployment of Kubernetes resources.
In this section, you will configure a Webhook URL on your forked Git repositories vote-ui
and vote-api
. This URL points to the publicly accessible EventListener service route.
Adding Webhooks requires administrative privileges to the repository. If you do not have administrative access to your repository, contact your system administrator for adding Webhooks. |
Get the Webhook URL:
$ echo "URL: $(oc get route el-vote-app --template='http://{{.spec.host}}')"
Note the URL obtained in the output.
Configure Webhooks manually on the front-end repository:
Open the front-end Git repository vote-ui
in your browser.
Click Settings → Webhooks → Add Webhook
On the Webhooks/Add Webhook page:
Enter the Webhook URL from step 1 in Payload URL field
Select application/json for the Content type
Specify the secret in the Secret field
Ensure that the Just the push event is selected
Select Active
Click Add Webhook
Repeat step 2 for the back-end repository vote-api
.
Whenever a push
event occurs in the Git repository, the configured Webhook sends an event payload to the publicly exposed EventListener
service route. The EventListener
service of the application processes the payload, and passes it to the relevant TriggerBinding
and TriggerTemplate
resource pairs. The TriggerBinding
resource extracts the parameters and the TriggerTemplate
resource uses these parameters and specifies the way the resources must be created. This may rebuild and redeploy the application.
In this section, you push an empty commit to the front-end vote-ui
repository, which then triggers the pipeline run.
From the terminal, clone your forked Git repository vote-ui
:
$ git clone git@github.com:<your GitHub ID>/vote-ui.git -b pipelines-1.3
Push an empty commit:
$ git commit -m "empty-commit" --allow-empty && git push origin pipelines-1.3
Check if the pipeline run was triggered:
$ tkn pipelinerun list
Notice that a new pipeline run was initiated.
For more details on pipelines in the Developer perspective, see the working with Pipelines in the Developer perspective section.
To learn more about Security Context Constraints (SCCs), see the Managing Security Context Constraints section.
For more examples of reusable tasks, see the OpenShift Catalog repository. Additionally, you can also see the Tekton Catalog in the Tekton project.