Customizing the build pipeline

To meet your specific needs, you can customize the way that Konflux builds components. For instance:

  • Change the parameters of the build pipeline of a component.

  • Configure the timeouts for the pipeline.

  • Extend the pipeline with your own tasks (or delete existing tasks).

For example, you might decide to limit how long Konflux saves the images that it builds for pull requests. You can set a limit by changing the value of the image-expires-after parameter. Or you can ensure that images are compliant with regulations for your industry, by adding a compliance check as a task to the build pipeline.

Changing parameters

Procedure

  1. In the PipelineRun YAML files in the .tekton directory, customize the parameters in the .spec.params section:

    1. You can change the default value of existing parameters. For example, for PRs, you could change the value of image-expires-after from the default 5d to 1w, so images built for PRs expire after a week rather than 5 days.

       spec:
   params:
   - name: image-expires-after
-    value: 5d
+    value: 1w
   - name: dockerfile
     value: Dockerfile

    2. You can also add parameters. New parameters must include a name and value.

       spec:
   params:
+  - name: hermetic
+    value: "true"
   - name: image-expires-after
     value: 1w
   - name: dockerfile
     value: Dockerfile

      Refer to the params section in your Pipeline to see the available parameters. When looking at a PipelineRun file, you can typically find the Pipeline definition in:

      • The PipelineRun file itself, inlined in .spec.pipelineSpec

      • A separate file in the .tekton directory, referenced by name in .spec.pipelineRef

      • An external location, referenced by a resolver in .spec.pipelineRef

  2. Commit your changes to the repository of the component.

Configuring timeouts

By default, your pipeline has a timeout of 1 hour (as does each task in the pipeline individually). Your builds may take longer than that, in which case you will need to increase the timeouts accordingly.

Procedure

For example, let’s configure the timeouts for a pipeline where the build itself may take up to 3 hours and antivirus scanning may take up to 2 hours. We’ll set corresponding timeouts for both the tasks and a 6 hour total timeout for the pipeline.

  1. In the PipelineRun files in the .tekton directory, set the .spec.timeouts. This is the overall timeout for the pipeline.

    kind: PipelineRun
spec:
  timeouts:
    # See https://tekton.dev/docs/pipelines/pipelineruns/#configuring-a-failure-timeout
    pipeline: 6h

  2. In the Pipeline definitions in the .tekton directory, set the timeout for the relevant tasks:

    kind: Pipeline
spec:
  tasks:
    # ...
    - name: build-container
      timeout: 3h
      runAfter:
        - clone-repository
      taskRef: ...
    - name: clamav-scan
      timeout: 2h
      runAfter:
        - build-container
      taskRef: ...
    This example shows a separate Pipeline file, but your pipeline may also be defined directly in your PipelineRun file(s) (in .spec.pipelineSpec), in which case you configure all the timeouts in the PipelineRun.

Extending the build pipeline with your own tasks

Before you extend your build pipeline, be aware that doing so may cause builds to fail an Enterprise Contract check, if you are using EC. To resolve this issue, please reference the next procedure.

Procedure

  1. In each Pipeline definition in the .tekton directory, add a new task to the tasks section.

    Example task:

      name: example-task
  params:
  - name: example-param
    value: "Example"
  runAfter:
  - build-container  # You can be more specific by choosing another task
  taskRef:
    params:
    - name: name
      value: example-task  # metadata.name field of the Task
    - name: bundle
      value: quay.io/tekton-bundle-catalog/example-task-bundle:1.0
      # For more details on tekton bundles, refer to https://tekton.dev/docs/pipelines/pipelines/#tekton-bundles
    - name: kind
      value: task
    resolver: bundles
  when:
  - input: $(params.skip-checks)  # This references the pipeline parameters
    operator: in
    values:
    - "false"

    An example of a custom task added to the pipeline that sends a slack notification when the Pipelinerun fails:

      name: slack-webhook-notification
  params:
    - name: message
      value: PipelineRun $(context.pipelineRun.name) failed
    - name: secret-name
      value: my-secret  # name of secret in the your namespace which contains slack web-hook URL under key specified in 'key-name' parameter below
    - name: key-name
      value: dev-team
  taskRef:
    params:
    - name: bundle
      value: quay.io/konflux-ci/tekton-catalog/task-slack-webhook-notification:0.1
    - name: name
      value: slack-webhook-notification
    - name: kind
      value: Task
    resolver: bundles
  when:
    - input: $(tasks.status)
      operator: in
      values: ["Failed"]

  2. Commit your changes to the repository of the component.

  • To use slack-webhook-notification task, you need to create a secret in your namespace with at least one key where the value is the webhook URL. For example, to create a secret for Slack, run oc create secret generic my-secret --from-literal dev-team=https://hooks.slack.com/services/XXX/XXXXXX

  • If you want to define a task directly in this file, rather than using taskRef, you can use taskSpec. See the Tekton documentation on specifying the target task for more details.

Preventing issues with the Enterprise Contract

Custom Tasks may need access to data from other Tasks. However, in order to not break the chain of trust in a build Pipeline, there are restrictions in modifying such data. For example, a custom Task should not be allowed to modify the component’s source code. If you are using the Enterprise Contract (EC) to verify your builds, introducing a custom Task may violate the Trusted Tasks rule. See Trusted Artifacts for how to safely allow share data between Tasks.

Exchanging the build pipeline build task with higher memory limits

If possible, prefer overriding compute resources directly in your PipelineRun file. It is a much more flexible and simpler approach. Use this procedure as a last resort, if the relevant features are not enabled in your cluster.

The buildah task, which builds components from a Dockerfile, has a memory limit of 4 GB. To build components with memory requirements greater than 4 GB, use the following tasks:

Procedure

To exchange the build task with a memory limit of 6 GB, complete the following steps. For a memory limit of 8 or 10 GB, replace the references to 6 GB with the appropriate values.

  1. Go to the GitHub repo of your component.

  2. In each Pipeline definition in the .tekton directory, under tasks, locate the task named build-container:

    1. Under .taskRef.params, set name to buildah-6gb.

    2. Under .taskRef.params, change the value of bundle - replace the repository name with task-buildah-6gb. Keep the version tag (e.g. 0.2) and remove the @sha256:…​ digest.

       spec:
   pipelineSpec:
     tasks:
       # ...
       - name: build-container
         taskRef:
           resolver: bundles
           params:
             - name: kind
               value: task
             - name: name
-              value: buildah
+              value: buildah-6gb
             - name: bundle
-              value: quay.io/konflux-ci/tekton-catalog/task-buildah:0.2@sha256:3f0913dfb85e9aeb9916e412d10329528ddf4c8fba9958cba5291ca8ee247f7e
+              value: quay.io/konflux-ci/tekton-catalog/task-buildah-6gb:0.2

    3. If you’d like, pin the task-buildah-6gb bundle to a digest. Take the output of the following script (requires skopeo) and append it to the bundle value:

      my_bundle=quay.io/konflux-ci/tekton-catalog/task-buildah-6gb:0.2

skopeo inspect --format '@{{.Digest}}' --no-tags docker://"${my_bundle}"

      Konflux task version tags (e.g. 0.2) are "floating" - they move to the latest release every time the task gets an update. By pinning to a digest, you ensure your pipeline will always use the exact same version of the task. Konflux automatically sends out PRs to update task digests, letting you opt into updating the task rather than having it update on its own behind your back. The task update PR will trigger the on-pull-request pipeline, testing the changes before they can affect the on-push pipeline or other PRs.

Bring your own Quay repository to the build pipeline

By default, all pipelines push the images to a local repository that is set up as a part of installation. Ths registry address is registry-service.kind-registry:5001. It is not mandatory to use this local repo, so if you want to use your own Quay repo to control user permissions, you can do this by following the instructions for configuring a push secret for the build piepline.

Verification

When you commit changes to these .yaml files in your repository, Konflux automatically triggers a new build. Wait for Konflux to complete the new build, then verify your changes have been made by following these steps:

  1. Navigate to Activity > Pipeline runs.

  2. Select the most recent build pipeline run.

  3. In the Details tab, confirm that there are new tasks that you added in the pipeline visualization.

  4. In the Logs tab, confirm the following:

    1. Any new tasks are in the navigation bar.

    2. If you changed a parameter’s value, and that value gets printed, the new value is in the log.

Troubleshooting

If you experience any issues with your customized pipeline, try the following solutions:

  • If you believe that your desired parameter values are not being passed into the pipeline, make sure that your assignment of that value doesn’t get overwritten later in the .yaml file.

  • If your new task is not appearing in the pipeline run, ensure the following:

    • You added it to the correct place in the .yaml files, so that it has the path .spec.tasks or .pipelineSpec.tasks.

    • You specified a valid runAfter field, and that the task in that field completed successfully.

  • For problems with both parameters and tasks, make sure you committed your changes to the .tekton directory in the repository that Konflux references for the component.

Additional resources