Dealing with Workflows

Overview

Hello everyone, This article is part of the CircleCI series, The knowledge in this series is built in sequence.

in the last Article #2 Pipeline configuration we went through a quick view of understanding the YAML file structure by surfing on CricleCI Docs, and Viewing the CircleCI tool capabilities, Also created a simple pipeline.

Today's lesson will continue configuring our pipeline YAML file and will try to cover the Workflow topic

Workflows overview

As explained in CircleCI Docs, A workflow is a set of rules for defining a collection of jobs and their run order. Workflows support complex job orchestration using a simple set of configuration keys.

In the past lesson, we have configured our config.yaml file workflow section as the below:

workflows:
  default:
    jobs:
      - Build
      - Scan:  # this stage will run after Build stage
          requires: [Build]

So, Again the Scan job won't start till the Build job is finished, and the workflow graph appears like this:

So, This graph means the Build job will run first then the Scan job.

If you need to run both of them concurrently just remove the "Requires" step, as below:

workflows:
  default:
    jobs:
      - Build
      - Scan

and its graph will appear like this:

Let's take another example, Assuming that we have four sequential jobs each job waiting to start until the required job finishes successfully, the whole YAML file will be like this:

# Use the latest 2.1 version of CircleCI pipeline process engine.
version: 2.1

jobs: 
  Build:  # Job Name
    docker:  # in this example will built our code in docker container, however you can use linux, macOS, etc..
      - image: cimg/python:3.11.3  # Spcecify The python image.
    steps:  # Job Steps
      - checkout  # checkout code from GitHub
      - run: python --version  # print out python version
      - run:
          name: "Build Python app"
          command: |
            cd python
            python3 main.py


  Scan_Build:  # Job Name
    docker:  # in this example will built our code in docker container, however you can use linux, macOS, etc..
      - image: cimg/python:3.11.3  # Spcecify The python image.
    steps:  # Job Steps
      - checkout  # checkout code from GitHub
      - run: python --version  # print out python version
      - run:
          name: "Scan Python app"
          command: |
            cd python
            python3 main_scan.py

  Deploy:  # Job Name
    docker:  # in this example will built our code in docker container, however you can use linux, macOS, etc..
      - image: cimg/python:3.11.3  # Spcecify The python image.
    steps:  # Job Steps
      - checkout  # checkout code from GitHub
      - run: python --version  # print out python version
      - run:
          name: "Build Python app"
          command: |
            cd python
            python3 main.py 

  Scan_Deploy:  # Job Name
    docker:  # in this example will built our code in docker container, however you can use linux, macOS, etc..
      - image: cimg/python:3.11.3  # Spcecify The python image.
    steps:  # Job Steps
      - checkout  # checkout code from GitHub
      - run: python --version  # print out python version
      - run:
          name: "Scan Python app"
          command: |
            cd python
            python3 main_scan.py


# Orchestrate jobs using workflows
# See: https://circleci.com/docs/configuration-reference/#workflows
workflows:
  default:
    jobs:
      - Build
      - Scan_Build:
          requires: [Build]
      - Deploy:
          requires: [Scan_Build]
      - Scan_Deploy:
          requires: [Deploy]

And its graph will appear like this:

Another example, Assuming we wanna accelerate our pipeline, to run the deploy job after the Build job directly, and the Scan_deploy requires The Scan_build and deploy jobs, the workflow config will be:

workflows:
  default:
    jobs:
      - Build
      - Scan_Build:
          requires: [Build]
      - Deploy:
          requires: [Build]
      - Scan_Deploy:
          requires: [Deploy, Scan_Build]

And the graph will be:

Also, I wanna share with you an error I got while configuring the pipeline, to view how it's easy to troubleshoot with CircleCI

very simple, Right!?

Workflows branches Filter

Workflows allow you to filter branches if you have set your triggers on multiple branches, Therefore you can filter a branch for a specific job.

if you need to setup multiple branches, just select all branches and set the trigger pipeline config. Here in my example, I configured it just for the main branch, However if you want to set it for all branches and in your pipeline needs to run a specific job only from a specific branch follow the below YAML config.

View the config file workflows section below:

workflows:
  default:
    jobs:
      - Build
      - Scan_Build:
          requires: [Build]
      - Deploy:
          requires: [Build]
      - Scan_Deploy:
          requires: [Deploy, Scan_Build]
          filters:
            branches:
              only: [master]

The Scan_deploy job won't run due to we don't have The Master branch we have the main instead, however you got the idea; The graph will be:

Workflow manual approval

As explained in CircleCI Docs, Workflows can be configured to wait for manual approval of a job before continuing, Assuming That your pipeline applies some configuration on the production application and any change sure will affect the running application, Therefor you requested to Hold this job and ask for approval before continuing.

The config.yaml file workflows section should be like the below:

workflows:
  default:
    jobs:
      - Build

      - Scan_Build:
          requires: [Build]

      - hold:
          type: approval
          requires: [Scan_Build]

      - Deploy:
          requires: [hold]

      - Scan_Deploy:
          requires: [Deploy]

Here we created a hold job with an approval type, and it requires The Scan_Build job; So after the Scan_Build job the pipeline will ask for approval; The pipeline waiting for approval.

Press on the hold job to approve:

After approval, the pipeline will continue with the other jobs.

That's it, Hope this article inspired you and will appreciate your feedback.
Thank you.

#3 Dealing with Workflows

#circleci #cicd

Table of contents

Overview

Workflows overview

Workflows branches Filter

Workflow manual approval