Gitlab CI run multiple similar, but independent pipelines

Development Software Development
1

Since a few here are using gilab-ci I figure this might be either interesting for people to find out or someone might already know.

I’ve played with gitlab-ci yesterday and also right now.

The end result looks somewhat like this

image
image.png776×173 5.43 KB

Now that looks GREAT, but what I wanna achieve is two identical pipelines for building a docker image for JRiver Media Center, both the stable and the latest version. But have the stable and latest build be able to fail independetly. This is good if the latest build stops working or most likely whenever version 26 comes out there won’t be a stable version right away. Right now if latest where to fail the build the test stage would not even begin, even though stable might have succeeded.

The current gitlab-ci.yml you can find here. If you are interested.

or there... 

image: docker:latest
services:
  - docker:dind

stages:
  - build
  - test
  - release

before_script:
  - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY

# JRiver Media Center 25 (stable)

build-master-stable:
  stage: build
  script:
    - ./scripts/build.sh 25 stable
  only:
    - master

test-master-stable:
  stage: test
  script:
    - ./scripts/test.sh jrivermc:stable
  dependencies:
    - build-master-stable
  only:
    - master

release-master-stable:
  stage: release
  script:
    - ./scripts/release.sh jrivermc:stable stable
  dependencies:
    - test-master-stable
  only:
    - master

# JRiver Media Center 25 (latest)

build-master-latest:
  stage: build
  script:
    - ./scripts/build.sh 25 latest
  only:
    - master

test-master-latest:
  stage: test
  script:
    - ./scripts/test.sh jrivermc:latest
  dependencies:
    - build-master-latest
  only:
    - master

release-master-latest:
  stage: release
  script:
    - ./scripts/release.sh jrivermc:latest latest
  dependencies:
    - test-master-latest
  only:
    - master

I already tried to add more stages. Like build-stable, build-latest, test-stable… You get the idea. That ends up being a single long line. Accomplishes nothing, other than the build jobs can’t even run in parallel anymore.

I’m not sure yet from what I found if that’s possible at all quite the way I would want it. I would know workarounds by making pipelines not-fail and instead write something somewhere that indicates it failed until I get to the release stage, then it can fail. But it’s awkward, tedious and not very pretty.

2

So, this does not look like a thing you can do quite yet.

But I’ve found a better workaround, than the file thing. (Did not try yet).

You can have 1 pipeline that triggers when you trigger it with the gitlab api with various input variables and then another stage that triggers that pipeline twice with the different inputs when you push to a branch.

3

Turns out this is much more dead simple than I thought it would be. Since I don’t actually need to run different pipelines. In the pipeline Schedules you can set environment variables too. So the yml is now this.

image: docker:latest

services:
  - docker:dind

variables:
  CI_REGISTRY: "docker.io"
  CI_JRIVER_RELEASE: "25"

stages:
  - build
  - test
  - release

before_script:
  - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"

build-master-stable:
  stage: build
  script:
    - ./scripts/build.sh "$CI_JRIVER_RELEASE" "$CI_JRIVER_RELEASE_TAG"
  only:
    - triggers
    - schedules

test-master-stable:
  stage: test
  script:
    - ./scripts/test.sh "jrivermc:$CI_JRIVER_RELEASE_TAG"
  only:
    - triggers
    - schedules

release-master-stable:
  stage: release
  script:
    - ./scripts/release.sh "jrivermc:$CI_JRIVER_RELEASE_TAG" "$CI_JRIVER_RELEASE_TAG"
  only:
    - triggers
    - schedules

And I define CI_JRIVER_RELEASE_TAG and the repository for every schedule individually. CI_JRIVER_RELEASE is probably gonna get the same treatment, but I wanna see what it does when I set it to 24 from the Schedule and it’s 25 in the yml.

Anyways, now I have my 2 same pipeline with different variables that run without killing each other if one or the other where to fai. :slight_smile: