[Xenomai-images PATCH 1/4] ci: Build and test from gitlab.com

Gylstorff Quirin quirin.gylstorff at siemens.com
Tue Apr 21 11:19:56 CEST 2020



On 4/21/20 11:09 AM, Jan Kiszka wrote:
> On 21.04.20 10:31, Q. Gylstorff wrote:
>> From: Quirin Gylstorff <quirin.gylstorff at siemens.com>
>>
>> Adapt scripts and gitlab-ci.yml to build on public available
>> gitlab infrastructure.
>>
>> As the internal gitlab system has no artifact storage and
>> it is not possible to deactivate arifact upload with a variable
>> split the gitlab-ci.yml. The .gitlab-ci.yml in the root points
>> to .ci/gitlab-com.yml.
> 
> Why hidden ".ci/" and not just "ci/"?

I modeled it after the .gitlab-ci.yml. If it is better to have it
visible a will ajust that in a second patch set.

> 
>>
>> Signed-off-by: Quirin Gylstorff <quirin.gylstorff at siemens.com>
>> ---
>>   .ci/gitlab-ci-base.yml                        | 153 ++++++++++++++++++
>>   .ci/gitlab-com.yml                            |  26 +++
>>   .ci/siemens-ci.yml                            |  22 +++
>>   .gitlab-ci.yml                                | 145 +----------------
>>   scripts/deploy_for_testing.sh                 |   4 +-
>>   scripts/get-build-id.sh                       |  29 ++++
>>   scripts/install-lavacli.sh                    |   5 +-
>>   scripts/run-lava-tests.sh                     |   8 +-
>>   ...bone.yml => xenomai-beagle-bone-black.yml} |   4 +-
>>   tests/jobs/xenomai-hikey.yml                  |   4 +-
>>   tests/jobs/xenomai-ipc227e.yml                |   6 +-
>>   tests/jobs/xenomai-minnow.yml                 |   6 +-
>>   tests/jobs/xenomai-qemu-amd64.yml             |   6 +-
>>   tests/jobs/xenomai-qemu-arm64.yml             |   6 +-
>>   tests/jobs/xenomai-qemu-armhf.yml             |   6 +-
>>   ...omai-x86-64.yml => xenomai-x86-64-efi.yml} |   6 +-
>>   16 files changed, 265 insertions(+), 171 deletions(-)
>>   create mode 100644 .ci/gitlab-ci-base.yml
>>   create mode 100644 .ci/gitlab-com.yml
>>   create mode 100644 .ci/siemens-ci.yml
>>   create mode 100755 scripts/get-build-id.sh
>>   rename tests/jobs/{xenomai-beaglebone.yml => 
>> xenomai-beagle-bone-black.yml} (91%)
>>   rename tests/jobs/{xenomai-x86-64.yml => xenomai-x86-64-efi.yml} (86%)
>>
>> diff --git a/.ci/gitlab-ci-base.yml b/.ci/gitlab-ci-base.yml
>> new file mode 100644
>> index 0000000..e8d259f
>> --- /dev/null
>> +++ b/.ci/gitlab-ci-base.yml
>> @@ -0,0 +1,153 @@
>> +#
>> +# Xenomai Real-Time System
>> +#
>> +# Copyright (c) Siemens AG, 2019 - 2020
>> +#
>> +# Authors:
>> +#  Quirin Gylstorff <quirin.gylstorff at siemens.com>
>> +#
>> +# SPDX-License-Identifier: MIT
>> +#
>> +stages:
>> +  - build
>> +  - lava-test
>> +
>> +variables:
>> +  GIT_STRATEGY: clone
>> +  http_proxy: "$HTTP_PROXY"
>> +  https_proxy: "$HTTPS_PROXY"
>> +  ftp_proxy: "$FTP_PROXY"
>> +  no_proxy: "$NO_PROXY"
>> +  XENOMAI_BUILD_OPTION: ":opt-xenomai-next.yml"
>> +  LINUX_BUILD_OPTION: ":opt-linux-latest.yml"
>> +  ISAR_IMAGE: demo-image
>> +  ISAR_DISTRIBUTION: xenomai-demo
>> +
>> +default:
>> +  image: kasproject/kas-isar:latest
>> +
>> +# add lavacli to the container and install the ssh keys
>> +# for the test infrastructure
>> +.add-lava-ssh-config:
>> +  before_script:
>> +    - mkdir -p -m=700 ~/.ssh
>> +    - if [ -n "$https_proxy" ]; then echo "ProxyCommand socat - 
>> PROXY:$(echo $https_proxy | sed 
>> 's|.*://\([^:]*\).*|\1|'):%h:%p,proxyport=$(echo $https_proxy | sed 
>> 's|.*:\([0-9]*\)$|\1|')" >> ~/.ssh/config && chmod 600 ~/.ssh/config; fi;
>> +    - echo "$LAVA_SSH_UPLOAD_KEY" | tr -d '\r' > ~/.ssh/lava_id_rsa 
>> && chmod 600 ~/.ssh/lava_id_rsa
>> +    - echo "$LAVA_SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts && chmod 644 
>> ~/.ssh/known_hosts
>> +
>> +.test:
>> +  extends: .add-lava-ssh-config
>> +  stage: lava-test
>> +  script:
>> +    - scripts/install-lavacli.sh
>> +    - export BUILD_JOB_ID=$(scripts/get-build-id.sh ${BUILD_JOB_NAME})
>> +    - scripts/run-lava-tests.sh ${TARGET}
>> +  variables:
>> +    BUILD_JOB_NAME: "build:board-${TARGET}${TARGET_EXTENSION}"
>> +  only:
>> +    variables:
>> +      - $LAVA_SSH_USER
>> +
>> +
>> +build:board-qemu-amd64:
>> +  extends: .build
>> +  variables:
>> +    TARGET: qemu-amd64
>> +
>> +lava-test:qemu-amd64:
>> +  needs: [ "build:board-qemu-amd64" ]
>> +  dependencies: [ "build:board-qemu-amd64" ]
>> +  extends: .test
>> +  variables:
>> +    TARGET: qemu-amd64
>> +
>> +build:board-qemu-armhf:
>> +  extends: .build
>> +  variables:
>> +    TARGET: qemu-armhf
>> +
>> +lava-test:qemu-armhf:
>> +  needs: [ "build:board-qemu-armhf" ]
>> +  extends: .test
>> +  variables:
>> +    TARGET: qemu-armhf
>> +
>> +build:board-qemu-arm64:
>> +  extends: .build
>> +  variables:
>> +    TARGET: qemu-arm64
>> +
>> +lava-test:qemu-arm64:
>> +  needs: [ "build:board-qemu-arm64" ]
>> +  extends: .test
>> +  variables:
>> +    TARGET: qemu-arm64
>> +
>> +build:board-hikey:
>> +  extends: .build
>> +  variables:
>> +    TARGET: hikey
>> +    BUILD_OPTIONS: ":opt-lava-test.yml"
>> +
>> +lava-test:board-hikey:
>> +  needs: [ "build:board-hikey" ]
>> +  extends: .test
>> +  variables:
>> +    TARGET: hikey
>> +
>> +build:board-beagle-bone-black:
>> +  extends: .build
>> +  variables:
>> +    TARGET: beagle-bone-black
>> +    BUILD_OPTIONS: ":opt-lava-test.yml"
>> +
>> +lava-test:board-beagle-bone-black:
>> +  needs: [ "build:board-beagle-bone-black" ]
>> +  extends: .test
>> +  variables:
>> +    TARGET: beagle-bone-black
>> +
>> +build:board-x86-64-efi:
>> +  extends: .build
>> +  variables:
>> +    TARGET: x86-64-efi
>> +    BUILD_OPTIONS: ":opt-lava-test.yml"
>> +
>> +lava-test:board-x86-64-efi:
>> +  needs: [ "build:board-x86-64-efi" ]
>> +  extends: .test
>> +  variables:
>> +    TARGET: x86-64-efi
>> +
>> +build:board-beagle-bone-black:xenomai-3.0.x:
>> +  extends: .build
>> +  variables:
>> +    TARGET: beagle-bone-black
>> +    XENOMAI_BUILD_OPTION: ":opt-xenomai-3.0.x.yml"
>> +    BUILD_OPTIONS: ":opt-lava-test.yml"
>> +    DEPLOY_DIR_EXTENSION: "xenomai-3.0.x"
>> +
>> +lava-test:board-beagle-bone-black:xenomai-3.0.x:
>> +  needs: [ "build:board-beagle-bone-black:xenomai-3.0.x" ]
>> +  extends: .test
>> +  variables:
>> +    TARGET: beagle-bone-black
>> +    DEPLOY_DIR_EXTENSION: "xenomai-3.0.x"
>> +    TARGET_EXTENSION: ":xenomai-3.0.x"
>> +
>> +build:board-x86-64-efi:xenomai-3.0.x:
>> +  extends: .build
>> +  variables:
>> +    TARGET: x86-64-efi
>> +    XENOMAI_BUILD_OPTION: ":opt-xenomai-3.0.x.yml"
>> +    BUILD_OPTIONS: ":opt-lava-test.yml"
>> +    DEPLOY_DIR_EXTENSION: "xenomai-3.0.x"
>> +
>> +lava-test:board-x86-64-efi:xenomai-3.0.x:
>> +  needs: [ "build:board-x86-64-efi:xenomai-3.0.x" ]
>> +  extends: .test
>> +  variables:
>> +    TARGET: x86-64-efi
>> +    DEPLOY_DIR_EXTENSION: "xenomai-3.0.x"
>> +    TARGET_EXTENSION: ":xenomai-3.0.x"
>> +
>> diff --git a/.ci/gitlab-com.yml b/.ci/gitlab-com.yml
>> new file mode 100644
>> index 0000000..0abb3fb
>> --- /dev/null
>> +++ b/.ci/gitlab-com.yml
>> @@ -0,0 +1,26 @@
>> +#
>> +# Xenomai Real-Time System
>> +#
>> +# Copyright (c) Siemens AG, 2019 - 2020
>> +#
>> +# Authors:
>> +#  Quirin Gylstorff <quirin.gylstorff at siemens.com>
>> +#
>> +# SPDX-License-Identifier: MIT
>> +#
>> +.build:
>> +  extends: .add-lava-ssh-config
>> +  stage: build
>> +  script:
>> +    - kas build 
>> kas.yml:board-${TARGET}.yml${XENOMAI_BUILD_OPTION}${LINUX_BUILD_OPTION}${BUILD_OPTIONS} 
>>
>> +  artifacts:
>> +    paths:
>> +    - build/tmp/deploy/images/${TARGET}/
> 
> Can't we put that path completely into a variable and leave that empty 
> for CI setups that do not support (large) artifacts?
> 
> Jan


I tried that and it leads to an invalid gitlab-ci.yml. This is kind of a 
workaround to work with artifacts. If we use another storage backend 
(aws) with our own upload scripts we can use your sugestion.

Quirin
> 
>> +    expire_in: 1 week
>> +  only:
>> +    variables:
>> +      - $USE_GITLAB_ARTIFACTS
>> +include:
>> +  - local: '/.ci/gitlab-ci-base.yml'
>> +
>> +
> 

-- 
Quirin Gylstorff

Siemens AG
Corporate Technology
Research in Digitalization and Automation
Smart Embedded Systems
CT RDA IOT SES-DE
Otto-Hahn-Ring 6
81739 Muenchen, Germany
Mobile: +49 173 3746683
mailto:quirin.gylstorff at siemens.com
www.siemens.com/ingenuityforlife

Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Jim 
Hagemann Snabe; Managing Board: Joe Kaeser, Chairman, President and 
Chief Executive Officer; Roland Busch, Lisa Davis, Klaus Helmrich, 
Cedrik Neike, Michael Sen, Ralf P. Thomas; Registered offices: Berlin 
and Munich, Germany; Commercial registries: Berlin Charlottenburg, HRB 
12300, Munich, HRB 6684; WEEE-Reg.-No. DE 23691322

Important notice: This e-mail and any attachment thereof contain 
corporate proprietary information. If you have received it by mistake, 
please notify us immediately by reply e-mail and delete this e-mail and 
its attachments from your system. Thank you.



More information about the Xenomai mailing list