This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
civilinfrastructureplatform:ciptesting:centalisedtesting [2019/03/20 12:23] patersonc [Test Scheduler] Add a link to howto page. |
civilinfrastructureplatform:ciptesting:centalisedtesting [2023/12/19 08:55] (current) swvanbuuren Add SQUAD watch job workflow |
||
---|---|---|---|
Line 2: | Line 2: | ||
The first goal in CIP testing was the creation of the [[civilinfrastructureplatform:ciptesting#action_1board_at_desk_-_single_developer_b_d|B@D]] (Board at Desk) virtual machine. This allows developers to easily run automated testing on a local Beaglebone Black or Renesas RZ/G1M iwg20m platform. | The first goal in CIP testing was the creation of the [[civilinfrastructureplatform:ciptesting#action_1board_at_desk_-_single_developer_b_d|B@D]] (Board at Desk) virtual machine. This allows developers to easily run automated testing on a local Beaglebone Black or Renesas RZ/G1M iwg20m platform. | ||
- | The next goal for CIP is to centralise testing, so that developers can run tests without having local access to a platform. This will become more and more useful for CIP as the list of [[civilinfrastructureplatform:ciptesting:cipreferencehardware|reference platforms]] grows. It also means that tests can be triggered and run automatically 24/7. | + | The second goal for CIP testing was to centralise testing so that developers can run tests without having local access to a platform. This is becoming more and more useful for CIP as the list of [[civilinfrastructureplatform:ciptesting:cipreferencehardware|reference platforms]] grows. It also means that tests can be triggered and run automatically 24/7. |
+ | The third goal for CIP testing was to set up Continuous Integration (CI) testing to automatically test CIP software on CIP hardware. See the [[civilinfrastructureplatform:ciptesting:centalisedtesting:cioverview|Continuous Integration testing overview]] page for more details. | ||
+ | |||
+ | The fourth goal for CIP testing is to integrate with the [[https://kernelci.org/|KernelCI project]]. This work is underway and the latest results can be viewed at [[https://kernelci.ciplatform.org/|kernelci.ciplatform.org]]. | ||
===== CIP Testing Architecture ===== | ===== CIP Testing Architecture ===== | ||
The block diagram below provides an overview of CIP's centralised test infrastructure. | The block diagram below provides an overview of CIP's centralised test infrastructure. | ||
- | {{ :civilinfrastructureplatform:ciptesting:20190318_cip_testing_overview.png |}} | + | {{ :civilinfrastructureplatform:ciptesting:20190807_cip_testing_overview.png |}} |
Explanations for each block are below. | Explanations for each block are below. | ||
Line 13: | Line 16: | ||
This schedules submitted test jobs to run on available hardware targets. | This schedules submitted test jobs to run on available hardware targets. | ||
- | CIP have set up their own instance of LAVA (Linaro Automated Validation Architecture). LAVA is a continuous integration system for deploying operating systems onto physical and virtual hardware for running tests. See the [[https://www.lavasoftware.org/|LAVA website]] for more details. | + | CIP has set up their own instance of LAVA (Linaro Automated Validation Architecture). LAVA is a continuous integration system for deploying operating systems onto physical and virtual hardware for running tests. See the [[https://www.lavasoftware.org/|LAVA website]] for more details. |
- | CIP are using a version of LAVA maintained by the [[https://kernelci.org/|KernelCI project]] which [[https://github.com/kernelci/lava-docker/|containerises]] support in a Docker image. | + | CIP is using a version of LAVA maintained by the [[https://kernelci.org/|KernelCI project]] which [[https://github.com/kernelci/lava-docker/|containerises]] support in a Docker image. |
=== Links === | === Links === | ||
* CIP LAVA master website: https://lava.ciplatform.org | * CIP LAVA master website: https://lava.ciplatform.org | ||
+ | * CIP LAVA documentation: [[civilinfrastructureplatform:ciptesting:centalisedtesting:ciplava|CIP LAVA Wiki]] | ||
* cip-lava-docker source code: https://gitlab.com/cip-project/cip-testing/lava-docker | * cip-lava-docker source code: https://gitlab.com/cip-project/cip-testing/lava-docker | ||
* How to submit a LAVA test job: [[civilinfrastructureplatform:ciptesting:centalisedtesting:submittinglavajobs|Submitting a LAVA test job]] | * How to submit a LAVA test job: [[civilinfrastructureplatform:ciptesting:centalisedtesting:submittinglavajobs|Submitting a LAVA test job]] | ||
+ | |||
+ | ==== Test scheduler workflow ==== | ||
+ | |||
+ | The following schematic shows the workflow, that describes how test images are built and tested with LAVA. | ||
+ | |||
+ | {{:civilinfrastructureplatform:ciptesting:cip-testing-6.1-workflow.png?800 |}} | ||
+ | |||
+ | === Links === | ||
+ | |||
+ | * CIP Testing group: https://gitlab.com/cip-project/cip-testing | ||
+ | * [[https://gitlab.com/cip-project/cip-testing/linux-cip-ci|linux-cip-ci]] | ||
+ | * [[https://gitlab.com/cip-project/cip-testing/linux-cip-pipelines|linux-cip-pipelines]] | ||
+ | * CIP Kernel group: https://gitlab.com/cip-project/cip-kernel | ||
+ | * [[https://gitlab.com/cip-project/cip-kernel/linux-cip|linux-cip]] | ||
==== Target Control ==== | ==== Target Control ==== | ||
A LAVA worker (or lab) manages target (virtual/physical hardware) control. There can be one or many different LAVA workers. | A LAVA worker (or lab) manages target (virtual/physical hardware) control. There can be one or many different LAVA workers. | ||
- | Currently CIP has one LAVA worker (lab-cip-renesas) based in the Renesas UK office. Targets attached to the worker can be viewed on the [[https://lava.ciplatform.org/scheduler/worker/lab-cip-renesas|master website]]. | + | Currently CIP has three LAVA workers. A summary of all of the attached targets can be viewed on the [[https://lava.ciplatform.org/scheduler/alldevices|LAVA master]]. |
CIP encourage as many LAVA workers to be set up as possible and have provided a [[https://gitlab.com/cip-project/cip-testing/lava-docker/blob/cip-lava-docker/documentation/setup_cip_lava_lab.md|guide]] on how to do this. | CIP encourage as many LAVA workers to be set up as possible and have provided a [[https://gitlab.com/cip-project/cip-testing/lava-docker/blob/cip-lava-docker/documentation/setup_cip_lava_lab.md|guide]] on how to do this. | ||
=== Links === | === Links === | ||
- | * lab-cip-renesas: https://lava.ciplatform.org/scheduler/worker/lab-cip-renesas | + | * lab-cip-cybertrust: [[https://lava.ciplatform.org/scheduler/worker/lab-cip-cybertrust]] |
+ | * lab-cip-mentor: [[https://lava.ciplatform.org/scheduler/worker/lab-cip-mentor]] | ||
+ | * lab-cip-renesas: [[https://lava.ciplatform.org/scheduler/worker/lab-cip-renesas]] | ||
* CIP worker setup guide: https://gitlab.com/cip-project/cip-testing/lava-docker/blob/cip-lava-docker/documentation/setup_cip_lava_lab.md | * CIP worker setup guide: https://gitlab.com/cip-project/cip-testing/lava-docker/blob/cip-lava-docker/documentation/setup_cip_lava_lab.md | ||
Line 54: | Line 74: | ||
KernelCI is a widely used project that aims to unify all upstream Linux kernel testing efforts in order to provide a single place where to store, view, compare and track these results. | KernelCI is a widely used project that aims to unify all upstream Linux kernel testing efforts in order to provide a single place where to store, view, compare and track these results. | ||
- | At the moment CIP has their own instance of the [[https://kernelci.ciplatform.org/|KernelCI website]] where test results can be tracked. | + | CIP plans to submit our test results to kernelci.org so that they can be viewed by all. |
== Mailing list == | == Mailing list == | ||
Line 60: | Line 80: | ||
LAVA can also email any email address specified in the job description. | LAVA can also email any email address specified in the job description. | ||
+ | |||
+ | == SQUAD == | ||
+ | |||
+ | For detailed test reporting CIP uses the Software Quality Dashboard (SQUAD). Build success and test results are reported to the [[https://squad.ciplatform.org/|CIP SQUAD]]. | ||
+ | |||
+ | The following diagram shows how build results are reported to SQUAD and test jobs are submitted to LAVA and monitored by SQUAD. | ||
+ | |||
+ | {{:civilinfrastructureplatform:ciptesting:cip-testing-with-squad.png?400|}} | ||
=== Links === | === Links === | ||
Line 65: | Line 93: | ||
* CIP KernelCI instance: https://kernelci.ciplatform.org | * CIP KernelCI instance: https://kernelci.ciplatform.org | ||
* CIP test results mailing list: https://lists.cip-project.org/mailman/listinfo/cip-testing-results | * CIP test results mailing list: https://lists.cip-project.org/mailman/listinfo/cip-testing-results | ||
+ | * CIP SQUAD Reporting: https://squad.ciplatform.org/ | ||
==== Artifact Storage ==== | ==== Artifact Storage ==== | ||
Line 121: | Line 150: | ||
=== Links === | === Links === | ||
* Mainline Linux Kernel: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git | * Mainline Linux Kernel: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git | ||
- | * LTS Kernel repository: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git | + | * Stable Kernel repository: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git |
+ | * Stable Kernel release candidate repository: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git/ | ||
* Real Time Kernel repository: https://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git | * Real Time Kernel repository: https://git.kernel.org/pub/scm/linux/kernel/git/rt/linux-rt-devel.git | ||