civilinfrastructureplatform:ciptesting
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
civilinfrastructureplatform:ciptesting [2017/05/24 09:50] Agustin Benito Bethencourt changed links to gitlab |
civilinfrastructureplatform:ciptesting [2018/04/27 13:17] (current) rajmarshall remove irrelevant text |
||
|---|---|---|---|
| Line 14: | Line 14: | ||
| - Define kernel testing as a service within CIP: define the testing environment within CIP assuming that in some cases, Members would share the tests, test results and laboratory and in others they will not. | - Define kernel testing as a service within CIP: define the testing environment within CIP assuming that in some cases, Members would share the tests, test results and laboratory and in others they will not. | ||
| - | For more details, please check the [[ciptestingfaq|CIP Testing FAQ]] | + | For more details, please check the [[:civilinfrastructureplatform/ciptestingfaq|CIP Testing FAQ]]. If you are interesting in following up how the project is progressing, please check the [[:civilinfrastructureplatform/ciptesting/management|CIP Testing Project Management page]]. |
| ==== Action 1: Board At Desk - Single Developer (B@D) ==== | ==== Action 1: Board At Desk - Single Developer (B@D) ==== | ||
| - | [[http://www.kernelci.org|kernelci.org]] is one of the most sucessful Open Source testing projects. Its distributed nature improve the possibilities of collaborating among different developers and corporations. Distributed boards farms across the world are working together to test every merge of an upstream Linux kernel tree, detecting bugs and regressions in a timely manner and reporting them back to kernel developers. | + | [[http://www.kernelci.org|kernelci.org]] is one of the most successful Open Source testing projects. Its distributed nature improve the possibilities of collaborating among different developers and corporations. Distributed boards farms across the world are working together to test every merge of an upstream Linux kernel tree, detecting bugs and regressions in a timely manner and reporting them back to kernel developers. |
| Board At Desk - Single Developer, given name for action 1, has as goal to create and publish a VM that contains all the kernelci tools so a single developer can test the CIP kernel (or any other Linux kernel) on a board connected to the user's development machine. This board will initially be the selected hardware platforms for the CIP project which at the moment is Beaglebone Black. Any other board would work too after some basic configuration. {{ :civilinfrastructureplatform:board_at_desk_single_dev_block_diagram_low_quality.jpg?500|Board At Desk - Single Dev. Schema}} | Board At Desk - Single Developer, given name for action 1, has as goal to create and publish a VM that contains all the kernelci tools so a single developer can test the CIP kernel (or any other Linux kernel) on a board connected to the user's development machine. This board will initially be the selected hardware platforms for the CIP project which at the moment is Beaglebone Black. Any other board would work too after some basic configuration. {{ :civilinfrastructureplatform:board_at_desk_single_dev_block_diagram_low_quality.jpg?500|Board At Desk - Single Dev. Schema}} | ||
| Line 27: | Line 27: | ||
| * The wiki pages that will guide you through the process of using B@D are: | * The wiki pages that will guide you through the process of using B@D are: | ||
| - Instructions to deploy and configure B@D, connect the board and try the reference tests. | - Instructions to deploy and configure B@D, connect the board and try the reference tests. | ||
| - | - Deploy B@D through [[ciptestingboardatdeskdingledevdeployment#b-d-deployment-method-2-building-vm-from-scratch-using-vagrant-14|Vagrant]] or by downloading the [[ciptestingboardatdeskdingledevdeployment#b-d-standalone-virtual-machine-box|B@D standalone VM box]] directly. | + | - Deploy B@D through [[ciptestingboardatdeskdingledevdeployment#b-d-deployment-method-2-building-vm-from-scratch-using-vagrant-15|Vagrant]] or by downloading the [[ciptestingboardatdeskdingledevdeployment#b-d-standalone-virtual-machine-box|B@D standalone VM box]] directly. |
| - Once deployed, [[[[ciptestingboardatdesksingledevsetup|configure B@D]]. | - Once deployed, [[[[ciptestingboardatdesksingledevsetup|configure B@D]]. | ||
| - | - Connect the board ([[beagleboneblackboard|Beaglebone Black]] in this case) to the host machine, where B@D is deployed. | + | - Connect the board ([[beagleboneblackboard|Beaglebone Black]] or [[renesasboard|Renesas IWG20M]]) to the host machine, where B@D is deployed. |
| + | - [[cipsystembuildhowto|Build the kernel (system)]] to be tested and required artefacts. | ||
| - Check the [[civilinfrastructureplatform/ciptestingreferencetestcases|reference test cases]] and try them as examples. | - Check the [[civilinfrastructureplatform/ciptestingreferencetestcases|reference test cases]] and try them as examples. | ||
| - If you experience any problem following the above instructions, please check the [[ciptestingknownissues|Known Issues and Workarounds]] wiki page before reporting any issue. | - If you experience any problem following the above instructions, please check the [[ciptestingknownissues|Known Issues and Workarounds]] wiki page before reporting any issue. | ||
| - | * Contribute to the CIP testing project | + | * Do you want to know more about the release process followed with the CIP Testing project of tools and code? You should take a look at the following pages then: |
| - | * Check the code at [[https://gitlab.com/cip-project/board-at-desk-single-dev/tree/master|Gitlab.com]] | + | * Check the CIP Testing project [[:civilinfrastructureplatform/ciptesting/management|management wiki page]] to get a description of the process followed and links to other related content. |
| + | * Interested in previous releases of B@D? Please check the [[/civilinfrastructureplatform/ciptesting/releases|CIP Testing Project Releases]] wiki page. | ||
| + | * All the marketing related material created for every release is published on the CIP Testing Project [[/civilinfrastructureplatform/ciptesting/batdmarketing|marketing page]]. | ||
| + | |||
| + | === Contribute to the CIP testing project === | ||
| + | |||
| + | If you want to contribute to the CIP Testing project, please consider reading the following information: | ||
| + | * Check the code at [[https://gitlab.com/cip-project/cip-testing/board-at-desk-single-dev/tree/master|Gitlab.com]] | ||
| * Please read the [[ciptestingfaq|CIP Testing FAQ]] to learn more about B@D. | * Please read the [[ciptestingfaq|CIP Testing FAQ]] to learn more about B@D. | ||
| - | * The delivery process is described in the [[ciptestingmanagement|CIP testing management]] wiki page. | + | * The delivery process is described in the [[/civilinfrastructureplatform/ciptesting/management|CIP testing management]] wiki page. |
| * Join the technical mailing list ([[https://lists.cip-project.org/mailman/listinfo/cip-dev|cip-dev]]) to follow this effort. | * Join the technical mailing list ([[https://lists.cip-project.org/mailman/listinfo/cip-dev|cip-dev]]) to follow this effort. | ||
| - | * Report a [[https://gitlab.com/cip-project/testing/boards|bug]]. | + | * Report a [[https://gitlab.com/cip-project/cip-testing/testing/boards/322796?&label_name[]=bug|bug]]. |
| + | * Any contributions for the CIP Testing project need to have licences that are compatible with the existing licences (APGL-3.0), source files must have a licence header and any repositories must have a LICENSE file and a LICENSE.spdx file - see the current repositories for style and see also [[https://commonsconservancy.org/faq/licenseinfo/|the commons conservancy]] guidance. | ||
| ==== Action 2: CIP kernel testing ==== | ==== Action 2: CIP kernel testing ==== | ||
| - | Once the VM with the needed tools has been finished, it is time to focus on CIP use case. The goals of this Action 2 are the following: | + | Now that the VM with the needed tools has been finished, it is time to focus on CIP use case. The goals of this Action 2 are the following: |
| - Test CIP kernel in a transparent way so the test environment, the tests themselves, the results and the logs got from each test are reproducible by anyone at any given time as well as traceable. . | - Test CIP kernel in a transparent way so the test environment, the tests themselves, the results and the logs got from each test are reproducible by anyone at any given time as well as traceable. . | ||
| Line 52: | Line 61: | ||
| The most effective and scalable way to achieve this goal is by creating a testing service in a similar way that the kernel does through KernelCI which is creating a public testing service. Even more effective is if the tools used by the public service can be downloaded and easily configured by any contributor in order to replicate the tests and test results in his own environment, like openSUSE does with openQA. | The most effective and scalable way to achieve this goal is by creating a testing service in a similar way that the kernel does through KernelCI which is creating a public testing service. Even more effective is if the tools used by the public service can be downloaded and easily configured by any contributor in order to replicate the tests and test results in his own environment, like openSUSE does with openQA. | ||
| - | As a first step on the journey towards transparency, CIP will create an environment in which the tools, the configuration files, the tests, the kernel to be tested and the outcome of the tests are public under GPL based licenses so anyone can reproduce the outcome of the tests performed, as well as the environment. All those inputs will also be signed so they can be verified at any stage of the testing process and life time of the CIP kernel. | + | As a first step on the journey towards transparency, CIP will create an environment in which the tools, the configuration files, the tests, the kernel to be tested and the outcome of the tests are public under Open Source licenses so anyone can reproduce the outcome of the tests performed, as well as the environment. All those inputs will also be signed so they can be verified at any stage of the testing process and life time of the CIP kernel. |
| === 2.- Testing performed by any developer at any time === | === 2.- Testing performed by any developer at any time === | ||
| Line 60: | Line 69: | ||
| === 3.- Share inputs, outputs and configurations === | === 3.- Share inputs, outputs and configurations === | ||
| - | All inputs, configurations and outputs, including metadata and environment variables will be shared through publicly accessible repositories or mailing lists. Based on the current LAVA and KernelCI licenses, AGPLv3, CIP will adopt the spirit of the tooling licenses and use GPL based licenses. | + | All inputs, configurations and outputs, including metadata and environment variables will be shared through publicly accessible repositories or mailing lists. |
| === 4.- Increase CIP kernel test coverage === | === 4.- Increase CIP kernel test coverage === | ||
| Line 66: | Line 75: | ||
| The main goal for CIP testing project when it comes to the kernel is to detect potential regressions and security issues in those backported patches applied to our current CIP kernel. An outcome of this action should be to select the best tests available and add a few more basic ones. At some point, early in the process, no patch can be merged without associated tests. | The main goal for CIP testing project when it comes to the kernel is to detect potential regressions and security issues in those backported patches applied to our current CIP kernel. An outcome of this action should be to select the best tests available and add a few more basic ones. At some point, early in the process, no patch can be merged without associated tests. | ||
| - | ==== Action 3: Define kernel testing as a service within CIP ==== | + | ==== Action 3: Extend the number of technologies used to deploy B@D ==== |
| - | Once the CIP kernel is being tested on regular basis, it will be time to improve the tooling and extend it to new technologies specially on the deployment side, to reach more potential users. | + | In order to reach the maximum number of developers, CIP testing team plans to enhance the deployment adding additional virtualisation technologies like KVM. Using containers will also be evaluated. |
| ==== Action 4: From kernel testing to system testing ==== | ==== Action 4: From kernel testing to system testing ==== | ||
| Line 74: | Line 83: | ||
| CIP is not just about the kernel. The goal is to create a base system for the industry. In order to achieve the general goal, this initiative will require more of a journey than the one taken for the kernel, but for a base/core system. | CIP is not just about the kernel. The goal is to create a base system for the industry. In order to achieve the general goal, this initiative will require more of a journey than the one taken for the kernel, but for a base/core system. | ||
| - | ==== Action 5: Define kernel testing as a service within CIP ==== | + | ==== Action 5: Define kernel/system testing as a semi-distributed service within CIP ==== |
| Once CIP develops a testing culture, it will be time for scaling up the testing activities by turning the project into a service for Members and CIP friends, in collaboration with the kernel community. | Once CIP develops a testing culture, it will be time for scaling up the testing activities by turning the project into a service for Members and CIP friends, in collaboration with the kernel community. | ||
| [[start|Back to the parent page]] | [[start|Back to the parent page]] | ||
civilinfrastructureplatform/ciptesting.1495619457.txt.gz · Last modified: 2017/05/24 09:50 by Agustin Benito Bethencourt
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Noncommercial-Share Alike 4.0 International
