Differences

This shows you the differences between two versions of the page.

--- civilinfrastructureplatform:ciptestingreferencetestcases [2018/05/16 13:19]
rajmarshall [Procedure:] update results for new LAVA version
+++ civilinfrastructureplatform:ciptestingreferencetestcases [2018/11/08 13:24] (current)
rajmarshall add nowiki to the lavacli command so that '--'s appear correctly
@@ Line 1: / Line 1: @@
 ====== B@D Reference Test Cases (5/5)======
-These instructions provide some Example Tests and Test Cases you can use to validate your CIP Board-at-Desk Single-Developer (B@D) v1.0 Virtual Machine (VM) or from which, you can learn how all of the tooling works together.
+These instructions provide some Example Tests and Test Cases you can use to validate your CIP Board-at-Desk Single-Developer (B@D) Virtual Machine (VM) or from which, you can learn how all of the tooling works together.
 This set of instructions assumes you have already connected the board to the machine where you have installed B@D. If that is not the case please go back to the [[beagleboneblackboard|Beaglebone Black Setup & Configuration wiki page]] or [[renesasboard|Renesas IWG20M]] depending on the board you are using, which is step 3.
@@ Line 10: / Line 10: @@
 If you have a kernel that you have built on another machine that you wish to test with B@D please refer to [[ciptestingreferencetestcases#test-6testing-externally-built-kernels|Test 6]].
-// Note that if you are using the latest version of B@D with LAVA 2018-4 (LAVA version is checkable by clicking the Help button and examining the header of that Help page) then see [[ciptestingreferencetestcases#LAVA-2018-4|Version specific information]] //
+// Note that if you are using the latest version of B@D with LAVA 2018-4 or greater (LAVA version is checkable by clicking the Help button and examining the header of that Help page) then see [[ciptestingreferencetestcases#LAVA-2018-4|Version specific information]] //
 ===== Test #1: Running the Health Check for the QEMU Device =====
@@ Line 16: / Line 16: @@
 ==== Prerequisites: ====
-   * A working CIP Board-at-Desk Single-Developer (B@D) v1.0 Virtual Machine
+   * A working CIP Board-at-Desk Single-Developer (B@D) Virtual Machine
 ==== Procedure: ====
@@ Line 96: / Line 96: @@
 ==== Prerequisites: ====
-   * A working CIP Board-at-Desk Single-Developer (B@D) v1.0 Virtual Machine
+   * A working CIP Board-at-Desk Single-Developer (B@D) Virtual Machine
    * A Beaglebone Black that is powered up, connected to the LAN with an ethernet cable and connected to the host machine with an FTDI cable.
    * A locally built CIP kernel and initramfs.
@@ Line 123: / Line 123: @@
 '' vagrant@guest:~/git-repos/linux-cip$ **export TREE_NAME=cip-example** ''
-// Note: This should be the same name you used when you built the kernel. //
+// Note: This must be the same name you used when you built the kernel. //
 . Create a test - from within the ''linux-cip'' git repository.
@@ Line 163: / Line 163: @@
    downloading http://localhost:8010/initramfs/initramfs.cpio.gz
    saving as /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/ramdisk/initramfs.cpio.gz
    <snip>
@@ Line 198: / Line 197: @@
 ==== Prerequisites: ====
-   * A working CIP Board-at-Desk Single-Developer (B@D) v1.0 Virtual Machine
+   * A working CIP Board-at-Desk Single-Developer (B@D) Virtual Machine
    * A Beaglebone Black that is powered up, connected to the LAN with an ethernet cable and connected to the host machine with an FTDI cable.
    * A locally built CIP kernel and initramfs.
@@ Line 238: / Line 237: @@
 This is what the start of the Health Check looks like:
-   Root tmp directory created at /var/lib/lava/dispatcher/tmp/11
+   lava-dispatcher, installed at version: 2018.4-2~bpo9+1
    start: 0 validate
+   Start time: 2018-05-10 12:14:13.326013+00:00 (UTC)
    Validating that http://localhost:8010/initramfs/initramfs.cpio.gz exists
-   Validating that http://localhost:8010/cip-releaseTest/cip_v4.4.83/v4.4.83-cip8/arm/omap2plus_defconfig/zImage exists
+   Validating that http://localhost:8010/cip-test/cip_v4.4.126/v4.4.126-cip22/arm/omap2plus_defconfig/zImage exists
-   Validating that http://localhost:8010/cip-releaseTest/cip_v4.4.83/v4.4.83-cip8/arm/omap2plus_defconfig/dtbs/am335x-boneblack.dtb exists
+   Validating that http://localhost:8010/cip-test/cip_v4.4.126/v4.4.126-cip22/arm/omap2plus_defconfig/dtbs/am335x-boneblack.dtb exists
-   Validating that http://localhost:8010/cip-releaseTest/cip_v4.4.83/v4.4.83-cip8/arm/omap2plus_defconfig/modules.tar.xz exists
+   Validating that http://localhost:8010/cip-test/cip_v4.4.126/v4.4.126-cip22/arm/omap2plus_defconfig/modules.tar.xz exists
-   no device environment specified
+   validate duration: 0.02
-   booti, bootm and bootz are being deprecated soon, please use 'image','uimage' or 'zimage'
+   case: validate
-   device may need manual intervention to reboot
+   case_id: 128
-   validate duration: 3.01
+   definition: lava
-   start: 1 tftp-deploy (max 240s)
+   result: pass
-   start: 1.1 download_retry (max 240s)
+   start: 1 tftp-deploy (timeout 00:04:00) [common]
-   start: 1.1.1 http_download (max 240s)
+   start: 1.1 download-retry (timeout 00:04:00) [common]
+   start: 1.1.1 http-download (timeout 00:04:00) [common]
    Not decompressing ramdisk as can be used compressed.
-   No compression specified.
+   downloading http://localhost:8010/initramfs/initramfs.cpio.gz
-   downloading http://localhost:8010/initramfs/initramfs.cpio.gz as /var/lib/lava/dispatcher/tmp/11/tftp-deploy-Blt0p9/initramfs.cpio.gz
+   saving as /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/ramdisk/initramfs.cpio.gz
    <snip>
@@ Line 259: / Line 260: @@
    </snip>
-   case: finalize
+   Finalising connection for namespace 'common'
+   end: 4.2 read-feedback (duration 00:00:01) [common]
+   end: 4 finalize (duration 00:00:01) [common]
+   Cleaning after the job
+   Cleaning up download directory: /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/ramdisk
+   Cleaning up download directory: /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/kernel
+   Cleaning up download directory: /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/dtb
+   Cleaning up download directory: /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/modules
+   Override tmp directory removed at /var/lib/lava/dispatcher/tmp/11
+   Root tmp directory removed at /var/lib/lava/dispatcher/tmp/11
+   Job finished correctly
+   case: job
+   case_id: 142
    definition: lava
    result: pass
-   level: 4
-   duration: 0.105773925781
-   extra: ...
-   Cleanup: removing /var/lib/lava/dispatcher/tmp/11
 ===== Test #5: Running the Health Check for the Renesas IWG20M Device =====
@@ Line 271: / Line 281: @@
 ==== Prerequisites: ====
-   * A working CIP Board-at-Desk Single-Developer (B@D) v1.0 Virtual Machine
+   * A working CIP Board-at-Desk Single-Developer (B@D) Virtual Machine
    * An IWG20M that is powered up, connected to the LAN with an ethernet cable and connected to the host machine with a USB cable.
    * A locally built CIP kernel and initramfs.
-   * A Remote Power on Switcher - we have tested with... // - need connection info in the board page//
+   * A Remote Power on Switcher - see [[renesasboard|Renesas IWG20M]] for these details.
 ==== Procedure: ====
@@ Line 299: / Line 309: @@
 '' vagrant@guest:~/git-repos/linux-cip$ **export TREE_NAME=cip-example** ''
-// Note: This should be the same name you used when you built the kernel. //
+// Note: This must be the same name you used when you built the kernel. //
 . Create a test - from within the ''linux-cip'' git repository.
@@ Line 321: / Line 331: @@
 This is what the start of the Health Check looks like:
-   // insert log section //
+   lava-dispatcher, installed at version: 2018.4-2~bpo9+1
+   start: 0 validate
+   Start time: 2018-05-17 12:20:24.171357+00:00 (UTC)
+   Validating that http://localhost:8010/initramfs/initramfs.cpio.gz exists
+   Validating that http://localhost:8010/cip-checks/cip_v4.4.126/v4.4.126-cip22/arm/shmobile_defconfig/zImage exists
+   Validating that http://localhost:8010/cip-checks/cip_v4.4.126/v4.4.126-cip22/arm/shmobile_defconfig/dtbs/r8a7743-iwg20d-q7.dtb exists
+   Specifying a type in the boot action is deprecated. Please specify the kernel type in the deploy parameters.
+   validate duration: 0.02
+   case: validate
+   case_id: 114
+   definition: lava
+   result: pass
+   start: 1 tftp-deploy (timeout 00:05:00) [common]
+   start: 1.1 download-retry (timeout 00:05:00) [common]
+   start: 1.1.1 http-download (timeout 00:05:00) [common]
+   Not decompressing ramdisk as can be used compressed.
+   downloading http://localhost:8010/initramfs/initramfs.cpio.gz
+   saving as /var/lib/lava/dispatcher/tmp/9/tftp-deploy-ge8cmxso/ramdisk/initramfs.cpio.gz
+   total size: 5813656 (5MB)
    <snip>
@@ Line 327: / Line 355: @@
    </snip>
-   case: finalize
+   Finalising connection for namespace 'common'
+   end: 4.2 read-feedback (duration 00:00:01) [common]
+   end: 4 finalize (duration 00:00:01) [common]
+   Cleaning after the job
+   Cleaning up download directory: /var/lib/lava/dispatcher/tmp/9/tftp-deploy-ge8cmxso/ramdisk
+   Cleaning up download directory: /var/lib/lava/dispatcher/tmp/9/tftp-deploy-ge8cmxso/kernel
+   Cleaning up download directory: /var/lib/lava/dispatcher/tmp/9/tftp-deploy-ge8cmxso/dtb
+   Override tmp directory removed at /var/lib/lava/dispatcher/tmp/9
+   Root tmp directory removed at /var/lib/lava/dispatcher/tmp/9
+   Job finished correctly
+   case: job
+   case_id: 127
    definition: lava
    result: pass
-   level: 4
-   extra: ...
-   Cleanup: removing /var/lib/lava/dispatcher/tmp/22
-. You can see an example of a successful test log file [[ link | the Renesas IWG20M Test results ]].
+. You can see an example of a successful test log file [[ https://gitlab.com/snippets/1716407 | the Renesas IWG20M Test results ]].
 ----
 ===== Test #6: Use lava-tool utility =====
+Note that lava-tool is deprecated so, in preference, set up and use lavacli see [[ciptestingreferencetestcases#test-7use-lavacli-utility|lavacli instructions]].
 ==== Prerequisites: ====
-   * A working CIP Board-at-Desk Single-Developer (B@D) v1.0 Virtual Machine
+   * A working CIP Board-at-Desk Single-Developer (B@D) Virtual Machine
-   * A Beaglebone Black that is powered up, connected to the LAN with an ethernet cable and connected to the host machine with an FTDI cable.
+   * A Beaglebone Black or a Renesas IWG20M board that is powered up, connected to the LAN with an ethernet cable and connected to the host machine with an FTDI cable.
-   * A built kernel for the Beaglebone Black (see Test #3)
+   * A built kernel for the appropriate board (see Test #3 or #5 depending upon your board)
-   * The Beaglebone Black has a current successful health check (it is marked as online in LAVA)
+   * The board has a current successful health check (it is marked as online in LAVA)
 ==== Procedure: ====
@@ Line 358: / Line 397: @@
     In the LAVA web interface, click on API->Authentication Tokens and then on the 'new' button,
     give it a description and then click the 'View Token Hash' button and copy the displayed hash value.
-    Then ''lava-tool auth-add http://lavauser@localhost:8080/RPC2
+    Then:
+'' vagrant@guest:~/git-repos/linux-cip$ **lava-tool auth-add http://lavauser@localhost:8080/RPC2** ''
     Paste token for http://lavauser@localhost:8080/RPC2/: **paste token - which will not be echoed**
@@ Line 367: / Line 407: @@
 '' vagrant@guest:~/git-repos/linux-cip$ **export TREE_NAME=cip-example** ''
-// Note: This should be the same name you used when you built the kernel. //
+// Note: This must be the same name you used when you built the kernel. //
 . Create a test for use with the ''lava-tool'' utility from within the ''linux-cip'' git repository.
@@ Line 398: / Line 438: @@
 This is an example what the start of the test looks like in the Beaglebone case:
-   Root tmp directory created at /var/lib/lava/dispatcher/tmp/12
+   lava-dispatcher, installed at version: 2018.4-2~bpo9+1
    start: 0 validate
+   Start time: 2018-05-10 12:14:13.326013+00:00 (UTC)
    Validating that http://localhost:8010/initramfs/initramfs.cpio.gz exists
-   Validating that http://localhost:8010/cip-example/cip_v4.4.83/v4.4.83-cip8/arm/omap2plus_defconfig/zImage exists
+   Validating that http://localhost:8010/cip-test/cip_v4.4.126/v4.4.126-cip22/arm/omap2plus_defconfig/zImage exists
-   Validating that http://localhost:8010/cip-example/cip_v4.4.83/v4.4.83-cip8/arm/omap2plus_defconfig/dtbs/am335x-boneblack.dtb exists
+   Validating that http://localhost:8010/cip-test/cip_v4.4.126/v4.4.126-cip22/arm/omap2plus_defconfig/dtbs/am335x-boneblack.dtb exists
-   Validating that http://localhost:8010/cip-example/cip_v4.4.83/v4.4.83-cip8/arm/omap2plus_defconfig/modules.tar.xz exists
+   Validating that http://localhost:8010/cip-test/cip_v4.4.126/v4.4.126-cip22/arm/omap2plus_defconfig/modules.tar.xz exists
-   no device environment specified
+   validate duration: 0.02
-   booti, bootm and bootz are being deprecated soon, please use 'image','uimage' or 'zimage'
+   case: validate
-   device may need manual intervention to reboot
+   case_id: 128
-   validate duration: 1.07
+   definition: lava
-   start: 1 tftp-deploy (max 240s)
+   result: pass
-   start: 1.1 download_retry (max 240s)
+   start: 1 tftp-deploy (timeout 00:04:00) [common]
-   start: 1.1.1 http_download (max 240s)
+   start: 1.1 download-retry (timeout 00:04:00) [common]
+   start: 1.1.1 http-download (timeout 00:04:00) [common]
    Not decompressing ramdisk as can be used compressed.
-   No compression specified.
+   downloading http://localhost:8010/initramfs/initramfs.cpio.gz
-   downloading http://localhost:8010/initramfs/initramfs.cpio.gz as /var/lib/lava/dispatcher/tmp/11/tftp-deploy-iZGX26/initramfs.cpio.gz
+   saving as /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/ramdisk/initramfs.cpio.gz
    <snip>
@@ Line 419: / Line 461: @@
    </snip>
-   finalize duration: 0.10
+   Finalising connection for namespace 'common'
-   case: finalize
+   end: 4.2 read-feedback (duration 00:00:01) [common]
+   end: 4 finalize (duration 00:00:01) [common]
+   Cleaning after the job
+   Cleaning up download directory: /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/ramdisk
+   Cleaning up download directory: /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/kernel
+   Cleaning up download directory: /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/dtb
+   Cleaning up download directory: /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/modules
+   Override tmp directory removed at /var/lib/lava/dispatcher/tmp/11
+   Root tmp directory removed at /var/lib/lava/dispatcher/tmp/11
+   Job finished correctly
+   case: job
+   case_id: 142
    definition: lava
    result: pass
-   level: 4
-   duration: 0.10351395607
-   extra: ...
-   Cleanup: removing /var/lib/lava/dispatcher/tmp/12
 // Note: After running the test, if the (4) user LED's are all off, manually reboot the board by pressing the reset button (S1) or disconnect and reconnect the power cable to the board before re-running the test.//
-. You can see an example of a successful test log file [[https://gitlab.com/cip-project/cip-testing/testing/snippets/1666438| the Beaglebone Black Test results ]].
+. You can see an example of a successful test log file [[https://gitlab.com/snippets/1715994| the Beaglebone Black Test results ]].
-===== Test #7: Testing Externally Built Kernels =====
+===== Test #7: Use lavacli utility =====
 ==== Prerequisites: ====
-   * A working CIP Board-at-Desk Single-Developer (B@D) v1.0 Virtual Machine
+   * A working CIP Board-at-Desk Single-Developer (B@D) Virtual Machine
+   * A Beaglebone Black or a Renesas IWG20M board that is powered up, connected to the LAN with an ethernet cable and connected to the host machine with an FTDI cable.
+   * A built kernel for the appropriate board (see Test #3 or #5 depending upon your board)
+   * The board has a current successful health check (it is marked as online in LAVA)
+==== Procedure: ====
+. On the Virtual machine change to the ''~/git-repos/linux-cip'' directory.
+''vagrant@guest:~$ **cd ~/git-repos/linux-cip**''
+. If you have not done so you will need to install lavacli:
+''vagrant@guest:~$ **sudo apt install lavacli** ''
+''vagrant@guest:~$ ** mkdir ~/.config** ''
+. //If you have not already done so//, you will need to create an authentication token
+    In the LAVA web interface, click on API->Authentication Tokens and then on the 'new' button,
+    give it a description and then click the 'View Token Hash' button and copy the displayed hash value.
+    Then:
+'' vagrant@guest:~/git-repos/linux-cip$ **<nowiki>lavacli identities add --token <paste token here> --uri http://localhost:8080/RPC2 --username lavauser boardAtDesk</nowiki>** ''
+. Set the TREE_NAME environment variable.
+'' vagrant@guest:~/git-repos/linux-cip$ **export TREE_NAME=cip-example** ''
+// Note: This must be the same name you used when you built the kernel. //
+. Create a test for use with the ''lava-tool'' utility from within the ''linux-cip'' git repository.
+'' vagrant@guest:~/git-repos/linux-cip$ **/vagrant/scripts/create_test.sh /vagrant/tests/bbb_debian_local.yaml ~/mytest-board.yaml**''
+- if you are using the Renesas IWG20M use /vagrant/tests/iwg20m-test-ramdisk.yaml as the second parameter!
+// Note: ''create_test.sh'' must be run in the ''linux-cip'' directory so that the script can retrieve the git settings. //
+// Note: ''mytest-board.yaml'' is an example filename. Substitute your own test file name in its place. //
+. Run the ''lavacli command'' to submit the job to LAVA.
+'' vagrant@guest:~/git-repos/linux-cip$ **<nowiki>lavacli -i boardAtDesk jobs submit ~/mytest-bbb.yaml</nowiki>**''
+    // a job number will be echoed //
+// Note: Please see the LAVA documentation for help [[https://validation.linaro.org/static/docs/v2/developing-tests.html|writing tests]] in LAVA, //
+. On your host machine, open a web browser and navigate to the LAVA Website:
+''**http://localhost:8080**''
+. Click on the Scheduler menu and select the All Jobs option.
+. Click on the View Job Results icon for the top job to watch the build.
+. The results are displayed in the web page as the test runs. When the test is done, the status of the Job will show as **Complete** at the top of the page.
+This is an example what the start of the test looks like in the Beaglebone case:
+   lava-dispatcher, installed at version: 2018.4-2~bpo9+1
+   start: 0 validate
+   Start time: 2018-05-10 12:14:13.326013+00:00 (UTC)
+   Validating that http://localhost:8010/initramfs/initramfs.cpio.gz exists
+   Validating that http://localhost:8010/cip-test/cip_v4.4.126/v4.4.126-cip22/arm/omap2plus_defconfig/zImage exists
+   Validating that http://localhost:8010/cip-test/cip_v4.4.126/v4.4.126-cip22/arm/omap2plus_defconfig/dtbs/am335x-boneblack.dtb exists
+   Validating that http://localhost:8010/cip-test/cip_v4.4.126/v4.4.126-cip22/arm/omap2plus_defconfig/modules.tar.xz exists
+   validate duration: 0.02
+   case: validate
+   case_id: 128
+   definition: lava
+   result: pass
+   start: 1 tftp-deploy (timeout 00:04:00) [common]
+   start: 1.1 download-retry (timeout 00:04:00) [common]
+   start: 1.1.1 http-download (timeout 00:04:00) [common]
+   Not decompressing ramdisk as can be used compressed.
+   downloading http://localhost:8010/initramfs/initramfs.cpio.gz
+   saving as /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/ramdisk/initramfs.cpio.gz
+   <snip>
+This is what the end of a successful test looks like:
+   </snip>
+   Finalising connection for namespace 'common'
+   end: 4.2 read-feedback (duration 00:00:01) [common]
+   end: 4 finalize (duration 00:00:01) [common]
+   Cleaning after the job
+   Cleaning up download directory: /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/ramdisk
+   Cleaning up download directory: /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/kernel
+   Cleaning up download directory: /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/dtb
+   Cleaning up download directory: /var/lib/lava/dispatcher/tmp/11/tftp-deploy-pozjv6jf/modules
+   Override tmp directory removed at /var/lib/lava/dispatcher/tmp/11
+   Root tmp directory removed at /var/lib/lava/dispatcher/tmp/11
+   Job finished correctly
+   case: job
+   case_id: 142
+   definition: lava
+   result: pass
+// Note: After running the test, if the (4) user LED's are all off, manually reboot the board by pressing the reset button (S1) or disconnect and reconnect the power cable to the board before re-running the test.//
+. You can see an example of a successful test log file [[https://gitlab.com/snippets/1715994| the Beaglebone Black Test results ]].
+===== Test #8: Testing Externally Built Kernels =====
+==== Prerequisites: ====
+   * A working CIP Board-at-Desk Single-Developer (B@D) Virtual Machine
    * A board on which to test that kernel.
    * A defined device and device dictionary for the new device. Copy the dictionary entry into /etc/lava-server/dispatcher-config/devices (see /vagrant/integration-scripts/configure_lava.sh for examples).
@@ Line 450: / Line 606: @@
 ----
 ===== LAVA 2018-4 =====
-With this version of LAVA, in the above test cases replace any of the following sequences:
+With this version of LAVA, in the above test cases, replace any of the following sequences:
 >  Click on Scheduler and then select All Devices.
@@ Line 458: / Line 614: @@
 >  Click the Force Health Check button.
-With
+With:
    In Administration scroll down to Devices and select it, tick the appropriate device and then in the Action pull down select 'Update health of selected devices to Unknown' and click the 'Go' button.
@@ Line 465: / Line 621: @@
    vagrant@guest:~$ sudo cp testFileName.yaml /etc/lava-server/dispatcher-config/health-checks/DeviceName.yaml
-If you wish to just test then use the [[ciptestingreferencetestcases#test-6use-lava-tool-utility|lava-tool instructions]], replacing the file with your appropriate test.
+So for the supported devices replace DeviceName with beaglebone-black.yaml or renesas-iwg20m.yaml as appropriate.
+If you wish to just test then use the [[ciptestingreferencetestcases#test-7use-lavacli-utility|lavacli instructions]], replacing the file with your appropriate test.
 ----
 Return to the B@D [[ciptestingboardatdesksingledevfeaturepage|feature page]]

Wiki

User Tools

Site Tools

Differences

Page Tools