573 lines
18 KiB
Markdown
573 lines
18 KiB
Markdown
bladeRF Test Checklist
|
|
================================================================================
|
|
|
|
This file serves as a quick summary of tests that should be run regularly by
|
|
developers to detect defects and regressions.
|
|
|
|
Over time these procedures should become automated, and more fine-grained
|
|
tests should be introduced. At that time, this document shall be updated to
|
|
reflect usage of the automated tests.
|
|
|
|
When running through the items contained in this file, developers should
|
|
maintain an annotated copy of this file, containing:
|
|
|
|
* The date when tests were run
|
|
* The device versions used when performing tests
|
|
* "Checks" in the "boxes" ([ ]) and any associated observations or notes
|
|
|
|
--------------------------------------------------------------------------------
|
|
|
|
Build Tests
|
|
================================================================================
|
|
|
|
## FPGA Bitstream ##
|
|
|
|
* Perform FPGA images build with Quartus Prime (free edition) 17.1 (or latest
|
|
version) for the following:
|
|
|
|
* Board: bladeRF
|
|
* Revision: hosted
|
|
* [ ] Size: x40
|
|
* [ ] Size: x115
|
|
|
|
* Revision: atsc_tx
|
|
* [ ] Size: x40
|
|
* [ ] Size: x115
|
|
|
|
* Board: bladeRF-micro
|
|
* Revision: hosted
|
|
* [ ] Size: A2
|
|
* [ ] Size: A4
|
|
* [ ] Size: A9
|
|
|
|
## FX Firmware ##
|
|
|
|
* Perform firmware builds for both variants:
|
|
* [ ] Release
|
|
* [ ] Debug
|
|
|
|
* Ensure the build operates on the FX3 SDKs supported operating systems:
|
|
* [ ] Windows
|
|
* [ ] Linux
|
|
|
|
* Flash the images to appropriate boards and verify:
|
|
* [ ] Correct version number reported by `bladeRF-cli -e 'version'` (tagged
|
|
builds should have no commit hash; other builds should)
|
|
* [ ] Correct VID/PID and product descriptors are present on `lsusb -v` or
|
|
`dmesg`
|
|
* bladeRF: "Nuand bladeRF" 2cf0:5246
|
|
* bladeRF Micro: "Nuand bladeRF 2.0" 2cf0:5250
|
|
|
|
## Host software ##
|
|
|
|
Verify that build and installation completes from the top-level, from a clean
|
|
build directory, for each of below configurations.
|
|
|
|
Keep the "treat warnings as errors" option enabled; warnings should be regarded
|
|
as defects.
|
|
|
|
With a device attached, use the following commands to verify the build has
|
|
installed and that the desired backend is being used:
|
|
```bladeRF-cli -e 'version' -e 'info'```
|
|
|
|
Additionally, ensure ```libbladeRF_test_c``` and ```libbladeRF_test_cpp```
|
|
build and run.
|
|
|
|
* [ ] Debug build: ```-DCMAKE_BUILD_TYPE=Debug```
|
|
* [ ] Release build: ```-DCMAKE_BUILD_TYPE=Release -DTAGGED_RELEASE=Yes```
|
|
|
|
Perform the above builds with the following:
|
|
|
|
* Linux (various distributions; see host/misc/docker)
|
|
* [ ] GCC
|
|
* [ ] Clang
|
|
|
|
* macOS (latest)
|
|
* [ ] Apple LLVM
|
|
|
|
* Windows (currently Windows 10 w/ Visual Studio Community 2017)
|
|
* [ ] Build with support for both libusb and the Cypress backend. This should
|
|
be the default behavior when both present on the build system.
|
|
* [ ] Build with support for each of the two available backends.
|
|
* [ ] Perform builds targeting Win32 and x64
|
|
|
|
* Misc
|
|
* [ ] Build with documentation (from a clean CMake directory):
|
|
```-DENABLE_DOCUMENTATION=ON```
|
|
* Any Doxygen warnings should be regarded as defects.
|
|
* [ ] Clang `ccc-analyzer` & `scan-build` (with `-maxloop 100` or greater)
|
|
* Any items in the static analysis report should be regarded as defects.
|
|
|
|
|
|
Functional Tests
|
|
================================================================================
|
|
|
|
Most of the functional tests listed here are system-wide integration tests.
|
|
|
|
Unit tests, Built-in self-tests (BISTs), and top-level test-benches are
|
|
very much desirable additions for improving QA and testing efforts.
|
|
(Patches are always welcome!)
|
|
|
|
When appropriate, repeat each test for both USB 2.0 and USB 3.0 connections,
|
|
and for both the bladeRF and bladeRF Micro.
|
|
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
|
|
|
|
## FX3 Firmware Compliance Testing ##
|
|
|
|
The USB IF Compliance test tools shall be used to identify missing/incorrect
|
|
functionality. Run these and record any failures or warnings.
|
|
|
|
* [USB30CV tool](http://www.usb.org/developers/tools/#usb30tools)
|
|
* [USB20CV tool](http://www.usb.org/developers/tools/usb20_tools/#usb20cv)
|
|
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
|
|
|
|
## SPI Flash Access ##
|
|
|
|
The following items are intended to exercise SPI flash accesses.
|
|
|
|
### OTP region readback ###
|
|
|
|
There should be no warnings, info, or debug messages reported while performing
|
|
this operation.
|
|
|
|
* [ ] ```bladeRF-cli -e 'info' -v debug``` should report the serial number.
|
|
|
|
### Calibration region readback ###
|
|
|
|
There should be no warnings, info, or debug messages reported while performing
|
|
this operation.
|
|
|
|
* [ ] ```bladeRF-cli -e 'info' -v debug``` should report the VCTCXO DAC
|
|
calibration and FPGA size.
|
|
|
|
### Writing calibration data ###
|
|
|
|
During the following steps, there should be no error, warning, info, or debug
|
|
messages other than the information about flash page read operations.
|
|
|
|
* [ ] Back up calibration data via:
|
|
```bladeRF-cli -e 'flash_backup cal.bin cal' -v debug```
|
|
|
|
* [ ] Write new calibration data (using the appropriate FPGA size) via:
|
|
```bladeRF-cli -e 'flash_init_cal 40 0x8000' -v debug```
|
|
|
|
* [ ] Power cycle the device and verify the new values (see the previous
|
|
subsection).
|
|
|
|
* [ ] Restore the original calibration data via:
|
|
```bladeRF-cli -e 'flash_restore cal.bin cal' -v debug```
|
|
|
|
* [ ] Power cycle the device and verify the new values.
|
|
|
|
### Firmware Programming ###
|
|
|
|
Consider any errors or warnings to be defects during these operations.
|
|
Info messages are expected, and are intended to provide the user with
|
|
feedback about the status of flash operations.
|
|
|
|
* [ ] Force the device back to the FX3 bootloader via:
|
|
```bladeRF-cli -e 'jump_to_boot' -v debug```
|
|
|
|
* [ ] Power cycle the device. It should continue to boot into the bootloader,
|
|
instead of the firmware.
|
|
|
|
* [ ] "Recover" the device and Flash firmware via the bladeRF-cli:
|
|
```
|
|
bladeRF> recover <bus> <addr> bladeRF_fw_vX.Y.Z.img
|
|
bladeRF> open
|
|
bladeRF> load fx3 bladeRF_fw_vX.Y.Z.img
|
|
```
|
|
|
|
* [ ] Power cycle the device and verify that it boots the desired firmware,
|
|
and that the calibration data is intact:
|
|
```bladeRF-cli -e 'version' -e 'info' -v debug```
|
|
|
|
* [ ] Flash a different version of firmware via:
|
|
```bladeRF-cli -f bladeRF_fw_vA.B.C.img -v debug```
|
|
|
|
* [ ] Power cycle the device and verify that the new firmware has been loaded,
|
|
and that the calibration data is correct.
|
|
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
|
|
|
|
## FPGA Loading ##
|
|
|
|
### FPGA Flash-based Autoloading ###
|
|
|
|
Consider any errors or warnings to be defects during these operations.
|
|
Info messages are expected, and are intended to provide the user with
|
|
feedback about the status of flash operations.
|
|
|
|
Ensure that you do not have any FPGA bitstreams on the host set up for
|
|
host-based autoloading.
|
|
|
|
* [ ] Erase any previously stored FPGA images:
|
|
```bladeRF-cli -L X -v debug```
|
|
|
|
* [ ] Power cycle the device, and verify that this has not corrupted the
|
|
calibration data or firmware:
|
|
```bladeRF-cli -e 'version' -e 'info' -v debug```
|
|
|
|
* [ ] Write an FPGA bitstream to flash:
|
|
```bladeRF-cli -L <fpga image> -v debug```
|
|
|
|
* [ ] Power cycle the device and verify that the firmware boots, the FPGA is
|
|
loaded, and that the calibration data has not been corrupted.
|
|
```bladeRF-cli -e 'version' -e 'info' -v debug```
|
|
* Ensure this works when powered from a USB host, from a USB charger, and
|
|
from a power supply connected to the DC barrel plug.
|
|
|
|
* [ ] Erase the FPGA and perform the associated verifications.
|
|
|
|
### Host-based Autoloading ###
|
|
|
|
* [ ] Place a FPGA image to an autoload directory.
|
|
|
|
* [ ] Power cycle the device.
|
|
|
|
* [ ] Verify that the intended FPGA is loaded:
|
|
```bladeRF-cli -e 'version' -e 'info' -v debug```
|
|
|
|
### Manual FPGA Loading ###
|
|
|
|
* [ ] Verify that FPGA loading via ```bladeRF-cli -l <fpga> -v debug```
|
|
succeeds.
|
|
|
|
* [ ] Verify that the expected version was loaded:
|
|
```bladeRF-cli -e 'version' -e 'info' -v debug```.
|
|
|
|
Load a different version with the FPGA already loaded, and preform the
|
|
verification step again.
|
|
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
|
|
|
|
## Basic Device Control: bladeRF ##
|
|
|
|
These device control operations may be performed with the `bladeRF-cli`, or for
|
|
simplicity, third party tools, as they should utilize the same underlying
|
|
`libbladeRF` API calls.
|
|
|
|
A signal generator and spectrum analyzer are required for many of these items.
|
|
|
|
### Frequency Tuning ##
|
|
|
|
When performing these tests, make notes of the frequencies that you are using.
|
|
If issue occur, it is important to know if it is when crossing "bands" or
|
|
internal device ranges.
|
|
|
|
#### RX ####
|
|
|
|
* [ ] Verify reception of reference signals at various frequencies in
|
|
300 MHz - 3.8 GHz, ***without*** a XB-200 attached.
|
|
* [ ] Verify reception of reference signals at various frequencies up to
|
|
3.8 GHz, ***with*** a XB-200 attached, using each filter configuration:
|
|
* [ ] Auto 3 dB filterbank
|
|
* [ ] Auto 1 dB filterbank
|
|
* [ ] 6m filterbank
|
|
* [ ] 2m filterbank
|
|
* [ ] 1.25m filterbank
|
|
* [ ] Custom path
|
|
|
|
#### TX ####
|
|
|
|
Suggested reference signal: W-CDMA QPSK test pattern
|
|
|
|
* [ ] Verify that a VSA can receive and demodulate reference signals at
|
|
frequencies in 300 MHz - 3.8 GHz, ***without*** a XB-200 attached.
|
|
* [ ] Verify that a VSA can receive and demodulate reference signals at
|
|
frequencies up to 3.8 GHz, ***with*** a XB-200 attached, using each
|
|
filter configuration:
|
|
* [ ] Auto 3 dB filterbank
|
|
* [ ] Auto 1 dB filterbank
|
|
* [ ] 6m filterbank
|
|
* [ ] 2m filterbank
|
|
* [ ] 1.25m filterbank
|
|
* [ ] Custom path
|
|
|
|
### Gain ###
|
|
|
|
#### RX ####
|
|
|
|
Supply the device with a reference signal and verify the effect of changing
|
|
the gain values for:
|
|
|
|
* [ ] Overall system gain (`set gain rx <dB>`)
|
|
* [ ] RX LNA
|
|
* [ ] RX VGA1
|
|
* [ ] RX VGA2
|
|
|
|
#### TX ####
|
|
|
|
Transmit a reference signal to a VSA and verify the effect of changing the
|
|
gain values for:
|
|
|
|
* [ ] Overall system gain (`set gain tx <dB>`)
|
|
* [ ] TX VGA1
|
|
* [ ] TX VGA2
|
|
|
|
### Bandwidth ###
|
|
|
|
#### RX ####
|
|
|
|
* [ ] Supply a reference signal of a wide bandwith and verify the effect of
|
|
changing the RX bandwidth.
|
|
|
|
#### TX ####
|
|
|
|
* [ ] Transmit a reference signal of a wide bandwith to a VSA and verify the
|
|
effect of changing the TX bandwidth.
|
|
|
|
### DC Offset ###
|
|
|
|
Repeat the following steps for a variety of frequency/gain combinations and make
|
|
note of the results.
|
|
|
|
* [ ] Use the bladeRF-cli to perform an initial ```cal lms``` at a desired
|
|
frequency and gain.
|
|
|
|
* [ ] In the CLI, perform a ```cal dc rx``` operation, and note the effect on
|
|
received IQ values and on an FFT plot.
|
|
|
|
* [ ] Create and load an RX calibration table. Verify that corrections are
|
|
applied over the range of the table.
|
|
|
|
* [ ] In the CLI, perform a ```cal dc tx``` operation, and transmit a tone into
|
|
a VSA. Make note of the difference in the DC offset.
|
|
|
|
* [ ] Create and load a TX calibration table. Verify that corrections are
|
|
applied over the range of the table.
|
|
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
|
|
|
|
## Basic Device Control: bladeRF Micro ##
|
|
|
|
These device control operations may be performed with the `bladeRF-cli`, or for
|
|
simplicity, third party tools, as they should utilize the same underlying
|
|
`libbladeRF` API calls.
|
|
|
|
A signal generator and spectrum analyzer are required for many of these items.
|
|
|
|
### Frequency Tuning ##
|
|
|
|
When performing these tests, make notes of the frequencies that you are using.
|
|
If issue occur, it is important to know if it is when crossing "bands" or
|
|
internal device ranges.
|
|
|
|
#### RX ####
|
|
|
|
* [ ] Verify reception of reference signals at various frequencies in
|
|
70 MHz - 6 GHz
|
|
* [ ] RX1
|
|
* [ ] RX2
|
|
* [ ] Verify that `print rssi` prints reasonable values that vary based on
|
|
signal level.
|
|
* [ ] RX1
|
|
* [ ] RX2
|
|
|
|
#### TX ####
|
|
|
|
Suggested reference signal: W-CDMA QPSK test pattern
|
|
|
|
* [ ] Verify that a VSA can receive and demodulate reference signals at
|
|
frequencies in 70 MHz - 6.0 GHz
|
|
* [ ] TX1
|
|
* [ ] TX2
|
|
|
|
### Gain ###
|
|
|
|
#### RX ####
|
|
|
|
Supply the device with a reference signal and verify the effect of changing
|
|
the gain values for:
|
|
|
|
* [ ] Overall system gain (`set gain rx{1,2} <dB>`)
|
|
* [ ] RX1
|
|
* [ ] RX2
|
|
* [ ] `full` stage (`set gain rx{1,2} full <dB>`)
|
|
* [ ] RX1
|
|
* [ ] RX2
|
|
|
|
#### TX ####
|
|
|
|
Transmit a reference signal to a VSA and verify the effect of changing the
|
|
gain values for:
|
|
|
|
* [ ] Overall system gain (`set gain tx{1,2} <dB>`)
|
|
* [ ] TX1
|
|
* [ ] TX2
|
|
* [ ] `dsa` stage (`set gain tx{1,2} dsa <dB>`)
|
|
* [ ] TX1
|
|
* [ ] TX2
|
|
|
|
### Bandwidth ###
|
|
|
|
#### RX ####
|
|
|
|
* [ ] Supply a reference signal of a wide bandwith and verify the effect of
|
|
changing the RX bandwidth.
|
|
* [ ] RX1
|
|
* [ ] RX2
|
|
|
|
#### TX ####
|
|
|
|
* [ ] Transmit a reference signal of a wide bandwith to a VSA and verify the
|
|
effect of changing the TX bandwidth.
|
|
* [ ] TX1
|
|
* [ ] TX2
|
|
|
|
### ADF4002 ###
|
|
|
|
Connect a 10 MHz reference source (e.g. GPSDO, or 10 MHz ref output from VSA)
|
|
to `REFIN` (J95). The signal should be [TODO: voltage here ... sine/square?]
|
|
|
|
Note: Ensure the VSA is synchronized to the same clock.
|
|
|
|
Caution: UFL connectors are fragile and have limited mating cycles.
|
|
|
|
Transmit a reference signal (e.g. QPSK constellation into EVM analysis, or a
|
|
tone into spectrum analysis) into a VSA.
|
|
|
|
* [ ] With the ADF4002 disabled (`set adf_enable 0`), note the frequency offset
|
|
of the reference signal.
|
|
* [ ] `print adf_enable` should report `ADF Chip Enable: disabled`
|
|
* [ ] Enable the ADF4002 (`set adf_enable 1`).
|
|
* [ ] `print adf_enable` should report `ADF Chip Enable: enabled`
|
|
and `ADF Chip Locked: locked`
|
|
* [ ] The frequency offset of the reference signal should be very small,
|
|
within a few Hz.
|
|
* [ ] `print trimdac` should report `0xc000` for current VCTCXO trim
|
|
* [ ] Change the ADF4002's reference frequency: `set adf_refclk 9M`
|
|
* [ ] `print adf_enable` should report `ADF Chip Enable: enabled`
|
|
and `ADF Chip Locked: unlocked`
|
|
* [ ] Change the ADF4002's reference frequency back to 10 MHz:
|
|
`set adf_refclk 10M`
|
|
* [ ] `print adf_enable` should report `ADF Chip Enable: enabled`
|
|
and `ADF Chip Locked: locked`
|
|
* [ ] Disable the reference clock while the ADF is enabled (note: don't
|
|
do this on the UFL connector)
|
|
* [ ] `print adf_enable` should report `ADF Chip Enable: enabled`
|
|
and `ADF Chip Locked: unlocked`
|
|
* [ ] Disable the ADF4002 (`set adf_enable 0`)
|
|
* [ ] `print adf_enable` should report `ADF Chip Enable: disabled`
|
|
* [ ] `print trimdac` should NOT report `0xc000` for current VCTCXO trim,
|
|
but should instead return to its previous value.
|
|
|
|
### Bias tees ###
|
|
|
|
Note: disconnect RX1/RX2/TX1/TX2 when testing the bias tees to avoid possible
|
|
damage to test equipment.
|
|
|
|
* [ ] Disable the bias tees: `set biastee rx 0 ; set biastee tx 0`
|
|
* [ ] Verify 0 VDC between RX1/RX2/TX1/TX2 center pin and ground
|
|
* [ ] Enable the RX bias tees: `set biastee rx 1`
|
|
* [ ] Verify ~5 VDC between the center pins of RX1 & RX2 and ground, and
|
|
0 VDC between the center pins of TX1 & TX2 and ground
|
|
* [ ] Enable the TX bias tees: `set biastee tx 1`
|
|
* [ ] Verify ~5 VDC between the center pins of RX1/RX2/TX1/TX2 and ground
|
|
* [ ] Disable the RX bias tees: `set biastee rx 0`
|
|
* [ ] Verify ~5 VDC between the center pins of TX1 & TX2 and ground, and
|
|
0 VDC between the center pins of RX1 & RX2 and ground
|
|
* [ ] Disable the TX bias tees: `set biastee tx 0`
|
|
* [ ] Verify 0 VDC between RX1/RX2/TX1/TX2 center pin and ground
|
|
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
|
|
|
|
### libbladeRF test programs ###
|
|
|
|
The libbladeRF test programs are intended to exercise specific functionalities
|
|
provided by the API. However, these programs inherently exercise device control
|
|
functionality not included above. A selection of the ```libbladeRF_test_*```
|
|
programs is provided here to attempt to maximize coverage with a small number
|
|
of programs.
|
|
|
|
#### ```libbladeRF_test_ctrl``` ####
|
|
|
|
This test ensures that various device control calls complete without error.
|
|
|
|
Run all of its tests **without** an XB-200 attached.
|
|
|
|
* [ ] ```sampling```
|
|
* [ ] ```lpf_mode```
|
|
* [ ] ```enable_module```
|
|
* [ ] ```loopback```
|
|
* [ ] ```correction```
|
|
* [ ] ```samplerate```
|
|
* [ ] ```bandwidth```
|
|
* [ ] ```gain```
|
|
* [ ] ```frequency```
|
|
* [ ] ```threads```
|
|
|
|
Run all of its tests **with** an XB-200 attached.
|
|
|
|
* [ ] ```sampling```
|
|
* [ ] ```lpf_mode```
|
|
* [ ] ```enable_module```
|
|
* [ ] ```loopback```
|
|
* [ ] ```correction```
|
|
* [ ] ```samplerate```
|
|
* [ ] ```bandwidth```
|
|
* [ ] ```gain```
|
|
* [ ] ```frequency```
|
|
* [ ] ```threads```
|
|
|
|
#### ```libbladeRF_test_rx_discont``` ####
|
|
|
|
* [ ] Run this program over a variety of valid sample rates (for the particular
|
|
USB connection). No discontinuities should occur.
|
|
|
|
* [ ] Run this program over for sample rates that your USB controller does not
|
|
support. Discontinuities should be reported.
|
|
|
|
#### ```libbladeRF_test_repeater``` ####
|
|
|
|
This program uses the async interface to immediately retransmit received
|
|
samples.
|
|
|
|
Supply a reference signal from a signal generator, and verify that it can
|
|
be demodulated without failures on a VSA.
|
|
|
|
Record the sample rates, bandwidths, frequencies, and stream parameters used.
|
|
|
|
* [ ] Verify that transmitted samples are representative of the received signals.
|
|
|
|
|
|
#### ```libbladeRF_test_timestamps``` ####
|
|
|
|
This program verifies timestamp functionality. Run all available test at
|
|
a variety of sample rates. Not that some tests (e.g., loopback_onoff) are not
|
|
designed to support high sample rates.
|
|
|
|
* [ ] ```rx_gaps```
|
|
* [ ] ```rx_scheduled```
|
|
* [ ] ```tx_onoff```
|
|
* [ ] ```tx_onoff_nowsched```
|
|
* [ ] ```tx_gmsk_bursts```
|
|
* [ ] ```loopback_onoff```
|
|
* [ ] ```format_mismatch```
|
|
* [ ] ```readback```
|
|
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
|
|
|
|
### libbladeRF sample programs ###
|
|
|
|
As these are intended to exemplify API usage, these programs must be correct.
|
|
|
|
Verify that the following run as intended:
|
|
|
|
* [ ] ```libbladeRF_example_open_via_serial```
|
|
* [ ] ```libbladeRF_example_sync_rxtx```
|
|
* [ ] ```libbladeRF_example_sync_rx_meta```
|
|
* [ ] ```libbladeRF_example_sync_tx_meta```
|
|
|
|
|
|
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
|
|
|
|
## Third-party Tools ##
|
|
|
|
Testing against third-party tools with bladeRF support to ensure changes have
|
|
not introduced compatibility issues is highly recommended.
|
|
|
|
Make note of the tool and its version, for future comparison, should any issues
|
|
arise.
|