Integration tests of SNBmodules¶

The integration tests of snbmodules are located in the integtest folder, and the tests are usually executed under the snbmodules/integtest/ folder as follows:

pytest -s <test_file_name>.py

Nomenclature¶

Bookkeeper : Monitor file transfers status and registers/logs metadata about these transfers
TransferClient : Client object that contains transfer implenetation (e.g.: torrent, rclone, etc.) sessions (upload, download)
Transfer metadata : Contains informations about file transfers and their states
Group metadata : Contains multiple Transfer metadata uploaded by the same transfer client

snb_minimal_system_test.py¶

This very simple integration test contains 4 daq_applications (processes), each with a single module, running on a single host.

Aim of the test: Verify that the different SNB file transfer modules respect the state machine, the control interfaces are working as expected and can be communicated with via the run control. It also ensures that the SNB plugin configuration generators are working as expected. This test doesn't test actual transfers! There are individual tests for testing transfer implementations of different protocols.

Test steps:

Generate default configs for snbmodules.
Modify snbmodules configurations.
Add bookkeeper to snbmodules configuration.
Add client apps, in this case three.
Populate dunerc command list:

dunerc_command_list = "integtest-partition boot conf start 111 wait 1 enable_triggers ".split()
dunerc_command_list += ["wait"] + [str(run_duration)]
dunerc_command_list += "stop_run wait 2 scrap terminate".split()

Pass criteria:

test_dunerc_success: dunerc completed processes return code is 0 (no errors)
test_log_files: log files are error free
test_local_transfer_snbmodules: snb client process spawned and controlled successfully
test_bookkeeper_snbmodules: snb bookkeeper process spawned and controlled succesffully

Caveats/extra steps needed: None

Troubleshooting: None

snb_1node_1app_torrent_system_quick_test.py¶

This simple integration test contains a full system integration, exercising recorded SNB file transfer between different processes, running on a single host. The test exercises the torrent based file transfer implementation, and the SNB recording within the readout subsystem. The test contains a single SNB client process, which has both uploader and downloader sessions.

Aim of the test: Verify that files can be transferred between processes using the torrent based implementation. For source files, the integtest uses the readout raw recording features, in order to exercise the file registration ("new file transfer") command of SNBModules (bookkeeper and client). This also aims to verify the functionality and command sequence between readout and dataflow subsystems. The configuration parameters and aspects of the torrent based transfer implementation is also demonstrated and tested here.

Test steps:

Generate default configs for subsystems.
Modify snbmodules configurations.
Add bookkeeper to snbmodules configuration.
Prepare record-cmd.json to record raw content for 1 second from every data producers in the rulocalhosteth0 app.
Prepare new-torrent-transfer.json expert command for registering a transfer -> source and destination clients and file list. (Protocol arguments: select BITTORRENT and port to be used)
Prepare start-torrent-transfer.json expert command to start the upload/download procedure for the registered transfers.
Populate dunerc command list:

# The commands to run in dunerc, as a list
dunerc_command_list="integtest-partition boot conf start 111 wait 1 enable_triggers wait ".split() + [str(run_duration)] + \
("expert_command /json0/json0/ru" + interface_name + f" {root_path_commands}/record-cmd.json ").split() + \
["wait"] + [str(record_duration)] + \
f"expert_command /json0/json0/snbclient {root_path_commands}/new-torrent-transfer.json ".split() + \
f"expert_command /json0/json0/snbclient {root_path_commands}/start-torrent-transfer.json ".split() + \
["wait"] + [str(send_duration)] + "stop_run wait 2 scrap terminate".split()

Pass criteria:

test_dunerc_success: dunerc completed processes return code is 0 (no errors)
test_log_files: log files are error free
test_data_files: sanity check of the nominal request/response path (expected num. of Fragments and content is correct)
test_local_transfer_snbmodules: checks if the content of transferred files are matching with the source files (size and byte-by-byte match)
test_bookkeeper_snbmodules: snb bookkeeper reported correctly the transfer registration and state.

Caveats/extra steps needed:

The pytest area can result in moderately big size: With 2 data producers, a successful test will results with 2x150MB source files, and their copies after transfer in the destination folder. So in total, the pytest run area will exceed 600MBs! It's advised to remove the pytest run area after successful runs. Potentially, we could add a raw file cleanup to the test itself.
During the configuration generation step, the test looks up the IP address of the host. If the hostname is not IP resolvable, this information is not found in the system. The IP address is required for correct behavior of the torrent based transfer implementation.

Troubleshooting: In case the hostname resolvable IP address seems to be a too strict requirement, we should modify the integtest to pass the expected IP address via configuration or command line argument, instead of the following line in the test:

L123: ip_addr = socket.gethostbyname(socket.gethostname())
L124: assert ip_addr is not '192.168.0.1' and ip_addr is not 'localhost', 'Hostname resolvable IP address on the host is needed by this test!'

snb_1node_1app_rclone_http_system_quick_test.py¶

This simple integration test contains a full system integration, exercising recorded SNB file transfer between different processes, running on a single host. The test exercises the RClone based file transfer implementation served through HTTP protocol, and the SNB recording within the readout subsystem. The test contains a single SNB clientprocess, which has both uploader and downloader sessions.

Prerequisites: This test requires an rclone server process running, that is serving a remote access point over a protocol. In order to execute this test, there must be an rclone process started as follows:

rclone serve http / --addr localhost:8080  --copy-links --dir-cache-time 2s

The service is available from a standard DUNE-DAQ workarea environment. The lifetime and usability of this service spans across multiple integtest runs, tests, and any other processes that use rclone for file transfers.

Aim of the test: Verify that files can be transferred between processes using the RClone HTTP based implementation. For source files, the integtest uses the readout raw recording features, in order to exercise the file registration ("new file transfer") command of SNBModules (bookkeeper and client). This also aims to verify the functionality and command sequence between readout and dataflow subsystems. The configuration parameters and aspects of the RClone based transfer implementation is also demonstrated and tested here.

Test steps:

Generate default configs for subsystems.
Modify snbmodules configurations.
Add bookkeeper to snbmodules configuration.
Prepare record-cmd.json to record raw content for 1 second from every data producers in the rulocalhosteth0 app.
Prepare new-RClone-transfer.json expert command for registering a transfer -> source and destination clients and file list. (Protocol arguments: select RClone, HTTP protocol and port to be used -which is specified by the server process spawned in the prerequisites subsection of this test description)
Prepare start-transfer.json expert command to start the upload/download procedure for the registered transfers.
Populate dunerc command list:

# The commands to run in dunerc, as a list
dunerc_command_list="integtest-partition boot conf start 111 wait 1 enable_triggers wait ".split() + [str(run_duration)] + \
("expert_command /json0/json0/ru" + interface_name + f" {root_path_commands}/record-cmd.json ").split() + \
["wait"] + [str(record_duration)] + \
f"expert_command /json0/json0/snbclient {root_path_commands}/new-RClone-transfer.json ".split() + \
f"expert_command /json0/json0/snbclient {root_path_commands}/start-transfer.json ".split() + \
["wait"] + [str(send_duration)] + "stop_run wait 2 scrap terminate".split()

Pass criteria:

test_dunerc_success: dunerc completed processes return code is 0 (no errors)
test_log_files: log files are error free
test_data_files: sanity check of the nominal request/response path (expected num. of Fragments and content is correct)
test_local_transfer_snbmodules: checks if the content of transferred files are matching with the source files (size and byte-by-byte match)
test_bookkeeper_snbmodules: snb bookkeeper reported correctly the transfer registration and state.

Caveats/extra steps needed:

The pytest area can result in moderately big size: With 2 data producers, a successful test will results with 2x150MB source files, and their copies after transfer in the destination folder. So in total, the pytest run area will exceed 600MBs! It's advised to remove the pytest run area after successful runs. Potentially, we could add a raw file cleanup to the test itself.
There is an extra check before the configuration generation of the test, where it looks for a running rclone serve http process/service on the system. If the process is not running, the test immediately fails with an assertion error.

Troubleshooting: None

snb_1node_multiclientapps_rclone_http_system_quick_test.py¶

This simple integration test contains a full system integration, exercising the same targets as the 1app variant, but uploading the same source files to two target destination clients. Both of the destination clients reside in their own daq_application.

Prerequisitis, main test steps, and pass criterias are equivalent with the 1app variant. Only the topology is different.

Last git commit to the markdown source of this page:

Author: Kurt Biery

Date: Thu Apr 16 16:09:29 2026 -0500

If you see a problem with the documentation on this page, please file an Issue at https://github.com/DUNE-DAQ/snbmodules/issues