utils.datautils

General I/O and preprocessing utilities for BRIDGE-style datasets.

This module collects small helper functions used across data preparation and experiment scripts. The active functions focus on:

• filesystem helpers (creating output folders, listing dataset files)

• simple run bookkeeping (checking whether an expected results file is complete)

• sequence preprocessing (DNA/RNA → one-hot encoding with optional centered padding)

• dataset splitting (simple stratified train/validation split for binary targets)

• lightweight formatting helpers (array → string)

Who this is for

• Users running BRIDGE-related scripts that need quick dataset utilities.

• Developers who want a single place for shared preprocessing functions.

Dependencies

• Standard library: os, sys, copy.deepcopy

• Third-party: numpy, h5py (imported here but only required by legacy/extended loaders)

API overview

Filesystem

make_directory(path, foldername, verbose=1)

Ensures path exists, then creates/returns os.path.join(path, foldername).

get_file_names(dataset_path)

Returns all filenames in dataset_path with extension exactly .h5 (not full paths).

Bookkeeping

finished(path, line_num)

Returns True if path exists and contains exactly line_num lines.

Formatting

mat2str(m)

Converts a 1D/2D numeric array into a comma-separated string with '%.3f,' formatting (note: a trailing comma is included).

Sequence preprocessing

convert_one_hot(sequence, max_length=None)

Converts a list of DNA/RNA strings into one-hot arrays with channel order A, C, G, (U/T). Optional centered zero-padding to max_length.

Dataset splitting

split_dataset(data, targets, valid_frac=0.2)

Performs a simple stratified split using a binary threshold on targets: negatives: targets < 0.5, positives: targets >= 0.5.

Input/Output conventions

One-hot encoding

• Input: sequence is a list[str] (each sequence is uppercased internally).

• Output: np.ndarray of shape (N, 4, L) or (N, 4, max_length).

• Channel order:

  • channel 0: A

  • channel 1: C

  • channel 2: G

  • channel 3: U or T

Padding behavior (convert_one_hot)

If max_length is provided, sequences are centered and padded symmetrically:

• offset1 = (max_length - seq_length) // 2

• offset2 = max_length - seq_length - offset1

The function does not truncate sequences longer than max_length; in that case negative offsets would lead to unexpected behavior. Ensure max_length >= len(seq) upstream or add explicit truncation.

Split behavior (split_dataset)

• data is assumed to be indexed by sample along axis 0: (N, ...).

• targets must align with data along axis 0 (shape (N,) or (N, 1)).

• Randomness comes from np.random.permutation; set np.random.seed(...) upstream for reproducibility.

How to use

Convert sequences to one-hot:

from utils import convert_one_hot
seqs = ["ACGT", "AUGU"]   # 'U' is treated like 'T' in the 4th channel
X = convert_one_hot(seqs, max_length=10)   # shape (2, 4, 10)

Split into train/test:

import numpy as np
from utils import split_dataset

X = np.random.randn(100, 4, 101)
y = (np.random.rand(100, 1) > 0.5).astype(np.float32)

(X_tr, y_tr), (X_te, y_te) = split_dataset(X, y, valid_frac=0.2)

Notes and caveats

• Character handling in convert_one_hot: Characters other than A/C/G/U/T are silently ignored (left as all-zeros). If you need explicit handling of N or other IUPAC codes, extend the implementation.

Functions

convert_one_hot(sequence[, max_length])	Convert DNA/RNA sequences to one-hot encoding with optional center padding.
finished(path, line_num)	Check whether a text results file contains an expected number of lines.
get_file_names(dataset_path)	List all HDF5 filenames in a directory.
make_directory(path, foldername[, verbose])	Make a directory
mat2str(m)	Convert a 1D or 2D numeric array to a comma-separated string with 3-decimal formatting.
md5(string)	Compute the MD5 hex digest of a UTF-8 string.
split_dataset(data, targets[, valid_frac])	Stratified split of a dataset into train/test partitions by a binary threshold.

utils.datautils.make_directory(path, foldername, verbose=1)

Make a directory

utils.datautils.finished(path, line_num)

Check whether a text results file contains an expected number of lines.

Parameters:

• path (str) – Path to the results file.

• line_num (int) – Expected total number of lines in the file.

Returns:

bool – True if file exists and line count matches line_num, else False.

utils.datautils.get_file_names(dataset_path)

List all HDF5 filenames in a directory.

Parameters:

dataset_path (str) – Directory containing dataset files.

Returns:

list[str] – Filenames (not full paths) whose extension is exactly ‘.h5’.

utils.datautils.md5(string)

Compute the MD5 hex digest of a UTF-8 string.

Parameters:

string (str) – Input string to hash.

Returns:

str – Lowercase MD5 hex digest.

utils.datautils.mat2str(m)

Convert a 1D or 2D numeric array to a comma-separated string with 3-decimal formatting.

Parameters:

m (np.ndarray) – 1D array shape (D,) or 2D array shape (R, C).

Returns:

str – Comma-separated string with each value formatted as ‘%.3f,’. Note the string has a trailing comma.

Notes

• For 2D arrays, rows are flattened in row-major order.

• Does not insert line breaks between rows.

utils.datautils.convert_one_hot(sequence, max_length=None)

Convert DNA/RNA sequences to one-hot encoding with optional center padding.

Encoding:

Channel order: A, C, G, (U/T) - ‘A’ -> channel 0 - ‘C’ -> channel 1 - ‘G’ -> channel 2 - ‘U’ or ‘T’ -> channel 3

Parameters:

• sequence (list[str]) – List of sequence strings. Characters are uppercased internally.

• max_length (int, optional) – If provided, sequences are zero-padded (centered) to this length. Padding is applied symmetrically as:

  offset1 = (max_length - seq_length) // 2 offset2 = max_length - seq_length - offset1

  The resulting shape becomes (N, 4, max_length).

Returns:

np.ndarray – One-hot encoded array of shape (N, 4, L) or (N, 4, max_length), dtype float64 by default due to np.zeros usage.

Notes

• Characters other than A/C/G/U/T are ignored (remain all-zeros at that position). If you want explicit handling of ‘N’ etc., add it upstream.

utils.datautils.split_dataset(data, targets, valid_frac=0.2)

Stratified split of a dataset into train/test partitions by a binary threshold.

This function defines:

negatives: targets < 0.5 positives: targets >= 0.5

and then samples approximately valid_frac from each group into the test set.

Parameters:

• data (np.ndarray) – Input samples array. First dimension is assumed to be sample axis (N, …).

• targets (np.ndarray) – Target values aligned with data along the first axis. Can be shape (N,) or (N,1) as long as comparisons and indexing work.

• valid_frac (float, optional) – Fraction of each class to allocate to the validation/test split. Default: 0.2.

Returns:

tuple[tuple[np.ndarray, np.ndarray], tuple[np.ndarray, np.ndarray]] –

(train, test) where:

train = (X_train, Y_train) test = (X_test, Y_test)

Notes

• Within each class, indices are randomly permuted via np.random.permutation.

• The returned train/test are concatenations of positives then negatives (as implemented).