utils.structureFeatures

utils.structureFeatures#

Secondary-structure feature utilities: RNAplfold profiles and per-base icSHAPE scores.

This module provides two independent ways to obtain “structure” inputs aligned to nucleotide positions for downstream models (e.g., BRIDGE):

Via generateStructureFeatures(), this module can run external RNAplfold wrapper executables to compute loop-type probabilities per position and parse them into a tensor shaped (N, L, 5).
Channels (column order in the returned array):
- P: pairedness probability (computed as residual probability mass)
- H: hairpin-loop probability
- I: internal-loop probability
- M: multi-loop probability
- E: external-region probability
The combined profile is cached on disk and parsed by read_combined_profile().
Via build_structure_tensor(), this module can convert per-sequence structure strings such as icSHAPE reactivity tracks into a padded numeric tensor shaped (N, 1, L).

In this representation, each sequence has a single structure channel where the i-th value corresponds to the i-th nucleotide position (same length/alignment as the sequence).

The typical upstream format is a comma-separated string, for example:
```
"0.12,0.03,0.50,0.10,..."
```
This is referred to as “icshape” in some parts of the codebase.

Who this is for#

Users preparing token-aligned structure inputs for training/inference.
Developers maintaining preprocessing and cache behavior.

Input / output conventions#

Token alignment

All structure tensors produced here are position-aligned. The caller is responsible for ensuring that the chosen L matches the sequence length convention used elsewhere (e.g., fixed-length 101 in many BRIDGE pipelines).

RNAplfold path (multi-channel)

Input: a FASTA file path dataset_path readable by external wrapper executables.
Output: np.ndarray of shape (N, L, 5) and dtype=float.

icSHAPE path (single-channel)

Input: structs is a list of comma-separated numeric strings, one per sequence.
Output: np.ndarray of shape (N, 1, max_length) (float64 by NumPy default).
Length constraint: each string must contain exactly max_length comma-separated values.

External dependency (RNAplfold only)#

run_RNA() uses os.system to invoke four wrapper executables under script_path:

E_RNAplfold, H_RNAplfold, I_RNAplfold, M_RNAplfold

These wrappers are expected to: - read FASTA from stdin (< fasta_path) - write two-line-per-record profiles to *_profile.txt

Warning

If these executables are missing or not executable, the RNAplfold path will fail.

How to use#

RNAplfold multi-channel features:

feats = generateStructureFeatures(
    dataset_path="inputs.fa",
    script_path="path/to/wrappers",
    basic_path="workdir/struct_cache/",
    W=101, L=101, u=1,
    dataset_name="my_dataset"
)
# feats: (N, L, 5) with columns [P, H, I, M, E] after parsing

icSHAPE / icshape single-channel tensor:

structs = [
    "0.1,0.2,0.3,0.4",
    "0.0,0.5,0.5,0.2",
]
x = build_structure_tensor(structs, max_length=4)
# x: (2, 1, 4)

Important notes / caveats#

These two structure representations are not interchangeable: RNAplfold returns 5 loop/pairedness channels, while icSHAPE returns a single per-base score.
Pairedness computation (RNAplfold): P = 1 - E - H - I - M assumes the four probabilities sum to <= 1 per position. If they sum to > 1 (numerical issues or wrapper semantics), P may become negative.
Cache path check mismatch (behavior preserved): generateStructureFeatures checks for basic_path + "/combined_profile.txt" but writes to <basic_path>/<dataset_name>/combined_profile.txt. Ensure your cache layout matches, or adjust the check if you standardize caching.
Parsing of icSHAPE strings: build_structure_tensor() does not trim whitespace or trailing commas. Upstream strings should be clean (e.g., no trailing comma). A mismatch in value count will raise an error (or fail assignment/broadcasting).

Functions

`build_structure_tensor`(structs, max_length)	Convert comma-separated structure score strings into a padded 3D tensor.
`concatenate`(pairedness, hairpin_loop, ...)	Combine multiple whitespace-delimited structure tracks into a per-position feature matrix.
`defineExperimentPaths`(basic_path, name_id)	Create and return directory paths used for RNAplfold-derived structure profiles.
`generateStructureFeatures`(dataset_path, ...)	Generate per-position RNA secondary-structure features using RNAplfold and cache results.
`list_to_str`(lst)	Convert a list of values into a tab-delimited string.
`mk_dir`(dir)	Create a directory.
`read_combined_profile`(file_path)	Parse a combined structure profile file into a numeric tensor.
`run_RNA`(fasta_path, script_path, E_path, ...)	Run external RNAplfold wrapper executables to generate structure profile text files.

utils.structureFeatures.mk_dir(dir)[source]#: Create a directory.

utils.structureFeatures.list_to_str(lst)[source]#: Convert a list of values into a tab-delimited string.

utils.structureFeatures.concatenate(pairedness, hairpin_loop, internal_loop, multi_loop, external_region)[source]#

Combine multiple whitespace-delimited structure tracks into a per-position feature matrix.

Parameters:

pairedness (str) – Whitespace-delimited numeric tokens for the pairedness (P) track.
hairpin_loop (str) – Whitespace-delimited numeric tokens for the hairpin-loop (H) track.
internal_loop (str) – Whitespace-delimited numeric tokens for the internal-loop (I) track.
multi_loop (str) – Whitespace-delimited numeric tokens for the multi-loop (M) track.
external_region (str) – Whitespace-delimited numeric tokens for the external-region (E) track.

Returns:

np.ndarray – Array of shape (L, 5) where L is the number of positions/tokens and columns correspond to [P, H, I, M, E] in the order provided to this function.

Notes

Each input string is split with .split(); multiple spaces are treated as separators.
All input tracks are assumed to have the same token length L.

utils.structureFeatures.defineExperimentPaths(basic_path, name_id)[source]#

Create and return directory paths used for RNAplfold-derived structure profiles.

This function creates the following directories under:

basic_path/<name_id>/: E/, H/, I/, M/

Parameters:

basic_path (str) – Root directory for storing intermediate outputs.
name_id (str or int) – Dataset identifier appended to basic_path.

Returns:

Tuple[str, str, str, str, str] – (path, E_path, H_path, I_path, M_path), each ending with ‘/’.

utils.structureFeatures.read_combined_profile(file_path)[source]#

Parse a combined structure profile file into a numeric tensor.

Expected file format:

The file is assumed to contain repeating 6-line blocks:: line 0: an identifier line (ignored by this parser) line 1: pairedness probabilities (P) as whitespace-separated numbers line 2: hairpin-loop probabilities (H) line 3: internal-loop probabilities (I) line 4: multi-loop probabilities (M) line 5: external-region probabilities (E)

This function reads lines 1..5 of each block and concatenates them into an (L, 5) array.

Parameters:

file_path (str) – Path to the combined profile text file.

Returns:

np.ndarray –

Float array of shape (N, L, 5), where:: N = number of records (blocks), L = number of positions/tokens in the profile lines, 5 = number of structure channels.

Notes

Uses linecache.getlines, which reads the whole file into memory.
Whitespace is normalized with re.sub(‘[s+]’, ‘ ‘, …) before splitting.
Assumes every record is exactly 6 lines and all profile lines have equal token length.

utils.structureFeatures.run_RNA(fasta_path, script_path, E_path, H_path, I_path, M_path, W, L, u)[source]#

Run external RNAplfold wrapper executables to generate structure profile text files.

This function invokes four commands via os.system:

<script_path>/E_RNAplfold …
<script_path>/H_RNAplfold …
<script_path>/I_RNAplfold …
<script_path>/M_RNAplfold …

Each command reads from stdin redirected from fasta_path and writes output to:

E_path/E_profile.txt, H_path/H_profile.txt, I_path/I_profile.txt, M_path/M_profile.txt

Parameters:

fasta_path (str) – Path to an input FASTA file for RNAplfold to process.
script_path (str) – Directory containing the RNAplfold wrapper executables.
E_path (str) – Output directory for E_profile.txt.
H_path (str) – Output directory for H_profile.txt.
I_path (str) – Output directory for I_profile.txt.
M_path (str) – Output directory for M_profile.txt.
W (int) – RNAplfold window size argument (-W).
L (int) – RNAplfold maximum base pair span argument (-L).
u (int) – RNAplfold “unpaired” length argument (-u).

Returns:

None.

utils.structureFeatures.generateStructureFeatures(dataset_path, script_path, basic_path, W, L, u, dataset_name='')[source]#

Generate per-position RNA secondary-structure features using RNAplfold and cache results.

Workflow:

Create output directories under: basic_path/<dataset_name>/[E,H,I,M]/
If <basic_path>/combined_profile.txt does NOT exist, run RNAplfold wrappers and write a combined profile file at: <path>/combined_profile.txt
Parse the combined profile file into a numeric tensor via read_combined_profile.

Parameters:

dataset_path (str) – Path to input FASTA file to process.
script_path (str) – Directory containing RNAplfold wrapper executables.
basic_path (str) – Root directory for intermediate outputs and cache files.
W (int) – RNAplfold window size (-W).
L (int) – RNAplfold maximum base pair span (-L).
u (int) – RNAplfold unpaired length (-u).
dataset_name (str, optional) – Identifier used to create subdirectories under basic_path. Default: ‘’.

Returns:

np.ndarray – Structure feature tensor of shape (N, L, 5), dtype float, as returned by read_combined_profile.

Notes

The cache existence check currently uses basic_path + ‘/combined_profile.txt’ while the file is written to path + ‘combined_profile.txt’ (where path=basic_path/<dataset_name>/). This behavior is preserved; ensure your basic_path/dataset_name usage matches expectation.
Pairedness is computed as:
P_prob = 1 - E - H - I - M

assuming the four probabilities sum to <= 1 per position.

utils.structureFeatures.build_structure_tensor(structs, max_length)[source]#

Convert comma-separated structure score strings into a padded 3D tensor.

Parameters:

structs (List[str]) – List of comma-separated numeric strings, one per sequence. Example: “0.1,0.2,0.3,…”
max_length (int) – Expected sequence length. Each structs[i] should contain exactly max_length comma-separated values.

Return type:

ndarray

Returns:

np.ndarray – Array of shape (N, 1, max_length), dtype float64 (due to np.zeros default), containing the parsed structure values.

Raises:

ValueError – If a structure string cannot be parsed into floats.
ValueError or broadcasting error – If the number of values is not equal to max_length (assignment will fail).

utils.structureFeatures

Contents

utils.structureFeatures#

Who this is for#

Input / output conventions#

External dependency (RNAplfold only)#

How to use#

Important notes / caveats#