ahcore.utils package

Submodules

ahcore.utils.callbacks module

Ahcore’s callbacks

ahcore.utils.data module

Utilities to describe the dataset to be used and the way it should be parsed.

class ahcore.utils.data.DataDescription(*, mask_label: str | None = None, mask_threshold: float | None = None, roi_name: str | None = None, num_classes: int, data_dir: Path, manifest_database_uri: str, manifest_name: str, split_version: str, annotations_dir: Path, training_grid: GridDescription, inference_grid: GridDescription, index_map: Dict[str, int] | None, remap_labels: Dict[str, str] | None = None, use_class_weights: bool | None = False, convert_mask_to_rois: bool = True, use_roi: bool = True, apply_color_profile: bool = True)

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.

annotations_dir: Path

apply_color_profile: bool

convert_mask_to_rois: bool

data_dir: Path

index_map: Dict[str, int] | None

inference_grid: GridDescription

manifest_database_uri: str

manifest_name: str

mask_label: str | None

mask_threshold: float | None

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'annotations_dir': FieldInfo(annotation=Path, required=True), 'apply_color_profile': FieldInfo(annotation=bool, required=False, default=True), 'convert_mask_to_rois': FieldInfo(annotation=bool, required=False, default=True), 'data_dir': FieldInfo(annotation=Path, required=True), 'index_map': FieldInfo(annotation=Union[Dict[str, int], NoneType], required=True), 'inference_grid': FieldInfo(annotation=GridDescription, required=True), 'manifest_database_uri': FieldInfo(annotation=str, required=True), 'manifest_name': FieldInfo(annotation=str, required=True), 'mask_label': FieldInfo(annotation=Union[str, NoneType], required=False), 'mask_threshold': FieldInfo(annotation=Union[float, NoneType], required=False), 'num_classes': FieldInfo(annotation=int, required=True, metadata=[AfterValidator(func=<function is_positive>)]), 'remap_labels': FieldInfo(annotation=Union[Dict[str, str], NoneType], required=False), 'roi_name': FieldInfo(annotation=Union[str, NoneType], required=False), 'split_version': FieldInfo(annotation=str, required=True), 'training_grid': FieldInfo(annotation=GridDescription, required=True), 'use_class_weights': FieldInfo(annotation=Union[bool, NoneType], required=False, default=False), 'use_roi': FieldInfo(annotation=bool, required=False, default=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

num_classes: PositiveInt

remap_labels: Dict[str, str] | None

roi_name: str | None

split_version: str

training_grid: GridDescription

use_class_weights: bool | None

use_roi: bool

class ahcore.utils.data.GridDescription(*, mpp: float | None, tile_size: Tuple[int, int], tile_overlap: Tuple[int, int], output_tile_size: Tuple[int, int] | None = None)

Bases: BaseModel

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'mpp': FieldInfo(annotation=Union[Annotated[float, AfterValidator], NoneType], required=True), 'output_tile_size': FieldInfo(annotation=Union[Tuple[int, int], NoneType], required=False), 'tile_overlap': FieldInfo(annotation=Tuple[Annotated[int, AfterValidator], Annotated[int, AfterValidator]], required=True), 'tile_size': FieldInfo(annotation=Tuple[Annotated[int, AfterValidator], Annotated[int, AfterValidator]], required=True)}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

mpp: PositiveFloat | None

output_tile_size: Tuple[int, int] | None

tile_overlap: Tuple[NonNegativeInt, NonNegativeInt]

tile_size: Tuple[PositiveInt, PositiveInt]

ahcore.utils.data.basemodel_to_uuid(base_model: BaseModel) → UUID

Create a unique identifier for a pydantic model.

This is done by pickling the object, and computing the sha256 hash of the pickled object and converting this to an UUID. The UUID is generated using the sha256 hash as a namespace, ensuring similar lengths. The chance of a collision is astronomically small.

Returns:

uuid.UUID

A unique identifier for the BaseModel.

ahcore.utils.database_models module

Database models for ahcore’s manifest database.

class ahcore.utils.database_models.Base(**kwargs: Any)

Bases: DeclarativeBase

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

metadata: ClassVar[MetaData] = MetaData()

Refers to the _schema.MetaData collection that will be used for new _schema.Table objects.

See also

orm_declarative_metadata

registry: ClassVar[_RegistryType] = <sqlalchemy.orm.decl_api.registry object>

Refers to the _orm.registry in use where new _orm.Mapper objects will be associated.

class ahcore.utils.database_models.CacheDescription(**kwargs)

Bases: Base

Cache description table.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

cache: Mapped[ImageCache]

created

crop

grid_order

id

last_updated

mask_threshold

mpp

tile_mode

tile_overlap_height

tile_overlap_width

tile_size_height

tile_size_width

class ahcore.utils.database_models.CategoryEnum(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: str, Enum

FIT = 'fit'

PREDICT = 'predict'

TEST = 'test'

VALIDATE = 'validate'

class ahcore.utils.database_models.Image(**kwargs)

Bases: Base

Image table.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

annotations: Mapped[List[ImageAnnotations]]

caches: Mapped[List[ImageCache]]

created

filename

height

id

labels: Mapped[List[ImageLabels]]

last_updated

masks: Mapped[List[Mask]]

mpp

patient: Mapped[Patient]

patient_id

reader

width

class ahcore.utils.database_models.ImageAnnotations(**kwargs)

Bases: Base

Image annotations table.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

created

filename

id

image: Mapped[Image]

image_id

last_updated

reader

class ahcore.utils.database_models.ImageCache(**kwargs)

Bases: Base

Image cache table.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

created

description: Mapped[CacheDescription]

description_id

filename

id

image: Mapped[Image]

image_id

last_updated

num_tiles

reader

class ahcore.utils.database_models.ImageLabels(**kwargs)

Bases: Base

Image labels table.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

created

id

image: Mapped[Image]

image_id

key

last_updated

value

class ahcore.utils.database_models.Manifest(**kwargs)

Bases: Base

Manifest table.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

created

id

last_updated

name

patients: Mapped[List[Patient]]

class ahcore.utils.database_models.Mask(**kwargs)

Bases: Base

Mask table.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

created

filename

id

image: Mapped[Image]

image_id

last_updated

reader

class ahcore.utils.database_models.Patient(**kwargs)

Bases: Base

Patient table.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

created

id

images: Mapped[List[Image]]

labels: Mapped[List[PatientLabels]]

last_updated

manifest: Mapped[Manifest]

manifest_id

patient_code

splits: Mapped[List[Split]]

class ahcore.utils.database_models.PatientLabels(**kwargs)

Bases: Base

Patient labels table.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

created

id

key

last_updated

patient: Mapped[Patient]

patient_id

value

class ahcore.utils.database_models.Split(**kwargs)

Bases: Base

Split table.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

category: Column[CategoryEnum]

created

id

last_updated

patient: Mapped[Patient]

patient_id

split_definition: Mapped[SplitDefinitions]

split_definition_id

class ahcore.utils.database_models.SplitDefinitions(**kwargs)

Bases: Base

Split definitions table.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

created

description

id

last_updated

splits: Mapped[List[Split]]

version

ahcore.utils.io module

Input/ Output utils.

A part of the functions in this module are derived/taken from pytorch lightning template at: ashleve/lightning-hydra-template This template is licensed under the MIT License.

ahcore.utils.io.debug_function(x: int) → int

Function to use for debugging (e.g. github workflow testing) :param x: :return: x^2

ahcore.utils.io.extras(config: DictConfig) → None

A couple of optional utilities, controlled by main config file: - disabling warnings - forcing debug friendly configuration - verifying experiment name is set when running in experiment mode Modifies DictConfig in place. Args:

config (DictConfig): Configuration composed by Hydra.

ahcore.utils.io.fullname(obj: Any) → str

ahcore.utils.io.get_cache_dir() → Path

ahcore.utils.io.get_enum_key_from_value(value: str, enum_class: Type[Enum]) → str | None

ahcore.utils.io.get_logger(name: str = 'ahcore.utils.io') → Logger

Initializes multi-GPU-friendly python command line logger.

ahcore.utils.io.log_hyperparameters(config: DictConfig, model: LightningModule, trainer: Trainer) → None

Controls which config parts are saved by Lightning loggers.

Additionally saves: - number of model parameters

ahcore.utils.io.print_config(config: DictConfig, fields: Sequence[str] = ('trainer', 'model', 'experiment', 'transforms', 'datamodule', 'callbacks', 'logger', 'test_after_training', 'seed', 'name'), resolve: bool = True) → None

Prints content of DictConfig using Rich library and its tree structure. Arguments ——— config : DictConfig

Configuration composed by Hydra.

fieldsSequence[str], optional

Determines which main fields from config will be printed and in what order.

resolvebool, optional

Whether to resolve reference fields of DictConfig.

ahcore.utils.io.validate_config(cfg: Any) → None

ahcore.utils.manifest module

All utilities to parse manifests into datasets. A manifest is a database containing the description of a dataset. See the documentation for more information and examples.

class ahcore.utils.manifest.DataManager(database_uri: str)

Bases: object

close() → None

get_image_by_filename(filename: str) → Image

Fetch the metadata for an image based on its filename.

Parameters:

filenamestr

The filename of the image.

Returns:

Image

The image from the database.

get_image_metadata_by_id(image_id: int) → ImageMetadata

Fetch the metadata for an image based on its ID.

Parameters:

image_idint

The ID of the image.

Returns:

ImageMetadata

Metadata of the image.

get_image_metadata_by_patient(patient_code: str) → list[ahcore.utils.manifest.ImageMetadata]

Fetch the metadata for the images associated with a specific patient.

Parameters:

patient_codestr

The unique code of the patient.

Returns:

list[ImageData]

A list of metadata for all images associated with the patient.

get_image_metadata_by_split(manifest_name: str, split_version: str, split_category: str | None = None) → Generator[ImageMetadata, None, None]

Yields the metadata of images for a given manifest name, split version, and optional split category.

Parameters:

manifest_namestr

The name of the manifest.

split_versionstr

The version of the split.

split_categoryOptional[str], default=None

The category of the split (e.g., “fit”, “validate”, “test”).

Yields:

ImageMetadata

The metadata of the image.

get_records_by_split(manifest_name: str, split_version: str, split_category: str | None = None) → Generator[Patient, None, None]

class ahcore.utils.manifest.ImageMetadata(*, filename: Path, height: int, width: int, mpp: float)

Bases: BaseModel

Model to hold image metadata

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

__init__ uses __pydantic_self__ instead of the more common self for the first arg to allow self as a field name.

class Config

Bases: object

frozen = True

filename: Path

height: PositiveInt

model_config: ClassVar[ConfigDict] = {'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'filename': FieldInfo(annotation=Path, required=True), 'height': FieldInfo(annotation=int, required=True, metadata=[AfterValidator(func=<function is_positive>)]), 'mpp': FieldInfo(annotation=float, required=True, metadata=[AfterValidator(func=<function is_positive>)]), 'width': FieldInfo(annotation=int, required=True, metadata=[AfterValidator(func=<function is_positive>)])}

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

mpp: PositiveFloat

width: PositiveInt

ahcore.utils.manifest.create_tables(engine: Engine) → None

Create the database tables.

ahcore.utils.manifest.datasets_from_data_description(db_manager: DataManager, data_description: DataDescription, transform: Callable[[TileSample], RegionFromWsiDatasetSample] | None, stage: str) → Generator[TiledWsiDataset, None, None]

ahcore.utils.manifest.fetch_image_metadata(image: Image) → ImageMetadata

Extract metadata from an Image object.

ahcore.utils.manifest.get_labels_from_record(record: Image | Patient) → list[tuple[str, str]] | None

Get the labels from a record of type Image or Patient.

Parameters:

recordImage | Patient

The record containing the labels.

Returns:

list[tuple[str, str]] | None

The labels if they exists, else None.

ahcore.utils.manifest.get_mask_and_annotations_from_record(annotations_root: Path, record: Image) → tuple[dlup.annotations.WsiAnnotations | dlup._image.SlideImage | None, dlup.annotations.WsiAnnotations | dlup._image.SlideImage | None]

Get the mask and annotations from a record of type Image.

Parameters:

annotations_rootPath

The root directory of the annotations.

recordType[Image]

The record containing the mask and annotations.

Returns:

tuple[WsiAnnotations, WsiAnnotations]

The mask and annotations.

ahcore.utils.manifest.open_db(database_uri: str) → Session

Open a database connection.

Parameters:

database_uristr

The URI of the database.

Returns:

Session

The database session.

ahcore.utils.manifest.parse_annotations_from_record(annotations_root: Path, record: list[ahcore.utils.database_models.Mask] | list[ahcore.utils.database_models.ImageAnnotations]) → WsiAnnotations | SlideImage | None

Parse the annotations from a record of type ImageAnnotations.

Parameters:

annotations_rootPath

The root directory of the annotations.

recordlist[Type[ImageAnnotations]]

The record containing the annotations.

Returns:

WsiAnnotations

The parsed annotations.

ahcore.utils.rois module

Utilities to work with regions-of-interest. Has utilities to compute balanced ROIs from masks

ahcore.utils.rois.compute_rois(mask: WsiAnnotations, tile_size: tuple[int, int], tile_overlap: tuple[int, int], centered: bool = True) → list[tuple[tuple[int, int], tuple[int, int]]]

Compute the ROIs from a WsiAnnotations object. The ROIs are computed by:

1. The bounding box of the whole mask object is computed (by dlup).

2. The whole region up to the bounding box is computed.

3. For each of the regions in the result, the bounding box is determined.

If centered is True, the bounding box is centered in the region. This is done as follows:

1) The effective region size is computed, depending on the tiling mode. This is the size of the region which is tiled by the given tile size and overlap. Tiles always overflow, so there are tiles on the border which are partially covered. 2) The ROIs obtained in the first step are replaced by the effective region size, and centered around the original ROI.

Parameters:

maskWsiAnnotations

tile_sizetuple[int, int]

tile_overlaptuple[int, int]

centeredbool

Returns:

list[tuple[tuple[int, int], tuple[int, int]]]

List of ROIs (coordinates, size).

ahcore.utils.types module

class ahcore.utils.types.InferencePrecision(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: str, Enum

FP16 = 'float16'

FP32 = 'float32'

UINT8 = 'uint8'

get_multiplier() → ndarray[Any, dtype[generic]]

class ahcore.utils.types.NormalizationType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: str, Enum

LOGITS = 'logits'

SIGMOID = 'sigmoid'

SOFTMAX = 'softmax'

normalize() → Callable[[...], Any]

ahcore.utils.types.is_non_negative(v: int | float) → int | float

ahcore.utils.types.is_positive(v: int | float) → int | float

Module contents