mindpose.data¶
- mindpose.data.create_dataset(image_root, annotation_file=None, dataset_format='coco_topdown', is_train=True, device_num=None, rank_id=None, num_workers=1, config=None, **kwargs)[source]¶
Create dataset for training or evaluation.
- Parameters:
image_root (
str
) – The path of the directory storing imagesannotation_file (
Optional
[str
]) – The path of the annotation file. Default: Nonedataset_format (
str
) – The dataset format. Different format yield different final output. Default: coco_topdownis_train (
bool
) – Wether this dataset is used for training/testing: Default: Truedevice_num (
Optional
[int
]) – Number of devices (e.g. GPU). Default: Nonerank_id (
Optional
[int
]) – Current process’s rank id. Default: Nonenum_workers (
int
) – Number of workers in reading data. Default: 1config (
Optional
[Dict
[str
,Any
]]) – Dataset-specific configurationuse_gt_bbox_for_val – Use GT bbox instead of detection result during evaluation. Default: False
detection_file – Path of the detection result. Default: None
- Return type:
GeneratorDataset
- Returns:
Dataset for training or evaluation
- mindpose.data.create_pipeline(dataset, transforms, method='topdown', batch_size=1, is_train=True, normalize=True, normalize_mean=[0.485, 0.456, 0.406], normalize_std=[0.229, 0.224, 0.255], hwc_to_chw=True, num_workers=1, config=None)[source]¶
Create dataset tranform pipeline. The returned datatset is transformed sequentially based on the given list of transforms.
- Parameters:
dataset (
Dataset
) – Dataset to perform transformationstransforms (
List
[Union
[str
,Dict
[str
,Any
]]]) – List of transformationsmethod (
str
) – The method to use. Default: “topdown”batch_size (
int
) – Batch size. Default: 1is_train (
bool
) – Whether the transformation is for training/testing. Default: Truenormalize (
bool
) – Perform normalization. Default: Truenormalize_mean (
List
[float
]) – Mean of the normalization: Default: [0.485, 0.456, 0.406]normalize_std (
List
[float
]) – Std of the normalization: Default: [0.229, 0.224, 0.255]hwc_to_chw (
bool
) – Wwap height x width x channel to channel x height x width. Default: Truenum_workers (
int
) – Number of workers in processing data. Default: 1config (
Optional
[Dict
[str
,Any
]]) – Transform-specific configuration
- Return type:
Dataset
- Returns:
The transformed dataset
mindpose.data.dataset¶
- class mindpose.data.dataset.BottomUpDataset(image_root, annotation_file=None, is_train=False, num_joints=17, config=None)[source]¶
Bases:
object
Create an iterator for ButtomUp dataset, return the tuple with (image, boxes, keypoints, target, mask, tag_ind) for training; return the tuple with (image, mask, center, scale, image_file, image_shape) for evaluation.
- Parameters:
image_root (
str
) – The path of the directory storing imagesannotation_file (
Optional
[str
]) – The path of the annotation file. Default: Noneis_train (
bool
) – Wether this dataset is used for training/testing. Default: Falsenum_joints (
int
) – Number of joints in the dataset. Default: 17config (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: None
- Items in iterator:
- image: Encoded data for image filekeypoints: Keypoints in (x, y, visibility)mask: Mask of the image showing the valid annotationstarget: A placeholder for later pipline usingtag_ind: A placeholder of later pipline usingimage_file: Path of the image fileboxes: Bounding box coordinate (x0, y0), (x1, y1)
Note
This is an abstract class, child class must implement load_dataset_cfg and load_dataset method.
- load_dataset()[source]¶
Loading the dataset, where the returned record should contain the following key
- Keys:
- image_file: Path of the image file.keypoints (For training only): Keypoints in (x, y, visibility).boxes (For training only): Bounding box coordinate (x0, y0), (x1, y1).mask_info (For training only): The mask info of crowed or zero keypoints instances.
- Return type:
List
[Dict
[str
,Any
]]- Returns:
A list of records of groundtruth or predictions
- class mindpose.data.dataset.COCOBottomUpDataset(image_root, annotation_file=None, is_train=False, num_joints=17, config=None)[source]¶
Bases:
BottomUpDataset
Create an iterator for ButtomUp dataset, return the tuple with (image, boxes, keypoints, mask, target, tag_ind) for training; return the tuple with (image, mask, center, scale, image_file, image_shape) for evaluation.
- Parameters:
image_root (
str
) – The path of the directory storing imagesannotation_file (
Optional
[str
]) – The path of the annotation file. Default: Noneis_train (
bool
) – Wether this dataset is used for training/testing. Default: Falsenum_joints (
int
) – Number of joints in the dataset. Default: 17config (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: None
- Items in iterator:
- image: Encoded data for image filekeypoints: Keypoints in (x, y, visibility)mask: Mask of the image showing the valid annotationstarget: A placeholder for later pipline usingkeypoints_coordinate: A placeholder of later pipline usingimage_file: Path of the image fileboxes: Bounding box coordinate (x0, y0), (x1, y1)
- load_dataset()[source]¶
Loading the dataset, where the returned record should contain the following key
- Keys:
- image_file: Path of the image file.keypoints (For training only): Keypoints in (x, y, visibility).boxes (For training only): Bounding box coordinate (x0, y0), (x1, y1).mask_info (For training only): The mask info of crowed or zero keypoints instances.
- Return type:
List
[Dict
[str
,Any
]]- Returns:
A list of records of groundtruth or predictions
- class mindpose.data.dataset.COCOTopDownDataset(image_root, annotation_file=None, is_train=False, num_joints=17, use_gt_bbox_for_val=False, detection_file=None, config=None)[source]¶
Bases:
TopDownDataset
Create an iterator for TopDown dataset based COCO annotation format. return the tuple with (image, center, scale, keypoints, rotation, target, target_weight) for training; return the tuple with (image, center, scale, rotation, image_file, boxes, bbox_ids, bbox_score) for evaluation.
- Parameters:
image_root (
str
) – The path of the directory storing imagesannotation_file (
Optional
[str
]) – The path of the annotation file. Default: Noneis_train (
bool
) – Wether this dataset is used for training/testing. Default: Falsenum_joints (
int
) – Number of joints in the dataset. Default: 17use_gt_bbox_for_val (
bool
) – Use GT bbox instead of detection result during evaluation. Default: Falsedetection_file (
Optional
[str
]) – Path of the detection result. Defaul: Noneconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: None
- Item in iterator:
- image: Encoded data for image filecenter: A placeholder for later pipline usingscale: A placeholder of later pipline usingkeypoints: Keypoints in (x, y, visibility)rotation: Rotatated degreetarget: A placeholder for later pipline usingtarget_weight: A placeholder of later pipline usingimage_file: Path of the image fileboxes: Bounding box coordinate (x, y, w, h)bbox_id: Bounding box id for each single imagebbox_score: Bounding box score, 1 for ground truth
- load_dataset()[source]¶
Loading the dataset, where the returned record should contain the following key
- Keys:
- image_file: Path of the image filebbox: Bounding box coordinate (x, y, w, h)keypoints: Keypoints in [K, 3(x, y, visibility)]bbox_score: Bounding box score, 1 for ground truthbbox_id: Bounding box id for each single image
- Return type:
List
[Dict
[str
,Any
]]- Returns:
A list of records of groundtruth or predictions
- class mindpose.data.dataset.ImageFolderBottomUpDataset(image_root, annotation_file=None, is_train=False, num_joints=17, config=None)[source]¶
Bases:
BottomUpDataset
Create an iterator for ButtomUp dataset based on image folder. It is usually used for demo usage. Return the tuple with (image, mask, center, scale, image_file, image_shape)
- Parameters:
image_root (
str
) – The path of the directory storing imagesannotation_file (
Optional
[str
]) – The path of the annotation file. Default: Noneis_train (
bool
) – Wether this dataset is used for training/testing. Default: Falsenum_joints (
int
) – Number of joints in the dataset. Default: 17config (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: None
- class mindpose.data.dataset.TopDownDataset(image_root, annotation_file=None, is_train=False, num_joints=17, use_gt_bbox_for_val=False, detection_file=None, config=None)[source]¶
Bases:
object
Create an iterator for TopDown dataset, return the tuple with (image, center, scale, keypoints, rotation, target, target_weight) for training; return the tuple with (image, center, scale, rotation, image_file, boxes, bbox_ids, bbox_score) for evaluation.
- Parameters:
image_root (
str
) – The path of the directory storing imagesannotation_file (
Optional
[str
]) – The path of the annotation file. Default: Noneis_train (
bool
) – Wether this dataset is used for training/testing. Default: Falsenum_joints (
int
) – Number of joints in the dataset. Default: 17use_gt_bbox_for_val (
bool
) – Use GT bbox instead of detection result during evaluation. Default: Falsedetection_file (
Optional
[str
]) – Path of the detection result. Default: Noneconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: None
- Item in iterator:
- image: Encoded data for image filecenter: A placeholder for later pipline usingscale: A placeholder of later pipline usingkeypoints: Keypoints in [K, 3(x, y, visibility)]rotation: Rotatated degreetarget: A placeholder for later pipline usingtarget_weight: A placeholder of later pipline usingimage_file: Path of the image filebbox: Bounding box coordinate (x, y, w, h)bbox_id: Bounding box id for each single imagebbox_score: Bounding box score, 1 for ground truth
Note
This is an abstract class, child class must implement load_dataset_cfg and load_dataset method.
- load_dataset()[source]¶
Loading the dataset, where the returned record should contain the following key
- Keys:
- image_file: Path of the image filebbox: Bounding box coordinate (x, y, w, h)keypoints: Keypoints in [K, 3(x, y, visibility)]bbox_score: Bounding box score, 1 for ground truthbbox_id: Bounding box id for each single image
- Return type:
List
[Dict
[str
,Any
]]- Returns:
A list of records of groundtruth or predictions
mindpose.data.transform¶
- class mindpose.data.transform.BottomUpGenerateTarget(is_train=True, config=None, sigma=2.0, max_num=30)[source]¶
Bases:
BottomUpTransform
Generate heatmap with the keypoint coordinatess with multiple scales.
- Parameters:
is_train (
bool
) – Whether the transformation is for training/testing. Default: Trueconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: Nonesigma (
float
) – The sigmal size of gausian distribution. Default: 2.0max_num (
int
) – Maximum number of instances within the image. Default: 30
- Inputs:
- data: Data tuples need to be transformed
- Outputs:
- result: Transformed data tuples
- transform(state)[source]¶
Transform the state into the transformed state. state is a dictionay storing the informaton of the image and labels, the returned states is the updated dictionary storing the updated image and labels.
- Parameters:
state (
Dict
[str
,Any
]) – Stored information of image and labels- Return type:
Dict
[str
,Any
]- Returns:
Updated inforamtion of image and labels based on the transformation
Note
Required keys for transform: keypointsReturned keys after transform: target, tag_ind
- class mindpose.data.transform.BottomUpHorizontalRandomFlip(is_train=True, config=None, flip_prob=0.5)[source]¶
Bases:
BottomUpTransform
Perform randomly horizontal flip in bottomup approach.
- Parameters:
is_train (
bool
) – Whether the transformation is for training/testing. Default: Trueconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: Noneflip_prob (
float
) – Probability of performing a horizontal flip. Default: 0.5
- transform(state)[source]¶
Transform the state into the transformed state. state is a dictionay storing the informaton of the image and labels, the returned states is the updated dictionary storing the updated image and labels.
- Parameters:
state (
Dict
[str
,Any
]) – Stored information of image and labels- Return type:
Dict
[str
,Any
]- Returns:
Updated inforamtion of image and labels based on the transformation
Note
Required keys for transform: image, mask, keypointsReturned keys after transform: image, mask, keypoints
- class mindpose.data.transform.BottomUpPad(is_train=True, config=None)[source]¶
Bases:
BottomUpTransform
Padding the image to the max_image_size.
- Parameters:
is_train (
bool
) – Whether the transformation is for training/testing. Default: Trueconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: None
- transform(state)[source]¶
Transform the state into the transformed state. state is a dictionay storing the informaton of the image and labels, the returned states is the updated dictionary storing the updated image and labels.
- Parameters:
state (
Dict
[str
,Any
]) – Stored information of image and labels- Return type:
Dict
[str
,Any
]- Returns:
Updated inforamtion of image and labels based on the transformation
Note
Required keys for transform: imageReturned keys after transform: image, mask
- class mindpose.data.transform.BottomUpRandomAffine(is_train=True, config=None, rot_factor=30.0, scale_factor=(0.75, 1.5), scale_type='short', trans_factor=40.0)[source]¶
Bases:
BottomUpTransform
Random affine transform the image. The mask and keypoints will be rescaled to the heatmap sizes after the transformation.
- Parameters:
is_train (
bool
) – Whether the transformation is for training/testing. Default: Trueconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: Nonerot_factor (
float
) – Randomly rotated in [-rotation_factor, rotation_factor]. Default: 30.scale_factor (
Tuple
[float
,float
]) – Randomly Randomly scaled in [scale_factor[0], scale_factor[1]]. Default: (0.75, 1.5)scale_type (
str
) – Scaling with the long / short length of the image. Default: shorttrans_factor (
float
) – Translation factor. Default: 40.
- transform(state)[source]¶
Transform the state into the transformed state. state is a dictionay storing the informaton of the image and labels, the returned states is the updated dictionary storing the updated image and labels.
- Parameters:
state (
Dict
[str
,Any
]) – Stored information of image and labels- Return type:
Dict
[str
,Any
]- Returns:
Updated inforamtion of image and labels based on the transformation
Note
Required keys for transform: image, mask, keypointsReturned keys after transform: image, mask, keypoints
- class mindpose.data.transform.BottomUpRescale(is_train=True, config=None)[source]¶
Bases:
BottomUpTransform
Rescaling the image to the max_image_size without change the aspect ratio.
- Parameters:
is_train (
bool
) – Whether the transformation is for training/testing. Default: Trueconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: None
- transform(state)[source]¶
Transform the state into the transformed state. state is a dictionay storing the informaton of the image and labels, the returned states is the updated dictionary storing the updated image and labels.
- Parameters:
state (
Dict
[str
,Any
]) – Stored information of image and labels- Return type:
Dict
[str
,Any
]- Returns:
Updated inforamtion of image and labels based on the transformation
Note
Required keys for transform: imageReturned keys after transform: image, center, scale, image_shape
- class mindpose.data.transform.BottomUpResize(is_train=True, config=None, size=512, base_length=64)[source]¶
Bases:
BottomUpTransform
Resize the image without change the aspect ratio. The length of the short side of the image will be equal to the input size.
- Parameters:
is_train (
bool
) – Whether the transformation is for training/testing. Default: Trueconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: Nonesize (
int
) – The target size of the short side of the image. Default: 512base_length (
int
) – The minimum size the image. Default: 64
- transform(state)[source]¶
Transform the state into the transformed state. state is a dictionay storing the informaton of the image and labels, the returned states is the updated dictionary storing the updated image and labels.
- Parameters:
state (
Dict
[str
,Any
]) – Stored information of image and labels- Return type:
Dict
[str
,Any
]- Returns:
Updated inforamtion of image and labels based on the transformation
Note
Required keys for transform: imageReturned keys after transform: image, mask, center, scale, image_shape
- class mindpose.data.transform.BottomUpTransform(is_train=True, config=None)[source]¶
Bases:
Transform
Transform the input data into the output data based on bottom-up approach.
- Parameters:
is_train (
bool
) – Whether the transformation is for training/testing. Default: Trueconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: None
- Inputs:
data: Data tuples need to be transformed
- Outputs:
result: Transformed data tuples
Note
This is an abstract class, child class must implement transform method.
- class mindpose.data.transform.TopDownAffine(is_train=True, config=None, use_udp=False)[source]¶
Bases:
TopDownTransform
Affine transform the image, and the transform image will contain single instance only.
- Parameters:
is_train (
bool
) – Whether the transformation is for training/testing. Default: Trueconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: Noneuse_udp (
bool
) – Use Unbiased Data Processing (UDP) affine transform. Default: False
- Inputs:
- data: Data tuples need to be transformed
- Outputs:
- result: Transformed data tuples
- transform(state)[source]¶
Transform the state into the transformed state. state is a dictionay storing the informaton of the image and labels, the returned states is the updated dictionary storing the updated image and labels.
- Parameters:
state (
Dict
[str
,Any
]) – Stored information of image and labels- Return type:
Dict
[str
,Any
]- Returns:
Updated inforamtion of image and labels based on the transformation
Note
Required keys for transform: image, center, scale, rotation, keypoints (optional)Returned keys after transform: image, keypoints (optional)
- class mindpose.data.transform.TopDownBoxToCenterScale(is_train=True, config=None)[source]¶
Bases:
TopDownTransform
Convert the box coordinate to center and scale. If is_train is True, the center will be randomly shifted by a small amount.
- Parameters:
is_train (
bool
) – Whether the transformation is for training/testing. Default: Trueconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: None
- Inputs:
- data: Data tuples need to be transformed
- Outputs:
- result: Transformed data tuples
- transform(state)[source]¶
Transform the state into the transformed state. state is a dictionay storing the informaton of the image and labels, the returned states is the updated dictionary storing the updated image and labels.
- Parameters:
state (
Dict
[str
,Any
]) – Stored information of image and labels- Return type:
Dict
[str
,Any
]- Returns:
Updated inforamtion of image and labels based on the transformation
Note
Required keys for transform: boxesReturned keys after transform: center, scale
- class mindpose.data.transform.TopDownGenerateTarget(is_train=True, config=None, sigma=2.0, use_different_joint_weights=False, use_udp=False)[source]¶
Bases:
TopDownTransform
Generate heatmap from the coordinates.
- Parameters:
is_train (
bool
) – Whether the transformation is for training/testing. Default: Trueconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: Nonesigma (
float
) – The sigmal size of gausian distribution. Default: 2.0use_different_joint_weights (
bool
) – Use extra joint weight in target weight calculation. Default: Falseuse_udp (
bool
) – Use Unbiased Data Processing (UDP) encoding. Default: False
- Inputs:
- data: Data tuples need to be transformed
- Outputs:
- result: Transformed data tuples
- transform(state)[source]¶
Transform the state into the transformed state. state is a dictionay storing the informaton of the image and labels, the returned states is the updated dictionary storing the updated image and labels.
- Parameters:
state (
Dict
[str
,Any
]) – Stored information of image and labels- Return type:
Dict
[str
,Any
]- Returns:
Updated inforamtion of image and labels based on the transformation
Note
Required keys for transform: keypointsReturned keys after transform: target, target_weight
- class mindpose.data.transform.TopDownHalfBodyTransform(is_train=True, config=None, num_joints_half_body=8, prob_half_body=0.3, scale_padding=1.5)[source]¶
Bases:
TopDownTransform
Perform half-body transform. Keep only the upper body or the lower body at random.
- Parameters:
is_train (
bool
) – Whether the transformation is for training/testing. Default: Trueconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: Nonenum_joints_half_body (
int
) – Threshold number of performing half-body transform. Default: 8prob_half_body (
float
) – Probability of performing half-body transform. Default: 0.3scale_padding (
float
) – Extra scale padding multiplier in generating the cropped images. Default: 1.5
- Inputs:
- data: Data tuples need to be transformed
- Outputs:
- result: Transformed data tuples
- transform(state)[source]¶
Transform the state into the transformed state. state is a dictionay storing the informaton of the image and labels, the returned states is the updated dictionary storing the updated image and labels.
- Parameters:
state (
Dict
[str
,Any
]) – Stored information of image and labels- Return type:
Dict
[str
,Any
]- Returns:
Updated inforamtion of image and labels based on the transformation
Note
Required keys for transform: keypointsReturned keys after transform: center, scale
- class mindpose.data.transform.TopDownHorizontalRandomFlip(is_train=True, config=None, flip_prob=0.5)[source]¶
Bases:
TopDownTransform
Perform randomly horizontal flip in topdown approach.
- Parameters:
is_train (
bool
) – Whether the transformation is for training/testing. Default: Trueconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: Noneflip_prob (
float
) – Probability of performing a horizontal flip. Default: 0.5
- Inputs:
- data: Data tuples need to be transformed
- Outputs:
- result: Transformed data tuples
- transform(state)[source]¶
Transform the state into the transformed state. state is a dictionay storing the informaton of the image and labels, the returned states is the updated dictionary storing the updated image and labels.
- Parameters:
state (
Dict
[str
,Any
]) – Stored information of image and labels- Return type:
Dict
[str
,Any
]- Returns:
Updated inforamtion of image and labels based on the transformation
Note
Required keys for transform: image, keypoints, centerReturned keys after transform: image, keypoints, center
- class mindpose.data.transform.TopDownRandomScaleRotation(is_train=True, config=None, rot_factor=40.0, scale_factor=0.5, rot_prob=0.6)[source]¶
Bases:
TopDownTransform
Perform random scaling and rotation.
- Parameters:
is_train (
bool
) – Whether the transformation is for training/testing. Default: Trueconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: Nonerot_factor (
float
) – Std of rotation degree. Default: 40.scale_factor (
float
) – Std of scaling value. Default: 0.5rot_prob (
float
) – Probability of performing rotation. Default: 0.6
- Inputs:
- data: Data tuples need to be transformed
- Outputs:
- result: Transformed data tuples
- transform(state)[source]¶
Transform the state into the transformed state. state is a dictionay storing the informaton of the image and labels, the returned states is the updated dictionary storing the updated image and labels.
- Parameters:
state (
Dict
[str
,Any
]) – Stored information of image and labels- Return type:
Dict
[str
,Any
]- Returns:
Updated inforamtion of image and labels based on the transformation
Note
Required keys for transform: scaleReturned keys after transform: scale, rotation
- class mindpose.data.transform.TopDownTransform(is_train=True, config=None)[source]¶
Bases:
Transform
Transform the input data into the output data based on top-down approach.
- Parameters:
is_train (
bool
) – Whether the transformation is for training/testing. Default: Trueconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: None
- Inputs:
data: Data tuples need to be transformed
- Outputs:
result: Transformed data tuples
Note
This is an abstract class, child class must implement transform method.
- class mindpose.data.transform.Transform(is_train=True, config=None)[source]¶
Bases:
object
Transform the input data into the output data.
- Parameters:
is_train (
bool
) – Whether the transformation is for training/testing. Default: Trueconfig (
Optional
[Dict
[str
,Any
]]) – Method-specific configuration. Default: None
- Inputs:
data: Data tuples need to be transformed
- Outputs:
result: Transformed data tuples
Note
This is an abstract class, child class must implement load_transform_cfg, transform and setup_required_field method.
- load_transform_cfg()[source]¶
Loading the transform config, where the returned the config must be a dictionary which stores the configuration of this transformation, such as the transformed image size, etc.
- Return type:
Dict
[str
,Any
]- Returns:
Transform configuration
- setup_required_field()[source]¶
Get the required columns names used for this transformation. The columns names will be later used with Minspore Dataset map func.
- Return type:
List
[str
]- Returns:
The column names
- transform(state)[source]¶
Transform the state into the transformed state. state is a dictionay storing the informaton of the image and labels, the returned states is the updated dictionary storing the updated image and labels.
- Parameters:
state (
Dict
[str
,Any
]) – Stored information of image and labels- Return type:
Dict
[str
,Any
]- Returns:
Updated inforamtion of image and labels based on the transformation