artist.core =========== .. py:module:: artist.core .. autoapi-nested-parse:: Bundle all classes responsible for the core functions in ``ARTIST``. Submodules ---------- .. toctree:: :maxdepth: 1 /autoapi/artist/core/core_utils/index /autoapi/artist/core/heliostat_ray_tracer/index /autoapi/artist/core/kinematic_reconstructor/index /autoapi/artist/core/learning_rate_schedulers/index /autoapi/artist/core/loss_functions/index /autoapi/artist/core/motor_position_optimizer/index /autoapi/artist/core/regularizers/index /autoapi/artist/core/surface_reconstructor/index Classes ------- .. autoapisummary:: artist.core.DistortionsDataset artist.core.HeliostatRayTracer artist.core.RestrictedDistributedSampler artist.core.KinematicReconstructor artist.core.MotorPositionsOptimizer artist.core.SurfaceReconstructor Package Contents ---------------- .. py:class:: DistortionsDataset(light_source: artist.scene.LightSource, number_of_points_per_heliostat: int, number_of_heliostats: int, random_seed: int = 7) Bases: :py:obj:`torch.utils.data.Dataset` Initialize the dataset. This class implements a custom dataset according to the ``torch`` interface. The content of this dataset are the distortions. The distortions are used in our version of "heliostat"-tracing to indicate how each incoming ray must be multiplied and scattered on the heliostat. According to ``torch``, this dataset must implement a function to return the length of the dataset and one function to retrieve an element through an index. Parameters ---------- light_source : LightSource The light source used to model the distortions. number_of_points_per_heliostat : int The number of points on the heliostats for which distortions are created. number_of_heliostats : int The number of heliostats in the scenario. random_seed : int The random seed used for generating the distortions (default is 7). .. py:method:: __len__() -> int Calculate the length of the dataset, i.e., the number of items contained. Returns ------- int The length of the dataset. .. py:method:: __getitem__(idx: int) -> tuple[torch.Tensor, torch.Tensor] Select an item from the dataset. Parameters ---------- idx : int The index of the item to select. Returns ------- torch.Tensor The distortions in the up direction for the given index. torch.Tensor The distortions in the east direction for the given index. .. py:class:: HeliostatRayTracer(scenario: artist.scenario.scenario.Scenario, heliostat_group: artist.field.heliostat_group.HeliostatGroup, world_size: int = 1, rank: int = 0, batch_size: int = 100, random_seed: int = 7, bitmap_resolution: torch.Tensor = torch.tensor([artist.util.index_mapping.bitmap_resolution, artist.util.index_mapping.bitmap_resolution])) Initialize the heliostat ray tracer. "Heliostat"-tracing is one kind of ray tracing applied in ``ARTIST``. For this kind of ray tracing, the rays are initialized on the heliostats. The rays originate in the discrete surface points. There they are multiplied, distorted, and scattered, and then they are sent to the aim points. Letting the rays originate on the heliostats, drastically reduces the number of rays that need to be traced. Parameters ---------- scenario : Scenario The scenario used to perform ray tracing. heliostat_group : HeliostatGroup The selected heliostat group containing active heliostats. world_size : int The world size i.e., the overall number of processes (default is 1). rank : int The rank, i.e., individual process ID (default is 0). batch_size : int The amount of samples (heliostats) processed in parallel within a single rank (default is 100). random_seed : int The random seed used for generating the distortions (default is 7). bitmap_resolution : torch.Tensor The resolution of the bitmap in both directions. (default is torch.tensor([256,256])). Tensor of shape [2]. .. py:attribute:: scenario .. py:attribute:: heliostat_group .. py:attribute:: world_size :value: 1 .. py:attribute:: rank :value: 0 .. py:attribute:: batch_size :value: 100 .. py:attribute:: light_source .. py:attribute:: distortions_dataset .. py:attribute:: distortions_sampler .. py:attribute:: distortions_loader .. py:attribute:: bitmap_resolution .. py:method:: trace_rays(incident_ray_directions: torch.Tensor, active_heliostats_mask: torch.Tensor, target_area_mask: torch.Tensor, device: torch.device | None = None) -> torch.Tensor Perform heliostat ray tracing. Scatter the rays according to the distortions, calculate the intersections with the target planes, and sample the resulting bitmaps on the target areas. The bitmaps are generated separately for each active heliostat and can be accessed individually or they can be combined to get the total flux density distribution for all heliostats on all target areas. Parameters ---------- incident_ray_directions : torch.Tensor The direction of the incident rays as seen from the heliostats. Tensor of shape [number_of_active_heliostats, 4]. active_heliostats_mask : torch.Tensor A mask where 0 indicates a deactivated heliostat and 1 an activated one. An integer greater than 1 indicates that this heliostat is regarded multiple times. Tensor of shape [number_of_heliostats]. target_area_mask : torch.Tensor The indices of the target areas for each active heliostat. Tensor of shape [number_of_active_heliostats]. device : torch.device | None The device on which to perform computations or load tensors and models (default is None). If None, ``ARTIST`` will automatically select the most appropriate device (CUDA or CPU) based on availability and OS. Raises ------ ValueError If not all heliostats used for ray tracing have been aligned. Returns ------- torch.Tensor The resulting bitmaps per heliostat. Tensor of shape [number_of_active_heliostats, bitmap_resolution_e, bitmap_resolution_u]. .. py:method:: scatter_rays(distortion_u: torch.Tensor, distortion_e: torch.Tensor, original_ray_direction: torch.Tensor, device: torch.device | None = None) -> artist.scene.rays.Rays Scatter the reflected rays around the preferred ray directions for each heliostat. Parameters ---------- distortion_u : torch.Tensor The distortions in up direction (angles for scattering). Tensor of shape [number_of_active_heliostats, number_of_rays, number_of_combined_surface_normals_all_facets]. distortion_e : torch.Tensor The distortions in east direction (angles for scattering). Tensor of shape [number_of_active_heliostats, number_of_rays, number_of_combined_surface_normals_all_facets]. original_ray_direction : torch.Tensor The ray direction around which to scatter. Tensor of shape [number_of_active_heliostats, number_of_combined_surface_normals_all_facets, 4]. device : torch.device | None The device on which to perform computations or load tensors and models (default is None). If None, ``ARTIST`` will automatically select the most appropriate device (CUDA or CPU) based on availability and OS. Returns ------- Rays Scattered rays around the preferred reflection directions. .. py:method:: sample_bitmaps(intersections: torch.Tensor, absolute_intensities: torch.Tensor, active_heliostats_mask: torch.Tensor, target_area_mask: torch.Tensor, device: torch.device | None = None) -> torch.Tensor Sample bitmaps (flux density distributions) of the reflected rays on the target areas. The bitmaps are saved for each active heliostat separately. Parameters ---------- intersections : torch.Tensor The intersections of rays on the target area planes for each heliostat. Tensor of shape [number_of_active_heliostats, number_of_rays, number_of_combined_surface_points_all_facets, 4]. absolute_intensities : torch.Tensor The absolute intensities of the rays hitting the target planes for each heliostat. Tensor of shape [number_of_active_heliostats, number_of_rays, number_of_combined_surface_points_all_facets]. active_heliostats_mask : torch.Tensor Used to map bitmaps per heliostat to correct index. Tensor of shape [number_of_heliostats]. target_area_mask : torch.Tensor The indices of target areas on which each heliostat is raytraced. Tensor of shape [number_of_active_heliostats]. device : torch.device | None The device on which to perform computations or load tensors and models (default is None). If None, ``ARTIST`` will automatically select the most appropriate device (CUDA or CPU) based on availability and OS. Returns ------- torch.Tensor The flux density distributions of the reflected rays on the target areas for each active heliostat. Tensor of shape [number_of_active_heliostats, bitmap_resolution_e, bitmap_resolution_u]. .. py:method:: get_bitmaps_per_target(bitmaps_per_heliostat: torch.Tensor, target_area_mask: torch.Tensor, device: torch.device | None = None) -> torch.Tensor Transform bitmaps per heliostat to bitmaps per target area. Parameters ---------- bitmaps_per_heliostat : torch.Tensor Bitmaps per heliostat. Tensor of shape [number_of_active_heliostats, bitmap_resolution_e, bitmap_resolution_u]. target_area_mask : torch.Tensor The mapping from heliostat to target area. Tensor of shape [number_of_active_heliostats]. device : torch.device | None The device on which to perform computations or load tensors and models (default is None). If None, ``ARTIST`` will automatically select the most appropriate device (CUDA or CPU) based on availability and OS. Returns ------- torch.Tensor Bitmaps per target area. Tensor of shape [number_of_target_areas, bitmap_resolution_e, bitmap_resolution_u]. .. py:class:: RestrictedDistributedSampler(number_of_samples: int, world_size: int = 1, rank: int = 0) Bases: :py:obj:`torch.utils.data.Sampler` Set up a custom distributed sampler to assign data to each rank or leave them idle. Parameters ---------- number_of_samples : int The length of the dataset or total number of samples. world_size : int The world size or total number of processes (default is 1). rank : int The rank of the current process (default is 0). .. py:attribute:: number_of_samples .. py:attribute:: world_size :value: 1 .. py:attribute:: rank :value: 0 .. py:attribute:: number_of_active_ranks .. py:attribute:: number_of_samples_per_rank .. py:method:: __iter__() -> Iterator[int] Generate a sequence of indices for the current rank's portion of the dataset. Returns ------- Iterator[int] An iterator over indices for the current rank. .. py:class:: KinematicReconstructor(ddp_setup: dict[str, Any], scenario: artist.scenario.scenario.Scenario, data: dict[str, artist.data_parser.calibration_data_parser.CalibrationDataParser | list[tuple[str, list[pathlib.Path], list[pathlib.Path]]]], optimization_configuration: dict[str, Any], reconstruction_method: str = config_dictionary.kinematic_reconstruction_raytracing) Initialize the kinematic optimizer. Parameters ---------- ddp_setup : dict[str, Any] Information about the distributed environment, process_groups, devices, ranks, world_Size, heliostat group to ranks mapping. scenario : Scenario The scenario. data : dict[str, CalibrationDataParser | list[tuple[str, list[pathlib.Path], list[pathlib.Path]]]] The data parser and the mapping of heliostat name and calibration data. optimization_configuration : dict[str, Any] The parameters for the optimizer, learning rate scheduler, regularizers and early stopping. reconstruction_method : str The reconstruction method. Currently only reconstruction via ray tracing is available (default is ray_tracing). .. py:attribute:: ddp_setup .. py:attribute:: scenario .. py:attribute:: data .. py:attribute:: optimization_configuration .. py:method:: reconstruct_kinematic(loss_definition: artist.core.loss_functions.Loss, device: torch.device | None = None) -> torch.Tensor Reconstruct the kinematic parameters. Parameters ---------- loss_definition : Loss The definition of the loss function and pre-processing of the prediction. device : torch.device | None The device on which to perform computations or load tensors and models (default is None). If None, ARTIST will automatically select the most appropriate device (CUDA or CPU) based on availability and OS. Returns ------- torch.Tensor The final loss of the kinematic reconstruction for each heliostat in each group. Tensor of shape [total_number_of_heliostats_in_scenario]. .. py:method:: _reconstruct_kinematic_parameters_with_raytracing(loss_definition: artist.core.loss_functions.Loss, device: torch.device | None = None) -> torch.Tensor Reconstruct the kinematic parameters using ray tracing. This reconstruction method optimizes the kinematic parameters by extracting the focal points of calibration images and using heliostat-tracing. Parameters ---------- loss_definition : Loss The definition of the loss function and pre-processing of the prediction. device : torch.device | None The device on which to perform computations or load tensors and models (default is None). If None, ARTIST will automatically select the most appropriate device (CUDA or CPU) based on availability and OS. Returns ------- torch.Tensor The final loss of the kinematic reconstruction for each heliostat in each group. Tensor of shape [total_number_of_heliostats_in_scenario]. .. py:class:: MotorPositionsOptimizer(ddp_setup: dict[str, Any], scenario: artist.scenario.scenario.Scenario, optimization_configuration: dict[str, Any], incident_ray_direction: torch.Tensor, target_area_index: int, ground_truth: torch.Tensor, bitmap_resolution: torch.Tensor = torch.tensor([256, 256]), device: torch.device | None = None) Initialize the motor positions optimizer. Parameters ---------- ddp_setup : dict[str, Any] Information about the distributed environment, process_groups, devices, ranks, world_Size, heliostat group to ranks mapping. scenario : Scenario The scenario. optimization_configuration : dict[str, Any] The parameters for the optimizer, learning rate scheduler, regularizers and early stopping. incident_ray_direction : torch.Tensor The incident ray direction during the optimization. Tensor of shape [4]. target_area_index : int The index of the target used for the optimization. ground_truth : torch.Tensor The desired focal spot or distribution. Tensor of shape [4] or tensor of shape [bitmap_resolution_e, bitmap_resolution_u]. bitmap_resolution : torch.Tensor The resolution of all bitmaps during optimization (default is torch.tensor([256,256])). Tensor of shape [2]. device : torch.device | None The device on which to perform computations or load tensors and models (default is None). If None, ``ARTIST`` will automatically select the most appropriate device (CUDA or CPU) based on availability and OS. .. py:attribute:: ddp_setup .. py:attribute:: scenario .. py:attribute:: optimization_configuration .. py:attribute:: incident_ray_direction .. py:attribute:: target_area_index .. py:attribute:: ground_truth .. py:attribute:: bitmap_resolution .. py:method:: optimize(loss_definition: artist.core.loss_functions.Loss, device: torch.device | None = None) -> torch.Tensor Optimize the motor positions. The motor positions are optimized through a reparameterization to ensure stable training across different heliostats with widely varying initial motor positions and ranges. Motor positions can range from 0 to up to ~80000. Instead of directly optimizing the absolute motor positions, which can differ in magnitudes, an unconstrained parameter is optimized. Directly optimizing the absolute motor positions, would have very different effects depending on the scale of the motors. For small initial motor positions (e.g. ~100), a gradient update of size 10 may cause a ~10% relative change, drastically altering the motor positions of this heliostat. For large initial motor positions (e.g. ~50000), the same optimizer step would correspond to only a 0.02% relative change in motor positions, effectively freezing the optimization of this heliostat. This mismatch makes it impossible to choose a single learning rate that works robustly across all heliostats. The reparametrization of the optimizable parameter (motor positions) defines the optimizable parameter as: .. math:: \text{motor\_positions\_optimized} = \tanh( \text{torch.nn.Parameter(optimizable\_parameter)} ) The true motor positions can be reconstructed by: .. math:: \text{motor\_positions} = \text{initial\_motor\_positions} + \text{motor\_positions\_normalized} \cdot \text{scale} where scale defines the range (e.g. up to ~80000) for adjustments. By optimizing as explained above instead of raw motor positions, every heliostat sees updates of comparable relative magnitude, regardless of the absolute size of its motors positions. Parameters ---------- loss_definition : Loss The definition of the loss function and pre-processing of the prediction. device : torch.device | None The device on which to perform computations or load tensors and models (default is None). If None, ``ARTIST`` will automatically select the most appropriate device (CUDA or CPU) based on availability and OS. Returns ------- torch.Tensor The final loss of the motor position optimization. .. py:class:: SurfaceReconstructor(ddp_setup: dict[str, Any], scenario: artist.scenario.scenario.Scenario, data: dict[str, artist.data_parser.calibration_data_parser.CalibrationDataParser | list[tuple[str, list[pathlib.Path], list[pathlib.Path]]]], optimization_configuration: dict[str, Any], number_of_surface_points: torch.Tensor = torch.tensor([50, 50]), bitmap_resolution: torch.Tensor = torch.tensor([256, 256]), device: torch.device | None = None) Initialize the surface reconstructor. Parameters ---------- ddp_setup : dict[str, Any] Information about the distributed environment, process_groups, devices, ranks, world_Size, heliostat group to ranks mapping. scenario : Scenario The scenario. data : dict[str, CalibrationDataParser | list[tuple[str, list[pathlib.Path], list[pathlib.Path]]]] The data parser and the mapping of heliostat name and calibration data. optimization_configuration : dict[str, Any] The parameters for the optimizer, learning rate scheduler and early stopping. number_of_surface_points : torch.Tensor The number of surface points of the reconstructed surfaces (default is torch.tensor([50,50])). Tensor of shape [2]. bitmap_resolution : torch.Tensor The resolution of all bitmaps during reconstruction (default is torch.tensor([256,256])). Tensor of shape [2]. device : torch.device | None The device on which to perform computations or load tensors and models (default is None). If None, ``ARTIST`` will automatically select the most appropriate device (CUDA or CPU) based on availability and OS. .. py:attribute:: ddp_setup .. py:attribute:: scenario .. py:attribute:: data .. py:attribute:: optimization_configuration .. py:attribute:: number_of_surface_points .. py:attribute:: bitmap_resolution .. py:method:: reconstruct_surfaces(loss_definition: artist.core.loss_functions.Loss, device: torch.device | None = None) -> torch.Tensor Reconstruct NURBS surfaces from bitmaps. Parameters ---------- loss_definition : Loss The definition of the loss function and pre-processing of the prediction. device : torch.device | None The device on which to perform computations or load tensors and models (default is None). If None, ``ARTIST`` will automatically select the most appropriate device (CUDA or CPU) based on availability and OS. Returns ------- torch.Tensor The final loss of the surface reconstruction for each heliostat in each group. Tensor of shape [total_number_of_heliostats_in_scenario]. .. py:method:: lock_control_points_on_outer_edges(gradients: torch.Tensor, device: torch.device | None = None) -> torch.Tensor :staticmethod: Lock the u and v values of the control points on the outer edges of each facet. As the knots of the first and last knots on each facet have full multiplicity, the NURBS surfaces all start and end in control points. If the outer control points are not fixed in their u and v values, the reconstructed surfaces may not be rectangular anymore. To keep them rectangular, this function zeros the gradients of the u and v coordinates of all outer control points. Parameters ---------- gradients : torch.Tensor The gradients of the outer control points. Tensor of shape [number_of_active_heliostats, number_of_facets_per_surface, number_of_control_points_u_direction, number_of_control_points_v_direction, 3]. device : torch.device | None The device on which to perform computations or load tensors and models (default is None). If None, ``ARTIST`` will automatically select the most appropriate device (CUDA or CPU) based on availability and OS. Returns ------- torch.Tensor The updated gradients. Tensor of shape [number_of_active_heliostats, number_of_facets_per_surface, number_of_control_points_u_direction, number_of_control_points_v_direction, 3].