Modeling your industrial assets at scale using AWS IoT SiteWise

Posted on by

Introduction

Industrial and manufacturing customers increasingly rely on AWS IoT SiteWise to collect, store, organize, and monitor data from industrial equipment at scale. AWS IoT SiteWise serves as an industrial data foundation that enables stakeholders to remotely monitor equipment across multiple sites, track asset performance using metrics such as Overall Equipment Effectiveness (OEE), proactively detect abnormal equipment behavior, and support other advanced analytics use cases.

Building such as a data foundation typically involves modeling your assets and ingesting live and historical telemetry data. Asset modeling, in particular may require significant effort when dealing with tens of thousands of equipment and ever-changing operations in pursuit of reducing waste and improving efficiency.

We launched three new features for AWS IoT SiteWise at re:Invent 2023 to improve your asset modeling efforts. These features allow customers effortlessly model their equipment using Metadata bulk operations, use their own identifiers leveraging User-defined unique identifiers for consistency across the organization, and represent equipment components using Asset model components and promote reusability.

In this blog post, you will examine eleven real-world customer scenarios related to asset modeling, with the help of a code sample, and learn how you can benefit from new AWS IoT SiteWise features in each scenario.

Prerequisites

  1. An AWS account
  2. AWS IoT SiteWise supported region
  3. Basic knowledge of Python
  4. Some familiarity with asset modeling in AWS IoT SiteWise

Setup the environment

  1. Create a Cloud9 environment using Amazon Linux 2 platform (recommended) or use any on-prem machine as a developer workstation
  2. Configure AWS credentials
  3. To perform bulk operations, you must create an AWS Identity and Access Management (IAM) policy with permissions that allow the exchange of AWS resources between Amazon S3, AWS IoT SiteWise, and your local machine. Refer to IAM permissions.
  4. Ensure Python 3.x is installed on your system, you can verify by running python3 --version or python --version (on Windows)
  5. Using terminal, install Git and clone the metadata-bulk-operations-sample-for-aws-iot-sitewise repository from Github 
    sudo yum install git
git --version
git clone https://github.com/aws-samples/metadata-bulk-operations-sample-for-aws-iot-sitewise.git

  6. Install required Python packages by running pip3 install -r requirements.txt
  7. Update config/project_config.yml to provide necessary information for the job
    • s3_bucket_name: Name of the S3 bucket where bulk definitions will be stored
    • job_name_prefix: Prefix to be used for the bulk operations jobs

Onboard and manage assets at scale

AWS IoT SiteWise now supports the bulk import, export, and update of industrial equipment metadata for modeling at scale. These bulk operations are accessible through new API endpoints such as CreateMetadataTransferJob, ListMetadataTransferJobs, GetMetadataTransferJob and CancelMetadataTransferJob.

With this new capability, users can effortlessly bulk onboard new assets and asset models into AWS IoT SiteWise, seamlessly update existing assets and asset models, and easily migrate assets and asset models between different AWS IoT SiteWise accounts.

You will primarily use metadata bulk import jobs for this blog. The following diagram and steps explain the workflow involved in a metadata bulk import job.

Steps in Metadata Bulk Import Flow

  1. Prepare a job schema JSON file for AWS IoT SiteWise resources such as asset models and assets, following the AWS IoT SiteWise metadata transfer job schema, and upload this file to an Amazon S3 bucket
  2. Make a metadata bulk import call to AWS IoT SiteWise, referencing the uploaded JSON file
  3. AWS IoT SiteWise will import all the resources specified in the JSON file
  4. Upon completion, AWS IoT SiteWise will return the status and a pre-signed S3 URL for any failures encountered
  5. In the event of failures, access the provided report to investigate and understand the root cause

You can also perform bulk operations using the console by navigating to Build → Bulk Operations. Now that you understand how metadata bulk operations work, you will see how this feature can help in the following real-world scenarios.

Scenario 1 – Onboard initial asset models & assets

During a Proof of concept (POC), our customers typically onboard a subset of their equipment to AWS IoT SiteWise. Using metadata bulk operations, you can import thousands of asset models and assets to AWS IoT SiteWise in a single import job.

For a fictitious automotive manufacturing company, import asset models and assets related to the welding lines at one of its manufacturing plants.
python3 src/import/main.py --bulk-definitions-file 1_onboard_models_assets.json

Scenario 2 – Define asset hierarchy

Once the asset models and assets are created in AWS IoT SiteWise, you can define the relationship between assets and create an asset hierarchy. This hierarchy allows users to easily track performance across different levels, from the equipment level to the corporate level.

Create an asset hierarchy for Sample_AnyCompany Motor manufacturing company
python3 src/import/main.py --bulk-definitions-file 2_define_asset_hierarchy.json

Scenario 3 – Associate data streams with asset properties

Our customers typically start ingesting data from their data sources such OPC UA server, even before modeling their assets. In these situations, the data ingested into SiteWise is stored in data streams that are not associated with any asset properties. Once the modeling exercise is complete, you need to associate the data streams with specific asset properties for contextualization.

Associate the data streams for Sample_Welding Robot 1 and Sample_Welding Robot 2 with corresponding asset properties.

python3 src/import/main.py --bulk-definitions-file 3_associate_data_streams_with_assets.json

So far, for demonstration purposes, you were asked to create three separate metadata bulk import jobs for creating asset models and assets, defining the asset hierarchy, and associating data streams with asset properties. In a real-world implementation, you can perform all these actions using a single metadata bulk import job.

Scenario 4 – Onboard additional assets

After demonstrating the business value during POC, you may need to scale the solution to include remaining assets in the same plant and then roll-out the solution to other plants.

Onboard additional welding robots (#3 & #4) and a new production line (#2) from the same Chicago plant.
python3 src/import/main.py --bulk-definitions-file 4_onboard_additional_assets.json

Scenario 5 – Create new properties

As time goes on, you may need to regularly enhance asset models to accommodate changes in data acquisition. For example, when new sensors are installed to capture additional data, you may want to update the corresponding asset models to reflect these changes.

Add a new property Joint 1 Temperature to Sample_Welding Robot asset model
python3 src/import/main.py --bulk-definitions-file 5_onboard_new_properties.json

Scenario 6 – Fix manual errors

We often see customers making mistakes during asset modeling especially when the source data is not available in other systems and therefore manually entered by the user. Some examples include asset serial number, asset description, and unit of measurement.

Correct the serial number of Sample_Welding Robot 1 asset by replacing the old serial number S1000 with S1001.
python3 src/import/main.py --bulk-definitions-file 6_fix_incorrect_datastreams.json

Scenario 7 – Relocate assets

Production line operations change for several reasons, such as process optimization, technological advancements, and equipment maintenance. As a result, some equipment may need to physically move from one production line to another. Using Metadata bulk operations, you can effortlessly update the asset hierarchy to adapt to the changes in line operations.

Move Sample_Welding Robot 3 asset from Sample_Welding Line 1 to Sample_Welding Line 2.
python3 src/import/main.py --bulk-definitions-file 7_relocate_assets.json

Scenario 8 – Backup asset models and assets

As you continue to update asset modeling, you may want to take regular backups of asset models and assets in case if you need to roll back to a previous state. AWS IoT SiteWise tracks changes in asset modeling but currently does not allow you to roll back to a previous version. Therefore, it is recommended to take regular backups. Using the bulk export operation, you can filter specific asset models and assets that you want to include in the export JSON file.

Take a backup of definitions of all welding robots under welding line 1. Replace in 6_backup_models_assets.json with the Asset ID of Sample_Welding Line 1.

python3 src/export/main.py --job-config-file 8_backup_models_assets.json

Scenario 9 – Promote asset models and assets to another environment

Once you model the assets in a development environment, the next step is typically to promote the resources such as asset models and assets to a testing environment. By using the metadata bulk export operation followed by the bulk import operation, you can seamlessly promote a set of asset models and assets from one environment to another.

Promote all the asset models and assets from development to QA environment
python3 src/import/main.py --bulk-definitions-file 9_promote_to_another_environment.json

Maintain consistency throughout the organization

Many industrial companies may have modeled some or most of their industrial equipment in multiple systems such as asset management systems and data historians. It is important for these companies to use common identifiers across the organization to maintain consistency. Previously, identifiers in SiteWise were automatically generated when creating a new asset or asset model, making it challenging to maintain consistency with existing identifiers from external systems.

AWS IoT SiteWise now supports the use of external ID and user-defined UUID for assets and asset models. With the external ID feature, users can seamlessly map their existing identifiers from other systems with SiteWise UUIDs and interact with asset models and assets using these external IDs. The user-defined UUID feature, on the other hand, allows users to reuse the same UUID across different environments such as development, QA, and production.

Scenario 10 – Apply external identifiers

For existing asset models and assets without any external IDs in AWS IoT SiteWise, you can apply external IDs using the AWS IoT SiteWise console, API or metadata bulk import job.

Apply external ID to an existing asset, for example, Sample_Welding Robot 4.
python3 src/import/main.py --bulk-definitions-file 10_apply_external_identifier.json

Promote standardization and reusability using model composition

AWS IoT SiteWise now supports component model, a new asset model type that helps industrial companies model smaller pieces of equipment and reuse them across asset models. This allows for standardization and reusability of commonly used equipment components, such as motors.

For example, a CNC Lathe (asset model) is made of components such as servo motors. With the new feature, a servo motor can be modeled independently as a component model and reused in another asset model, such as a CNC Machining Center.

Scenario 11 – Compose asset models

You can compose asset models using the AWS IoT SiteWise console, API or metadata bulk import job.

Compose the Sample_Welding Robot asset model by independently modeling components in a welding robot, such as a robot joint.
python3 src/import/main.py --bulk-definitions-file 11_compose_models.json

Clean Up

If you no longer require the sample solution, consider removing the resources.

Run the following to remove all the asset models and assets created using this sample repository.
python3 src/remove_sitewise_resources.py --asset-external-id External_Id_Company_AnyCompany

Conclusion

In this post, you have learned how to use the new AWS IoT SiteWise features such as Metadata bulk operations, User-defined unique identifiers, and Asset model components to scale and enhance your asset modeling initiatives. Together, these new features promote standardization, reusability, and consistency across your organization.

The Metadata transfer Job schema provides the JSON definitions used in bulk import operations. To learn about the differences between external IDs and UUIDs, refer to external IDs. Custom composite models explains the two types of composite models and provides guidance on when to use inline versus component-model-based composite models.

About the authors

Raju Gottumukkala is a Senior WorldWide IIoT Specialist Solutions Architect at AWS, helping industrial manufacturers in their smart manufacturing journey. Raju has helped major enterprises across the energy, life sciences, and automotive industries improve operational efficiency and revenue growth by unlocking true potential of IoT data. Prior to AWS, he worked for Siemens and co-founded dDriven, an Industry 4.0 Data Platform company.