Making a Digital Twin in Azure IoT Central

Perhaps you heard about Internet of Things (IoT). The “Things” can be whatever: vehicles, machines, even fridges, assuming they are connected to the internet. But what is on another end of this connection? In some cases – Digital Twins of the “things”. They are kind of virtual twins of real devices. If some characteristics of a device are changed – they are immediately changed in the twin. And vice versa.

Today we will create a digital twin of a device, measuring temperature and humidity, which I described in my previous article How to make a thermometer. We’ll connect it to the Azure IoT Central and link it with its twin.

Digital Twin

We will implement the following architecture:

Azure IoT Central Application

First what we need to do is to create an application in Azure IoT Central. Open Azure IoT Central site, select “My apps” tab, and click on the “New application” button. Fill in required fields, following the prompts, appearing on the screen, and create an application.

Device Template

When the application is created, select “Device templates” tab in “App settings” of the application, and create a new template. Select “IoT device” type of the template. You will need to give a name of the template:

Now you need to create a model of your twin. If you are creating a new model from scratch, you will choose a Custom model:

Then, select your model in the list of models, and press the “Add capability” button. We will add two capabilities:

  • Temperature with measurement unit “Degree Celsius”
  • Humidity with measurement unit “Percent”.

Add OptimalTemperature property with measurement unit “Degree Celsius”, and make it “Writable” because we need to be able to change this property in a twin.

Let’s add a command just to demonstrate how to send a command (with some parameters) from a twin to a device. Add Reset command with “delay” parameter in its request:

Do not forget to save your changes.

Now we can create a dashboard, visualizing twin’s properties and telemetry. Select “Views” tab and choose “Visualizing the device” as a new view. Give a name of the view, and add tiles for Temperature and Humidity telemetry, and a tile for the Optimal Temperature property.

You can add (optionally) a view called “Editing device and cloud data”. Try to add a section for the Optimal Temperature property.

Device

Select “Devices” tab on the left side menu and press the “New” button. Enter a device name, a unique device id, and choose a device template you created on the previous step:

Now select “Administration” tab on the left side menu, and select “Device connection” tab. You will see a page with device connection parameters:

Save the ID scope in the same text file.

Find a “SAS-IoT_Devices” link on the same page and click on it.

Copy and save a “Primary key” in your text file.

You have three pieces of the puzzle. Now you need to generate a Device Key. To do that, you shall login to the Azure Portal, and click on the “Cloud Shell” button in the upper navigation menu.

Check that you have azure-iot extension installed, and give the command:

az extension add --name azure-iot

Next, generate a Device Key, using Device ID and Primary Key you saved before:

az iot central device compute-device-key --device-id <device_id> --pk <primary_key>

Write down the generated Device Key (without quotes).

IoT device

Now we are ready to assemble our IoT device. We’ll be using a Raspberry Pi 4, a DHT22 temperature sensor, and a LCD display. Read my article How to make a thermometer, and wire up the sensor and the LCD display to the Pi as described there.

Turn on power of the Raspberry Pi and log in to it using your favorite client. You will need to install some libraries:

pip3 install adafruit-circuitpython-dht
sudo apt-get install libgpiod2
sudo apt-get install i2c-tools
sudo apt-get install python-smbus
pip3 install azure-iot-device
pip3 install asyncio

The code

Clone the azure-iot-blog GitHub repository into a folder in your Raspberry Pi:

git clone https://github.com/jevgenij-p/azure-iot-blog.git

Change current working directory to iot-central/digital-twin. There are three files:

I2C_LCD_driver.py
send_telemetry.py
set_env.sh

First, we will need to check the I2C address of your LCD using the i2cdetect command, as described in the article How to make a thermometer. If needed, update the LDC address in the I2C_LCD_driver.py file.

Second, open the set_env.sh in your favorite editor. It looks like this:

export PROVISIONING_HOST=global.azure-devices-provisioning.net
export ID_SCOPE=00000000000
export DEVICE_ID=0000000000
export DEVICE_KEY=00000000000000000000000000000000000000000000

Replace zeroes in this file with the values of the ID scope, Device ID and Device Key you saved before. Save the file and run the command to export variables:

source set_env.sh

And, finally, you can run the program, collecting telemetry, and sending it to the digital twin:

python3 send_telemetry.py

Program output will look like this, if everything is done right:

If you open your Azure IoT Central application, you can see the telemetry, reflected in the digital twin of your device:

Select “Desired properties” and change the Optimal Temperature value (don’t forget to press Enter in the input field):

If your program is still running, the LCD display will show you updated “Desired” value:

Select “Command”, specify some delay in the input field and press the Run button:

A “Reset” command with delay = 10.5 as an input parameter will be sent from the twin to your IoT device and displayed on the LCD:

Summary

There are many thing we can do with the digital twin: we can browse raw data, export it to a storage for future analysis, create rules, triggering some actions based on the data, and much more…

I do not want to explain each line in the code – that would take too much time and make the article even more boring ūüôā But I can suggest you to take a look at the Microsoft tutorial, explaining similar code: Tutorial: Create and connect a client application to your Azure IoT Central application.

How to make a thermometer…

… using a Raspberry Pi 4, a DHT22 temperature and humidity sensor, and a LCD display.
I am using a SunFounder IIC I2C TWI 1602 Serial LCD Display with integrated I2C module to display measurements.

Let’s wire up all parts together. Connect the DHT22 sensor to GPIO pins of the Raspberry Pi, using the following wiring:

DHT22 SENSOR PINSGPIO PINS
VCC (+)5V0
GND (-)GND
DATAGP04

Connect the SDA pin of the LCD display to the SDA1 pin of the Raspberry Pi, and the SCL pin to the SCL1 pin on the Pi:

LCD PINSGPIO PINS
SDASDA1
SCLSCL1
VCC 5V0
GNDGND

Installing libraries

Now we’ll need to install some libraries and tools. Turn on your Raspberry Pi 4, and run the commands:

pip3 install adafruit-circuitpython-dht
sudo apt-get install libgpiod2
sudo apt-get install i2c-tools
sudo apt-get install python-smbus

Check that your LCD display is connected, reboot your Raspberry Pi, and type the following command to see all the devices connected to the I2C bus:

i2cdetect -y 1

You will see a table of addresses for each I2C device connected to your Raspberry Pi:

The I2C address of my¬†LCD is 27.¬†You can see another address. Write it down – we’ll need it later.

The code

Clone the azure-iot-blog GitHub repository into a folder in your Raspberry Pi:

git clone https://github.com/jevgenij-p/azure-iot-blog.git

Open the raspberry/thermometer/ folder. You will see two files:

Open the I2C_LCD_driver.py file in your favorite editor and find the Line 22:

# i2c bus (0 -- original Pi, 1 -- Rev 2 Pi)
I2CBUS = 1

# LCD Address
ADDRESS = 0x27

import smbus
from time import sleep

As I mentioned before, my I2C address is 27. If your address is different – put the I2C address of your LCD in line 22, and save the code.

Our program, measuring temperature and humidity, is measure_temperature.py:

import time
import board
import adafruit_dht
import I2C_LCD_driver

DELAY = 2.0

def main():

    dhtDevice = adafruit_dht.DHT22(board.D4, use_pulseio=False)
    lcd = I2C_LCD_driver.lcd()
    
    print("Measurement started. Press Ctrl-C to exit")
    while True:
        try:
            temperature = dhtDevice.temperature
            humidity = dhtDevice.humidity
            print(f"Temp: {temperature:.1f} C   Humidity: {humidity}% ")

            lcd.lcd_display_string("Temp: {:.1f}{} C".format(temperature, chr(223)), 1)
            lcd.lcd_display_string("Humidity: {:.1f}%".format(humidity), 2)
    
        except KeyboardInterrupt:
            dhtDevice.exit()
            raise SystemExit
        except RuntimeError as error:
            print(error.args[0])
            time.sleep(DELAY)
            continue
    
        time.sleep(DELAY)

if __name__ == '__main__':
    main()

Line 10 is initializing a DHT22 sensor, using Adafruit Blinka (CircuitPython) library. The sensor is connected to GP04 pin.

Line 11 is initializing the LCD. Lines 20 and 21 print text to the screen. Last parameter (1 or 2) of the lcd.lcd_display_string() method is a row number.

Now you can run the example:

python3 measure_temperature.py

If everything is connected and configured correctly, you will see temperature and humidity on your LCD display:

Useful links

Installing CircuitPython Libraries on Raspberry Pi or BeagleBone Black
How to setup an I2C LCD on the Raspberry Pi

Sending Telemetry to Azure IoT Hub

As I promised in my previous article, I will show you how to connect a temperature sensor to a Raspberry Pi, read real telemetry data, and send it to the Azure IoT Hub:

I am using a Raspberry Pi 4, a DHT11 temperature sensor, and a GPIO Extension Board to connect the sensor to the Raspberry. We’ll be using Azure IoT Python SDK enabling connection to the Azure IoT Hub. And we need DHT11 Python library for reading temperature and humidity from DHT11 sensor on Raspberry Pi.

First, we need to create an IoT Hub and to set up a Raspberry Pi. Repeat the steps described in my article “Connect Raspberry Pi 4 to Azure IoT Hub“:

Connect the sensor to Raspberry Pi

The DHT11 sensor can collect temperature and humidity data. Use the following wiring to connect the sensor to GPIO pins:

DHT11 Sensor pinsGPIO pins
Vcc (+)5V (pin 2)
Ground (-)GND (pin 6)
DataGPIO17 (pin 11)

Turn on your Raspberry Pi and connect it to your network.

Run the application

When it is done, clone the azure-iot-blog GitHub repository into a folder in your Raspberry Pi:

git clone https://github.com/jevgenij-p/azure-iot-blog.git

Find the Python application send_sensor_data.py in the raspberry-to-iot-hub/send-telemetry folder:

import asyncio
import time
import board
import RPi.GPIO as GPIO
import dht11
from azure.iot.device import Message
from azure.iot.device.aio import IoTHubDeviceClient

CONNECTION_STRING = ""

DELAY = 5
TEMPERATURE = 20.0
HUMIDITY = 60
PAYLOAD = '{{"temperature": {temperature}, "humidity": {humidity}}}'

async def main():

    try:
        # Create instance of the device client
        client = IoTHubDeviceClient.create_from_connection_string(CONNECTION_STRING)

        # Initialize GPIO
        GPIO.setwarnings(False)
        GPIO.setmode(GPIO.BCM)
        GPIO.cleanup()

        # Read data using pin GPIO17
        dhtDevice = dht11.DHT11(pin=17)

        print("Simulated device started. Press Ctrl-C to exit")
        while True:

            try:
                result = dhtDevice.read()
                if result.is_valid():
                    temperature = result.temperature
                    humidity = result.humidity

                    data = PAYLOAD.format(temperature=temperature, humidity=humidity)
                    message = Message(data)

                    # Send a message to the IoT hub
                    print(f"Sending message: {message}")
                    await client.send_message(message)
                    print("Message successfully sent")
                else:
                    # print("Error: %d" % result.error_code)
                    continue

                await asyncio.sleep(DELAY)

            except KeyboardInterrupt:
                print("Simulated device stopped")
                GPIO.cleanup()
                break

    except Exception as error:
        print(error.args[0])

if __name__ == '__main__':
    asyncio.run(main())

IMPORTANT

Make sure you copy-paste the Primary Connection String you saved when you created your IoT device, into the quotes in the Line 9.

Now, open your Raspberry Pi terminal and install Python packages:

pip3 install azure-iot-device
pip3 install asyncio
pip2 install dht11

Run the application:

python3 send_sensor_data.py

If everything is configured and connected correctly, the program will be sending sensor data every 5 seconds to your IoT hub:

Open Azure Portal, find your IoT hub, and select Overview in the left navigation menu. You should see number of messages received by your IoT hub from your device:

Stay tuned. I’ll continue the “Azure IoT” series.

Connect Raspberry Pi 4 to Azure IoT Hub

I have a Raspberry Pi 4 running Raspbian and I wanted to implement the simplest IoT Architectural pattern of direct connecting an IoT device to an Azure IoT Hub:

Create IoT Hub

First, we need to create an Azure IoT Hub. Login to Microsoft Azure Portal, find an “IoT Hub” in the Azure Marketplace, and create it:

Then, we need to specify as existing Resource Group, or create a new one, specify a Region closest to you, and a globally unique IoT hub name:

On the Management tab, select F1: Free tier and press Review + create button to create the hub:

When the hub is created, we need to create a device identity in the identity registry in your IoT hub. In the left navigation menu, open IoT devices, and then press New to add a device:

In Create a device panel, provide a name for your device (Device ID), and press Save:

Soon, you will see your device in IoT devices:

Click on it to open its properties. You need to copy its Primary Connection String and save it somewhere in a text file. We will need this string a bit later to establish connection between your device and the IoT hub.

Set up your Raspberry Pi

Before we continue, you need to prepare you Raspberry Pi. First, make sure SSH and SPI interfaces are enabled in your Raspberry Pi Configuration.

You can use PuTTY to connect to your Raspberry. Copy the IP address of your Raspberry Pi into the Host name and select SSH as the connection type:

Then, open your terminal and log in to your device:

Run the application

I created a simple Python application, sending simulated data to the Azure IoT hub. The code is in the azure-iot-blog GitHub repository. You can clone it into a folder in your Raspberry Pi:

git clone https://github.com/jevgenij-p/azure-iot-blog.git

Or, you can clone it to a folder on your computer, and copy only the send_simulated_messages.py file to the Raspberry Pi using WinSCP (if you are working on a machine with Windows).

The Python application is send_simulated_messages.py in the raspberry-to-iot-hub/send-telemetry folder:

import asyncio
import random
from azure.iot.device import Message
from azure.iot.device.aio import IoTHubDeviceClient

CONNECTION_STRING = ""

TEMPERATURE = 20.0
HUMIDITY = 60
PAYLOAD = '{{"temperature": {temperature}, "humidity": {humidity}}}'

async def main():

    try:
        # Create instance of the device client
        client = IoTHubDeviceClient.create_from_connection_string(CONNECTION_STRING)

        print("Simulated device started. Press Ctrl-C to exit")
        while True:

            temperature = round(TEMPERATURE + (random.random() * 15), 2)
            humidity = round(HUMIDITY + (random.random() * 20), 2)
            data = PAYLOAD.format(temperature=temperature, humidity=humidity)
            message = Message(data)

            # Send a message to the IoT hub
            print(f"Sending message: {message}")
            await client.send_message(message)
            print("Message successfully sent")

            await asyncio.sleep(5)

    except KeyboardInterrupt:
        print("Simulated device stopped")

if __name__ == '__main__':
    asyncio.run(main())

IMPORTANT

Make sure you copy-paste the Primary Connection String you saved before, into the quotes in the Line 6.

Next, open your Raspberry Pi terminal and install some Python packages:

pip3 install azure-iot-device
pip3 install asyncio

Azure IoT Python SDK is a library, providing functionality for communicating between IoT devices and the Azure IoT Hub.

Now, you can run the application:

python3 send_simulated_messages.py

The program is sending data every 5 seconds to your IoT hub:

Let it work a few minutes and open Microsoft Azure Portal. Find your IoT hub, and select Overview in the left navigation menu to see your IoT Hub Usage:

Now you know how to send telemetry data (simulated in this case) to your Azure IoT hub. Next time I will show you how to send real data.

Camera calibration using OpenCV

Being not ideal, all cameras distort their images. Usually, it’s not so important and can be even funny. But if your image analysis requires quite precise images, you should take care about calibrating your camera to remove distortions.

Luckily, OpenCV provides everything we need for that. I don’t want to repeat the theory and description of that process – you can easily find it on the OpenCV website. I just created a simple Python application calibrationlab, calibrating cameras:

calibration-screen1

The application was written on Python 3.6 using opencv v3.4.1 and wxpython v4.0.3. So you’ll need to install them first, using pip or conda. Also, you will need to print out a chessboard.docx.

Start the application running¬†calibrationlab.py. If you have a few cameras connected to your computer, you can select one using the “Camera” menu. Place the chessboard image in front of your camera and press the “Calibrate” button. Make a few tries to get minimal Mean Error. Resulting calibration parameters can be saved in a file. They contain camera matrix and distortion coefficients. Later you can use them loading from the file without the need to recalibrate your camera again. As an example, I included such a file (camera.json) into the source repository.

When the calibration process is done, you can check the”Undistort” checkbox to apply calibration parameters. The undistorted image is a bit smaller than the original one. I intentionally left a black border around the image. But you can crop the image, using roi¬†(region of image)¬†parameter, returned by the cv2.getOptimalNewCameraMatrix() method.

If you are using a good camera you hardly spot the difference between original and undistorted images. Otherwise, you will notice that straight lines in the real world become straight in the image.

The source code: GitHub download