Home IoT – Part 4 – The ‘final’ solution and the things I learned

This blog post is part of a series of posts were I’ve automated, and Internet of Things (IoT) enabled my garage door, swing gate and Bluetooth enabled garden irrigation system. Links below to each part!

Last changes from lessons learnt

So it’s been a good couple of weeks now with the home IoT setup, and everything has been pretty good. However, it hasn’t been all perfect, and some changes have been made since I last blogged.

Firstly, a weird issue where the Holman BTX-8 Bluetooth server would stop working after a couple of days kept cropping up. I suspect this is because the ESP32 is always connected with it, unlike the phone app which connects only when it writes a characteristic. To mitigate this, I’m simply calling ESP.restart() every night at 2 am via the webserver to give the Holman device some time alone which seems to do the trick.

Another significant change since the last write up is the way I handle the webserver on the ESP32 itself. While part 1’s example code gave me a great starting point, it became annoying when wanting to make small front-end tweaks to the webserver, which resulted in recompiling the C++ code every time. To mitigate this problem, I have since moved the front-end code into a standalone .Net Core web app that makes HTTP Gets to the ESP32.

.Net Core WebApp communicating with HTTP Get to the ESP32

This approach has made life so much better. Not only do I now essentially have the ESP32 working like an API service, but I can further improve the .Net core web app and allow external access to it backed by Azure AD authentication. Now, with AzureAD permissions, I can grant just me and the stakeholder access to the app which can be accessed anywhere in the world

For those interested to know why I deployed the .Net core app on-premises on an IIS server vs. using Azure, this was because of a cost-saving exercise and being able to use my own private DNS and PKI infrastructure.

The .Net Core app has been super valuable. Not only do I no longer need to carry around with me my garage and gate remotes, but over the past few weeks, I’ve come to find how hit and miss IFTTT can be. Sometimes there is a lengthy 2-3 minute delay making the call to the PowerShell runbook, which is not great when your sitting in your car, waiting for the gate and garage to open. The .Net core app doesn’t have any of these delays!

Below is a code snippet of the ActionResult in the .Net Core WebApp which makes the HTTP get calls. All I’m doing is providing this action with the full URL (base64 encoded) to the call from the front-end with some simple JS and jQuery. Simple, and does the job.

        public static string Base64Decode(string base64EncodedData)
        {
            var base64EncodedBytes = System.Convert.FromBase64String(base64EncodedData);
            return System.Text.Encoding.UTF8.GetString(base64EncodedBytes);
        }
        public async Task<ActionResult> GetFromExternal(string url)
        {
            string urlDecode = Base64Decode(url);

            var client = new HttpClient();

            Task<string> getStringTask =
                client.GetStringAsync(urlDecode);

            string contents = urlDecode;
            try
            {
                contents = await getStringTask;
            }
            catch
            {
                contents = "Something went wrong.";
            }
            return Content(contents);
        }

Making the garden irrigation timing smarter using BOM data

Another improvement I’ve made is to the irrigation timings and smarts around changing the running times based on the actual weather.

Like many, every summer, I tend to forget bumping up the watering times or leave it too late, and my garden suffers because of it. To stop this happening ever again, I have put together two PowerShell scripts which also run locally to save on cost.

The first script pulls data from the Bureau of Meteorology of the last few days temperature and evaporation loss. Using that data with a few switch statements the script creates a JSON file with the timings to run the stations for that day. The range could be from not at all, right up to an extreme 40 minutes per station, which would only happen if there were maximum temperatures over 50 degrees for a whole week!

The second script reads the JSON file to water the garden based on that schedule if it’s one of the permitted watering days.

Two PowerShell scripts using BOM data to trigger irrigation run on the ESP32

Get BOM Data PowerShell script.

<#
        .SYNOPSIS
        Downloads BOM data and creates a JSON file.

        .DESCRIPTION
        This script creates a JSON file with irrigation timings to be invoked with another PowerShell script.
        The script is designed to get the best irrigation timings based on the tempature in the last X days and some other factors like how often the irrigation would run per WaterCorp regulations.

        .PARAMETER JSONFile
        Specifies the file name.

        .OUTPUTS
        JSON File used for the other PowerShell script to schedule irrigation.

        .EXAMPLE
        C:\PS> Invoke-IrrigationTimings.ps1 -JsonFile "payload.json"
        File.txt

        .NOTES
        Requires access to the internet.
#>
[CmdletBinding()]
param (
    [Parameter(Mandatory=$true)]
    [string]$JSONFile
)

Write-Host "`r`nInvoke-IrrigationTimings.ps1 is running" -ForegroundColor Cyan 
Write-Host "---------------------------------------`r`n" -ForegroundColor Cyan 

$a = "http://www.bom.gov.au/climate/dwo/{0}/text/IDCJDW6111.{0}.csv"

$datelast = (Get-Date).AddMonths(-1);
$datelast = $datelast.ToString("yyyyMM");

$date = Get-Date -f "yyyyMM"

$combined = @{};

$a -f $date
$data = (Invoke-WebRequest -Uri ($a -f $date)).Content
$combined = $data

$a -f $datelast
$data = (Invoke-WebRequest -Uri ($a -f $datelast)).Content
$combined += "`r`n"
$combined += $data

$combined = $combined -replace "�", "" -split "`r`n" | Where-Object { $_ -like ",*" -and $_ -notlike ",`"*" } 

$data = foreach ($point in $combined) {

    $data = $point -split ","
    New-Object psobject -Property @{
        Date                  = [datetime]::Parse($data[1])
        "Maximum temperature" = $data[3]
        "Minimum temperature" = $data[2]
        "Rainfall"            = $data[4]
        "Evaporation"         = $data[5]
    }
}
$data = $data | Sort-Object Date 

$daystocheckdata = switch ( Get-Date -f "%M") {
    { 11..12 -contains $_ -or 1..2 -contains $_ } { 5 }
    { 9..10 -contains $_ -or 3..5 -contains $_ } { 7 }
    { 6 -contains $_ -or 8 -contains $_} { 5 }
    { 7 -contains $_ } { 0 }
}
Write-Host -ForegroundColor Cyan "`r`nHow many days to check =" $daystocheckdata 

$data = ($data | Select-Object -Last ($daystocheckdata + 1)) | Select-Object -First $daystocheckdata

$data | Format-Table

$rainfall = ($data."Rainfall" | Measure-Object -Sum).sum
$evaporation = ($data."Evaporation" | Measure-Object -Sum).sum
$mintemp = ($data."Minimum temperature" | Measure-Object -Sum).sum
$maxtemp = ($data."Maximum temperature" | Measure-Object -Sum).sum


Write-Host "Rainfall =" ([math]::Round($rainfall, 2))
Write-Host "Evaporation =" ([math]::Round($evaporation, 2))
Write-Host -ForegroundColor Cyan "Sum of evaporation =" ([math]::Round($rainfall - $evaporation, 2)) "`r`n"
Write-Host "Sum Min Temp =" $mintemp
Write-Host "Sum Max Temp =" $maxtemp
Write-Host -ForegroundColor Cyan "Sum of Temps =" ($mintemp + $maxtemp) "`r`n"


$totalloss = ([math]::Round($rainfall - $evaporation, 2))
$totaltemp = ($mintemp + $maxtemp)

$calc1 = switch ($totaltemp) {
    { 401..1000 -contains $_ } { 40 }
    { 351..400 -contains $_ } { 30 }
    { 301..350 -contains $_ } { 25 }
    { 251..300 -contains $_ } { 20 }
    { 201..250 -contains $_ } { 15 }
    { 151..200 -contains $_ } { 10 }
    { 0..150 -contains $_ } { 0 }
}

$calc2 = switch ($totalloss) {
    { $_ -le -70 } { 40 }
    { $_ -le -55 } { 30 }
    { $_ -le -45 } { 25 }
    { $_ -le -35 } { 20 }
    { $_ -le -20 } { 15 }
    { $_ -gt -20 -and $_ -lt 10 } { 10 }
    { $_ -ge 10 } { 0 }
}

$calc = ([array]$calc1, $calc2 | Measure-Object -Maximum).Maximum
$calc_int = [convert]::ToInt32($calc, 10)
Write-Host -ForegroundColor Cyan "Recommended station watering =" $calc "minutes `r`n"

$startTime = "07:00 AM"

$arrayStations = @(1, 2, 3, 4)

[int]$count = 0;
$Object = @()

foreach ($station in $arrayStations) {
    $properties = @{
        station = $station
        time    = [datetime]::ParseExact($startTime, "hh:mm tt", $null).AddMinutes($calc_int * $count);
        runtime = $calc_int
    }
    $count++;

    $Object += New-Object psobject -Property $properties;
}
$json = $Object | ConvertTo-Json 

$json
$path = Join-Path $PSScriptRoot -ChildPath $JSONFile

Write-Host "`r`nWriting JSON file..." $path

$json | Set-Content $path

Write-Host "`r`nScript completed."

The end solution

The below diagram gives a great overview of how the whole solution has come together over these last few weeks.

The entire solution

So what did I learn through this project?

  • There are a lot of great tutorials out there that will get you started. They definitely helped me so I’d like to think anyone with IT and some coding experience should be able to pick this up. Caution though, some examples do naughty things like unencrypted MQTT!
  • Arduino sketches are pretty easy to put together once you’ve got the IDE configured correctly. E.g. Libraries, example sketches etc. Much like anything, make sure your tooling works first, before doing anything big.
  • Azure IoT Hub is probably a bit too big for this project. I’m contemplating switching MQTT over to Adafruit.io as it would probably make integration with IFTTT a lot easier.
  • Make sure to check out HomeAssistant and ESPHome first as it may cater to all your needs. I learnt about these services mid-way through the project so I kept on the same course of doing my own sketch on the ESP32.
  • Think about your ESP32 as an API endpoint or as something that just listens to MQTT subscriptions to do an action. Having it be a front-end webserver became far too unwieldy after a while, especially with no over-the-air (OTA) updates.
  • IFTTT recently announced a paid pro model, which limits what you can do with the free version. Consider and factor in running costs of your IoT solution, especially if actions can be triggered simply using on-prem scripts and cron jobs like I did.

In final, I absolutely loved this challenge and learned so much on this journey. If you’re not sure about doing your first IoT project, I hope this series has helped you in jumping right in!

Home IoT – Part 3 – What about my garden irrigation system

This blog post is part of a series of posts were I’ve automated, and Internet of Things (IoT) enabled my garage door, swing gate and Bluetooth enabled garden irrigation system. Links below to each part!

I was about to fall asleep, and then it hit me

So after a late night just completing part 2, I was laying in bed when my mind was wondering of all the things I could do with my ESP32. Hrmm, could I convince my key stakeholder to wire up the lights in the house to this controller? Nah, she was pretty clear with the rule, nothing inside the house. Hrmm, what is there outside the house? My mind went blank as I started to drift to sleep when I forgot about watering the pots in the patio. Crap! I’ll do it tomorrow.

Just as I’m about to nod off, it hits me like a train. WAIT FOR A SECOND, I hate that darn Android app for my Bluetooth garden irrigation system. The ESP32 has a Bluetooth radio! Maybe I can make the same calls to replace the app and water those pots? Maybe I can say to Google, hey, water my garden for 10 minutes. I was excited about the challenge! Okay, after some long, painful nights, I have successfully got my ESP32 making Bluetooth Low Energy (BLE) calls to the Holman BTX-8 Outdoor Garden Irrigation System. For details on the Holman unit, it’s just one that you can find at your local Bunnings store.

Here’s how I IoT enabled my garden irrigation system.

Capturing BLE packets on my Android Phone

So to start with this, I first needed to capture the BLE calls the phone was sending and receiving. If I was to have any chance of replacing the app, I first needed to know what the app does. Turning on developer mode and capturing HCI logs was relatively straight forward on my Samsung S9 phone. I followed this guide and was able to get the logs into WireShark where I could start seeing the important parts that make up Bluetooth client to server communication like Service UUID and the Characteristic UUID.

WireShark view of the logs

Not long after finding the right write characteristic, I could see the hexadecimal calls being made. Thankfully this write characteristic didn’t look to be based on a read value of something else, so the challenge was simplified a bit. Being honest though, this step took a while, with lots of trial and error, calling to stop and start stations so I could pick up the patterns in the captured logs. I could see numbers changing, but I was only 90% sure I had the logic right. At this point, I felt I had a chance of not making a bad write characteristic to the Holman device which could, though unlikely, scramble its brain. To test locally on my PC first before going all-in on coding it on the ESP32, I ended up getting an app called Bluetooth LE Lab. This app was great. I recommended it to anyone trying to reverse engineer Bluetooth calls.

Bluetooth BLE Lab App

After some fun figuring out some of the other values, I concluded that there was a 10-byte hex value that made the Holman device do something. This value broken down looks like:

  • Turns off all the solenoids.
    • 00 00 00 00 00 00 00 00 00 00 –
  • To run a station (open a solenoid)
    • 01 (run)
    • 00 (station 1, starting at 0)
    • 13 (19 hrs in hex)
    • 12 (18 mins in hex)
    • 00 00 00 00 00 00 (used for scheduling a station at day/time, instead of immediately running it)

To state the obvious because it would have been silly, I did not run a station for 19 hours and 18 minutes! During the testing and validation, I also decided that I didn’t need to schedule the station to run as I had a plan to manage those smarts outside of the controller itself.

Coding it up in the ESP32


// The remote service we wish to connect to.
static BLEUUID serviceUUID("C521F000-0D70-4D4F-X-X");
// The characteristic of the remote service we are interested in.
static BLEUUID charUUID("0000F006-0000-1000-X-X");

static boolean doConnect = false;
static boolean connected = false;
static boolean doScan = false;
static BLERemoteCharacteristic* pRemoteCharacteristic;
static BLEAdvertisedDevice* myDevice;

class MyClientCallback : public BLEClientCallbacks {
    void onConnect(BLEClient* pclient) {
    }

    void onDisconnect(BLEClient* pclient) {
      connected = false;
      Serial.println("onDisconnect");
    }
};

void connectToServer() {
  Serial.print("BLE - Forming a connection to ");
  Serial.println(myDevice->getAddress().toString().c_str());

  BLEClient*  pClient  = BLEDevice::createClient();
  Serial.println("BLE - Created client.");

  pClient->setClientCallbacks(new MyClientCallback());

  // Connect to the remove BLE Server.
  pClient->connect(myDevice);  // if you pass BLEAdvertisedDevice instead of address, it will be recognized type of peer device address (public or private)
  Serial.println("BLE - Client connected.");

  delay(1000);

  // Obtain a reference to the service we are after in the remote BLE server.
  BLERemoteService* pRemoteService = pClient->getService(serviceUUID);
  if (pRemoteService == nullptr) {
    Serial.print("BLE - Failed to find our service UUID: ");
    Serial.println(serviceUUID.toString().c_str());
    connected = false;
  }
  Serial.println("BLE - Found our service");

  pRemoteCharacteristic = pRemoteService->getCharacteristic(charUUID);
  if (pRemoteCharacteristic == nullptr) {
    Serial.print("BLE - Failed to find our characteristic UUID: ");
    Serial.println(charUUID.toString().c_str());
    connected = false;
  }
  Serial.println("BLE - Found our characteristic");
  connected = true;
}

class MyAdvertisedDeviceCallbacks: public BLEAdvertisedDeviceCallbacks {
    void onResult(BLEAdvertisedDevice advertisedDevice) {
      Serial.print("BLE - Advertised Device found: ");
      Serial.println(advertisedDevice.toString().c_str());

      if (advertisedDevice.haveServiceUUID() && advertisedDevice.isAdvertisingService(serviceUUID)) {
        BLEDevice::getScan()->stop();

        Serial.println("BLE - Correct device found.");
        myDevice = new BLEAdvertisedDevice(advertisedDevice);
        doConnect = true;
        doScan = true;
      }
    }
};

bool makeBLECall(uint8_t* value)
{
  char dataString[30] = {0};
  sprintf(dataString, "%02X %02X %02X %02X%", value[0], value[1], value[2], value[3]);
  String output = dataString;

  Serial.print("BLE - ");
  Serial.println(output);

  if (connected) {
    Serial.println("BLE - Call made.");
    pRemoteCharacteristic->writeValue(value, 4);
    return true;
  }
  return false;
}

void BLEEnable(){
  if (BLEDevice::getInitialized() == false){
    esp_bt_controller_mem_release(ESP_BT_MODE_CLASSIC_BT);
    BLEDevice::init("Cortana Design IoT");
    BLEDevice::setPower(ESP_PWR_LVL_P9);

    Serial.println("BLE - Enabling.");
    BLEScan* pBLEScan = BLEDevice::getScan();
    pBLEScan->setAdvertisedDeviceCallbacks(new MyAdvertisedDeviceCallbacks());
    pBLEScan->setActiveScan(true);
    pBLEScan->start(5, false);

    Serial.println("BLE - Delaying for 3 seconds.");
    delay(3000);

    if (doConnect == true) {
      connectToServer();
      doConnect = false;
    }

    Serial.println("BLE - Delaying for 3 seconds.");
    delay(3000);
  }
}

void BLEDisable(){
  if (BLEDevice::getInitialized() == true){
    Serial.println("BLE - Disabling.");
    BLEDevice::deinit(false);
  }
}


void setup() {
      server.on("/irrigation", HTTP_GET, [] (AsyncWebServerRequest *request) {
    String inputMessage0;
    String inputParam0;
    String inputMessage1;
    String inputParam1;
    String inputMessage2;
    String inputParam2;
    String inputMessage3;
    String inputParam3;
    if (request->hasParam(PARAM_INPUT_3) & request->hasParam(PARAM_INPUT_4) & request->hasParam(PARAM_INPUT_5) & request->hasParam(PARAM_INPUT_6)) {
      inputMessage0 = request->getParam(PARAM_INPUT_3)->value();
      inputParam0 = PARAM_INPUT_3;
      inputMessage1 = request->getParam(PARAM_INPUT_4)->value();
      inputParam1 = PARAM_INPUT_4;
      inputMessage2 = request->getParam(PARAM_INPUT_5)->value();
      inputParam2 = PARAM_INPUT_5;
      inputMessage3 = request->getParam(PARAM_INPUT_6)->value();
      inputParam3 = PARAM_INPUT_6;

      uint8_t value[4] = {inputMessage0.toInt(),(inputMessage1.toInt()-1),inputMessage2.toInt(),inputMessage3.toInt()};
      if (makeBLECall(value)){
        request->send(200, "text/plain", "OK");
      }
      else{
        request->send(400, "text/plain", "Something went wrong.");
      }
    }
    else {
      inputMessage0 = "BLE - Incorrect message sent.";
      inputParam0 = "none";
      Serial.println(inputMessage0);
      request->send(400, "text/plain", "Bad Request");
    }
  });
}

When it came to coding this up, I learnt a tough lesson about the importance of keeping your ESP32 codebase small, efficient and optimised as much as possible. It turns out the BLE library for ESP32 is quite big, and when trying to run that with WiFi, a WebServer and MQTT subscription and publishing to Azure IoT Hub, I was overflowing on the 4mb memory of the controller. To solve this, I had to change the memory partitions, but this meant that over the air (OTA) updates were now no longer possible.

This change made updating my C++ code and the webserver inside it only possible over USB, which made progress tedious and a lot slower. I also ran into another issue that as I kept moving the ESP32 back to where I wanted it to be, it couldn’t see or scan for the Holman device. This issue was quite problematic to troubleshoot as I wasn’t able to see the serial output as it wasn’t near my PC. But when I brought it back which subsequently meant it was nearer to the Holman device (located outside on the office PC wall), it worked fine! It took a good day to realise I needed to bump up the power gain settings on the ESP32 to reach the distance I wanted it to.

With a few changes to the webserver, I now had the irrigation controlled via this much simpler interface, which actually works. Did I mention the Holman app is horrible?! Though this webserver was still only on the network, so I also created a new subscriber in Azure IoT too.

It’s now possible, through the ESP32, to control my Bluetooth irrigation system from anywhere in the world!

ESP32 Webserver with the irrigation controls
Integration of the irrigation using IFTTT, the WebHook + Runbook and Azure IoT Hub

In the last blog post, I’ll share the final solution and recap some learnings that I’ve taken away from this project. Stay tuned!

Home IoT – Part 2 – Putting it together and integration

This blog post is part of a series of posts were I’ve automated, and Internet of Things (IoT) enabled my garage door, swing gate and Bluetooth enabled garden irrigation system. Links below to each part!

Okay, time to put this together

So not long after following Rui Santos’s blog post and video (see part 1), I had my ESP32 controller on our home Wi-Fi network and running a webserver that would switch on/off a relay with a toggle function.

To start, I had only wired up the first relay to the appropriate GPIO, but I had compiled the C++ code to make use of all four relays. Yep, I only have the garage and gate to automate, but why not think big!

ESP32 dev-board wired to one relay
Webserver on the ESP32

The first problem with the example code is that I needed my gate and garage door buttons to be a momentary press rather than a switch. A necessary change because I planned to hook up the relay to the normally open (NO) loop of both the garage and gate and with it potentially stuck in the closed position, they were essentially locked out from any other action from the conventional remotes. It also didn’t make sense to turn on and then immediately off the switch to get the functionality I needed. So with a bit of HTML, JS and CSS changes plus giving the GPIOs nicer labels in the C++ code, I had what I was after.

At this point, I was pretty excited. I had a web server that gave me the ability to control my gate and garage when on the network. However, I faced another problem the very next day. A bit more back story, the gate I’ve installed blocks any access to the front of the house and subsequently, the meter boxes. This design decision became a sticking point when the meter reader rang my Ring doorbell on the gate, and I couldn’t let him in because no one was home or on the network. At that point, I was like huh, maybe I could connect my Ring Doorbell to this? Perhaps get it to send a push notification to my phone when it’s rung, that I can acknowledge, and make the call to trigger the momentary button press on the gate?!

The gate in question

In comes Azure IoT Hub and the power of MQTT. Essentially, I needed my ESP32 to be listening to Azure IoT Hub for the call to action and do the thing I needed it to do.

Azure IoT Hub integration with the ESP32

The best way to describe this is a boy (the ESP32) constantly nagging for the chocolate bar (the action) at the exit aisle by pestering mum (Azure IoT Hub). When Azure IoT Hub says yes after persistently saying no, as in changes from 0 to 1, little Jimmy ESP32 gets his chocolate (opens the gate).

To call it out, as it’s sometimes misunderstood, there is no backdoor (inbound access) to the ESP32 from the outside world. MQTT is all about subscribing and publishing messages, so by nature of that principle; it’s always outbound traffic.

Setting up the subscribing and publishing of values was relatively straight forward. In my C++ code, all I needed to do was to connect to the Azure IoT Hub and set up a function in the loop() that checked if the value had changed, do the function to open the gate and then, have another function then publish back after write HIGH (back to LOW) 0 again.

void publishAzuredata(char* event, unsigned int value){
  client.publish(event, value);
}
void subscibeAzuredata(char* event){
  client.subscribe(iothub_subscribe_endpoint);
}

void reconnect() {
   while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    if (client.connect(iothub_deviceid, iothub_user, iothub_sas_token)) {
      Serial.println("connected");
  
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println("try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

void callback(char* topic, byte* payload, unsigned int length) {
  if (String(topic) == "device/deviceID/gate") {
    if (value == "1") {
      digitalWrite(gatePin, HIGH);
      delay(2000);
      client.publish(topic, 0);
      digitalWrite(gatePin, LOW);
    }
    else if (value == "0") {
      //do nothing really
      //digitalWrite(gatePin, LOW);
    }
  }
}

void setup(){
  client.setServer(mqttServer, mqttPort);
  client.setCallback(callback);
  subscibeAzuredata("device/deviceID/gate");
  subscibeAzuredata("device/deviceID/garage");
  connect_mqtt();
}
void loop(){
  if (!client.connected()) {
    reconnect();
  }
  client.loop();
}

To interface with Azure IoT Hub itself and change the value of 0 to 1, I had a few options like a Logic App or Function App in Azure. However, to keep it simple, I decided to stick with what I know and write a PowerShell script hosted in a Runbook that I could call with a WebHook. That way I could do an HTTP Post with a JSON body that could say open the gate or garage or even both at the same time.

Now that I have the WebHook, I could theoretically call it from anywhere in the world, but I wanted to have a nice way of doing it besides using Postman. Here is where I leveraged IFTTT applets, one for my phone (push notification with IFTTT app), and another for Google Assistant.

Integration with IFTTT and the WebHook

The end result of it all coming together…

ESP32 dev-board and the 4ch relay in a box to be installed

Stay tuned for the next part where I integrate my Bluetooth garden irrigation system with my ESP32 so I can smarten up my irrigation system!

Home IoT – Part 1 – The start of the journey

Over the past few weeks, I’ve automated, and Internet of Things (IoT) enabled my garage door, swing gate and never to be finished with everything, my Bluetooth enabled garden irrigation system which required some reverse engineering. If you’re interested to know more, and the challenges I faced, keep reading!

Background

So to give you some context and background, custom IoT is relatively new to me. Of course, we all have smartwatches and gadgets at home these days but I’ve never worked with micro-controllers and integration, protocols and services like Adafruit.io, Azure IoT Hub, and MQTT, I’m a bit nervous I’ll get tripped up here. To be really honest, I had to google MQTT to know what it was for, so you could say my experience was very much a beginner.

As for automation, I’ve got some pretty good experience with pipelines, scripting, logical flows (Logic Apps, PowerAutomate, IFTTT Runbooks) and function apps. Enough knowledge and know-how that I knew I would be relying heavily on these skills to help with the IoT learning curve I’m about to embark on.

On top of some knowledge around automation, to call it out early, you’ll definitely need some coding experience to customise your solution. However, there are great examples out there that give you the full solution of controlling relays, sensors and displays with a micro-controller. At the end of my journey, I ended up doing C++, HTML, CSS, Javascript, .Net Core and PowerShell, but really, you could do a lot of this by copying coding examples from reputable sources (Arduino ESP32 example library), plugging your variables and hitting the compile button.

Lastly, full disclaimer here, when I started with this project, I jumped right into it without much of a plan. The series of blog posts is more about a journey of tinkering that tested my knowledge, my patience and my sanity! There are probably a million better ways to do this yourself, such as Home-Assitant and ESPHome, but I wanted to get a deep understanding of the ins-and-outs of IoT, so when I do this the next time, for reals, I’d know 😉

Challenges

Like any project, this one not only had technical challenges but business challenges too. Yep, a business challenge in the home! This challenge was working with the key stakeholder, my wife. While she’s a lover of tech (though she doesn’t admit that), she’s not a big fan of seeing home automation on the news and how it’s spying and mining all our conversations (data) for advertising. 

So with the stakeholder uneasy already on what I’m about to set out to do, our at least the perception of what I was going to do, I had to set out some ground rules for myself.

Rules

Knowing the challenges I would face with the key stakeholder, I gave myself some rules and boundaries for this project. They were:

  1. I can’t control anything inside the house. The stakeholder was very clear about that.
  2. I wouldn’t be able to install any third-party apps on the stakeholder’s phone to assist with the home automation.
  3. No Google Homes, no Alexa’s, no anything that is listening in our home.

Let’s get started

Alright, to get started we need some hardware to run this on. Something like an Arduino, Rasberry Pi, ESP8266 or an ESP32. I wanted something cheap and did all the things, so here is where I made my first mistake and brought something from eBay.

The custom PCB that I fried

My recommendation if you’re starting out with IoT, buy a hardware option from a known manufacturer. Custom PCB’s, while sounding like a great idea (“Hey, this does everything, I can’t go wrong”), are very lacking in documentation. I learned my mistake by frying the eBay custom PCB by sending 5v somewhere I shouldn’t have because the PCB schematic was wrong. Turns out I has got a different iteration of the PCB from the doco.

What I ended up with is something much more manageable, buying from a local store my ESP32 development board that had integrated USB and a separate 4-ch relay controller. Not only did I have documentation of the pin layouts (trust me, knowing your pin layouts comes in handy when you start putting it all together) but the people at my local store were super helpful about how I should tackle the project.

Take two, using the ESP32 dev-kit board and 4ch relay

Links to the hardware:

So with the hardware, I need to start putting it together. Thankfully, following the tip from the guys at my local store, there is plenty of ESP32 how-to blogs with relay controls. My need was a webserver (noting I couldn’t use third-party apps on the stakeholder’s phone), so a quick Google and I landed on Rui Santos’s end to end example here:

The code you see in his post is C++, which you need to compile with an IDE. Arduino IDE is an excellent starting point with the ESP32 library, but you can also do this in VS Code (still requires Arduino IDE).

The ESP32 sketch in Arduino IDE

With my IDE ready, I’m now ready to get started. In the next part, I’ll talk about how I took Rui Santos tutorial and got my first iteration working! Stay tuned!